Senior Software Engineer - Crawler
Temps plein
ZoomInfo
Responsibilities
- Design and implement scalable, fault-tolerant web crawling and extraction pipeline components.
- Write clean, production-grade Java and Python code.
- Build and operate large-scale ETL/ELT pipelines for data extraction and transformation.
- Work with GCP and AWS cloud infrastructure, primarily on GKE.
- Improve observability, reliability, and operational excellence across contributed systems.
- Partner with product and data science teams to deliver solutions.
- Contribute to code reviews, documentation, and knowledge sharing.
- Keep current with web technologies, anti-crawling mechanisms, and AI-powered extraction approaches.
Requirements
- At least 5 years of professional software engineering experience building production systems.
- Strong knowledge of algorithms, data structures, concurrency, and distributed systems.
- Proficiency in Java and/or Python.
- Experience owning features end-to-end from design through deployment and operation.
- Experience making component-level architectural decisions.
- Hands-on experience with cloud data warehouses such as BigQuery or Snowflake.
- Experience designing and operating large-scale ETL/ELT pipelines.
- Experience with orchestration tools such as Apache Airflow.
- Experience with streaming or event-driven systems such as Apache Kafka.
- Production experience on GCP or AWS, with GCP preferred.
- Hands-on experience with Kubernetes, including GKE or EKS, for distributed workloads.
- Familiarity with infrastructure-as-code tools such as Terraform.
- Strong communication, pragmatic problem-solving, adaptability, and self-direction.
- Web crawling at scale, Scrapy, proxy infrastructure, anti-bot evasion, SERP extraction, AI/LLM-based extraction, or B2B data-company experience is preferred.
Benefits
- Remote work arrangement is indicated by the LI-REMOTE designation.
- Comprehensive benefits are offered, along with holistic mind, body, and lifestyle well-being programs.
L'offre d'emploi a été publiée il y a 4 heures
Des emplois similaires qui pourraient vous intéresserBasé sur l'offre Senior Software Engineer - Crawler à télétravail
56000 € par année
...investors, among which Microsoft M12, Point72 Ventures, Maxar, In-Q-Tel, SAFRAN, and ISAI/Capgemini. We are searching for a senior software engineer to play a key role in the further development of our AI platform for object detection. You will be part of the Geo...SuggéréTemps complet- ...India) creates quality solutions for its customers by combining software, back office, and knowledge processing operations to fulfill... ...Summary We are seeking a highly motivated and skilled Software Engineer with 5–8 years of professional experience in designing,...SuggéréTemps completBack office
110000 € - 135000 € par année
...About the team: Builder.io's Publish engineering team builds the platform that empowers marketing and content teams to launch pages... ...and AI-driven automation, this is the team for you. As a Senior Software Engineer at Builder.io, you will… Design and build the APIs...SuggéréTemps completÀ distanceTravail à domicile- ...cover and their potential bottlenecks Work with deployment engineers to ensure that what you build goes as quickly as possible to the... ...science or related field ~3+ years of experience as a Software Engineer (Backend) ~2+ years of experience using TypeScript...Suggéré
- ...At Cloudbeds, we're not just building software, we’re transforming hospitality. Our intelligently... ...How You'll Make an Impact: As a Senior Software Engineer on the POS team, you'll work directly... ...-first applications, synchronization engines, or conflict-resolution workflows ~...SuggéréÀ distanceTravail à domicileTravail postéTravail le weekend
116633 $ - 181243 $ par année
...Summary The Wikimedia Foundation is seeking a Senior Software Engineer to join the team supporting the Wikidata Platform — the structured data backbone of Wikimedia projects and a key part of the global open knowledge ecosystem. You’ll help scale and sustain the...Pour les contractantsTemps completÀ distance- ...and working with brilliant people. You are an accomplished senior Python programmer that enjoys challenging projects in mission-critical... ...one of the industry's pioneers in open source with intelligent engineers at every level from engineer to CTO and CEO level. At Canonical...Temps completÀ distance
56000 € - 74000 € par année
...the world, Mozillians design, build and distribute open-source software that enables people to enjoy the internet on their terms.... ...environment. Collaborate on security initiatives: Partner with engineers across teams to identify, prioritize, and address security...À distanceTravail à domicileRecrutement immédiat- ...EVs, heat pumps, and home batteries. The missing piece is software that can connect and orchestrate these devices at scale.... ...energy transition. What we're looking for: We’re hiring a Senior Backend Engineer to make Enode’s EV platform more reliable, observable, and...StageÀ distanceTravail à domicileTravail posté
- ...with our ecosystem of devices and cloud software. Like our products, we work better together... ...an adjacent team of Controls and Vision Engineers. This team also collaborates very... ...scripting and testing. As a strong, senior engineer - either level 1 or level 2 senior...Travail à distanceStage
- ...company founded by former Google Cloud leaders, architects, and engineers. We are seeking candidates with significant experience in... .... What we need... ~5+ years of experience working in a Software Engineering capacity ~ Excellent knowledge and experience with...Emploi contractuelTemps completÀ distance
- ...This is the general track for Engineering Director at Canonical, apply here if you are confident... ...staff engineers. Canonical’s largest software products are built by multiple teams... ...of the result. Even though this is a senior management position, managing managers,...Temps completÀ distance
- ...for task authoring, expert annotation, review, quality control, and structured data collection. Collaborate with researchers and engineers to ensure evaluation data and published outputs are accurate, consistent, and integrated. Requirements ~4+ years of...Temps completÀ distance
- ...Responsibilities Own delivery of high-quality software projects and coordinate with teammates and stakeholders on timelines. Contribute... ...ambiguity, and drive projects forward. Participate in engineering recruiting through interviewing, problem specification, and...Temps complet
- ...our platform scales globally, we're looking for people who can turn complex systems into practical, working solutions. As a Software Engineer in Partner Engineering, you’ll play a pivotal role in shaping technical solutions that directly impact our top-tier partners....Temps completÀ distance
- ...Responsibilities Own delivery of high-quality software projects and coordinate with teammates and stakeholders to meet agreed timelines... ...ambiguity, and drive projects forward. Participate in engineering recruiting through interviewing, problem specification, and role...Temps complet
- ...with customers and subject-matter experts, and take ownership of product outcomes. Requirements Four or more years of software engineering experience, most of it in frontend work, at companies with strong engineering practices. Strong frontend capability in either...StageTemps completÀ distance
210000 $ par année
...That's why you can do here what you can never do at a legacy software company. move fast and prove it, not theorize it. We think from... ...operates. What You'll Be Doing : Lead a small team of engineers, providing technical mentorship, code review, and...- ...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation, and IoT. Our customers include the world's leading...Temps completÀ distanceTravail à domicileÉquipe de nuitTravail le weekend
- ...around the globe rely on our products, and we’re continuing to invest deeply in engineering to scale and innovate. We’re looking for a Senior Software Engineer to join the Data Sync Engine team , the foundation of HubSpot’s Data Hub. This team is responsible for...Travail à distance
92267 $ - 144201 $ par année
...Summary The Wikimedia Foundation is looking for a software engineer to join the Product Safety and Integrity Team. Engineers at the Wikimedia Foundation get to work on technology that truly makes a global impact — powering Wikipedia and its sister projects, which serve...Pour les contractantsTemps completÀ distance- ...distinguish model limitations from infrastructure, harness, or environment failures. Requirements ~3–5+ years of professional software engineering experience, particularly in backend, infrastructure, platform, SRE, or distributed systems engineering. ~ Strong...Temps completÀ distance
- ...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation and IoT. Our customers include the world's leading...FreelanceÀ distance
- ...Your Next Successful Chapter with Veta Virtual! Are you a seasoned full-stack engineer with a passion for building secure, high-performance systems? Veta Virtual is hiring a Full-Stack Software Engineer for an important fintech company. This role is ideal for someone...Temps completÀ distance
- ...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation, and IoT. Our customers include the world's leading...Candidature spontanéeÀ distanceTravail à domicile
- ...won’t just be writing code; you’ll be shaping the future. As a Software Engineer at Xapo, you will design, build, and deploy high-impact... ...design to production operations, all while being mentored by senior engineers. You could join one of our dynamic, business domains...Temps completÀ distance
- ...means embracing thrilling challenges tailor-made for a star Senior Data Engineer! At Kodify, data is at the heart of how we make decisions.... ...analytical models. Comfort with Git, code review, and modern software engineering practices applied to data. Strong sense of...Temps completÀ distance
- ...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation and IoT. Our customers include the world's leading...FreelanceTemps completÀ distance
- ...with Linux systems, package management, build systems, release engineering, or artifact pipelines. Bachelor's degree or equivalent in... ...experience includes dependency and vulnerability assessment, software supply chain security, SBOMs, signatures, provenance, CVE reports...Temps completTravail à domicile
- ...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation, and IoT. Our customers include the world's leading...À distanceTravail à domicile
Voulez-vous recevoir plus d'offres d'emploi ?
S'abonner et recevoir des offres d'emploi similaires à Senior Software Engineer - Crawler. Soyez parmi les premiers à postuler !
