Senior Software Engineer, Data Infrastructure

the company is where careers accelerate. We move fast, think boldly, and empower you to do the best work of your life. You’ll be surrounded by teammates who care deeply, challenge each other, and celebrate wins. With tools that amplify your impact and a culture that backs your ambition, you won’t just contribute. You’ll make things happen–fast.

We're looking for a Senior Software Engineer to join our Web Data team and help build the next generation of the company's web crawling and data extraction infrastructure.

This is a hands-on engineering role with high impact. You'll work alongside experienced engineers building large-scale crawling and extraction systems that process billions of pages. Your day-to-day will involve solving real distributed systems problems, writing production code, and shipping features that directly impact data quality across the platform.

You'll partner with a dedicated people manager who handles HR and administrative responsibilities, a product manager who connects business needs with technical work, and a senior manager who removes roadblocks and supports career growth. Your focus is on engineering execution, technical excellence, and collaboration.

What You'll DoAs a Senior Software Engineer, you'll contribute to enterprise-scale crawling and extraction platforms that process massive volumes of web data.

You will:

• Design and implement components of scalable, fault-tolerant web crawling and extraction pipelines
• Write clean, production-grade code in Java and Python
• Build and operate ETL/ELT pipelines for large-scale data extraction and transformation
• Work with cloud infrastructure on GCP and AWS, primarily on GKE
• Improve observability, reliability, and operational excellence across the systems you contribute to
• Partner with product and data science teams to deliver impactful solutions
• Contribute to code reviews, documentation, and knowledge sharing across the team
• Stay current with evolving web technologies, anti-crawling mechanisms, and AI-powered extraction approaches

We're prioritizing strong software engineering fundamentals over deep crawling-specific experience. The right candidate is a great engineer first; the domain can be learned on the team.

• 5+ years of professional software engineering experience building production systems
• Strong CS fundamentals: algorithms, data structures, concurrency, distributed systems
• Proficiency in Java and/or Python
• Track record of owning features end-to-end from design through deployment and operation
• Comfortable making sound architectural decisions at the component level

• Hands-on experience with cloud data warehouses such as Big Query or Snowflake
• Experience designing and operating large-scale ETL/ELT pipelines
• Experience with orchestration tools such as Apache Airflow
• Experience with streaming or event-driven systems such as Apache Kafka

• Production experience on GCP (preferred) or AWS; multi-cloud exposure is a plus
• Hands-on experience with Kubernetes (GKE/EKS) for distributed workloads
• Familiarity with infrastructure-as-code tooling such as Terraform

• Strong communicator who can explain technical decisions clearly
• Comfortable operating in ambiguity and iterating quickly
• Bias toward action and pragmatic problem solving
• Self-starter who thrives in fast-paced, evolving environments

• Experience with web crawling at scale (Scrapy or similar frameworks)
• Familiarity with proxy infrastructure, rotation strategies, or anti-bot evasion techniques
• Experience extracting structured and unstructured web data from diverse site architectures
• Knowledge of SERP (Search Engine Results Page) extraction
• Comfort with…