Big Data Engineer

As a Big Data Engineer at Tavily, you’ll work at the heart of our systems, shaping the data backbone that powers real-time AI agents. From managing billions of records across No

SQL and SQL databases to optimizing high-throughput pipelines, your work will help our models think faster, our agents act smarter, and our users get answers in real time.

What You’ll Do:

• Design and build scalable, production-grade data pipelines using Airflow, dbt, and PySpark.
• Architect and manage large-scale Data Lakes / Delta Lakes.
• Work with big data processing frameworks to handle massive datasets efficiently.
• Manage and optimize data storage across modern table formats (Iceberg, Delta) and warehouses (Snowflake, Big Query, Redshift).
• Operate across a wide range of No
  
  SQL systems such as Mongo
  
  DB, Redis, and Neo4j.
• Deploy and scale data infrastructure in the cloud
  .

What We’re Looking For:

• Proven experience building scalable data pipelines and infrastructure.
• Strong background in big data ecosystems and distributed systems.
• Experience in data lake architecture and table format management.
• Comfort working across multiple types of databases (both SQL and No
  
  SQL).
• Familiarity with Dev Ops for data
  , including containerization (Docker), CI/CD for data pipelines.
• Experience with cloud-native architectures
  , (AWS , GCP or Azure is a plus).
• Bonus:
  Bachelor’s degree in Computer Science, Engineering, or a related field.

What Kind of Engineer Thrives Here:

• Moves fast, breaks bottlenecks, and loves getting their hands dirty.
• Embraces changing requirements and is motivated by real-world impact.
• Thrives in high-ownership, zero-handhold environments. You build it, you run it.