Senior Data Engineer

About the Role

We are looking for a Senior Data Engineer to design and build scalable data lakes, warehouses, and lakehouse architectures supporting a thematic research platform that processes large volumes of financial data daily. You will implement Python-based ETL/ELT pipelines, orchestrate workflows with Airflow, develop ingestion workflows from third‑party APIs, and work with Snowflake, Spark, and AWS to deliver high‑performance data infrastructure. The role combines hands‑on engineering with technical consulting responsibilities, translating business goals into data architecture roadmaps.

• Design and implement Python data engineering solutions.
• Build scalable data lakes, warehouses, and lake houses.
• Implement robust ETL/ELT processes at scale using Python with modern orchestration tools such as Airflow.
• Develop sophisticated ingestion workflows from diverse third‑party APIs and data sources.
• Manage and optimize file formats (Parquet, Avro, ORC) and columnar storage for high‑performance data retrieval.
• Work with AI development tools to support and accelerate machine learning initiatives and advanced analytics.
• Act as a technical consultant for stakeholders and leadership, gathering requirements, understanding business goals, and translating them into technical roadmaps.
• Use Terraform and other tools to build AWS and on‑premise infrastructure.

• Must be authorized to work for any employer in the US (e.g., Green‑card holders, TN visa holders, GC EAD, H4 EAD, U4U with EAD). We are unable to sponsor visa applications.
• Bachelor’s degree in computer science, engineering, or an equivalent technical field.
• 5+ years of experience with Python.
• 5+ years of experience with data processing, manipulation, and analytics libraries such as Pandas, Polars, PySpark, or DuckDB.
• 2+ years of experience with big‑data technologies (Spark, Snowflake).
• Expert‑level knowledge of pipeline orchestration using Airflow or a similar industry‑standard tool.
• Deep understanding of Medallion Architecture, columnar file formats, and various database technologies (SQL, No
  
  SQL, lakehouse architectures).
• Proven ability to work with third‑party APIs for complex data ingestion tasks.
• Proficiency with modern cloud platforms (AWS, GCP, Snowflake) and advanced SQL optimization.
• Exceptional soft skills, including gathering requirements from leadership and collaborating effectively across cross‑functional teams.
• Excellent ability to optimize complex data pipelines and troubleshoot latency or consistency issues in massive datasets.
• A self‑starter mindset that continuously seeks more efficient data architectures and AI development tools.
• High standards for data integrity and accuracy in end‑to‑end pipelines and architectures.
• Strong communication skills for global collaboration with stakeholders and distributed teams.
• Upper‑intermediate English level.

• Familiarity with the fintech industry, including financial data, regulatory requirements, and domain‑specific business processes.
• Documentation skills to record data pipelines, architecture designs, and best practices.
• Experience with Open Search or Elasticsearch.
• Experience with AWS Sage Maker Studio and Jupyter for data analysis.
• Knowledge of Terraform.
• Knowledge of Scala.

• Professional growth
  :
  Mentorship, Tech Talks, and personalized growth roadmaps.
• Competitive compensation
  : USD‑based pay with education, fitness, and team activity budgets.
• Exciting projects
  :
  Modern solutions with Fortune 500 and top product companies.
• Flextime
  :
  Flexible schedule with remote and office options.