Senior Data Engineer

SG Analytics a Straive company is a leading data, analytics, and AI services firm delivering end-to-end consulting and delivery solutions to global clients. Our Data, Analytics & AI vertical partners with enterprises to unlock value through advanced analytics, machine learning, and AI-driven transformation.

We are looking for an experienced Senior Data Engineer with deep expertise in Databricks, PySpark, and AWS. The ideal candidate will have a strong understanding of media measurement datasets (Nielsen) and a proven ability to lead multiple concurrent projects across onsite and offshore teams. This role requires a senior technical leader to define architecture, manage delivery, and mentor teams within the fast-paced New York media industry.

• Architect and deliver enterprise‑scale data platforms using Databricks, PySpark, Delta Lake, and AWS. Lead the development of media analytics pipelines, including the ingestion and modeling of Nielsen data.
• Manage scope, timelines, and risks for multiple simultaneous projects. Collaborate with New York-based stakeholders and drive sprint delivery.
• Act as the primary technical lead onsite, coordinating with offshore teams to ensure alignment, quality, and continuity.
• Mentor junior and mid‑level engineers through code reviews and architectural guidance.

• Databricks:
  Delta Lake, Workflows, Unity Catalog, Notebooks, and SQL.
• Data Processing:
  Strong PySpark and distributed computing experience.
• AWS: S3, EMR, Glue, Lambda, Step Functions, Kinesis, and IAM.
• Leadership:
  Experience leading large data engineering programs and managing senior stakeholders.
• Work experience:
  
  7+ years in Data Engineering.

• Knowledge of Snowflake, Airflow, or Dagster.
• Infrastructure as Code (Terraform / Cloud Formation).
• Exposure to ad‑tech ecosystems (Comscore, Kantar, IAS, Moat) or ML feature pipelines.

• Strong decision‑making and ownership mindset.
• Exceptional communication skills for interacting with senior business stakeholders.
• Strategic, problem‑solving, and delivery‑focused approach.