Big Data Engineer

Company Overview

Advan Six specializes in global supply chains, delivering products across building & construction, fertilizers, plastics, solvents, packaging, paints, coatings, adhesives, and electronics.

Advan Six is seeking a Big Data Engineer to build and operate our enterprise Unified Data Layer (UDL) spanning IT and OT to deliver trustworthy, performant data products for Finance, Operations, Supply Chain & Logistics, HSE, Commercial, and corporate analytics. The role involves building batch/CDC/streaming pipelines, modeling curated/semantic layers, and hardening run-state with testing, CI/CD, security, and observability. The engineer will partner closely with the data team and larger IT organization.

• Build ingestion pipelines (batch, CDC, streaming) from S/4
  
  HANA/Data Sphere, PHD/historian, LIMS, TMS, HSE, and other sources into landing → curated → semantic layers.
• Implement data contracts, schema/versioning, SCD handling, partitioning, and performance tuning (file formats, clustering, caching).
• Develop dimensional/semantic models that back certified Power BI datasets and APIs for apps/agents.
• Integrate OT data via OPC UA/MQTT, broker/DMZ patterns, read‑only historian feeds, and event/batch frames—no control‑net reads.
• Collaborate with plant controls on change control, signal quality, and downtime windows.
• Embed data quality rules, unit/integration tests, and validation checks (freshness, completeness, drift/PSI).
• Instrument lineage and end‑to‑end monitoring; build alerting and on‑call runbooks to minimize MTTR.
• Enforce RBAC, secrets management, PII/HSE classifications, and retention aligned to Governance/MDM policies.
• Automate build/test/deploy with Git‑based CI/CD (environments, approvals, blue/green).
• Track and optimize cost/performance (cluster sizing, autoscaling, cache strategy); contribute to Fin Ops reviews.
• Partner with Reporting & BI on semantic model contracts, RLS, and performance SLAs; avoid direct system scraping.
• Produce “readme” docs, data dictionaries, runbooks, and post‑incident reviews; support knowledge transfer with vendors.

• Minimum 5 years’ experience in data engineering building production pipelines at scale (batch/CDC/streaming).
• Hands‑on with Azure data stack:
  Databricks or Fabric/Synapse, ADF/Pipelines, ADLS/One Lake, Azure SQL/SQL MI, Key Vault.
• Strong SQL and Python/PySpark; comfort with Spark Structured Streaming and performance tuning.
• Experience implementing tests/observability (freshness, schema, expectations), and Git‑based CI/CD.
• Familiarity with SAP S/4
  
  HANA structures and SAP Data Sphere semantic modeling.
• OT concepts: historians (PHD/PI), OPC UA/MQTT, event/batch frames, ISA-95/99 basics.
• Understanding of Power BI consumption (semantic models, RLS) and APIs for downstream AI/ML apps/agents.

• Time‑series/data‑quality tooling (e.g., Great Expectations or equivalent patterns), feature/metric stores.
• MDM concepts (keys, survivorship), lineage/catalog tooling.
• TMS/WMS, LIMS, Historian, HSE domain exposure;
  Lean/Six Sigma mindset;
  Fin Ops awareness.

The base salary range for this position is $104,600 to $156,800 annually.

Benefits include:

• Competitive compensation, health, dental, vision & wellness programs
• Paid vacation and paid holidays
• 401 K with company matching
• Health savings program and flexible spending account eligibility
• Disability & life insurance with options to purchase additional coverage
• Employee assistance program
• Tuition reimbursement for continued education, certifications, training, and development
• Work within a fast‑paced and innovative company, meeting passionate colleagues and partners with diverse backgrounds and experiences

The pay range, incentives and benefits listed above are general guidelines only and not a guarantee of total compensation or benefits. The final offer will depend on multiple factors, including but not limited to the responsibilities of the job, experience, education, knowledge, skills, and abilities, as well as the job location, applicability of a collective bargaining agreement, length of service, internal equity, and alignment with market data.

The incentive pay is dependent on your role, business results and individual performance. All aspects of total rewards offered are subject to the terms and conditions of the specific plans.

Advan Six is an equal opportunity employer. Qualified applicants will be considered without regard to age, race, creed, color, national origin, ancestry, marital status, affectional or sexual orientation, gender identity or expression, disability, nationality, sex, religion, veteran status, or any other protected classification.