Data Engineer

Shift4 is seeking a hands-on Data Engineer to build reliable SQL and Python workloads in Snowflake for payments datasets and reporting.

• Design, write, and maintain Python jobs that execute SQL over Snowflake to produce transaction, settlement, fee, and reconciliation datasets
• Develop performant, readable SQL using window functions, CTEs, incremental logic, and semi-structured (JSON/VARIANT) handling on large transaction tables
• Package workloads as Docker containers and run them on the existing scheduler
• Ensure jobs are idempotent
  , runnable multiple times
  , parameterized
  , and observable with logging and alerting for failures or bad outputs
• Implement data-quality checks and reconciliation controls to verify accuracy and consistent totals across sources and reporting periods
• Monitor Snowflake cost through warehouse sizing, clustering, query profiling, and avoiding wasteful scans
• Collaborate with finance, operations, and product stakeholders to translate payments questions into correct and maintainable queries
• Apply payments-appropriate security controls
  : secrets handling, least-privilege access,
  PII masking
  , and audit trails
• Document the job library and contribute to code review, testing, and CI practices
• Explore and apply AI tools to improve efficiency across query authoring, testing, and documentation

• 1 to 3 years of experience in data engineering, analytics engineering, or backend roles building SQL-heavy data workloads in production
• Strong SQL skills including window functions
  , CTEs
  , aggregation and joins over large tables, query tuning, and reading execution plans
• Hands-on Snowflake experience: warehouses and cost awareness;
  stages and COPY INTO
  ; streams and tasks
  ; time travel
  ; VARIANT/semi-structured data
  ; Snowpark or the Snowflake Python connector
• Solid Python for data work:
  pandas or Polars
  , parameterized SQL, configuration and secrets management, structured logging, error handling, and unit tests
• Docker fundamentals: building images, running scheduled batch jobs in containers, and debugging container failures
• Git and basic CI/CD habits; comfort with code review and writing jobs that are testable and rerunnable
• Strong attention to numerical correctness
  , including investigating mismatched totals until the root cause is understood

• Python, SQL, Snowflake, Docker
• pandas, Polars
• Git, Snowpark, Snowflake Python connector
• VARIANT, JSON, CTEs, window functions
• Git Hub-style code review and CI/CD (generic)

• Payments domain knowledge: authorization, capture, settlement, refunds, chargebacks, interchange and scheme fees, merchant and acquirer data models, reconciliation
• Orchestration and transformation tooling:
  Airflow, Prefect, Dagster, dbt
• AWS experience from the developer side: S3, IAM, Secrets Manager, ECS/Fargate scheduled tasks
• Data-quality frameworks and monitoring:
  Great Expectations, dbt tests, Soda, pipeline monitoring/alerting
• Experience in a PCI or other regulated environment
• Exposure to BI tools consuming outputs:
  Sigma, Tableau, Power BI, Looker

Location: Center Valley, PA (onsite)

Experience: 1+ years