Data Integration Engineer Remote

Data Integration Engineer In dependant Visa Preferred Primary Responsibilities

• Integration Development:
  Design, build, and maintain integration pipelines connecting source systems to downstream targets, using containerized Python services on Azure Container Apps with Service Bus and Event Hub.
• Event-Driven and Streaming Integration:
  Build and operate flows against Salesforce's Pub/Sub API over gRPC, including Platform Event publish and subscribe, Avro decode, durable replay checkpointing, and reconnection handling for long-lived subscriptions.
• Reliability and Failure Handling:
  Implement idempotent processing, deduplication, retry policies, and dead-letter consumers so at-least-once delivery does not produce duplicate or lost records in target systems.
• API Design and Authentication:
  Design and consume REST APIs, including OAuth 2.0 with token caching, rate limit handling, and error semantics that distinguish retryable from terminal failures.
• Transformation and Business Logic:
  Build and maintain Python-based transformation and validation services that enforce business rules as data moves between systems.
• Data Platform Integration:
  Land request and response event streams into Snowflake via Event Hub as a durable parallel stream, so integration activity is available for reporting, reconciliation, and audit.
• Observability and Handoff to Operations:
  Instrument integrations for distributed tracing, structured logging, and correlation across asynchronous hops, and deliver monitoring hooks, alert thresholds, runbooks, and escalation paths to Operations before go-live.
• Testing, CI/CD, and Requirements:
  Write unit and integration tests covering failure and replay paths, maintain CI/CD pipelines (e.g., Git Hub Actions) across DEV, SIT, UAT, and PROD, and translate stakeholder requirements into technical designs, including which pattern applies.