Principal Software Engineer, Platform Services; PMTS

Summary
Design, build, and maintain the backend services, APIs, data-platform automation, and AI platform / agent systems behind our data products. This is a backend-heavy role: distributed Python services, streaming. LLM/agent runtimes on AWS, RAG pipelines, and the dbt/Airflow automation that powers a Snowflake-based lakehouse.

You will work inside the Developer Experience team, whose mandate is to build the tooling, "paved paths," and automation that let internal data teams ship discoverable, governed, and consumable data products quickly and safely.

• Lakehouse & warehouse — Snowflake with Apache Iceberg tables in a medallion architecture (Bronze → Silver → Gold → Semantic Views).

• Transformation —
  dbt (Cloud on Fusion) with data contracts, tag-driven governance, and automated project-evaluation checks.

• Orchestration —
  Airflow on AWS MWAA
  ; DAGs and schedules authored and deployed through platform tooling and CI (metadata driven).

• Developer tooling — an internal CLI and reusable utilities (dbt, Iceberg operations, Snowflake load, REST API based ingestions, etc.) that abstract the platform for data teams.

• Data quality & governance —
  Monte Carlo for data observability (monitors as code) and centralized governance (tagging, lineage, PII classification, cataloging).

• AI layer — Agent/MCP/Skill deployment framework, RAG, semantic router / broker agents, over our data products and catalog metadata; conversational agents in Slack;
  model access via AWS Bedrock / Agent Core
  , centralized LLM Gateway, and Snowflake Cortex
  .

• Cloud & delivery — AWS (
  ECS Fargate, Lambda, API Gateway, DynamoDB, S3, IAM, etc.),
  Terraform IaC,
  Git Hub Actions CI/CD, containerized builds, structured logging, metrics, and tracing.

Responsibilities

• Backend services

• Design, build, and maintain backend services in Python / Go to support our Data Platform Services — Agent runtimes on AWS Bedrock Agent Core / ECS Fargate
  , and event-driven Lambda handlers for our AI Platform Services

• Design and maintain our abstraction layers for for our data domain customers via custom developed utilities

• Maintain and service our infrastructure using IaC (Terraform) for our AWS accounts

• AI / agents

• Build RAG pipelines and tool-calling AI agents for our data products: retrieval, orchestration, grounding/citations, and evaluation.

• Integrate model providers with our LLM Gateway with response streaming, prompt caching, and structured outputs in an auditable way (OTeL)

• Stand up eval harnesses, guardrails, and tracing so agent quality, latency, and cost are measurable and regressions are caught before release.

• Data-platform automation

• Build and maintain data-platform automation:
  dbt services,
  MWAA/Airflow orchestration, and tooling that makes data products discoverable, governed, and consumable.

• Extend the platform CLI and shared utilities that data teams use as their day-to-day interface to the platform.

• Identity & security

• Implement auth and identity:
  OAuth/OIDC flows (per-user 3LO, token vaulting, session binding), least-privilege IAM
  , and secrets management.

• Enforce multi-tenant isolation and per-user identity/RBAC across services and agents

• Reliability & delivery

• Own service reliability and delivery:
  Terraform
  , Git Hub Actions CI/CD, container builds, structured logging, metrics/tracing, alerting, and cost controls.

• Set technical direction: system and API design, code review, and mentoring.

Required skills

Backend engineering

• 15+ years designing, building, and maintaining production backend services at scale

• Expert-level Python / Go / Java for server-side development; solid grasp of relevant frameworks and the WSGI/ASGI model.

• Service & API design: REST (and/or gRPC), request/response and streaming patterns, pagination, versioning,…