Systems Architect II

The Systems Architect II – Data Streaming Platforms is a senior technical leader responsible for designing and guiding the evolution of streaming and integration solutions across CSX.

This role serves as a design authority for data streaming platforms, defining architecture patterns, integration approaches, and system designs that enable reliable, scalable data movement across applications, platforms, and infrastructure.

The Systems Architect II leads the design of complex, cross-domain streaming solutions leveraging platforms such as Kafka, Palantir Foundry, and Microsoft Fabric Real-Time Intelligence, ensuring alignment between platform capabilities, engineering delivery, and business outcomes.

The role owns architectural direction, trade-off decisions, and total cost of ownership (TCO) for streaming and integration solutions while advancing AI-enabled patterns across streaming and integration workflows.

Primary Responsibilities:

Streaming Architecture & Design Leadership

• Design and guide end-to-end data streaming architectures across applications, platforms, and infrastructure
• Define and evolve event-driven, messaging, and integration patterns, including:
  • Kafka-based architectures
  • Event pipelines into Palantir Foundry
  • Real-time ingestion and processing patterns within Microsoft Fabric Real-Time Intelligence
• Ensure designs meet requirements for scalability, resiliency, performance, and security
• Align solution designs with data streaming platform capabilities and constraints across:
  • Kafka (event backbone)
  • Palantir Foundry (data modeling and operational analytics)
  • Microsoft Fabric Real-Time Intelligence (real-time reporting and analytics)
• Guide integration between:
  • Data producers and consumers
  • Streaming platforms and downstream systems
  • Operational and analytical platforms
• Guide consistent adoption of approved platform patterns across engineering teams

Standards, Patterns, and Reusability

• Define standard patterns for streaming integration, including:
  • Topic design and partitioning strategies
  • Schema management and data contracts
  • Event design for cross-platform interoperability
• Establish reusable reference architectures and implementation patterns for engineering teams

Infrastructure as Code & Platform Automation

• Define and standardize Infrastructure as Code patterns for provisioning and managing:
  • Streaming infrastructure (Kafka clusters, topics, and supporting services)
  • Platform integrations with Palantir Foundry and Microsoft Fabric
• Guide the use of Terraform and ArgoCD (or equivalent tooling) for consistent, declarative, and repeatable deployments
• Ensure infrastructure is:
  • Version-controlled (source of truth in Git)
  • Environment-consistent (dev/test/prod)
  • Auditable and reproducible
• Promote automated environment provisioning over manual configuration
• Define CI/CD and deployment patterns for streaming and platform solutions, including:
  • Release orchestration, validation, and rollback
  • Code deployment (applications, stream processing logic)
  • Infrastructure deployment (IaC pipelines managed via Git workflows)
  • Configuration and schema promotion across environments
• Build, test, and release automation
• Release governance, validation, and rollback strategies
• Ensure integration of CI/CD pipelines with platform workflows across:
  • Kafka ecosystems
  • Foundry data pipelines
  • Fabric real-time analytics components
• Drive adoption of automated testing, validation, and continuous delivery practices
• Define and promote AI-enabled patterns across data streaming platforms that create measurable value in architecture and operations, including:
  • Real-time data enrichment and intelligent processing of event streams
  • Integration of streaming data with AI/ML and agent-based workflows
  • Event-driven triggers for AI-driven decisioning and automation
• Guide integration of…