Data Engineer

Open to Tysons Corner, VA or Rockville, MD locations

The Data Engineer works with moderate supervision across two equally weighted domains: (1) large-scale data pipeline development processing market events in a cloud environment, and (2) design and development of agentic AI systems including LLM-powered regulatory data assistants, MCP servers, and agent harness architectures. This position contributes to overall product quality throughout the software development lifecycle.

• Experience building data pipelines using Apache Spark (PySpark preferred) and SQL
• Experience with SQL query engines (Hive, Trino/Presto, or similar) and cloud data platforms (AWS S3, EMR, Lambda)
• Practical experience building LLM-powered agent systems
• Hands‑on experience with at least one agent framework:
  Lang Chain, Lang Graph, AWS Strands, or equivalent
• Proficiency in Python for data engineering and automation

• Build and maintain ETL/ELT pipelines using Apache Spark, Hive, and Trino across S3‑based data lake environments.
• Develop and optimize SQL for large‑scale datasets, including window functions, multi‑table joins, and complex aggregations.
• Build and engineer big data systems (EMR‑on‑EC2, EMR‑on‑EKS) and develop solutions on analytical platforms (Sage Maker, Domino, Dataiku).
• Participate in data quality monitoring, anomaly detection, and production incident investigation.
• Develop AI agent systems using AWS Bedrock and agent frameworks (Strands Agents SDK, Lang Chain/Lang Graph, or equivalent).
• Build agent‑harness architectures combining LLM reasoning with deterministic execution, including skill‑ and RAG‑based SQL generation and structured output validation.
• Implement agent memory, context management, and tool integration (MCP servers, API connectors, data catalog lookups) across data platforms.
• Build evaluation frameworks for agent accuracy, including paraphrase robustness, routing precision, and structural consistency.
• Stay informed of advances in LLM frameworks (Lang Graph, Google ADK, AWS Strands) and emerging AI capabilities.
• Write clean, well‑tested code and contribute to CI/CD pipelines and infrastructure‑as‑code on AWS.
• Ensure secure handling of sensitive and regulated data across both data pipelines and AI agent outputs, including auditable execution traces.
• Adhere to organizational standards for secure software development practices, governance, and technology policies.
• Partner across teams, communicate technical information effectively to both technical and non‑technical stakeholders, and maintain clear documentation.
• Actively learn from senior team members and contribute to continuous improvement initiatives, fostering collaboration, innovation, accountability, and technical excellence.