Senior Big Data Engineer; Java, Kafka, Spark, ETL

Location: Ashburn, VA (Hybrid/Remote)

Employment Type: Full-Time

Citizenship: U.S. Citizenship Required

Clearance: Ability to obtain and maintain a CBP Background Investigation (BI)

SDA is seeking a Senior Big Data Engineer / Java Data Integration Engineer to support the development and modernization of enterprise data ingestion, integration, search, and analytics platforms. This role is responsible for building scalable solutions that collect, normalize, enrich, transform, and distribute large volumes of data across enterprise systems, analytics platforms, and operational environments.

The ideal candidate combines strong Java development and data integration expertise with hands‑on experience building modern Big Data solutions using technologies such as Kafka, Spark, Elasticsearch, and cloud-based platforms. This position is ideal for engineers who enjoy solving complex data challenges and working across the full data lifecycle—from ingestion and transformation to search, analytics, and operational support.

Java | Kafka | Spark | Elasticsearch | Oracle | SQL | ETL | Data Integration | Data Mapping | REST APIs | MongoDB | AWS | Linux

• Design, develop, and maintain enterprise data ingestion, ETL, and data transformation solutions supporting both batch and real-time processing.
• Build and support scalable Kafka-based streaming architectures for processing and distributing high-volume data.
• Develop Java-based applications and services supporting enterprise data integration and transformation workflows.
• Create Spark and Spark Streaming solutions to process, enrich, and analyze large datasets.
• Design and support enterprise search capabilities using Elasticsearch to enable data discovery, investigative workflows, analytics, and operational decision-making.
• Analyze source and target data structures and develop source-to-target mapping specifications.
• Integrate data from multiple structured and semi-structured sources while ensuring data integrity, consistency, traceability, and performance.
• Develop REST APIs and system integration services that enable data exchange across enterprise platforms.
• Support data migration, modernization, and cloud transformation initiatives.
• Perform data validation, reconciliation, profiling, and quality assurance activities.
• Implement and support data quality, governance, lineage, and validation processes across ingestion and transformation pipelines.
• Optimize SQL queries, database processes, and distributed data pipelines.
• Troubleshoot production issues and improve platform reliability, scalability, and performance.
• Collaborate with architects, developers, analysts, and stakeholders to design and implement technical solutions.
• Support onboarding efforts for data consumers and integration partners.
• Evaluate emerging technologies and develop proof-of-concept solutions.
• Mentor junior engineers and contribute to technical leadership initiatives.

• Bachelor's degree in Computer Science, Information Systems, Engineering, or related field (or equivalent experience).
• 5-10+ years of experience in Data Engineering, Software Engineering, ETL Development, or Enterprise Data Integration.
• Experience designing and implementing ETL and data integration solutions.
• Experience building and supporting Apache Kafka producers, consumers, and streaming applications.
• Experience developing applications using Apache Spark and/or Spark Streaming.
• Strong SQL and relational database experience, including Oracle.
• Experience creating and maintaining source-to-target data mappings.
• Experience with data transformation, data migration, and data validation activities.
• Experience developing REST APIs and integration services.
• Experience working with XML, JSON, CSV, and other structured or semi-structured data formats.
• Strong analytical, troubleshooting, and problem-solving skills.

• Experience with Elasticsearch and enterprise search platforms.
• Experience with Hadoop ecosystem technologies.
• Experience with Scala development.
• Experience with MongoDB, Document DB, HBase, Delta Lake, Redshift, or other Big Data technologies.
• Experience with ETL tools…