Data Engineer; Forensic & Streaming

Oslitandi Tech LLC Data Engineer (Forensic & Streaming) Washington, DC
· Full time Company website Apply for Data Engineer (Forensic & Streaming)

This is a pivotal data-centric role responsible for managing the division's massive data requirements. The engineer will design and maintain scalable pipelines for ingesting, cleansing, and normalizing terabyte-scale forensic datasets. A key responsibility is building and administering high-throughput, real-time data streaming architectures (Kafka/Pulsar) and time-series databases (Timescale DB) to ensure the AI/ML models are continuously fueled with high-integrity, mission-critical sensor data.

Our company works with clients to achieve their tactical and strategic goals by unifying sustainable technology solutions which reduce costs, decrease cycle times, and seamlessly manage processes throughout the enterprise.

Oslitandi Tech specialty lies in integrating sustainable IT services & solutions, Management & IT Consulting, & Network/Cyber Security.

• ETL Pipeline Design: Design, build, and maintain fault-tolerant ETL pipelines to ingest, cleanse, transform, and normalize massive, multi-terabyte datasets of historical mission data (logs, transcripts, sensor records) using technologies like Apache Spark or equivalent distributed processing frameworks.
• Real-Time Streaming Architecture: Implement and administer high-throughput, low-latency real-time data streaming architectures utilizing Apache Kafka or Apache Pulsar to handle live feeds from numerous sensor sources.
• Time-Series Database Management: Administer and optimize specialized databases, such as Timescale DB or InfluxDB
  , for high-speed storage and retrieval of time-stamped sensor data.
• Data Governance & Lineage: Implement comprehensive data governance policies, metadata management, and data lineage tracking tools to ensure the integrity, quality, and audibility of data consumed by the AI/ML Squad.
• Query Optimization: Work closely with the Mission Software Squad to optimize complex SQL and distributed query performance for near real-time retrieval of historical mission data.
• API Integration: Perform integration activities to configure, connect, and pull data from 3rd party software APIs, and collaborate with separate engineering teams to configure data sources for pipeline integration.

• A minimum of 4+ years of progressive experience in Data Engineering, with specific experience handling high-volume (terabyte-scale), high-velocity datasets.
• At least 3+ years of experience designing and managing Apache Kafka or similar distributed messaging systems in production.
• Expert-level proficiency in advanced SQL and relational/time-series database optimization (e.g.,
  Timescale DB
  ).
• Strong scripting and development skills (
  Python
  ) for pipeline orchestration and data manipulation.
• Experience with distributed data processing frameworks such as Apache Spark
  .
• Proficiency in developing log ingestion, data normalization strategies, and implementing data models for complex datasets.
• The candidate shall have a Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field.
• Must be eligible for a U.S. Government Secret Clearance
  .