Data Engineer; Mid-Level) Washington, D.C. · On-site
-
Software Development
Job Description & How to Apply Below
We're hiring a mid-level data engineer to build pipelines that turn a client's messy data into something they can actually trust and use.
Beautiful Majestic Dolphin is a software company working in cybersecurity, AI infrastructure, and data systems, much of it for government and national security customers. Yes, the name is unusual. The work is not. We take it seriously, and we have fun doing it.
This is a hands-on, client-facing role. You'll sit with the client, hear what they need (including the parts they haven't figured out how to say yet), and then build it. Some of the data will arrive clean through APIs. A lot of it won't. Expect PDFs full of orders, scanned documents, inconsistent spreadsheets, and text that needs real structure before it's useful.
Whatyou'll do
- Work directly with the client, in person and in real time, to understand their data, their workflows, and what "done" actually looks like
- Design, build, and maintain batch and API-driven ingestion pipelines
- Build event-driven and streaming flows using pub/sub and message queue services
- Extract and structure data from unstructured sources: PDFs, scanned forms, documents, and free text
- Model, clean, validate, and load data so downstream users can rely on it
- Write clear requirements, technical docs in Markdown, and prompts and specs for coding agents
- Use AI-assisted development to move faster, and review that output with the same rigor you'd apply to anyone else's code
- Share what works with your teammates, so the whole team gets faster, not just you
Required
- 5+ years of hands-on data engineering experience
- Production experience with at least one orchestration or integration tool, such as Airflow, Dagster, Prefect, Mage, AWS Glue, or Azure Data Factory
- Proficiency in Python and SQL. You can write a gnarly query from scratch and explain why it's fast (or why it isn't)
- Solid working knowledge of Docker and Git
- Working knowledge of pub/sub and messaging systems, such as Kafka, RabbitMQ, AWS SNS/SQS, or Google Cloud Pub/Sub
- Comfort with messy data, and the organizational habits to keep it from becoming a mess for everyone else
- The ability to write a clear requirements doc, a useful README, and a prompt a coding agent can actually execute
- Strong communication and listening skills. You'll be talking to the client every day
Nice to have
- Experience extracting structured data from PDFs and scanned documents, including OCR
- Experience with a cloud data stack on AWS or Azure
- Exposure to machine learning workflows, or to preparing data for ML and LLM use
- Experience working with government or regulated-industry clients
You don't need every library below, but you should have real experience with a solid share of them and be able to pick up the rest quickly.
Data wrangling
- pandas
- Polars
- Num Py
- Py Arrow
Databases and SQL
- SQL Alchemy
- psycopg
- DuckDB
APIs and ingestion
- requests
- httpx
- aiohttp
Cloud
- boto3 (AWS)
- Azure SDK for Python
Validation and schemas
- Pydantic
- Great Expectations
- Pandera
Messaging
- confluent-kafka or kafka-python
- pika (RabbitMQ)
Distributed processing
- Py Spark
- Dask
PDFs and documents
- pdfplumber
- PyMuPDF
- Camelot
- pytesseract
Text and NLP
- spaCy
- regex
- NLTK
Machine learning
- scikit-learn
- Py Torch
- Hugging Face Transformers
Testing
- pytest
We love coding agents, and we're not shy about it. They make us faster and more efficient, and this role is built around using them well.
If you already work with voice-to-text tools like Wispr Flow, or hand off code to agents and review what comes back, you'll be right 'll have access to compute for deploying your own agents and managed agents, and you get to decide how to solve the problem.
That said, we're not pure AI maxima lists. We pick the right tool for the job. If an agent you built is too error-prone, stop using it and write the thing yourself. Good judgment about when not to automate matters as much as knowing how to automate.
We also want you to improve the way the team works. If you find a better workflow, show us.
Why on-site, and where you'll beThis role is in the office five days a week. We know a lot of people want remote work, and we get it. But some problems get solved fastest with a whiteboard, the client in the room, and a lot of face time. This is one of those roles.
Since we're asking you to come in, we made sure the office is somewhere you'll actually want to be:
- Right on the Metro in Washington, D.C.
- A comfortable, modern office
- A gym downstairs
- Plenty of restaurants within walking distance
The…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction