Senior Data Engineer

Clearance Required

• Active TS/SCI with Polygraph – must already hold this level of clearance.

Telework:
None

• 7 years of experience as a software engineer.
• Bachelor’s degree in a technical discipline (2 additional years of software engineering experience may be substituted for a degree).

• The Software Engineer will possess extensive expertise in dataflow design, data transport mechanisms, and Apache Spark-based distributed processing.
• Responsibilities include designing, implementing, and optimizing data ingress/egress pathways to ensure efficient, scalable, and reliable processing of the organization’s analytics workloads.

• Experience using the Linux CLI and Linux tools.
• Experience developing Bash scripts to automate manual processes.
• Recent software development experience using Python and Java.
• Experience with Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows.
• Experience with distributed big data processing engines including Apache Spark.
• Familiarity with SQL technologies such as MySQL, Maria
  
  DB, and Postgre
  
  SQL for querying, joining, and aggregating large datasets.
• Experience using Jupyter Notebook.
• Experience with data wrangling and preprocessing using tools such as pandas and Num Py.
• Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, XML.
• Familiarity with data quality concepts, data validation, and anomaly detection.
• Experience with Git Source Control System.

• Familiarity with HPC job scheduling tools including Slurm.
• Experience using the Atlassian Tool Suite (JIRA, Confluence).

Salary: $230,000 - $240,000 annually.

Excellent benefits package including 25 days PTO, 11 paid holidays, 100% employer-paid healthcare for employees and dependents (available day 1), 8% 401(k) employer match (immediate vesting).

Disclaimer:
The salary range provided is an estimate based on current market conditions and may be adjusted based on factors such as experience, skills, and qualifications. The final salary offer will be determined after a thorough review of the candidate’s background and alignment with the role. Please note that this range is subject to change and should be considered as a guideline rather than a definitive figure.

Shield Consulting Solutions is an equal opportunity/affirmative action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected veteran status, age, or any other characteristic protected by law.

This is a full‑time position.