ML Software Engineer - Production- ML & Benchmarking

Position: ML Software Engineer - Production-Ready ML & Benchmarking
Machine Learning
- Software Engineer
- Python, Mongo, Jupyter

Annapolis Junction, MD

Erias Ventures was founded to serve its customers with an entrepreneurial mindset. We valuecreative problem-solving,open communication, andempowering our employeesto make decisions and put forth new ideas.

Our staff includes technical experts working across multiple disciplines, bringing diverse perspectives to every project. We are seeking engineers who wish to grow their careers and want to become part of atechnically strongandgrowth-orientedcompany focused on bringinginnovative solutionsto the difficult mission problems facing our customers.


Description

We're seeking a Software Engineer to bridge the critical gap between cutting-edge ML research and production-ready solutions. You'll transform research prototypes into robust, deployable systems that end users can confidently put into production. This role uniquely combines software engineering, Dev Ops practices, and ML solution delivery.

• AI Model Benchmarks:
  Design, implement, and maintain benchmark suites that measure model performance against defined evaluation criteria, including dataset curation, scoring, and results reporting.
• Agentic AI Benchmarks:
  Develop benchmarks and test scenarios that evaluate agentic workflows and multi-step task performance, including tool-use, task completion, and reliability metrics.
• Evaluation Dataset Development:
  Curate, validate, and version evaluation datasets and test scenarios, including any adversarial or edge-case sets needed to stress agent behavior.
• Metrics, Analysis, and Reporting:
  Produce scoring frameworks, visualizations, and written analyses that translate raw evaluation results into actionable findings for stakeholders.

Clearance

A current Top-Secret/SCI with polygraph security clearance is required. Candidates cannot be sponsored or nominated for a government security clearance under this position.


Experience

Seven (7) years' experience as a SWE in programs and contracts of similar scope, type, and complexity is required. Bachelor’s degree in Computer Science or related discipline from an accredited college or university is required. Fouradditional (4) years of SWE experience on projects with similar software processes may be substituted for a bachelor’sdegree.

Required skills:

• InspectAI
• Python
• MongoDB
• Jupyter notebooks
• Data visualization tools
• Statistical analysis
• Data cleaning and validation
• Cloud or containerized environments
• Git-based version control
• Collaboration tools such as Jira or Confluence
• Experience operating in airgapped or constrained environments

Benefits

Erias Ventures provides a complete package of wealth, health, and happiness benefits.

The expected salary range for this position, depending on education and years of experience is $210,000 - $230,000.

Wealth Benefits:

• Above Market Hourly Pay
• 11% RothorTraditional 401kwith Immediate Vesting and Deposit
• Spot Bonusesfor Assisting with Business Development and Company Growth
• Professional Development Bonusesfor Certificates and Degrees

Health Benefits:

• Company subsidizedMedicalCoverage
• 100%Company PaidVisionandDentalCoverage
• 100%Company PaidLong Term Disability,Short Term Disability, andGroup Life Insurance
• MonthlyWellness Reimbursement

Happiness Benefits:

• Paid Time OffwithFlexible WorkSchedules andBirthday Off
• Amazon Prime Membershipand MonthlyInternetReimbursement
• TechnologyandProductivity Allowancefor Equipment and Supplies
• Morale BuildingandCompany Eventsto Celebrate our Successes and Build our Community
• Onboarding and AnnualSwag
• Company Paid Professional DevelopmentandTraining

At Erias Ventures, we are dedicated to fostering a diverse and inclusive workplace. As an equal opportunity employer, we ensure that all qualified applicants are considered for employment based on merit, without discrimination. We welcome individuals regardless of race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.


Referrals &Inquiries

Do you know a cleared professional seeking to advance their career? Interested in earning some extra cash? If so, refer them to us with their name and contact details, and you could be eligible for a referral bonus of up to$10,000for each successful hire.

Not seeing the right position right now? Reach out to us, and we’ll notify you as new contracts and opportunities become available!

Please send referrals and inquiries to:


To learn more about our company visit ourwebpage orLinked

In .#J-18808-Ljbffr