Senior Software Engineer – Inference Security Clearance

Position: Senior Software Engineer – Inference with Security Clearance
Location: Annapolis Junction

Salary Range: $260k-$285k per year with an additional 25% ($65k-$71k) in immediately vested company 401(k) contributions Shadowgate Partners is seeking a Senior Software Engineer to join the AI infrastructure team of one of our clients. Join a team building the next generation of AI infrastructure that will power innovation across the organization. In this role, you’ll help build and maintain the foundation for the organization’s AI capabilities while supporting a broader ecosystem of AI-enabled applications.

Your focus will be ensuring access to the highest-quality large language models (LLMs) available to users throughout the inference software stack. Requirements shift quickly as priorities evolve and new technologies emerge, so you’ll continually sharpen your skills and learn to turn loosely defined problems into working solutions.

Key Responsibilities
• Lead the evaluation, configuration, and deployment of inference models for end users.

• Translate high-level stakeholder goals into engineering outcomes.

• Mentor engineers and grow development best practices and standards for the team.

• Develop in-house services and techniques to ensure continual, high-quality inference service.

• Partner with other teams across the organization to establish solid infrastructure for services and integrate LLM-powered tools that meet user needs.

• Collaborate with teammates on short-term, high-priority efforts to meet urgent inference demands.
Required Skills and Experience

• Experience with Python and/or other modern programming languages.

• Experience with Argo CD and/or other CI/CD frameworks.

• Experience with Kubernetes and Helm.

• Experience with AWS or other cloud service providers.

• Strong communication skills and the ability to mentor other engineers.

• Excellent leadership and stakeholder management skills.

• Ability to balance hands-on engineering with leadership and coordination responsibilities. Desired Skills
• Experience with vLLM, LiteLLM, or similar inference-serving frameworks.

• Experience with other LLM hosting frameworks and practices.

• Experience supporting production software using Site Reliability Engineering (SRE) best practices.

• Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices.

• Experience with Docker and containerization.

• Experience in traffic shaping and quality-of-service engineering.

• Knowledge of and interest in hosting AI capabilities.

Education and Experience
• Twelve (12) or more years of relevant professional experience.

• B.S. in a technical discipline, or an additional four (4) years of experience in lieu of a degree.