DevOpsEngineer

Overview

hackajob is collaborating with Leo Technologies to connect them with exceptional professionals for this role. Dev Ops / Site Reliability Engineer - AI Systems for Corrections & Intelligence

Location: Palm Beach, FL •
Type: Full-time •
Reports to: Chief AI & Data Officer

We re hiring a Dev Ops / SRE to deploy, operate, and harden the AI systems that support corrections operations and intelligence analysis. Our data scientists build LLM-powered agents, RAG pipelines, and ontology-driven analytics — your job is to make sure those systems run reliably, securely, and auditably in environments where uptime, data segregation, and chain-of-custody actually matter. You ll own the path from a trained model or agent prototype to a production system that analysts depend on, in infrastructure that meets CJIS, FedRAMP, or equivalent standards.

• Design and operate the deployment platform for LLM applications, agentic systems, RAG pipelines, and supporting data services across cloud, on-prem, and air-gapped environments.
• Build CI/CD pipelines for model and application delivery — including model registries, prompt and config versioning, evaluation gates, and rollback paths.
• Stand up and maintain inference infrastructure: GPU clusters, model serving (vLLM, TGI, Triton, Ollama, TensorRT-LLM), vector databases (pgvector, Weaviate, Qdrant, Milvus), and graph databases (Neo4j, Neptune).
• Operate Kubernetes (EKS, AKS, GKE, or on-prem) as the backbone for AI workloads, with GPU scheduling, autoscaling, and workload isolation.
• Implement observability for AI systems specifically — not just CPU and latency, but token throughput, model drift, agent trace logs, tool-call success rates, retrieval quality, and cost per request.
• Harden environments to meet CJIS, FedRAMP Moderate/High, StateRAMP, or DoD IL4/5 controls as applicable — encryption at rest and in transit, key management, audit logging, FIPS-validated crypto, and boundary controls.
• Enforce data segregation, classification boundaries, and need-to-know access through network policy, IAM, and secrets management (Vault, AWS Secrets Manager, KMS/HSM).
• Build deployment patterns for air-gapped or classified enclaves — including offline model distribution, signed artifacts, and dependency mirroring.
• Manage incident response for AI systems: runbooks, on-call rotations, blameless postmortems, and the special failure modes that come with LLMs (hallucination spikes, prompt injection, retrieval poisoning, runaway tool loops).
• Partner with data scientists, security, and compliance teams to ship safely — and push back when a deploy would compromise security or reliability.

• 5+ years in Dev Ops, SRE, or platform engineering, with at least 2 years operating ML or AI workloads in production.
• Strong fluency with Kubernetes, container orchestration, and infrastructure-as-code (Terraform, Pulumi, or equivalent).
• Hands-on experience deploying LLM inference at scale — you know the tradeoffs between vLLM, TGI, Triton, and managed APIs, and when to use which.
• Solid Python skills for tooling, automation, and glue code; comfort with Bash and at least one systems language is a plus.
• Experience operating GPU infrastructure (NVIDIA drivers, CUDA, MIG, GPU operator, scheduling) in either cloud (A10/A100/H100 instances) or on-prem environments.
• Production experience with CI/CD (Git Hub Actions, Git Lab CI, Jenkins, ArgoCD) and Git Ops patterns.
• Strong security posture: IAM, secrets management, network segmentation, vulnerability scanning, supply-chain security (SBOMs, signed artifacts, SLSA).
• Experience with observability stacks (Prometheus, Grafana, Open Telemetry, Loki, Elastic, Datadog) and applying them to ML systems.
• Demonstrated ability to work with sensitive data and operate within compliance frameworks.

• Direct experience deploying systems in CJIS, FedRAMP, IL4/5, or equivalent regulated environments.
• Experience with air-gapped or cross-domain deployments.
• Familiarity with LLM-specific tooling:
  Lang Smith, Langfuse, Helicone, Phoenix, Weights & Biases, MLflow.
• Vector and graph database operations at scale — sharding, replication, backup,…