Lead DevOps Engineer; Remote

The Work:

ICF is seeking a Lead Dev Ops Engineer to own and evolve cloud platforms that support modern enterprise applications. The role will operate multi-account AWS environments, strengthen infrastructure automation and Git Ops delivery, and partner with application, security, database, and enterprise cloud teams to provide secure, observable, and recoverable services.

This position requires that the job be performed in the United States. If you accept this position, you should note that ICF monitors employee work locations, blocks access from foreign locations and foreign IP addresses and prohibits personal VPN connections.

• Own the architecture and day-to-day operation of multi-account AWS environments across development, QA, staging, and production, including VPC, IAM, EKS, Elastic Load Balancing, Cloud Front, Route 53, ACM, S3, and ECR.
• Develop and maintain reusable Terraform modules and Terragrunt environment configurations, including remote state, stack dependencies, drift detection, change planning, approvals, imports, and recovery procedures.
• Administer Amazon EKS Auto Mode and Kubernetes resources using Helm, Argo CD, and Application Sets; manage upgrades, access, capacity, ingress, secrets, and workload reliability.
• Support Git Ops controllers and Kubernetes infrastructure automation, including AWS Controllers for Kubernetes and Kubernetes Resource Orchestrator, and resolve reconciliation, ownership, and lifecycle failures.
• Build and maintain Git Hub Actions workflows, AWS OIDC authentication, self-hosted runners, container-image pipelines, environment promotion, deployment approvals, and rollback procedures.
• Operate Microsoft SQL Server on Amazon RDS, including parameter and option groups, backups, snapshots, point-in-time recovery, restore testing, Performance Insights, Cloud Watch monitoring, and AWS DMS migration workflows.
• Implement security and audit controls using IAM, KMS, Secrets Manager, AWS Config, Cloud Trail, Guard Duty, Security Hub, Inspector, WAF, security groups, and centralized logging.
• Build and maintain observability using Prometheus, Grafana, Amazon Cloud Watch, Open Telemetry and AWS Distro for Open Telemetry, with actionable dashboards, metrics, logs, traces, alerts, and service-level objectives.
• Partner with developers to review and troubleshoot Java and Spring Boot services, including application startup, JVM performance, API behavior, configuration, database connectivity, and container or Kubernetes deployment failures; use logs, metrics, and traces to distinguish application defects from infrastructure issues.
• Integrate application and supply-chain security into delivery pipelines, including SAST with Sonar Qube, DAST, software composition analysis, container-image scanning, dependency checks, and release quality gates.
• Lead incident response, root-cause analysis, disaster-recovery exercises, infrastructure upgrades, cost optimization, architecture documentation, operational runbooks, and knowledge transfer.

• Bachelor's degree in computer science, information technology, engineering, or a related field, or equivalent professional experience.
• 8+ years of Dev Ops, cloud infrastructure, platform engineering, or site reliability experience, including 5+ years of experience operating production workloads on AWS.
• 5+ years of experience with Terraform and infrastructure as code (IaC), including reusable modules, remote state, imports, drift reconciliation, and automated plan and apply workflows
• 4+ years of experience administering production Kubernetes environments, including Amazon EKS, Helm, ingress, RBAC, secrets, upgrades, observability, and Git Ops delivery with Argo CD or a comparable platform.
• 3+ years of experience building and supporting CI/CD pipelines with Git Hub Actions or a comparable platform, plus strong Linux, scripting, AWS networking, IAM, troubleshooting, and incident-response skills.
• Must be a US Citizen or Permanent Resident per contract requirements.

• Experience taking ownership of and improving an existing production cloud platform, including its architecture, automation, reliability, security, and operational practices.
• Hands-on experience with Terragrunt, EKS Auto Mode, AWS Pod Identity, Argo CD Application Sets, AWS Controllers for Kubernetes, or Kubernetes Resource Orchestrator.
• Experience operating Microsoft SQL Server on Amazon RDS and supporting AWS DMS, backup and restore,…