AI Security Application Engineer

Role Summary

Builds the three agent applications that sit on top of the agentic AI platform: the AI SOC analyst agent, the AI Pen testing agent, and the AI Secure Code Review agent. Owns agent prompts, workflows, tool wiring, domain knowledge, evaluation datasets and integration with the operational security stack.

• AI SOC Agent: build triage, enrichment, correlation and investigation workflows integrated with SIEM, EDR, Extra Hop, Imperva, and ticketing. Produce human‑readable incident narratives and recommended actions.
• AI Pen testing Agent: build reconnaissance, vulnerability hypothesis, exploitation planning and safe‑execution workflows within authorized scopes; integrate with Kali tooling, Burp, Nuclei, and sandboxed execution.
• AI Secure Code Review Agent: build repository ingestion, diff‑aware review, SAST‑finding triage, and developer‑facing explanation workflows integrated with Git Lab and Fortify.
• Curate domain knowledge bases (MITRE ATT&CK, D3
  
  FEND, OWASP, CWE, CVE, KEV, internal runbooks) for RAG.
• Design and maintain evaluation datasets and golden benchmarks for each agent; track precision, recall, hallucination rate and task success.
• Collaborate with the human SOC, pentest and code‑review contractors to capture expert workflows and convert them into agent behaviours; run shadow‑mode trials before any autonomous action.
• Implement human‑in‑the‑loop approval gates and blast‑radius controls for all state‑changing actions.
• Publish per‑agent documentation: scope, capabilities, limitations, failure modes, escalation paths.
• Ensure every agent operates within defined safety boundaries with full auditability and human oversight where required.
• Achieve measurable, monotonic quality improvement on each agent’s evaluation benchmark release over release.

• Within 30 days: shadow‑mode AI SOC agent triaging a defined alert class with measured quality metrics.
• Within 60 days: AI Secure Code Review agent running on selected repos in advisory mode; baseline evaluation set established.
• Within 90 days: AI Pentest agent running authorized, scoped reconnaissance and reporting tasks in lab environment.
• Within 6 months: all three agents graduated from shadow to advisory/assistive mode with published quality metrics, runbooks and escalation paths.

12‑month contract.

Dependent on AI Platform Engineer’s MVP (month
2).

Agent delivery: months 2‑9. Hardening and expansion: months 9‑12.

• 5+ years software engineering, with 2+ years building applied LLM/agent systems.
• Deep practical knowledge of prompt engineering, tool‑use design, multi‑step agent workflows, and evaluation methodology.
• Working knowledge of at least two of: SOC operations, offensive security, secure code review.
• Comfortable reading security tool APIs and SDKs (SIEM, Tenable, Git Lab, Burp, Fortify).
• Python proficiency; familiarity with at least one agent framework used by the platform engineer.
• Understanding of responsible‑AI practices: evaluation, red‑team, bias and hallucination mitigation, human oversight design.
• Strong written communication – agents must explain their reasoning to human operators.