AI Engineer

Own the AI safety layer for ’s Real Assist, an LLM-powered real-estate assistant used across web and mobile. On the AI Integrations Team at News Corp, you will build and tune LLM safety guardrails that act as a launch gate for Fair Housing compliance and content moderation, with runtime prompt-injection and jailbreak screening designed for production.

• Build, tune, and take end-to-end ownership of LLM-as-a-judge classifiers for Fair-Housing compliance and content moderation, targeting high recall on disallowed content without over blocking legitimate users
• Design and run guardrail evaluation pipelines
  , including labeled dataset curation, train/test/validate splits, and offline prod-replay to evaluate over blocking
• Use defensible metrics by reporting confusion matrices and precision/recall/F1 per category to guide safety decisions
• Integrate and operate cloud guardrail services as a runtime prompt-injection/jailbreak screening layer, including fail-open behavior and alerting
• Red-team the assistant and wire safety regression checks into CI to prevent silent guardrail degradation as the product grows
• Manage guardrail infrastructure as code using Terraform (templates, IAM, and project shape), and participate in technical design reviews and architecture discussions
• Partner with ML, backend, product, and legal/compliance stakeholders to define risk tiering and get new AI capabilities through safety review before launch
• Independently design, implement, and tune guardrail and evaluation systems with a focus on balancing safety, latency, and user experience
  ; write clean, maintainable, well-tested code
• Keep eval datasets calibrated to real traffic and emerging attack patterns, staying current with AI-safety research and new guardrail tooling

• AI Safety & Evaluation Expertise (required)
• 4+ years of professional software/ML engineering experience with hands‑on LLM application work
• Bachelor’s degree or equivalent experience
• Strong Python proficiency
• Real classification-metrics literacy:
  precision/recall/F1 (macro vs weighted), confusion matrix debugging, dataset curation, and calibration to production distribution
• Experience evaluating LLM prompts/classifiers with an eval framework such as Deep Eval, RAGAS, Arize Phoenix, Lang Smith, OpenAI Evals
  , or a homegrown harness
• Ability to turn small seed sets into robust labeled eval datasets using dataset-synthesis and prompt-optimization loops
• Familiarity with prompt-injection/jailbreak defense concepts, including OWASP LLM Top 10
  , input/output filtering, least-privilege tool access, and adversarial testing
• Hands‑on experience with cloud guardrail services:
  Google Cloud Model Armor
  , AWS Bedrock Guardrails
  , or Azure AI Content Safety
• Terraform/IaC for cloud infrastructure
• Experience with Google Cloud Vertex AI / Gemini
• Understanding of monitoring and observability tools (for example,
  New Relic or similar)
• Exposure to regulated, compliance-sensitive domains (fair housing, fair lending, healthcare, finance, trust & safety)
• Red‑teaming or AI‑security background

Python
, Deep Eval
, RAGAS
, Arize Phoenix
, Lang Smith
, OpenAI Evals
, OWASP LLM Top 10
, Google Cloud Model Armor
, AWS Bedrock Guardrails
, Azure AI Content Safety
, Terraform
, Google Cloud Vertex AI / Gemini
, New Relic

The AI Integrations Team is a cross-functional squad building how  integrates AI across its products. The work blends creativity and innovation with in-person collaboration. For most roles, employees work four or more days in office
.

• Inclusive and competitive medical, Rx, dental, and vision coverage
• Family forming benefits
• 13 Paid Holidays
• Flexible Time Off
• 8 hours of paid volunteer time off
• Immediate eligibility into the Company 401(k) plan with 3.5% company match
• Tuition Reimbursement for degreed and non-degreed programs
• 1:1 personalized Financial Planning Sessions
• Free snacks and refreshments in each office location
• Student Debt Retirement Savings Match program