AI Engineer

Reference: 3240

Posted:

Location:

Austin, TXCompany:
Realtor.comAI Engineer

Location:

Austin, TX, United States Category:
Technology Job  Now Description Recognized  as the No. 1 site trusted by real estate professionals,  has been at the forefront of online real estate for over 25 years, connecting buyers, sellers, and renters with trusted insights and expert guidance to find their perfect home. Through its robust suite of tools,  not only makes a significant impact on the real estate industry at large, but for consumers, navigating the biggest purchase they will make in their life, by providing a user experience that is easy to use, easy to understand, and most of all, easy to make decisions.

Join us on our mission to empower more people to find their way home by breaking barriers to entry, making the right connections, and building confidence through expert guidance.

We are seeking an AI Engineer to join our AI Integrations Team with deep expertise in AI safety, guardrails, and LLM evaluation to join our AI Integrations team. You will be responsible for building and tuning the classifiers that keep our AI assistant compliant, safe, and helpful - working at the intersection of AI and consumer-facing products.

About the Role:

The AI Integrations Team is a cross-functional squad that builds the future of how  integrates AI across its products. Our flagship is Real Assist, an LLM-powered real-estate assistant embedded across web and mobile. Because it advises consumers on housing, it operates under real Fair Housing Act compliance obligations - our AI safety layer is a launch gate, not an afterthought. This role owns that layer, including:

LLM-as-a-judge classifiers for Fair-Housing compliance and content moderation The guardrail evaluation harness that proves it all works before it shipsA cloud prompt-injection / jailbreak backstop (Google Cloud Model Armor) Technical Ownership Build, tune, and own LLM-as-a-judge classifiers for Fair-Housing compliance and content moderation, targeting high recall on disallowed content without overblocking legitimate users

Design and run guardrail evaluation pipelines: curate labeled datasets, maintain train/test/validate splits, and run offline prod-replay overblocking evals
-Report confusion matrices and precision/recall/F1 per category to make safety decisions defensible

Integrate and operate cloud guardrail services as a runtime prompt-injection/jailbreak screening layer, with fail-open behavior and alerting

Red-team the assistant and wire safety-regression checks into CI so guardrails don't silently degrade as the product grows

Manage guardrail infrastructure as code with Terraform (templates, IAM, project shape) - Participate in technical design reviews and architecture discussions

Partner with ML, backend, product, and legal/compliance stakeholders to define risk tiering and get new AI capabilities through safety review before launch

What You'll Bring:

AI Safety & Evaluation Expertise (Required)4+ years of professional software/ML engineering experience, with hands-on LLM application work Bachelor’s degree or equivalent experience

Strong Python proficiency

Real classification-metrics literacy - precision/recall/F1 (macro vs weighted), confusion matrix debugging, dataset curation, and calibration to a production distribution
-Experience building or tuning LLM prompts/classifiers and evaluating them with an eval framework (Deep Eval, RAGAS, Arize Phoenix, Lang Smith, OpenAI Evals, or a home grown harness)
Experience turning small seed sets into robust labeled eval datasets (dataset-synthesis tooling, prompt-optimization loops)
Familiarity with prompt-injection / jailbreak defense concepts (OWASP LLM Top 10, input/output filtering, least-privilege tool access,…