Content Engineer, Superintelligence Labs
Job in
New York, New York County, New York, 10261, USA
Listing for:
Meta
Full Time
position
Listed on 2026-11-20
Job specializations:
-
Business
Job Description & How to Apply Below
Location: New York
Content Engineering is a horizontal function within Meta Superintelligence Labs (MSL) that shapes AI product experiences through prompt engineering, evaluations, and quality frameworks. Sitting at the intersection of user experience and large language model behavior, content engineers partner closely with research science, engineering, product, and design teams to build and ship AI experiences from lab to production, across modalities and surfaces.
MSL is seeking a content engineer to improve the quality of AI product capabilities and agentic experiences. In this role, you'll independently create and improve model evaluations and training data content-quality within assigned product/model behavior areas to measure and improve the model behavior on a variety of different tasks. You'll define and apply quality standards, build evaluation materials, and turn qualitative feedback from internal and external users into measurable findings that inform model and product improvements.
You'll combine editorial judgment, analytical skills, and cross-functional collaboration to help create experiences that entertain, inform, and delight.
Content Engineer, Meta Superintelligence Labs Responsibilities:
Quality Definition & Frameworks:
Define and apply quality criteria for assigned AI product capabilities. Build and maintain guidelines, golden response sets, and benchmarks across features and surfaces. Develop failure-mode taxonomies that turn user-facing quality issues into structured, prioritized findings for technical partners
Evaluation & Measurement:
Design and run human evaluation work streams, including rubric development, annotator guidance, calibration, and analysis of results. Partner with research science to support automated evaluation and validate automated quality signals against human ratings. Track quality trends and regressions across model versions and product capabilities
Testing & Iteration:
Run structured internal testing programs, design test plans targeting specific failure modes, triage results, and deliver prioritized recommendations to product and engineering within development timelines. Craft and refine system prompts and agentic behavior in partnership with technical teams to support product goals and model outcomes
Cross-Functional Partnership:
Partner with engineering, research science, product, policy, and design teams on quality standards and priorities for assigned product areas. Communicate evaluation findings clearly, align partners on next steps, and independently deliver work streams through influence and collaboration
Agentic Workflows:
Use AI-native tools to build scalable, repeatable processes for evaluation design, data analysis, system prompting, and reporting. Experiment with emerging tools, validate their outputs, and document workflows that improve quality and reduce manual work
Minimum Qualifications:
Bachelor's degree in a directly related field, or equivalent practical experience4+ years of experience in digital content strategy, user experience, technical writing, journalism, production, or related fields
Experience with human evaluation workflows, including rubric or guideline development, golden set construction, and calibration
Experience translating qualitative content or user-experience feedback into structured quality criteria, failure categories, and measurable benchmarks
Experience designing and running structured testing programs, analyzing results, and communicating prioritized findings to technical partners
Experience making editorial and content-quality decisions, communicating complex concepts to cross-functional partners, and independently delivering work in environments with evolving requirements and competing priorities
Preferred Qualifications:
Experience in prompt engineering and agentic workflows
Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
Experience with AI-native tooling, such as LLM-based development tools, annotation platforms, or prototyping environments, and integrating those tools into workflows to improve efficiency or quality
Demonstrated ongoing AI skill development, such as prompt/context engineering or agent orchestration, and staying current with emerging AI technologies
Experience applying responsible AI practices, including bias mitigation and quality and accuracy reviews
Experience with LLM-as-judge development, including building automated quality signals aligned…
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction