AI Engineer

Our client, a leading technology organization, is seeking an AI Engineer - Prompt 2 to join their team. As an AI Engineer - Prompt 2, you will be part of the OSSE Ecosystems Engineering team supporting the development and operation of middle services that power generative AI experiences in office applications. The ideal candidate will have strong problem‑solving skills, adaptability, and a collaborative mindset, which will align successfully in the organization.

* No C2C

• Design and craft conversational prompts, messages, and responses for AI chatbots and virtual assistants.
• Support evaluation of Suggested User Actions (SUAs) across multiple office applications.
• Create and maintain test datasets for large language model (LLM) evaluation systems.
• Automate evaluation processes and integrate quality checks into build and deployment pipelines.
• Collaborate with partner teams to iterate on evaluation results and improve AI performance.

• Bachelor's degree in computer science, computer engineering, or a related technical field.
• 2‑4 years of experience in software development, data science, or related areas.
• The ideal candidate has experience with prompt engineering, LLM prompt evaluation, synthetic data and coding/scripting in Python.
• Ability to setup synthetic tenant data and data ingestion – test accounts, generate grounding data, configuration as code.
• Maintain, validate, and automate the creation of test datasets for an LLM evaluation system.
• Integrate evaluation quality checks in our build and deployment pipeline, ensuring performance and scalability.
• Experience with prompt engineering, LLM prompt evaluation, and synthetic data generation.
• Strong troubleshooting skills and experience with system testing and validation.

• Opportunity to work on cutting‑edge AI and LLM evaluation technologies impacting millions of users.
• Engage in practical prompt engineering work directly tied to shipping generative AI experiences.
• Work in a flexible US time zone with onsite presence at Redmond, Building 36.
• Be part of a team that is shaping the future of productivity tools and AI integration.
• Collaborate with a diverse and innovative group of professionals in a dynamic environment.

• Medical and Prescription Drug Plans
• Dental Plan
• Vision Plan
• Health Savings Account
• Health Flexible Spending Account
• Dependent Care Flexible Spending Account
• Supplemental Life Insurance
• Short Term and Long Term Disability Insurance
• Business Travel Insurance
• 401(k), Plus Match
• Weekly Pay

If this is a role that interests you and you'd like to learn more, click apply now and a recruiter will be in touch with you to discuss this great opportunity. We look forward to speaking with you!