AI Assistant Quality Automation Developer

Build automated quality systems for Poka’s AI assistant using Python, Type Script, and evaluation tools. This is a hybrid or remote position based on location.

As a quality assurance specialist, you will test assistant answers and surrounding features without manual regression passes. You will define quality with development, product, and design teams while building evaluations, automated tests, and practical tools. Your work supports the connected worker platform used across manufacturing sites by global leaders.

• Automate QA for features and build tools for evaluation analysis
• Maintain evaluation suites using real questions and production edge cases
• Write scoring rubrics and calibrate LLM judges against human samples
• Run CI regression checks for quality, cost, and speed
• Review conversations, identify failure patterns, and support fixes

• Two to four years of relevant experience
• AEC, DEC, BAC, or equivalent experience
• Background in test automation or software development engineering testing
• Comfort with Python, Type Script, and several programming languages
• Curiosity about LLM behavior and manufacturing users’ needs

Apply quality automation, AI evaluation, and programming skills to improve Poka’s manufacturing assistant for real users.