QA Engineer

Location: New York

Description Kforce has a client that is seeking a QA Engineer in New York, NY.Responsibilities:
* Design, develop, and maintain automated test suites for AI platforms and services
* Integrate testing capabilities into CI/CD pipelines to support rapid and reliable releases
* Build automated regression, functional, integration, and end-to-end testing frameworks
* Collaborate with platform engineering teams to improve testing coverage and release confidence
* Monitor test execution results and proactively identify quality risks
* Develop and maintain evaluation harnesses used to assess AI model and agent performance
* Create and curate golden datasets that represent expected business outcomes and user interactions
* Establish deterministic validation methods to verify AI responses against predefined expectations
* Design scoring methodologies, evaluation metrics, and performance rubrics aligned to business objectives
* Build adversarial and edge-case testing suites to assess model robustness, safety, and resilience
* Implement continuous evaluation capabilities that extend beyond pre-production testing into live environments
* Monitor AI outputs against established quality thresholds and service-level expectations
* Develop dashboards and reporting mechanisms that provide visibility into model quality and drift
* Identify performance degradation, anomalous behavior, and emerging risks
* Support root-cause analysis and remediation efforts when quality issues are identified
* Validate sandbox environments, platform guardrails, and operational control frameworks
* Verify that platform controls function as intended across development, testing, and production environments
* Support testing of AI safety mechanisms, access controls, content filters, and governance policies
* Ensure evaluation standards remain aligned with regulatory, compliance, and organizational requirements Requirements
* 1-3 years of experience in Software Quality Assurance, Test Automation, Quality Engineering, Software Development, or a related technical discipline
* Experience building automated tests using Python, Java, JavaScript, or similar programming languages
* Familiarity with CI/CD pipelines and modern software delivery practices
* Understanding of software testing methodologies, including integration, regression, and performance testing
* Strong analytical and troubleshooting skills
* Excellent written and verbal communication skills

Preferred Qualifications:

* Experience building evaluation frameworks or automated validation tools
* Experience with cloud platforms such as Azure, AWS, or Google Cloud
* Experience with testing frameworks such as PyTest, Playwright, Selenium, JUnit, or equivalent tools
* Exposure to Generative AI, Large Language Models (LLMs), AI agents, or machine learning systems
* Exposure to observability and monitoring platforms
* Familiarity with prompt engineering, retrieval-augmented generation (RAG), or agentic workflows
* Understanding of AI safety, governance, model evaluation, and risk management concepts

Technical

Skills:

* Testing & Automation
* Python* Py Test
* API Testing
* Test Automation Frameworks
* CI/CD Pipelines
* Git/GitHubAI & Data
* Generative AI
* LLM Evaluation
* Prompt Testing
* Golden Dataset Creation
* Evaluation Metrics
* Continuous Monitoring Cloud & Platforms:
* Azure* Azure Dev Ops
* Git Hub Actions
* Container Technologies
* Monitoring & Observability Tools Job Type Contract Compensation
32 - $38