QA Test Engineer GA, Alpharetta

Position: QA Test Engineer Jobs  GA, Alpharetta | Rose International Job
Education:

Bachelor's in Computer Science, Engineering, Data/Information Systems, or equivalent practical experience.

Job Description:

TOP 5

SKILLS REQUIRED:

- 3+ years in QA automation or SDET-type work (adjust by level); 1+ year exposure to AI/LLM or ML-driven features is a plus.

- Strong test automation in Python and/or Java/Type Script.

- We are a platform team, testing APIs for high performance, automation will be primary focus.

- Strong communication and analytical skills.

ADDITIONAL

SKILLS REQUIRED:

- Hands-on with frameworks/tools such as: UI:
Playwright / Cypress / Selenium and API: pytest + requests, Postman/Newman, REST Assured

- CI/CD integration:
Git, Git Hub Actions/Jenkins/Git Lab CI, test reporting, gating.

- Test design: equivalence partitioning, boundary testing, risk-based testing, defect triage. AI-Specific Testing Competencies (Key)

- LLM/application behavior testing: validating correctness when outputs are probabilistic.

- Evaluation strategies: golden datasets, scoring rubrics, human-in-the-loop reviews.

- Non-determinism handling: statistical assertions, repeated runs, variance thresholds.

- Prompt and regression management: versioning prompts, detecting prompt drift, replay tests.

- RAG testing (if applicable): retrieval quality (recall/precision), grounding checks, citation validation, doc freshness.

- Safety & quality checks: hallucination detection, toxicity/PII leakage checks, policy compliance tests.

Data & Observability

- Ability to create and maintain test datasets (structured + unstructured), including edge cases.

- Familiarity with telemetry for AI systems:

- logging prompts/outputs safely, traceability, correlation IDs - tools like Open Telemetry, ELK/Splunk, Datadog/Grafana (any equivalent)

- Understanding of data privacy constraints (masking/redaction) and secure test data practices.

- API / Microservices / Cloud

- Comfortable testing distributed systems: microservices, async workflows, queues/events.

- Basic cloud proficiency (AWS/Azure/GCP) and containerization (Docker, optional Kubernetes). Performance & Reliability Testing (AI-Aware)

- Load/performance testing for inference endpoints (latency, throughput, concurrency).

- Cost-aware testing (token usage, rate limits, fallbacks).

- Resilience tests: retries, circuit breakers, model timeouts, degraded-mode behavior.

Nice-to-Have Domain Knowledge

- Familiarity with NLP concepts (embeddings, context windows, temperature/top-p).

- Experience with AI tooling:
Lang Chain/Llama Index, evaluation tools, model gateways.

- Knowledge of regulatory/security needs relevant to the telecom domain.

Soft Skills / Ways of Working

- Strong communication

-able to explain AI quality issues clearly to product and engineering.

- Comfortable partnering with data science/ML engineers and backend teams.

- Ownership mindset: building reusable test harnesses, improving quality metrics, preventing regressions.

Benefits:

For information and details on employment benefits offered with this position, please visit

here
. Should you have any questions/concerns, please contact our HR Department via our secure
website

California Pay Equity:

For information and details on pay equity laws in California, please visit the State of California Department of Industrial Relations' website

here

Rose International is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, age, sex, sexual orientation, gender (expression or identity), national origin, arrest and conviction records, disability, veteran status or any other characteristic protected by law. Positions located in San Francisco and Los Angeles, California will be administered in accordance with their respective Fair Chance Ordinances.

If you need assistance in completing this application, or during any phase of the application, interview, hiring, or employment process, whether due to a disability or otherwise,

please contact our HR Department

Rose International has an official agreement (), effective June 30, 2008, with the U.S. Department of Homeland Security, U.S. Citizenship and Immigration Services, Employment Verification Program (E-Verify). (Posting required by OCGA 13/10-91.).