Automation Engineer II

Job Title:
Automation Engineer II

Job Location:

Jersey City, NJ
Job Duration: 8 months on W2

Job Description:
" 10+ years of proven experience in the software development industry, participating in a team environment.
" 5+ years of experience in UI automation, such as Selenium Web Driver.
" 5+ years of proven experience in testing web services such as RESTful API.
" 3+ years of proven experience in performance testing using tools such as Apache JMeter.
" 3+ years of working in a cloud-based development and production environment such as AWS.
" 3+ years of proven experience with programming languages such Java/C#. Familiarity with Python is a plus. Seeking candidates with solid coding proficiency in Java or C#.
" Proficient with both SQL and No

SQL databases.
" Experienced working in Continuous Integration/Continuous Deployment (CI/CD) environments.
" Having AWS/Cloud technologies and AI/ML powered applications knowledge are big plus.
"

Experience with AI/ML-powered applications, Generative AI, and Agentic AI systems is highly desirable.
" Understanding of AI testing concepts, including prompt validation, model output verification, hallucination detection, bias testing, guardrail validation, and AI quality evaluation.
" Familiarity with LLMs (Large Language Models), AI agents, retrieval-augmented generation (RAG), and AI-assisted workflows is a strong plus.
" Ability to develop QA strategies and test approaches for AI-driven and autonomous agent-based applications.

Key Responsibilities:
" Design and execute functional and automation testing for applications.
" Develop and execute automated test scripts using testing tools; document and summarize results.
" Build, update, and maintain UI automation test cases using Selenium with C# (flexibility to learn C# if not already proficient).
" Perform manual API testing using tools such as Postman or Insomnia.
" Perform API automation testing using industry-standard frameworks and tools.
" Participate actively in Agile ceremonies (daily stand-ups, sprint planning, retrospectives).
" Collaborate closely with Development and Business teams to ensure project success.
" Quickly adapt to new technologies and tools (e.g., Amazon Web Services).
" Work with SQL and No

SQL databases for backend validation and robust data testing.
" Create comprehensive end-to-end test plans, test specifications, and test case templates.
" Develop and document detailed test plans and test cases.
" Facilitate defect tracking, reporting, and resolution management.
" Review requirements (Functional Specifications, User Stories, Change Requests) to ensure full coverage.
" Identify and manage test data; prepare automation-ready test cases.
" Conduct system testing, regression testing, and support User Acceptance Testing (UAT).
" Apply test case design techniques (e.g., boundary value analysis, equivalence partitioning).
" Implement Shift Left Testing practices by participating in early requirement and design reviews, writing acceptance criteria up front, and integrating automated tests in CI/CD pipelines.
" Be actively involved in the deployment process, collaborating with Dev Ops and engineering teams to ensure smooth releases.
" Design and execute test strategies for AI/ML and Agentic AI applications, including:
o Validating AI-generated outputs for accuracy, consistency, and reliability.
o Testing AI agents for workflow execution, decision-making, and multi-step task completion.
o Evaluating AI model behavior, prompt responses, hallucinations, and edge cases.
o Verifying AI safety controls, guardrails, security, and compliance requirements.
o Creating test datasets and benchmarks for AI quality assessment.
o Monitoring and reporting AI model performance, accuracy, and user experience metrics.

Nice to Have:
"

Experience with application monitoring tools (e.g., New Relic, Datadog, Cloud Watch).
" Working knowledge of Python for scripting and automation.
" Familiarity with AWS services for cloud-based testing and deployment.
" Hands-on experience with performance and load testing using JMeter
"

Experience with testing AI/ML-powered applications, including validating model outputs, creating test datasets, evaluating model performance.