Software Engineer Calibration

Job Title

Own the formal Deliverable Acceptance process for TOP's external vendor engagement, reviewing each submitted Deliverable against the Acceptance Criteria defined in the vendor SOW and issuing written acceptance or a specific, actionable defect list. Verify vendor container image deliverables against the Software Bill of Materials (SBOM), confirming that all declared components are present and no unapproved components are included. Validate that vendor-delivered AI engine deployments comply with Client's technical architecture requirements, including confirming that fine-tuned model weights are stored in Client's Vertex AI Model Registry and not embedded in container images.

Design, build, and maintain automated test suites for TOP platform backend services, APIs, and data pipelines. Develop AI engine output evaluation frameworks that test inference quality against defined accuracy benchmarks for each dealer service use case. Own the UAT (User Acceptance Testing) process for dealer-facing interfaces, coordinating with Client Service stakeholders to recruit pilot users, design test scenarios, and capture structured feedback.

Manage Jira project bug triage workflow: creating Bug records for confirmed Tier 2 issues, assigning priority in alignment with SLA tiers, tracking vendor acknowledgment and resolution timelines, and reporting SLA compliance metrics. Perform regression testing for each new container image version before Client authorizes production deployment. Define test environments within Client's GCP project space in collaboration with the GCP Cloud Engineer, ensuring test environments accurately reflect production configurations.

Produce quality metrics reports for Client program leadership covering defect rates, SLA compliance, and test coverage across all TOP platform components.

• API - 3-5 years designing, executing, and validating API test cases using tools such as Postman or REST-assured. This includes verifying request/response contracts, authentication flows, error handling, and integration behavior across microservices within the Telemetry & Observability Platform.
• Cloud Infrastructure - 3-5 years of working knowledge of cloud-native infrastructure concepts including containerization, networking, IAM, and storage. Experience validating deployments and testing service behavior in cloud environments is expected.
• Google Cloud Platform - 2-5 years of hands-on experience working within GCP, including familiarity with services such as Pub/Sub, Big Query, GCS, or Cloud Run as they relate to testing data pipelines and telemetry workloads.
• User Acceptance Testing - 2-5 years coordinating and executing UAT cycles with internal stakeholders and platform consumers, ensuring delivered features meet business and functional requirements before production release.
• Application Testing - 3-5 years developing and executing test plans covering functional, regression, and smoke testing across platform applications and services throughout the SDLC.
• Software Testing - 3-5 years of broad software testing experience including unit, integration, system, and end-to-end testing, with the ability to contribute to or maintain automated test suites.
• Test Cases - 2-5 years authoring clear, traceable, and reusable test cases tied to acceptance criteria, with documentation maintained in Jira or a comparable test management tool.
• Jira
  - Ad Hoc Reporting - 2-5 years creating ad hoc Jira queries, dashboards, and filters to surface test coverage, defect trends, and sprint quality metrics for engineering and leadership audiences.
• Artificial Intelligence & Expert Systems - 1-3 years of familiarity with AI-assisted testing approaches, including the use of LLM-based tools for test generation, anomaly detection, or intelligent defect triage within developer workflows.
• Dynatrace - 1-3 years using Dynatrace for observability-driven testing, including leveraging traces, dashboards, and alerting to validate system health and performance during test cycles.
• Quality Assurance Concepts and Standards - 2-5 years applying QA methodologies such as shift-left testing, risk-based testing, and test-driven development,…