Software Engineer, Software Testing, Python

• Complete structured evaluation assignments that mirror authentic developer work, including version control, code review, issue tracking, and CI/CD workflow execution with AI-powered tools.
• Assess and grade technical outputs for correctness, validating results directly in relevant development environments and tools.
• Design, populate, and maintain reproducible test environments, including repositories with real commit histories and CI workflows.
• Configure, document, and troubleshoot integrations and authentication flows across multiple software services.
• Investigate and document previously undocumented product behavior through hands‑on exploration and systematic testing.
• Participate in calibration sessions to maintain consistent evaluation and grading practices across contributors.

• At least 3 years of professional software engineering experience.
• Strong Git and Git Hub workflow knowledge, including pull requests, code review, branching, diff analysis, and CI log diagnostics.
• Proficiency with CI/CD systems, ideally Git Hub Actions, including writing custom workflows and supporting reproducible builds.
• Experience using connectors across Claude and ChatGPT.
• Advanced Python or Bash scripting skills, including automation of environment setup and reset tasks.
• Solid understanding of REST APIs, OAuth, and webhook integrations.
• Working knowledge of enterprise SaaS platforms such as Slack, Google Workspace, or Microsoft 365. Cloud workspace administration experience is advantageous.
• Experience using AI assistants or agent tools in day‑to‑day development work. Familiarity with agent tool‑calling or the Model Context Protocol, MCP, is a plus.
• Technical QA, evaluation, or data annotation experience, with the ability to apply rubrics consistently and identify nuanced edge cases.
• Exceptional attention to detail, disciplined process adherence, and excellent written and verbal English communication.

Remote contract engagement.

$50 to $70 per hour.