Senior Manager, Automated Scoring

Job Overview:
As the Senior Manager of Automated Scoring, you will play an integral part of the machine learning team, which consists of mathematicians, data scientists, psychometricians, linguists, and software engineers. Our team focuses on natural language applications in educational measurement, with key foci in the automated scoring of student responses for both summative and interim assessments, automated feedback for formative assessments, and automated alerting of disturbing content in student responses.

Our team uses state-of-the-art deep learning tools and models different modalities (e.g., speech, text) and is responsible for both prototyping and deploying engines and models. This position will lead a team of scientists on data design and analytic efforts for our suite of products, work closely with internal and external stakeholders to ensure we meet their needs, present at planning and technical advisory meetings, write technical reports and contribute to our growing body of research by publishing, and contribute to enhancements of existing products or to new products.

This position will work closely with the other leads in the machine learning team on developing new applications for automated scoring methods.

• Support CAI’s innovation in natural language applications in support of our mission to improve educational outcomes for our students and educators.
• This includes automated scoring in large scale assessment programs (for writing, speech and other areas of educational measurement), detection of crisis alert content in student writing and the use of ML for feedback in student writing.
• Support CAI’s emerging product portfolio for products that employ state of the art machine learning methods that provide solutions to real world problems in K-12, higher education and other related domains.
• Contributes to R&D agenda and sets priorities for the automated scoring team
• Drive and provide mentorship to a team of Scientists
• Design and implement studies to obtain, score, model, and examine results for any of CAI’s current natural language applications, including automated scoring, crisis alert detection, and feedback.
• Pursue applied research to actively publish and represent CAI’s leadership in this area
• Present at planning and technical advisory committee meetings

• Ph.D. in psychometrics/measurement, NLP, or statistics or related computational field
• 10+ years professional experience in k-12 assessment
• 8+ years working with teams of software developers, NLP scientists, and data scientist
• Experience in working with client-facing work, including presentations
• Established publication record in NLP applications in k-12 assessment .

Remote First Work Environment Our Remote First approach gives employees the flexibility and trust they need to effectively balance work with life. It creates a culture in which all employees are valued and where success is measured in results. It allows us to work collaboratively, inclusively and for greater positive impact, regardless of our individual locations. If you will be working remotely, either occasionally or on a permanent basis, you must have a reliable internet connection through a cable or fiber-optic broadband service with minimum speeds of 10 Mbps download and 5 Mbps upload.

The successful candidate will be expected to actively participate in video-based interviews during the recruiting process and ongoing virtual meetings with their camera on, as part of their role. To maintain confidentiality and ensure a fair evaluation process, the use of note-taking tools, reference materials, or AI-powered tools (including generative AI, language models, or similar technologies) during interviews or other selection…