Sr. Machine Learning Scientist & Engineer

About Laivly
Seeking curious and creative types! We are an ambitious company of innovators building and shaping the future of customer service technology. Our solutions help the world’s biggest brands leverage artificial intelligence, machine learning, and digital automation in their contact centers to deliver better customer experiences. Led by a team of established contact center experts, Laivly addresses the unique needs and challenges of customer service programs, with an emphasis on ethics in AI and the customer service agent experience.

About the role
The team at Laivly is looking for a Sr. Machine Learning Scientist & Engineer to lead our prompt optimization and validation efforts for LLM‑powered features.

As a Sr. Machine Learning Scientist & Engineer, you’ll work closely with engineering and product teams to design robust, accurate, and scalable experiments and monitoring systems that support safe and reliable deployment of AI features. You’ll own the end‑to‑end design and evaluation of prompts, build synthetic datasets, and develop pre‑ and post‑release validation frameworks that measure performance, safety, and cost‑effectiveness.

This remote role is open to all Canadian Residents.
Responsibilities

Lead prompt design, optimization, versioning, and documentation

Generate and maintain synthetic datasets for pre‑release validation and testing

Design and run experiments to measure accuracy, robustness, and safety of LLM behavior

Define and monitor key metrics such as hallucination rates, latency, token cost, and regression risk

Build evaluation pipelines, dashboards, and drift detection tools

Implement rigorous statistical methods and produce reproducible reports with effect sizes and confidence intervals

Investigate regressions or prompt failures in production and coordinate mitigation with stakeholders

Collaborate with cross‑functional teams on model integration, rate limiting, and cost controls

Maintain prompt libraries, test suites, and operational runbooks

Qualifications

Bachelor’s degree in Computer Science, Computational Linguistics, Statistics, or related field;
Master’s preferred

2+ years working directly with LLMs, prompt design, or applied NLP in production

Experience creating synthetic datasets using programmatic generation or controlled sampling

Deep knowledge of experimental design, statistical inference, and power analysis

Strong Python programming skills and familiarity with statistical/ML frameworks

Experience defining observability for model outputs and instrumenting alerting systems

Familiarity with cloud environments (AWS, Azure), Kubernetes, and production deployment practices

Proficiency in SQL and experience with reproducible analysis pipelines

Excellent communication skills and strong documentation practices

at Laivly
Laivly gives you the opportunity to collaborate and grow your career with a creative, diverse, and passionate team. We work hard and play often, with a flexible environment that works with you. A career at Laivly means being part of a fun‑loving, dedicated team of creatives, risk takers, and game changers. It’s about sharing your talent and imagination to develop innovative tech that’s revolutionizing the way top brands interact with the world.

We’ve got a shared mission—and a Laivly future.

Join us today!

Equal Employment Opportunities
Laivly provides Equal Employment Opportunities in accordance with all provincial and federal laws. Laivly is committed to ensuring equality of opportunity in all aspects of employment and does not discriminate based on protected characteristics.

Accommodation
Laivly is committed to accommodating persons with disabilities. If you need accommodation at any stage of the application process or want more information on our accommodation policies, please let us know.

#J-18808-Ljbffr