Data Scientist

Measure how well AI models and features perform build predictive models where classical machine learning fits better than an LLM and turn usage data into decisions

• Design evaluation methods and build test data sets expected answers edge cases multilingual and adversarial prompts
• Run evaluations and score the results with automated metrics and LLM-as-judge calibrated against human review
• Benchmark models on quality latency and cost and recommend which to use
• Analyse errors including false positives and false negatives of safety and privacy filters
• Build and validate predictive and classification models on tabular time-series and signal data
• Add evaluation tests to CI pipelines so a regression blocks the release
• Analyse usage cost and quality data and build reports and dashboards

• Degree in data science statistics computer science or a related field
• Python with pandas Num Py and scikit-learn
• Statistics precision recall confidence intervals hypothesis testing
• SQL
• Git

• More experience is a plus professional years in data science or applied machine learning and models used in production
• Experience evaluating NLP or LLM systems
• PyTorch or Tensor Flow
• Time-series or signal-processing models
• LLM evaluation tools Langfuse Ragas promptfoo
• Experiment tracking with MLflow or Weights amp Biases
• Experience mentoring other data scientists
• A second language or a background in healthcare or another regulated domain

Degree in data science, statistics, computer science or a related field Python with pandas, Num Py and scikit-learn Statistics: precision, recall, confidence intervals, hypothesis testing SQL Git