Senior Data Scientist, NLP & LLM Fine-Tuning

Xenoss isanAI engineering and integration services company. Wehelp medium tolarge enterprises runAI transformation end-to-end: situation analysis and goal framing, data discovery and preparation, pipeline building, model development, retraining pipeline design, deployment, and support.

Webuild abroad range ofAIsolutions: user behaviour prediction, content generation, NLP, audience segmentation, pathfinding, AI assistants, edge computer vision, fraud detection, and more.

Wework with prominent companies such as Microsoft, Toshiba, AstraZeneca, Activision Blizzard, Verve Group, Voodoo Games, and Telefonica.

Weare inthe top 100 software companies onthe Inc.
5000list.

Weare looking for aSenior Data Scientist for along-term In-Call Assistant initiative with aworld-leading financial services company.

The project focuses on building areal-time conversational AI system that supports front-office employees during live customer conversations. The system identifies customer needs, objections, buying signals, and required process steps, and provides concise, context-aware recommendations.

You will primarily work onthe signal detection and trigger layer: turning live conversation streams into structured signals, confidence scores, and routing decisions under strict latency requirements.

The broader solution combines low-latency signal detection, context preparation, specialist recommendation generation, RAG over approved product and policy knowledge, and compliance guardrails.

The data islarge and messy. It includes over one million speech-to-text call transcripts with PIIredaction, ASR errors, and nogold labels.

Text data processing

• Build scalable pipelines toclean and normalise noisy ASR call transcripts.
• Handle PII-redacted text with typed placeholders. Detect and measure redaction errors.
• Segment conversations into turns. Repair turns broken by interruptions and overlap.
• Remove near-duplicate and low-quality samples (Min Hash/LSH, embedding similarity, heuristic and model-based quality filters).
• Mine and cluster objections and agent responses with embeddings and topic models.
• Build train, validation, and test sets with time-based splits and noleakage.

Supervision and labeling

• Build training targets from outcome signals: next customer reaction, call conversion, and agent performance.
• Create labels withLLM-assisted annotation. Validate them with sales subject-matter experts.
• Apply weak supervision and label-noise detection toweakly labeled data.
• Build preference datasets: chosen/rejected pairs for DPO and good/bad pools for KTO.
• Balance data across ataxonomy ofaround 50objection types.

LLM fine-tuning and post-training

• Run and improve supervised fine-tuning (SFT) with LoRA/QLoRA and full-parameter methods.
• Run preference optimization: DPO, KTO, ORPO, SimPO, and iterative on-policy DPO.
• Run reinforcement learning atturn level with GRPO (and PPO where useful).
• Train reward models for response quality and conversion likelihood.
• Design composite rewards (reward model + LLM judge + rule checks). Detect and limit reward hacking withKL control and reward ensembles.
• Run best-of-N sampling and reranking asastrong baseline.
• Train onasingle node of8× H100 80GB GPUs with Deep Speed ZeRO-3or FSDP.
• Prototype onsmaller models, then scale tothe 100B+ MoE model.

Evaluation

• Design offline evaluation for generated responses when nogold answer exists.
• Build LLM-as-judge rubrics: relevance, empathy, clarity, factual accuracy, compliance, and how easy the response istosay aloud.
• Use pairwise comparison with position swap. Calibrate judges against expert labels.
• Measure agreement between judges and experts (Cohen’s kappa, Krippendorff’s alpha).
• Estimate business impact…