Senior AI Data Engineer​/Data Scientist

Billennium is a global technology company with over 20 years of experience, committed to innovation and empowering businesses. As an employer, we offer a supportive, growth-focused environment where collaboration and creativity thrive. Join us to shape the future of technology together!

We are looking for a Senior AI Data Engineer / Data Scientist who can turn messy enterprise data into AI-ready, high-quality knowledge assets.

You will lead the cleanup, preparation, and enrichment of unstructured content (SharePoint/document repositories) and structured/semi-structured data (data lakes, databases) so our agents, copilots, and RAG systems are accurate, trustworthy, and scalable.

This is a senior, hands-on role. You will own data quality outcomes end-to-end: discovery - cleanup - enrichment - ingestion - refresh cycles - governance. We value AI-native generalists who can remove bottlenecks by working directly with AI Engineers, Architects, and business stakeholders to decide what data is worth using and how to structure it for retrieval and reasoning.

Our standardized stack includes (and this role actively uses it): ingestion/ETL foundations, Postgres + pgvector as default RAG store, Redis caching, LLM gateway patterns, Langfuse observability, Deep Eval/RAGAS evaluation, and Presidio for PII detection/masking when required.

• 5+ years in data engineering / applied data science / analytics engineering with ownership of production pipelines.
• Proven experience working with unstructured enterprise data (documents, PDFs, Office files, wikis, knowledge bases).
• Solid understanding of data quality engineering: validation, monitoring, lineage, refresh cycles.
• Strong stakeholder skill
  : can work with business to define what data matters and what “good” looks like.

• Experience with Postgres + pgvector (or similar vector stores), retrieval optimization, and hybrid search concepts.
• Familiarity with observability practices for AI pipelines and the use of RAG evaluation metrics (RAGAS-style).
• Experience with governance tooling and privacy controls for enterprise AI (e.g., PII workflows).

• Lead “data triage” for AI use cases: identify authoritative sources, duplicates, outdated content, and low-quality documents.
• Clean, normalize, deduplicate, and standardize enterprise content at scale (documents, PDFs, Word/Excel, wiki pages, etc.).
• Define what data should be excluded from AI systems (stale, contradictory, low-trust, or sensitive content).
• Unstructured ingestion (SharePoint + document repositories)
• Build robust ingestion pipelines for SharePoint and file repositories: parsing, text extraction, structure recovery, and metadata capture.
• Implement document normalization strategies (naming, taxonomy, metadata standards, canonical IDs).
• Design chunking strategies, metadata enrichment, and document structuring optimized for retrieval performance and cost.
• Improve retrieval quality through practical techniques such as filtered retrieval and post-retrieval optimization where appropriate (e.g., reranking), collaborating with AI Engineers on the retrieval interface.
• Prepare and maintain “AI-ready knowledge sets” that can be embedded and served via Postgres + pgvector (default).
• Data quality, evaluation, and feedback loops (non-negotiable)
• Define and implement data quality gates (freshness, completeness, relevance, dedupe rate, metadata coverage).
• Partner with AI Engineers to evaluate retrieval and RAG performance using frameworks like RAGAS (answer correctness, context recall/precision) and to monitor trust metrics over time.
• Establish human feedback loops where needed (review queues, sampling, targeted audits) to continuously improve data usefulness and user trust. Governance, privacy, and auditability
• Apply privacy and enterprise constraints; where required, implement PII detection/masking using Presidio patterns.
• Reuse Package reusable “data cleanup + RAG readiness” recipes: ingestion templates, metadata schemas, chunking playbooks, dedupe strategies.
• Build a repeatable data foundation that accelerates future use cases (not a one-off cleanup project).

• Comprehensive be…