AI Retrieval & Relevance Engineer; RAG​/Hybrid Search

Position: AI Retrieval & Relevance Engineer (RAG / Hybrid Search)
About iBusiness

iBusiness is a leading financial technology company transforming the way banks, credit unions, and lenders innovate. As a pioneer in secure AI, automation, and AI software development, iBusiness builds infrastructure and platforms that empower financial institutions to modernize faster-without sacrificing compliance or security. Its technology enables seamless digital transformation across lending, banking, and customer experience systems, giving institutions the tools to compete and innovate at enterprise scale.

Join us and be part of a team that's transforming the finance industry and empowering businesses to thrive!

Position Description

We are seeking an experienced AI Retrieval & Relevance Engineer to architect, implement, and optimize retrieval-augmented generation (RAG) and hybrid search systems that provide accurate, grounded context to LLMs and AI agents. This role owns the retrieval pipeline end-to-end-from indexing strategy and candidate generation through ranking/reranking and evaluation-to ensure our systems efficiently retrieve, contextualize, and support accurate outputs across business applications.

You will collaborate closely with Knowledge Representation engineering to leverage knowledge graphs and semantic signals in retrieval.

Major Areas of Responsibility

RAG Architecture & Hybrid Retrieval

• Architect, implement, and optimize RAG workflows integrating LLMs with retrieval mechanisms (vector search, Elasticsearch, FAISS, Weaviate).
• Implement and optimize dense/sparse/hybrid retrieval strategies, ranking algorithms, reranking, and query rewriting to maximize relevance and accuracy.
• Integrate graph-aware retrieval patterns (entity-centric expansion, metadata filters, constrained traversal) using signals defined by Knowledge Representation.
• Indexing, Ingestion-to-Retrieval Pipelines (Retrieval View)
• Design and maintain scalable pipelines for indexing and retrieval readiness: chunking, embedding, metadata enrichment, index refresh and backfills.
• Ensure reliable retrieval across structured and unstructured data with appropriate filtering, boosting, and context packaging strategies.
  
  Training Data Operations (Retrieval & Evals)
• Orchestrate and scale retrieval-related training/evaluation data operations, including:
  query sets / golden datasets, relevance judgments, regression suites and benchmarks
  lineage and versioning of eval datasets
  
  Evaluation, Observability, and Performance
• Define and run retrieval evaluation: recall@k, nDCG/MRR, context precision, and groundedness/citation success metrics.
• Instrument telemetry and dashboards for retrieval quality, drift, latency (p95/p99), and cost.
• Optimize performance and reliability: caching, rate limiting, tiered retrieval, fallbacks.
  
  Agent Tooling & Addressable Services
• Design and build addressable retrieval services/tools that can be invoked by LLMs and agents to orchestrate workflows (query endpoints, retrieval tools, context assembly services).
  
  Collaboration & Documentation
• Work with Knowledge Representation engineering to align on entity/metadata contracts and provenance signals used in retrieval.
• Maintain clear documentation of retrieval models, pipelines, evals, and runbooks.
• Evaluate and integrate new technologies and research in information retrieval, RAG, and vector search.

Required Knowledge, Skills, and Abilities

• Bachelor's or Master's degree in Computer Science, Data Science, Machine Learning, or related field (or equivalent experience).
• Proven experience designing and tuning information retrieval systems, vector search, and RAG frameworks.
• Strong knowledge of vector and hybrid search technologies (e.g., FAISS, Weaviate, Elasticsearch, Milvus/Pinecone equivalents).
• Proficiency in Python and familiarity with ML tooling (PyTorch/Tensor Flow helpful, especially for rerankers).
• Familiarity with distributed processing/orchestration tools (e.g., Spark, Airflow, Kubeflow) as needed for indexing and eval pipelines.
• Strong analytical and communication skills; able to collaborate cross-functionally.

Nice To Haves

• Experience with rerankers / learning-to-rank, query understanding, and relevance tuning.
• Experience with LLM fine-tuning, prompt engineering,…