Computational Linguist, Generative AI - Sr. Associate

We're looking for a versatile Computational Linguist to join our team focused on evaluating and supporting Generative and Agentic AI systems. This role combines linguistic expertise, data analysis, and hands‑on experimentation with large language models. This role is ideal for someone who can move between qualitative language analysis and quantitative evaluation. You'll work cross‑functionally with Machine Learning Engineers, Analytics team and annotators to design innovative, rigorous, and scalable evaluation processes for LLM‑powered workflows.

• Serve as a subject matter expert, engaging with various stakeholders throughout the product lifecycle, and maintain a strong understanding of the Chase Digital Assistant's model from both customer and technical perspectives.
• Manage, monitor, evaluate and version the Chase Digital Assistant's intent and entity taxonomy and the model training; enforce taxonomy versioning practices to ensure traceability and rollback capability.
• Work closely on the adaptation to LLM‑driven workflows, ensuring seamless integration of LLMs with existing conversational AI architectures and event tracking systems.
• Implement and evolve metrics and KPIs across Model Correctness, Customer Experience, AI Assurance, and Business Metrics and ensure evaluation is transparent, repeatable, and release‑decision‑ready.
• Maintain established metrics and introduce new guardrail metrics for LLM and generative use cases.
• Manage full artifact suite for LLM models including descriptions, prompts, evaluation rubrics, LLM‑as‑judge prompts, guidelines, calibration data, data statistics & reliability measures.
• Apply ontology design principles to improve semantic reasoning and data integration aligned to business standards.
• Design frameworks to incorporate knowledge graphs within classification and extraction model architectures.
• Align knowledge models with RAG pipelines and agent orchestration to enhance AI functionality.
• Work with data scientists, software engineers, and business stakeholders to translate requirements into robust solutions.
• Identify optimization opportunities across teams, supporting continuous improvement across model performance, data quality, and feature coverage for improving customer experience.

• Master's degree in Computational Linguistics, NLP, Linguistics, or related field.
• 2+ years experience in Computational Linguistics or NLP applied to chatbot or conversational AI development.
• Hands‑on experience with Generative AI and Agentic AI frameworks and evaluation (e.g., Auto Gen, Lang Graph, CrewAI, Sierra).
• Linguistic background in discourse & pragmatics.
• Advanced knowledge of conversational AI product development lifecycle—training, design, conversation analysis.
• Hands‑on experience with LLM integration, prompt engineering, evaluation, and performance monitoring.
• Proficient in Python, Git, Linux, and Bash scripting.
• Fluent with NLP/data science libraries: pandas, numpy, scikit‑learn, NLTK.
• Experience with transformer‑based models (e.g., BERT, GPT) – fine‑tuning and application.
• Experience with generative AI SDKs and frameworks (e.g., OpenAI, Google, Anthropic, Lang Chain).
• Comfortable with JSONL, CSV, and Jupyter notebook workflows.
• Experience with ontologies/taxonomies and knowledge graphs.
• Solid understanding of evaluation methodologies including human‑AI comparison and red teaming.
• Direct experience with financial institutions, financial products, and customer‑facing queries.
• Chase customer service experience highly desirable.
• Strong written communication for documenting experiments, results, and processes.

• Experience with hybrid conversational architectures, generative AI, and LLM‑driven flow design.
• Familiarity with LLM safety, bias, and compliance.
• Demonstrated success in a highly matrixed organization.
• Awareness of current GenAI trends and evaluation challenges in subjective NLP tasks.
• Ability to solve the cold start problem via synthetic data generation for new intents, flows, and low‑resource scenarios.

We offer a competitive total rewards package including base salary determined based on…