Computational Linguist, Generative AI - Sr. Associate

:

Category:
Analytics Solutions & Delivery

Job Schedule:

Full time

Posted Date: T19:54:47+00:00

Job Shift: Day

Base Pay/Salary:
New York,NY $-$

We're looking for a versatile Computational Linguist to join our team focused on evaluating and supporting Generative and Agentic AI systems. This role combines linguistic expertise, data analysis, and hands-on experimentation with large language models.

This role is ideal for someone who can move between qualitative language analysis and quantitative evaluation. You'll work cross-functionally with Machine Learning Engineers, Analytics team and annotators to design innovative, rigorous, and scalable evaluation processes for LLM-powered workflows.

Core Responsibilities

* Serve as a subject matter expert, engaging with various stakeholders throughout the product lifecycle, and maintain a strong understanding of the Chase Digital Assistant's model from both customer and technical perspectives.

* Manage, monitor, and evaluate and version the Chase Digital Assistant's intent and entity taxonomy and the model training;
Enforce taxonomy versioning practices to ensure traceability and rollback capability

* Work closely on the adaptation to LLM-driven workflows, ensuring seamless integration of LLMs with existing conversational AI architectures and event tracking systems.

* Implement and evolve metrics and KPIs across Model Correctness, Customer Experience, AI Assurance, and Business Metrics and ensure evaluation is transparent, repeatable, and release-decision-ready

* Maintain established metrics and introduce new guardrail metrics for LLM and generative use cases

* Manage full artifact suite for LLM models including descriptions, prompts, evaluation rubrics, LLM-as-judge prompts, guidelines, calibration data, data statistics & reliability measures.

* Apply ontology design principles to improve semantic reasoning and data integration aligned to business standards.

* Design frameworks to incorporate knowledge graphs within classification and extraction model architectures.

* Align knowledge models with RAG pipelines and agent orchestration to enhance AI functionality.

* Work with data scientists, software engineers, and business stakeholders to translate requirements into robust solutions.

* Identifying optimization opportunities across teams, supporting continuous improvement across model performance, data quality, and feature coverage for improving customer experience.

Required Qualifications , Capabilities, and Skills

* Master's degree in Computational Linguistics, NLP, Linguistics, or related field

* 2+ experience in Computational Linguistics or NLP applied to chatbot or conversational AI development

* Hands-on experience with Generative AI and Agentic AI frameworks and evaluation (e.g., Auto Gen, Lang Graph, CrewAI, Sierra)

* Linguistic background in discourse & pragmatics

* Advanced knowledge of conversational AI product development lifecycle - training, design, conversation analysis

* Hands-on experience with LLM integration, prompt engineering, evaluation, and performance monitoring

* Proficient in Python, Git, Linux, and Bash scripting

* Fluent with NLP/data science libraries: pandas, numpy, scikit-learn, NLTK

* Experience with transformer-based models (e.g., BERT, GPT) - fine-tuning and application

* Experience with generative AI SDKs and frameworks (e.g., OpenAI, Google, Anthropic, Lang Chain)

* Comfortable with JSONL, CSV, and Jupyter notebook workflows

* Experience with ontologies/taxonomies and knowledge graphs

* Solid understanding of evaluation methodologies including human-AI comparison and red teaming

* Direct experience with financial institutions, financial products, and customer-facing queries

* Chase customer service experience highly desirable

* Strong written communication for documenting experiments, results, and processes

Preferred Qualifications

* Experience with hybrid conversational architectures, generative AI, and LLM-driven flow design

* Familiarity with LLM safety, bias, and compliance

* Demonstrated success in a highly matrixed organization

* Awareness of current GenAI trends and evaluation challenges in subjective NLP tasks

* Solve the cold start problem via synthetic data generation for new intents, flows, and low-resource scenarios.