AI Engineer

About Us

The Patient-Centered Outcomes Research Institute (PCORI) is an independent nonprofit organization authorized by Congress in 2010. Its mission is to fund research that will provide patients, their caregivers and clinicians with the evidence-based information needed to make better-informed healthcare decisions. PCORI is committed to continually seeking input from a broad range of stakeholders to guide its work

Position Summary:

PCORI is seeking a highly motivated and innovative AI Engineer to design, build, and deploy advanced AI Capabilities. Examples include Natural Language Processing (NLP) and Machine Learning (ML) models that automate portfolio coding and classification tasks. This role will focus on developing text classification models to analyze and code Letters of Intent (LOIs) and research plans against PCORI's taxonomy framework-improving efficiency, accuracy, and scalability of manual coding processes.

We are also looking for an AI engineer who can support in deploying employee productivity tools (e.g., Microsoft Copilot).

The AI Engineer will be part of the Data & Technology Infrastructure team reporting to the Director of the unit and will partner and collaborate with all teams at PCORI to strategize, define, and implement artificial intelligence and machine learning solutions that solve complex business challenges.

In this role, you will work closely with cross-functional teams to leverage data, optimize algorithms, and deliver intelligent systems that enhance decision-making and automation in PCORI

Job Duties and Responsibilities:

Key responsibilities

* Design, build, and deploy AI/ML models to address business needs

* Collect, preprocess, and analyze structures and unstructured data

* Collaborate with product, engineering, PMO, security, and business team to identify AI opportunities

* Develop and maintain scalable data pipelines and model training environments

* Monitor model performance, retrain, and improve algorithms as needed

* Stay up to date with the latest AI/ML research, frameworks and tools

* Ensure AI systems comply with ethical, security, and regulatory requirements

Responsibilities

* Deploy AI solutions (e.g., Microsoft Copilot)

* Design, develop, and optimize NLP models (e.g., BERT, PubMedBERT, GPT-family) for multi-label classification of research documents

* Align AI models with taxonomy domains and coding frameworks, ensuring outputs meet organizational standards

* Build systems to generate annotated outputs that highlight coded sections of research documents for enhanced human review.

* Evaluate model performance against gold-standard datasets using accuracy, precision, recall, and F1-score.

* Deploy and manage models in production using MLOps practices (experiment tracking, monitoring, retraining).

* Collaborate with data engineers, analysts, and domain experts to refine models and integrate them into operational workflows.

* Explore advanced approaches such as LLM integration, retrieval-augmented generation (RAG), and semantic search.

* Work with Azure (preferred) or other cloud platforms to build scalable and secure deployment pipelines.

Incumbent(s) in this position may be required to perform other duties and special assignments not specifically stated above.

Statements outlined in this section are designated as essential job functions in accordance with the Americans with Disabilities Act of 1990.

Required

Skills and Experience:

* Master's degree in computer science, Business Analytics, Data Science, Engineering, Applied Mathematics, or related field

* Proven experience in Machine Learning and NLP, particularly in text classification and multi-label problems

* Strong programming skills in Python (PyTorch/Tensor Flow, Hugging Face Transformers), R or Java, and frameworks like Tensor Flow, PyTorch, or Scikit-learn

* Familiarity with end-to-end software development for AI models, from training to deployment

* Understanding of MLOps tools (MLflow, Weights & Biases, or similar)

* Experience with cloud platforms (Azure preferred; AWS or GCP acceptable).

* Solid understanding of algorithms, statistics, and model evaluation techniques

* Strong problem-solving skills and ability to work in a team-oriented environment

Preferred Skills

* Experience working with LLMs (GPT models, Microsoft OpenAI Service)

* Knowledge of retrieval-augmented generation (RAG) and semantic search techniques

* Proficiency in data augmentation techniques to expand small datasets

* Exposure to Lang Chain, agent workflows, or enterprise AI frameworks

Compensation and Benefits

Salary Range: $125,000 - $145,000

The above range represents the salary range expected for the position; however, final offers are based on several factors such as the position's responsibilities; the candidate's experience, education, and skills; location; travel required; and current market conditions.

Subject to the terms and conditions of the applicable plans then in effect, eligible employees may participate in PCORI-sponsored medical, dental, vision, and basic…