AI Modeler Job New York City area,New York USA,Software Development

Job Title: AI Modeler
Location: Malvern, PA/ Charlotte, NC
Duration:
Fulltime

Skills:
Artificial Intelligence | Python
Salary: 130K-160K/Year

Roles and Responsibilities:
Must Have Technical/Functional Skills

10+ years of software engineering experience with Python as your primary language
Hands-on experience with Amazon Bedrock model invocation, Agents, Knowledge Bases, and Guardrails
Experience with prompt engineering designing, versioning, and evaluating prompts in a production context, not just experimentation
Familiarity with LLM evaluation patterns: ground truth datasets, accuracy measurement, regression testing for model outputs
Strong AWS fundamentals:
Lambda, S3, SQS, SNS, Event Bridge, Cloud Watch, IAM, Secrets Manager
Experience with agentic patterns and tool use building systems where the model calls external APIs as part of a reasoning chain
Comfortable working with structured and unstructured document formats (PDF, Excel, DOCX) and document intelligence pipelines (AWS Textract or equivalent)
AWS ML certification is a plus; a demonstrated track record working with Bedrock and Foundation Models in production carries equal weight.
Understanding of how to build observable AI systems logging model inputs/outputs, tracking token costs, detecting accuracy drift in production

Nice to Have

Experience with traditional ML frameworks (PyTorch, scikit-learn) helpful for understanding what is being replaced
Background in high-stakes operational domains financial services, insurance, healthcare, or any environment where data accuracy and auditability are non-negotiable
Experience with PostgreSQL, and event-driven Lambda architectures
Prior work on parallel-run or shadow-mode validation patterns for AI system migrations

Roles & Responsibilities

We are modernizing core automation platforms that sit at the heart of our financial operations processing systems that handle high-volume document intelligence, transaction risk assessment, and intelligent workflow routing. You will be a key technical contributor on a high-visibility initiative, working alongside a Senior AI Engineer to migrate these platforms from traditional machine learning to Amazon Bedrock Foundation Models. The work directly impacts operational efficiency, compliance confidence, and the speed at which our teams can process and settle client transactions.

This is a production-grade AI engineering role at the intersection of financial services and cutting-edge cloud AI with real business outcomes tied to what you build.
This is a 100% hands-on individual contributor role. You will be writing code, building pipelines, and shipping to production. As one of the most experienced engineers on the team in this space, you will also be expected to mentor junior and mid-level engineers, sharing knowledge through pairing, code review, and hands-on guidance.

What You Will Do

Design, build, and optimize Bedrock-powered pipelines replacing custom-trained NLP and rule-based models across multiple production Lambda functions
Develop and iterate on prompts, agentic workflows, and tool-use patterns that extract transaction attributes from complex financial documents PDFs, Excel, Word, and images
Implement Bedrock Knowledge Bases to replace hard-coded business rules and fragile per-invocation API calls with managed, queryable reference data
Build and maintain an evaluation harness to measure FM extraction accuracy against histori cal ground truth, enforcing accuracy gates before any production promotion
Integrate Bedrock Agents with internal APIs using tool-use patterns, replacing legacy direct database queries
Participate in parallel-run infrastructure running old and new systems side by side in TEST, logging divergence, and driving statistical validation before cutover
Contribute to team knowledge transfer; this role works closely with the Senior AI Engineer who leads FM and Bedrock capability building across the teams

Generic Managerial Skills, If any

Planning & Execution Effectively prioritize tasks, estimate effort, and ensure timely delivery of project milestones while balancing multiple commitments.
Stakeholder Management Build strong relationships with business teams, clients, and leadership through proactive communication and expectation management.
Communication Convey technical and non-technical information clearly through meetings, presentations, documentation, and status updates.
Collaboration Foster teamwork across cross-functional groups, encourage knowledge sharing, and contribute to a positive team environment.
Decision Making Analyze options, assess risks, and make informed decisions that align with both technical and business objectives.
Leadership & Ownership Take accountability for deliverables, drive initiatives proactively, and lead by example to achieve team goals.
Risk & Issue Management Identify potential risks early, address blockers proactively, and implement mitigation strategies to ensure smooth project execution.
Customer & Business Focus Understand customer needs and…