Senior Software Engineer AI

Overview

Make Your Mark:
We’re looking for a Senior AI/ML Engineer to design, build, and optimize data pipelines that power our next-generation AI-driven accounting agents. You’ll lead the development of scalable, high-performance data infrastructure while collaborating closely across teams.

• Lead data pipeline development:
  Build and maintain PySpark ETL pipelines with high data quality and performance.
• Manage integrations:
  Establish robust connections to client data sources via APIs and tools like Five Tran, Plaid, and Black Line’s internal connector ecosystem.
• Ensure reliability:
  Monitor pipeline performance, automate testing, and validate data accuracy.
• Optimize for scale:
  Implement performance improvements (e.g., CDC mechanisms, indexing strategies) for large‑scale datasets.
• Collaborate & innovate:
  Work with business stakeholders to refine data requirements and integrate cutting‑edge AI and big data technologies.
• Lead incident response and reliability strategies for ML/AI systems.
• Collaborate with development teams to integrate AI solutions into existing workflows and applications.
• Define and manage MCP Registry for agentic component onboarding, lifecycle versioning, and dependency governance.
• Build CI/CD pipelines automating LLM agent deployment, policy validation, and prompt evaluation of workflows.
• Develop and operationalize experimentation frameworks for agent evaluations, scenario regression, and performance analytics.
• Implement logging, metering, and auditing for agent behavior, function calls, and compliance alignment.
• Create scalable observability systems: tracking conversation outcomes, factual accuracy, latency, escalation patterns, and safety events.
• Architect end‑to‑end guardrails for AI agents including prompt injection protection, identity‑aware routing, and tool usage authorization.
• Collaborate cross‑functionally to standardize authentication, authorization, and session governance for multi‑agent runtimes.
• Architect and standardize model registries and feature stores to support version tracking, lineage, and reproducibility across environments.
• Lead the deployment of machine learning models into production environments, ensuring scalability, reliability, and efficiency.
• Collaborate with software engineers to integrate machine learning models into existing applications and systems.
• Implement and maintain APIs for model inference.
• Design and manage training infrastructure including distributed training orchestration, GPU/TPU resource allocation, and automatic scaling.
• Implement CI/CD for model workflows using pipelines integrated with model validation, bias checks, and rollback automation.
• Build standardized experimentation frameworks for reproducible training, tuning, and deployment cycles (MLflow, W&B, Kubeflow).
• Manage and optimize the infrastructure required for machine learning operations in the cloud.
• Collaborate with other teams to ensure availability, security, and performance of machine learning systems.
• Implement robust monitoring solutions for deployed machine learning models to detect issues and ensure performance.
• Collaborate with data scientists and engineers to address and resolve model performance and data quality issues.
• Conduct regular system maintenance, updates, and optimizations to ensure optimal performance of machine learning solutions.
• Develop and maintain automation scripts and tools for managing machine learning workflows.
• Implement orchestration systems to streamline the end‑to‑end machine learning lifecycle, from data preparation to model deployment.
• Facilitate the transition of machine learning models from research to production, ensuring scalability and efficiency.
• Identify and implement optimizations to enhance the performance and efficiency of machine learning models in production.
• Conduct performance analysis and implement improvements based on resource utilization.
• Implement security measures to protect machine learning systems and data.
• Ensure compliance with regulatory requirements and industry standards related to machine learning and data privacy.
• Integrate audit controls, metadata storage, and lineage tracking across ML and AI workflows.
• Ensure complete…