×
Register Here to Apply for Jobs or Post Jobs. X

AIOps Technical Associate

Job in Orlando, Orange County, Florida, 32885, USA
Listing for: Milestone Technologies, Inc.
Full Time position
Listed on 2026-02-17
Job specializations:
  • IT/Tech
    Data Analyst, AI Engineer, Data Science Manager
Salary/Wage Range or Industry Benchmark: 60000 - 80000 USD Yearly USD 60000.00 80000.00 YEAR
Job Description & How to Apply Below

Overview

22-month W2 Contract (No C2C/No Visa Sponsorship/No Student Sponsorship).
Max rate for FL: $77.50, for CA/WA: $91.00 (no PTO and no paid holidays).
Hybrid role: 2-4 days/week onsite in Orlando, Glendale, Anaheim, or Seattle.
All submissions must include a completed questionnaire for client review.

Experience & Requirements

2‑4 years in Operations/Technical Operations (MLOps, AIOps, Data Ops, Platform Ops, or similar).
Must have experience with AI/ML concepts and Cloud Cost Management / Fin Ops.

Required:

GCP, SQL, reporting tools, basic Python, and monitoring/observability tools.

Overall Responsibilities
  • Manage operational workflows for model deployments, updates, and versioning across GCP, Azure, and AWS.
  • Monitor model performance metrics: latency, throughput, error rates, token usage, and inference quality.
  • Track model drift, accuracy degradation, and performance anomalies—escalating to engineering as needed.
  • Support knowledge base operations including vector embedding pipeline health, chunk quality, and refresh cycles in Vertex AI.
  • Maintain model inventory and documentation across multi‑cloud environments.
  • Coordinate model evaluation cycles with Responsible AI and Core Engineering teams.
Agent & MCP Server Operations
  • Monitor AI agent health, performance, and reliability (Auto Gen‑based agents, MCP servers).
  • Track agent execution metrics: task completion rates, tool call success/failure, latency, and error patterns.
  • Support agent deployment and configuration management workflows.
  • Document agent behaviors, known issues, and operational runbooks.
  • Coordinate with Core Engineering on agent updates, testing, and rollouts.
  • Monitor MCP server availability, connection health, and integration status.
Fin Ops & Cost Management
  • Track and analyze AI/ML cloud spend across GCP (Vertex AI), Azure (OpenAI), and AWS (Bedrock).
  • Build cost dashboards with breakdowns by model, application team, use case, and environment.
  • Monitor token consumption, inference costs, and embedding/storage costs.
  • Identify cost optimization opportunities—model selection, caching, batching, rightsizing.
  • Provide cost allocation reporting for chargeback/showback to consuming application teams.
  • Forecast spend trends and flag budget anomalies.
  • Partner with Infrastructure and Finance teams on AI cost governance.
Monitoring, Dashboarding & Reporting
  • Build and maintain dashboards for platform performance, model health, agent metrics, and operational KPIs.
  • Create executive and stakeholder reports on platform adoption, usage trends, and cost allocation.
  • Develop Responsible AI dashboards tracking hallucination rates, accuracy metrics, guardrail triggers, and safety incidents.
  • Monitor APIGEE gateway traffic patterns and API consumption trends.
  • Provide regular reporting to product management on use case performance.
Release Operations Support
  • Support release management processes with pre/post‑deployment validation checks.
  • Track release health metrics for models, agents, and platform components.
  • Maintain release documentation, runbooks, and operational playbooks.
  • Coordinate with QA, Performance Engineering, and Infrastructure teams during releases.
AI Operations
  • Monitor guardrail effectiveness and flag anomalies to the Responsible AI team.
  • Track and report on hallucination detection, content safety triggers, and accuracy trends.
  • Support LLM Red Teaming efforts by collecting and organizing evaluation data.
  • Maintain audit logs and compliance documentation for AI governance.
Cross‑Functional Coordination
  • Serve as operational point of contact for application teams consuming DxT AI APIs.
  • Coordinate with Corporate Security on audit requests and compliance reporting.
  • Partner with Infrastructure team on capacity tracking and resource utilization.
  • Support Performance Engineering with load test analysis and results documentation.
Basic Qualifications
  • 2‑4 years in an Ops, Analytics, or Technical Operations role (MLOps, AIOps, Data Ops, Platform Ops, or similar).
  • Understanding of AI/ML concepts: models, inference, embeddings, vector databases, LLMs, tokens, prompts.
  • Experience with cloud cost management and Fin Ops—tracking, analyzing, and optimizing cloud spend.
  • Strong proficiency with dashboarding…
Position Requirements
10+ Years work experience
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
 
 
 
Search for further Jobs Here:
(Try combinations for better Results! Or enter less keywords for broader Results)
Location
Increase/decrease your Search Radius (miles)

Job Posting Language
Employment Category
Education (minimum level)
Filters
Education Level
Experience Level (years)
Posted in last:
Salary