Applied AI Engineer

Overview

Job Title:

Applied AI Engineer

Location:

Remote (Florida based company)
Job Type: Contract and Permanent
Salary: $120K to $140K + Solid Benefits
Job #: 7365

If you believe coding with AI isn't good, then please do not apply to this position... otherwise:

Help us build the future of automation with AI at its core. If you care about shipping real products, solving hard problems with large language models, and building platform capabilities that help others move faster, this is the role for you.

We’re hiring across multiple teams, each with its own focus area in applied AI. Depending on the team, you may work on shared SDKs, evaluation and benchmarking systems, orchestration frameworks, retrieval infrastructure, guardrails, or user-facing AI features. What these teams share is a common operating model: you will ship to production, own meaningful problems end-to-end, and deliver impact across the organization.

Even if this description feels tailored to a very specific candidate, the reality is that the role often adapts to the strengths and experience of the person who joins. If you meet the core criteria below, we encourage you to apply.

Core engineering background

• 5+ years in software engineering, including 3+ years building distributed, cloud-native services (e.g., microservices, event-driven systems, asynchronous workers, API gateways).

• Hands-on experience with service reliability and performance: profiling, latency budgets, throughput tuning, back pressure, rate limiting, caching, and resilience patterns (timeouts, retries, idempotency, circuit breakers).

• Comfortable operating production systems with observability: structured logging, metrics, tracing (Open Telemetry), dashboards, and on-call style debugging.

Applied LLM experience (production-grade)

• 1+ year deploying LLM-powered features in production, including prompt design, tool/function calling, and multi-step workflows that must be reliable under real user traffic.

• Experience with agentic architectures: planners/executors, tool routers, memory management, and guardrailed action execution (e.g., constrained tool schemas, sandboxing, deterministic fallbacks).

• Familiarity with model behavior and failure modes: hallucinations, instruction hierarchy conflicts, tool misuse, prompt injection, and data leakage risks—and practical mitigations.

Model fundamentals and evaluation

• Working understanding of transformer-based models (attention, tokenization, context windows) and how these constraints influence product and system design.

• Experience building LLM evaluation pipelines:

  • Offline and online evals (golden datasets, regression tests, shadow deployments, A/B tests).

  • Metrics such as task success rate, exact match / rubric scoring, faithfulness, latency, cost per successful task, and safety outcomes.

  • Methods like judge-model scoring, human review workflows, and adversarial test sets.

• Exposure to prompt/version management and reproducible experimentation (dataset versioning, prompt diffs, model snapshot tracking).

Retrieval and knowledge systems (RAG)

• Experience designing and operating Retrieval-Augmented Generation systems:

  • Document ingestion, normalization, chunking strategies (semantic, recursive, structural), metadata enrichment, and deduplication.

  • Vector search and hybrid search (BM25 + dense retrieval), query rewriting, re-ranking, and caching strategies.

  • Managing vector stores and indexes, and tuning for latency, recall, and precision under load.

• Understanding of how retrieval choices affect grounding, citation quality, and freshness, and how to detect retrieval regressions.

• Experience deploying services on cloud infrastructure (AWS/GCP/Azure), including containers (Docker), orchestration (Kubernetes/ECS), and CI/CD automation.

• Comfort with storage and messaging primitives: relational DBs, Redis, queues/streams, object storage, and background job frameworks.

• Ability to document trade-offs clearly (quality vs. latency vs. cost; determinism vs. flexibility; synchronous vs. async execution).

• You love shipping. You translate ambiguous user needs into measurable outcomes and production-ready implementations,…