Prompt Engineer

Remote (US-Based)
· Optional hybrid in New York, NY

Locked In AI is the #1 real-time AI interview and meeting copilot, trusted by over one million users worldwide. We are a fast-growing company building the most advanced career preparation platform on the market.

Our platform delivers real-time, AI-powered assistance during live job interviews, coding assessments, and professional meetings — helping candidates communicate with clarity, confidence, and competence.

We are looking for a creative, technically sharp Prompt Engineer to design, craft, and optimize the prompts and instruction sets that power Locked In AI’s core AI experience. This is a hands-on, high-leverage role — the quality of every AI-generated response our users see flows directly through your work.

You will own the full prompt lifecycle — from understanding user needs and model behavior, to writing and testing prompt architectures, to evaluating output quality and iterating based on real-world performance data. You will work across multiple LLM providers (OpenAI, Anthropic, open-source models) and develop prompt strategies that maximize accuracy, relevance, tone, and speed while minimizing hallucination, cost, and latency.

The ideal Prompt Engineer combines exceptional language skills with a deep, practical understanding of how large language models work. You think in systems — building reusable prompt templates, chain-of-thought architectures, and evaluation rubrics — not just one-off instructions. You are obsessive about output quality and relentless about testing and iteration.

• Design, write, and optimize production-grade prompts and system instructions that drive Locked In AI’s real-time copilot across interview coaching, coding assistance, and meeting support use cases
• Develop and maintain prompt architectures including system prompts, few-shot examples, chain-of-thought reasoning, role-based instructions, and multi-turn conversation management
• Apply advanced prompt engineering techniques — including zero-shot, few-shot, chain-of-thought, self-consistency, tree-of-thought, and retrieval-augmented prompting — to maximize output quality for different product scenarios
• Build and maintain a structured prompt library and repository, documenting prompt patterns, version history, and performance benchmarks for reuse across the platform

• Design and execute rigorous prompt evaluation frameworks with clearly defined metrics — including accuracy, relevance, tone alignment, hallucination rate, response completeness, and latency
• Build and maintain benchmark datasets, golden answer sets, and scoring rubrics to systematically measure prompt performance across diverse user scenarios and edge cases
• Conduct systematic A/B tests and comparative experiments across prompt variations, model versions, and LLM providers to identify the highest-performing configurations
• Analyze prompt failure modes, identify patterns in low-quality outputs, and rapidly iterate on prompt designs to close quality gaps
• Work across multiple LLM providers (OpenAI, Anthropic, Google AI, open-source models) to understand model-specific behaviors, strengths, and limitations — and tailor prompt strategies accordingly
• Develop model routing logic and prompt adaptation layers that optimize for the best combination of quality, latency, and cost across different use cases and user segments
• Monitor and adapt prompts as models are updated or replaced, ensuring consistent output quality through model version changes and provider migrations
• Track and optimize token usage, prompt length, and inference costs — balancing response quality with sustainable unit economics

• Design prompts that effectively leverage retrieval-augmented generation (RAG) pipelines, ensuring AI responses are grounded in accurate, relevant, and up-to-date knowledge sources
• Develop context management strategies for real-time, multi-turn conversations — optimizing how conversation history, retrieved documents, and user context are structured within prompt context windows
• Collab…