AI Model Benchmarking Intern; Remote, Iowa-

• Rate: $22 – $28/hr
• Hours: 20 hours / week
• Experience: 1 - 3 years
• Location: United States | Canada + 23 more

Part-time, 15–20 hours/week
· Remote, Iowa-based
· 6-month internship · $22–$28/hour

Sure Host is an early-stage hospitality technology company based in Des Moines, Iowa. We help growing short-term rental operators confirm each property is guest-ready after a cleaning turn, using documented property standards, photo and video evidence, AI-assisted review, and human review where it matters.

We are working with an initial paying customer and preparing for broader growth in the U.S. market. Our customer and product work happens in Des Moines, and our technical team is based in Pune, India. Our stack is Python and Flutter, deployed on GCP.

Sure Host uses AI narrowly, where it can be measured and trusted. Language and vision models change quickly, and a model that looks strong in a demo can still fail on real property footage, cost too much to run, or make decisions an operator cannot follow.

In this role, you will help us tell the difference. You will build repeatable benchmarks that compare LLMs and VLMs on Sure Host use cases, and your findings will inform model choices and the product roadmap. You will work with our AI Chief Architect.

• Research and compare evolving LLM and VLM models that may support Sure Host's product use cases.
• Create representative evaluation datasets, test cases, scoring criteria, and repeatable benchmark workflows.
• Measure model quality across accuracy, reliability, latency, cost, and other product-relevant factors.
• Analyze failure patterns and trade-offs, including false positives and false negatives on issues operators care about.
• Present clear findings and recommendations to inform model strategy and the product roadmap.
• Maintain benchmark results and documentation as models, APIs, and requirements change.

• Interest in applied AI and familiarity with LLMs, VLMs, generative AI tools, or model APIs through coursework or projects.
• Logical, evidence-based thinking and comfort working with structured data and evaluation criteria.
• Basic Python or data-analysis skills.
• The ability to document methods and results clearly.
• Patience to run repeatable experiments, investigate edge cases, and separate a compelling demo from dependable performance.
• Clear communication, willingness to ask questions early, and responsible handling of customer, property, and company information.

Required

• Based in Iowa.

• Pay: $22–$28/hour, based on experience and how independently you can work.
• Hours: 15–20 hours per week, with flexible scheduling.
• Location: Remote, for Iowa-based candidates.
• Term: 6-month internship.
• Start date: Flexible. We are onboarding now and can work around school schedules, finals, and the holidays.
• Works with: AI Chief Architect.
• Team time: A monthly in-person check-in over coffee in Des Moines.

• Real responsibility on a live hospitality technology product, with few layers between customer needs and product decisions.
• Direct mentorship from the AI Chief Architect on applied model evaluation.
• Work you can point to: benchmark methods, results, and recommendations to reference in future roles.

Company:
Sure Host