Digital Experimentation, Analyst II

OVERVIEW

Discover the possibilities of our progressive, omnichannel approach to beauty retail. At Ulta Beauty, our E-commerce associates sit at the cutting edge of our efforts to reimagine the future of retail. Through a full suite of new and emerging technologies, this team is creating an enhanced buying experience for guests who crave simplicity, choice, personal insights and the joy of discovery.

With greater interactivity. Inspired applications of big data, AI and UI/UX. And creative that persuades and delights. All of this, with the opportunity to experience smart ideas come to life
.

At Ulta Beauty, we’re forever imagining. Forever striving. Forever investing in next-gen—with deep and rich opportunities for ongoing learning, growth and advancement. We encourage you to take control of your career and set your own ceiling. For curious and big-picture thinkers who can flex to the need and who want to stretch themselves through courage and conviction, Ulta Beauty provides an environment perfectly suited to your ambitions.

Let’s build brilliant together.

The Analyst, Experimentation & Testing plays a key role in advancing our digital experimentation program by delivering high-quality measurement, analysis, and insights across web and app experiences. This role is responsible for validating and analyzing test results, and translating data into actionable recommendations that drive incremental business impact.

The ideal candidate brings strong analytical rigor, curiosity, and storytelling skills, with the ability to distill complex findings into clear narratives for cross-functional teams. This role supports the Manager of Experimentation & Testing in ensuring experimentation insights inform strategic decision-making across digital teams (UX, Digital Experience, Product PODs, Personalization, Content, etc.).

• Analyze A/B and multivariate tests using Testing tools (AB Tasty and Adobe Target) and other data sources, ensuring data accuracy, statistical reliability, and clear interpretation of results.
• Validate experiment performance, including traffic allocation, segmentation, audience overlap, etc.
• Build post-test analyses that quantify incremental impact on KPIs, customer behavior, and conversion outcomes.
• Synthesize insights into compelling stories that highlight implications, risks, and recommended next steps.
• Create test readouts, dashboards, and self-service reporting that scale learnings across teams.
• Partner with the Digital Analytics and Strategic Analytics teams to analyze test performance and uncover insights within Customer Journey Analytics (CJA).
• Assist in defining success metrics, analytical requirements, and measurement plans for experimentation initiatives.
• Ensure learnings are cataloged and shared to support a strong organizational repository and test-and-learn culture.
• Support the Manager of Experimentation & Testing in refining analytical frameworks, governance, and best practices.
• Maintain transparency and communication of testing insights across business units.

• Bachelor’s Degree in Analytics, Statistics, Data Science, Marketing, or related quantitative field.
• 1–3 years of experience in digital analytics, experimentation, or customer behavior analytics; eCommerce or retail experience preferred.
• Experience with A/B Testing Tools – Adobe Target or AB Tasty preferred.
• Familiarity with Adobe Analytics and/or Customer Journey Analytics (CJA) a plus.
• Foundational knowledge in statistical analysis and hypothesis testing.
• Ability to translate complex data into clear, concise, and actionable business insights.
• Proficiency with data visualization, dashboard creation, and developing clear analytical documentation.
• Excellent communication and collaboration skills, with the ability to support both technical and non-technical partners.
• High attention to detail, strong critical thinking, and a natural curiosity about digital customer behavior.

The pay range for this position is $66,200.00 - $82,000.00 / Year with the opportunity for eligible associates to earn additional compensation pursuant to the Company’s bonus plan. Exact…