Researching and piloting 3D-led shopping experiences
Understanding if virtual stores raise conversion of online products.
Background
lululemon was exploring the world of 3D-led shopping experiences, known as virtual stores: digital activations/spaces embedded within retailers' ecommerce sites, allowing their consumers to experience and shop for their catalogs in a more interactive or video game-like way.
In the case of the brand, they were interested in whether virtual stores could be used as a way to hype up interest in an upcoming olympic collection. To know this we decided to conduct a virtual store pilot using an upcoming footwear collection as the focus.
My job as a Lead Product Researcher was to bring clarity on how we would test the riskiest assumptions behind a virtual store, telling the story behind the data captured within store and the research studies happening around it, and once wrapped provide a recommendation on its productization to Senior Leadership.
The project’s scope also meant a lot of cross-department collaboration (Product, Brand, Ecomm, Marketing & PR, UX, 3D & Product Development) and me balancing different POVs across every decision. This also meant being responsible for its launch: overseeing smallest to the biggest day-to-day activities of this work like 3D reviews, concept development feedback to virtual store vendor, and driving approvals from Senior Leaders.
Designing the journey towards a virtual store pilot
As the org had never launched or built an experience like this, I had to design not only how we would test the virtual store but the process to get to it.
After researching virtual store startups/vendors, I wrote a research brief and met with a selection of them (Emporia, Bitreel, Obsess, and Threedium) to discuss the details of the test.
The ask to them: during a showcase, they’ll have 20 mins to present a pitch on why they should be picked to be our virtual store to internal lululemon stakeholders. Pitch must have a breakdown of their 3D capabilities, tech integration needed, and a sample of their work.
Partners and Senior Leaders invited received scorecards to rank virtual store vendors across their different capabilities. We landed on Emperia.
Designing a virtual store test: goals & scope
In partnership with teams among Brand, UX, Product, Ecomm, 3D & Product Development, we aligned on the below goals and assumptions for this pilot:
| Understand potential engagement | How might these 3D-led novel shopping experiences impact the interest of an upcoming footwear collection for our guests? |
| Understand potential conversion | How might these 3D-led novel shopping experiences influence the online conversion of an upcoming footwear collection? |
| Capture learnings for potential future experience | If feasible and viable for the business to stand up another virtual store, what can we improve for the proposed Olympic experience? |
Scope wise, the virtual store launched exclusively for US & CA in both their app and website between August 1st and September 26th 2023.
The virtual store had a total of three spaces, containing all colourways from that footwear collection plus some extra styles (e.g., leggings, sports bras). Visitors could interact with 3D-versions of those products, see their PDP pages and fully ATC and transact through the store. Other features: AR/Virtual try-on and a workout video.
The store could be found through several entry points in the app and website:
Testing assumptions: quantitative & qualitative data
To have a holistic understanding of how the virtual store was received by guests and its performance, there were multiple data points captured and studies I designed to weigh in on the final product recommendation.
| Engagement metrics within experience | Anything that’s trackable once the customer lands into the store and starts interacting with it (e.g., number of visitors and sessions, CTR across products, average engagement time, etc.) |
| ATC value and experience's influence | How’s the store best found by the customer, revenue it brought, and its influence on visitor outside store itself (e.g., clicks/visits across all promo blocks, ATC and OCR, where the visitor goes after in site/app, etc.) |
| Post-experience walkthrough surveys & intercepts | A survey of lululemon guests experience using the virtual store – from UX to brand perception – and an in-situ virtual store intercept asking visiting guests thoughts on virtual store. |
How the virtual store performed
Experience wrapped up after 8 weeks. While analyzing the performance of the store, we started to get an early understanding of what happened when guests visited it.
Most visited through nav bar on mobile by core demographic, but exploration was mostly limited to the first room
- Approx. 153K customers visited the store, with the avg. session lasting 23s and avg. session among users that performed actions lasting 3 mins 36 secs.
- Most sessions happened in the first 3 weeks, and 95% of traffic came from mobile.
- Core demographic visited the most: 25-34 (30%) and 35-44 (25%) age range.
- MegaNav entry point (homepage) was most used to enter virtual store – 20K visits.
- In order of how space was explored by visitors: Run (58.60%), Train (6.53%), Go (3.25%)
Although it influenced conversion when visiting both virtual store and PDP, investment outweighed its revenue
- A total of 1749 add-to-cart instances among all products, with footwear style being the most added-to-cart SKU.
- Total ATC % for 8 weeks was 1.1% with a value of USD164K. For reference, investment in store’s production was ~USD175K.
- In comparison, the brand's then monthly average PDP conversion was 3-4% and ATC was 12-14%.
- Brand analytics uncovered Guests who viewed both the Virtual Store and a Footwear PDP had the strongest overall conversion rate at 8.30%.
Research studies: experience's UX frictions
We launched a survey focused on capturing lululemon’s guest panel opinions and reflections on the virtual store experience.
377 lululemon guests (male and female, 18-64 yrs, US & Canada) answered questions on the brand and then they proceeded to walk through the virtual store experience. At the end, they were asked questions on the experience (what worked vs what didn’t), how it impacted their perception of lululemon across areas like product education, and whether it would impact or influence their online shopping decisions.
Several themes emerged:
Immersive but frustrated by navigation
- Majority agreed experience was immersive, but only 2-in-5 (10%) of surveyed guests enjoyed the experience.
- Although gamification and 3D features were “cool” for a few, most guests were frustrated by the navigation, loading times, and questioned the overall purpose of the experience.
- "I had a hard time navigating it. I found it frustrating."
Unique and innovative, but it negatively impacted brand perception
- Virtual store drove significant increases in perceived uniqueness of the website (45%), app (42%) and footwear category (43%).
- At the same time, the experience drove significant decreases in sentiment towards the overall brand.
- "It was a unique experience, but shopping for what I was looking for wasn't easy."
Experience didn’t deliver a cohesive brand or creative message
- 52% of engaged guests believed the experience educated them on footwear collection.
- 60% didn’t think it educated them on the overall brand, showing a disjointed brand message.
- "It’s trying to be a whole experience, but it feels honestly unnecessary."
Novel but guests didn't trust it enough to shop through it
- 73% of engaged guests did not feel the experience would influence their purchase decision.
- 69% claimed they wouldn’t be likely to shop at a lululemon Virtual Store if engaging with it on their own.
- "It's not easy to shop and I can honestly say I'd never shop this way."
We also launched an intercept study meant to gather feedback from visiting guests. Although responses were low, synthesized qual data gathered told a similar story: novel experience but UX navigation made it frustrating to shop through it.
-
Impressed by novelty and newness: a subset of visiting guests were excited about it mainly by how different and new it was, either from the brand's perspective or just them experiencing a virtual store for the first time.
"Haven't seen anything like it before."
-
Cool but doubting purpose: Visitors noted it was 'cool' but with added doubts about what its designed intention was if it didn't improve upon the current online experience.
"It's cool but I'm not sure it enhances my shopping experience."
-
Majorly frustrated by having to shop through it: Visitors described their major pain-points being the UX navigation and lack of tried-and-tested ecomm features (like filters) that simply just work.
"Experience is too busy, and the interactive aspect does not feel essential to the online shopping experience."
Conclusions & Product Recommendations
In the end, productization wasn’t recommended for the upcoming Olympic experience that year. When marrying both qual and quant captured, the experience only landed at being "unique" but not at incentivizing conversion.
I posed two scenarios to leadership about the potential use of virtual stores in the future:
✓ Potential tool for Brand or Marketing
If intention is to generate brand awareness of a new campaign, celebrate it, and educate Guests on it by using different mediums like 3D storytelling, audio, and video; then Virtual Stores could make sense if there’s also a clear comms strategy and a willing high-level of support and investment.
✗ Unfit for Digital Business
If the intention is to deliver significant conversion of a new campaign through the set up of a Virtual Store and its’ PDP features, then Virtual Stores don’t make sense for the Digital Business if traffic and revenue aren’t up to par with our ecomm channel’s current performance.
More detailed reasons for the NO/GO productization decision:
Its navigation added friction to the online shopping experience guests had grown used to. The experience flipped shopper's mental models for online shopping too drastically, forcing them to shop in a way they didn’t trust or cared for. For reference, 67% of lululemon guests engaged hadn't visited a virtual store before.
Its intention and purpose questioned signaled we didn’t nail the value prop of the store to guests. From 7 entry points across the app and website, only one made a reference to a reason (“get a virtual check”). This might also explain the low space exploration beyond the first room: lack of incentive to do so.
The conversion was similar to another retailer’s with a higher marketing spend, but it was still short of breaking even when compared to the production costs of the experience (~USD175K) and the internal resources hours invested on it.
Feasibility wise, although an external partner built the store, internal teams still had to spend a considerable amount of time supporting it. Me dividing my time as lead researcher and product owner of the experience showcased an experience like this needed someone to manage it on a more full-time basis.
Even though guests claimed experience was good at educating them on the footwear product (52%), reframing them as education-only tools didn't make sense for the brand to tap into them again.
Collaborators & Credits
The project was led while at lululemon as part of their Digital Innovation team, Future Experiences.
The project couldn’t have happened without the support of team members within Product (Stacy Bodnarek), UX (Tim Armstrong), Brand (Steven Guillen, Alexis Morton), Marketing (Chris Nuelle), Ecomm (Jenna Renfrew), Footwear (Davoud Ohadi), Tech (Miles Fishman), Voice of Guest (Lisvet Parra), Analytics (Anya Andronov) and Leadership buy-in from Danny Ryder (Chief Digital Officer), Nick Engel (SVP Digital), Sarah Rogowsky (former VP Digital Strategy & UX) and Celeste Burgoyne (former President, Americas & Global Guest Innovation).
Within my team, Future Experiences, I was supported by Heather Kaiserani (Project Manager) and Ellisa Calder (Director, Digital Insights & Future Experiences).
Thank you to the Emperia team for supporting us with the launch and build of this experience.