Home

Home

Home

Scent-iment Shopping

An AI-enabled, multi-sensory retail experience that personalizes perfume shopping through voice interaction.

Scent-iment Shopping

An AI-enabled, multi-sensory retail experience that personalizes perfume shopping through voice interaction.

Interactive Environments

Experience Design

AI

Retail Experience

Interactive Environments

Experience Design

AI

Retail Experience

Role

Media Art, Experience Design, Backend

Tools

P5.js, FigJam, Google Vertex API, Woodwork, Projectors, Mics

Timeline

Aug - Dec 2022 (4 Months)

Methods

Opportunity Statements Concept Design - Experience, Physical, Tech ,Demo - Testing

Team

Tynan Purdy, Upasana B., Freja Zhang

Role

Media Art, Experience Design, Backend

Methods

Opportunity Statements Concept Design - Experience, Physical, Tech ,Demo - Testing

Tools

P5.js, FigJam, Google Vertex API, Woodwork, Projectors, Mics

Team

Tynan Purdy, Upasana B., Freja Zhang

Aug - Dec 2022 (4 Months)

Timeline

Problem

Retailers face a retention crisis; customers have endless digital options and fewer reasons to visit physical stores. The challenge was to create an immersive, value-added experience that offers something online shopping cannot: a multi-sensory journey that translates abstract feelings into a tangible product. We explore:

How might we reimagine the in-store retail experience to deeply engage younger consumers who are accustomed to digital personalization?

Opportunity

We are at an inflection point for the physical and digital integrations in retail stores at hand. Newer technologies along XR, haptics and targeted visual and audio technologies can allow us to explore new avenues for shopper participation.

What might enhancements look like for loyalty members that we could unlock across the store using digital tools and technology?

We looked at current and future retails trends and synthesized our learnings to reframe an opportunity statement

Brainstorming

We analyzed current retail trends and identified three core themes:

  1. Innovation Fatigue: Users want "magic," not just more screens.

  2. Hyper-Personalization: Users expect services that adapt to them instantly.

  3. Inclusivity: The experience must be intuitive and accessible, not complex or technical.

The Pivot: Initially, we explored concepts around shoe assembly, but user testing revealed that olfactory (scent) experiences offered a stronger emotional connection and a wider "gap" in the current market that needed filling.

Brainstorming

We analyzed current retail trends and identified three core themes:

  1. Innovation Fatigue: Users want "magic," not just more screens.

  2. Hyper-Personalization: Users expect services that adapt to them instantly.

  3. Inclusivity: The experience must be intuitive and accessible, not complex or technical.

The Pivot: Initially, we explored concepts around shoe assembly, but user testing revealed that olfactory (scent) experiences offered a stronger emotional connection and a wider "gap" in the current market that needed filling.

Opportunity Statement

As part of our efforts to acquire new customers, we, Perfume Co, will create an interactive browsing experience at retail stores that educates customers who are new to perfumes to discover their tastes and preferences.

We experimented with future retail shopping concepts and present olfactory shopping experiences to see room for wild ideas about our perfume shopping booth, considering the diversity in smells and the emotions they evoke for each individual.

Opportunity Statement

As part of our efforts to acquire new customers, we, Perfume Co, will create an interactive browsing experience at retail stores that educates customers who are new to perfumes to discover their tastes and preferences.

We experimented with future retail shopping concepts and present olfactory shopping experiences to see room for wild ideas about our perfume shopping booth, considering the diversity in smells and the emotions they evoke for each individual.

Research

Phase 1: Looking for the "Why"

Our research methodology followed a "Diverge-Converge" process. We started by exploring the macro-trends of retail, narrowed down to the psychology of smell, and iterated on the interaction design based constraint each modality offered.

We conducted a literature review and market analysis to understand the context of our intervention.

  • The State of Retail: Analyzing reports on "Future Retail Concepts" (Forbes, Shopify), we identified that physical store/brick-and-mortar locations could shift from transactional spaces to experiential community hubs.

  • The Physiology of Scent: Olfactory inputs trigger immediate, involuntary emotional responses.

Research

Phase 1: Looking for the "Why"

Our research methodology followed a "Diverge-Converge" process. We started by exploring the macro-trends of retail, narrowed down to the psychology of smell, and iterated on the interaction design based constraint each modality offered.

We conducted a literature review and market analysis to understand the context of our intervention.

  • The State of Retail: Analyzing reports on "Future Retail Concepts" (Forbes, Shopify), we identified that physical store/brick-and-mortar locations could shift from transactional spaces to experiential community hubs.

  • The Physiology of Scent: Olfactory inputs trigger immediate, involuntary emotional responses.

We initially started with planning to use facial recognition but pivoted to sentiment analysis of voice.


Phase 2: How do we read consumer emotions: Face vs. Voice?

Our initial concept relied on Facial Recognition technology. The hypothesis was that we could read a user’s micro-expressions (disgust, delight, surprise) to automatically determine if they liked a scent.

Challenge: Applying facial recognition in a retail context presented three hurdles:

  1. Privacy & Ethics: Scanning faces without explicit, enthusiastic consent felt invasive.

  2. Ambiguity: In a dimly lit, immersive booth, computer vision struggles to differentiate subtle emotions.

Pivoting to Voice Sentiment Analysis: We shifted the input modality from passive observation (Face) to active participation (Voice).

Implications to Experience: It gave the user agency. Instead of the camera guessing how they felt, the user told us.

Design Thought

We synthesized our research findings into a set of design heuristics that guided the physical build:

  1. Retail requires “Magic”.

The “Peep-Hole” Interface -> We avoided standard screens. We built a physical tube structure that required users to lean in, creating a private, magical window into the digital world.

  1. Scent is abstract and hard to describe.

Explore Generative Visualization -> We used P5.js to translate abstract scent data into concrete visual metaphors (e.g., Woody Scents = Branching fractals; Floral Scents = Blooming particles)

  1. Users need immediate system feedback.

Real-Time Responsiveness -> To build trust in the AI, the visual feedback had to be instant. As the user speaks, the visual changes immediately, confirming the system is “listening.”

Interaction Design

We synthesized our research findings into a set of design heuristics that guided the physical build:

  1. Imagining the Space

Designing a booth for interaction rooted in the culture, emotions, and objects around smell and perfume.

A moodboard for contents, elements and objects we thought complemented the experience.

  1. Mapping the Sensory Experience

Relating to use of senses to crafting the exeperience further.

  1. Identifying the Key Risks and Challenges at this stage

We acknowledge that olfactory was a hard to control as modality.

  1. Mapping a User Journey

Sketching a user journey across the experience

  1. Mapping the spatial aspect of the journey

Realising the physical aspects of the journey into physical computing elements

Tech Planning

We built a custom interaction engine using open-source tools for rapid iteration:


  • Voice Input

    We utilized the speech.p5 library to handle speech-to-text conversion, allowing users to naturally describe their sensory experience.


  • Sentiment Analysis:

    The text data was fed into the Google Vertex API. This backend analyzed the emotional weight of the user's feedback in real-time.


  • Generative Visuals:

    The sentiment score directly controlled the visual output. Positive feedback caused the digital flower to "bloom," while negative feedback caused it to "de-bloom," providing instant system feedback

Tech Planning

We built a custom interaction engine using open-source tools for rapid iteration:


  • Voice Input

    We utilized the speech.p5 library to handle speech-to-text conversion, allowing users to naturally describe their sensory experience.


  • Sentiment Analysis:

    The text data was fed into the Google Vertex API. This backend analyzed the emotional weight of the user's feedback in real-time.


  • Generative Visuals:

    The sentiment score directly controlled the visual output. Positive feedback caused the digital flower to "bloom," while negative feedback caused it to "de-bloom," providing instant system feedback

Dissecting the Tech behind the Prototype Design first before making the protoype

Prototyping Scent-iment Shopping

The final prototype integrated physical immersion with real-time digital feedback to create a "listening" retail experience. We constructed an abstract, tubular structure featuring a "peep-hole" window that compelled users to lean in, creating a private moment for interaction. Using the speech.p5 library for voice input and Google Vertex API for sentiment analysis, the system interpreted user verbal feedback instantly; positive words caused a projected digital flower to "bloom" while negative feedback made it "de-bloom." This emotional data was then processed through semantic similarity matching against a perfume dataset to generate a personalized scent recommendation for the user to wear.

Prototyping Scent-iment Shopping

The final prototype integrated physical immersion with real-time digital feedback to create a "listening" retail experience. We constructed an abstract, tubular structure featuring a "peep-hole" window that compelled users to lean in, creating a private moment for interaction. Using the speech.p5 library for voice input and Google Vertex API for sentiment analysis, the system interpreted user verbal feedback instantly; positive words caused a projected digital flower to "bloom" while negative feedback made it "de-bloom." This emotional data was then processed through semantic similarity matching against a perfume dataset to generate a personalized scent recommendation for the user to wear.

  1. Rendering Visual and Spatial Aspects

We created 3D spatial renders to imagine the retail space first before building a prototype

Explosion Diagram for the physcial booth prior to being built

  1. Building Scent-iment Shopping

  1. Showcasing the Prototype

Visitors trying out the experience

Frontal View

Reflections

This was a pivot from my previous screen-based design to Interactive Environments. I learned that designing for interactive environments requires a strategy that accounts for physical, user time, and sensory input. The project stuck out in planning a strategy that extended beyond the tech stack to include the emotional journey of the user in physical space.

Working with a multidisciplinary team highlighted the importance of shared vocabulary between design and engineering and that I had so much more to learn.

Reflections

This was a pivot from my previous screen-based design to Interactive Environments. I learned that designing for interactive environments requires a strategy that accounts for physical, user time, and sensory input. The project stuck out in planning a strategy that extended beyond the tech stack to include the emotional journey of the user in physical space.

Working with a multidisciplinary team highlighted the importance of shared vocabulary between design and engineering and that I had so much more to learn.

Funding

Materials funded by Cognizant as part of Course: Designing Interactive Environments