Recap on Biometrics in User Experience Research

by Andrew Schall

In a recent collaboration with User Experience Magazine I published an article entitled: The Future of UX Research: Uncovering the True Emotions of Our Users. The motivation for writing this article was to expose the UX community to the possibilities of using biometric and neurometric measurements to understand the emotions of our users. Current methods for understanding a user’s emotional response are at best limited, and at worst, entirely inaccurate. As the field of user experience evolves, we need to explore new methods for measuring emotional responses using technologies borrowed and refined from neuroscience and human biology. I also wanted to highlight some of the latest technologies available to user researchers, as well as the challenges of working with these tools.
I was thrilled to receive so many positive comments about the article as well as many questions about using these tools. This recap provides some of the key aspects of using biometrics for user experience research as well as answers to the most frequently asked questions about the topic.

Here are some do’s and don’ts for using biometrics in your user experience research projects:
Do:

  • Hire specialists on your team with a neuroscience, cognition, or experimental psychology background (preferably a Ph.D.). They will be most helpful during the study design and also during the analysis of your biometric data.
  • Run lots of pilot tests. Incorporating biometrics can add unforeseen challenges to your study and requires extra practice to ensure that data is collected accurately with actual participants.
  • Ensure that you have a sufficient sample size. Biometric studies require more participants than a typical usability test to account for the potentially large variability between participants. You will also encounter more situations where data needs to be excluded because due to improper equipment calibration, equipment failures, etc.

Don’t:

  • Examine biometric data in a vacuum. It is important to triangulate data across a combination of different methods including nonverbal observations, participant retrospectives, surveys & rating scales.
  • Assume that your data is perfect. Measuring biological and neurological responses, especially with the tools accessible to UX researchers, is not going to be 100% accurate. I would recommend looking at general trends in valence (i.e. positive/negative spectrum of emotions) changes instead of trying to pinpoint exact moments where very specific emotions were felt.

Questions & Answers
There have been a variety of tweets and posts generated as a result of publishing this article. Here are some of the most common questions with my thoughts:

Do participants mind wearing all of that equipment? Does anyone complain about it being uncomfortable?
Before making any purchases, I heavily researched all of the existing hardware currently available to make sure that it was as minimally intrusive to participants as possible. In many cases we are only using one or two measurements at a time. I try to use only one measurement that requires physical contact with the participant, such as wearing a GSR wristband combined with facial analysis that only requires a webcam. At the end of a session I always ask the participant whether wearing any of the gear was uncomfortable for them or if it impacted their experience at all. Most said that once the session began they quickly forgot that they were wearing anything.

Is it [biometrics] worth it? Wouldn’t it just be easier to interview an individual or give them a survey to complete?
We are at the very beginning of an exciting journey to uncover more about how our users are actually feeling. Biometrics is definitely not for everyone, and it is not a useful endeavor for all situations. I actually don’t think that these tools are ready for mainstream UX researchers just yet. The tools and software continue to get better every day, and will likely become more useful to UX researchers within the next few years. However, for those who are very serious about obtaining objective, quantitative measurements about your user’s emotional journey, I would recommend trying out these tools to see if they are useful for your team.

I’m very interested in learning more about biometrics! How did you learn about this topic? Would you recommend any resources?
There really isn’t one place to find out everything you would want to know. Biometrics is still a very new topic for the UX community and most of the resources out there are geared for people in the fields of human biology and neuroscience. I started reading a lot of academic papers to understand how to collect and analyze the data. There are a few worthwhile articles in the ACM Digital Library that provide case studies where biometrics were used in the context of human-computer interaction. I would also recommend looking into each biometric and neurometric measurement (e.g. EEG, GSR, etc.) individually. You can get a basic sense of how these tools work by visiting the websites of the device hardware and software manufacturers. My article provides links to many of the major vendors who make UX researcher-friendly products.

PRESS RELEASE: Norwegian Cruise Line Hosts the Best In Class Web Experience; Leaves Competition in Their Wake

Key Lime Interactive Ranks Norwegian Cruise Line’s Website Best-In-Class in Comprehensive User Experience Report.  

Miami, FL – Key Lime Interactive (KLI), an industry leader in competitive index reporting across a variety of industries, released the results of their Contemporary and Premium Brands 2014 Cruise Competitive Benchmark Study at the close the third quarter ranking Norwegian Cruise Line as Best-In-Class. The report compares the website for seven cruise brands including Carnival Cruise Lines, Holland America, Disney Cruise Line, Princess Cruises, Celebrity Cruises, Royal Caribbean Cruise Ltd. and Norwegian Cruise Line. The report reveals that Norwegian Cruise Lines offered the best “overall site satisfaction” over a variety of indices such as brand affinity. The Key Lime Interactive Competitive Benchmark study defines these indices as “behavioral loyalty.”

“Consistently positive emotional experiences result in consumer loyalty,” reported Ania Rodriguez, Founder and CEO of Key Lime Interactive. “Norwegian’s site visitors indicated that they had a high likelihood to return to the site and book a cruise in the future. Most importantly, the site significantly outperformed its competitors’ in terms of time spent on the site. Time spent increased the brand’s perception relating to attributes such as ‘entertaining’, ‘appealing’, ‘fun’, and ‘family-oriented’.”

These emotional experiences and demonstrated behavior were measured using a task-based user research approach in which hundreds of users participate in a study passively, in their natural context, from geographically widespread locations. A mix of experienced and inexperienced cruisers were recruited to visit a cruise website and perform a series of tasks. Before, during and after task completion, several lines of questioning were asked so that Key Lime Interactive’s team of researchers could evaluate a series of drivers and determine how each site performs. These detailed results help design and digital marketing professionals at each brand better understand the online customer journey. Digital marketing and product development teams count on these findings to meet the needs and expectations of their target audience in strategic planning sessions.

Norwegian significantly outperformed the other players in areas such as “Offering Satisfaction” and “Value Proposition.” Additional analysis across six other drivers is featured in the report.

Norwegian stakeholders responded favorably. “…Not only are we prioritizing what users claim they need, but we get a first-hand view of what users are doing on the site. In our experience, what our guests say, and how they actually behave online doesn’t always align. This methodology provides the details we need to in order to focus our development plans in the coming year,” says Rob Casas, Vice President of eCommerce at Norwegian Cruise Line.

Casas and others rely on Key Lime Interactive reporting to drive the development roadmap into the next quarter and beyond. In many cases, deficits or strengths easily translate to quick fixes or strategic change to improve prospective buyer experience.

Additional Report Information

Key Lime Interactive’s Cruise Experience Mobile Heuristics were used to determine the overall score and ranking of seven (7) companies including Carnival Cruise Lines, Holland America, Disney Cruise Line, Princess Cruises, Celebrity Cruises, Royal Caribbean Cruise Ltd. and Norwegian Cruise Lines. These heuristics consisted of a capabilities assessment, comparative feature ranking and overall user score and ranking.

The methodology first aimed to understand by using consumers’ self-reported preferences; which features and tasks were must-haves when it came to interacting with the site. These identified features were then placed into priority order to highlight their importance. This ranking was used in an algorithm to tabulate the final scores.

Key Lime Interactive recognizes the importance of aligning business-centric goals with consumer-centric needs to produce winning solutions and makes this a core component to all deliverables. This incorporation of user feedback is a primary component for this and all studies across various industries ranging from finance, retail, hospitality, etc.

“Other reports highlight what we’re doing well from an expert user experience researcher perspective. This report is different because it tells us what we’re doing well from the user perspective. When adjusting the opportunities for improvements listed we can see the direct impact on overall site success.” says Casas as he anticipates Norwegian’s scores in the next report release.

All assessments are as of August 22, 2014. The next iteration of the study will be available in the summer of 2015.

To purchase the published report, please reach out to Key Lime Interactive for more information at sales(at)keylimeinteractive(dot)com

Researchers, Designers, Executives, Journalists and Bloggers in the Cruise, Travel and Leisure Industry are encouraged to contact Key Lime Interactive with any additional questions.

About Key Lime Interactive

Key Lime Interactive is an user experience research firm with proven excellence in both quantitative and qualitative user and consumer testing. To serve our growing client list of Fortune 100 companies, we conduct competitive research, true intent / voice of customer studies, and prototype studies using quantitative methods. Additionally, we’re experienced in moderating one-on-one interviews/ usability studies, focus groups, and eye-tracking studies for both the traditional and mobile web. Currently, Key Lime is working to help top brands better understand their customer via behavioral personas and customer journey mapping. Ultimately, our goal is to empower teams to use consumer/user experience data at any and all phases of product development; from strategy to implementation. We aim to provide the true perspective of target users and build exceptional consumer-driven solutions.

PRESS RELEASE: Norwegian Cruise Line Hosts the Best In Class Web Experience; Leaves Competition in Their Wake

Key Lime Interactive Ranks Norwegian Cruise Line’s Website Best-In-Class in Comprehensive User Experience Report.  

Miami, FL – Key Lime Interactive (KLI), an industry leader in competitive index reporting across a variety of industries, released the results of their Contemporary and Premium Brands 2014 Cruise Competitive Benchmark Study at the close the third quarter ranking Norwegian Cruise Line as Best-In-Class. The report compares the website for seven cruise brands including Carnival Cruise Lines, Holland America, Disney Cruise Line, Princess Cruises, Celebrity Cruises, Royal Caribbean Cruise Ltd. and Norwegian Cruise Line. The report reveals that Norwegian Cruise Lines offered the best “overall site satisfaction” over a variety of indices such as brand affinity. The Key Lime Interactive Competitive Benchmark study defines these indices as “behavioral loyalty.”

“Consistently positive emotional experiences result in consumer loyalty,” reported Ania Rodriguez, Founder and CEO of Key Lime Interactive. “Norwegian’s site visitors indicated that they had a high likelihood to return to the site and book a cruise in the future. Most importantly, the site significantly outperformed its competitors’ in terms of time spent on the site. Time spent increased the brand’s perception relating to attributes such as ‘entertaining’, ‘appealing’, ‘fun’, and ‘family-oriented’.”

These emotional experiences and demonstrated behavior were measured using a task-based user research approach in which hundreds of users participate in a study passively, in their natural context, from geographically widespread locations. A mix of experienced and inexperienced cruisers were recruited to visit a cruise website and perform a series of tasks. Before, during and after task completion, several lines of questioning were asked so that Key Lime Interactive’s team of researchers could evaluate a series of drivers and determine how each site performs. These detailed results help design and digital marketing professionals at each brand better understand the online customer journey. Digital marketing and product development teams count on these findings to meet the needs and expectations of their target audience in strategic planning sessions.

Norwegian significantly outperformed the other players in areas such as “Offering Satisfaction” and “Value Proposition.” Additional analysis across six other drivers is featured in the report.

Norwegian stakeholders responded favorably. “…Not only are we prioritizing what users claim they need, but we get a first-hand view of what users are doing on the site. In our experience, what our guests say, and how they actually behave online doesn’t always align. This methodology provides the details we need to in order to focus our development plans in the coming year,” says Rob Casas, Vice President of eCommerce at Norwegian Cruise Line.

Casas and others rely on Key Lime Interactive reporting to drive the development roadmap into the next quarter and beyond. In many cases, deficits or strengths easily translate to quick fixes or strategic change to improve prospective buyer experience.

Additional Report Information

Key Lime Interactive’s Cruise Experience Mobile Heuristics were used to determine the overall score and ranking of seven (7) companies including Carnival Cruise Lines, Holland America, Disney Cruise Line, Princess Cruises, Celebrity Cruises, Royal Caribbean Cruise Ltd. and Norwegian Cruise Lines. These heuristics consisted of a capabilities assessment, comparative feature ranking and overall user score and ranking.

The methodology first aimed to understand by using consumers’ self-reported preferences; which features and tasks were must-haves when it came to interacting with the site. These identified features were then placed into priority order to highlight their importance. This ranking was used in an algorithm to tabulate the final scores.

Key Lime Interactive recognizes the importance of aligning business-centric goals with consumer-centric needs to produce winning solutions and makes this a core component to all deliverables. This incorporation of user feedback is a primary component for this and all studies across various industries ranging from finance, retail, hospitality, etc.

“Other reports highlight what we’re doing well from an expert user experience researcher perspective. This report is different because it tells us what we’re doing well from the user perspective. When adjusting the opportunities for improvements listed we can see the direct impact on overall site success.” says Casas as he anticipates Norwegian’s scores in the next report release.

All assessments are as of August 22, 2014. The next iteration of the study will be available in the summer of 2015.

To purchase the published report, please reach out to Key Lime Interactive for more information at sales(at)keylimeinteractive(dot)com

Researchers, Designers, Executives, Journalists and Bloggers in the Cruise, Travel and Leisure Industry are encouraged to contact Key Lime Interactive with any additional questions.

About Key Lime Interactive

Key Lime Interactive is an user experience research firm with proven excellence in both quantitative and qualitative user and consumer testing. To serve our growing client list of Fortune 100 companies, we conduct competitive research, true intent / voice of customer studies, and prototype studies using quantitative methods. Additionally, we’re experienced in moderating one-on-one interviews/ usability studies, focus groups, and eye-tracking studies for both the traditional and mobile web. Currently, Key Lime is working to help top brands better understand their customer via behavioral personas and customer journey mapping. Ultimately, our goal is to empower teams to use consumer/user experience data at any and all phases of product development; from strategy to implementation. We aim to provide the true perspective of target users and build exceptional consumer-driven solutions.

Do You Love Your Job?

by Rick Damaso

Love your job? Chances are if you work in UX, you do.
My recent trip to UXPA 2014 offered me more than the opportunity try authentic Fish & Chips. It gave me great insights to the UX industry and as it turns out, great perspective on the people that make up the industry. Perhaps the most interesting aspect was the consistently, inconsistent career & educational backgrounds of UX professionals. After nearly a dozen conversations that described incredibly diverse paths to their current role, I decided this warranted a closer look. After some digging it turns out that not only are UX professionals incredibly diverse in their educational backgrounds, but their career satisfaction ratings were noticeably high¹. In order to understand why our colleagues were rating their job satisfaction so highly, I decided to first look at the characteristics of successful researchers. I wondered how current growth within the industry and optimism on future developments within their profession shaped their feelings towards their job.
In order to answer these questions I first looked at the qualities that make up professional UX research associate.  I asked participants at UXPA2014, employees within Key Lime Interactive and colleagues across the industry to chip in. I received some pretty interesting opinions. Most success stories were based on educational backgrounds. Some of the responses described researchers as having “solid computer programming skills, a background in Anthropology or Psychology and some exposure to Human Factors training.”
To test the validity of these statements, and in true UX fashion, I conducted some ethnographic studies on some unsuspecting colleagues and UXPA2014 delegates. Knowing their education background and comparing it to the “suggested educational paths” would allow me to determine if it was their training or other inherent qualities that make great researchers. It turns out that only about half of the participants I observed were trained in the previously mentioned disciplines. So what was the common thread between the greater population? It seems that at the heart of almost all UX professionals lies a naturally inquisitive spirit, the willingness to pursue a career of lifelong learning and excellent communication skills.
Armed with this information I wondered how such a diverse group of men and women would describe their job satisfaction. Would they express optimism towards the future of their industry? According to a study conducted by the Nielsen Norman Group, UX professionals rank their job satisfaction a 5.4² on a Likert scale. They also ranked pay/benefits satisfaction as a 5.2³. But does earnings truly correlate to higher job satisfaction? Harvard Business Review author Tomas C. Premuzic argues that “there is less than a 2% overlap between individuals pay and pay satisfaction”⁴.  This leads us to infer that job satisfaction and satisfaction of pay are independent of each other. So what other factors could be driving the overall satisfaction from such a diverse pool of workers? Nielsen researchers noted that after polling over 1,000 UX professionals, the core activities of a UX researcher included: presenting solutions/concepts, persuading others and critically analyzing tasks or activities, which strongly correlates to the qualities observed during my impromptu ethnographic study.
As a whole, the UX industry finds itself in a period of tremendous growth. With more than half of all UX professionals living in the United States⁵, other markets will soon look to qualified professionals for expertise and guidance. So ask yourself this, do you feel well-rewarded and highly valued? Do you see your work as being intrinsically good for humanity? Do you enjoy being engaged and being able to use many of your skills on a daily basis? Then smile, you are probably a UX professional.
1. Nielsen, Jakob, and Susan Farrell. “User Experience Careers.” User Experience Careers. 2013. Web.
2. Nielsen, Jakob, and Susan Farrell. “User Experience Careers.” User Experience Careers. 2013. Web.
3. Nielsen, Jakob, and Susan Farrell. “User Experience Careers.” User Experience Careers. 2013. Web.
4. Chamorro-Premuzic, Tomas. “Does Money Really Affect Motivation? A Review of the Research.” Does Money Really Affect Motivation. Harvard Business Review, 10 Apr. 2013. Web. 28 Aug. 2014.
5. Nielsen, Jakob, and Susan Farrell. “User Experience Careers.” User Experience Careers. 2013. Web.

Do Alignment Diagrams Really Work?

by Kelly Nercess
Ever think about the interactions you have with a company? How each precise moment may seem tailored to your needs and catered towards the experience you are looking for? In the UX industry there is a technique used to ensure this end result and it’s called alignment diagrams. I had the opportunity to read a very interesting article on the idea of alignment diagrams and how they serve a perpetual purpose among the growth of companies. , written by James Kalbach & Pul Kahn, shared their discoveries on the business process and provided advice to take full advantage of each customer touch point.
There are three major design avenues that play a component in the user/customer process, those being: Information Architecture (IA), User Experience Design (UXD), and Service Design (SD). Along with these visual techniques to map out the strategic thinking to guarantee customer satisfaction, the idea of alignment diagrams allows a business to visually map out the touch points they have with a customer throughout the business process. Alignment diagrams are created to allow for a structure to reveal the customer touch points and give constructive insights to where the design and business process needs to be altered in order to optimize the experience.
There are two very important components to a successful alignment diagram, the customer behavior and the business process. Common visualizations that are used in the UX industry include: site maps, overview diagrams, workflows, customer journeys and service blueprints. These visualizations are gathered from a number of data collection methods, including: ethnographic observation, user interviews and card sorting. All of these methods play a unique role in gathering insights towards a positive user experience.
“The business focuses on maximizing a target market’s profits and potential for growth. Customer-centered design focuses on maximizing the value of a product or service to the customer” writes Kalbach & Kahn. The idea is to be able to pinpoint the location in the business process where there is an overlap between the consumer and the company. This maximizes the value for both parties and allows for a hyper focus on that exact moment of intersection. Decreasing the noise between the customer interactions and keeping the experience seamless proves to not only be beneficial towards the customer but also increase value for the business. Once the business understands customer perception and loyalty they can begin to align their business goals. Kalbach & Kahn also note, “Mapping the steps in a production and consumption process is the best way to see opportunities for improvement.”
Several other design techniques come into play when mapping out the steps of the business process. Using a service blueprint to help structure the customer journey is the best way for a designer to visually see where the issues occur. Furthermore, mental model diagrams allow you to visualize where your business strategy aligns with the current user experience. Journey maps play a large role in the experience design, allowing to fully map out the beginning to end. This will include the first time a potential customer researches the company to the point where they decide to make the purchase. All these interactions reiterate the idea of ‘touch points’, providing an opportunity to reconstruct that interaction to a more positive one. The journey map defines the customer perception of the company and illustrates the amount of time invested in to their ‘relationship’ with a business. Each touch point may be different for each customer. Someone who tends to spend more time researching the internet prior to a purchase will need a more personalized experience tailored to their specific needs. Someone who skips the research portion and spends an extensive amount of time interacting directly with the company versus purely researching it.
By using these techniques, specifically alignment diagrams, benefits both sides of the transaction – the customer and the business. The PJIM article’s closing statement says, “These visualizations reveal value that can not only aid in the creation of products and services, but also improve changes in the business strategy as well.” The important takeaway is to consider all aspects of the relationship between a business and customer. By leveraging these diagrams, you can discover opportunities that increase your customer loyalty and benefit your business strategy.