New Director of Quantitative Research Leads Competitive Initiatives

KLI couldn’t be happier to welcome Dana Bishop to our team as our new Director of Quantitative Research. Dana has been working in the field of user research for 20 years and flaunts extensive experience with a variety of research methods. Above all, Dana has perfected the art and science of creating simple, yet highly-informative large-scale online user experience research studies. Her graceful orchestration of traditional scaled questions and directed tasks for users results in detailed feedback, thoughtful analysis and poignant evidence that informs design for clients far and wide.
Prior to joining Key Lime Interactive, Dana was lead researcher and manager of Keynote Systems’ Competitive Research group. While at Keynote, Dana led longitudinal quantitative research studies across numerous verticals and global markets for companies such as Carnival, Expedia, Travelocity, Wells Fargo, U.S. Bank, Yahoo!, and State Farm Insurance. Dana began her career in the 1990s in San Francisco where she spent 3 years at Charles Schwab & Co conducting a nationwide field study and weekly in-lab sessions with customers; as well as time spent running usability testing for edu-tainment software in school environments.
After just three short months as part of the KLI team, Dana’s expertise is in high-demand! Custom studies are exceeding client expectations and all the while, Dana and other Key Limers are preparing the following types of reports for incremental release:
KLI Competitive Research. Naturally, with the addition of Dana to the Key Lime team, we’ve both expanded and refined our competitive research. Dana is spearheading several existing and new reports that fall under the following categories:
Competitive Index:
Currently our Auto Insurance Competitive Index and our Mobile Banking Competitive Index are widely used by nearly all top players in their respective industries. For this research KLI runs a survey to deeply understand the perceptions, beliefs, needs and desires of users when using their mobile devices (both web and apps) in context of a given industry and then indexes and compares capabilities across major players; ultimately ranking them and revealing strengths and opportunities for the industry and individual companies to move ahead. Inquire about the purchase of either of these reports, or suggest an index for your industry…
Competitive Benchmark Studies
Additionally, KLI publishes annual Cruise Competitive Benchmark results in June each year. This is a task-based assessment of the leading cruise industry websites by users (a mix of first-time and experienced users). The study analyzes the user experience in trying to learn about the cruise line, find a cruise of interest, and book online. It measures the user experience in terms of satisfaction, site reliability and performance, as well as NPS and likelihood to return and purchase. Dana’s keen understanding of what the cruise industry needs and pays attention to when executing sound design changes is part of what makes this benchmark study novel and desired. The study capitalizes on the value proposition offered by the various brands: Are they selling the ratio of cost to experience well to their digital consumers? Are they painting a clear picture that informs decisions and promotes action? At present, leaders in the industry are working with Dana to refine the June release to include exactly what they’ve been missing. Want to be involved in that conversation? Have ideas for a similar study in a different vertical? Learn more…
Custom Competitive Benchmark Studies
To take this one-step farther and truly meet the demands of KLI clients, Dana is leading the development of Custom Competitive Benchmark studies for several clients in the retail, travel, medical and financial industries. These studies are quite similar to the general Competitive Benchmark studies in that they are also task-based assessments of sites within a given industry by users. They also focus on which site(s) are providing the best user experience; but differ in that they allow companies to custom design aspects of the study along with KLI researchers. Companies can “ customize” by selecting the competitors they are most interested in benchmarking themselves against, as well as having input about the tasks users complete, and timing of when the study fields. Need to benchmark yourself against competitors in your industry? Learn more…

Consumer Testing in a Mobile World

Shopping, for me, has taken on a whole new form.
I’ve been browsing bridal registries at Pier 1 while waiting in the dentist office and purchasing new running shoes from Zappos while on the train.  Lately, I’ve even taken the opportunity to update my home grocery delivery list on my mobile phone while cooking dinner!  It’s been fantastic – more efficient shopping decisions supported by in-app reviews, free shipping offers and most importantly less time at the store…which translates to more time for other things.

Our clients are noticing this, too, because we’re seeing a drastic rise in the demand to put apps in front of prospective consumers before they launch.  Our retail clients want to exploit the interplay between store, online, and mobile channels in consumer outreach and they want to do it well.   With the holidays only being a short 5 months away and the holiday shopping season ramping up in a mere 2, retailers are expanding their mobile budgets and adjusting their second half of the year marketing plans to ensure they can meet the needs of users like me who would just rather not head to the store.

What kind of mobile testing is happening?
Feature priorities are broad and users’ expectations are demanding.  The trends in mobile are changing rapidly. Clients cannot simply take their traditional site and miniaturize it to fit on the screen of your smart phone or they’ll lag behind their competitors.

Let’s consider everything that a consumer can do from the palm of their hand these days:  text-to-list to add oneself to marketing lists, access location based deals, get feedback from their social networks about purchasing decisions, accurately compare pricing between e-retailers and local retail stores and this list goes on and on.  Consumers seek instant gratification and retailers have new avenues for branding opportunities to meet that instant search protocol that is becoming standard.

Our clients get it and they have expressed the need to get their offering right the first time.  (…before the app store makes them go through a lengthy resubmission process.)

Although there are the occasions where a template app can be used to showcase the clients’ offering, we’re finding that this is increasingly rare in the retail segment.   Clients are customizing and they’re considering their audience as a critical component to developing a successful solution.

What are we doing to help?
KLI specializes in primary research – we work to evaluate user behavior, blend our observations and user feedback with our expertise and deliver actionable recommendations to make things intuitive to the target audience.  So, as mobile expands, we need to continue to be forward-thinking as we gather data.

In an effort to help our clients evaluate user behavior across form factors and operating systems as they develop this critical mobile presence, we’ve found a few tools that help us move from the lab and into the wild.

  • The Spy Cam. In a nutshell, this is a pair of glasses with a small video camera mounted in the center between the two eyes – they look and feel like a geeky pair of sunglasses.
    When we’re using it:
    Several clients have the need to understand the interplay between their mobile shopping app and the physical store shelf.  In scenarios where the app offers reviews, or additional product information or helps comparative shop, users aren’t neglecting this tool.  So, we’ve used “the spy cam” to monitor this behavior without having to stand two feet away from them as they navigate a store aisle.  They’re more comfortable and we get more information to make informed generalizations about their behavior.
  • Mirrored Screen.  Depending on the smart phone, there is an HDMI out cable that can be used to mirror the screen on a PC.   We use this, combined with screen recording software to capture the navigation path of the user.
    When we’re using it:
    Sometimes the setting is such that we don’t want to observe first hand what’s going on because we may interfere with the users’ most natural behavior.  In this scenario we send the mirrored screen to a more comfortable viewing location so that we can see the navigation path as the user executes a task, explores freely, etc.   In many cases, we stream this mirrored screen to the client in tandem with our notes so that they can check-in freely as they wish to observe the data collection process.  Bringing our clients on board during data collection makes them feel like they’re more involved and increases the integrity of our reporting.
  • Diary Studies. We’ve setup easy to access blogs like tumblr, etc. for users to send screenshots, audio clips, quotes, thoughts, and more as they log their usage behavior.
    When we’re using it:
    In some scenarios we need data testing over time.  Take the airline industry study we did for example… from booking, to checking in, to changing seats, to boarding, the app is accessed at various time points.  It’s an effective delivery method for us to encourage users to log their experience at various time points so the entire service can be properly evaluated.

We’re also using mobile eyetracking solutions, split screen monitors to synchronize concurrent activities, in-lab mobile sleds and camera systems, on-screen simulators, and more depending on the research questions on hand.

To learn more about the presentation from the Big (D)esign 2011 conference in Dallas reach out to us at

Engaging your Audience: Unlocking and Optimizing Multi-Screen Opportunities

We can’t imagine we’re alone as we notice a dramatic change in the needs of our customers.  It seems that rather than independently testing how users consume the lean-back broadcast experience and the lean-forward web and mobile experience as we did in the past, top brands are seeking our expertise to understand how these two mediums collide and complement one another.
With Nielsen reporting that 60 percent of people surf the web while watching TV, we certainly can’t blame them.  Consumers are hungry for a complete “infotainment” experience and are consuming media on multiple screens that target specific content, device or individual demographics.  Most importantly, though, these people still indicate a refusal to abandon traditional broadcasting altogether.  This leaves a lot for our clients to learn by observing and collecting data as users toggle their attention between the technology (e.g. iPad, laptop, or mobile phone) in their hand and that big HD flat screen on their wall.
TV is one, but what are the other screens?
This concept has evolved rapidly.  In a short time, we went from simply gaining a bit more information by using the digital menus that our cable providers deliver to being able to chit chat about what the stars have on at the Oscars in chat forums and via twitter feeds on the traditional web.  Now, we have specially designed applications, synched by sound, that are designed to simultaneously augment the viewing experience by promoting social and engaging experiences!  The second and third screen is alive and well and the delivery system is our laptop, tablet and/or smartphone.
Who is taking advantage? (…and who SHOULD be.)
As our investigative research continues, we can’t help but think there is still a lot of room for improvement as companies endeavor to blend screen one with the others.  Some have given it a shot but still haven’t found that sweet spot of allowing a passive and active experience to live in harmony, and some have yet to consider this opportunity.  Take Jeopardy for example.  I can only imagine that the viewing experience on Jeopardy would be enhanced with the ability to play along with the audience and compare myself to the contestants as they play live!  Sure, I can download their HD iPad app, but it wouldn’t be easy to respond to both the questions on the app AND participate on TV.  I don’t know about you, but they’d lose me on one of these interfaces and there’s always a chance that they’d lose me for good.   Others, like Disney, have pushed this to the next level.  With the release of their TRON and Bambi second screen app your iPad or laptop interacts with you along the same timeline as your movie.  The design promotes the synchronized companion experience that they’re seeking and they hang on to the viewership and therefore the ability to advertise to you on both channels.
But it’s not over.  The race is on to create a solution that executes this perfectly.
What is KLI doing to enhance these experiences?
One of our clients said it best:  “I want to create a unified and seamless user experience for my audience, regardless of how tech-savvy (or not) they may be”.  And it was born.  Appropriately, we’ve called this experimental design “Unified User Experience Research”.  We dig in and learn what users love and what frustrates them about splitting their attention between these screens.  We listen to and observe them, and we take their feedback and ultimately arrive at qualitative and quantitative data points to inform design – and put bigger smiles on the faces of the users AND the clients.  In this process, we’re helping our clients understand how to blend the channels, how to hook their audience and encourage use of the second channel even when after the credit screen has rolled on the primary broadcast.
The results of this work have been enlightening.  We’ve seen dramatic increases in retention and engagement!  Above all, we’re thrilled to be part of the process as this concept is optimized and evolves to become standard.
Take a look at these other second screen experiences:
Prime Time Audience:
ABC: Greys Anatomy Synchronized Experience
FOX Broadcasting for Bones & Glee
NBA GameTime Courtside by NBADigital
MSNBC’s Lean Forward Campaign
For Kids (or the kid-at-heart):
Disney: TRON and Bambi Second Screen
& Vanessa, a two-year old little munchkin loved by all of us at KLI, has a favorite interactive experience that keeps Dora the Explorer on her mind.  What’s better than watching Dora WHILE playing along with the Explorer on the iPad?