IA Issues? Online Card Sorting is Not Enough.

by Andrew Schall


Understanding how your users think about the organization of content.
If your users can’t find the information that they are seeking, it might as well not be there at all. An intuitive information architecture (IA) is a core part of a user’s experience, but how do you know what would make sense to them?

Card sorting is a technique that can help gain insights into how your users think about the organization of your content. This user research method can be performed using an online tool or in-person using physical cards.

The benefits of online card sorting.
Online card sorting has become a very convenient and common way to collect this information from users. The benefits of online card sorting include:

  • Fast and easy data collection – participants can log into a website and perform the activity at their convenience.
  • Large sample size – it is easy to obtain a large sample size for statistical analysis.
Image provided by Optimal Workshop (www.optimalworkshop.com/optimalsort)

“Image provided by Optimal Workshop (www.optimalworkshop.com/optimalsort)”

Online card sorting has become the predominant way for user researchers to collect this type of data, but it can’t tell you everything you need to know about how to design an intuitive information architecture.

Online card sorting tells you what, but not why.
Collecting data on how your users sort items into categories tells you what they did, but not why they did it. The reality is that designing an information architecture is a messy job, and card sorting can sometimes lead to inconclusive results, especially if the items are particularly challenging for participants to sort.

Qualitative survey questions at the end of an online card are useful, but can’t really help you to get a full understanding of the user’s mental model. There is no way to probe deeper, and participants often give minimal responses to these questions.

Gaining deeper insights with moderated in-person card sorting.
Moderated in-person card sort sessions are typically scheduled for an hour and are conducted one-on-one with a facilitator. Participants are provided with physical materials that include cards labeled with the name of each item to be sorted, pens or markers for making notes, and sometimes color-coded stickers for further annotations.

Image from chapter 6, Information Architecture & Web Navigation, Eye Tracking in User Experience Design.

“Image from chapter 6, Information Architecture & Web Navigation, Eye Tracking in User Experience Design.”

The ability to manipulate physical cards provides many benefits to the participants including:

  • Easier to complete – physically sorting cards into groups requires less instruction compared with learning how to use a new online tool.
  • More flexibility – participants can put aside cards on a table and spread out as much as they need. They can write on the cards, cross things out, make suggestions, and draw out connections between cards, including cross-linking.
  • Higher engagement – participants tend to be more engaged when they are physically moving cards around. They are also less likely to be distracted compared with performing the activity remotely on their own.

The ability to sit with a participant as they sort cards provides many benefits to the researcher including:

  • Understand participant’s thought process – think-aloud protocol can be used to have the participant explain why they are sorting cards in a certain way.
  • Observe nonverbal responses – do they appear confused or frustrated? Do they visibly hesitate to place a card or create a label for a category?
  • Provide motivation – card sorting is a laborious and mentally demanding task. Participants often lose motivation to complete the activity. The facilitator can provide encouragement to the participant to keep going.

Moderated in-person card sorting is more time-consuming both in data collection and analysis than online studies. However, tools are available to reduce the time needed to analyze data.

  • Bar code scanner – bar codes can be used to tag each card and can then be scanned into a computer.
  • Utilize software tools – the same analysis tools can often be used to analyze the results from an in-person card sort.

Combining the best of both methods.

A hybrid approach that includes online and offline card sort activities will provide a more holistic understanding of how your users envision an information architecture. In this approach, it is recommended to start with an online open card sort study to see general trends in how users sort items. Next, follow up with an in-person study with a smaller sample size to understand a participants’ thought process including insights into how they would use the content.

ESPN Revamps Their Web Design – Data Driven or Data Informed?

by Eugenio Santiago
For all you sports fans I’m sure you’ve seen and formed an opinion about the new ESPN responsive website. Most of the comments I’ve seen have been negative. In fact, most site visitors who have expressed their dislike for the redesigned site have done so in a very colorful way:

Screen Shot 2015-04-09 at 10.15.04 AM

Screen Shot 2015-04-09 at 10.14.55 AM

I spent a fair amount of time on Apr 2, 2014 reading through thousands of user comments and was only able to find three that were not negative:
Screen Shot 2015-04-09 at 10.22.01 AM
Screen Shot 2015-04-09 at 10.22.16 AM
Screen Shot 2015-04-09 at 10.22.07 AM
I can’t really say they were positive, just not the common variety of “this is the worst thing EVER!” or “is this an April Fools joke?!”.
Now we all have heard the adages: “If it ain’t broke don’t fix it” or “people don’t like change”, but many of the comments were very specific in detailing WHY they disliked the new design. In short, it is different, but more importantly it forces them to change the way they normally interact with the site. No longer are they able to scan headlines, but now must scroll endlessly through entire stories that ESPN deemed newsworthy**This has been changed since the writing of this article. The homepage now features bi-lines of top stories instead of displaying them in full. 
Here’s an example of how site visitors indicated they typically interacted with the siteI would normally come to ESPN for a quick glance of the top stories then, as time permitted, would dig deeper into the topics / articles that are of highest interest to me. 
What I read over and over in the user comments was that site visitors would normally use the homepage of ESPN primarily as a means to scan. Therein lies the problem with the new design. It’s much harder to do that now. The number of stories/headlines that are available, above the fold, has decreased.
Those in charge at ESPN have taken their 6+ months of user testing along with their business objectives and have decided that the homepage interaction should now be different. Did they underestimate or misinterpret their data and come away with a design that didn’t meet the needs of their site visitors? Or did they intentionally redesign the homepage in a way that would force users to interact with it differently?
As of the afternoon of April 3, 2014 the answer is clear, they goofed. Midway through my article I noticed a change on the homepage. The endless scroll of the homepage where you had to view full stories before advancing to the next was replaced to only include snippets of the main stories. Scanning is now much easier than its initial release, however, some may still not enjoy the endless scroll. While many site visitors commented the design was yet another example that ESPN didn’t “listen” to them, I’d have to say this is clear evidence of how responsive ESPN can be.
If this is any indication of things to come, I could see ESPN making continued updates to further refine the interaction, but how many customers did they alienate in the first two days of this new release? Since I started this story, not only have they changed how they display stories on the homepage, they have darkened the background from white to a gray to increase the contrast, which too, was another complaint site visitors had.
There still remains much to do. Some secondary pages still have the endless scroll from story to story (e.g., ESPN Cities blog pages), which is even more difficult to deal with when on your smartphone. Navigating horizontally through the site content once you go 1 or 2 levels deep is a major issue, and not every team page has the same layout (e.g., Boston Red Sox = New/Good, Boston Bruins = Old). These are some of the first enhancements that would go a long way to improving the user experience.

Screen Shot 2015-04-09 at 10.43.13 AM

New Team Page Layout

Screen Shot 2015-04-09 at 10.43.18 AM

Old Team Page Layout

Conducting user research is important, but what you take away from that research is much more important in our opinion. At KLI, we like to call that ‘being data informed rather than data driven.’ ESPN has shown they can be responsive and make on-the-fly changes that correct/improve the user experience, but are you as flexible?

We at KLI pride ourselves in making sure the insights you learn from research are in alignment with user needs and respect, not only what they do, but why they do it. Whether your user research need is at the concept/ideation stage, pre-launch phase, or post-launch phase, Key Lime Interactive is ideally suited to partner with you.
To learn more about what Key Lime Interactive offers and how we can help you with enhancing your user experience, contact us at: 305-809-0555.