Card sorting as user research for content projects

This resource is for:

• People working in small charities developing collections of information, support, guidance or blogs

• People new to user research who want new ideas to support content design

It covers:

1. An overview of standard card sorting
2. 3 ways to use card sorting to help you design your content
3. Resources to help you learn more/ get started

This is our first #HowIUseIt style article. This series looks at how members of the Catalyst Network or re-purpose standard tools to support organisations.

What is a card sort and how do you do it? 

User researchers typically use card sorting to help plan information architecture (website structure, or how your menus or filters work) 

The activity works like this:

• You write a list of topics or page titles on cards
• You give them to people and ask them to sort them into groups. What would they expect to find together?  

The sorting step can be open or closed.

Open: The people sorting decide on the groups, and once they have finished sorting cards name the groups. This is common very early on in a research process.

Closed: You provide a short set of group headings and ask people to sort within those headings. This is useful when your in house team has an idea of a draft structure. 

You can run card sorting as a quantitative exercise with a focus on the end placement of the cards as the main research goal.  You can also run it more slowly with a focus on asking people to talk out loud whilst they are sorting.

Different tools sort different situations and goals:

• Note cards or post-its on a table or wall grid in a face to face session. 
• Online user research tools such as Optimal Workshop or Maze.
• Any Kanban board type tool -  creating your own card list and drag and drop space

If you use a specialist tool it will also provide you with a statistical analysis of the most common groupings. You can also do this yourself by documenting card placement in a spreadsheet.

Making the most of card sorting

I’ve been using card sorting since I first started doing user research at Neontribe in 2015. In its common form, focusing on information architecture, I think it is really hard to get right.

• You have to be careful with how you write your topics. For example people group titles with similar word patterns together. For example people group "Running X, Running Y and Running Z" together. But they also group other things relating to X, Y and Z into categories that they call X, Y and Z.

• You need to be careful that you do card sorts with a diverse range of your website users. You need to include new and potential users. If you don't, your card sort may reflect organisational thinking more than it reflects human understanding.

• Often organisations in the charity sector already have a reasonable grasp of how people might group their content. It would be a better use of their time with new users to test those groupings rather than starting from scratch with sorting. 

But I still love card sorting. I think it’s a really useful tool for all charities to use when they are working on improving their website content.  I use it for insights that are relevant to the page titles and their content, not just menu structure.

Here are my top 3 ways to make it work for you. I’ve used these on several projects with organisations of different sizes.

‍Focus on the “talk out loud as you sort” part of the activity

Do card sorting as an activity with a user researcher present. This could be part of a face to face or online interview or testing session.  It can be a closed or open sort.

For your cards, use existing in house topic categories or page titles OR use new draft titles OR mix and match.

Emphasise that you’d like to know the participants’ thinking as they sort the cards.   This will help you understand some key things about your content design:

• Does the topic or page title have the same meaning for them that you intended? What can this tell you about the way you tend to phrase things as an organisation?
• Are they categorising things based on the topic or by something else about how the topic is worded? Is this an issue only for menus or could it affect how people use pages too?  
• How sure are they that they understand what the topics/page titles are? Do you need to improve this?

Add an “I don’t really know what this” category to the possible groups

You can do this even if you are running an “open” sort where the participants label their own categories. It helps give people permission to be unsure.

This will help you:

• Spot jargon
• Find complicated phrasing that you need to improve

Forget about menu structure - change the way your card sort works

Card sorts usually look at menu structure. But you can change them to give a sense of what topics are most important to people.

Create a closed card sort using any method.

Create topic/title cards covering the range of topics that you  already have content about AND that you want to provide content about.

Create categories that relate to whether people have looked for information on a topic in the last six months.  This follows the user research principle that past behaviour is a good predictor of what people will actually do.  

For example:
Looked for it and found something
Looked for it but never found anything helpful
Didn’t look for it
Don’t know what it is

Use the “talk aloud method” and be ready to ask follow up questions. This is most important when people put things in the “Looked for it but never found anything helpful” category. It will help you spot pain points in finding information. 

This won’t give you an exact priority order list of content to work on. But it will give you:

• Indications of what people are looking for and what they are not
• Pain points or indications of why people are struggling to find what they need at present
• Indications of why people do or don’t look for certain information
• All the same benefits of the first two tips above

Using card sorting with any or all of these 3 tips in place will help you create a collection of insights. Some of those insights will apply to your whole information, guidance or blog collection. Others will be user stories that can help shape specific pages.

Resources to help you learn more/ get started:

• Read: Want a more thorough guide on running a standard card sort for information architecture? Nielsen Norman have a thorough guide.
• Listen: Thinking about how to plan a content design project overall? Listen to Jack Garfinkel from Scope on Content Design project planning.  
• Watch: When you are working on information architecture, card sorts alone are not enough. Software provider Optimal Workshop explain how to use them alongside tree-jacking or tree testing activities.