Between June and October 2020, we (Amanda Norrlander from Renaisi, Tom French and Tom Watson from The Good Ship) came together to research and evaluate the Design Hop - a short course which introduces the main concepts of digital service design to charities.
Five themes from the research (among the many specific findings) really stood out for us.
1) The voluntary sector is massively diverse and catering for it with a single entity is hard, perhaps impossible
Charities come in a huge number of varieties - from the £1bn-a-year Wellcome Trust to the local WI, with no money to speak of at all. When creating services to the sector, it’s important to think which bits of the sector you can usefully target.
The Design Hops offer meant different things to voluntary sector organisations of different size (and income) and self-defined starting points in terms of their ‘digital journey’. Attendee motivations and the challenges they wanted to address (as expressed in the signup form) varied greatly.
For example, smaller organisations tended to be motivated by a desire for better digital communications. Larger organisations, on the other hand (and as you might expect), talked of wanting to develop strategy and culture change around digital. Whether delivered as online elearning or a half-day face to face workshop, Design Hop cohorts are designed to be a mix of organisations of different sizes and maturity levels. While many attendees reported that they appreciated this aspect, it does present a challenge of where to focus the content to keep everyone engaged - and that’s without even getting into the many interpretations of the word ‘digital’, which we’ll come to shortly.
Size also appears to relate to digital confidence. Smaller organisations self-assessed at lower digital maturity levels upon signup, and comprised a comparatively low percentage of overall attendees when we compared NCVO’s voluntary sector analysis.. The highest cancellation rates were among those who stated that they were at the beginning of their digital journey. This may of course be a phenomenon seen across many learning initiatives.
2) Language matters and it’s best not to assume there’s a common vocabulary about anything
Language of ‘digital’ is still understood and perceived very differently across the sector. One person’s ‘digital’ or ‘design’ is not the same as the next, and as a consequence, expectations of what they’re going to experience differ. This, in turn, affects the outcomes that a person or organisation goes on to achieve.
Several people we spoke to mentioned a mismatch between what they thought they were signing up to and what they actually experienced. The Design Hop essentially offers a framework for effective problem-solving (‘the design process’), but many signed up expecting tangible tips for social media management or lists of useful tech tools, because that’s what ‘digital’ meant to them. Furthermore, organisations’ ‘digital’ needs and priorities cover a broad spectrum, including things that are internal or very technical, while the Design Hop’s focus on user-centred design is geared towards better responding to the needs and behaviours of beneficiaries. There were references to not having read the small print, suggesting that assumptions are made based on language that means different things to different groups and can sometimes feel jargony.Should we even be using the word ‘digital’ at all to describe some of the concepts introduced in design training like Design Hops?
3) What people gain from attending a half-day workshop might be less specific than what we would like to think
When we started to dig through CAST’s data, we found some evidence of concrete takeaways and next steps people had taken after attending Design Hops, such as using specific tools, speaking to colleagues about the content of the course and/or having an idea to test out. Attendees’ next steps and outcomes depended on a variety of factors, including digital maturity, support from other colleagues internally, and their own inner drive to change the way they work.
We found the most impactful takeaways, however, were far more conceptual, mindset-changing, and comprehensive. Three takeaways were repeated over and over again – attendees reported having gained confidence, having gained a new way of thinking, and having a greater understanding of what resources are out there.
“The most you can hope for in 0.5 day is that people are enthused – that they are shown the art of the possible and that it starts to point the way of how they would like to progress in this area.” (CREW)
And considering what people can actually expect people to gain from a half-day workshop, should we really expect them to take specific actions, or is it more beneficial to them that they have gained a general boost of confidence to ‘speak and do’ digital? Perhaps the more important question is, how can CAST and other similar organisations support people to translate that confidence into actions through additional follow-up activities and support?
4) There is a need for support participants post-workshop to keep up the momentum
Several people we spoke to expressed difficulties when they had to action what they’d learnt in the Hops. The most common barriers to action were lack of time, capacity, resources and of course the onset of Covid-19. However, there’s a sense that these barriers are not insurmountable, given the right support.
The hardest bit for many, particularly in these times of extensive home working, is to work in isolation;not being surrounded by like-minded people to remind you of what you’ve set out to do. People described having felt excited and motivated during the Design Hop and shortly after, but then normal day-to-day life with its full inboxes and other pressing issues got in the way.
How can we overcome this? What can we do to support people to continue walking down the design thinking and digital road that they’ve already started on?
One thing that came out from this study was the need for a network – a place where you can tap into others’ knowledge and experience, and where you can find people that understand the design process in a way your colleagues might not. Another thing people mentioned was the value of having an expert they could check-in with at regular intervals, so they could be held accountable.
In this sector, we often lead busy lives and getting things done that are outside our immediate focus can be challenging. It can almost feel impossible at times. However, connecting with other people and receiving support from others are two things that help push new projects forward.
5) There is a tension between dynamic delivery and consistency in evaluative data collection
One of the great strengths of Design Hops (and CAST as an organisation) is its ability to flex; to respond to needs quickly and adapt delivery, both locally and at scale. The Design Hops service has been built using a range of off-the-shelf tools, such as elearning platforms to deliver online training (Thinkific), and services that facilitate easy sign-ups and data collection (Airtable). The challenge of using a smorgasbord of tools is that it creates a lot of disparate, hard-to-connect data points. This makes it harder to evaluate, as it’s hard to piece together an end-to-end user experience using the quantitative data available as an ‘outsider’ tasked with doing this.
This is something that needs thinking about in more depth across the sector. How do we all work towards consistency and clarity with data collection without constricting the flexibility of delivery that is so key to being able to respond to change, and to learning about the most effective ways to deliver? Answers on a postcard, please…
Overall, the evaluation and service review of Design Hops uncovered how positively they have been received by those attending. This positivity is largely due to the informal and exploratory nature of the Hops, as well as the relationships between peers and facilitators. As highlighted above, the research activities have also unearthed some bigger questions
We would welcome any thoughts on how these things have resonated with you. Please get in touch with any comments or questions that you may have (see below). And watch this space for more reflections on how we worked in partnership to do this research; there was plenty of learning in the process and methodology too.
@MrTomFrench on Twitter
@a_norrlander on Twitter