Museum evaluation: we have nothing to fear but fear itself
Evaluation counts, says Nicky Boyd, but we need to stop asking the same questions and start sharing our data with others
For more than 12 years now I’ve worked within the museum and heritage sector in the field of evaluation and audience research, and I’m reasonably confident the sector can now see the benefits of evaluation. However, I often feel frustrated at the barriers that we, as museum professionals, collectively raise, which often prevent us from sharing what we find out from the evaluation process.
We are still reticent about telling others what hasn’t worked, we still ask the same old questions, and we don’t look internally or externally as much as we could for useful data. We just don’t seem to learn as well as we potentially could from the experiences of our peers.
A colleague suggested that perhaps it’s just part of the human condition that we need to keep asking the same questions of museum audiences. Does this suggest we need reassurance or is it, fundamentally, about the fact that we believe our organisations are so unique we don’t feel we can apply what others have found out about their audiences to our own situation?
Does it feel easier to keep asking the same questions rather than spending time applying someone else’s research to our own organisation? I’ve come to the conclusion that breaking down the barriers to sharing is much more than the question of whether we publish our findings. Many museums already do this well, for example the V&A, who commission a lot of research and put reports online, and the Natural History Museum, who share evaluations of exhibitions and learning programmes. I think we need to do more.
We know that carrying out evaluation provides evidence about how projects and programmes are meeting aims and objectives, and encourages ongoing project improvements. We know it can act as an excellent tool for telling others about the great work being done in the sector, enabling us to share lessons learned with colleagues.
Evaluation can help the project team and participants feel like they ‘own’ the project, it can develop relationships with visitors and play a really important role in developing ideas and planning future projects. We also know that if we don’t carry out any evaluation we could waste time and money, produce something that we can’t change, lose interest from our target audience, waste an opportunity to learn something useful, and lose funding.
There are a number of excellent online toolkits readily available that steer willing participants through the whole evaluation process, for example the East of England Museum Hub’s Evaluation Toolkit for Museum Practitioners. The Smithsonian Centre for Education and Museum Studies has just set up an online community of practice around evaluation, which also looks really exciting.
So we understand the benefits of evaluation and how to do it, but why are we not sharing what we find out through the evaluation process more effectively? One of the important factors may be that many of us are still reticent about telling others about what hasn’t worked so well. I really understand the fear behind this.
For many reasons we want to portray projects, programmes and events in the best light, to secure future funding or to retain professional standing, but in its purest sense evaluation should be seen as an opportunity for us to learn and improve our professional practice. It should not be done to point fingers or assign blame. Rather it should help create a culture where we’re able to take risks and share what hasn’t gone so well. Only then can we learn and move forward as a sector.
I think a possible way forward is to re-package the data already held within organisations and think more strategically about how we plan evaluation. The organisations I regularly work with are often surprised with how much data they actually have once they start looking! Working with the education and interpretation teams in a heritage organisation, a colleague and I tried this as a practical exercise. After some initial hesitation, followed by discussion and deliberation, the team realised they could pull out useful audience information from their existing evaluation data which they could put to efficient use in planning future projects and programmes.
Maybe the next step would be to work with two or three similar organisations to share audience consultation and evaluation data around specific audiences. Maybe a group of museums could agree some overarching research questions so all the evaluation and audience consultation work they do individually could collectively feed into answering these questions.
Let’s be confident about what we know
As a profession we need to be confident about what we already know. I am often asked to find out the barriers secondary school teachers have in taking their students out to museums. I can safely say that I’ve asked a lot of teachers this question and I know what the answers usually are, so I encourage the organisations I work with to ask something which might be fundamentally more useful
A recent project with the National Maritime Museum (NMM) in Greenwich took this a step further when they asked secondary teachers what the NMM could do to help teachers advocate internally for a museum trip with their students. Should the NMM guarantee that the school could bring a whole year group at once or guarantee specialist input from a member of museum staff for the visit? Should they provide evidence of impact on attainment targets or information that highlights the benefits of learning outside the classroom?
The museum plans to share the results of their research with the sector. Having the confidence that they know about the barriers teachers have to visit the museum with students has enabled the NMM to find out new information which will ultimately improve their school offer.
Sharing documents is not enough
As a member of the London Museums Group, I know there are certainly more conversations we can have as to how Share London, our online forum for sharing practice, can help us to share evaluation and audience consultation findings more widely. We need to remember that posting our analysed and interpreted evaluation and audience research data might not be enough and more work and conversations need to be had about how we stop reinventing the wheel.
Ultimately, time is always precious, and if we can stop asking the same questions, use the data we already have and share our findings more efficiently we – and our audiences – can all only benefit in the longer term.
Nicky Boyd is a museum consultant for audience research and evaluation – follow LMG on Twitter @LonMuseumsGroup