Monitoring and Evaluation

Introduction

As the UK City of Culture 2021, a detailed Monitoring and Evaluation programme has been established in Coventry and will run until 2024. The evaluation of Coventry 2021 is being led by the University of Warwick and Coventry University with a number of external consultancies contributing to it.

Released in 2019 and revised in early 2020, the Performance Measurement & Evaluation Strategy details the Trust’s Theory/Story of Change which highlights the change that is sought as a result of Coventry being the UK City of Culture. Developed in partnership between the University of Warwick, Coventry University, Coventry City Council and Coventry City of Culture Trust, the strategy outlines how the UK City of Culture 2021 programme and activities intend to impact on the city and its peoples along cultural, social, economic, health and wellbeing, and other impacts. The planned outcomes of the Theory/Story of Change drove the work and planning of Coventry City of Culture Trust in preparation and delivery for Coventry 2021.

The monitoring and evaluation of Coventry UK City of Culture 2021 is a step-change in expectations around evaluation, impact reporting and learning. These interim and supplementary reports have been prepared and written by the Core Monitoring and Evaluation Team led by the University of Warwick and Coventry University. They cover the period from winning the bid to be UK City of Culture 2021 in December 2017 to the end of the delivery of the City of Culture Year in May 2022. Reporting against Coventry City of Culture Trust's Theory of Change, they detail achievements and tell the story of the delivery of Coventry UK City of Culture 2021. 

INTERIM REPORT: Dec 17 - Nov 21

SUPPLEMENTARY REPORT: Dec 21 - May 22

Faith and Culture

How did programming and cultural leadership engage faith and community groups in Coventry and the region?

Integrating the Environment

How successfully has the environmental theme been integrated into Coventry 2021?

In previous UK City of Culture programmes, the natural environment has been largely ignored, including the original bid document for Coventry UK City of Culture 2021. However, the environment was subsequently incorporated into programme planning, as well as overarching governance and reporting. It was therefore granted an explicit position within the cultural spaces of the event, whilst the environmental impacts of running individual activities also came into focus.

Given the rapidly increasing attention being given to human impacts upon biodiversity and climate, such a focus upon environmental issues in the cultural space was welcomed.

Conclusions

• Events within the Green Futures programme acted as a vehicle for re-focusing attention on nature and green-related issues across the city. Participants appreciated the impact that the connection with nature had on them (such as benefits to their mental health) and their families.
• A community-focused approach − supported by adequate resources and co-operation between local communities, green and/or environmental organisations and key local decision-makers − can increase engagement in nature and wider green issues.
• Overall, Coventry 2021 created an opportunity to foreground a ‘Green Agenda’ for the city, linking communities with nature and the environment creating hope that the City of Culture experience would leave an enduring Green legacy.

Integrating the Environment Focus Study Report

Civic, Cultural and Business Partnerships

What effect has the title of UK City of Culture had on civic, cultural and business partnership co-working and to what extent can these partnerships be sustainable?

A three-part qualitative study was carried out through a series of in-depth interviews with staff and a range of new and existing partners of four organisations covering the different types of partnerships. The research team specifically focussed on partners and protagonists away from the larger city institutions to ensure that diverse voices were included in articulating the overall impact of the title on the city.

Conclusions

• The effects of the title on the city were felt across all three types of entities, as they impacted on the scale and scope of their operation.
• All four organisations worked with new collaborators and partners (e.g. national and international artists, residents from other international cities, arts professionals, and press and media).
• They all underlined the importance of their existing partnerships in supporting their activities and believe that they have been strengthened during the year.
• The sustainability of any new ways of working, collaborations and partnerships formed has been queried. How far they continue without the funding, specific plans for future activity, and the external visibility brought about by the title is a matter of some debate.

Civic, Cultural and Business Partnerships Focus Study Report

Volunteering, Wellbeing and Civic Pride

How and to what extent has the City Hosts’ volunteering programme impacted on those taking part, particularly in terms of subjective wellbeing and civic pride?

The aim of this focus study was to understand how and to what extent the Coventry UK City of Culture City Host volunteering programme impacted on those taking part, particularly in terms of their subjective wellbeing and civic pride. Additionally, the study aimed to explore the mechanisms of change and intermediate outcomes between volunteering and subjective wellbeing. The What Works Centre for Wellbeing Theory of Change was used to guide the study’s analysis.

Conclusions

• The focus study suggests that the City Host programme increased Volunteers’ wellbeing and that this was achieved through many mechanisms, particularly through increasing social connectedness and sense of belonging.
• Strengths of the City Host programme have been identified that other volunteer programmes could emulate to promote positive outcomes for their volunteers.

Volunteering, Wellbeing and Civic Pride Focus Study Report

Policing, Culture and Community

How can the police use Coventry UK City of Culture 2021 as a platform to engage the public and improve public perceptions of policing, whilst simultaneously helping to manage crime and protect event attendees?

West Midlands Police was a principal partner of Coventry City of Culture Trust in the delivery of Coventry UK City of Culture 2021.

In addition to the obvious community safety aspects of running a city-based cultural mega-event (from policing large audiences, to licensing venues and outdoor spaces), collaboration was underpinned by numerous shared values and objectives around community engagement, public safety and protecting vulnerable populations.

The University of Warwick led a study sought to understand the potential for police partnerships around arts and culture, to positively impact reducing crime, protect vulnerable people, increase diversity in recruitment, and on police relationships with young people and seldom heard communities.

A report will be made available shortly.

To gauge the awareness, understanding and sentiment towards Coventry City of Culture 2021, M·E·L Research were commissioned to conduct interviews with a representative sample of Coventry residents. Establishing a baseline level prior to the announcement of the events programme for the year, the study was repeated to gather a midpoint and endpoint assessment.

Sentiment Research Baseline Report

Sentiment Research Midpoint Report

Sentiment Research Endpoint Report

In April 2020, the Trust commissioned MB Associates to undertake a Social Return On Investment study to help understand and quantify the social, environmental and economic value that the UK City of Culture title has brought. The study will examine five key projects and will be reported on in early 2023.

The team at Coventry City of Culture Trust refocused their work following the end of the City of Culture Year, with a series of Legacy Projects that support the creation of new arts and culture projects that contribute to a better, greener, fairer future for Coventry, alongside the continuation of the popular temporary festival site in the city centre and a major new visitor attraction for the city.

The following evaluations have been produced by Coventry City of Culture Trust in consultation with evaluation partners to support its data-led approach to decision making.

To understand the economic benefit to Coventry and the region, the Trust commissioned AMION Consulting in April 2020 to undertake an externally validated long-term economic impact assessment into the economic benefit which the UK City of Culture title has brought to Coventry.

Running until 2024, the study will examine key economic indicators. A baseline report has been completed and is available below. Further reporting will take place in April 2023 and September 2024.

Economic Impact Assessment Baseline Report

Commissioned by the Arts and Humanities Research Council and part of the Warwick UK Cities of Culture Project, this review promotes a research-led approach to how culture might lead to impacts that address the specific needs of a place and its diverse communities.  A key driver for the UK City of Culture programme is to invest in places that have a clear ambition and evidenced plan for culture-led city transformation. The bidding process itself produces benefits for shortlisted as well as title cities and regions.

From accounts of lived experiences and hyperlocal cultural impacts, to large scale social and economic benefits and increased civic pride, research and evaluation can enhance the understanding of the conditions for success and sustainability of the UK City of Culture programme.

The review includes a series of interim proposals to drive a more strongly research-informed approach to the UK City of Culture programme. The Appendices to the review give an overview of geo-demographic and other data for four rounds of short-listed and title cities and regions.

Towards a Research-Informed Approach Review

The Future Trends series was completed by University of Warwick as part of the Warwick UK Cites of Culture Project and explores different aspects of Coventry UK City of Culture 2021. It aims to provide accessible, research-led accounts of issues related and relevant to the development of the UK City of Culture programme.

Innovations In Economic Impact Assessment

Within UK public policy, economic impact assessment has been a long-run mainstream staple for assessing the benefits to society of a policy intervention. It has thus supported learning and policy design.

Coventry UK City of Culture 2021 has sought to use an innovative approach to economic impact assessment to gain a more complete and better-grounded understanding of the benefits generated by the cultural mega-event. The aim has been to support learning and the development of good practice in the use of non-reductive valuation frameworks within the cultural and heritage sphere.

Innovations In Economic Impact Assessment Report

Social Value Creation and Measurement in the Cultural Sector

Can arts and culture change someone’s life? Can a City of Culture change the lives of every person within a region?

The term ‘social value’ has increasingly been used within the cultural sector to categorise a way in which the arts impact individuals and groups. This paper provides a cross-disciplinary overview of this trend and illustrates how social value can be generated and measured in cultural contexts. Social value represents the value that people experience as a result of changes in their lives. As such, social value is subjective to those who experience change.

Social Value Creation and Measurement in the Cultural Sector Report

Reasons to Co-create

Co-creating is difficult, so why do it?

Co-creation is a popular term but the outcomes of co-creative practices and the reasons for doing it are poorly understood. This is a problem because, arguably, the value of co-creation cannot be understood independently of the reasons for which people co-create.

Reasons to Co-create Report

Addressing Cultural and Other Inequalities at Scale

Publicly funded arts organisations are seeking new ways of engaging with communities to ensure art can be for the many and not the few.

While the UK City of Culture programme has – like similar cultural mega-events rooted within place, and the priority area funding programmes from Arts Council England – reduced spatial inequalities, they do not seek to reduce social inequalities as a priority.

By focusing on hyper-local offerings, the programme offered local proximity and the potential for a sense of ownership by the kinds of people who are most often missing from cultural spaces.

Addressing Cultural and Other Inequalities at Scale Report

Maximising and Measuring the Value of Heritage in Place

Heritage (tangible, natural, and cultural) and Place have a mutually-symbiotic relationship, and much intangible heritage has a strong place-based association and origin.

UK Cities of Culture provide a valuable opportunity to drive place-shaping efforts and improve impacts from local heritage engagement through the involvement of host communities and the development of participatory co-produced research that employs socially-engaged practices and spatial and visualisation approaches.

Maximising and Measuring the Value of Heritage in Place Report

Measuring the Impact of Arts and Culture on Wellbeing

What does the growing body of evaluation research say about the wellbeing value of arts and culture interventions? What are the factors that enable wellbeing outcomes and reduce social inequalities?

Key findings from a new rapid review on what works to improve wellbeing in arts and culture interventions, connecting research findings with the priorities of policy-making. Looking ahead to future wellbeing and place-based research, making recommendations on how to generate and sustain wellbeing value creation through arts, culture, and heritage interventions.

Measuring the Impact of Arts and Culture on Wellbeing Report

Building Trust in Policing through Arts Collaboration

Can police engagement through creative partnerships help to rebuild public trust and confidence?

Coventry UK City of Culture 2021 offered the opportunity for the police to explore how to use arts and culture to build positive relationships with vulnerable, at-risk, and seldom heard communities through collaborative working with arts practitioners and 3rd sector organisations.

Building Trust in Policing through Arts Collaboration Report

Coventry UK City of Culture 2021 aimed to engage the youthfulness and diversity of the city in its cultural offer. To ensure the programme met local and regional needs and is evidence-based, Coventry City of Culture Trust and its partners developed a proactive performance measurement & evaluation strategy based on a theory of change model.

Performance Measurement & Evaluation Strategy

Theory of Change

Data captured by the Trust as part of its monitoring activity, and any derived findings concerning the Theory/Story of Change outcomes and impacts are reviewed by the Universities. There is also an external Technical Reference Group, of experts in social, cultural and economic evaluation that meets quarterly to receive reports and comment on the rigour and progress of the PM&E Strategy. This group signs off on findings and other evidence of impact. The terms of reference for the Technical Reference Group are below:

The Technical Reference Group will act as an independent and impartial group advising the Core Monitoring and Evaluation Team around current evaluation practices, methodologies, ethics and data aggregation. The group will be key in aiding and assisting in the delivery and execution of monitoring and evaluation activities for Coventry being UK City of Culture in 2021 and the delivery of key evaluation reports in early 2022. Monitoring and evaluation activities are being delivered through a partnership between the Coventry City of Culture Trust, University of Warwick, Coventry University, Coventry City Council and other city partners as required. The remit of the Technical Reference Group will be:

• Support the monitoring and evaluation activities through guidance on appropriate methodological approaches, validation of surveys and ethics around the collection of data;
• Assist in identifying gaps in the reporting data and how new data sources can be aggregated alongside current data sets;
• Provide input to ongoing monitoring and evaluation activities where appropriate, drawing on the expert knowledge and experience of the members of the group;
• Guidance on ensuring relevant evaluation is being undertaken to meet funding body requirements;
• Advise on the dissemination of data and key results from the monitoring and evaluation activities to key stakeholders, funders and other interested parties.

For the period up to June 2021, six Progress Reports have been published by the Monitoring and Evaluation team and these are available to download here.

Working with our Monitoring and Evaluation partners, we created a handy document designed to help organisations who were applying to be an Arts Council England National Portfolio Organisation.

What Data Tells Us About Coventry is full of useful information that covers demographics, cultural engagement, education, employment, green futures, economics, health, tourism and young people.

What Data Tells Us About Coventry

University of Warwick

Coventry University

Coventry City Council