Agnieszka Chalas (Queen’s University)

Title of Project: Painting a Portrait of Organizational Evaluation Capacity in the Canadian Art Museum Sector

Research Questions: Three central research questions guide this study:

1. What are the key dimensions that describe EC in the Canadian art museum sector?
2. How is EC manifested across Canada’s largest publicly owned art museums, and to what extent does this capacity vary from museum to museum?
3. How is EC manifested in those Canadian art museums most active in evaluating their educational programs, and which factors and conditions, according to practitioners’ self-reported understandings, enabled their capacities to develop to their current levels?

Project Description: As education has taken a more prominent role in museums, governments and external funding agencies have increased demands for evaluations of publicly-funded programs in the name of accountability. Despite this trend, research on evaluation practice and capacity in art museums is largely absent. Consequently, not only is little known about what such museums are doing to evaluate their educational programs but the actual dimensions of evaluation capacity (EC) as they could be observed in art museums have not before been conceptualized for these settings or defined based on empirical data. This study seeks to: (a) empirically validate a literature-derived conceptual framework of EC developed to bound this research, (b) describe how is EC manifested both in the sector writ large (sector-wide) and in those museums most active in evaluation across the various dimensions in my conceptual framework, and (c) elucidate specific examples of institutional successes (i.e., exemplary program evaluation, mainstreaming, and capacity building practices, etc.) for the purpose of sharing them across the sector. A two-phase multiple method qualitative research design will be used to address the purposes of this study. Phase One involves conducting an interview study to establish an initial knowledge base on program evaluation (i.e., practice, capacity, building efforts, etc.) in the sector and both revise and validate the initial conceptual framework. Phase Two involves conducting qualitative multiple case studies of three art museums that demonstrate the highest capacity in order to draw out specific examples of good practice. This research will make an important contribution to the evaluation literature by deepening the field’s understandings of EC across a yet unexplored sector—Canadian art museums.

Nest Steps for this Project: I am currently in the process of recruiting participants for the first phase of my study. A key informant purposeful sampling strategy is being used to identify art museum practitioners with responsibility for evaluation working in Canada’s 15 largest publicly owned art museums.

CERA’s impact on Agnieszka’s work: Being a member of CERA has enabled me to gain valuable insights into new methods, methodologies, and evaluation approaches that are being used within the education and museum fields to evaluate the success and impacts of educational programs. It has also allowed me to reconnect with colleagues and network with other professionals in the museum and education fields.