Developing a Culture of Quality Enhancement: Self-assessment Tools for Digital Education

The Digital Education Action Plan (2021-2027) has a strategic priority of “fostering the development of a high-performing digital education ecosystem” (European Commission, 2020). Set against this ambitious goal and increasing focus on digital education in the shadow of the COVID-19 crisis, a NIDL team was pleased to contribute to a study reviewing a range of self-assessment tools and frameworks that support quality enhancement and strategic institutional planning for the future.

EUA, 2021

Current evidence suggests use of self-assessment and/or benchmarking instruments for digital education capacity development is not particularly common in Europe, as illustrated above in the findings of a related survey of digital learning and teaching in European higher education institutions (EUA, 2021). This survey along with the review of self-assessment instruments is part of the Digi-HE project co-funded by the Erasmus+ Programme of the European Union. 

Volungevičienė, Brown, Greenspon, Gaebel, & Morrisroe, 2021

The outcome of this work is a report published by the European Universities Association (EUA) that should be of particular interest to higher education institutions, but also to policy makers and quality assurance agencies. Advice and guidance is provided for higher education institutions on how to select the right instruments, use them as part of self-assessment, and share and act on the findings. To begin with the review was framed by the following research question:

What does digital education look like from an institution-wide perspective when successfully implemented in a mature way? Not surprisingly the answer to this question is complex.  

A brief description of each self-assessment instrument is provided, along with an overall summary of their strengths and weaknesses. Insightful observations are offered in terms of striving for digital education maturity in higher education institutions, and, in particular, the use (or non-use) of these tools and frameworks for critical self-reflection, capacity development and for fostering a quality culture of continuous self-improvement.

Two observations are particularly noteworthy. Firstly, the concept of maturity, which features in a number of the instruments, is potentially problematic if narrowly interpreted as being static or linear in terms of progression.

The key point is that the edge of digital education is constantly evolving and being pushed and pulled in multiple directions, which makes it difficult to shoehorn an unknown future into a self-assessment tool or framework developed with today’s thinking.

Secondly, an overly narrow focus on digital education outside of other quality review and strategic planning processes may not take sufficient account of wider change forces and global developments facing higher education (e.g. globalisation, demographic changes, new funding models, etc.), which all have an influence on the development of a future-ready, digitally capable organisation. 

Importantly, the report highlights the role of local institutional contexts in the selection of the most appropriate tool(s) for the intended purpose. It also underscores the value of collaboration both internally within higher education institutions and externally with other stakeholders in order to support the development of high performing digital education ecosystems. 

At risk of borrowing a cliché, the report concludes there is no one-size-fits-all model of critical self-assessment for digital education capacity development as institutional context is crucial. 

Rather a dialogical, “pick and mix” approach may be more productive in terms of efforts to help scaffold critical self-assessments that lead to real and transformative change in higher education institutions. Such an approach recognises that the process of continuous development in digital education needs to be infused throughout institutional culture, as well as part of a wider ecosystem that promotes critical self-assessment as a shared ethos and collective responsibility of all educators. 

DCU’s Experience of Critical Self-assessment

In 2020, DCU engaged in a comprehensive institutional review and self-assessment of digital learning to reflect on previous investments and help steer and shape future strategic plans for teaching and learning.  This self-assessment exercise actually began at the end of 2019, but work in 2020 was delayed due the Covid-19 crisis. A Steering Committee was established consisting of 18 staff representing all faculties and most service units across the University. The reflective self-assessment was informed by several recent surveys as well as by focus group interviews with DCU staff and students. Completion of institutional benchmarking using the ACODE Framework proved to be particularly useful in helping to scaffold critical reflections and resulted in many productive conversations. 

To enhance the process of self-improvement through critical reflection an external review panel was appointed at the start of 2020 and eventually met virtually in early December with a range of DCU staff and relevant stakeholders. The comprehensive self-assessment report completed before the external review reflected on strategy and policy; investment in both hard and soft infrastructure; strategic partnerships; research, innovation and thought leadership; staff development; student learning resources and supports; implementation across faculties; staff and student perspectives; response to Covid-19; external engagement; and alignment to national and international standards and best practice. As noted by Professor Mark Brown, a meeting of the DCU Steering Group:

“Self-assessment of digital learning has been highly beneficial in terms of engaging a wider number of stakeholders in critical reflections and helping to better understand our strengths and weaknesses. It has helped to affirm DCU’s strategic approach to digital learning and provided a stronger evidence base for future decisions and investments.”

Finally, a copy of the full report, Developing a High Performance Digital Education Ecosystem: Institutional Self-assessment Instruments, can be downloaded from the EUA website. You can also register for a forthcoming webinar on Tuesday January 26th that will discuss the main findings of the report. A recording of this webinar will be available shortly after the event from the Digital-HE project website.

.