Developing a Culture of Quality Enhancement: Self-assessment Tools for Digital Education

The Digital Education Action Plan (2021-2027) has a strategic priority of “fostering the development of a high-performing digital education ecosystem” (European Commission, 2020). Set against this ambitious goal and increasing focus on digital education in the shadow of the COVID-19 crisis, a NIDL team was pleased to contribute to a study reviewing a range of self-assessment tools and frameworks that support quality enhancement and strategic institutional planning for the future.

EUA, 2021

Current evidence suggests use of self-assessment and/or benchmarking instruments for digital education capacity development is not particularly common in Europe, as illustrated above in the findings of a related survey of digital learning and teaching in European higher education institutions (EUA, 2021). This survey along with the review of self-assessment instruments is part of the Digi-HE project co-funded by the Erasmus+ Programme of the European Union. 

Volungevičienė, Brown, Greenspon, Gaebel, & Morrisroe, 2021

The outcome of this work is a report published by the European Universities Association (EUA) that should be of particular interest to higher education institutions, but also to policy makers and quality assurance agencies. Advice and guidance is provided for higher education institutions on how to select the right instruments, use them as part of self-assessment, and share and act on the findings. To begin with the review was framed by the following research question:

What does digital education look like from an institution-wide perspective when successfully implemented in a mature way? Not surprisingly the answer to this question is complex.  

A brief description of each self-assessment instrument is provided, along with an overall summary of their strengths and weaknesses. Insightful observations are offered in terms of striving for digital education maturity in higher education institutions, and, in particular, the use (or non-use) of these tools and frameworks for critical self-reflection, capacity development and for fostering a quality culture of continuous self-improvement.

Two observations are particularly noteworthy. Firstly, the concept of maturity, which features in a number of the instruments, is potentially problematic if narrowly interpreted as being static or linear in terms of progression.

The key point is that the edge of digital education is constantly evolving and being pushed and pulled in multiple directions, which makes it difficult to shoehorn an unknown future into a self-assessment tool or framework developed with today’s thinking.

Secondly, an overly narrow focus on digital education outside of other quality review and strategic planning processes may not take sufficient account of wider change forces and global developments facing higher education (e.g. globalisation, demographic changes, new funding models, etc.), which all have an influence on the development of a future-ready, digitally capable organisation. 

Importantly, the report highlights the role of local institutional contexts in the selection of the most appropriate tool(s) for the intended purpose. It also underscores the value of collaboration both internally within higher education institutions and externally with other stakeholders in order to support the development of high performing digital education ecosystems. 

At risk of borrowing a cliché, the report concludes there is no one-size-fits-all model of critical self-assessment for digital education capacity development as institutional context is crucial. 

Rather a dialogical, “pick and mix” approach may be more productive in terms of efforts to help scaffold critical self-assessments that lead to real and transformative change in higher education institutions. Such an approach recognises that the process of continuous development in digital education needs to be infused throughout institutional culture, as well as part of a wider ecosystem that promotes critical self-assessment as a shared ethos and collective responsibility of all educators. 

DCU’s Experience of Critical Self-assessment

In 2020, DCU engaged in a comprehensive institutional review and self-assessment of digital learning to reflect on previous investments and help steer and shape future strategic plans for teaching and learning.  This self-assessment exercise actually began at the end of 2019, but work in 2020 was delayed due the Covid-19 crisis. A Steering Committee was established consisting of 18 staff representing all faculties and most service units across the University. The reflective self-assessment was informed by several recent surveys as well as by focus group interviews with DCU staff and students. Completion of institutional benchmarking using the ACODE Framework proved to be particularly useful in helping to scaffold critical reflections and resulted in many productive conversations. 

To enhance the process of self-improvement through critical reflection an external review panel was appointed at the start of 2020 and eventually met virtually in early December with a range of DCU staff and relevant stakeholders. The comprehensive self-assessment report completed before the external review reflected on strategy and policy; investment in both hard and soft infrastructure; strategic partnerships; research, innovation and thought leadership; staff development; student learning resources and supports; implementation across faculties; staff and student perspectives; response to Covid-19; external engagement; and alignment to national and international standards and best practice. As noted by Professor Mark Brown, a meeting of the DCU Steering Group:

“Self-assessment of digital learning has been highly beneficial in terms of engaging a wider number of stakeholders in critical reflections and helping to better understand our strengths and weaknesses. It has helped to affirm DCU’s strategic approach to digital learning and provided a stronger evidence base for future decisions and investments.”

Finally, a copy of the full report, Developing a High Performance Digital Education Ecosystem: Institutional Self-assessment Instruments, can be downloaded from the EUA website. You can also register for a forthcoming webinar on Tuesday January 26th that will discuss the main findings of the report. A recording of this webinar will be available shortly after the event from the Digital-HE project website.

.

Two Months on… Reflections on DCU’s Teaching and Learning Week

It’s two months ago since the Teaching Enhancement Unit (TEU) at DCU hosted Teaching & Learning Week prior to the start of the new academic year. This event normally takes place over a full-day on a DCU campus and has become a real highlight in the annual calendar of university events. This year a decision was made quite early in light of ongoing Covid-19 challenges to host an online event.

T&L Week set us up for the new “hybrid” semester

While there have been many online professional development events since March a lot of time and planning was involved in trying to design something that would be engaging and model good practice in online learning for DCU staff. For this reason the event was intentionally spread over the week of 14th to 18th September.  Just to remind those of you with short memories, this week was just before further restrictions were introduced in Dublin and then more widely due to the pandemic.

Building on DCU’s adoption of an hybrid learning approach, the event was carefully designed to graft together the best of several approaches. There were showcases of good practice from across faculties and time for reflection to ensure that the DCU community could continue to share and engage in discussions about teaching, learning and assessment both for and during these extraordinary times.

Strong interest from DCU staff in the launch of T&L Week

The event started with a welcome message from DCU’s President and some valuable insights from award winning educators from this year’s President’s Awards for Excellence in Teaching & Learning, To engage staff in a topical issue a live workshop followed on the issue of “contract cheating”. Other lively interactive sessions included a discussion on “students as partners in assessment”, supported by podcasts and asynchronous online discussions which were centred on the theme of “Pedagogy in Practice: Teaching Excellence, In-class and Online.”

Live sessions augmented with short video nuggets

This mix of synchronous and asynchronous resources and activities in a hybrid format were offered through our Moodle instance, internally called Loop, supported by Zoom webinars and recordings which enabled staff to engage with the resources at times and places that best suited their busy schedules leading up to the new semester. A highlight over the week was the launch of the “Edge of Discovery” podcast series which attracted over 175 downloads by the end of the week. There are now 9 podcasts as part of this series with an increasing level of interest in listening to these short recordings with experienced DCU educators.

The dedicated Loop page had over 3,300 interactions by DCU staff during the week incorporating 415 unique visitors. Importantly, the online space facilitated learning to continue beyond the designated week with over 200 interactions occurring over the following week.

The continuing echo from the week even two months on highlights one of the benefits of a longer online event as opposed to our more traditional single-day, in-place, face to face event. Drawing on this experience and the benefit of our further reflections we see real value in taking lessons from this year’s Teaching and Learning Week to refine the design of future events across both formats to develop an even better hybrid model.

Given that plans for a return to campus changed shortly after Teaching & Learning Week, the TEU team was pleased to create an opportunity for so many DCU staff to connect with each other and engage with best practices around online pedagogy during these challenging times.  We hope the event helped many DCU staff to transition back to teaching online following DCU’s hybrid model with a few new ideas and suggestions to enhance the student learning experience.

Sincere thanks to all our contributors who made this year’s hybrid Teaching & Learning Week such a success:  Ann Marie Farrell, Martin Brown, Joanne Lynch, Lucien Waugh-Daly, Suzanne Stone, Orla Bourke, Roisin Lyons, Fiona O’ Riordan, Rob Lowney, and Orna Farrell.