“Out of Intense Complexities, Intense Simplicities Emerge”: Assessment and the Pivot Online

By Dr Orna Farrell

On the 20th of April, I gave a presentation at a webinar organised by European Distance and Elearning Network (EDEN) about how to design and manage assessments for online learning which was part of a webinar series called Education in Time of a Pandemic. The webinar contributors included Prof. Mark Brown from DCU’s National Institute for Digital Learning, Dr. Lisa Marie Blaschke from the University of Oldenburg and Dr. Alfredo Soeiro from the University of Porto. I started my presentation with a quotation from Winston Churchill:

“Out of intense complexities, intense simplicities emerge”.

I felt this quote was appropriate for the current educational context because although it is a very uncertain time, there could be potential to reimagine our approaches to assessment. For example, we could take this opportunity to clear out some of the deadwood or kill a few sacred cows in the educational system. The sacred cow I focused on was the campus-based examination system.

a27d24_44d4cfc1080b4edd8870dcc661db11c2~mv2
Photo by Biel Morro on Unsplash

In the literature on assessment, exams are considered to be a poor measure of student learning which emphasizes knowledge reproduction rather than critical thinking. For example:

“they tend to measure lower order thinking skills in a decontextualized manner at a time when the literature frequently argues for the benefits of a richer, authentic approach to assessment” (Villarroel, Boud, Bloxham, Bruna, 2020, p. 38)

If the literature indicates that richer authentic approaches to assessment can benefit student learning, why are higher education institutions so attached to exams? Face-to-face campus-based exams are surrounded by ritual, bureaucracy and awe in higher education. I think we should take advantage of the opportunity that the pivot online created by the Coronavirus pandemic and kill the sacred cow of closed book campus based exams permanently.

Replicating campus-based exams online through timed proctored online exams is not the solution. In her recent presentation at #Gastagoesglobal, Sheila McNeill talked about how online proctored exams promote a culture of distrust and surveillance and how we should instead be creating a culture of support, trust and development for our students.

Student Engagement and Covid-19

When designing alternative emergency assessments for our students we need to think about student engagement in the context of Covid-19 pandemic. Adapting Kahu’s (2013) model of student engagement to encompass the Covid19 pandemic context is a useful lens to help us understand how our students can engage and learn successfully during this crisis.

a27d24_d49dfe8ee2ac45e3b8ce52b1e1414a18~mv2.png

Important factors that we should consider are lifeload, course and assessment design, access to adequate broadband and computing equipment, the availability and accessibility of institutional remote online supports. Kahu (2013, p. 767) describes lifeload as “the sum of all the pressures a student has in their life, including university”, and is seen as being a critical factor influencing student engagement. Everyone including students and staff are experiencing increased lifeload pressures due to the pandemic, such as illness, caring responsibilities, home schooling, and remote working. Time management and organisational skills are key skills for student success (Farrell & Brunton, 2020). In addition, students are experiencing difficulties with broadband, access to computers, finding quiet study space and sufficient time to study. These pressures are impacting on our students well-being, their time, and their ability to learn successfully.

Our Approach to Crisis Assessment

In Dublin City University (DCU), the institution identified four key principles for crisis assessment:

  1. Validity

  2. Equivalence

  3. Proportionality

  4. Academic Integrity

These principles were applied across the University coupled with coupled advice for choosing alternative assessments to adapt exams into more appropriate alternative assessments. In our DCU Connected Humanities programmes, for example, which are modular online degrees we applied these four principles and adapted our exams into openbook take home assessments. Openbook take home assessments adhered to these principles and were a flexible, low bandwidth asynchronous assessment approach.

Alternative Assessment Ideas

In the webinar, we discussed a number of alternative assessment ideas, such as:

    • Eportfolio

    • Collaborative wiki

    • Asynchronous online debate

    • Vlog

    • Blog

    • Podcast

    • Online presentation

    • Asynchronous group project-Gdocs

    • Video creation

    • Reflective journal

    • Peer assessment

My colleagues from the NIDL, Suzanne Stone and Rob Lowney have created an edited open access resource called Exemplars and Case Studies of Technology Enhanced Assessment in HE and FE which has more details on these alternative assessment ideas. For more resources on alternative assessment and the pivot online, check out the NIDL Teaching Online Resource Bank.

Academic Integrity

The issue of academic integrity came up frequently in the discussion at the webinar. Our approach to academic integrity involves creative design of authentic assessment, moderation of marking, text matching software, clear guidelines to students about our expectations around referencing and the use of vivas to verify student academic work. These are detailed further in the resource below, and our Academic Integrity Self-Assessment Checklist.

a27d24_c6e451e614ab4275a45c3d4ccb482688~mv2.jpg

DCU (2018) Academic Integrity Principles

Final thoughts

In these unprecedented times, it is important that we support the well-being of our students, while addressing the need to have quality assessment. In the context of student engagement and Covid-19, using low bandwidth, flexible and asynchronous assessments may enable our students to succeed at completing their studies and #keeplearning.

References

DCU Teaching Enhancement Unit (2018). Academic Integrity Principles. Retrieved from Academic Integrity for Quality Teaching and Learning in Higher Education (INTEGRITY Project): https://teuintegrityproject.wordpress.com/

EDEN Webinar resources and recording https://www.eden-online.org/how-to-design-and-manage-assessments-for-online-learning/

Farrell, O. (2020) “Out of intense complexities, intense simplicities emerge.”: Assessment and the pivot online. Presentation at EDEN NAP Webinar.

Farrell, O., Brunton, J. (2020). A balancing act: a window into online student engagement experiences. Int J Educ Technol High Educ 17, 25. https://doi.org/10.1186/s41239-020-00199-x

Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38(5), 758–773. https://doi.org/10.1080/03075079.2011.598505.

NIDL Resource bank https://www.dcu.ie/nidl/resources/Swiftly-Moving-Online-Coronavirus.shtml#Assessment

Villarroel, V., Boud, D., Bloxham, S., Bruna, D., & Bruna, C. (2020;2019). Using principles of authentic assessment to redesign written examinations and tests. Innovations in Education and Teaching International, 57(1), 38-49. doi:10.1080/14703297.2018.1564882

Stone, S. Lowney, R. (2020). Exemplars and Case Studies of Technology Enhanced Assessment in HE and FE

Note: This blog post originally appeared as a guest commentary on this Portuguese site established to support teaching online in response to the Covid-19 crisis.

A Rabbit Caught in the Headlights

Ken’s successful journey with DCU Connected…

It was Saturday October 5th 2013 when I first visited the Dublin City University (DCU) campus in Glasnevin, Dublin. Strange as it may sound it was only on my third visit in March 2015 that I received my parchment for the degree I completed online through DCU Connected.  Thanks to the Irish Government’s Springboard+ scheme and the help from DCU I received along the way.  I had promised myself for a long time that I would go back and complete my studies someday, but time and money conspired against me to keep that from happening.  With an unwanted redundancy I fixed the time problem and thanks to Springboard+ I did not have to worry about the cost.

0.jpeg

That first day I was not at all sure what I was setting myself up for or the journey that I was about to embark on.  Feeling like the proverbial rabbit stuck in the headlights I wondered was this “online learning” really something that I could do or was I really just setting myself up to fail.  Can I do this?  Will I have the self-discipline required? How will I manage my time and combine a demanding study schedule with the rest of my life?  So many questions, but feeling like I had so few answers.Thankfully I had plenty of fellow students making the same journey and the help and assistance from my tutors and the DCU Connected support staff was always on hand. Clear expectations were set from the start and a very detailed timetable of assignments, notes, and online tutorials outlined exactly what I had to do and when it was that I had to do it.  The online tutorials provided clear guidance on the standard that was expected in our assignments and offered us the opportunity to ask questions and get answers.  The online forums in Loop, DCU’s virtual learning environment, were also hugely beneficial in this regard. The notes provided were excellent and very clearly laid out with lots of useful references and additional reading.

Online study has the potential to be lonely; however, I was glad to have new friends that I made on that initial day in DCU as we were able to offer support and guidance to each other along the way.  The nature of the assignments was such that there was almost always some group or team component and that provided an opportunity to interact with other students also.  Sharing the work but also sharing stories of our own experiences meant that you never felt overly isolated.  All of the assignments also provided the opportunity to share a part of your work with your peers and to both give and receive peer feedback in the form of an online critique and this provided a yardstick of sorts to gauge your own progress through the programme.

The real measure however was provided by the tutors.  For each and every assignment we received timely and comprehensive feedback, with really useful guidance on what could be done to improve the work for future assignments and what the strengths and weaknesses of the assignment were.  Seeing the list of completed assignments grow and the list of remaining assignments shrink provided the motivation to stick the course and complete the programme. It was hard, but it was worthwhile and I have no regrets that I made that trip to DCU in 2013. Ken tells his story here…

At Dublin City University through DCU Connected and with the support of Springboard+ opportunities are available for people to study at no cost or 90% subsidised fees towards graduate certificates in:

• Management of Internet Enterprise Systems

• Management of Operations

• Management of Information Systems Strategy

These programmes are designed so that students can study part-time whilst they work. Students who start with the above qualifications can progress to complete a full Masters degree. DCU has a long history of online distance learning and through the National Institute for Digital Learning (NIDL) has considerable expertise in designing online courses. Notably in November DCU will be hosting the ICDE World Conference on Online Learning.

DCU Connected also provides a range of undergraduate programmes through the Springboard+ scheme. If you are interested in finding out more, please see our website or contact a member of our team:

openeducation@dcu.ie

0.png

0-1.jpeg