“Out of Intense Complexities, Intense Simplicities Emerge”: Assessment and the Pivot Online

By Dr Orna Farrell

On the 20th of April, I gave a presentation at a webinar organised by European Distance and Elearning Network (EDEN) about how to design and manage assessments for online learning which was part of a webinar series called Education in Time of a Pandemic. The webinar contributors included Prof. Mark Brown from DCU’s National Institute for Digital Learning, Dr. Lisa Marie Blaschke from the University of Oldenburg and Dr. Alfredo Soeiro from the University of Porto. I started my presentation with a quotation from Winston Churchill:

“Out of intense complexities, intense simplicities emerge”.

I felt this quote was appropriate for the current educational context because although it is a very uncertain time, there could be potential to reimagine our approaches to assessment. For example, we could take this opportunity to clear out some of the deadwood or kill a few sacred cows in the educational system. The sacred cow I focused on was the campus-based examination system.

a27d24_44d4cfc1080b4edd8870dcc661db11c2~mv2
Photo by Biel Morro on Unsplash

In the literature on assessment, exams are considered to be a poor measure of student learning which emphasizes knowledge reproduction rather than critical thinking. For example:

“they tend to measure lower order thinking skills in a decontextualized manner at a time when the literature frequently argues for the benefits of a richer, authentic approach to assessment” (Villarroel, Boud, Bloxham, Bruna, 2020, p. 38)

If the literature indicates that richer authentic approaches to assessment can benefit student learning, why are higher education institutions so attached to exams? Face-to-face campus-based exams are surrounded by ritual, bureaucracy and awe in higher education. I think we should take advantage of the opportunity that the pivot online created by the Coronavirus pandemic and kill the sacred cow of closed book campus based exams permanently.

Replicating campus-based exams online through timed proctored online exams is not the solution. In her recent presentation at #Gastagoesglobal, Sheila McNeill talked about how online proctored exams promote a culture of distrust and surveillance and how we should instead be creating a culture of support, trust and development for our students.

Student Engagement and Covid-19

When designing alternative emergency assessments for our students we need to think about student engagement in the context of Covid-19 pandemic. Adapting Kahu’s (2013) model of student engagement to encompass the Covid19 pandemic context is a useful lens to help us understand how our students can engage and learn successfully during this crisis.

a27d24_d49dfe8ee2ac45e3b8ce52b1e1414a18~mv2.png

Important factors that we should consider are lifeload, course and assessment design, access to adequate broadband and computing equipment, the availability and accessibility of institutional remote online supports. Kahu (2013, p. 767) describes lifeload as “the sum of all the pressures a student has in their life, including university”, and is seen as being a critical factor influencing student engagement. Everyone including students and staff are experiencing increased lifeload pressures due to the pandemic, such as illness, caring responsibilities, home schooling, and remote working. Time management and organisational skills are key skills for student success (Farrell & Brunton, 2020). In addition, students are experiencing difficulties with broadband, access to computers, finding quiet study space and sufficient time to study. These pressures are impacting on our students well-being, their time, and their ability to learn successfully.

Our Approach to Crisis Assessment

In Dublin City University (DCU), the institution identified four key principles for crisis assessment:

  1. Validity

  2. Equivalence

  3. Proportionality

  4. Academic Integrity

These principles were applied across the University coupled with coupled advice for choosing alternative assessments to adapt exams into more appropriate alternative assessments. In our DCU Connected Humanities programmes, for example, which are modular online degrees we applied these four principles and adapted our exams into openbook take home assessments. Openbook take home assessments adhered to these principles and were a flexible, low bandwidth asynchronous assessment approach.

Alternative Assessment Ideas

In the webinar, we discussed a number of alternative assessment ideas, such as:

    • Eportfolio

    • Collaborative wiki

    • Asynchronous online debate

    • Vlog

    • Blog

    • Podcast

    • Online presentation

    • Asynchronous group project-Gdocs

    • Video creation

    • Reflective journal

    • Peer assessment

My colleagues from the NIDL, Suzanne Stone and Rob Lowney have created an edited open access resource called Exemplars and Case Studies of Technology Enhanced Assessment in HE and FE which has more details on these alternative assessment ideas. For more resources on alternative assessment and the pivot online, check out the NIDL Teaching Online Resource Bank.

Academic Integrity

The issue of academic integrity came up frequently in the discussion at the webinar. Our approach to academic integrity involves creative design of authentic assessment, moderation of marking, text matching software, clear guidelines to students about our expectations around referencing and the use of vivas to verify student academic work. These are detailed further in the resource below, and our Academic Integrity Self-Assessment Checklist.

a27d24_c6e451e614ab4275a45c3d4ccb482688~mv2.jpg

DCU (2018) Academic Integrity Principles

Final thoughts

In these unprecedented times, it is important that we support the well-being of our students, while addressing the need to have quality assessment. In the context of student engagement and Covid-19, using low bandwidth, flexible and asynchronous assessments may enable our students to succeed at completing their studies and #keeplearning.

References

DCU Teaching Enhancement Unit (2018). Academic Integrity Principles. Retrieved from Academic Integrity for Quality Teaching and Learning in Higher Education (INTEGRITY Project): https://teuintegrityproject.wordpress.com/

EDEN Webinar resources and recording https://www.eden-online.org/how-to-design-and-manage-assessments-for-online-learning/

Farrell, O. (2020) “Out of intense complexities, intense simplicities emerge.”: Assessment and the pivot online. Presentation at EDEN NAP Webinar.

Farrell, O., Brunton, J. (2020). A balancing act: a window into online student engagement experiences. Int J Educ Technol High Educ 17, 25. https://doi.org/10.1186/s41239-020-00199-x

Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38(5), 758–773. https://doi.org/10.1080/03075079.2011.598505.

NIDL Resource bank https://www.dcu.ie/nidl/resources/Swiftly-Moving-Online-Coronavirus.shtml#Assessment

Villarroel, V., Boud, D., Bloxham, S., Bruna, D., & Bruna, C. (2020;2019). Using principles of authentic assessment to redesign written examinations and tests. Innovations in Education and Teaching International, 57(1), 38-49. doi:10.1080/14703297.2018.1564882

Stone, S. Lowney, R. (2020). Exemplars and Case Studies of Technology Enhanced Assessment in HE and FE

Note: This blog post originally appeared as a guest commentary on this Portuguese site established to support teaching online in response to the Covid-19 crisis.

Schools Out: Diary on an Incredible Week that Changed Irish Education

A lot can happen in a week! This time last week we were still coming to terms with the dramatic news from the previous day that all schools and college campuses across Ireland were to close at 6:00pm on Thursday March 12th. At the time few people could have predicted the impact of this decision as the Government adopted a serious response to the Covid-19 pandemic. Taking a little time out, this post briefly reflects on events of the past week and in particular the incredible response from Irish educators. Drawing on DCU’s response it offers a diary of insights and selected highlights from the experiences of the NIDL team.

DDD.jpg

Our preparations in the NIDL for the prospect of restricted campus access began in earnest on Sunday, March 8th, when we first produced our “Keep Teaching” guidelines for DCU staff. On Tuesday, March 10th, we produced a similar “Keep Learning” guide for DCU students. Over the next few days we developed and facilitated a series of additional online professional development events for staff, including two workshops on “The ABC of Teaching Online” and several “Using Zoom for Teaching” sessions. There was a large response to these workshops and no doubt they contributed, coupled with our considerable experience of online education through DCU Connected, to the relative smooth pivot on Monday, March 16th, to fully teaching online.

Unknown.jpegAlthough we developed a plan as early as the previous Monday to increase our frontline services to cope with a significant increase in demand for online teaching support, by the end of Monday, March 16th, only 5 ticket requests were waiting to be resolved. And as the week continued our Loop support team managed with a huge effort to stay largely on top of requests for assistance. By close of business of Friday, therefore, only 14 tickets remained to be opened and most of these are likely to be responded to by the end of the evening. A big thanks goes to the commitment of this small team who notably received a personal message of thanks from DCU’s President.

111But the real story and success of the first full week of teaching fully online at DCU needs to be attributed to the incredible efforts of those who teach across all of our five faculties. There are some amazing stories that will hopefully get told when the time is right. For example, Loop was being used in particularly creative ways and on Monday alone there were over 700 separate live sessions in Zoom. This number had doubled by the end of the week with over 1,400 separate live events on Friday. The incredible response from educators in the face of such challenging and unprecedented circumstances appears to be a standout theme right across Ireland. Perhaps we only really learn about ourselves, and the values we hold, in times of adversity.

The NIDL team has also spoken throughout the week about our wider responsibility to the Irish education community. On Saturday March 14th, we launched the first version of a dedicated webpage designed to support educators moving to teaching online in a hurry. Our “Swiftly Moving to Teaching Online: Resource Bank” now contains a wealth of resources, including quality checklists, tips for online teaching, suggestions for alternative assessments and student learning guides. This resource continues to be updated on a daily basis.

padlet_wall.jpg

On Monday, March 16th, we hosted in partnership with the European Association for Distance Teaching Universities (EADTU) and the European Distance and E-Learning Network (EDEN) a webinar on swiftly moving to teaching online which attracted several hundred participants.

IMG_6718 Coincidently, a team in the NIDL was already scheduled to launch a free online course on Teaching Online as part of the #OpenTeach project. This course developed with funding from the National Forum for the Enhancement of Teaching and Learning in Higher Education starts this coming Monday with over 450 participants. However, over the last week to help meet the demand for resources on teaching online the team shared a number of video lessons from the keyface on relevant topics. The full suite of videos can be found on the Resource Bank along with a number of other useful guides on getting started with teaching online.

fl_course.jpg

Also, on Monday, we first engaged with FutureLearn about the development of a new free online course for educators affected by Covid-19. After a rapid design and development process, this course was ready for its official launch on Thursday March 19th and in the first 24 hours attracted over 2000 participants from 125 countries. At the time of writing over 150 Irish educators have registered for the course. The FutureLearn course, ‘How to Teach Online: Providing Continuity for Students’ aims at providing evidence-based, just-enough, just-in-time and just-in-case practical learning for educators new to online teaching.

mark_head_2015_small4.jpgAs part of the course and DCU’s support for this initiative, Professor Mark Brown was asked to provide his responses to 7 key questions about teaching online. Although there have been many lists of top tips for teaching online published over the past week, we share Mark’s response to these questions below as another small contribution to supporting the education community at this challenging time.

7 Key Questions About Teaching Online

  1. What three key considerations should educators make when moving from face to face to online?

Firstly, take time to decide what type of online teacher you want to be. Don’t leave this to chance. Draw on your educational philosophy to identify a term or metaphor that best encapsulates how you would want your students to describe your approach to online teaching. For example, are you going to be a manager, guide, coach, facilitator, gardener, time traveller or something else more akin to your teaching philosophy?

Secondly, think about your online learners. Who are they? What needs are they likely to have? What skills and knowledge gaps are they likely to have? What prior experiences do they bring and how can you draw on them as a valuable online learning resource? Ask yourself, if I was doing this course as an online learner why would I be excited and even delighted to engage in the variety of learning experiences?

Thirdly, be creative in how you intentionally design for learning. What innovative online learning experiences can you embed in your course that might not be possible in a traditional classroom? With the luxury of more time avoid merely replacing one traditional method or technology with another in your online environment without any clear transformative advantage. Remember if you want to develop creative, innovative and highly imaginative learners, then we need teachers willing to apply these dispositions or “habits of mind” in the design of their online courses.

  1. What’s the hardest lesson you’ve learnt when teaching online?

Adding too much “stuff” to my online course. Although with the best intentions of helping my students by giving them access to lots of links and additional resources, I discovered that inadvertently I was actually increasing their workload. Indeed, I was even placing their success at risk as a clear flag was needed to differentiate between essential and non-essential information.

  1. What practical steps can set your learners up for success for learning online?

Make sure they have enough time. Set clear expectations in terms of the amount of time required for both directed and self-directed learning activities each week. Ensure your students understand whether they have enough time to continue their online study on top of existing commitments, particularly in light of changing circumstances. To this end, invite your learners to complete this “Do I Have Enough Time” calculator. And plan for the unfortunate fact that other life commitments may get in their way, including getting sick. So be flexible and show compassion to your learners.

  1. What’s the biggest wrong assumption you’ve made when teaching online?

When I first started teaching online over 20-years ago I assumed that all you needed to do was put your lecture notes on the internet and open a bulletin board for discussion. With the benefit of hindsight, it was hardly surprising that very few students responded, posted comments or even asked questions. What I discovered is that you have to design for interaction and help your students learn how to ask the right questions. I would even go so far as to say more often than not the question is the answer!

  1. What’s your best time saving tip for teachers?

Don’t waste your time creating lots of new content as your students are one of your best resources to help locate and co-construct the knowledge required to meet your learning intentions. In many respects, you are better to think of your students as the co-authors rather than the audience of your content. More to the point, embrace emptiness! For example, if you really want to be creative, then consider developing an asynchronous online role play where students drive interactivity in their assigned roles over a defined period, as they debate and try to resolve an authentic problem.

  1. What’s your best time saving tip for students?

Don’t wait until you think you will have a block of long, uninterrupted, quiet, rested perfect time. You may be procrastinating. You do need some longer spaces of time but do also use shorter intervals when you can. Keep your train of thought going by refocusing on your learning in smaller slices of time. During that break, that queue, journey, or any other short space of free time you have. Develop new habits of how you use those in-between moments. Keep your focus on your learning, and remember why you are doing it and what your goal is.

  1. What’s the one thing you’d recommend to educators new to teaching online?

Design your course as if you’re a learner. Think very carefully about the workload and new technical demands you’re placing on your students. From this learner’s perspective also consider the inherent value proposition in different types of activities you ask them to complete. When you’re done ask yourself: Is this a great looking course that I you would really like to take? Will it really grab my learners? While the best online courses have an X factor the main thing is your willingness to learn, innovate and remain open to trying new ideas. Hence, good online education begins with having the right mindset rather than depending on a strong technical skillset.

If you would like to hear more from Mark and other experienced online educators, or share your own lessons with people keen to learn, then we highly recommend that you register for the FutureLearn course.

CCCC.jpgFinally, as recently as November 2019, when the National Institute for Digital Learning (NIDL) at DCU hosted last year’s World Conference on Online Learning, few could have predicted in the space of just a few months that online education would become so crucial to the continuity of Irish education. Arguably, online education will never be the same. However, responding to a crisis is one thing but the challenge over the weeks ahead, and beyond, is to better understand and embrace the opportunities that new digital technologies offer for a more transformative model of education which prepares students for tomorrow’s world, today.

img_6709.jpg