By Clare Gormley & Fiona O’Riordan, DCU Teaching Enhancement Unit
The recent DCU Assessment Hackathon invited staff across the university to come together to hack the challenge of designing authentic and sustainable assessment. Frequently associated with digital innovation, a hackathon is an intense, time-bound event where people collaborate in teams to solve challenges (Briscoe and Mulligan, 2014). At hackathons participants work with their peers and a variety of stakeholders to develop solutions to real-world challenges of urgency and significance.
While DCU is no stranger to the hackathon approach – DCU Business School have been running large scale social innovation hackathons for years – this was the first time the Teaching Enhancement Unit (TEU) ran such an event specifically for the professional learning of teaching staff. Written by the event facilitators, this blog post will share some initial reflections on the experience and present initial feedback from some of those who took part.
A hackathon is a form of Challenge-based Learning or CBL for short. Because CBL is such a central pillar of the DCU Futures and ECIU projects, this event emerged from the need to offer practical and pedagogical support to staff who are interested in learning about and potentially applying the hackathon approach. Furthermore, because hackathons revolve around exploring solutions to significant issues, a focus on the challenges of assessment was deemed particularly relevant. The objectives of the event were therefore threefold:
- Provide a first-hand experience of a hackathon for academic staff interested in implementing a similar approach;
- Highlight the potential value and application of hackathons as an innovative and engaging learning experience;
- Facilitate an opportunity for participants to learn about designing authentic and sustainable assessments.
So it was that on Tuesday May 24th approx. 50 DCU staff members, supported by mentors and a local resource team, collaborated intensively over the course of a full day to hack the following ‘Big Idea’:
How can we design an authentic and sustainable assessment experience for all?
The event was the culmination of several months of planning by the TEU team. One of the first stages of the process was to identify mentors who would work alongside participants in thinking through their solutions. 11 mentors were selected from across the Irish and UK HE sector for their expertise in assessment and/or learning design. Cast primarily as critical friends, their role was to ask questions and act as a sounding board to support their team in teasing out a potential solution based on a challenge statement of their choice. The challenge statements emerged from ‘pitches’ which attendees were invited to submit about a week before the event.
The day itself was structured in accordance with a three phase CBL framework (Engage – Investigate – Act), the same overarching framework as that used by ECIU. The various stages of the process were introduced and kept on course by Frank van den Berg, our invited ‘CBL Ninja’, from the University of Twente. Attendees were kept busy (and well fed and watered!) throughout an intense day of discussion, negotiation, and activity. For the last hour or so, teams presented their proposed solutions. Following this the Judges were invited to give their feedback on the ideas proposed and identify the winning teams.
Throughout the day, snapshots of the hackathon activities were shared with the outside world through Twitter via #HackathonDCU. Indeed in the run up to the event, on the advice of our colleague Rob Lowney, video ‘stings’ were created as part of a social media plan to create interest and inform participants about what to expect.
The DCU Digital Communications team took care of photography and we liaised with local videographers to record various elements of the day.
Facilitator Perspective
Hackathons are increasingly being used as a basis for collaborative group projects in various educational domains (Lyons, Brown and Donlon, 2021; Kienzler and Fontanesi, 2017). Some higher education institutions (including DCU) are using them to foster transversal skills such as collaboration, critical thinking, problem-solving, and creativity in students (Lake, 2022).
As facilitators, we found that a joint approach to all planning and preparation made it possible for us to organise an event of this nature. Indeed we would go so far as to caution anyone against even trying to organise something like this alone. There are multiple stakeholders to be considered and consequently a surprisingly high number of administration tasks to be planned and tracked through meetings, lists, and frequent communication. Everything from organising room bookings and catering to liaising with mentors and developing judging criteria was documented.
Based on our impressions of the day and the feedback we received from the post event survey, some aspects of the hackathon seemed to work particularly well:
- The presence of student ambassadors provided a quick and well-informed source of information and feedback to teams ‘on the fly’
- The CBL framework and structure, scaffolded by the invited expert CBL Ninja, provided a logical pathway – and a sense of togetherness – through the day
- Being flexible and open to change on the day helped us deal with unanticipated issues (such as no shows) that inevitably crop up at the last minute
Amongst the items that did not go as well as hoped:
- Acoustics – the room was sometimes too noisy. While it was great to hear a lively hubbub of conversation, at times it was simply too loud for ease of communication across teams.
- The prizes and competitive element of ‘winning’ teams was perhaps unnecessary in retrospect, especially given the limited time – and it was an additional administrative task that took considerable effort to organise
- Reflection opportunities were extremely scarce and while hackathons are fast-paced by their nature, we didn’t feel that participants had the time to adequately reflect on the process whilst it was underway
Participant Perspective
Some of the above points were echoed by the participants. Asked what three things were most useful from the event, the most frequently cited benefit was the opportunity to discuss/collaborate on teaching and assessment approaches with colleagues from other disciplines. ‘Table discussions and recommendations by DCU and outside DCU colleagues’ were listed by participants as the most useful aspects of the event. The exchange of ideas about assessment with colleagues from different Schools/departments was mentioned several times, with a number stating how external mentors added value to this.
The next most important benefit was the opportunity to learn about CBL (the framework in particular) and the running of hackathons which several participants said would feed into how they design their own hackathons. Indeed several respondents welcomed the recommended emphasis on the learning in CBL, not the outcome, and revealed that that was new to their way of thinking.
The value placed on the input of the student ambassadors was another frequently-mentioned benefit. While some participants recognised that these were excellent students, and perhaps not representative of those ‘not-so-engaged’, the volume of comments commending the transformative input of the students was notable. As one participant summed it up:
‘The challenge was really a wicked challenge. The diversity of the team added to this challenge. The day was very well structured and there were clear instructions for the most part. The voices of the students on the day was paramount to the success. The constant access to food and drink was great as it was energy depleting partaking for the length of the day. I was wrecked afterwards.’
On the other hand, there were certain limitations recognised and felt by participants. A small number mentioned challenges with group dynamics and this would be intriguing to explore further. By far the most frequently mentioned limitation was the time constraint of attempting to come up with a meaningful solution in one day. All respondents recognised that time pressures were to be expected at a hackathon but that still didn’t resolve the difficulty – and their experience did influence their intention on whether or not to use one in their own context. Strongly associated with this constraint was the extremely limited time for reflection which several stated as a drawback to the hackathon approach. As one participant said:
‘I think that given the importance of reflection of learning in the process that there could be more time given to this element.’
Mentor Perspective
The presence of the student voice, the opportunity to observe a hackathon, and the structure/format of the event were reiterated again in the feedback from mentors. Reflecting a common theme, one of the mentors found the most useful aspects to be
‘Networking in person again post-Covid; the enthusiasm of the participants; brainstorming with strangers’
Based on the feedback, the mentors appeared to feel adequately prepared for the hackathon and the meetings with the ‘ninja’, other mentors and experts were deemed helpful. As one mentor expressed it: ‘It was a step into the unknown, and there was discomfort, but I felt supported.’
When asked to make recommendations for the role for future hackathons, it was indicated that there may be scope for preparing both participants and the mentors themselves for the expectations and boundaries of the role. While there were a couple of short regrouping pep talks at various points throughout the hackathon, perhaps more could have been done to set expectations across the board.
‘I am not sure that the participants around my table knew how to make use of my role.’
Another suggestion indicated that some additional time for icebreaking would not have gone amiss:
“I think it would be useful to introduce the mentors to the larger group, and give allocated time and/or a short activity where everyone at the table introduces themselves. I noticed that the group began discussing things right away. Providing some information about discipline/course numbers/contexts etc. would have been really useful, even just briefly.”
Issues with acoustics and lack of time for reflection were reiterated and the benefits of a longer format were repeated:
“It was hard not to have more thinking time including time to go away and come back in order to let ideas germinate. Also, the groups were just formed on the day; for longer term work I imagine the groups would get to know each other and that might contribute to more productive working”.
Where to next?
While the above shows feedback was generally very positive, there are certainly aspects that we will take on board (and pass on to others) to aid planning and optimal use of the hackathon approach. Some of the limitations are down to the hackathon format and illustrate how important it is for teams to refine the challenge to something achievable in one day. Perhaps more detail should have been added to the challenge statements to assist with that and perhaps the timeframe should be lengthened to more than one day to enable deeper development of potential assessment solutions. There are pros and cons to all of these changes, in our view, and no easy answers apparent.
But in terms of giving an experience of a real-life hackathon, showcasing its potential as an engaging format, and providing a space for learning about very different but topical approaches to assessment, we are satisfied that the objectives were met. While important not to overdo use of the format, this type of event seemed to work well as a form of professional development, in our view, and it’s something that others might consider. Some promising impacts include:
- At least two of the teams have met up independently to progress their assessment solutions
- One of the participants has professed to now ‘seeing hackathons everywhere’ and is planning to run at least one hackathon with colleagues over the coming year
- Lessons learned by participants should quickly feed in to more considered design and application of hackathons within innovative curricula right across DCU
References
Kienzler, H. and Fontanesi, C. (2017) ‘Learning through inquiry: a Global Health Hackathon’, Teaching in Higher Education, 22(2), pp. 129–142. Available at: https://doi.org/10.1080/13562517.2016.1221805.
Lyons, R., Brown, M., & Donlon, E. (2021) ‘Moving the hackathon online: Reimagining pedagogy for the digital age’, Distance Education in China, 7, [English Version].
Digital Learning Dispatches 