Top 10 Articles – Digital Learning Dispatches

Digital Learning, Learning Design, Learning Futures, Online Learning, Publications, Research, Teaching

Top 10 Good Reads from 2022: From Theory to Practice and Back Again

Selecting our annual top 10 open-access journal articles after reviewing over 100 nominated papers for consideration is never easy and last year was no exception. Within our defined selection criteria, it’s always difficult to judge what counts as a “good read”, as this depends on your interests and perspectives. In selecting our 7th annual list of “good reads”, we are also conscious of giving profiles to some articles that appear in lesser-known publications, as they may be outside your normal reading in the digital learning area.

This point is emphasised by Macgilchrist, Potter and Williamson (2022) in their excellent article, which asks…

Who are we reading and who are we citing?

Their analysis shows some of the biases in our publication and citation practices by revealing an extensive body of literature in developing countries and/or typically outside of the better-known journals published in the English language. Whether you realise it or not, what you choose to read is a political decision or practice, which says much about your orientation to the literature. What we have attempted to do in selecting our 2022 list of “good reads” is balance the inclusion of several seminal works with those you may not have previously seen through your normal reading channels.

About this Year’s Selections

As in previous years, deciding on the order of the top 10 articles is contentious. Except for our top article, which discusses the entangled nature of pedagogy, the selection order largely reflects the narrative we develop through our brief curated remarks on each work rather than implying one article is measurably better than another. The narrative and order of articles begin with a focus on definitions and conceptual frameworks, with the first few papers addressing different ways of theorising and conceptualising the ecosystem in which digital learning inhabits.

Several of the articles help to illustrate the complexity of what might fall under the umbrella of digital learning and serve to demonstrate the importance of criticality and the role of theorical constructs and conceptual frameworks in shaping our thinking—for better and worse.

We have also included a major review article looking at the framing of learning design versus instructional design, as reflected in the literature. This article continues the focus on the importance of the explicit or implicit theories and language we use to express the nature of our work.

At this point, we pivot to a series of articles with a future-focused perspective. The first of these articles provides a thoughtful piece on the emergence of speculative education fiction as a methodology in helping to frame hopeful (and less optimistic) learning futures. This article provides a bridge to the final three papers that focus on new and emerging technological developments in digital learning.

As you will see from these articles, our selections for 2022 were influenced to some extent by the considerable interest generated late in the year by the launch of ChatGPT-3. We have chosen two major literature reviews on the use of Artificial Intelligence (AI) for teaching, learning and assessment that help to categorise and synthesize the emerging literature in this major development area. If you have a particular interest in AI in higher education, we suggest you also read from last year the systematic review of the top 50 most-cited articles written by Chu et al. (2022) and the cautionary notes Neil Selwyn (2022) offers on future developments in the area. The final article continues a futures-focus theme by providing a literature review that asks whether the Metaverse is a blessing or a curse. In many respects, the answer to this question requires readers to circle back to the entanglement thesis contained in our top article for the year.

Celebrating Open Access

We would like to acknowledge all the authors who appear in our selections and many others whose articles we reviewed in this process. This annual exercise for us is hugely valuable in taking stock of the literature and requires us to go beyond the skim reading we typically do when publications are first released over the course of the year. The decision people make to publish their work in an open-access journal, despite, in many cases, strong institutional pressures to submit manuscripts to more prestigious and higher-ranked closed publications, is a moral and political decision that we aim to strongly endorse through this exercise.

By the Numbers

A quick analysis of the top 10 “good reads” from 2022 reveals that the articles come from nine journals and collectively recognise 44 authors. This is the highest number of individual journals and authors since we began this exercise seven years ago. As Table 1 shows, three articles were written by single male authors, two by co-authors and the remaining five by multiple authors, but it’s not possible, based on names alone, to provide an accurate breakdown by gender across all 10 articles and 44 authors. However, this type of binary analysis has also become increasingly problematic.

One author, Aras Bozkurt, appears twice, which is not the first time he has been featured in our selections. Indeed, Bozhurt was a co-author of three articles in our list of “good reads” for 2021. There are 14 countries represented with seven UK-based authors, five from Finland, four from Canada, the United States and Turkey, two from France and South Africa, with the remaining authors distributed across Ireland, Spain, Palestine, China, India, Korea and Pakistan.

Thus, the “good reads” from 2022 represent a wide geographical spread of authors, although none are from Latin, South America, and Australasia. As Table 2 shows, this is the first time that the Australasian Journal of Educational Technology does not appear on the list, and notably, neither does the highly ranked International Journal of Educational Technology in Higher Education and the International Review of Research in Open and Distributed Learning feature. We believe their omission partly reflects our commitment to identifying open-access articles outside the better-known journals. Table 3 shows that for the first time the top article was published in Postdigital Science and Education, which is the only journal that contributes two paper.

The Top 10 Good Reads

Here are this year’s top 10 “good reads” with brief curated remarks to help explain why each article was selected…

No 1

Fawns, T. (2022). An entangled pedagogy: Looking beyond the pedagogy—technology dichotomy. Postdigital Science and Education, https://doi.org/10.1007/s42438-022-00302-7

This article challenges the validity of a popular slogan that has emerged in recent years—‘Pedagogy first’. It demonstrates why this is a naïve and potentially dangerous mantra to promote as it falls into the trap of pedagogical determinism and fails to recognise the entangled nature of the complex constellation of change forces influencing the current focus on digital learning. The call for ‘pedagogy first’ implies or can be misinterpreted to suggest that technology is just a tool. In this regard, the paper shows that new digital technology influences us, our institutions, and our communities as much as we can influence how technology is deployed for educational purposes. Importantly, following this point, the work also rejects deterministic language claiming that new digital technology is a force driving educational change independent from society. Put simply, the central thesis is woven around the entangled relationship between technology and pedagogy. It responds to binary discourses that do little to help us understand the reality of complexity, uncertainty, and imperfection. In looking beyond the pedagogy—technology dichotomy, an aspirational view is offered of how teachers, students and other stakeholders can engage with collective agency underpinned by values and ethics. The paper makes a valuable contribution to the literature as it demonstrates the complex interdependencies which are a reality of the many and varied educational contexts in which we work.

No 2

Johnson, N., Seaman, J., & Poulin, R. (2022). Defining different modes of learning: Resolving confusion and contention through consensus. Online Learning Journal, 26(3), 91-110. DOI: http://dx.doi.org/10.24059/olj.v26i3.3565

This article contributes to the challenge of defining what constitutes ‘digital learning’ and the many variants. The issue of definitions is not new, with a long history of contentious terms and ill-defined concepts under the historical umbrella of Educational Technology. However, the question of nomenclature has become even more topical following the COVID-19 crisis, with a renewed focus on hybrid forms of teaching and learning. The study reported in this article helps to identify how different teaching modalities and the associated terms we use to describe them largely fall into two big buckets. Importantly, these buckets represent the location in which learning primarily takes place: distance or in-person. While the proposed Modes of Learning Spectrum is unlikely to put an end to definition wars, as ‘distance learning’ is unfortunately associated with the deficit language of ‘remote learning’, and some educators might like to argue all learning is ‘in-person’ even when occurring virtually, the framework provides useful a tool for thinking about different learning experiences by mode. It also highlights the importance of being intentional about the language we use and the meanings we adopt at an institutional and sector-wide level.

No 3

Passey, D. (2022). Theories, theoretical and conceptual frameworks, models and constructs: Limiting research outcomes through misconceptions and misunderstanding. Studies in Technology Enhanced Learning, 1(1), 95-114. https://doi.org/ 10.21428/8c225f6e.56810a1a

This article is particularly relevant to research and doctoral students. The underlying premise is that any research study needs to be firmly anchored within one or more theoretical or conceptual frameworks. However, this is easier said than done and is often “considered a ‘doctoral or research challenge’ in itself” (p. 95). Importantly, the explicit or underlying models, theories or frameworks are crucial in determining how a study might (or might not) contribute to new knowledge. Framed by this assumption, the paper offers several examples of theoretical constructs in the field. It provides a valuable analysis of their roles in drawing on the literature, shaping our perspectives and contributing to new theories and understandings. Several useful recommendations are provided to ensure criticality is paramount when selecting, interpreting, developing, and applying such models, constructs, and frameworks throughout the research process. While the paper is primarily written for doctoral students or those relatively new to undertaking research, it is also valuable reading for more experienced researchers. After all, Bond et al. (2021) showed that many articles and reports published during the COVID-19 crisis lacked a deeper theoretical underpinning and fit the description of emergency remote research.

No 4

Atenas, J., Beetham, H., Bell, F., Cronin, C., Vu Henry, J., & Walji, S. (2022). Feminisms, technologies and learning: Continuities and contestations. Learning, Media and Technology, 47(1), 1-10, DOI: 10.1080/17439884.2022.2041830

This challenging article extends the call for criticality and serves as an editorial for a special issue providing a feminist perspective on learning technology. It speaks to some of the biases and marginalised voices in the literature and across theory, research, and practice. The Editorial begins by illustrating through several previous articles published almost three decades ago how concerns about equity, diversity, and inclusion (EDI) are nothing new in the literature. Indeed, the authors further argue that the gaps and structural issues have become even more hardwired as “…gender and other inequalities are encoded into the technologies we routinely use for learning and everyday life” (Eynon, 2018; cited in Atenas et al., 2022). In (re)exploring the territory and intersections between pedagogy, digital technology, and feminism, the authors anchor digital learning in wider social practice. A notable feature of the paper is a meta-discussion around the open process of editing the journal, followed by insightful commentary illustrating how feminist theory is a diverse repertoire of methods involving critique and interventions. They share a common commitment to activism and critical action for justice. Importantly, the editorial points out that feminists have not ceased to innovate and maintain their hope and resilience, as evidenced by the collection of works in the special issue.

No 5

Downes, S. (2022). Connectivism. Asian Journal of Distance Education, 17(1), 58-87. http://www.asianjde.com/ojs/index.php/AsianJDE/article/view/623

This article takes the award of having the shortest title. It provides a comprehensive overview of Connectivism, one of the most influential theoretical constructs over the previous two decades, influencing thinking about learning and how we understand knowledge in today’s digitally connected world. The central thesis of Connectivism, according to the paper, is “that knowledge is constituted of the sets of connections between entities, such that a change in one entity may result in a change in the other entity, and that learning is the growth, development, modification or strengthening of those connections” (p. 58). There has been considerable debate over the years about whether Connectivism is a new theory of learning, and the article addresses this point. It argues that this is a fundamental misunderstanding of Connectivism even though it does provide an account of learning in the digital age. At the core of the thesis is the crucial role networks and connections play in the learning process, which contribute to knowledge. Importantly, the author distinguishes his interpretation of learning from that adopted or implied by George Siemens‘ (2004) in his original seminal paper on Connectivism. It also outlines how Connectivism differs or can be distinguished from other learning theories. Close attention needs to be given to the following theoretical discussion, which describes how learning occurs. Several practical examples help to illustrate key points, concepts, and principles for the design of networks, our understanding of pedagogy and for engaging in connectivist forms of learning. In this respect, the discussion challenges what we currently measure as learning and has wider implications for how we live and learn in an interconnected world, especially as the Internet’s architecture evolves.

No 6

Saçak, B., Bozkurt, A., & Wagner, E. (2022). Learning design versus instructional design: A bibliometric study through data visualization approaches. Education Sciences, 12, 752, 1-14. https://doi.org/10.3390/educsci12110752

This article responds to the fact that ‘instructional design’ and ‘learning design’ are common terms used to describe a discipline concerned with enhancing the design of teaching, learning and assessment. While the terms share a common goal, they are not always interchangeable. Indeed, they often reflect a different underlining philosophy or approach to designing, implementing, and evaluating effective pedagogies, resources, and environments for learning. Mindful of this point, the study seeks to better understand the similarities and differences between the terms and how they have evolved by mapping the intersections and common themes using text mining and social network analysis. More specifically, a triangulated bibliometric study is reported, which analyses 514 publications (326 for instructional design and 157 for learning design) indexed in the Scopus database. In the case of instructional design, the paper reports four broad themes: Theory-driven approaches; technology-informed designs; instructional design for higher education; and assessment and evaluation. For learning design, four major themes emerged: Design thinking and user experience-driven approaches; online learning informed designs and online environments; analytical approaches for assessment and evaluation; and engagement-based learning design. In comparing these themes, the authors conclude that instructional design, as reflected in the literature, is about developing, assessing, and evaluating instruction, whereas learning design places a stronger focus on “learner engagement and experience, which can be assessed and enhanced by analytical and technological approaches” (p. 1). While the recent adoption of the term ‘learning design experience’ (LDX) is not included in this study, the paper offers a useful ‘ecosystem map’ of what the authors call the instructional and learning design metaverse.

No 7

Houlden, S., & Veletsianos, V. (2022). Impossible dreaming: On speculative education fiction and hopeful learning futures. Postdigital Science and Education. https://doi.org/10.1007/s42438-022-00348-7

This article explores the emergence of speculative fiction as a method for reimagining education futures. Set against the backdrop of the COVID-19 crisis, the authors observe that while speculative fiction offers a powerful lens for examining possible futures, including both the risks and threats and possible changes facing institutions and the education system, the methodology also informs our understanding of the present. In this sense, “futures fictions are not strictly an imagined or fictional endeavour but are concurrently somehow nonfictional in nature” (p. X). Arguably, this point is not fully reflected in some of the speculative fiction produced since the pandemic. However, this line of thinking would benefit from more explicitly anchoring the present and the future in the past. Importantly, many indigenous cultures see time as circular and interconnected with the past, such that Western conceptions of the future based on ‘clock time’ do not fit well with these worldviews. Many indigenous cultures do not distinguish the future from the past. This point raises important questions about who is currently telling the future through the median of speculative fiction. Notwithstanding this critique, the paper speaks to the issue of whose voice is being heard and recognises the power of story in a non-Western, specifically Indigenous ontology. It makes a valuable contribution to the literature by drawing our attention to an increasing tendency towards telling more negative, pessimistic, or apocalyptic stories of the future. Indeed, one of the central tenets of the article is a call to reframe speculative education fiction to develop more hopeful futures. In this respect, the paper conveys a similar message to Atenas et al. (2022) by emphasising agency and community with imagines of hope and more liberatory education futures.

No 8

Holmes, W., & Tuomi, I. (2022). State of the art and practice in AI in education, European Journal of Education, 57(4),542–570. https://doi.org/10.1111/ejed.12533

As the title indicates, this article provides a state-of-the-art account of Artificial Intelligence (AI) in education. Of course, since the article was published, the launch of ChatGPT-3 has placed even greater attention on the problems and possibilities associated with new and emerging AI platforms. Notably, at the time of writing, the authors suggest that ChatGPT-3 is posed to have an even greater impact, although they also note the training involved in producing it is “… estimated to have required as much energy as driving a car to the moon and back, thus generating an equivalent of 85,000 kg of CO2 emissions” (p. 448). The authors begin by addressing the question of what AI is and how we define it? The paper notes that “AI is a complex domain of research that includes many different conceptual approaches and domains of expertise, and sometimes emphasises the point that there is no such a thing as AI” (p. 2). After introducing several competing definitions and the debate about how it should be defined in the European context, the paper discusses an important distinction between data-driven AI and knowledge-driven AI. Importantly, readers are cautioned to be wary of grand claims or sweeping speculative predictions about AI as their validity depends on which of the multiple variations are being discussed in an educational context. To help address this issue, a typology is presented that describes different ways of using AI in education for teaching and learning with three distinct yet overlapping categories: (1) student-focused, (2) teacher-focused, and (3) institution-focused. While recognising the categories are open to conjecture, the remainder of the paper elaborates on each category, with examples of specific AI applications, including whether they are speculative, already researched or commercially available. This taxonomy is probably the most valuable contribution of the article as it describes in practical terms how AI can be applied in education in multiple ways. The discussion is solidly anchored in the literature, with the article concluding with a consideration of the roadblocks on the AI highway. This final section raises some fundamental and controversial issues, including ethics, colonialism, techno-solutionism, and the commercialisation of education, to name a few. Overall, the paper is excellent reading for people trying to make sense of the implications of AI for education, with a critical perspective on the near and longer-term future.

No 9

Celik, I., Dindar, M., Muukkonen, H., & Järvelä, S. (2022). The promises and challenges of artificial intelligence for teachers: A systematic review of research. TechTrends, 66, 616–630. https://doi.org/10.1007/s11528-022-00715-y

This article complements the above paper by providing an overview of research on teachers using artificial intelligence (AI) to support their work. It begins with several underlying premises. Firstly, AI developers know little about learning sciences and lack pedagogical knowledge for its effective implementation in education. Secondly, they often fail to take into consideration the expectations and experiences of AI end-users. Thirdly, teachers are among the most crucial stakeholders for successfully adopting AI in education. While primarily focused on the schooling sector, the basic thesis is that limited attention in AI-based education has been paid to teachers and the nature of their work, a gap in the research the study sets out to address. From the outset, the authors acknowledge that AI does not refer to a single technology and can be used in education differently, as illustrated in the above paper. Six research questions are presented, followed by a description of the literature review methodology, which identified 44 articles suitable for inclusion in this study. Not surprisingly, the authors found that research on teachers’ AI use has intensified in the last four years. In terms of the advantages AI offers teachers, the study reveals three main categories: planning, implementation, and assessment. After summarising these advantages, the paper describes some of the challenges in AI use by teachers, with few surprises based on previous educational technology research. While the study does not address some of the deeper concerns that were raised by Holmes and Tuomi (2022) and contained in last year’s special issue on AI in the European Journal of Education, it helps map the literature. It is also commendable for its strong focus on the teacher. Given the systematic literature review of AI in higher education conducted by Zawacki-Richter et al. (2019) found limited input by educators, this remains an important focus for future research.

No 10

Tlili, A., et al. (2022). Is Metaverse in education a blessing or a curse: a combined content and bibliometric analysis. Smart Learning Environments, 9(24), https://doi.org/10.1186/s40561-022-00205-x

This article claims to provide the first systematic review of the literature on the Metaverse in education. It applies content and bibliometric analysis methods to reveal this research topic’s major trends, focus, and limitations. However, before reporting the research questions, methodology and findings, the authors attempt to define the Metaverse by borrowing an educational definition they acknowledge is techno-centric based on four categories: Augmented Reality (AR), Lifelogging, Mirror Worlds, and Virtual Worlds. Two axes run across these categories: Augmentation versus Simulation and External versus Intimate. A key assumption underpinning the study is that there are generational differences in how people engage with the Metaverse as it evolves, which are worthy of further investigation. Importantly, the review found that few publications explain one or more of the Metaverse categories of technology to a high level of complexity. Moreover, the current literature is dominated by studies from higher education in the fields of Natural Science, Mathematics, and Engineering. Potentially, the most interesting aspect of the analysis relates to the proposed learning scenarios, which try to incorporate more of a pedagogical perspective. While this analysis would benefit from adopting an established pedagogical framework offering greater depth, it shows that most Metaverse applications in education are being used in virtual learning—whatever that means. The discussion then focuses on digital identities followed by a taxonomy of tools consisting of seven categories: immersive, artificial intelligence (AI), game application, educational, modelling and simulation, mobile, sensors, and wearable. Again, some of the analysis and related claims about learner motivation, generational differences, and the tools’ nature would benefit from a greater critique.

While this final paper concludes with several critical questions, a large disconnect exists between these types of systematic literature reviews, which have become increasingly popular, and the ideas put forward by Fawns (2022) and the depth of criticality in other theoretical works in this year’s selection of good reads.

The decision to include a handful of major review articles, some of which adopt a more instrumentalist approach to the field, was influenced by this disconnect. The final article, for example, serves as a useful bookend or rejoinder to our first article, as it helps to demonstrate the importance of the theoretical lenses and critical perspectives we adopt (or not) to interpreting, understanding, and shaping the use of digital learning for better futures in an ever-changing world.

Good Reads from the Closed Literature

Finally, the above selections raise the question of what was published last year in closed journals that we should have on our reading radar? As in previous years, we have identified a single article by Martin et al. (2022) from the wider collection of closed journals or those requiring a fee to access that we think is worthy of reading.

Martin, F. Sun, T., Westine, C., & Ritzhaupt, A. (2022). Examining research on the impact of distance and online learning: A second-order meta-analysis study. Educational Research Review, 6, https://doi.org/10.1016/j.edurev.2022.100438

A wider review of the closed literature is beyond our capacity, but we invite you to consider what you might include in such a list of comparative articles. This year’s chosen article offers an important meta-analysis of the research at a time when the benefits of online learning are being challenged in the post-COVID return to more traditional delivery modes. Hopefully, you can find a way of accessing this publication if your institution or organisation does not provide access.

References

Bond, M., Bedenlier, S., Marín, V.I., Händel, M. (2021). Emergency remote teaching in higher education: Mapping the first global online semester.International Journal of Educational Technology in Higher Education, 18 (50). https://doi.org/10.1186/s41239-021-00282-x

Chu, H-C., Hwang, G-H., Tu, Y-F., & Yang, K-H. (2022). Roles and research trends of artificial intelligence in higher education: A systematic review of the top 50 most-cited articles. Australasian Journal of Educational Technology, 38(3), 22-42. https://ajet.org.au/index.php/AJET/article/view/7526

Holmes, W., & Tuomi, I. (2022). State of the art and practice in AI in education, European Journal of Education, 57(4),542–570. https://doi.org/10.1111/ejed.12533

Macgilchrist, F., Potter, J., & Williamson, B. (2022). Reading internationally: If citing is a political practice, who are we reading and who are we citing? Learning, Media and Technology, 47(4), 407-412, https://doi.org/10.1080/17439884.2022.2140673

Selwyn, N. (2022). The future of AI and education: Some cautionary notes. European Journal of Education, 57(4), 620-631. https://doi.org/10.1111/ejed.12532

Siemens, G. (2004). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology & Distance Learning, 2(1). http://www.itdl.org/Journal/Jan_05/article01.htm

Zawacki-Richter, O., Marin, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0

Digital Learning, Highlights, Leadership, Learning Design, Learning Futures, MOOCs, Online Learning, Open Educational Practices, Policy and Strategy, Publications, Quality, Research, Teaching, Technology, Think Piece

Good Reads from 2021: Our NIDL Top 10 Journal Articles – Part 3

This is the third and final blog post in the process of introducing our NIDL top 10 “good reads” for 2021. Next week we will be sharing a separate list of “good reads” from last year with a specific COVID-related focus. In our first and second blog posts, we established the background context and explained how we go about selecting our annual list of top 10 “good reads”.

In this final post, we begin by sharing some of the descriptive features of our 2021 selections and report on how they compare with those chosen over the previous five years. We then share our list of top 10 “good reads” for 2021 along with a brief commentary on each article helping to explain the rationale behind each selection. Finally, we offer five additional highly recommended articles and briefly comment on the gaps in our selections and the importance of continuing to access and keep a watchful eye on literature that can only be found in restricted publications.

What’s included in this year’s top 10?

We begin by sharing some interesting observations and providing comparative data on this year’s top 10 articles. Firstly, for the third year running a higher proportion of women authors feature in the list, as illustrated in Table 1. This reverses a trend over the first three years of our selections.

Table 2 shows that the number of multiple authored articles in the top 10 list consists of 50% of the final selection. Only two articles are single authored which is consistent with previous years.

Notably, one author features in three of this year’s top 10 selections and another in two of them. The former author, Aras Bozkurt, has now featured in eight of the top 10 articles over the past six years. This number is matched by Olaf Zawacki-Richter who also features in this year’s selections and their prominence, along with Melissa Bond (n=4), reflects our bias towards major literature reviews. Another nine authors (Gourley, Kalz, Knox, Lee, Lundin, Sangra, Selwyn, Shea, & Weller) have featured twice in our annual top 10 “good reads” since we began this exercise.

This year’s No 1 article appears in E-Learning and Digital Media which appears for the first time. As shown in Table 3, an article published in the Australasian Journal of Educational Technology took the No 1 spot over the first four years of selections. This year, two articles once again feature from this journal. In total, eight journals feature in this year’s top 10 list, with three appearing for the first time. Another new journal features in our list of five highly recommended articles, with the other four appearing in the International Review of Research in Open and Distributed Learning and the International Journal of Educational Technology in Higher Education.

Table 4 illustrates the distribution of journals included in the top 10 selections over the past six years. While some journals remain a consistent source of “good reads”, it is noteworthy that around 30% of the selected articles come from other types of journals, which we believe reflects our deliberate attempt to source a wide and diverse range of publications. In a similar pattern to 2020, this year half of our top 10 articles feature in such journals.

We are less successful in selecting journal articles that are evenly spread in terms of geographical distribution. Table 5 reveals the dominance of English language speaking countries, with around 25% of the top 10 articles over the last six years having authors located in North America. While there is a strong trend of collaboration by authors across regions, which helps to increase the actual level of geographical inclusion, Asia, Africa and South America rarely feature in our selections.

This year, six articles fall under the ‘across region’ category and for only the second time in six years there are no publications from authors exclusively based in North America. Having said that, three of our highly recommended articles have authors from this region, although we are pleased to report one is written by scholars based in South America. The other comes from Australasia.

What are the top 10 articles for 2021?

The specific rank order of each article remains one of the most contentious aspects of the selection exercise, but to maintain consistency we have retained this practice.

Before listing our top 10 “good reads” in rank order, we would like to congratulate all of the authors who appear in our selections, including those in the highly recommended category.

We hope that you value this recognition of your work as it arises from lots of reading and a systematic review process. Additionally, we also trust that people find our brief notes on each article useful and they do justice to the authors and adequately describe the main focus and major contribution of each selection. The intention is to help explain why the article was selected as a top 10 “good read” and hopefully entice more people to slow read the ideas, thinking and perspectives contained in the writing and analysis of data.

No 1 – Selwyn, N. (2021). Ed-Tech within limits: Anticipating educational technology in times of environmental crisis. E-Learning and Digital Media, 18 (5), 496–510. https://doi.org/10.1177/20427530211022951

This article meets the criteria of challenging conventional thinking and stands out for the way it locates educational technology within wider societal concerns about climate change, ecological instability and environmental issues. While there have been several claims over the years that distance education is environmentally friendly and offers the opportunity to develop a low carbon higher education system, this article goes beyond any ‘feel good’ factor by raising serious questions about the ‘greening’ of schools and university provision through new digital technologies.

As Facer and Selwyn (2021) point out in a related UNESCO background paper, online learning may be an environmental solution to lowering emissions but massively increased global use of digital technologies in education require unsustainable levels of energy and place huge demands on natural resource consumption. They note that this includes the ‘dirty’ aspects of digital hardware production, the vast energy requirements of data-processing centres and the increasing problem of e-waste.

The uncomfortable truth is that these concerns feature rarely in the literature, despite evidence that the COVID reset has been a boon for the EdTech sector and this growth is likely to continue bolstering the uptake of new digital technologies for teaching and learning purposes. While the author acknowledges that technology can also be of the solution during a time of crisis, the sobering point is further planetary degradation may over the next decade put paid to ‘abundant’ and taken-for-granted forms of digital technology use. Therefore, the article calls for a new ‘within limits’ paradigm of educational technology that is both sustainable and more intentionally targeted towards addressing major societal challenges and those facing disadvantaged groups.

There is no question that this paper achieves its stated goal of provoking the field of educational technology to revalue, reconfigure, reprioritise and move beyond a complacency when it comes to the environmental crisis. The urgency and seriousness of this challenge left us with little choice when it came to ranking this year’s No 1 article, and we encourage everyone to ‘slow read’ and critically reflect on the big issues raised by the author as we plan for an unknown future.

No 2 – Gourlay, L. (2021). There is no ‘virtual learning’: The materiality of digital education. Journal of New Approaches in Educational Research, 10 (1), 57-66. https://doi.org/10.7821/naer.2021.1.649

This article stands out for what is says on the tin—that is, ‘there is no virtual learning’. It offers a deep, thoughtful and convincing argument why the notion of ‘virtual learning’ is a flawed one. The author draws on Sociomaterial and Posthuman theoretical perspectives to challenge the false binary between virtual and face-to-face forms of learning, with a central thesis highly relevant to practitioners, institutional leaders and educational policy-makers.

Even if readers are unfamiliar with some of the theoretical literature, the article illustrates, through several examples, why we need to challenge our traditional conceptions and understandings of space, distance, absence, and presence in digital higher education. It demonstrates the embodied nature of digital technologies and how they are entangled in our day-to-day living and learning. In practical terms, the article makes the case that all learning and digital engagement is ‘in person’ and therefore we need to change our thinking, modify our language and redefine traditional delivery modes to reflect this more complex reality.

More generally, the paper reminds us to be critical of dominant technologically deterministic notions which position ‘the digital’ as a separate or independent driving force from education and society. Further extrapolating the notion of ‘entanglement’ reveals the naivety of the popular tool metaphor for learning technology, which falsely implies a degree of neutrality. It also brings into question efforts to disembody ‘the digital’ from the interwoven nature of pedagogy as educators rally to the appeal of ‘pedagogy-first’.

No 3 – Gourlay, L., Rodríguez-Illera, J.L. et al. (2021). Networked learning in 2021: A community definition. Postdigital Science and Education 3, 326–369. https://doi.org/10.1007/s42438-021-00222-y

This article also focuses on the language we use and the values and thinking this reflects in a highly engaging discussion on networked conceptions of learning. It begins by outlining how instrumentalist understandings and managerialist approaches to digital education have permeated our thinking in the field and traces the emergence of networked learning as a response to the dominant discourses of the day. This discussion returns to the fore the critical and emancipatory agenda underlying the notion of networked learning and those who formed the original academic and research community.

However, the paper acknowledges that a lot has changed since the definition of network learning over two decades ago and therefore it takes up the challenge to reframe the current meaning and understanding for future directions and developments. What makes the article unique is the way it is collectively authored by 40 contributors from six continents working across many fields of education. Moreover, the reviewers are acknowledged as authors as their feedback and viewpoints are incorporated within the paper.

While there was always a risk that contributions from so many different authors under separate headings may have resulted in a disjointed collection of works, this concern is unwarranted as the conversation is seamless and insightful as each section adds a new perspective and often critical twist to the search for a refined and potentially unified definition. What really stands out from the discussion is how difficult it is to define the undefinable and how our search for commons definitions and to pin down our language can inadvertently narrow thinking and foreclose on different perspectives.

As Siân Bayne writes in her contribution to the article, “To define a field is necessarily to put boundaries around it, to determine which writings, conversations, people are ‘inside’ and which are ‘outside’ (Gourlay, L., Rodríguez-Illera, J.L. et al., 2021, p. 333). This point has wider relevance beyond efforts to redefine network learning as there several ongoing definition wars in the wider field of educational technology. In summary, we recommend this article as it helps to stretch the current digital horizon to longer-term societal goals of equity and social justice through a rich variety of perspectives.

No 4 – Castaneda, L., & Williamson, B. (2021). Assembling new toolboxes of methods and theories for innovative critical research on educational technology. Journal of New Approaches in Educational Research, 10 (1), 1-14. http://dx.doi.org/10.7821/naer.2021.1.703

This article continues the theme of critique and reminds us of the importance of ‘public intellectuals’ in the face of greater interest, reinvigorated attention and competing agenda promoting the potential of educational technology. Importantly, the authors point out that many of the concerns that have played out in the backdrop of the COVID crisis mirror issues already raised in previous research.

They highlight enduring tensions and often politicized debates between techno-utopian enthusiasts, sometimes referred to as ‘Boosters’ or ‘Deschoolers’ (Brown, et al, 2019), who promote the disruptive and transformative benefits of digital technology, and those offering more critical voices. While not singled out in the article it is useful to emphasise that those labelled as ‘critics’ are often confused with a group of techno-dystopian detractors, or ‘Doomsters’ (Brown, et al, 2019), who repeat arguments from previous moral panics or draw on a demon perspective to suggest a technocratic nightmare.

Arguably, true critics adopt a multi-focal lens recognising that educational technology needs to be framed beyond crude binary positions of good or bad. As the authors point out, neither boom or threat positions offer a constructive path forward. More specifically, they go on to share their concerns that current preoccupation with evidence of ‘what works’ in relation to ‘EdTech’ risks deflecting attention away from crucial educational questions that consider more complex and wider societal issues. In this sense, the article helps us to keep the bigger picture in mind alerts us to the danger of getting caught up in the digital forest.

A related concern is the need for critical research and theorising to evolve fast if we are to shape and influence the field, especially given the emergence of “Big EdTech” (Brown, 2021) with powerful new actors attempting to accelerate the pace and scale of change. With the objective of helping to assemble new toolboxes for critical research and innovative approaches, the article identifies several current gaps and under-researched issues. It introduces a valuable collection of papers that offer exemplars and useful guiding beacons for future lines of research and development.

No 5 – Tamim, R. M., Borokhovski, E., Bernard, R. M., Schmid, R. F., Abrami, P. C., & Pickup, D. I. (2021). A study of meta-analyses reporting quality in the large and expanding literature of educational technology. Australasian Journal of Educational Technology, 37(4), 100–115. https://doi.org/10.14742/ajet.6322

This article makes a valuable contribution by recognising that not all meta-analyses of educational technology are equal. In recent years, rapid reviews, scoping reviews, systematic literature reviews and other variations of meta-analyses have become increasingly common with many featuring in our list of “good reads” partly due to the selection criteria. However, we have become wary of a tendency towards technicist or instrumentalist approaches to these major literature reviews as they often lack a framing theoretical perspective and deeper level of critique.

Indeed, the reader may find it difficult to judge their true value and contribution to the field due to increasing techno-sophistication effects in the way the findings are presented using new data analysis software. In reviewing 52 meta-analyses in the field of educational technology over almost 30 years using the Meta-Analysis Methodological Reporting Quality Guide (MMRQG), the authors confirm our own suspicion that many studies meet only moderate levels of quality. Thus, there is a risk that this type of analysis potentially misleads researchers and practitioners alike. Accordingly, we suggest that anyone embarking on a literature review in the field should familiarise themselves with this article.

We also suggest that many educators and researchers, including current and prospective doctoral candidates, would benefit from delving more deeply into the literature on how to undertake a quality meta-analyses. Three recent publications are worthy of slow reading: i) Systematic reviews in educational research: Methodology, perspectives and application (Zawacki-Richter, et al., 2020); Rapid reviews as an emerging approach to evidence synthesis in education (Wollscheid & Tripney, 2021); and iii) The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews (Steel, Beugelsdijk & Aguinis, 2021).

No 6 – Jiménez‐Cortés, R., & Aires, L. (2021). Feminist trends in distance and hybrid higher education: a scoping review. International Journal of Educational Technology in Higher Education, 18:60, 1-20.https://doi.org/10.1186/s41239-021-00297-4

This article is the first in a sequence of four major literature reviews exploring different aspects of the field. It appears first in this sequence in light of our previous concerns about the growth of descriptive analyses of the literature lacking a deeper level of critique and theoretical perspective. Few could argue that feminist perspectives have featured prominently in the educational technology literature and there is growing evidence of what is known as the glass escalator in terms of how women are disadvantaged in the field, as Bond et al. (2021) recently illustrate in their survey of U.S. based instructional designers.

Therefore, this article reporting a review of 160 journals and 10 articles that meet the inclusion criteria is a welcome contribution to the field. Notably, the search protocol, methodology and review process over three phases is guided by the PRISMA-SCR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews), which helps to enhance the trustworthiness of data extraction and interpretation. Findings in terms of distance and hybrid models of education are reported based on three feminist approaches: i) feminist-pragmatist, ii) eco-dialogical feminist, and iii) intersectional-technofeminist. Those unfamiliar with feminist theories and pedagogical research will find these three perspectives relatively accessible and a real strength of the article is its description of how feminist digital pedagogy goes beyond critical digital pedagogy.

Importantly, the authors argue that digital enhanced teaching and learning platforms can limit feminist responses and reproduce dominant structures that reinforce existing power relations. While the article serves to highlight networked learning ideologies that from a feminist approach value connections, relationships, and collaborations, it also illustrates contrasting perspectives and their relationship to other theories concerning materiality, embodiment and hierarchies of power. Thus, the article flags an important gap in the literature and helps to border cross with other theoretical developments which have practical implications in terms of creating a more equitable and inclusive digital education ecosystem.

No 7 – Bozkurt, A., & Zawacki-Richter, O. (2021). Trends and Patterns in Distance Education (2014–2019): A Synthesis of Scholarly Publications and a Visualization of the Intellectual Landscape. The International Review of Research in Open and Distributed Learning, 22(2), 19-45. https://doi.org/10.19173/irrodl.v22i2.5381

This article builds on several previous studies by the same authors exploring trends and patterns in distance education. Accordingly, in reviewing this article we had to ask the question: what makes it different from previous work and how does it contribute to new knowledge? Whether the six journals chosen for this analysis are truly representative of the field is open to conjecture, but two aspects standout in terms of answering this question. Firstly, the authors demonstrate how social network analysis (SNA) can be used to reveal new insights into a field.

More specifically, they model how researchers can deploy powerful data analysis software to undertake t-SNE analysis and content and co-word analysis using text mining techniques to visually identify and illustrate patterns, pivotal contributions and turning points. The underlying assumption that this type of visual analysis of the landscape has potential to better inform future research and development in the field is sound.

The second valuable contribution of this visual analysis is evidence of how since the 2000s, the fields of distance education and educational technology have intersected and triggered innovations in each other. Inclusion of more journals from the wider educational technology literature and mainstream publications would provide a means of further validating this type of cross fertilisation or symbiotic relationship. Also, greater consideration of what is missing from the published literature in terms of the issues raised by Castaneda and Williamson (2021) would further enhance the value of this type of analysis.

Drawing on their findings the authors conclude with recommendations for future research directions. Like several other articles in this year’s selection of “good reads”, they ask what our research agenda will and should be in the changing world.

No 8 – Liu, C., Zou, D., Chen, X., Xie, H., & Chan, W. H. (2021). A bibliometric review on latent topics and trends of the empirical MOOC literature (2008–2019). Asia Pacific Education Review, 22 (3), 515-534. https://doi.org/10.1007/s12564-021-09692-y

This article ensures that we do not overlook the continuing influence of MOOCs on the education and digital learning landscape. As Bozkurt (2021) suggests in a separate analysis of the literature using data mining and analytic approaches, the MOOC continues to evolve and as new waves emerge their impact on traditional models and methods of education warrant ongoing investigation. We chose this particular bibliometric review of over 1,000 peer-reviewed MOOC studies published between 2008 and 2019 as it helps to bring an Asian perspective to the literature.

Notably, several of the research questions seek to investigate differences and collaborations across countries/regions, which gives further weight to the global perspective we hoped this article might provide. Not surprisingly, the review extracted its data from three reputable and influential publication databases, which unfortunately have a strong English language bias. This is something we have increasingly recognised over the years and places linguistic boundaries around our thinking and opportunities to share and co-construct new knowledge.

Mindful of this limitation and that many of us are unable to read what is published in other languages, the authors report that 11 countries contributed 81% of the total MOOC publications. The US (n=266) was the most prolific, with the UK (n=103), Spain (n=116) and China (n=172) featuring prominently in the literature. In terms of the latter, we know through our NIDL research collaborations with Chinese partners that the MOOC movement has generated an active local research community. To the author’s credit, some of this community and the level of global collaboration is revealed through social network analysis identifying scientific activity across countries/regions. The US collaborated with the most countries, followed by Spain, the UK, the Netherlands, China, Australia, and Germany. Ireland also features in this analysis, which is further illustrated through a visual representation of the most prolific institutions.

The main research topics and trends are also analysed at a country and institutional level helping this study to go beyond previous MOOC literature reviews. Thus, the authors generally meet their claim to have provided a deep and comprehensive understanding of current MOOC research up until 2019, which, in turn, builds on other reviews and supports future research.

No 9 – Tlili, A., Burgos, D., Huang, R., Mishra, S., Sharma, R. C., & Bozkurt, A. (2021). An analysis of peer-reviewed publications on Open Educational Practices (OEP) from 2007 to 2020: A bibliometric mapping analysis. Sustainability, 13, 10798, 1-15. https://doi.org/10.3390/su131910798

This article is the fourth major literature review featuring in this year’s selections with a focus on Open Education Practices (OEPs). The study reports a bibliometric mapping analysis of 156 publications listed in the Scopus and Web of Science databases that meet the inclusion criteria for OEPs. Importantly, the authors begin the paper by making a distinction between OEPs and more content-centred Open Educational Resources (OERs). The authors reveal how there has been a steady growth in OEP literature, with a peak in 2020. In light of the above comment about English language bias in research publication databases, perhaps not unexpectedly, 88% of the studies were in English, 6% were in Spanish, with 2% or less in French, Portuguese, Korean, and Russian.

The study identifies the leading journals publishing work in this area, with Distance Education contributing the most articles (n=18), which is ironic since this prestigious journal published by ODLAA is not fully open. Having said that, a special issue on OEPs does help to explain the journal’s position. The UK leads the country ‘league table’ with 38 documents followed by Spain (n=21) and Australia (n=20). The influence of North America on the OEP literature is better reflected when the tally for the US and Canada is combined as together they contribute 30 documents. Ireland is 10^th in the list of countries with 4 documents, which is somewhat surprising given the level of interest in this area, although this number may reflect more of a content-centred emphasis on OERs.

While the study shows on-going international cooperation regarding OEP amongst leading institutions and the contribution of several well-known scholars, the scale of global collaboration is limited. We believe this is an important finding. It would be interesting to compare the level of global collaboration between the OEP and OER communities, although many of the leading scholars work across both communities; and there is a question of where does the “Open Pedagogy” community ‘fit’ or map in terms of this type of relational analysis? We suggest the article needs to be read alongside a recent literature review on Open Pedagogy (Tietjen & Asino, 2021) as this is another branch of the literature.

The article is generally methodologically sound and helps to demonstrate how VOSviewer software can be deployed to undertake this type of bibliometric analysis. However, in light of the above comments, we would have liked a little more from the article. It offers limited analysis and critical insight into the competing drivers and attractors underlying the OEP movement. Given the thorny issue of definitions, different branches of the literature, and the politics of Openness, the question left unanswered is how do different OEP viewpoints, perspectives and understandings influence what is being researched, who is doing the research and what is being publishing in this area. It would also be interesting to analyse how the OER and Open Pedagogy literature is influencing OEP, and back again, based on citations and mapping the publication landscape.

Overall, the findings provide useful suggestions for future research and it is hard to disagree with the call for more inclusive practices that promote efforts to support and accommodate students with disabilities. In a similar vein, the value of focusing on cultural differences in education to internationalise OEPs makes good sense if we are to advance the United Nation’s Sustainable Development Goals and ensure that everyone benefits from the openness movement.

No 10 – Lodge, J. M., Corrin, L., Hwang, G.-J., & Thompson, K. (2021). Open science and educational technology research. Australasian Journal of Educational Technology, 37(4), 1–6. https://doi.org/10.14742/ajet.7565

This final article has a strong focus on the quality of research informing decisions and the direction of new developments in educational technology. Borrowing from other fields the authors describe the ‘replication crisis’ that helps to reveal questionable research practices (QRPs) and the robustness and trustworthiness of quantitative research across many disciplines. They suggest one of the greatest problems is that many published works are low powered—that is, the research fails to establish whether an effect is occurring at greater than chance levels.

Another concern is that of hypothesising after the results are known (HARKing), although this is difficult to identify based on a published article alone. Serving as an editorial, the paper goes on to explore what the replication crisis means for educational technology research with suggestions for future responses by the academic and professional community. While the authors observe that educational technology research does not rely as much on experimental designs, there is often a degree of intervention which then involves the study of effects raising inherent questions of replicability and generalisability. Although not singled out by the authors, the novelty of any intervention, often described as the ‘Hawthorn Effect’, also needs to be taken into account.

Overall, this is a novel and timely critique of the published research where the authors conclude that ‘Low powered studies are common’ (p. 3). Poor statistical practices characterise the field and there is evidence of widespread publication bias. Looking to the future, the article promotes the benefits of greater transparency as part of the ‘Open Science’ movement and renewed emphasis on statistical practices.

While it is hard to disagree with the concluding statement that quality must take precedence over what attracts the most clicks, perhaps future editorials could give more attention to the contestable nature of what counts as quality in educational technology research. After all, the idea of replication has Positivist undertones and can be interpreted from many different research perspectives.

What else is worthy of reading?

In this final section we identify five additional articles that are worthy of slow reading if you have not already come across them. In alphabetical order we highly recommended the following articles.

Dell, D. (2021). Resonance and current relevance of IRRODL Highly-cited articles: An integrative retrospective. The International Review of Research in Open and Distributed Learning, 22(1), 243-258. https://doi.org/10.19173/irrodl.v22i1.5315

This article which notably is written by a current doctoral candidate at Athabasca University provides a useful summary of the top 100 highly cited publications from past issues in this leading open access journal.

Murphy, M. (2021). Belief without evidence? A policy research note on Universal Design for Learning. Policy Futures in Education, 19 (1), 7–12. https://doi.org/10.1177/1478210320940206

This article might not be to everyone’s taste, but it asks some important questions about the quality of evidence supporting the growing adoption of UDL principles in both policy and practice.

Guzman‑Valenzuela , C., Gomez‑Gonzalez, C., Rojas‑Murphy Tagle, A, & Lorca‑Vyhmeister, A. (2021). Learning analytics in higher education: a preponderance of analytics but very little learning? International Journal of Educational Technology in Higher Education, 18:23, 1-19. https://doi.org/10.1186/s41239-021-00258-x

This article helps to keep the rapidly evolving area of learning analytics to the fore of thinking and offers a valuable critical perspective through an analysis of 385 papers by showing that much of the focus to date has been more on analytics than on learning.

Moon, J., & Park, Y. (2021). A scoping review on Open Educational Resources to support interactions of learners with disabilities. The International Review of Research in Open and Distributed Learning, 22 (2), 314-341. https://doi.org/10.19173/irrodl.v22i1.5110

This paper through a scoping review of the literature responds to the challenge presented in one of our top 10 selections to promote more inclusive practices that support students with disabilities.

Nkomo, L., Daniel, B., & Butson, R. (2021). Synthesis of student engagement with digital technologies: a systematic review of the literature. International Journal of Educational Technology in Higher Education, 18:34, 1-19. https://educationaltechnologyjournal.springeropen.com/articles/10.1186/s41239-021-00270-1

This article adopts a tripartite model to provide a systematic review of the literature on the degree to which social media, video, and collaborative learning technologies have supported student engagement over the past decade.

Where are the gaps?

We conclude with a brief comment on what’s missing from this year’s list as arguably the gaps are just as important as what we choose to profile. It’s simply not possible to include everything and so we acknowledge there are valuable publications offering a synthesis of the literature on issues of quality, creativity, wellbeing, teacher competence, mobile learning, blockchain, immersive VR, virtual laboratories, use of video, the growth of surveillance and online exam proctoring, and the unbundling movement, to name a few.

This last topic also serves to illustrate that the abundance of open access journals does not diminish the need to monitor the literature that often is only available in restricted publications.

Therefore, we finish by drawing your attention to one such article on this topic that makes good reading if your institution provides access to the journal:

Matkin, G. (2021). Reshaping university continuing education: Leadership imperatives for thriving in a changing and competitive market. American Journal of Distance Education, https://doi.org/10.1080/08923647.2021.1996217

References

Bond, J., Dirkin, K., Tyler, A.J., & Lassitter, S. (2021). Ladders and escalators: Examining advancement obstacles for women in instructional design. Journal of Applied Instructional Design,https://edtechbooks.org/jaid_10_2/examining_advancemen

Bozkurt, A. (2021). Surfing on three waves of MOOCs: An examination and snapshot of research in Massive Open Online Courses. Open Praxis, 13 (3), 296–311. DOI: http://doi.org/10.5944/openpraxis.13.3.132

Brown, M. (2021). What are the main trends in online learning? A helicopter view of possible futures. Asian Journal of Distance Education, 16 (2), http://asianjde.com/ojs/index.php/AsianJDE/article/view/605

Brown, M., Conole, G., & Beblavỳ, M. (2019). Education outcomes enhanced by the use of digital technology: Reimagining the school learning ecology. EENEE Analytical Report No. 38 Prepared for the European Commission. March.

Facer, K., & Selwyn, N. (2021). Digital technology and the futures of education – towards ‘non-stupid’ optimism.Background paper for the Futures of Education initiative, Paris: UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000377071.locale=en

Steel, P., Beugelsdijk, S. & Aguinis, H. (2021). The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews. Journal of International Business Studies 52,23–44. https://doi.org/10,.1057/s41267-020-00385-z

Tietjen, P., & Asino, T. I. (2021). What is open pedagogy? Identifying commonalities. The International Review of Research in Open and Distributed Learning, 22 (2), 185-204. https://doi.org/10.19173/irrodl.v22i2.5161

Wollscheid, S., & Tripney, J. (2021). Rapid reviews as an emerging approach to evidence synthesis in education. London Review of Education, 19 (1), https://ucl.scienceopen.com/hosted-document?doi=10.14324/LRE.19.1.32

Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., & Buntins, K. (eds.). (2020). Systematic reviews in educational research: Methodology, perspectives and application.Wiesbaden, Germany: Springer VS.