Good Reads from 2021: Our NIDL Top 10 Journal Articles – Part 3

This is the third and final blog post in the process of introducing our NIDL top 10 “good reads” for 2021. Next week we will be sharing a separate list of “good reads” from last year with a specific COVID-related focus. In our first and second blog posts, we established the background context and explained how we go about selecting our annual list of top 10 “good reads”. 

In this final post, we begin by sharing some of the descriptive features of our 2021 selections and report on how they compare with those chosen over the previous five years. We then share our list of top 10 “good reads” for 2021 along with a brief commentary on each article helping to explain the rationale behind each selection. Finally, we offer five additional highly recommended articles and briefly comment on the gaps in our selections and the importance of continuing to access and keep a watchful eye on literature that can only be found in restricted publications. 

What’s included in this year’s top 10? 

We begin by sharing some interesting observations and providing comparative data on this year’s top 10 articles. Firstly, for the third year running a higher proportion of women authors feature in the list, as illustrated in Table 1. This reverses a trend over the first three years of our selections.

Table 2 shows that the number of multiple authored articles in the top 10 list consists of 50% of the final selection. Only two articles are single authored which is consistent with previous years. 

Notably, one author features in three of this year’s top 10 selections and another in two of them. The former author, Aras Bozkurt, has now featured in eight of the top 10 articles over the past six years. This number is matched by Olaf Zawacki-Richter who also features in this year’s selections and their prominence, along with Melissa Bond (n=4), reflects our bias towards major literature reviews. Another nine authors (Gourley, Kalz, Knox, Lee, Lundin, Sangra, Selwyn, Shea, & Weller) have featured twice in our annual top 10 “good reads” since we began this exercise.

This year’s No 1 article appears in E-Learning and Digital Media which appears for the first time. As shown in Table 3, an article published in the Australasian Journal of Educational Technology took the No 1 spot over the first four years of selections. This year, two articles once again feature from this journal. In total, eight journals feature in this year’s top 10 list, with three appearing for the first time. Another new journal features in our list of five highly recommended articles, with the other four appearing in the International Review of Research in Open and Distributed Learning and the International Journal of Educational Technology in Higher Education.

Table 4 illustrates the distribution of journals included in the top 10 selections over the past six years. While some journals remain a consistent source of “good reads”, it is noteworthy that around 30% of the selected articles come from other types of journals, which we believe reflects our deliberate attempt to source a wide and diverse range of publications. In a similar pattern to 2020, this year half of our top 10 articles feature in such journals. 

We are less successful in selecting journal articles that are evenly spread in terms of geographical distribution. Table 5 reveals the dominance of English language speaking countries, with around 25% of the top 10 articles over the last six years having authors located in North America. While there is a strong trend of collaboration by authors across regions, which helps to increase the actual level of geographical inclusion, Asia, Africa and South America rarely feature in our selections. 

This year, six articles fall under the ‘across region’ category and for only the second time in six years there are no publications from authors exclusively based in North America. Having said that, three of our highly recommended articles have authors from this region, although we are pleased to report one is written by scholars based in South America. The other comes from Australasia. 

What are the top 10 articles for 2021?

The specific rank order of each article remains one of the most contentious aspects of the selection exercise, but to maintain consistency we have retained this practice.

Before listing our top 10 “good reads” in rank order, we would like to congratulate all of the authors who appear in our selections, including those in the highly recommended category.

We hope that you value this recognition of your work as it arises from lots of reading and a systematic review process. Additionally, we also trust that people find our brief notes on each article useful and they do justice to the authors and adequately describe the main focus and major contribution of each selection. The intention is to help explain why the article was selected as a top 10 “good read” and hopefully entice more people to slow read the ideas, thinking and perspectives contained in the writing and analysis of data.

No 1 – Selwyn, N. (2021). Ed-Tech within limits: Anticipating educational technology in times of environmental crisis. E-Learning and Digital Media, 18 (5), 496–510. https://doi.org/10.1177/20427530211022951

This article meets the criteria of challenging conventional thinking and stands out for the way it locates educational technology within wider societal concerns about climate change, ecological instability and environmental issues. While there have been several claims over the years that distance education is environmentally friendly and offers the opportunity to develop a low carbon higher education system, this article goes beyond any ‘feel good’ factor by raising serious questions about the ‘greening’ of schools and university provision through new digital technologies.

As Facer and Selwyn (2021) point out in a related UNESCO background paper, online learning may be an environmental solution to lowering emissions but massively increased global use of digital technologies in education require unsustainable levels of energy and place huge demands on natural resource consumption. They note that this includes the ‘dirty’ aspects of digital hardware production, the vast energy requirements of data-processing centres and the increasing problem of e-waste.

The uncomfortable truth is that these concerns feature rarely in the literature, despite evidence that the COVID reset has been a boon for the EdTech sector and this growth is likely to continue bolstering the uptake of new digital technologies for teaching and learning purposes. While the author acknowledges that technology can also be of the solution during a time of crisis, the sobering point is further planetary degradation may over the next decade put paid to ‘abundant’ and taken-for-granted forms of digital technology use. Therefore, the article calls for a new ‘within limits’ paradigm of educational technology that is both sustainable and more intentionally targeted towards addressing major societal challenges and those facing disadvantaged groups.

There is no question that this paper achieves its stated goal of provoking the field of educational technology to revalue, reconfigure, reprioritise and move beyond a complacency when it comes to the environmental crisis. The urgency and seriousness of this challenge left us with little choice when it came to ranking this year’s No 1 article, and we encourage everyone to ‘slow read’ and critically reflect on the big issues raised by the author as we plan for an unknown future. 

No 2 – Gourlay, L. (2021). There is no ‘virtual learning’: The materiality of digital education. Journal of New Approaches in Educational Research, 10 (1), 57-66. https://doi.org/10.7821/naer.2021.1.649

This article stands out for what is says on the tin—that is, ‘there is no virtual learning’. It offers a deep, thoughtful and convincing argument why the notion of ‘virtual learning’ is a flawed one. The author draws on Sociomaterial and Posthuman theoretical perspectives to challenge the false binary between virtual and face-to-face forms of learning, with a central thesis highly relevant to practitioners, institutional leaders and educational policy-makers.

Even if readers are unfamiliar with some of the theoretical literature, the article illustrates, through several examples, why we need to challenge our traditional conceptions and understandings of space, distance, absence, and presence in digital higher education. It demonstrates the embodied nature of digital technologies and how they are entangled in our day-to-day living and learning. In practical terms, the article makes the case that all learning and digital engagement is ‘in person’ and therefore we need to change our thinking, modify our language and redefine traditional delivery modes to reflect this more complex reality.

More generally, the paper reminds us to be critical of dominant technologically deterministic notions which position ‘the digital’ as a separate or independent driving force from education and society. Further extrapolating the notion of ‘entanglement’ reveals the naivety of the popular tool metaphor for learning technology, which falsely implies a degree of neutrality. It also brings into question efforts to disembody ‘the digital’ from the interwoven nature of pedagogy as educators rally to the appeal of ‘pedagogy-first’. 

No 3 – Gourlay, L., Rodríguez-Illera, J.L. et al. (2021). Networked learning in 2021: A community definition. Postdigital Science and Education 3, 326–369. https://doi.org/10.1007/s42438-021-00222-y

This article also focuses on the language we use and the values and thinking this reflects in a highly engaging discussion on networked conceptions of learning. It begins by outlining how instrumentalist understandings and managerialist approaches to digital education have permeated our thinking in the field and traces the emergence of networked learning as a response to the dominant discourses of the day. This discussion returns to the fore the critical and emancipatory agenda underlying the notion of networked learning and those who formed the original academic and research community.

However, the paper acknowledges that a lot has changed since the definition of network learning over two decades ago and therefore it takes up the challenge to reframe the current meaning and understanding for future directions and developments. What makes the article unique is the way it is collectively authored by 40 contributors from six continents working across many fields of education. Moreover, the reviewers are acknowledged as authors as their feedback and viewpoints are incorporated within the paper.

While there was always a risk that contributions from so many different authors under separate headings may have resulted in a disjointed collection of works, this concern is unwarranted as the conversation is seamless and insightful as each section adds a new perspective and often critical twist to the search for a refined and potentially unified definition. What really stands out from the discussion is how difficult it is to define the undefinable and how our search for commons definitions and to pin down our language can inadvertently narrow thinking and foreclose on different perspectives.

As Siân Bayne writes in her contribution to the article, “To define a field is necessarily to put boundaries around it, to determine which writings, conversations, people are ‘inside’ and which are ‘outside’ (Gourlay, L., Rodríguez-Illera, J.L. et al., 2021, p. 333). This point has wider relevance beyond efforts to redefine network learning as there several ongoing definition wars in the wider field of educational technology. In summary, we recommend this article as it helps to stretch the current digital horizon to longer-term societal goals of equity and social justice through a rich variety of perspectives. 

No 4 – Castaneda, L., & Williamson, B. (2021). Assembling new toolboxes of methods and theories for innovative critical research on educational technology. Journal of New Approaches in Educational Research, 10 (1), 1-14. http://dx.doi.org/10.7821/naer.2021.1.703

This article continues the theme of critique and reminds us of the importance of ‘public intellectuals’ in the face of greater interest, reinvigorated attention and competing agenda promoting the potential of educational technology. Importantly, the authors point out that many of the concerns that have played out in the backdrop of the COVID crisis mirror issues already raised in previous research.

They highlight enduring tensions and often politicized debates between techno-utopian enthusiasts, sometimes referred to as ‘Boosters’ or ‘Deschoolers’ (Brown, et al, 2019), who promote the disruptive and transformative benefits of digital technology, and those offering more critical voices. While not singled out in the article it is useful to emphasise that those labelled as ‘critics’ are often confused with a group of techno-dystopian detractors, or ‘Doomsters’ (Brown, et al, 2019), who repeat arguments from previous moral panics or draw on a demon perspective to suggest a technocratic nightmare.

Arguably, true critics adopt a multi-focal lens recognising that educational technology needs to be framed beyond crude binary positions of good or bad. As the authors point out, neither boom or threat positions offer a constructive path forward. More specifically, they go on to share their concerns that current preoccupation with evidence of ‘what works’ in relation to ‘EdTech’ risks deflecting attention away from crucial educational questions that consider more complex and wider societal issues. In this sense, the article helps us to keep the bigger picture in mind alerts us to the danger of getting caught up in the digital forest.

A related concern is the need for critical research and theorising to evolve fast if we are to shape and influence the field, especially given the emergence of “Big EdTech” (Brown, 2021) with powerful new actors attempting to accelerate the pace and scale of change. With the objective of helping to assemble new toolboxes for critical research and innovative approaches, the article identifies several current gaps and under-researched issues. It introduces a valuable collection of papers that offer exemplars and useful guiding beacons for future lines of research and development.

No 5 – Tamim, R. M., Borokhovski, E., Bernard, R. M., Schmid, R. F., Abrami, P. C., & Pickup, D. I. (2021). A study of meta-analyses reporting quality in the large and expanding literature of educational technology. Australasian Journal of Educational Technology37(4), 100–115. https://doi.org/10.14742/ajet.6322

This article makes a valuable contribution by recognising that not all meta-analyses of educational technology are equal. In recent years, rapid reviews, scoping reviews, systematic literature reviews and other variations of meta-analyses have become increasingly common with many featuring in our list of “good reads” partly due to the selection criteria. However, we have become wary of a tendency towards technicist or instrumentalist approaches to these major literature reviews as they often lack a framing theoretical perspective and deeper level of critique.

Indeed, the reader may find it difficult to judge their true value and contribution to the field due to increasing techno-sophistication effects in the way the findings are presented using new data analysis software. In reviewing 52 meta-analyses in the field of educational technology over almost 30 years using the Meta-Analysis Methodological Reporting Quality Guide (MMRQG), the authors confirm our own suspicion that many studies meet only moderate levels of quality. Thus, there is a risk that this type of analysis potentially misleads researchers and practitioners alike. Accordingly, we suggest that anyone embarking on a literature review in the field should familiarise themselves with this article.

We also suggest that many educators and researchers, including current and prospective doctoral candidates, would benefit from delving more deeply into the literature on how to undertake a quality meta-analyses. Three recent publications are worthy of slow reading: i) Systematic reviews in educational research: Methodology, perspectives and application (Zawacki-Richter, et al., 2020); Rapid reviews as an emerging approach to evidence synthesis in education (Wollscheid & Tripney, 2021); and iii) The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews (Steel, Beugelsdijk & Aguinis, 2021).  

No 6 – Jiménez‐Cortés, R., & Aires, L. (2021). Feminist trends in distance and hybrid higher education: a scoping review. International Journal of Educational Technology in Higher Education18:60, 1-20.https://doi.org/10.1186/s41239-021-00297-4

This article is the first in a sequence of four major literature reviews exploring different aspects of the field. It appears first in this sequence in light of our previous concerns about the growth of descriptive analyses of the literature lacking a deeper level of critique and theoretical perspective. Few could argue that feminist perspectives have featured prominently in the educational technology literature and there is growing evidence of what is known as the glass escalator in terms of how women are disadvantaged in the field, as Bond et al. (2021) recently illustrate in their survey of U.S. based instructional designers.

Therefore, this article reporting a review of 160 journals and 10 articles that meet the inclusion criteria is a welcome contribution to the field. Notably, the search protocol, methodology and review process over three phases is guided by the PRISMA-SCR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews), which helps to enhance the trustworthiness of data extraction and interpretation. Findings in terms of distance and hybrid models of education are reported based on three feminist approaches: i) feminist-pragmatist, ii) eco-dialogical feminist, and iii) intersectional-technofeminist. Those unfamiliar with feminist theories and pedagogical research will find these three perspectives relatively accessible and a real strength of the article is its description of how feminist digital pedagogy goes beyond critical digital pedagogy.

Importantly, the authors argue that digital enhanced teaching and learning platforms can limit feminist responses and reproduce dominant structures that reinforce existing power relations. While the article serves to highlight networked learning ideologies that from a feminist approach value connections, relationships, and collaborations, it also illustrates contrasting perspectives and their relationship to other theories concerning materiality, embodiment and hierarchies of power. Thus, the article flags an important gap in the literature and helps to border cross with other theoretical developments which have practical implications in terms of creating a more equitable and inclusive digital education ecosystem. 

No 7 – Bozkurt, A., & Zawacki-Richter, O. (2021). Trends and Patterns in Distance Education (2014–2019): A Synthesis of Scholarly Publications and a Visualization of the Intellectual Landscape. The International Review of Research in Open and Distributed Learning22(2), 19-45. https://doi.org/10.19173/irrodl.v22i2.5381

This article builds on several previous studies by the same authors exploring trends and patterns in distance education. Accordingly, in reviewing this article we had to ask the question: what makes it different from previous work and how does it contribute to new knowledge? Whether the six journals chosen for this analysis are truly representative of the field is open to conjecture, but two aspects standout in terms of answering this question. Firstly, the authors demonstrate how social network analysis (SNA) can be used to reveal new insights into a field.

More specifically, they model how researchers can deploy powerful data analysis software to undertake t-SNE analysis and content and co-word analysis using text mining techniques to visually identify and illustrate patterns, pivotal contributions and turning points. The underlying assumption that this type of visual analysis of the landscape has potential to better inform future research and development in the field is sound.

The second valuable contribution of this visual analysis is evidence of how since the 2000s, the fields of distance education and educational technology have intersected and triggered innovations in each other. Inclusion of more journals from the wider educational technology literature and mainstream publications would provide a means of further validating this type of cross fertilisation or symbiotic relationship. Also, greater consideration of what is missing from the published literature in terms of the issues raised by Castaneda and Williamson (2021) would further enhance the value of this type of analysis.

Drawing on their findings the authors conclude with recommendations for future research directions. Like several other articles in this year’s selection of “good reads”, they ask what our research agenda will and should be in the changing world. 

No 8 – Liu, C., Zou, D., Chen, X., Xie, H., & Chan, W. H. (2021). A bibliometric review on latent topics and trends of the empirical MOOC literature (2008–2019). Asia Pacific Education Review22 (3), 515-534. https://doi.org/10.1007/s12564-021-09692-y

This article ensures that we do not overlook the continuing influence of MOOCs on the education and digital learning landscape. As Bozkurt (2021) suggests in a separate analysis of the literature using data mining and analytic approaches, the MOOC continues to evolve and as new waves emerge their impact on traditional models and methods of education warrant ongoing investigation. We chose this particular bibliometric review of over 1,000 peer-reviewed MOOC studies published between 2008 and 2019 as it helps to bring an Asian perspective to the literature.

Notably, several of the research questions seek to investigate differences and collaborations across countries/regions, which gives further weight to the global perspective we hoped this article might provide. Not surprisingly, the review extracted its data from three reputable and influential publication databases, which unfortunately have a strong English language bias. This is something we have increasingly recognised over the years and places linguistic boundaries around our thinking and opportunities to share and co-construct new knowledge.

Mindful of this limitation and that many of us are unable to read what is published in other languages, the authors report that 11 countries contributed 81% of the total MOOC publications. The US (n=266) was the most prolific, with the UK (n=103), Spain (n=116) and China (n=172) featuring prominently in the literature. In terms of the latter, we know through our NIDL research collaborations with Chinese partners that the MOOC movement has generated an active local research community. To the author’s credit, some of this community and the level of global collaboration is revealed through social network analysis identifying scientific activity across countries/regions. The US collaborated with the most countries, followed by Spain, the UK, the Netherlands, China, Australia, and Germany. Ireland also features in this analysis, which is further illustrated through a visual representation of the most prolific institutions.

The main research topics and trends are also analysed at a country and institutional level helping this study to go beyond previous MOOC literature reviews. Thus, the authors generally meet their claim to have provided a deep and comprehensive understanding of current MOOC research up until 2019, which, in turn, builds on other reviews and supports future research.

No 9 – Tlili, A., Burgos, D., Huang, R., Mishra, S., Sharma, R. C., & Bozkurt, A. (2021). An analysis of peer-reviewed publications on Open Educational Practices (OEP) from 2007 to 2020: A bibliometric mapping analysis. Sustainability, 13, 10798, 1-15. https://doi.org/10.3390/su131910798

This article is the fourth major literature review featuring in this year’s selections with a focus on Open Education Practices (OEPs). The study reports a bibliometric mapping analysis of 156 publications listed in the Scopus and Web of Science databases that meet the inclusion criteria for OEPs. Importantly, the authors begin the paper by making a distinction between OEPs and more content-centred Open Educational Resources (OERs). The authors reveal how there has been a steady growth in OEP literature, with a peak in 2020. In light of the above comment about English language bias in research publication databases, perhaps not unexpectedly, 88% of the studies were in English, 6% were in Spanish, with 2% or less in French, Portuguese, Korean, and Russian.

The study identifies the leading journals publishing work in this area, with Distance Education contributing the most articles (n=18), which is ironic since this prestigious journal published by ODLAA is not fully open. Having said that, a special issue on OEPs does help to explain the journal’s position. The UK leads the country ‘league table’ with 38 documents followed by Spain (n=21) and Australia (n=20). The influence of North America on the OEP literature is better reflected when the tally for the US and Canada is combined as together they contribute 30 documents. Ireland is 10th in the list of countries with 4 documents, which is somewhat surprising given the level of interest in this area, although this number may reflect more of a content-centred emphasis on OERs.  

While the study shows on-going international cooperation regarding OEP amongst leading institutions and the contribution of several well-known scholars, the scale of global collaboration is limited. We believe this is an important finding. It would be interesting to compare the level of global collaboration between the OEP and OER communities, although many of the leading scholars work across both communities; and there is a question of where does the “Open Pedagogy” community ‘fit’ or map in terms of this type of relational analysis? We suggest the article needs to be read alongside a recent literature review on Open Pedagogy (Tietjen & Asino, 2021) as this is another branch of the literature.

The article is generally methodologically sound and helps to demonstrate how VOSviewer software can be deployed to undertake this type of bibliometric analysis. However, in light of the above comments, we would have liked a little more from the article. It offers limited analysis and critical insight into the competing drivers and attractors underlying the OEP movement. Given the thorny issue of definitions, different branches of the literature, and the politics of Openness, the question left unanswered is how do different OEP viewpoints, perspectives and understandings influence what is being researched, who is doing the research and what is being publishing in this area. It would also be interesting to analyse how the OER and Open Pedagogy literature is influencing OEP, and back again, based on citations and mapping the publication landscape.

Overall, the findings provide useful suggestions for future research and it is hard to disagree with the call for more inclusive practices that promote efforts to support and accommodate students with disabilities. In a similar vein, the value of focusing on cultural differences in education to internationalise OEPs makes good sense if we are to advance the United Nation’s Sustainable Development Goals and ensure that everyone benefits from the openness movement.

No 10 – Lodge, J. M., Corrin, L., Hwang, G.-J., & Thompson, K. (2021). Open science and educational technology research. Australasian Journal of Educational Technology37(4), 1–6. https://doi.org/10.14742/ajet.7565

This final article has a strong focus on the quality of research informing decisions and the direction of new developments in educational technology. Borrowing from other fields the authors describe the ‘replication crisis’ that helps to reveal questionable research practices (QRPs) and the robustness and trustworthiness of quantitative research across many disciplines. They suggest one of the greatest problems is that many published works are low powered—that is, the research fails to establish whether an effect is occurring at greater than chance levels.

Another concern is that of hypothesising after the results are known (HARKing), although this is difficult to identify based on a published article alone. Serving as an editorial, the paper goes on to explore what the replication crisis means for educational technology research with suggestions for future responses by the academic and professional community. While the authors observe that educational technology research does not rely as much on experimental designs, there is often a degree of intervention which then involves the study of effects raising inherent questions of replicability and generalisability. Although not singled out by the authors, the novelty of any intervention, often described as the ‘Hawthorn Effect’, also needs to be taken into account.

Overall, this is a novel and timely critique of the published research where the authors conclude that ‘Low powered studies are common’ (p. 3). Poor statistical practices characterise the field and there is evidence of widespread publication bias. Looking to the future, the article promotes the benefits of greater transparency as part of the ‘Open Science’ movement and renewed emphasis on statistical practices.

While it is hard to disagree with the concluding statement that quality must take precedence over what attracts the most clicks, perhaps future editorials could give more attention to the contestable nature of what counts as quality in educational technology research. After all, the idea of replication has Positivist undertones and can be interpreted from many different research perspectives. 

What else is worthy of reading? 

In this final section we identify five additional articles that are worthy of slow reading if you have not already come across them. In alphabetical order we highly recommended the following articles. 

Photo by Dmitry Ratushny on Unsplash

Dell, D. (2021). Resonance and current relevance of IRRODL Highly-cited articles: An integrative retrospective. The International Review of Research in Open and Distributed Learning22(1), 243-258. https://doi.org/10.19173/irrodl.v22i1.5315

  • This article which notably is written by a current doctoral candidate at Athabasca University provides a useful summary of the top 100 highly cited publications from past issues in this leading open access journal. 

Murphy, M. (2021). Belief without evidence? A policy research note on Universal Design for Learning. Policy Futures in Education, 19 (1), 7–12. https://doi.org/10.1177/1478210320940206

  • This article might not be to everyone’s taste, but it asks some important questions about the quality of evidence supporting the growing adoption of UDL principles in both policy and practice. 

Guzman‑Valenzuela , C., Gomez‑Gonzalez, C., Rojas‑Murphy Tagle, A, & Lorca‑Vyhmeister, A. (2021).  Learning analytics in higher education: a preponderance of analytics but very little learning? International Journal of Educational Technology in Higher Education18:23, 1-19. https://doi.org/10.1186/s41239-021-00258-x

  • This article helps to keep the rapidly evolving area of learning analytics to the fore of thinking and offers a valuable critical perspective through an analysis of 385 papers by showing that much of the focus to date has been more on analytics than on learning.

Moon, J., & Park, Y. (2021). A scoping review on Open Educational Resources to support interactions of learners with disabilities. The International Review of Research in Open and Distributed Learning22 (2), 314-341. https://doi.org/10.19173/irrodl.v22i1.5110

  • This paper through a scoping review of the literature responds to the challenge presented in one of our top 10 selections to promote more inclusive practices that support students with disabilities.

Nkomo, L., Daniel, B., & Butson, R. (2021). Synthesis of student engagement with digital technologies: a systematic review of the literature. International Journal of Educational Technology in Higher Education18:34, 1-19. https://educationaltechnologyjournal.springeropen.com/articles/10.1186/s41239-021-00270-1

  • This article adopts a tripartite model to provide a systematic review of the literature on the degree to which social media, video, and collaborative learning technologies have supported student engagement over the past decade. 
Where are the gaps?

We conclude with a brief comment on what’s missing from this year’s list as arguably the gaps are just as important as what we choose to profile. It’s simply not possible to include everything and so we acknowledge there are valuable publications offering a synthesis of the literature on issues of qualitycreativitywellbeingteacher competencemobile learningblockchainimmersive VRvirtual laboratoriesuse of video, the growth of surveillance and online exam proctoring, and the unbundling movement, to name a few.

This last topic also serves to illustrate  that the abundance of open access journals does not diminish the need to monitor the literature that often is only available in restricted publications.

Therefore, we finish by drawing your attention to one such article on this topic that makes good reading if your institution provides access to the journal: 

References

Bond, J., Dirkin, K., Tyler, A.J., & Lassitter, S. (2021). Ladders and escalators: Examining advancement obstacles for women in instructional design. Journal of Applied Instructional Design,https://edtechbooks.org/jaid_10_2/examining_advancemen

Bozkurt, A. (2021). Surfing on three waves of MOOCs: An examination and snapshot of research in Massive Open Online Courses. Open Praxis13 (3), 296–311. DOI: http://doi.org/10.5944/openpraxis.13.3.132

Brown, M. (2021). What are the main trends in online learning? A helicopter view of possible futures. Asian Journal of Distance Education, 16 (2), http://asianjde.com/ojs/index.php/AsianJDE/article/view/605

Brown, M., Conole, G., & Beblavỳ, M. (2019). Education outcomes enhanced by the use of digital technology: Reimagining the school learning ecology.  EENEE Analytical Report No. 38 Prepared for the European Commission. March.

Facer, K., & Selwyn, N. (2021). Digital technology and the futures of education – towards ‘non-stupid’ optimism.Background paper for the Futures of Education initiative, Paris: UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000377071.locale=en

Steel, P., Beugelsdijk, S. & Aguinis, H. (2021). The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews. Journal of International Business Studies 52,23–44. https://doi.org/10,.1057/s41267-020-00385-z

Tietjen, P., & Asino, T. I. (2021). What is open pedagogy? Identifying commonalities. The International Review of Research in Open and Distributed Learning22 (2), 185-204. https://doi.org/10.19173/irrodl.v22i2.5161

Wollscheid, S., & Tripney, J. (2021). Rapid reviews as an emerging approach to evidence synthesis in education. London Review of Education, 19 (1), https://ucl.scienceopen.com/hosted-document?doi=10.14324/LRE.19.1.32

Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., & Buntins, K. (eds.). (2020). Systematic reviews in educational research: Methodology, perspectives and application.Wiesbaden, Germany: Springer VS. 

Good Reads from 2021: Our NIDL Top 10 Journal Articles – Part 2

This is the second blog post in a series of three introducing our NIDL top 10 “good reads” for 2021. We will also be sharing next early week a separate list of “good reads” in the area of digital learning that have a specific COVID-related focus. In our first blog post, we established why professional reading matters, the value of taking time to do more slow or critical reading, and why it’s important to more intentionally think about who, what, where, when, why and how we read. 

Photo by Kirill Sharkovski on Unsplash

In this post, we explain why we originally chose to focus on open access journal articles, how the publication landscape is evolving and the process we follow in selecting our annual list of top 10 “good reads”. 

Why Open Access?

The original decision to limit article selections to open access journal publications was primarily a reflection of our commitment to openness and more specifically Open Science. Another consideration was to help profile the growing number of open access journals, which at the time Perkins and Lowenthal (2016) claimed were around one-third of an estimated 270 journals in the field of Educational Technology.

Given that no one person could be expected to read all of these journals, a secondary intention was to help people to recognise and better understand which open access journals are worthy of following and most likely to publish quality articles.

A related goal was to model and help to make more explicit the critical searching and reading strategies that experienced educators and researcher use to interrogate the literature. Copyright constraints and accessibility issues were also factors around our ability to share links to restricted journal publications that people at other institutions may not have been able to access. Hence, we chose to focus only on articles available in open access publications.

However, over the years our original definition of openness has become more challenging to apply and manage as traditionally closed journals increasingly adopt a hybrid publication model – that is, there has been a growing trend towards including a mix of open and closed articles in each issue. In some cases, early release articles are available in an open format before being restricted in a published issue and/or authors are given the rights to share a limited number of open copies.

Photo by Jan Huber on Unsplash

Thus, the openness boundaries have become blurred making our selection process far more demanding and time-consuming to ensure that we do justice to the exercise.

How are open access journals evolving?

The open access publishing ecology is continuing to evolve. We are discovering that open is not forever, as Laakso, et al. (2021) illustrate in their study of vanished open access journals. Notably, they found 174 open access journals that disappeared from the Internet between 2000 and 2019, across all major disciplines and geographic regions.

Three recent examples from the field of Educational Technology further illustrate this point. In 2017, the Journal of Learning Design closed after being launched back in 2005. After 16 years of publication, the Journal of Interactive Online Learning closed in 2018. And first launched in 1999, the most recent issue of the Malaysian Journal of Distance Education was back in 2019. On a related point, wherever possible, we endeavour to provide a link to the archives of closed journals on our comprehensive NIDL list of scholarly journals in the field. 

On a more positive note, new journals continue to emerge in response to new areas of development.

For example, in 2019 the International Journal of Open Educational Resources was launched in response to the growth of the OER movement. Also in the 2019, the inaugural issue of Postdigital Science and Education was published to help move the current thinking ‘beyond digital’. This new journal has already made a significant contribution to the field and once again features in our selections. Another new open access publication has emerged over recent months with the OTESSA Journal welcoming papers on all aspects of educational technology, with a strong Canadian focus.

One of the side benefits of producing our annual list of “good reads” is that it helps us to keep up to date with journal developments and refresh our webpage of online journals.

Lastly, we would like to congratulate Open Praxis and the European Journal of Open, Distance and E-Learning for major upgrades to their host platforms in 2021, which greatly enhance the ‘look and feel’ of each publication.

Bibliometrics data for AJET

Another trend we have noticed over the past six-years accompanying platform enhancements is increased sharing of journal metrics, with most of the better-known open access publications displaying the number and frequency of views and downloads. In some cases, Editors identify the most popular articles and publish more detailed analytics providing comparative data with previous years. For example, the Australasian Journal of Educational Technology regularly shares this information, as contained in the latest Editorial which reports bibliometrics in terms of submissions, readership interest, views and download numbers, and review statistics by year. 

This is a practice that we strongly encourage and adopt in our association with the International Journal of Educational Technology in Higher Education, which the NIDL formally supports as one of four institutional partners. We greatly value our role in helping to increase the profile and impact of this Springer journal, which is now ranked in 11th place out of a total of over 1,500 journals in the Scopus database of international scientific publications for the Education and Computer Science categories.

The journal is now the number 1 ranked open access publication in the field. Notably, in 2021, over 1.6 million articles were downloaded from the journal website, up from 1.088 million in 2020.

This level of usage illustrates how open access publications have grown in status and credibility over the past decade. Onwards and upwards for open access but there is still the question of whether we are doing enough slow reading as we increasingly download all of this new literature.

How do we select our top reads?

Before we get to the business end of sharing our “good reads” for 2021, it is useful to revisit the selection process and several methodological points. Our selection of top 10 articles for 2016201720182019 and 2020 has generally followed the same selection process. This five-step process remains largely unchanged for 2021 using the same selection criteria.

Photo by Zan on Unsplash

To recap, these criteria privilege major literature reviews, think pieces presenting new perspectives, papers reporting new and emerging areas of development, and/or articles addressing important gaps in the literature.

We are always looking for something offering a fresh or different perspective which challenges conventional thinking. 

Diversity is another important criterion. And we automatically exclude anything written by one of our team and are careful to limit the number of articles published in the above mentioned journal that the NIDL formally supports. 

In terms of the selection process, we try to promote an inclusive approach open to all members of the NIDL team. Firstly, a shared Google folder allows colleagues to upload their own nominations at any point during the year. We have a practice of frequently sharing “good reads” through a mailing list to alert people to specific publications of potential interest, which also supports a culture of professional reading. 

Secondly, towards the end of the year several reminders are sent to NIDL colleagues to encourage their nominations, which usually results in a reasonable number of suggested articles. Importantly, at this stage of the process, nothing is rejected until the article has been more thoroughly read and critically reviewed by one of the three members of the core selection panel. 

Thirdly, in the latter part of December, the list of journals appearing on the NIDL website is systematically reviewed to ensure that nothing has been missed from the initial long list of publications.

This process can take several days and usually throws up some overlooked articles and interesting reads from lesser-known journals. It also helps to identify open articles published in traditionally closed or restricted access journals.

Over the years we have found that many journals do not publish their final issue until December and so this final ‘casting of the net’ allows us to accommodate late nominations in the selection process. In terms of meeting the criteria of openness, as a rule, if the article is freely available at the time of publication and can still be downloaded at no cost or with no restrictions by the end of December, then it is deemed eligible for inclusion.

Fourthly, the long list of articles, which this year excluding COVID-related publications totalled 85 papers, is reviewed by at least one member of the core selection panel and evaluated against the criteria to generate a short list. Usually, around 25 articles make it to this stage of the process.

This is a time-consuming process as it usually takes a minimum of one hour to slow read and write notes on each article. 

Finally, after closer reading and analysis, and often debate and even disagreement, the short-list is refined to identify the final top 10 “good reads” along with an additional five highly recommended articles. The latter is a relatively new feature as we have found it increasingly difficult to reduce the list to just 10 articles.

Also, it is important to note that not everyone’s “pet” read makes the final cut.

Ultimately, to ensure our selections are finalised and ready to share early in the New Year, the NIDL Director has executive privilege to make the final decision of what is included in the list. For this, and reasons previously mentioned in our first blog post in this series, we do not claim that our NIDL top 10 articles are unequivocally the best “good reads” in the area of digital learning over 2021. 

What constitutes a “good read” remains open to debate. However, we are willing to standby and defend this year’s selections that we reveal in the third and final blog post in this series. They reflect a great deal of thinking based on a explicit criteria, a robust methodology, systematic interrogation of the literature and lots of close reading to find what we believe is worthy of read irrespective of whether you have a policy, research, professional and/or practitioner interest in the field. 

References

Laakso, M., Matthias, L., & Jahn, N. (2021). Open is not forever: a study of vanished open access journals. Journal of the Association for Information Science and Technology, 1-14. https://arxiv.org/abs/2008.11933

Perkins, R., & Lowenthal, P. (2016). Open access journals in educational technology: Results of a survey of experienced users. Australasian Journal of Educational Technology, 32 (3). 18-37. https://ajet.org.au/index.php/AJET/article/view/2578/1358