Selecting our Top 10 Articles: What was Worth Reading in 2017?

Over 2017 we have been keeping a record of particularly interesting articles in the general areas of Blended, Online and Digital (BOLD) Education. More specifically, since March we have maintained a folder of articles published in open access journals that might qualify later in the year for our 2017 list of “top 10” widely accessible reads. Our list of top reads will be shared progressively via Twitter over the next three weeks as we countdown to our No. 1 article.

Interesting Questions

It follows that this exercise, which builds on last year’s experience, raises some interesting questions:

  • What selection criteria do you adopt to help identify a really good (open access) journal article?
  • What selection methodology do you use to help identify the top 10 open access journal articles for the year?
  • Who do you involve in the selection process to help enhance the validity of the list of top 10 (open access) journal articles?

With these questions in mind the following comments are intended to help explain and frame this year’s selection.

top-95717_960_720

Our Selection Criteria

When we first began this task we didn’t have any clearly defined selection criteria. This was partly in the interests of promoting inclusion and recognition of the fact that we all have different interests and perspectives. However, as we began the wider nomination process and refined the short-listed articles we recognised the need to more explicitly anchor our selection in some guiding criteria. Accordingly, the final selection of the top 10 open access journal articles for 2017 were loosely guided or informed by the following criteria:

• Published in open access journals listed on NIDL website

• Restricted to Higher Education (inclusive of teacher education)

• Strong preference to journal articles with international focus or relevance

• Minor preference to journal articles published by professional associations

• Strong preference to journal articles offering major literature reviews

• Strong preference to journal articles addressing major gaps in the literature

• Minor preference to journal articles exploring new and emerging topics

• Strong preference to journal articles which challenge conventional thinking

• Minor preference to journal articles relevant to current NIDL’s projects

• Overall selection of top 10 journal articles reflects a mix of gender, cultural and geographical diversity

Our Selection Methodology

The formal methodology initially involved a nomination process open to 20 members of the NIDL team. A related internal objective of the task was to raise awareness and encourage team members to more deeply engage with the published literature. While a shared folder for the collection of top journal articles was available from the beginning of March 2017, the wider open nomination process didn’t begin until the start of November.

In addition to this open nomination methodology and the collection of interesting journal articles in a shared folder over the course of the year, the NIDL Director systematically went through the full list of open access journals available on our website to help identify specific journal articles published in 2017 which might qualify for inclusion in the short-list. Notably, although extensive, this list of journals does not include all of the 270 publications identified last year by Perkins and Lowenthal (2016) in their comprehensive analysis of open access journals. From this more selective list taken from our website a draft collection of top articles for the year was created using a shared folder in Google Drive, which contained copies of the original articles.

transparency-1938335_960_720Towards the end of November this list was refined to a draft list of 10 articles, which the aforementioned NIDL team members were then invited to rank in order of merit and wider readership value. During this time the draft top 10 list was relatively dynamic as the ranking process threw up additional articles and a handful of new journal issues were published during the month (e.g., AJET, IRRODL & EDUCAUSE Review). As a consequence, a handful of articles on the original list were replaced with late additions, which posed some challenges in the ranking process but we believe is evidence of the inclusive nature of our overall methodology.

Lessons from the Selection Process

Before we share (initially via Twitter) our final selection of top 10 open access journal articles for 2017 it is useful to reflect on several lessons arising from the process.

1. Blurring of boundaries

Firstly, there is an increasing blurring of boundaries between open and closed publications. During the selection process the question arose what constitutes an open access article?

One of our top 10 articles, for example, appears in a highly ranked closed journal published by Taylor & Francis which is managed by a professional association. The publisher now provides an open select service where the author(s) have the option of paying a fee to ensure downloads of their article are freely available. In this case we decided to include such publications for consideration in our list of top 10 as we wanted to recognise the authors commitment to openness and more widely disseminating their work.

Publications

The question of what constitutes an open access publication also arose with pre-print uploads of articles by authors to institutional repositories and websites such as Research Gate, especially when the published article also appears in a closed journal.

More specifically, this issue came up when we considered publications such as George Veletsianos’ article on who participates on MOOC hashtags and in what ways in trying to develop a generalizable understanding of Twitter and social media use. Although a pre-print version of the article is openly available from ResearchGate and George’s personal blog, it is formally published in the Journal of Computing in Higher Education which is a closed publication.

Similarly, on the theme of MOOCs, we had to consider how to handle a useful publication on designing Massive Open Online Courses to take account of participant motivations and expectations, which is available as a pre-print version on Gilly Salmon’s personal blog, when the final version of this article appears in the British Journal of Educational Technology.

In the end, after carefully reflecting on this issue, we decided to exclude such articles from our list; but the wider lesson is the way that some authors are strategically navigating and intentionally managing both open and closed spaces to help more widely disseminate their work.

2. Growth of review articles

Secondly, there appears to be a growing trend and increasing popularity towards the publication of review articles on topical issues following a systematic review methodology. For example, amongst the list of nominations we considered Krull and Duart’s (2017) article reporting a systematic review of research on mobile learning in higher education. Similarly, we also considered Liyanagunawardena and colleagues’ article reporting a systematic review of literature on open badges published in the European Journal of Open and Distance Learning. In addition, Mnkandla and Minnaar’s (2017) metasynthesis of the literature on the use of social media in e-learning was considered for inclusion given our preference for major review articles.

nigerianman

Also, with an interesting focus on how authors collaborate in written publications in the area of e-learning, we reviewed Mohammadi, Asadzandi and Malgard’s paper in the International Review of Research in Open and Distributed Learning analyzing articles in the Web of Science over the 10-year period.

While all of the above publications explore important topics and our final selection of top 10 articles for 2017 include a number of major literature reviews, the standout observation from evaluating this type of work is that not all review articles are created equally. Polanin, Maynard and Dell (2017) support this observation in their critical analysis of this line of research recently published in the Review of Educational Research where they report:

Despite their popularity, few guidelines exist and the state of the field in education is unclear (p. 172).

They go on to observe that such ‘overviews are a relatively nascent and undeveloped synthesis method that pose unique methodological challenges and may be problematic’ (Polanin, Maynard & Dell, 2017, p. 173). Building on this concern the literature is never neutral and the challenge is to critically interpret the body of published work and methodologies adopted based on explicit theoretical frameworks, which go beyond closet positivist methods simply describing what has been published. Not all of the review articles we include in our top 10 list fully address this point and we encourage readers to be critical of such publications.

3. Value of closed publications

Thirdly, despite the focus of this exercise being on open access publications, which is both philosophical and pragmatic as we want as many people as possible to read the articles we select, many of the so-called best articles (depending on your personal selection criteria) continue to feature in more traditional closed journals. Put another way, our list of top 10 reads for the year would be very different if we adopted an hybrid sample of both open and closed publications. This point begs the question, what might we have included or at least considered in our selection from a wider sample of more traditional closed or restricted journals?

This is a difficult question to answer without repeating a systematic selection methodology; however, for your interest we have listed below another 10 publications that we may have considered for this list, although it needs to be stressed that there are many other journal articles worthy of consideration and further evaluation depending on your specific interests:

Final Comment

Our final list of top 10 open access articles for 2017 come from just five well known journals. Partly by design, professional associations, with one notable exception, manage the majority of the journals. This exception is the International Review of Research in Open and Distributed Learning which continues to be ranked as one of the top journals in the field. After we have announced the full list of top 10 publications we intend to post to our NIDL blog a brief explanation for our selection of each article and further remarks on the publications missing from this list. Importantly, in the end the final selection is not intended to be a definitive list of the “top 10 reads” for 2017, as this very much depends on the eyes and interests of the reader. What we hope is that our list is helpful in confirming your own selections, challenges you to reflect on what is missing and/or alerts you to something valuable that you may have missed in your professional reading over the year.

Success of DCU-Fuse in Reimagining the Future

DCU-Fuse was a unique 24 hour online envisioning and collaborative brainstorming experience, which took place over 30th & 31st March 2017, in partnership with students to engage stakeholders in development of a new 5-year Strategic Plan (2017-2022). FuseThis innovative project, first of its kind in Europe, was framed by Dublin City University’s (DCU) mission of ‘transforming lives and societies’ and strongly anchored in five core principles of ethical leadership: inclusion, openness, collaboration, sustainability and transformation for better futures. The objective was to canvas the community for ideas using an innovative online platform during World Open Education Week to shape and reimagine the next phase of the University’s development.

The unique DCU-Fuse platform was structured to support 32 separate online conversations over 24-hours across four channels, covering a variety of themes, which engaged the DCU community in meaningful dialogue related to the University’s future. NIDL staff was actively involved in leading and contributing to these conversations. Almost 6000 separate online posts and contributions and over 7000 likes were shared, along with around 80,000 page views, over 24-hours. Importantly, the DCU-Fuse experience also established thriving physical hubs on each academic campus supported by students throughout the 24 hours.

3

A distinctive feature of the DCU-Fuse experience was the underlying conceptual framework designed to reflect the University’s mission, core principles of ethical leadership, and contemporary models of open, inclusive and transformative online learning. More specifically. DCU-Fuse integrated key elements from three seminal theories on learning technology innovation. Twitter Fuse

Firstly, Laurillard’s (2013) Conversational Framework informed the platform design and engagement model by ensuring rich feedback loops. Secondly, Garrison, Anderson and Archer’s (2000) Community of Inquiry Framework was influential in fostering a strong sense of presence and community amongst participants, augmented by rich interactive content. Notably, each conversation was facilitated by a member of the University community, with support from a team of moderators who participated in relevant training. Thirdly, Hattie’s (2015) concept of Visible Learning based on a synthesis of over 1200 meta-analyses informed the platform usability and structure of conversations to enhance the flow of activities over the 24 hour period.

The DCU-Fuse experience was also invaluable in leveraging the University’s wider digitalisation agenda. Put simply, DCU “walked the talk” by infusing technology throughout our institutional culture in a truly open, inclusive and transformative experience. As Niall Behan, the new incoming Students’ Union President, comments:

“DCUFuse brought large-scale town hall meetings to the 21st Century online environment. This level of commitment to ensuring all opinions are heard, is part of a holistic model everyone at DCU is proud of. By being self-reflective and inquisitive in this manner, all aspects of the student experience can be improved, creating an even better institution”.

The following presentation on DCU-Fuse experience was later presented in June by Professor Mark Brown at the annual European Distance and e-Learnng Network (EDEN) conference.