Author Archives: David Jenkins

Research 4.0: Research in the age of automation

On Monday, I was at the LIS-Bibliometrics 10th anniversary conference, “The Future of Research Evaluation”.

Dr Steven Hill* and Prof James Wilsdon**, highlighted the rise of AI in research evaluation as a major trend in their keynotes. This is particularly significant as they are two of the most prominent figures in research evaluation in the UK. They both cited the recent Research 4.0: Research in the age of automation interim report by Demos.

They pointed out how AI can already do/help out with research (e.g. discovering new antibiotics) and how AI can potentially revolutionise research evaluation. Regarding the latter, systems are being developed to:

  • generate research grant applications and evaluate them
  • spot relevant papers missed from citation lists
  • identify reviewers
  • identify journals to publish in
  • undertake actual reviews of research papers, perhaps as a complement to human peer review

It was noted that AI could potentially reduce existing biases in some of these procesesses but that they equally could introduce new biases or cement existing biases. There are also numerous issues with the transparency of AI.

The responsible use of metrics may well need to cover the responsible use of AI in future and the UK Forum for Responsible Research Metrics are already looking into this.

Please see Stephen Hill’s post about his talk (link to slides included) for more information. I will Share James Wilsdon’s slides when they become available.

*Director of Research at Research England, chair of the steering group for the 2021 Research Excellence Framework (REF)

** Digital Science Professor of Research Policy in the University of Sheffield’s Department of Politics, Director of the Research on Research Institute, chaired the independent review of the role of metrics in the management of the research system resulting in The Metric Tide report

Literature searching for postgraduate researchers – online training

We have a series of online training for OU postgraduate researchers, full details and sign-up information is below:

Literature Searching for Postgraduate Researchers 1 (online)

Mon, 7 October 2019

14:00 – 15:00 BST

Online access – via the Research Support online training room

This session involves reflecting on a model of the literature search process in order to (re)conceptualize literature searching, increase confidence with the process and assess the model in relation to your practice. We will then look at formulating and revising a search strategy in order to perform a systematic and comprehensive search – this includes choosing databases, choosing keywords and recording your searches.

Please note that you are required to undertake a brief exercise in advance of this session and be prepared to discuss your thoughts on the exercise in the session itself. Details of this exercise are on the booking page.

To book a place, please go to https://www.eventbrite.co.uk/e/literature-searching-for-postgraduate-researchers-1-online-tickets-73822925357

Literature Searching for Postgraduate Researchers 2 (online)*

Wed, 16 October 2019

15:30 – 16:30 BST

Online access – via the Research Support online training room

This session involves identifying techniques for narrowing and broadening searches and when to apply them in order to construct and revise a search strategy. We will then identify and reflect on means of saving and exporting search results, this will allow us to manage search results effectively and understand the benefits of doing so.

Please note that you are required to undertake a brief exercise in advance of this session and be prepared to discuss your thoughts on the exercise in the session itself. Details of this exercise are on the booking page.

To book a place, please go to https://www.eventbrite.co.uk/e/literature-searching-for-postgraduate-researchers-2-online-tickets-73824522133

Literature Searching for Postgraduate Researchers 3 (online)*

Mon, 21 October 2019

10:00 – 11:00 BST

Online access – via the Research Support online training room

This session involves analysing search results using the CRAAP framework in order to identify the most appropriate papers on a topic and revise your search strategy. We will then describe and apply a scoping search process in order to establish the extent of the literature that exists on a topic.

Please note that you are required to undertake a brief exercise in advance of this session and be prepared to discuss your thoughts on the exercise in the session itself. Details of this exercise are on the booking page.

To book a place, please go to https://www.eventbrite.co.uk/e/literature-searching-for-postgraduate-researchers-3-online-tickets-73825523127

 

*Note – Advanced literature searching 1, 2 and 3 are designed to complement each other. You are very welcome to attend (or watch the recordings of) whichever of the sessions you need but please note that familiarity with content from previous sessions will be assumed and won’t be recapped in detail.

We look forward to seeing you at the training!!

Postgraduate researcher (PGR) training survey results – 2019

The Library Research Support team recently undertook its annual survey to capture the training needs and communication preferences of postgraduate researchers (PGRs). We got a good response rate (19.3% of PGRs available). Here are some takeaways:

Over a quarter (25.6%) had not attended any OU training whatsoever, whether run by their faculty, the Graduate School, the Library or anyone else

Email was the most common way that PGRs heard about the training they attended (75.7%). Twitter (2.7%) and Facebook (0%) were not well used in this regard

A clear majority felt Library training met their learning needs (89.2%), had clear learning outcomes (89.2%) and included sufficient interactivity (75.6%)

PGRs who attended Library sessions using flipped learning felt it improved their understanding of the topic (86.3%), led them to reflect more about the topic than they would have done otherwise (81.8%) and led to more efficient use of session time (81.8%). Although, fewer (68.2%) felt it enabled the group to discuss issues in more depth

Additional training needs were diverse and the majority were in areas outside the Library’s traditional remit – Nvivo, presentation skills and academic writing were the most requested training topics

 The results will inform the planning and communication of Library Research Support training in the 2019/20 academic year as we continually develop a tailored programme in response to PGR feedback. We are also sharing findings with colleagues in relevant departments/faculties.

“Measuring research: what everyone needs to know” – new ebook available

Want to know more about quantitative research indicators (a.k.a. bibliometrics) such as Journal Impact Factor, h-index or altmetrics? We have a new ebook in stock, containing thorough discussion of this field, the pros and cons of various indicators and the future of measuring research:

Sugimoto, C. R. and Larivière, Vincent (2018) Measuring research : what everyone needs to know(OU login required)

It also answers questions such as:

  • What are the data sources for measuring research?
  • How is impact defined and measured?
  • How is research funding measured?
  • What is the relationship between science indicators and peer review?

 

“Getting the most out of your doctorate” – new ebook available

We’ve just procured a new ebook at the request of academic staff:

Dollinger, M. (2019) Getting the most out of your doctorate : the importance of supervision, networking and becoming a global academic. First edition. (OU login required).

It focuses on the importance of networking and building relationships as a postgraduate researcher or early career academic:

Beyond the doctoral thesis itself, the most significant factors in the progression of PhD candidature and early academic careers are: the relationships between the researcher and their supervisor(s), the ability to network, and understanding one’s place in the global research arena. Navigating these critical factors and moving from a novice to expert, is a critical undertaking for every PhD candidate and a process that will continue for years following one’s PhD. In this book, scholars from around the world offer practical advice on how to get the most out of one’s doctorate. Readers will get helpful tips on how to sustain healthy and long-lasting relationships with their supervisors, learn how to develop their networks, and understand the important changes impacting the modern PhD candidate.

Edited by Dr Mollie Dollinger (Higher Education Researcher at La Trobe University, Australia) it features contributions from a wide variety of academics, including our very own Prof. Bart Rientes.

Investigating signing DORA in response to funder policy changes

Adopting a responsible metrics approach is seen as good practice
across the research community.

However, there is now an additional need for The Open University to sign up to an
external responsible metrics statement, such as the San Francisco Declaration on Research Assessment (DORA) or the Leiden Manifesto, or develop one of its own. Certain
major funders have changed their policies, which could impact our eligibility to receive research funding:

“We [The Wellcome Trust] are committed to making sure that when we assess research outputs during funding decisions we consider the intrinsic merit of the work, not the title of the journal or publisher.

All Wellcome-funded organisations must publicly commit to this principle. For example, they can sign the San Francisco Declaration on Research Assessment, Leiden Manifesto or equivalent.

We may ask organisations to show that they’re complying with this as part of our organisation audits.”

(The Wellcome Trust, 2019)

 

“cOAlition S* supports the principles of the San Francisco Declaration on Research Assessment (DORA) that research needs to be assessed on its own merits rather than on the basis of the venue in which the research is published. cOAlition S
members will implement such principles in their policies by January 2021.”

(cOAlition S, 2019)

* cOAlition S is a group of funders co-ordinated by Science Europe. It includes UKRI, Wellcome, the European Research Council (ERC), the European Commission and The Bill and Melinda Gates Foundation. They are responsible for Plan S, a radical proposal regarding open access to research publications from which the above quote is taken

 

The Library Research Support team recently brought a paper to Research Committee, which investigates the University’s options in terms of responding to these policy changes. We are looking into how publication metrics are used at The OU and whether any current practices are in tension with these policy changes. The aim is that, all being well, The Open University will look at signing DORA.

We will keep you updated on our progress and would welcome any feedback on this issue.

 

References

cOAlition S (2019) Plan S: Principles and Implementation. Available at: https://www.coalitions.org/principles-and-implementation/ 

The Wellcome Trust (2019) Open access policy 2021. London: The Wellcome Trust. Available at:
https://wellcome.ac.uk/sites/default/files/wellcome-open-access-policy-2021.pdf

New academic writing mini-collection

We have a new mini-collection of books on academic writing, purchased on the recommendation of PACE (Professional and Academic Communication in English) in order to support postgraduate reseachers.

There are a handful of print books, which are housed on the second floor of the Library building alongside the research methods print books (near The Park). They are available on a reference basis (i.e. they cannot be taken out of the library) in order to maximise the number of people who can use them.

There are also a number of ebooks, which you can access online:

We will be adding a few more titles to this collection in the near future.

We hope you enjoy using them and do feed back any comments regarding the collection!

New research methods print books

In response to feedback from postgraduate researchers, we now have a collection of over 70 print research methods books.

They are housed on the second floor of the Library building (near The Park) and are available on a reference basis (i.e. they cannot be taken out of the library) in order to maximise the number of people who can use them.

These titles were recommended by postgraduate researchers and cover many topics, including:

  • literature searching
  • writing research proposals
  • surveys
  • SPSS
  • ethnography
  • qualitative data analysis
  • mixed methods research

There are specific titles addressing subject areas such as education, business, management and the social sciences more broadly.

We hope you enjoy using them and do feed back any comments regarding the collection!

Public speaking and presentation skills for early career researchers

The Charlesworth Group, a publishing services company, are running a webinar on public speaking and presentation skills for early career researchers.

It’s on Tuesday July 30th @ 10:00am or @ 14:00pm BST and you can get more info and sign up here on their webinar schedule for the year:

https://www.cwauthors.com/article/webinar-schedule-2019

 

What is a systematic review and how does it differ from a ‘regular’ literature review?

There are a lot of different types of literature review and there is a lot of different terminology surrounding literature reviews.

This creates confusion and there is a particularly large amount of confusion regarding systematic reviews. The term, strictly speaking, refers to a specific and particularly rigourous method that has its origins in biomedicine and healthcare (although it is adapted and used in other disciplines). However, many people use the term to refer to a ‘regular’ literature review that is methodical and comprehensive.

In short, if somebody asks you to carry out a systematic review, it is worth clarifying exactly what they have in mind.

Here, we will spell out the differences between ‘regular’ literature reviews and systematic reviews as we see them:

‘Regular’ literature reviews

A regular literature review involves finding, analysing and synthesizing relevant literature, then presenting it in an organised way to the reader.

Regular literature reviews can be methodical and comprehensive. They can involve attempting to find all the literature there is on a topic, recording results and reflecting on strategies. We could even describe them as being “systematic” in an informal way but they do not employ the full formal methods of a systematic review, as outlined below.

Systematic reviews

In biomedicine and healthcare a systematic review aims to be exhaustive, objective, transparent and replicable, employing specific methods to reach these goals. It typically involves stages such as:

  • Creation of a structured research question to guide the process
  • Writing a protocol or following a previously established protocol, which sets out the methods the systematic review will use
    • A protocol covers things like which databases will be used, why they will be used, what keywords will be used, what other search techniques will be used. The protocol is usually developed through testing and is often peer-reviewed
  • A methods sections, including:
    • A list of all databases and/or journals that were searched
    • The exact keywords, limiters etc. that were used
    • When each search was undertaken
    • How many results each seach found
  • The titles and abstracts of articles found are compared against inclusion criteria
  • Meta-analysis may be undertaken
    • In this context, meta-analysis refers to the statistical analysis of data from comparable studies
  • Reporting on the results of all included studies, highlighting any similarities and differences between them

A systematic review is often preceded by a scoping review, a relatively brief search of relevant databases, which aims to tell you whether your research question, in its current form, is worth pursuing or whether it needs changing. This a process tells researchers whether a recent or ongoing review of the topic already exists – if it does then a new systematic review may not be necessary.

The description above is necessarily brief and partial. We recommend that you consult guidance such as that produced by the Centre for Reviews and Dissemination (CRD) for a fuller explanation of how systematic reviews work in biomedicine and healthcare.

As mentioned, the systematic reviews method has been adapted by other disciplines. For example, the Campbell Collaboration have adopted the method, defining systematic reviews and producing guidance with a focus more on the social sciences. There are also books (e.g. this book we have in print at the Library) and articles (e.g. this article which is open access) on systematic reviews in the social sciences.

If you want to know more about systematic reviews, you can also watch the recording of the online training session by Library Services (OU login required).