ce Rebecca's Research Blog » Rigour in journal articles

Rigour in journal articles

These points are taken from a talk by Sara Hennessy about assessing rigour in papers submitted to the British Journal of Educational Technology.

  • Is the account analytical or purely descriptive?
  • Do the authors critique the literature and present a balanced account or does the review gloss over known issues with educational technology initiatives?
  • How do the theoretical assumptions and explanations of the case compare with alternative explanations?
  • Are links made in the discussion/conclusions back to the conceptual framework?
  • Are the findings critically interpreted in relation to the existing literature?
  • Do the research questions convey genuine inquiry or do they assume that educational technology is a Good Thing?
  • What is the theory of change?
  • Were there any potential sources of bias?
  • What measures were taken to counter them? eg were any counter examples sought when collecting and analysing data?
  • Does reporting seem selective? Are the analyses systematic and explicit? Is the account reflective and evaluative?
  • Was there any kind of control in a technology intervention design?
  • Are there threats to validity and reliability? Does inadequate control of extraneous factors threaten validity of theoretical inferences from data?
  • Are limitations acknowledged?
  • Could reactivity have played a role? Did novelty value of shiny new tools increase motivation?
  • How robust are the claims – how adequately is the argument supported by the evidence provided?
  • Does the sampling strategy permit empirical generalization to a larger population?
  • Who is the audience? Is the work relevant and current?
  • Are there clear conclusions that generalized beyond specific case/context? Do they apply in other institutions?
  • Is the work applicable / of interest in other countries? Is the focus overly parochial?
  • Is educational technology serving only the privileged in developing countries who already have access?
  • SES and gender inequity, rural/urban divide, language, computer literacy, bandwidth and intermittent connectivity / electricity maintenance, technical support, gatekeepers / stakeholders, culturally appropriate content….
  • Is the research innovative or, at least, original
  • What is the significance and contribution to existing knowledge – theory? methodology? empirical data?
  • Has the innovation been tested with real users?
  • Are there any convincing learning outcomes?
  • Are there explicit implications for practice? Policy?
  • Many reports of interventions motivated by ‘how can I use…’ rather than educational need
  • How is it used, by and with whom, how often, for what purpose, under what conditions, with what support…
  • What is the role of the teacher? Has pedagogy-focused professional development been offered? What cultural shift in teacher and learner roles is necessary?


Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>