These points are taken from a talk by Sara Hennessy about assessing rigour in papers submitted to the British Journal of Educational Technology.

• Is the account analytical or purely descriptive?
• Do the authors critique the literature and present a balanced account or does the review gloss over known issues with educational technology initiatives?
• How do the theoretical assumptions and explanations of the case compare with alternative explanations?
• Are links made in the discussion/conclusions back to the conceptual framework?
• Are the findings critically interpreted in relation to the existing literature?
• Do the research questions convey genuine inquiry or do they assume that educational technology is a Good Thing?
• What is the theory of change?
• Were there any potential sources of bias?
• What measures were taken to counter them? eg were any counter examples sought when collecting and analysing data?
• Does reporting seem selective? Are the analyses systematic and explicit? Is the account reflective and evaluative?
• Was there any kind of control in a technology intervention design?
• Are there threats to validity and reliability? Does inadequate control of extraneous factors threaten validity of theoretical inferences from data?
• Are limitations acknowledged?
• Could reactivity have played a role? Did novelty value of shiny new tools increase motivation?
• How robust are the claims – how adequately is the argument supported by the evidence provided?
• Does the sampling strategy permit empirical generalization to a larger population?
• Who is the audience? Is the work relevant and current?
• Are there clear conclusions that generalized beyond specific case/context? Do they apply in other institutions?
• Is the work applicable / of interest in other countries? Is the focus overly parochial?
• Is educational technology serving only the privileged in developing countries who already have access?
• SES and gender inequity, rural/urban divide, language, computer literacy, bandwidth and intermittent connectivity / electricity maintenance, technical support, gatekeepers / stakeholders, culturally appropriate content….
• Is the research innovative or, at least, original
• What is the significance and contribution to existing knowledge – theory? methodology? empirical data?
• Has the innovation been tested with real users?
• Are there any convincing learning outcomes?
• Are there explicit implications for practice? Policy?
• Many reports of interventions motivated by ‘how can I use…’ rather than educational need
• How is it used, by and with whom, how often, for what purpose, under what conditions, with what support…
• What is the role of the teacher? Has pedagogy-focused professional development been offered? What cultural shift in teacher and learner roles is necessary?