A time to review and reflect for eSTEeM

22nd June 2016

In April this year eSTEeM commissioned Professor Lynn Clouder, Director of the Centre for Excellence in Learning Enhancement at Coventry University, to carry out an evaluation of eSTEeM.  

It is concluded that eSTEeM is in an excellent position to consolidate its STEM leadership, influence and impact to achieve even greater prominence. The aim of this review was to assess the effectiveness and current vision of eSTEeM, addressing four specific questions posed by the eSTEeM directors. These are presented below with summary responses.  

1. How can we ensure that eSTEeM is ready for the external community with regard to its credibility and potential to initiate and contribute to UK and international STEM scholarship?

On the basis of formal documentation, interviews and observations at the 2016 eSTEeM conference, eSTEeM appears to be well placed in this regard. Specific recommendations are:

• increase emphasis on robust evaluation/research that demonstrates originality, reach and significance;
• evaluation/research strategy, should include synthesis of the findings across projects, to focus from individual case studies to more generalisable and transferrable contribution to knowledge across STEM;
• tenured academic staff and ALs should be supported to present at carefully targeted national and international conferences and to publish in high quality peer reviewed journals;
• externally funded projects and targeted collaborative networks will raise profile and will need same professional office support as conventional STEM research, including training in bid preparation;
• investment in PhD scholarships and visiting scholars, would move the research agenda forward whilst increasing impacts and outputs and enhancing research profile (for REF);
• create a more distributed leadership model.

2. What is the effect of eSTEeM in terms of impacting teaching and benefitting students?  Is it an effective use of their chief resource which is academic staff time?

Maintaining a centre for STEM education as an autonomous centre within the Faculty as a hub for STEM scholarship, creativity and innovation ensures that high level attention to teaching and learning as opposed to substantive disciplinary knowledge.  There is evidence of significant impact on teaching and learning and benefit to students’ progression, retention and engagement. Projects over the past 5 years have addressed a wide range of fundamental issues. It is recommended that:

• there is a need for more structured, consistent and systematic evaluation of the outputs of eSTEeM projects;
• allocate academic staff time to and evidencing of impact.  

3. Is eSTEeM adequately resourced for current and future projects (particularly those relating to software development and onscreen engagements which are generally more expensive)?

Projects have been successfully carried out with relatively small pots of money and the themed projects have optimized use of resources on focused aspects of the curriculum. Since this approach works it should continue. It is recommended that eSTEeM:

• fosters closer links with KMi and IET to ensure that projects are adequately resourced in terms of research skills;
• supplement existing resources with external funding;
• allocate academic staff time for writing up projects and for funding proposals.

4. What steps should eSTEeM take to ensure its outputs are quantifiable and well matched to the proposed Teaching Excellence Framework?

Specific recommendations are:

• increase emphasis on structured, consistent and systematic evaluation of interventions against progression, retention and engagement to provide evidence of quality and impact;
• investigate current debates about measures of‘learning gain’;
• facilitate HEA accreditation forall levels of staff;
• appoint a TEF champion to ensure alignment with metrics.

If you would like to read the report in full please contact us at esteem@open.ac.uk