Author Archives: Shi Min Chua

End of Summer Term 2021

The sunny summer is calling! The long-awaited summer holiday season marks the end of our series of seminars and it is time to get moving rather than sitting in front of the screen. I can’t wait to go out and about, though I would be careful since Covid-19 is pretty much still around us.

In this series, we have had colleagues from IET, including Rob, Simon, Jane, Maria, sharing their research on open education, digital and supporting learning during Covid-19. Agnes, Bart and Corina also held a session on the Innovating Pedagogy 2021 introducing some of the topics covered, i.e., chatbots, telecollaborations and gratitude. Our alumni Katy talked about her work on Ed-tech hub in low-income context and the challenges facing her team during the pandemic. Mike also hosted an interactive session that brought us on a journey of mobile learning through 10 objects. IET PhD students Wendy and Xinyu also presented their work-in-progress, both of which about using human intelligent agents for learning.

Beside colleagues in IET, we were also pleased to have Mathijs from WELS sharing with us about designing serious games for adolescent mental health and Zsuzsanna discussing about language MOOCs. A big thanks for all of you who have offered to present in our sessions and those of you who have participated and initiated conversations with us!

Last but not least, we have had a fruitful two-day online conference. A total of 85 participants have attended our conference. We have had the pleasure to have IET alumni Prof Simeon Yates talking about digital inequalities and Assoc Prof Tamara Clegg from United States discussing about communitizing sciences and data analytics. A total of 25 papers were presented by our colleagues and alumni across OU, local and international collaborators.

If you would like to revisit some of the topics mentioned above, or simply just missing us, how about watching some of the recording of the seminars? The conference recordings are also available.

Our next series of weekly seminars starts on 14 October 2021. If you are interested in presenting or coming up with some creative 1-hour sessions, please feel free to write to us at to express your interest to present.

Till then!

Best wishes

Shi Min




Teaching and learning in OU

Day 2: Wednesday 16 June 2021, 11.10 – 12.15

  1. Testing and Learning Digital Assistants at the OU
  2. Open University Models: Towards Enhancing Inclusive, Equitable and Quality Higher Education in Kenya
  3. Are virtual visits an effective way of engaging learners?


Scroll down for abstract

Testing and Learning Digital Assistants at the OU

Selina Griffin, Massimiliano Zattera and Serge Plata

Building on a proof of concept case involving the use of Alexa (and demonstrated at the 2020 CALRG) the Test and Learn team have a staged vision of what the future for digital assistants at the OU could look like. Beginning with a “planning and baselining” phase, this has been an enormous data exercise; analysing the above 8 million of call transcripts, texts, emails and webchats that the OU receives. By performing a deep analysis of this dataset we can as a first step along this roadmap, determine the kinds of questions that students and enquirers ask that are large enough in volume but are user-blind in that they don’t require the assistant to know who they are speaking to in order to provide the requested information. With this stage now completed there are huge possibilities for the OU to make the most of this technology to support enquirers, students and staff as part of a modern, digital university offering.
This session showcases some of the findings from the analysis work carried out and shows a road map of what we could explore in the future and our next steps for summer 2021.

Open University Models: Towards Enhancing Inclusive, Equitable and Quality Higher Education in Kenya

Denise Whitelock, Rebecca Ferguson, Simon Cross, Andrew Law, Fereshte Goshtasbpour, Olivier Biard 

The Kenya Vision 2030 provides the blueprint of Kenya’s journey to meet the Sustainable Development Goals (SDGs). It aims to transform Kenya into an industrialising, middle-income country providing a high quality of life to all citizens by 2030. A major support to the Kenyan government to deliver its commitment under SDG4 is establishing the National Open University of Kenya (NOUK) to ensure inclusive, equitable and quality higher education for all.
This presentation reports on the Open University’s work in progress (under the Skills for Prosperity Project) to co-develop a NOUK model Options Paper and a relevant roadmap for Kenya. It specifically discusses the type of challenges and problems a NOUK can address and outlines a range of Open University models that have successfully addressed the discussed challenges. The models include Open Entry Distance, Open Distance, Hybrid, Micro-credentials and Catalyst models. Additionally, the strength, weaknesses, opportunities, and threats related to each model will be briefly reviewed.

Are virtual visits an effective way of engaging learners?

David Conway, Christine Gardner and Janet Hughes

Off campus visits have wide ranging benefits to students including reinforcing and expanding upon taught learning (Streule and Craig, 2016), improved ability to relate theory to practice (Claiborne et al., 2020) and enhancement of motivation (Hutson et al., 2011).

Mature students often choose distance learning (DL) due to its potential to fit around life priorities such as caring responsibilities (Rasheed, 2020). However, the reasons mature students often choose DL also act as motivational constraints which could prevent them from participating in extra-curricular activities (Roosmaa and Saar, 2006).

Advances in technology mean it is now possible to design and implement virtual insight visits for students which produce many of the same benefits as traditional insight visits.

The aim of this project was to investigate if a live virtual visit to Bletchley Park Museum using interactive onscreen technology effectively engages students and enhance their experience.

Over 100 students participated in the virtual visit, many of whom were identified as being in the lowest 50% of the index of multiple deprivation. A small number of participants completed a survey asking their perceptions of the virtual visit. Over half stated that they would normally find it difficult to visit Bletchley Park. All said they would participate in future virtual visits and that they would like to visit Bletchley Park Museum in person.

Initial results indicate that students are engaged by the concept of virtual visits and that they can widen participation in extra-curricular activities. Furthermore, virtual visits may be an alternative promotional strategy for museums to increase visitor numbers.


Learning analytics

Day 2: Wednesday 16 June 2021, 9.40 – 10.45

  1. Language Learners’ Characteristics and Behavioral Patterns in LMOOCs: The Case of Learn Turkish LMOOC
  2. Supporting teachers’ adoption of learning analytics dashboards through design-based research: Insights from two institutions
  3. Predicting drop-out in Toastmasters; scoping out the application of learning analytics to support professional education.


Scroll down for abstract


Language Learners’ Characteristics and Behavioral Patterns in LMOOCs: The Case of Learn Turkish LMOOC

Hilal Seda Yildiz, Bart Rienties and Tevfik Volkan Yuzer

In this two-stage exploratory research, we determined the characteristics of nearly 11,000 learners in a Turkish Language MOOC (LMOOC) and whether their engagement differs according to these characteristics, and also we unveiled which variables are caused these differences. The findings of this study can help researchers and designers to re-think learning design, learner support, and overall learner experience in LMOOCS.

Within the scope of the first study, we discovered learners’ demographics (age, gender, employment status, education level), language-related skills (online course experience and Turkish language level), and online learning environment engagement (watching videos, participating in short exams and module activities). We conducted Exploratory Data Analysis (EDA) procedures for the exploration purpose then visualised the findings. One of the interesting results we achieved as a result of EDA is that more than half of the learners are Syrian citizens, and half of them are men living in Turkey. More than half of the learners are at the undergraduate and above educational level as well as their online learning experience and their prior level in the Turkish language are low. When we explored learners’ engagement on grammar tutorials and drama videos investigated, we saw that the tutorials were preferred more, and also video and visual activities were preferred more than the activities with text and keyboard input.

In the second study, we investigated learner subgroups in terms of their demographics, language-related skills (and online learning environment engagement. At this stage, the TwoStep clustering algorithm was used to include all categorical and continuous variables in the analysis. As a result of the analysis, five different subgroups emerged. These groups are named as (I) the highest engagement and diversity, (II) low engagement and non-online learning, (III) online learner and high engagement, (IV) adult and professional, (V) young and student. A two-stage approach has been adopted to validate the cluster analysis results. After the validation phase, we saw that online course experience and education level are the variables with the highest discrimination rate. These two variables are followed by age, quiz attendance, additional activity attendance and the video watching frequency, respectively.

Supporting teachers’ adoption of learning analytics dashboards through design-based research: Insights from two institutions

Anna Gillespie and Rogers Kaliisa

While teachers are considered as the key users of learning analytics (LA), best practice examples that involve teachers in the development of relevant LA interventions, (e.g. dashboards) remain limited. In this paper, we present insights from two on-going doctoral projects that seek to develop and implement teacher-facing LA dashboards using design-based approaches that involve teachers as co-designers. The first project is based at the Open University (OU) in the United Kingdom and is focused on the use of the Early Alert Indicators dashboard (EAI) that provides predictions about students’ learning activities. The second project is at the University of Oslo in Norway, developing a teacher-facing dashboard (CADA) aimed at providing teachers with real-time insights about students’ participation and discourse in online discussions. Based on experiences gained through qualitative analysis of user-experiences, observation reports, and eye-tracking, preliminary findings from both cases suggest promises and challenges. Teachers reported the usefulness of LA dashboards as valuable tools to support individualised-interventions, enhanced their existing teaching skills, and timely learning-design changes. The involvement of teachers in the design and co-design process was highlighted as useful in connecting dashboards and teachers’ pedagogical intentions. However, it was evident across the two case studies that systematic use of LA dashboards by teachers remains problematic, with teachers using dashboards erroneously and having challenges to understand the visualisations. Moreover, ethical issues, and limited support from management were cited as key concerns. The two cases provide guidelines and implications for teachers, researchers and technology developers concerning advancing the use and development of LA dashboards.

Predicting drop-out in Toastmasters; scoping out the application of learning analytics to support professional education.

Selina Griffin

After my first year of my EdD at the OU, I am keen to share some of the findings of my preliminary literature review on learning analytics and how such practices could be applied to alternative contexts.

The use of learning analytics to build predictive models identifying students who might be at risk of failing or dropping out of their studies is becoming more common (to different degrees) in higher education. In the realm of professional education, focussed on skills rather than knowledge-based education, the practice is still relatively unheard of. My research is based in the realm of Toastmasters International (TMI); a global public speaking organisation of over 300,000 Members. Members attend local Club meetings where they give speeches and take on roles in order to progress through the educational programme in public speaking and leadership skills. TMI have recently developed Pathways; a complete overhaul of their educational programme (largely unaltered in over 50 years) which also for the first time moves materials from paper-based to a Learning Management System (LMS). My research seeks to examine these developments and then look to the field of learning analytics to see whether such techniques could be adapted to the field of professional skills training, examining behaviour in the trace data to flag individuals at risk of dropping out of Toastmasters. For the first time in Toastmasters close to one-hundred-year history, Pathways creates a dataset whereby we can examine the behaviour and engagement of Members with the materials and look for early warning signs of disengagement.

Teachers’ education and professional development

Day 1: Tuesday 15 June 2021, 15.40 – 17.05

  1. Catalogue of new forms of teaching, learning and assessment in Computer Science in Edu 4.0 and related teachers’ skills and competences
  2. The CPD2 Change Cycle: Enabling development for Online Higher Education
  3. Equity Focused Digital Clinical Simulations for K-12 Computer Science Teacher Education: Exploring the detection and dynamic response to confusion
  4. Online work-based learning: a systematic review

Day 2: Wednesday 16 June 2021, 11.00 – 11.10

Extending UTAUT toward acceptance of OERs in the context of higher education


Scroll down for abstract


Catalogue of new forms of teaching, learning and assessment in Computer Science in Edu 4.0 and related teachers’ skills and competences

Bart Rienties, Rebecca Ferguson, Christothea Herodotou, Francisco Iniesto, Julia Sargent and Igor Balaban

The overall aim of Teach4EDU is to enable the creation of an environment that supports implementation of new Education 4.0 learning and teaching approaches in Computer Science (CS). A systematic literature review was carried out, focusing on three research questions. RQ1: Which pedagogic approaches are used to support the teaching of CS?; RQ2: Which of these approaches align with Education 4.0?; RQ3: What skills and competences do HE educators require in order to align CS with Education 4.0? Our literature search identified 66 articles.

Perhaps surprisingly none of the articles explicitly mentions “Education 4.0”. The most common Education 4.0 characteristic was “5) students will be exposed to more hands-on learning through field experience” (73%), followed by “9) students will become more independent in their own learning” (67%), “4) students will be exposed to more project-based learning” (61%). A cluster analysis indicated a three-cluster solution: 1) EDU 4.0 light (n = 18), 2) project-based/hands-on learning (n = 22), and 3) full EDU 4.0 (n = 26). In EDU 4.0 light studies teachers mostly focussed on more independent learning (61%), learning anytime anywhere (44%), and personalised learning (39%). The second cluster had a strong focus on project-based (86%) and hands-on learning (86%), with relatively limited focus on choice how to learn (5%), personalised learning (5%), and learning anytime anywhere (18%). The third and final cluster, full EDU 4.0, was strongly focussed on hands-on learning (100%), becoming more independent (96%), personalised learning (85%), learning anytime anywhere (77%) and choice how to learn (77%). Overall, while there are some engaging and diverse practices in CS and Education 4.0 in Europe, it seems that relative to other countries (e.g., USA) more work needs to be done. This is one of the aims of the TEACH4EDU project will address.

The CPD2 Change Cycle: Enabling development for Online Higher Education

Anne Adams, Elizabeth Fitzgerald, Julia Sargent, Duygu Bektick, Nashwa Ismail, Carina Bossu and Gill Clough

The Covid-19 Pandemic has challenged Higher Education to change practices towards online provision requiring a huge shift in developing teachers, administrators and university management to understand these activities. The Institute of Educational Technology has a long history of providing academic development for the Open University in its 50 years as an international distance learning university. Key to our development practices to support change in crisis has been to use innovation, adaptability and a wholistic approach to professional development. In particular, the professional and personal development required to facilitate online learning is truly interdisciplinary and can be viewed both from a bottom-up, life-long learning or a top-down, learning organisation perspective. However, bottom-up innovation can suffer from poor sustainability and top-down strategies can have limited adaptation and relevance to contextual needs. This presentation provides a research-informed connection between these perspectives of change during crisis by pulling together the advances and research evaluations, often related to technology enhanced learning. These wholistic perspective reviews cross-disciplinary professional development activities that come from, or have relevance to, the Higher Education context. Through this work we have identified three key factors that are critical elements of professional development for developing online learning in a crisis: Context, Philosophy and Delivery (CPD). We have further identified, through connecting the individual and organisational level of change, a CPD2 cycle. This cycle can support effective implementation (from organisational to individual needs) and evaluation (from personal to contextual impacts) of professional development enablers and barriers. There is an evidenced account of delivery tools, methods and approaches that have been used within three CPD2 cycle case studies (online learning design and Tricky Topics, knowledge exchange and Evidence Cafés, and personalisation in online learning). The findings identify impacts from these professional development activities which are discussed with reference to ‘squaring the CPD2 cycle’ in crisis change situations (i.e. completing the impact evaluation side of the cycle). Finally, we conclude with insights for HE to help implement and advance their own philosophies and contexts for professional development to support developing online HE learning in a time of crisis.

Equity Focused Digital Clinical Simulations for K-12 Computer Science Teacher Education: Exploring the detection and dynamic response to confusion

Garron Hillaire, Laura Larke, Deborah Kariuki, Jack Chen, Alison Fang, Danilo Symonnette, Natalie Mionis and Justin Reich

In this talk we will share the progress and setbacks we experienced in seeking to model confusion based on audio recorded responses (and transcripts of audio) from simulations about equity focused problems of practice in K-12 computer science teaching. The aim of the presentation is to illustrate a chain of evidence (which ranges from null findings to significant results) that helped shape the direction of detecting and responding to confusion in equity-focused simulations. The coordination of multiple investigations has led to our current understanding with the first set of results focused on detection of confusion and the second set of results focused on responding to confusion. We first review and report results from four approaches explored to detect confusion: 1) transcripts of audio recordings, 2) prosodic features of audio files, 3) self-evaluation by participants, 4) and researcher coding. Second we report results focused on responding to confusion considering both how confusion related to moments where teachers would provide students with support as well as the potential for technology to provide dynamic support. This talk serves two purposes: First it illustrates interesting results related to equity-focused research and second it highlights the work coordinated by Garron Hillaire, a Leverhulme Scholar from the Open World Learning program at the Open University. The set of studies reported illustrate how his thesis work informed his current research trajectory.

Online work-based learning: a systematic review

Bart Rienties, Blazenka Divjak, Francisco Iniesto, Katarina Pažur Aničić and Mirza Zizak and Iona Jivet

It is widely acknowledged that graduates need to develop skills and competences beyond theoretical knowledge nurtured within higher education curricula. In the last twenty years there has been a push to support undergraduates and post-graduates with work-integrated or work-based learning (WBL) opportunities in the form of apprenticeships, practice-based lab sessions, project-based learning, and hands-on learning activities. With COVID-19 there has been a push to support and provide these opportunities for skills development online.

In this systematic literature review as part of the EU-funded RAPIDE project, we will explore 1) how to design inclusive online WBL practices, and 2) how effective are these online WBL practice in terms of developing graduate skills. We conducted a review using Web of Science and Sciencedirect using the search terms (“online” OR “virtual” OR “remote” OR “distance”) AND (“work-based learning” OR “WBL” or “work-integrated learning”) AND “higher education” for the period 2016-2021. In total 264 unique articles were identified, which were manually analysed to identify appropriate fit. 41 articles were subsequently read in detail by the research team and subsequently double coded. The preliminary findings will be presented at CALRG.

Extending UTAUT toward acceptance of OERs in the context of higher education

Samia Almousa

Knowledge is arguably the most valuable asset one can have.Even when shared, its value does not get any less. In most cases, when you share your knowledge with someone, you do not risk losing anything, unlike when you share money or any other materialistic objects, such as food, property, and so forth.In fact, sharing knowledge is a win-win situation for both the sharer and the receiver.For the former,sharing knowledge with others is an opportunity to engage in discussions that may broaden their already existing knowledge. For the latter the newly gained knowledge is highly valuable and can be a tool to accomplish many things.
As an academic,after using Open Educational Resources (OER) with my students,I realised that by using OER, it is possible to make education more effective by providing every student with a personal,free, softcopy of the textbook for the subjects they study.However, in the Arab context, we suffer to find appropriate resources due to many reasons,the most important is language and different context.
Saudi Arabia has the National Transformation Program,which aims to improve all life aspects, including educational technology aspects,and realise the Saudi Vision 2030,led to the establishment of the National Centre for E-Learning.The centre plays the role of a supporter of e-learning for public and higher education. In 2018, this centre launched “SHMS” as an OERs platform for educational institutions.The platform is dedicated to connecting people and ideas for the enrichment of all communities and is committed to improving educational outcomes through sharing and collaboration. As the adoption of this platform is new in my country,it is vital to study OERs implementation in Higher Education (HE). Thus, this study aims to understand the academics’ perceptions of the use of OERs, and determine how to mainstream OERs in HEIs.To achieve this, a mixed-methods approach for data collection was adopted through two stages, distributing questionnaires to the universities academics, as well as conducting semi-structured interviews with the academics and eLearning assistants.This presentation will report the findings of the first phase which was a questionnaire method used to evaluate the developed Unified Theory of Acceptance and Use of Technology model

An exciting series of talk in the Summer 2021!

Welcome back! I hope all of you have had a well-deserved break over the easter and have enjoyed a bit of sun amid some gloomy rain.

This week marks the start of our summer series of CALRG seminars! In this series, we will have colleagues from IET, including Rob, Simon, Jane, Maria, sharing their research on open education, teacher development and supporting learning during Covid-19. Agnes, Bart and Corina will also hold a session on the Innovating Pedagogy 2021 introducing some of the topics covered. Our alumni Katy will also talk her work on Ed-tech hub while Mike will introduce mobile learning through 10 objects.

Beside colleagues in IET, we are also pleased to be able to invite teams from WELS to talk about their research on MOOCs, language learning and e-health.

For more information on individual seminars, please visit the Event page, though we are still updating each event with abstract when it is around the corner.

Last but not least, our annual conference will be held on 15-16 June, and it will be another online conference! This conference will have two prestigious keynote speakers, one of whom is our alumni Simeon and another, Tamara from the United States.  And more importantly, you, who will be presenting or attending the conference. Do submit your work to share with us in the conference.

We look forward to e-meeting you in one of our weekly session and annual conference!


End of Spring term 2021

Today’s talk by Dr Louise Drumm from Edinburgh Napier University brought an end to our series of talk in the spring term 2021. Beside Louise, we also had Nils from University of Cologne and Tony from University of Cape Town in this series of talk. We are grateful to have external speakers joining us and expanded our reach outside of OU.

We have had a variety of topics in the past spring term. Rebecca and Tony had got us into taking chance to revolutionize online meetings, not limited to just responding to the Covid-19 restriction. Tim and colleagues from WELS, Alison, Claire, Tom presented their collaboration with educators in Africa to promote digital education, while Saraswati and colleagues from other universities presented their wok in Nepal and Bangladesh. Johanna, our alumni, also presented her PhD on creativity and game-based learning. Tina, our micro-credentials lead, presented guidance on making learning@scale accessible. Thea talked about citizenship. This variety of talk not only touched on digital learning, but also international development and online practices. To revisit some of these presentations, you can view the recording in the seminar recordings.

After such a wonderful series of sessions, we will have a two-week break for easter. We will then kickstart our summer term in mid-April. The summer series consists of themes on open education, Covid-19 and presentation from outside of IET and OU. So there are a lot to look forward to, not to mention our annual summer conference! We will soon send out our call for papers. Watch this space!





A CALRG manifesto for online meetings

It has been one year since we last have in-person meetings. Although most of us might be missing the ‘physical touch’ of conferences, seminars and meetings, we have also been able to adapt and utilize technology to come up with new ways of online meetings. It is not a replacement of in-person meeting, but an innovative development of meeting!

On 11 March, Rebecca from IET gathered us together to share our experiences in  online meetings, and what we envision as a good online meeting. Here are the contributions from some of the participants.



shared with us