Learning analytics: the patchwork quilt visualisation

In the Open University, we have developed a suite of LA (Learning Analytics) visualisations called ‘Action for Analytics’ (A4A: slides from a presentation giving more detail) designed to help those responsible for producing modules to see the effects of their designs.  For example, it’s possible to track just how much use videos we produce for the module get watched and therefore see whether doing more would be a good investment.

sample learning analytics graph of students resource use with time from OU module
Tracking how many students have accessed a particular section of the module in any week. Weeks along the bottom and blue bars are weeks with Assignments in them.

This has been very successful with our colleagues outside the Learning Design team (mostly academics) being able to track what is going on with their modules real time and also see the effects of changes as they are bought in.

However, the tool is limited to a set of ‘baked in’ dashboards so its not possible to split the above data into students who ended up failing the module from those who passed and compare the two graphs.  This could give useful insight into the value of individual parts of a module and also if students are accessing it or not.

Drilling down into the data:  A4A isn’t the only route to exploring statistics about students on modules.  There are a number of databases underlying the visualisations and these can be accessed directly by specialist staff.  Using our access rights, we have been experimenting with producing bespoke visualisations not in the current suite that we think could help those writing and designing modules.  These are currently prototypes but show some promise:

Screenshot showing patchwork quilt visualisation
Patchwork quilt visualisation. Sections of the module are arranged in columns, rows at the top represent individual students showing sections they have visited at least once. At the bottom, these individual visits are collated to show percentage access to each element for various groups: Withdrawers at the top, still registered below this and Low economic status (SES) below this.

In this visualisation, individual students are shown one per row at the top.  If they have accessed any element of the course (one section per column) the corresponding cell is blue.  If they have never accessed it, it’s shown white.  At the bottom, students are grouped (e.g. ‘withdrawers’ and ‘registered’ – not withdrawn) and cells are now coloured with hot colours showing low usage and cool colours showing high usage.

Example Interpretation:  As an example of its use, the last column is the block assignment.  It can clearly be seen that section 18 (column 2nd from right, expanded up left) is attracting a high percentage of students visiting it at least once.  Section 17 (3rd from right) is attracting considerably lower numbers of students, especially amongst withdrawers.  This is a factor of inclusion of section 18 in the assignment, whereas 17 is not and, as a result, students are choosing to skip it.  From a design point of view, should it be included at all?

More granularity:  In our work investigating this graphic, we think it will become even more useful when there are improvements in the granularity, at present we can only see that students have accessed a whole section.  For example, it will be much more useful to see how far they got within a section itself – did they give up half way through?  Improvements in the learning analytics the VLE records should help with this.

Next Steps:  This is a work in progress, already we are making the patchwork quilt visualisation more sophisticated and have plans for other experiments.

Richard Treves, Senior Learning Designer.

Carl Small, Analyst.

Analytics4Action: emphasis on the ACTION

It is no secret that we live in a world driven by data. The insight gained from analytics underpins success in almost every industry, from strategic consumer research in commercial sales to the in-play behavioural patterns of leading men and women in sports; a great deal of importance is afforded to the reporting of “The Stats” and the stories they tell.

Effective data reporting connects an audience with a subject, providing a level of understanding that would usually only be experiential. Having worked in analysis roles across multiple fields this concept was not new to me, however prior to joining The Open University I had never considered the sheer necessity for data within education.

Analytics – The Gift and The Curse

At The Open University we have a variety of tools that offer insight into different aspects of the institution. At any moment in time, Open University staff can use data to map out the journey of prospective students, or to see the number of students attaining their sought-after qualifications. It is extremely easy to spend hours fascinated, absorbing the wealth of numbers and eye-catching dashboards on offer, using them to comprehend exactly what is happening at The Open University. We readily accept the gift of this intricate and insightful data, but so what?

Data for data’s sake

Imagine running a small business, you have an efficient analyst in your team that provides you a monthly report of all the data you need. You know the numbers inside out and could reproduce the graphs from memory with ease, there is no doubt that you are in tune with the business at an operational level, this however is only flint for the fire. A data table on its own will not drive change, nor will an aesthetically pleasing graph bring you more customers. The actions made from the insight the data provides is the spark that needed to create the fire of success.


This thought process is core to the Learning Design Team at the Open University. The Analytics4Action programme ensures Open University staff have an understanding of students on live courses, during presentation. At a distance learning institution (in which the face to face interaction between those learning and those teaching is severely reduced) this understanding is empowering. It allows for decisions to be made that can positively influence the development of a course which will in turn contribute to the development and success of our students; which naturally is the common goal for everyone here at the Open University.

In data we trust

In summary, we should always recognise the importance of accurate and well presented data, however what we should really focus on is the potential for change  granted by accurate and well presented data.



OneNote: an eportfolio-type tool to support PDP

By Sue Lowe, Senior Learning Designer

What is PDP?

The OU Student employability policy states that all students are entitled to be supported in Personal Development Planning (PDP) (Open University, 2011).

The OU defines PDP as a framework to support students in their analysis of and development of their own skills, and a means of collecting evidence for use in the workplace (Open University, 2017). The Quality Assurance Agency (QAA) defines PDP as a “structured and supported process undertaken by a learner to reflect upon their own learning, performance and/or achievement and to plan for their personal, educational and career development” (QAA, 2009).

Expanding this, the OU careers team advise students work through five stages of PDP – Identify, Plan, Act, Record, Review (Figure 1, Open University, 2017) which ties in with the process recommended by the QAA (2009). Further, the QAA (2009) defines PDP as a process undertaken by students in relation to three areas – personal, professional, academic (Figure 2).

Five stages of PDP                     Figure 1: Five stages of PDP

Three areas of PDP

               Figure 2: Three areas of PDP


My exploration into OneNote to support PDP began when several module teams I was working with asked me about tools they could suggest students use to take notes, to carry out self-assessments and to reflect on their progress. At the time – going back a couple of years – this was a gap. My instructional design qualification and study of H819 The critical researcher: educational technology in practice both taught me that a defined gap starts a process of exploration.

Step 1 in this process involved a perhaps more traditional OU approach of developing an in-house resource. This was for the Open Degree programme and working with various colleagues in the Open Degree team, Learning and Teaching Innovation (LTI) and Careers. We produced an html5 asset (Figure 3) and Structured Content activities/guidance. The aim was for students to establish where they had gaps in relation to the seven employability skills, and plan for and carry out work to improve weaknesses and build on strengths. This is a solid first step, but now, looking back, it doesn’t provide a space for students to make notes and upload work they’re proud of, or to reflect on tutor feedback.

Interactive skills audit tool

Figure 3: Interactive skills audit tool

Step 2 involved working with Maria Luisa Perez Cavana, a language academic in the faculty of WELS who was keen to trial an external eportfolio tool, EPOS, developed by the University of Bremen and which we adapted to suit OU PDP (Figure 4). We ran a pilot on L161 Exploring languages and cultures in which we explored students’ reactions to the concept of PDP in its complete sense (as in the QAA definition, not ‘just’ reflecting on the academic elements of a single module but relating to the three developmental areas and the five stages of the process), how specially designed activities might help steer them through the – admittedly – complex process, and whether a separate space – an eportfolio – might help. While students found the concept of PDP challenging in the beginning, the more they worked with it, the more they could see its benefits and the easier it became. The tool was a hit in the sense of having somewhere to do this stuff, but not in the sense it was a tad complicated to use. (The report is available on the OU Scholarship exchange.)

EPOS eportfolio tool

Figure 4: EPOS eportfolio tool

Step 3 evolved from the fact that OU students now (since Spring 2018) have the opportunity to access Office 365 for free via StudentHome. Part of the Office 365 suite of tools is OneNote and talking to several TEL colleagues, we established that OneNote would be worth investigating. In line with the work the Enhanced Employability and Careers Progression (EECP) team are doing around promoting employability, OneNote is a tool students are likely to be familiar with in their professional – and personal – spheres. Consequently, as part of the usual Learning Design process for the three new Level 2 language modules (L222, L223 and L226), Maria Luisa and I carried out developmental testing with students. (The report is available on the OU Scholarship exchange.) We’re also working with three other module teams (L116 Spanish studies 1, E209 Developing subject knowledge for the primary years and EE812 Educational leadership: exploring strategy) as part of a cross-WELS scholarship project. In all these cases, we are exploring the use of OneNote for PDP. Here’s what we’re trialling…


We wanted to replicate the things that worked well in the L161 pilot whilst simplifying the student experience. We recreated the tabs to represent the stages in the OU PDP process. Under each of those tabs were a small number of pages with minimal frameworks (Figure 5). We exported the notebook we had created and provided this to students. Once students had set themselves up with an Office 365 account and had access to OneNote, they could install the template we had created. Students were directed to the relevant tabs and frameworks within the template by activities provided in Structured Content on a VLE website. The activities serve as a bridge between module and student PDP space.

The cross-WELS scholarship project is still underway, a report should be available in the summer. Initial impressions are that these students do see the benefit of PDP and ask for time to be allocated to it. Once they’re into the OneNote template they find it easy to use, in fact they find OneNote easy to use in general. So far so good. One issue though is the template file format is not compatible for Macs. (We estimate about 10% of OU students are currently Mac users.) Another issue is that students who already have an Office 365 account sometimes found it difficult to locate the template once installed. Not so good. However, I would like to explore solutions in Step 4. I still believe OneNote is a useful tool in supporting students in PDP.

PDP tabs in OneNote

Figure 5: PDP tabs in OneNote

Step 4 (proposed). Why not continue to guide students through their PDP via activities on module websites. Why not continue to suggest students use OneNote to collate their plans, evidence and reflections. However, perhaps we should not necessarily provide a template, instead leaving OneNote as an entirely student-owned space (Weller and Sclater, 2016), an idea I explored when studying H800 Technology-enhanced learning: practices and debates, part of my OU Master’s in Online and Distance Education. Yes, suggest how students might want to structure the notebook, such as having the tabs to represent the PDP process, but leave the rest up to them. This should avoid issues of Mac compatibility; avoid issues of templates being misplaced between different accounts; and, importantly, it should enable students to use the same digital space throughout their entire degree, and beyond, because it will not be tied to individual modules via a template. To promote this idea further, why not place PDP guidance on the subject (qualification) websites? This is what we’re going to be exploring next in WELS…

For further information, please contact Sue Lowe (Twitter: @DisEdTech,) or the OU Learning Design team in TEL (LTI-LearningDesign@open.ac.uk).


Open University (2011) ’Student Employability Policy Statement’, Essential documents. Available at http://www.open.ac.uk/students/charter/essential-documents/student-employability-policy-statement

Open University (2017) ‘Personal development plan (PDP)’, Develop your career. Available at http://www2.open.ac.uk/students/help/pdp

QAA (2009) Personal Development Planning: Guidance for institutional policy and practice in Higher Education, The Quality Assurance Agency. Available at: http://www.qaa.ac.uk/en/Publications/Documents/Personal-development-planning-guidance-for-institutional-policy-and-practice-in-higher-education.pdf

Weller, M, and Sclater, N. (2016) ‘Weeks 21-22 Debate about VLEs versus PLEs’, Sclater/Weller podcast, H800 Weeks 21/22 Implications and future trends [Online]. Milton Keynes, Open University

Flavours of design – what’s in a name?

Our team at The Open University has gone through a rebrand. Where before we had Learning Design (LD) and TEL Design components to our team, we are now all Learning Designers. A small change in terminology, but hopefully a big change in mindset. So why have we done this?

First, it better encapsulates what we’re actually doing and seeking to do as a team. Many of the activities carried out as part of TEL Design were actually LD activities happening at a more detailed level. For instance, visualising the student learning journey, planning for a block of study, designing a collaborative online activity or helping the module team to review student workload expectations. You could see these activities as implementation of the Learning Design.

Secondly, it helps to reinforce that LD is not a one-off activity, you need to continue making design decisions and reviewing how you are progressing toward the planned design throughout the module development and production, and even into presentation. The online LD tools we have (at http://www.open.ac.uk/iet/learning-design/downloads) support this approach. Encouraging teams to engage with these and continue to review their design is something we’ll continue to push on.

And lastly, having two design teams gave the sense that these were two very different activities whereas they were always really a continuation of the Learning Design process, with TEL Design going into more detail and identifying appropriate uses of technology as we did so.

Whilst we won’t be changing our approach to design overnight we will be looking at how we can get even more value out of LD workshops and at how we can enable the subsequent design activities to tie in further with the outputs from those workshops.

Through this new blog we’ll share some of our experiences as we adjust to the new team name and on any enhancements we make to our processes, it promises to be a busy and exciting time for us!