SocialLearn

I recently  attended Alpine Rendez-Vous 2011, meeting people from all over Europe with an interest in technology-enhanced learning (TEL). For most of the time, we were split into eight workshops – my group was exploring ‘Methods and models of next-generation technology enhanced learning‘, examining the roles of assessment and evaluation in learning.

A focus for the workshop was to identify the ‘Grand Challenges’ for future research – constructing a framework for the development of TEL over the next ten years. The ideas were wide-ranging and exciting, for example:  Develop new technology to harness the power of emotions for learning. One challenge was closely related to our work at SocialLearn on learning analytics and recommendations engines: ‘How can learning be assessed in an open, social TEL environment?’

Our current model for the assessment of learning is primarily summative and individual, firmly bound to hierarchical education structures. It is a model that was developed for use when educational technology had only just moved beyond the horn book, knowledge was not abundantly available, groups of learners were taught and examined at the same time in the same physical location, teachers and learners were clearly differentiated and online collaboration and publication were undreamed-of possibilities. As new models of learning have been widely adopted, this model of assessment is no longer fit for purpose. We need new approaches and new models.

Open, social TEL environments have made new models of learning possible. Learners now draw upon many different people and resources, knowledge is dispersed and distributed, individuals may move rapidly between the roles of teacher and learner, and their collaborations extend across time and space. Despite these changes, the group work, the growing archives of activity and the available data about learning networks, there is still a focus on summative assessment of individuals.

TEL environments offer learners and educators a wealth of new resources in the form of the data they record – learners’ demographics, activities, interactions, participation and engagement – but little of this is currently harnessed to support assessment and even less is used to provide formative feedback and to help learners develop their metacognitive skills, their learning dialogue, their skill-sets or their knowledge.

Colleges and universities are beginning to make use of this data, but educational data mining and analytics are often viewed from an institutional point of view – how can we recruit more students? how can we retain students for longer? how can this institution maximise its income? A wordsearch for ‘teaching’ or ‘learning’ in the current literature produces woefully limited results.

Our challenge is to make use of new resources and technologies to develop and build on learning analytics – analytics that can make a real difference for learners, producing measurable improvements in areas such as:

• Engagement with learning – supported by appropriate and personalised feedback
• Quality of online learning dialogue
• Engagement with online learning networks
• Enjoyment – due, in part, to development of a students-in-trouble alerting system
• Learners’ and teachers’ awareness of the value of learning analytics and recommendations.

Here in Banff, we’re wrapping up the 1st International Conference on Learning Analytics & Knowledge in the outstanding Banff Centre. I have to say that I’ve also never been anywhere so cold, and this is the only conference where they provide free tissues on every delegate table!

It’s been an exciting event to be at, with a tangible sense that this is only going to get bigger, and fast. More reflections on this later, but here are the slides from the session we ran yesterday, with a focus on how analytics might help us understand not just the quantitative analytics on online discourse (e.g. how many people are engaging in discourse, and how often, etc), but what’s the quality of that discourse? A lot of learning concerns making your thinking visible through the way in which you construct your contributions, whether to a forum, or an essay or article. As confirmed by extensive research, learners have to be inducted into the practices of scholarly writing as defined by their particular discipline – they must learn to show, quite explictly, the line of argument or nature of contribution and how it connects constructively to what has been said previously, either by a peer, or in the literature.

This strand of our work in SocialLearn might go under the heading of discourse-centric learning analytics. In the first approach, we seek to detect quality discourse in textchat, and in the second, we deploy a platform for structured deliberation.

Ferguson, R. and Buckingham Shum, S. (2011). Learning Analytics to Identify Exploratory Dialogue within Synchronous Text Chat. Proc. 1st International Conference on Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff [PDF]

Abstract. While generic web analytics tend to focus on easily harvested quantitative data, Learning Analytics will often seek qualitative understanding of the context and meaning of this information. This is critical in the case of dialogue, which may be employed to share knowledge and jointly construct understandings, but which also involves many superficial exchanges. Previous studies have validated a particular pattern of ‘exploratory dialogue’ in learning environments to signify sharing, challenge, evaluation and careful consideration by participants. This study investigates the use of sociocultural discourse analysis to analyse synchronous text chat during an online conference. Key words and phrases indicative of exploratory dialogue were identified in these exchanges, and peaks of exploratory dialogue were associated with periods set aside for discussion and keynote speakers. Fewer individuals posted at these times, but meaningful discussion outweighed trivial exchanges. If further analysis confirms the validity of these markers as learning analytics, they could be used by recommendation engines to support learners and teachers in locating dialogue exchanges where deeper learning appears to be taking place. Slides: Learning Analytics & Exploratory Dialogue [PPTX/PDF]

De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. (2011). Discourse-Centric Learning Analytics. Proc. 1st International Conference on Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff. Eprint: http://oro.open.ac.uk/25829

Abstract. Drawing on sociocultural discourse analysis and argumentation theory, we motivate a focus on learners’ discourse as a promising site for identifying patterns of activity which correspond to meaningful learning and knowledge construction. However, software platforms must gain access to qualitative information about the rhetorical dimensions to discourse contributions to enable such analytics. This is difficult to extract from naturally occurring text, but the emergence of more-structured annotation and deliberation platforms for learning makes such information available. Using the Cohere web application as a research vehicle, we present examples of analytics at the level of individual learners and groups, showing conceptual and social network patterns, which we propose as indicators of meaningful learning. Slides: Discourse-Centric Learning Analytics [PPTX/PDF]

Simon Buckingham Shum from the SL team will be in Barcelona next month for the co-located Open Education Conference and Mozilla Drumbeat Festival.

Here’s a preview of the article on SocialLearn that we’ll present at OpenEd, reviewing some of the design rationale for SocialLearn, currently in internal testing here at Open U:

Buckingham Shum, S. and Ferguson, R. (2010). Towards a social learning space for open educational resources. OpenEd 2010: Seventh Annual Open Education Conference, 2-4 Nov 2010, Barcelona. Eprint: http://oro.open.ac.uk/23351

Look forward to seeing you there, and look out at Drumbeat for the related demos of Cohere (opening night Science Fair), a social web annotation and knowledge mapping tool tuned for inquiry, sensemaking and learning. This forms part of our thinking on what a Collective Intelligence infrastructure might be not only for social learners, but also as a resilience platform for stakeholders in the open educational resources movement:

Buckingham Shum, S. and De Liddo, A. (2010). Collective intelligence for OER sustainability. OpenEd 2010: Seventh Annual Open Education Conference, 2-4 Nov 2010, Barcelona. Eprint: http://oro.open.ac.uk/23352

YouTube replay

Social learning is about bringing people and resources together, making meaningful connections between them and enabling discussion. But, as Google Buzz showed, there are limits to the connections that people want to make. Individuals, groups of people, and organisations are all engaged in representation management. There’s some information that we are happy to share with everybody, and some that we want to keep private and there are some discussions that we only want to have with trusted individuals. In the case of The Open University, there is an enormous amount of content that is shared freely via OpenLearn, iTunesU and  OUView. At the same time, there are also paid-for courses that you won’t be able to access without registering.

In the case of SocialLearn, we’re aiming to make it as easy as possible to link different resources and different networks. You can sign in to SocialLearn using your Twitter ID, your LinkedIn ID, your Facebook ID and many others. Once you’ve signed in, you can set your privacy levels, and you can join groups with their own privacy levels. On an individual basis this works – but it doesn’t always work on an institutional basis. If another university or organisation joins us, bringing lots of individuals, lots of resources, lots of connections and lots of information, we need a straightforward way for the organisation to manage privacy levels without putting the onus on every student/staff member to deal with this individually. At the same time, we don’t want all those new people cut off from their learning outside that organisation. A related question is, how do we provide minors with safe access to social learning, so that schools can make use of SocialLearn?

We’ve been discussing models for ‘walled gardens’ that allow a high degree of openness and sociability, while allowing us to bring together resources from big providers. Model one is the separate build.

Separate walled garden

Two builds of SocialLearn – one for Organisation X (which might be a university, school or company). They don’t talk to each other, but they are internally sociable. This is straightforward – but an obvious problem is that it negates the need for SocialLearn. People already have their learning divided up into silos – SocialLearn is trying to bring those silos together.

The next model is the internal garden. In this model, the walled garden sits within SocialLearn and forms a subset of it. This is more complicated to construct, because groups are nested within groups. If the wall around the walled garden is impermeable, this is effectively just a more complicated version of separate walled gardens. If the wall is permeable, then is it useful?

Internal walled garden

The other obvious model is the overlapping garden – where the walled garden sits partly inside and partly outside SocialLearn. Again, the problem is with what this means in practice. How freely can connections be made between SocialLearn and the walled garden? Can learners move from one to the other without effort, or do they have to pass through a gate and get permission from a gatekeeper at each point?

With all the models, we found we were less interested in the location of the wall, than in its doors and windows. How easy will it be to look over the wall, and to see what, and who, is on the other side? Can we wriggle through gaps in the wall, or do we have to go through a single gateway? Are people, information and resources trapped within the walled garden or can they leave with a permit? Can learners invite friends in, or link up with friends outside their garden?

These questions suggest that the visualisations on this page aren’t helpful in the long run, because our concern is not with location, but with connections and with movement. The metaphor of the walled garden proved limiting. We are now trying to visualise the problem in terms of information flow, or in terms of connected networks. We haven’t solved the problem, but we’re clearer about what solving the problem will involve.

Overlapping walled garden

The 1st International Conference Learning Analytics & Knowledge will be held February 27-March 1, 2011 in Banff, Alberta. This is an extremely exciting development, reflecting what in many people’s view is going to be a key dimension to future learning environments, with a strong Open U. presence in the steering committee (join the Learning Analytics Google Group).

From the conference announcement:

The growth of data surpasses the ability of organizations to make sense of it. This concern is particularly pronounced in relation to knowledge, teaching, and learning. Learning institutions and corporations make little use of the data learners “throw off” in the process of accessing learning materials, interacting with educators and peers, and creating new content. In an age where educational institutions are under growing pressure to reduce costs and increase efficiency, analytics promises to be an important lens through which to view and plan for change at course and institutions levels. Corporations face pressure for increased competitiveness and productivity, a challenge that requires important contributions in organizational capacity building from work place and informal learning. Learning analytics can play a role in highlighting the development of employees through their learning activities.

In enterprise settings, information flow and social interactions can yield novel insights into organizational effectiveness and capacity to address new challenges or adapt rapidly when unanticipated event arise.

Thirdly, as we witness the expansion of learning and knowledge work beyond formal institutional boundaries, myriad platforms in the cloud hosting the activity of individuals will be providing/exchanging analytics.

Advances in knowledge modeling and representation, the semantic web, data mining, analytics, and open data form a foundation for new models of knowledge development and analysis. The technical complexity of this nascent field is paralleled by a transition within the full spectrum of learning (education, work place learning, informal learning) to social, networked learning. These technical, pedagogical, and social domains must be brought into dialogue with each other to ensure that interventions and organizational systems serve the needs of all stakeholders.

Learning Analytics 2011 will focus on integrating the technical and the social/pedagogical dimensions of learning analytics.

Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs