Monday 16th June 2025
The Importance of Context in Deploying AI in Educational Arenas
Audrey Ekuban, John Domingue
Knowledge Media Institute, The Open University, UK.
Abstract:
Integrating Artificial Intelligence (AI) into educational environments can transform how Distance Learning students access learning resources. This paper explores Retrieval-Augmented Generation (RAG) strategies used in the AIDA@OpenLearn innovation project. In Education based scenarios, RAG enhances learning by finding relevant information from ingested trusted course material and then using it to generate comprehensive explanations or answers. Chunking, a fundamental part of RAG, involves dividing content into manageable sections to improve retrieval efficiency. RAG ensures that students receive accurate, course related educational content tailored to their specific queries or topics of study.
Student opinions were gained through surveys and interviews. The initial study indicated that students had a more positive sentiment towards using an AI system with curated content provided by the institution, rather than commercial solutions such as ChatGPT.
The RAG strategies in this paper leverage the XML data structures in the OU’s OpenLearn Virtual Learning Environment (VLE). XML is used in VLEs for structured data representation and standardisation, ensuring customisable and extensible content management. RAG with XML content offers structured retrieval and flexibility, enhancing accuracy and adaptability. However, challenges include processing complexity, especially in nested content, and ensuring data consistency across schemas.
This paper examines two RAG strategies. Strategy 1 employs chunking the leveraged XML structure to focus retrieval on the most relevant paragraphs within a student’s current section or subsection of focus. Strategy 2 adopts a different chunking approach, incorporating a holistic retrieval process that considers parent chunks, titles, headings, and commentary. Our research involved deploying the strategies into studies aimed at Distance Learners. The results indicate that Strategy 2 improved students’ perception of accuracy and specificity.
Thus, the studies demonstrate the importance of context-aware retrieval mechanisms in AI-driven educational tools, and that broader contextual elements enhance AIDA@OpenLearn’s effectiveness as a Digital Assistant. Future work will include moving to GraphRAG which will further enhance the way context can be embedded. GraphRAG combines graph-based knowledge representation with RAG to offer an improved method for capturing complex semantic and relational connections within educational content.
Designing and Evaluating an OU AI Digital Assistant (AIDA): What do students value? How will educators maintain influence?
Tim Coughlan and Bart Rienties
Institute of Educational Technology, The Open University
Abstract
This presentation will share findings and reflections from an ongoing Design-Based Research project to understand the potential for an AI Digital Assistant to be embedded into Open University study, working in collaboration across the Institute of Educational Technology (IET), Knowledge Media Institute (KMi), OpenLearn, Digital Services, and faculties.
Many students already use publicly available AI Digital Assistants like ChatGPT for academic purposes. However, there are concerns around the use of such tools in areas such as academic integrity, data privacy, intellectual property, and the impact on the quality of education. Therefore, we explored perspectives on and experiences with 315 students and 20 staff across five consecutive studies using multiple methods and data sources (including surveys, interviews, think-aloud, alpha- and beta-testing). The findings indicated that 24/7 immediate feedback relevant to academic learning was essential for learners. The beta-testing with students indicated that, beyond chat, participants particularly appreciated features such as flashcards and quizzes to support their understanding. Participants’ perspectives on i-AIDA significantly improved after their engagement with a prototype version.
We are also exploring where and how educators’ technological, pedagogical and content expertise could be brought to bear in the way in which the assistant behaves. We draw on feedback from 20 educators and education researchers who took part in one of the trials of the prototype system and reflect on ways in which the system has been designed to embed educator expertise. We highlight methods through which educators can exert influence over the behaviour of such assistants and reflect on the alignment, possibilities and limitations of these to meet educator goals.