Skip to content

Toggle service links
  1. eSTEeM
  2. Projects
  3. Themes
  4. Technologies Stem Learning
  5. Ask Programs Aloud ‐‐ Making Programming Concepts Naturally Accessible at a Distance

Ask Programs Aloud ‐‐ Making Programming Concepts Naturally Accessible at a Distance

Project leader(s): 
Yijun Yu and Anton Dil
Faculty: 
STEM
Status: 
Current

The Open University commits to redesign programming-related modules so that students could gain first-hand support from the very start, and learn more advanced concepts continuously throughout the course of study.

One of the major obstacles to this goal is to support those students with visual disabilities, who require sound as a modality to drive adaptive user interface design; however, existing accessibility helpers only read out the texts sequentially whilst the presentations of programs are hierarchical in nature. With the advent of voice-interaction technology such as Alexa Skill Kit (ASK), we propose to translate sequential narratives into hierarchical ones, driven by the requirements of students.

The new proposal aims to focus on any part of their programs whist maintaining the overview relevant to the studied concepts. To implement this, we introduce a knowledge map-based pedagogy called ``Program Aloud", a personal assistant that can respond to voice commands that triggers an explanation of programming concepts, a summary of programming contexts, etc. The new hands-free mode of interactions could generate intelligent dialogues that answer students' questions about the program or a programming concept, meaningfully. 

Some tangible outcome of the project include, but not limited to the following:

  • Turning certain types of iCMA questions (including those with structured answers such as programs) into multi-step voice dialogue sessions;
  • Turning the table of contents and the list of learning outcomes per units into voice navigable inquiries;
  • Recommending relevant course materials and reference materials to students given the context of their explorations.

The overall outcomes of the project would be an initial ASK Skill hosted to the Amazon Alexa ecosystem to recommend Open University course(s) just like the iTunes University open course contents.

Yijun Yu presentation (PDF)

Projects