Authorship – Alastair Barrow and Imogen Clare of Generic Robotics
This blog was kindly sent from Dr Alistair Burrow and his project assistant Imogen Clare. These two colleagues work a researchers within a start up business initiated and developed with via Oxford and Hertfordshire University. Through network links I have been working with both these colleagues and their new product TOIA (as outlined below), to understand if design OU students could benefit from haptic augmentation, as a research tool or as a teaching and learning tool. This blog was created to allow OU academic colleagues the chance to read about where the current state of the art is within haptics and what we could expect for the future of touch augmentation within creative subjects. I do hope it opens up your own creative thinking on what interactive VR technologies can do for Design Group modules.
Introduction to haptics
Haptics, put simply, is the science of touch feedback. It is a two-way process concerning the interplay between a user and computer generated device that gives you the sense you are touching something real that isn’t physically there. In order for a device or interface to be genuinely haptic, the user’s input movements must have a meaningful relationship to the touch feedback the computer generates. A standard computer mouse is only haptic if it has the ability to vibrate and do so differently depending on where the cursor is on the screen. In its simplest form, haptic technology is a handy way to feel if you correctly clicked a button on a touch panel; as an example, imagine using your fingers to select your favorite piece of music or latest podcast on your smartphone without having to look.
Advanced haptically enabled simulators let users freely pick up objects, feel their weight, how they deform under pressure and interact with other objects. How we perceive the world through touch impacts the way a haptic device is built. A haptics expert can manipulate sense of touch, providing sensations, forces, vibrations and experiences they choose and apply “human” oriented engineering to make the interaction believable, realistic, immersive and ideally enjoyable and/or impactful. Haptic devices store a representation of a virtual world which responds to the user’s touch and changes depending on a variety of factors; directional movement (rolling, gripping, tapping), pressure, temperature and weight. The haptic device then applies forces to the user depending on their input to give the illusion they are interacting with objects of substance which aren’t actually there.
To be a haptic device, the computer must be able to affect the user’s tactile or kinaesthetic senses based on their actions i.e. if the user presses something the haptic charge pushes back in response. The perfect haptic experience would make the user believe they are in direct contact with a virtual object to the point where they are not aware of the haptic device at all. You could be feeling the surface, edges and grooves of a fossil through a haptic device and not be able to distinguish the difference between the virtual object and the real thing. Haptic technology is still fairly new, but it has already reached a point where it is a genuinely useful tool for all sorts of medical and surgical training applications, creative applications, e-Learning, gaming, even agriculture.
How well Toia lends itself to accessible creative learning and teaching: The power of exploration: perception, sense of touch, haptics and application to design. ____________________________________________________________
It is not hard to understand how sense of touch, and so the idea of haptics, where the fast, largely subconscious link between touch sensation and the purposeful actions which produce them are integral to so much of the creative process across many domains. It is perhaps easier to overlook the importance of the haptic (touch) sense since the finished article doesn’t necessarily portray what went on during the making of what becomes a finished piece of work. So, if we wanted, we could separate the role of haptics in the design pipeline into: process and creation by the designer, and experience and appreciation by the audience. But, with technology, we can take this further and blur the boundary between the creative process and audience experience. Let’s explore this concept a little and see where it takes us.
While music, textiles, dance, various forms of painting and one hundred other creative disciplines and mediums all embody touch, we can, and probably should leave that discussion to another time for fear of losing ourselves by trying to cover too much. Let’s start with a simple, useable example; digital sculpture, which lends itself well to research, consumer applications of haptic technology and forming a ready understanding of how haptics works well in the design space.
Process
The process of sculpting clay by the artist involves an innate understanding of the materials in use, the properties as conditions change (wet/dry, warm/cold), the forces and pressures required to make the material take and hold a shape and the structural properties required to produce a desired form. This includes the fineness of detail that can be reproduced and the degree of tolerable overhang or support required. This delicate and subtle knowledge is imparted into the artist’s medium through haptics and is received and understood by the same sensory process; we know how hard we are pressing the clay and how much it is deforming beneath our fingers by the interplay of force produced and sensed by the muscles and the sensations of pressure through our fingertips. Equally, material properties such as temperature and wetness are received via sensors in our fingertips and sent to the brain to form an innate understanding of materials at any given moment in varying conditions.
The artist can draw on this understanding to help translate an idea about a design idea beginning in the mind to the execution of a physical creation that exists in the world and elicits response from the audience, which has great potential to feeding the teaching and learning experience. So, what about the digital analogue of this excuse (excuse the pun)? In haptics, the fast feedback loop involving sense of touch us absent from typical, traditional human computer interface paradigms. Though a computer mouse doesn’t give any meaningful physical feedback corresponding the movements of the artist’s hand and fingers when designing a 3D sculpture, the graphical user interface tools and techniques have evolved to make up for, replace, and admittedly in certain ways, improve on, the missing physical relationship with 3D creation.
Although there are, I am certain, many would fiercely argue that making mistakes is an integral and necessary part of the creative process, I’m willing to bet there are few artists who wouldn’t relish a real-life undo button. The problem is many teachers and students of design actively need the “making a mistake” event to inform and decide the next design phase before coming to an intended, or perhaps surprising, end point. Taking this a step further then, where haptic technology enters the arena is to provide a genuine, two-way relationship in the digital creative process. Through the use of clever software and a force and/or tactile feedback interface, a computer can recreate a physical relationship between the user and 3D modelling process, allowing him/her to physically experience contact with the digital medium and achieve an intuitive force relationship between action and the resulting change to the digital sculpture. Although, as is generally the case with digital tools, is it neither as subtle and sophisticated as reality, but nor is it limited either.
The range of interaction types permitted in a haptically enabled modelling tool can be far greater than what’s achievable in the real world. The directly comparable actions such as adding and removing material, pushing and pulling or applying affects and surface textures (e.g. making patterns with a fork) can all be recreated. Further, as we know, the computer can create mirror effects allowing all actions performed on one side be automatically added to the other. Examples might be applying complex patterns like clouds or numerical effects and filters similar to a photo editing package.
Experience
So what about the second of our creative digital haptic processes; experiencing? Once the creative process is complete and we have a digital piece of art, how can the work be experienced by someone else? In many circumstances sculpture is visually appreciated anyway, so no problem there. But why not allow the viewer to touch the digital work? You could certainly 3D print the sculpture and physically explore it; as amazing a breakthrough as this is when you think about what this gives the world, the use of computer haptics lets us explore the form of an object instantly with no waiting incurred. Some fascinating work has been carried out that allows museum visitors to virtually touch exhibits they would not ordinarily be allowed to interact with due to age or fragility; all by having a haptic device and simulation setup by the real piece. Thus, visitors can explore the digital representation at the same time as seeing the real work in front of them. But, that’s not the end of the story. You could, through the magic of computers, also record the creative process as it happens and then store that as well as the final result; a 4D sculpture if you like.
There is plenty of fascinating work looking at the original pencil sketches beneath great works of art, but a sculpture where the whole process from start to finish has been captured opens up a completely new and deeper way to connect with the final result and understand the creative process. The viewer now not only sees but feels the work taking shape as it is played back on the computer. And so, when we get to the third creative digital haptic process, things get really interesting if the experience isn’t purely passive. This could have huge implications for teaching, learning and devising imaginative ways to explore the design process. What if the viewer or learner was able to pause the playback and make his/her own changes part way through, exert different pressures on a tool, or select different effects while the sculpture was in motion? As a thread makes up a strand of wool, the viewer could become an integral part of a community-based creative process. A project could be initiated by one person at a distance with then a back and forth interplay between viewers/learners to build, review and influence the final work of art but the process as it unfolds over time.
Leave a Reply