Research on haptic interfaces has historically developed around grounded kinesthetic devices. The pursuit of more wearable technologies led next to the development of exoskeletons. Although exoskeletons can be considered wearable, they are often quite heavy and cumbersome. More recent research has sought to extend the definition of a “wearable device” beyond something merely suitable to be worn. A wearable haptic device should also be small, easy to carry, comfortable, and it should not impair the motion of the wearer. But wearable haptics is not only applying haptic stimuli in a seamless way. Recent advancements in textile engineering are showing remarkable results in innovative smart clothes, which embed lightweight and thin sensors, either haptic or inertial. For example, Levi’s and Google developed a haptic jacket which enables users to control their smartphones by touching an instrumented sleeve.

Wearability is a determining factor to design new interfaces that will simplify and enhance the way humans interact with digital and physical worlds. I will introduce the scientific and technological foundations for wearable haptics and I will show how wearability is redefining the way we interact with other humans, machines, and the environment. In the last few years we have developed devices with different form factors such as thimbles, rings and armbands. While the haptic thimble design has been proved to be very effective to display haptic cues in VR, the haptic ring has been the first device we proposed which is explicitly designed for Augmented Reality. The haptic ring leaves the fingertip free to interact with the physical objects while applying shearing forces to the base of the finger. I will introduce the most recent developments about the design and the applications of our haptic ring and armbands devices. The tradeoff between realism and wearability is one of the most challenging aspects of our research. Clearly, when the goal is to simulate fingertip interactions in VR environments, the haptic thimbles outperform the haptic rings: indeed, the latter display tactile cues at the base of the finger, where they are not expected to be. On the other hand, the usability of the rings is much more interesting than thimbles as for example while wearing them you can type on your keyboard with no need to take off the thimbles from your fingertips. The armbands results to be even more usable than rings because they are worn on the arm and leave the hand completely free to interact with the environment. We have been using armbands for applications where the realism of the interaction is not the primary objective such as in teleoperation and human body guidance. Finally I will introduce our research on wearable sensorimotor interfaces to 'connect' the human to more complex systems like robots,, e.g. extra fingers and extra arms, that are used as sensorimotor extensions of the human body to enhance the sensorimotor abilities of humans.

There is clear bifurcation of VR haptic solutions today. They are either custom application-specific solutions designed for high force and high realism haptic rendering or simple vibrotactile solutions built into the millions of VR controllers in mainstream use today. Truly realistic haptic rendering, while a noble research goal of many in the haptic field, remains far on the horizon. Instead of realism, human-computer interaction perspective argues that we should focus on what kind of haptics makes a better, more fun, more engaging VR experience. Given that the handheld controller is a de facto standard interaction device for VR today, augmenting such controllers with compelling haptic capabilities has a high potential to improve the overall VR experience. However, doing so is not easy. These VR haptic controllers need to be small, light, reliable in operation, easy to carry, comfortable to hold, capable of multipurpose haptic feedback, and they should not impair the motion of the user. If not done well, these haptic augmentations can do more harm than good, by breaking the user’s sense of presence. However, if finely tuned as one modality of an otherwise multi-sensory experience, they offer a path to convincingly fool the user and deliver a fundamentally better VR experience. In this talk I outline five key directions for VR haptic controllers that push the hardware beyond vibro-tactile solutions. I illustrate each direction with examples from my past and soon-to-published projects, including Haptic Retargeting, NormalTouch, CLAW, Haptic Links, and Haptic Revolver. Through those examples, I offer a blueprint for the next generation of VR haptic controllers.

The presentation will focus on three findings on the subject of the biomechanics and biotrobology of touch. The presentation will first comment on the significance of the dynamics properties of the human fingerpad reported in 'Mechanical Behavior of the Fingertip In The Range of Frequencies and Displacements Relevant to Touch' by Wiertlewski and Hayward, 2012. There, it was found that the bulk mechanical properties of the fingertip could be characterized by elasticity from DC to about 100 Hz and by viscosity above this frequency. The presentation will then address the 'Effect of Skin Hydration on the Dynamics of Fingertip Gripping Contact' by André, Levesque, et al. 2011 and its consequences for the design of tactile displays. In this study it was found that the grip force deployed when gripping object was more related to a strategic, anticipatory motor behavior designed to reduce the likelihood of contact failure rather than to a response to the coefficient of friction. Lastly, the presentation will introduce recent findings about the dynamics of true skin contact formation reported in 'Why pens have rubbery grips' by Dzidek, Bochereau, et al. 2017. There, it was found that because the keratin of the skin outer layer is stiff and rough at a small scale, when encountering a smooth, stiff, and impermeable surface, such as polished metal or glass, the actual contact area is initially small as is the friction. Because keratin softens when it is hydrated by the moisture secreted from the sweat pores, it requires many seconds for the contact area to increase to the value reached almost instantaneously with a soft material, such as a rubber. This mechanism might be used by our tactile sense to identify materials and has implications for the design of tactile displays.

Clothing- and textile-based haptic interfaces afford exciting benefits: clothing as a structure is socially pervasive and grants access to a large portion of the body surface. For haptics, this means the ability to affect a more complete scope of the human sensory capacity. However, there exist several important tradeoffs in the pursuit of large-area haptic displays: significantly, between the comfort and wearability of the display and the efficacy of the actuation technology. Clothing that moves and looks like everyday garments is much less likely to enable a consistent and robust interface with the skin. Further, effective communication through large-area displays is poorly understood. This talk will explore the key challenges and opportunities that lie between the state of the art and the long-term potential of haptics in clothing-based interfaces.

In recent years, wearable haptic systems (WHS) have gained increasing attention as a novel and exciting paradigm for human–robot interaction (HRI). These systems can be worn by users, carried around, and integrated in their everyday lives, thus enabling a more natural manner to deliver tactile cues. At the same time, the design of these types of devices presents new issues: the challenge is the identification of design guidelines, with the two-fold goal of minimizing system encumbrance and increasing the effectiveness and naturalness of stimulus delivery. The latter point can be successfully addressed moving from the correct understanding and mathematical modeling of human sensory mechanisms, e.g. to characterize mappings between cutaneous stimulation, motor acts, and percepts. Such a representation scheme could hence help to identify the most useful information for users for task accomplishment and to develop simple, effective, affordable and intuitive interfaces for HRI. Under this regard, fabrics can represent a viable technological solution for system design, which enables to successfully address the need for wearability. They are indeed specifically thought “to be worn”, imposing minimal constraints to users, and could be the key ingredient to develop WHS conceived for a natural human-machine interaction. In this talk, the author will present some examples of fabric-based WHS that can be applied to different body locations, and elicit different haptic perceptions, as well as sense user’s related information, for various application fields, including robotic-teleoperation, assistive robotics, augmented reality and human studies. The mutual inspiration between neuroscience and system design will be stressed. Finally, the emotional response that haptic systems can elicit in users will be discussed, an aspect that is often scarcely considered but that could significantly contribute to increase device acceptance and adoption in our everyday life – the latter represents the main motivation for wearable haptics. To achieve this goal, a general framework to evaluate the emotional counterpart of haptic stimuli through psycho-physiological tests and analysis of nervous system-related recordings will be described.

We present a loosely-fitting tactile jacket. On the jacket, large number of vibrators are implemented without one-to-one wiring. Vibrators can be added at arbitrary positions, in the design process of the jacket and even in the actual operation by the users themselves. When a user wears a loosely-fitting jacket, some part of the cloth contacts the user’s skin, and other parts are apart from the skin. By increasing the number of vibrators and forming clusters of vibrators, one or more vibrators in each cluster can tightly contact the skin and contribute to generate tactile sensation. The cluster-based approach improves the certainty of generating the sensation. For such an implementation of large number of vibrators, ordinary one-to-one electrical wiring would significantly degrade the flexibility. Our jacket is fabricated with double-layered conductive fabric. The fabric is woven with non-conductive and conductive threads, and both the sides of the fabric are electrically conductive. A special connector consisting of a tack and a clutch are stuck through the fabric and electrically contacts at both conductive sides without shorting them. Possible approaches to form clusters include: dividing the conductive fabric into some patches and distribute control signal to the patches; driving vibrators in the manner of frequency-division multiplexing; and combination of a serial communication technique and frequency-division multiplexing. The presented scheme can enable flexible implementation of tactile actuators on wearable systems.

Evidence for successful speech communication through the sense of touch is provided by the natural methods of tactual communication that have been used for decades by persons with profound auditory and visual impairments. However, there are still currently no wearable devices that can support speech communication effectively without extensive learning. What does it take to realize the potential of speech communication through touch that has been demonstrated by an extensive body of research? Let’s discuss the challenges and opportunities for developing wearables for sensory communication. I will also share our current efforts in developing a wearable haptic speech communication system.

Wearable robotics is opening new fields of applications in clinical neuroscience, either as augmentation of human capabilities and, possibly, as rehabilitative tools. Our experience in this sense is twofold: 1) the development and use of a supernumerary thumb (i.e., the sixth finger) for patients with paralysis of a hand following a stroke: By the sixth finger, these patients are able to relearn the use of the paretic hand for a number of everyday tasks activities, meanwhile even promoting the amount of otherwise impossible bimanual tasks; 2) the development and the use of vibrating anklets to improve walking and freezing of gait in Parkinson's Disease patients, two motor symptoms that are often difficult to treat pharmacologically. The advantages of the use of the (implicit) haptic channel versus traditional auditory or visual (explicit) cues is that the patient can maintain full interactions with the surrounding environment, without allocate his/her attention to the cue itself.

Fingertips and hands captivate the attention of most haptic interface designers, but humans can feel touch stimuli across the entire body surface. Trying to create devices that both can be worn and can deliver good haptic sensations raises challenges that rarely arise in other contexts. Most notably, tactile cues such as vibration, tapping, and squeezing are far simpler to implement in wearable systems than kinesthetic haptic feedback. This interactive discussion will present a variety of relevant projects to which I have contributed, attempting to pull out common themes and ideas for the future. My interest in arm-worn tactile displays began when I was a doctoral student with Günter Niemeyer at Stanford University. Motivated by the goal of enabling a surgeon to feel contact along the slender instruments one teleoperates in an Intuitive Surgical da Vinci system, I embedded eight eccentric- rotating-mass (ERM) motors in a line inside a long glove and drove them to simulate contact at a stationary location in space. The first NSF grant I received as an assistant professor at the University of Pennsylvania built somewhat on that early glove prototype. Working with collaborators from Moss Rehabilitation Research Institute, Karlin Bark and others on my team created a series of systems that sought to guide the motion of the wearer’s arm by controlling the vibrations delivered by eight ERM motors (four around the wrist and another four around the bicep). These vibrotactile cues aided users in performing simple but not complex arm motions. Frustrated by the slow response times and spatially diffuse sensations of ERM actuators, as well as the above system’s inability to guide wrist pronation/supination, Andrew Stanley and I created a suite of wrist-worn actuators that deliver tactile cues more like a person touching your wrist. We then compared these tapping, dragging, squeezing, and twisting actuators to a wrist band that contained six ERM motors. The best overall performance was enabled by the actuator that repeatedly taps on the subject’s wrist on the side toward which they should turn. Jeremy Brown and others recently employed the squeezing device to let the operator of a da Vinci robot feel the magnitude of the force the instruments are exerting. Finally, tactile displays can also be used to alter the wearer’s body perception. As a visiting masters student in my lab, Yosuke Kurihara wanted to make interactive games more compelling. He first recorded the vibrations that occur when a Puma 260 robot arm moved one of its gear-head-driven joints at different speeds. In his final system, the user wore a set of Tactile Labs Haptuators strapped to their joints plus an immersive VR headset that depicted a robot body that moved as they moved. The user felt robot vibrations corresponding to the present speed of each of their joints. Arm-worn tactile displays have great potential, but the design space is large and under-explored. I am currently interested in integrating motion sensing into garments and creating new wearable tactile actuation systems.

Textiles are everywhere. All of us are in direct contact with some form of textile for almost every hour of every day and night. This presents huge opportunities for interacting with our environment, but also poses many challenges. The textile and technology industries have common roots and share many processes and materials - where can crossover occur naturally and where do new solutions need to be created? What requirements to we need to be aware of when integrating electronics into textiles and what should we be mindful of when adding functionality to a system - clothing - that is thousands of years old?

Wearable haptic feedback devices for virtual reality, human-robot interaction, motion guidance, and physical support require lightweight actuators that display clearly discernible cues and potentially even large forces to the user. These goals motivate the design of a class of pneumatically actuated, body-grounded soft haptic devices. Their common feature is inflation of flexible but not stretchable thermoplastic pneumatic actuators – which can be attached to limb segments for kinesthetic feedback or directly touching the skin for tactile feedback. One example is an exomuscle, which is an inflatable device for shoulder abduction support. Designed for assistance and rehabilitation for stroke patients, the exomuscle supports the weight of the arm, offloading shoulder abductor muscles that have become erroneously coupled to elbow muscles. We constructed one type of exomuscle by reinforcing a plastic bladder with a fabric bag that is sewn to supporting straps. The bladder can then be inflated with pressurized air to provide expansive forces between the user’s torso and arm, supporting shoulder abduction. A seam acting as a hinge joint connects the exomuscle to the torso. A second example is WRAP (Wearable, Restricted-Aperture Pneumatics), which inflates a pouch with heat-sealed with switchback channels placed in direct contact with the skin, either on a held object or placed on the human body. It is capable of a variety of pulse frequencies higher and avoids sensory adaptation while delivering salient cues by exciting multiple types of mechanoreceptors. A third example is HapWRAP, A soft growing wearable haptic device that uses controlled air flow to cause an inverting plastic tube to grow out of a compact housing unit and provide a combination of directional and force feedback to a user. When activated, HapWRAP grows up and around the forearm; it loops form a temporary sleeve. After growth, WRAP-type pneumatic actuators inflate and deflate to stimulate mechanoreceptors in the skin at distinguishable locations. Future challenges in soft haptic design include potable, low cost, and quiet air supply, automated manufacturing, methods of personalization/fitting for different people, and development of real world applications demonstrating effective realism and guidance.

In the past human robotic interaction was mainly conceived as the interaction between humans and external robots. Recent advances have made clear that robotic technologies can be easily worn and of help in a variety of tasks, from augmentation of human capabilities both in terms of sensory feedback and power augmentation to novel interaction devices for exploration of virtual worlds. Now that thanks to the spread of head mounted displays virtual reality is within reach of everyone, novel interaction technologies are needed capable of providing immersive and realistic experience on all body, for applications that go beyond the traditional gaming domain. In this sense wearable haptic devices exploit a variety of sensory mechanisms and stimulations, from skin stretch or indentation to heat transfer, to elicit novel modalities of natural interaction. On the other side the advent of novel wearable robotic solutions, ranging from soft robotics to underactuated devices, has made possible to reduce fatigue over long lasting walking or during repetitive lifting loads, working in symbiosis with humans. All the above innovations are particular relevant in the medical assistance domain, where technologies offer new opportunities for rehabilitation, e.g. PCI children rehabilitation , or assistance, e.g. assistance to grasp in arm hemiparesis. But what are the challenges that new robotic designers have to address to build novel wearable devices? Science of materials, new tracking technologies, novel actuators, bioengineering based design and ergonomics are the main ingredients to achieve naturalistic human-robot symbiosis.

Head-mounted displays and unobtrusive tracking systems are making Virtual Reality (VR) experiences increasingly immersive and engaging. A further step towards realistic interactions is represented by wearable haptic devices that enable natural forms of communication between the wearer and the VR environment, including the feedback of contact forces and the rendering of objects material in terms of texture and temperature. However, wearing haptic devices on the fingertips significantly impairs the wearer’s ability of interacting with real objects, which is of paramount importance in AR. For this reason, there is a growing interest toward ring-shaped wearable interfaces capable of providing simpler haptic stimuli and that also represent a lightweight, compact, and inexpensive input technology with several potential applications in the field of computer-human interaction. Among all the applications, it is worth mentioning the control of innovative robotic devices for the compensation of hand functions in chronic stroke patients, such as supernumerary fingers. In this case, the wearable ring can be used to control the extra-finger as well as to provide cutaneous cues about the forces exerted by it on the grasped object.

In the last years, we have witnessed a drastic change in the form factor of audio and vision technologies, from heavy and grounded machines to lightweight devices that naturally fit our bodies. However, only recently, haptic systems have started to be designed with wearability in mind. The wearability of haptic systems enables novel forms of communication, cooperation, and integration between humans and machines. Compact, unobtrusive, inexpensive, easy-to-wear, and lightweight haptic devices enable researchers to provide compelling touch sensations to multiple parts of the body, significantly increasing the applicability of haptics in many fields, such as robotics, rehabilitation, gaming, and immersive systems. In this respect, wearable and portable haptics have a particularly great potential in the fields of virtual reality (VR) and augmented reality (AR). Being able to touch virtual objects in a wearable and unobtrusive way may indeed open new exciting avenues for the fields of haptics and VR. Development of these types of haptic devices is quite active, and we can already see applications targeting VR/AR entertainment applications as well as immersive robotics scenarios. This interactive discussion will focus on my recent work as well as on the perspectives of the field. Specifically, I will introduce a novel framework for the combination of tangible objects and wearable haptics to improve the display of stiffness sensations in Mixed Reality; a wearable forearm device for the shared control of robotic manipulators and aerial robots in Virtual Reality; and a set of unobtrusive wearable fingertip devices for immersive tactile experiences.

Wearable haptics refers to systems worn on the body or integrated into clothing that can provide haptic feedback to the wearer in a wide range of forms, such as vibrotactile, cutaneous, skin stretch, pressure, and kinesthetic feedback. The potential for wearable haptics is supported by technological advancements in sensing and actuation that allow for the design and deployment of flexible materials, fabrics, and small-footprint actuators that can augment and enhance human performance. While rigid and grounded haptic feedback devices have been explored as a means to train new motor skills, rehabilitate movement coordination after neurological injury, and even augment human force output capabilities in tasks such as lifting or running, the application space is limited due to the weight and scale of such devices. Vibrotactile feedback has long been used for notifications and alerts in portable devices such as phones and now smartwatches and fitness trackers, but the set of possible notifications is limited by the nature of the eccentric motors so often used to deliver such cues. In recent years, my group has been shifting to wearable haptic systems that provide multiple haptic modalities of feedback to the user, including vibration, skin stretch, pressure, and kinesthetic feedback, all packaged in bands that can be worn on the arm, or in hybrid rigid-soft exosuits that can apply forces directly to the limbs. We are particularly interested in applications where the provision of such feedback has a measurable impact on human performance. For example, we have shown the value of adding vibrotactile slip feedback to kinesthetic feedback when manipulating virtual objects, and deployed a skin stretch system to provide proprioceptive information to an amputee using an advanced prosthetic device. We have recently developed methods for real-time feedback of movement smoothness in complex manual control tasks, and have showed that such feedback can beneficially alter task completion strategies. As well, we have demonstrated that compact cutaneous feedback and guidance can be as effective at training a new motor skill as a rigid, grounded, kinesthetic feedback device. With further development of novel actuation technology, new materials, and sensors that can even be woven into our clothing, our wearable haptic systems will benefit from rich, high-resolution data that capture human intent and movement, and will be able to provide ever increasing fidelity of feedback to maximize human capabilities.

While haptics conferences today feature “wearable” as a submission keyword, what we see in terms of hardware is generally a mix of mechanisms designed for wearing, and familiar vibrotactile displays arrayed in creative arrangements. Meanwhile, in other scientific realms, smart materials are on their way: polymers and fabrics with varied “superpowers”. These materials are or will be able (in various combinations) to sense shear, pressure, stretch, bending and proximity, and display vibration, shape, and heat; while being stretchy, transparent and potentially inexpensive. While the potential is there, the design space is large and the optimization laborious. These scientists welcome application input to focus their efforts on high-value applications. What do we need? This interactive discussion will focus on brainstorming around 3-10- year-out sensing and display needs for flexible haptically active materials, for wearable and “situated” (in the world, e.g. wrapped over objects and manual controls) applications. Collaborating materials scientists may be able to participate remotely.

Compression is widely used as a physiological intervention for clinical and therapeutic applications (e.g., elastic stockings for treatment of diabetes, lymphedema, and orthostatic intolerance, etc.); however, because it is difficult to create dynamic/responsive compression on the body without using bulky inflation systems, compression is a relatively understudied mode of non-clinical haptic interaction. Recent advances in soft robotics design enables real-time, remotely controllable, dynamic (both spatially- and temporally-) compression systems. These technologies create a new and exciting haptic modality with wide-ranging potential as both a method for communication/display as well as for affective/behavioral intervention. In this discussion section, attendees will have an opportunity to learn about, and interact with, a variety of dynamic on-body compression systems using integrated shape-memory materials. Approaches that enable dynamic compression on the body will be included for discussion and demonstration.

While much has been learned on the perceptually relevant dimensions of hearing and vision, an equally comprehensive understanding of touch is still in its infancy. This challenge seeks to bring together perspectives of experts and stakeholders from a wide range of backgrounds to converge on the “dimensions” that contribute to the haptic perception of surfaces. Understanding these dimensions may ultimately help define the requirements of next-generation devices that can effectively record and playback touch for all or a specific subset of surfaces, enabling new applications in e-commerce, telerobotics and AR/VR where users can ultimately feel the virtual, remote or pre-recorded surfaces they’re touching.

Quantifying and communicating the way humans perceive touch has confounded product manufacturers for decades. While a wide range of test instruments exist to measure engineering properties (e.g. friction coefficients, surface roughness, etc.) domain experts understand that this information relates poorly to human perception (i.e. precise but not relevant). There are many reasons for this, but the simple explanation is that these instruments do not possess the sensory capabilities or mechanical properties of human skin to both elicit and sense interactions that human fingertips would otherwise perceive. As a result, those interested in measuring the haptic attributes as they relate to human perception have relied on humans – or more appropriately, trained sensory panels (the haptic equivalents to professional wine tasters). However, these approaches are largely qualitative in nature due to issues with human variability within and between subjects (i.e. relevant, but not precise). Over the last decade, our team has been developing new ways to measure tactile information that combines machine precision with relevant human perception. The approach has been heavily inspired by biomimicry. The BioTac tactile sensor developed by our group provides a decent surrogate for a human fingertip with similar mechanical properties and sensory capabilities. Human exploratory movements were investigated, modeled and optimized to produce the most salient information to discriminate objects. Tactile properties inspired by human language were modeled after theories of how these neurophysiological computations might be done in the body. Gratifyingly, the biomimetic approach achieved a very biomimetic performance in the ability to discriminate objects. The current tactile evaluation system developed by SynTouch (Toccare) measures a total of 15 tactile properties. While we have been successful in validating the comprehensiveness of these 15 dimensions to reliably discriminate materials that humans can discriminate, it is not immediately clear that this dimensionality is optimized. Ideally, such an optimized set of dimensions would be similar to the well known RGB model for color – reduced in complexity to measure, directly relatable to producing tactile perceptions, and intuitive to understand. There are indeed improvements to be made. Recently we have been exploring a methodology for developing an inherited hierarchical taxonomy to describe touch that includes four categories (texture, grip, hardness, and temperature), with more nuanced attributes built into deeper levels. My objective for this talk is to discuss processes used and challenges encountered to inspire discussion into what an optimal dimensional space for touch might look like. The ability to record and playback the human senses of sight and hearing have triggered technological revolutions. The next sense to be commercialized will be touch, so a strong understanding and definition of this dimensionality will be imperative to its success.

Any characterization of the texture of an object requires the integration of actual human perception. The science of Sensory Evaluation aims to do that by uncovering and defining the attributes that can be measured somesthetically on the skin and kinesthetically through muscle movement using descriptive analysis.

The concept of Perception Delivery at RISE means the deliberate control, delivery and prediction of the perceptual and sensory response to a product, device or material. Quantification of surface and material properties and perceptual responses are combined to increase the knowledge regarding how to modify and design surfaces to control the haptic response, which could reduce trial and error during new product development and consequently save time and cost. RISE (including former SP and YKI) has during ten ears employed a psychophysical approach in order to study the link between material properties and haptic perception. Similarity scaling of printing papers displayed rough/smooth as main perceptual dimension. Attribute scaling of perceived coarseness, coolness, dryness and smoothness all seemed related to the Ra average surface roughness parameter and most probably related to the real contact area with the finger. The role of topography was studied systematically using manufactured wrinkle‐patterned surfaces varying from 300 nm to 90 µm in wavelength, where the two perceptual dimensions were directly linked to tactile friction coefficient and wrinkle wavelength. The lowest amplitude of the wrinkles that was distinguished from the blank surface was approximately 10 nm (760 nm in wavelength). In a follow‐up study, perceived roughness and slipperiness were quantified using magnitude estimation. The results point to slipperiness as the key perception dimension for textures below 100 μm and roughness above 100 μm. In this study the same structures were replicated in NOA with elastic modulus, where the material with the higher elastic modulus was perceived slightly more slippery. The blank surfaces was scaled as rough, supporting the theory that perceived roughness itself is a multidimensional construct with both surface roughness and friction components. This was observed also on the scaling of coarseness on printing papers, where some people considered coarse as being the resistance and some considered the structure being rough. In summary, small modifications of the surface texture on fine textures affect the real contact area and consequently the tactile friction. When removing the structure, we have showed that the perceptual response can be modified by altering the surface chemistry, as shown by similarity scaling of various coated glass. Friction (slipperiness) was the main cue to distinguish among the glass surfaces. A pleasantness ranking of showed that high pleasantness was related to a lower friction coefficient. Thus both texture and surface chemistry can be used to control and modify the haptic response of a surface. A study on injection moulded master plaques (car interior surfaces) with different textures in 4 different plastic materials, showed that the texture was the dominant dimension, where some of the roughness parameters correlated better than others with the perception scale values. The relative importance of other material properties increased as the size of the texture decreased. Thus; with fine textures the choice of plastic material is more important than when having a prominent texture. The perceived quality was also more dependent on the texture than type of plastic, however; when allowing both visual and tactile cues, the material had a greater impact. In summary our studies have shown that the texture is incredible important for the haptic perception also on surfaces with smooth, small and fine structures.

Among the challenges associated with this theme are those that apply specifically to haptic displays. Here are a few: What is the range of experiences (the “tactile gamut”) that a given display can support? How can the largest possible tactile gamut be achieved? How can the realism of surface haptics be improved? Satisfactory answers to these questions don’t seem particularly close at hand, I’m afraid. In this talk, therefore, I am going to impose a few constraints in the hopes that they will lead to more narrowly defined and actionable problems … that are nonetheless interesting! I will begin by limiting myself to variable friction displays in which the gross level of friction experienced by the fingertip (µ) can be tightly controlled as a function of measured variables including finger position (x), velocity (v), normal force (Fn), and time (t). For much of the talk, I will focus mainly on µ(x) where x is a one or two-dimensional vector. Unfortunately, this doesn’t do much to limit the complexity of the problem. To illustrate why, suppose that µ(x) could take on two possible values (high or low) at each pixel on a typical smartphone screen (about two million pixels). Then the number of tactile patterns that we could generate would be two raised to the power of two million … basically infinity. And yet, the vast majority of those patterns would feel much the same: like tactile noise! This framing of the problem may seem like academic folly, a tactile version of monkeys sitting at typewriters! But, in my experience, it is thoroughly practical. One of the first tools that we built for programming haptics on the TPaD Phone was the tactile bitmap: start with an 8-bit grayscale bitmap, and convert the gray of each pixel into a friction value to be experienced as the center of the finger passes over that pixel. It is a simple tool that anyone can use (and hundreds of people have!). But, if truth be told, it is a terrible way to program haptics: for anything beyond the simplest patterns, it is impossible to predict the tactile experience by looking at the grayscale image; pictures that look very different may feel much the same; it is rarely clear how to change an image to achieve a particular tactile intent; and so on. We need a better parameterization. We need to know how to throw away extraneous information while scaffolding designers’ mental models and giving them control over a broad tactile gamut. In this talk, I’ll review the work that we have been doing toward those goals including: the notion of a “texel”, the largest region for which phase information may be disregarded; the use of low-order statistics to capture macro textures (the “Tactile Paintbrush”); the reduction of spectral information associated with fine texture; and the importance of other variables, such as velocity and time.

This Interactive Discussion will present insights on the haptic dimensions of tool-based virtual surfaces (roughness, hardness, and slipperiness) drawn from work on data-driven modeling the rendering of tool mediated surfaces. First, I will discuss how the haptic surface models are created using measurements of force, position, and acceleration recorded during physical interactions with real textured surfaces. Next, I will discuss how these surface models are used to render virtual surfaces using both vibrotactile and kinesthetic haptic devices. The haptic signals, which are a function of the interaction conditions and motions used during rendering, must respond realistically to the user’s motions in the virtual environment. We ran extensive human subject studies on both rendering schemes to test which haptic dimensions were captured purely by vibration feedback, and which ones required additional kinesthetic feedback. Our studies indicated both limitations in the surface models themselves as well as the rendering hardware in its ability to fully capture the haptic dimensions of interest. I will then conclude with a discussion of the current challenges in both haptic tool-mediated surface and rendering and the types of surfaces that current modeling method struggle to fully capture.

The changes in skin temperature that occur when the hand makes contact with an object provide information about the thermal properties of the object and these in turn assist in identifying the material composition of the object. In order to recreate these sensations in virtual environments or teleoperated robotic systems, thermal feedback has been incorporated in haptic displays. The objective of the display is to assist in object recognition in situations in which visual information may be limited or absent, and to create a more realistic experience of the contact between the hand and the object in a virtual environment. Thermal displays that assist in object identification typically consist of thermal stimulators, thermal sensors, and a temperature control system that monitors and controls the surface temperature of the thermal display. The results from a number of experiments have demonstrated that the thermal cues presented by such displays convey sufficient information to enable users to identify and discriminate between materials with a level of accuracy that is comparable to that achieved with real materials. Thermal displays have been incorporated into tactile displays with the aim of presenting a more realistic sensory experience of objects in virtual or remote environments. In these multisensory displays, thermal feedback provides information about the material composition of virtual objects and vibrotactile feedback conveys surface texture cues. Information regarding changes in skin temperature is relayed to the cerebral cortex in pathways that are separate from those that convey information about mechanical stimulation of the skin. In order to create effective multi-sensory displays, it is important to understand how these two sensory pathways interact. Results from recent experiments in which vibrotactile stimulation has been delivered concurrently with thermal stimuli indicate that vibration can have a substantial effect on the ability to identify thermal patterns. These studies demonstrated that dynamic tactile stimuli have a greater effect on thermal pattern identification than a continuous signal. Interestingly, vibrotactile stimulation had a greater effect on the perception of cold than warmth. This is an intriguing finding that is of particular relevance to perceiving the properties of objects in that most objects in contact with skin are at ambient temperature, and so it is the decrease in skin temperature that provides cues about the object’s material composition. The challenge in designing multi-sensory cutaneous displays is understanding such interactions and optimizing the temporal and spatial properties of stimulus presentation so that the user perceives a single perceptual entity and not independent sources of information.

The micrometer-scale features at the surface of an object accounts for an important part of the tactile perception of the material properties and in particular of what is defined as the texture. Running our fingers on the surface asperities creates a unique vibratory signal which can be interpreted by the somatosensory system as leather, fine grain wood or a sanded coating. Tactile texture accounts for an important part of the realism of simulated surfaces, and it has been a long-time goal of the haptic community to produce accurate virtual analogs. This interactive discussion will introduce some of the work that has been done to capture, process and reproduce realistic textures. Recording the interaction of a finger sliding onto a surface can be done by observing the minute fluctuations of the friction force that are produced during the tactile exploration. With the help of a custom-built force sensor, able to resolve micro-Newton variation, we observed that the relationship between the micro-geometry and the force fluctuations is not trivial and can vary as a function of space and time. A simple sinusoidal grating produces vibrations that vary from trial to trial and contains energy not only at the fundamental frequency but also on all observable harmonics. In addition to a complex signal, a rich background noise which amplitudes decays with the inverse of the frequency. The amplitude and the decay of the background noise is influenced by the nature of the surface and relates to the pleasantness of a particular texture. More complex structured textures exhibit frequency-spectra that are modulated as a function of the user’s position, leading to formant-like structures that seems to inform about the finish of the surface. This highly detailed force profiles can be reproduced using high-fidelity rendering devices by modulating the lateral deformation of the finger pad according to the speed of exploration of the user. Slow movement will produce low frequency oscillation while rapid stroke will increase the pitch of the same pattern. The production of a sensation congruent with the user exploration, creates a sensation similar to touching the original texture. Psychophysical experiments show that participants are able to use the spatially distributed force profile to discriminate and identify virtual texture with their real counterpart. This discussion will go over the intricacy of the development of high fidelity apparatuses to capture the finger-surface interaction and use the signals to produces realistic sensations.

In prior work, Okamoto et al. summarized the dimensions of tactile perception of textures as 1) rough/smooth, 2) hard/soft, 3) sticky/slippery, and 4) warm/cold. Among these dimensions, roughness and stickiness play important roles for texture recognition. Klatzky and Lederman showed that vibratory cues, together with spatial cues, were utilized to perceive roughness when the surface was explored with a bare finger. Their study suggests the viability of vibration in roughness augmentation. Yoshioka et al., on the other hand, studied perception of stickiness in addition to roughness. They attributed roughness to vibration, and stickiness to friction. Roughness is related to the size of surface irregularities, and stickiness or friction represents how much a finger adheres to the surface. Our central hypothesis in this discussion is that roughness and stickiness are the primary dimensions of virtual textures generated by electrostatic attraction. These textures are rendered by modulating surface friction creating vibration on the fingerpad as the finger moves across the surface. This friction-induced vibration is also called “electrovibration. We investigated above hypothesis through an experimental study. In this study, we used a data-driven texture rendering method on an electrostatic tactile display to create virtual textures. Data-driven haptic rendering is a reliable method to record data from real objects and replay them on a system without explicit knowledge about or setting of material properties. Characteristics of the input signal are of paramount importance in tactile perception on touchscreens. Considering that roughness perception is a result of vibratory stimuli, our datadriven method relies on contact accelerations occurring when an object moves on a texture. We used the haptic texture models provided in the Penn Haptic Texture Toolkit (HaTT) to obtain the input signals for the electrostatic touchscreen. Seven models were selected from the HaTT, and the real samples were obtained from the University of Pennsylvania. A psychophysical experiment was carried out with 10 participants, during which subjects rated similarities among the rendered virtual textures and the real samples. Confusion matrices were created, and multidimensional scaling (MDS) analysis was performed to create a perceptual space for further examination and to extract underlying dimensions of the textures. The results show that the virtual textures generated on the electrostatic touchscreen were similar to the real textures. Roughness and stickiness were the primary dimensions of texture perception.

Many exciting developments of friction based haptic displays have taken place in the recent years. The new generation of electrostatic and ultrasonic devices is capable to generate textures and other features such as buttons with a high-precision by modulating the friction occurring at the finger surface interface. We can now manufacture transparent devices that can generate simultaneous visual, auditory and tactile cues. However, our understanding of how the frictional cues should be designed for these textures and buttons to be felt as natural and unambiguous is still at an early stage. In the recent years, our lab has conducted several studies on the perception of frictional cues that occur naturally on materials or are generated by a high-fidelity haptic display. These studies point out key parameters that mediate our perception of patterns as well as some important factors that stem from the natural physiological properties of the fingertip such as the chemistry of the interfacial materials or the continuous variations of the skin’s humidity. In the context of ultrasonically generated changes in friction, our research enabled us to understand better how humans perceive tactile cues that can typically be generated by friction based haptic displays. One recently published study focused on the psychophysical thresholds associated to the human perception of transient changes in friction in various conditions. We found out that humans are very sensitive to sudden changes in friction and that their perceptual threshold is consistent across interfaces made from different materials. In addition, both the interfacial material and the properties of the skin were found to influence the efficiency of the ultrasonic actuation. In another study, we investigated the time intervals for which two consecutive frictional steps are sensed as one unified percept instead of two distinct cues. We found that such cues could be very ambiguous between 70ms and 90ms of duration. Based on the gathered knowledge, we have recently proposed a rendering technique that combines large spatial taxels and precise temporal modulation of the friction within each taxel. This technique enables to benefit from the spatial localization of the standard 50Hz capacitive screen and provide the fine temporal cues within the area where the user expects them. This mixed approach enables to haptic device to generate accurately a large range of periodic textures to the user independently of the speed of its tactile movement. Nevertheless, these findings are rather a beginning than the end. Based on their own psychophysical observations, other labs have different approaches based on much smaller taxels and use different waveforms to generate the desired frictional sceneries. The high number of exciting new results and the disparity of the methods and findings make us believe that an interactive discussion about the frontiers of the human perception of frictional cues will generate a large interest among the researchers who are active in the field of haptic surfaces and will initiate a lively discussion on the different results and methods for creating frictional patterns with a natural feel.

Many experimental approaches corroborate that there exist 3 to 5 factors for textural perception. The 5 factors include macroscopic and microscopic surface roughness, friction, softness/hardness, and warmness/coldness. These factors are not physically and perceptually independent from each other; nonetheless, many researchers regard them as texture dimensions and have discussed the perceptual mechanism of each dimension. These dimensions have been investigated in terms of contact mechanics, neurophysiology, and psychophysics. In order to understand each dimension, it is also beneficial to consider exploratory motion that is how we unconsciously touch surfaces. For example, human can sense surface roughness ranging from a few tens millimeter to sub-micrometer that is ten to the 5th power of spatial period. Human takes advantage of the combinations of some perceptual mechanisms and exploratory motions in order to achieve the perception of a wide range of physical information, which is demanding for artificial sensors. This talk provides how human achieves roughness perception of macro-, microscopic and super fine surfaces. Similarly to roughness perception, other textural dimensions are also addressed. Human seems to have a unique mechanism for softness and hardness perception. However, we unconsciously use at least 3 types of mechanisms depending on the hardness of objects. The speaker summarizes what are understood and missing regarding textural dimensions. Especially, hardness and friction perception remain to be studied because there do not exist mechanical receptors specific to hardness and friction. This cross-cutting talk would be a good base for comprehending material recognition and developing affectively positive materials. Furthermore, it would help discuss the directions of the studies on textural perception.

A real surface has several distinct physical properties, or physical dimensions, that must be captured and re-created in a virtual surface for realistic haptic rendering. The most prominent approach for that is data-driven modeling and rendering, which is also called record and play, haptic camera, or haptography (haptic photography). In this approach, data are measured from real haptic interaction, and the input-output relationship of the interaction is learned using a non-parametric model such as radial-basis functions, neural networks, and even images. In this talk, the presenter will describe the results of series of his research done for data-driven modeling of surface features including viscoelasticity, friction, texture, and thermal conductivity. Haptic devices used for rendering are also diverse, e.g., a force-feedback interface, a vibration actuator, a Peltier element, and a variable friction display. The presenter will also introduce some pros and cons of the data driven approach based on his experiences and lead a discussion with audience.

The presentation will cover three studies relating to surfaces. In the first we hold the unconventional view that, when we are in contact with surfaces, touch informs proprioception rather than the conventionally adopted opposite view. The case in point is the perceptual task of deciding whether an object is stationary or is moveable. This task is an example of the general perceptual problem that is faced by any organism to relate internal and external sensory information to explain a stimulus. In touch, a frequent occurrence, not shared by vision, is the modification of the state of objects by manipulation through pushing, squeezing, etc. In 'The Brain Uses Extra-Somatic Information To Estimate Limb Displacement' by Terekhov and Hayward, 2015, we show through a proprioceptive illusion induced by tactile inputs that the brain uses the prior knowledge of the mechanical properties of touched surfaces to estimate the state of the body. In 'Looking for Physical Invariants in the Mechanical Response of a Tactually Scanned Braille Dot' by Bochereau, Sinclair, and Hayward, 2015, we found that our perceptual system does not encode the intensity and the duration of brief tactile events independently, not any more than in other sensory modalities. The implications of this finding — which relates to how we sense asperities and coarse textures — will be discussed in terms of early touch since we have collected recent evidence that this tradeoff is already encoded in the spiking characteristics of peripheral afferents. The third study “Touch uses frictional cues to discriminate flat materials” by Gueorguiev, Bochereau, et al., 2016 shows that the material of optically smooth surfaces can be discriminated by touch and that the sense of touch is sensitive to materials probably through differences in hydrophobicity.

The development of epidermal electronics provides an enabling means for continuous monitoring of physiological signals and physical activities without affecting the quality of life. Such devices require high sensitivity for low-magnitude signal detection, noise reduction for motion artifacts, wearability for long-term comfort, and low-cost production for scalable manufacturing. However, the current epidermal pressure sensing devices, usually involving complex multilayer structures, have not fully addressed the aforementioned challenges. In this paper, we have presented a novel epidermal iontronic sensing (EIS) device, that is, a novel wearable pressure sensor with a single-sided configuration and reversible attachability to human skin. Notably, incorporating skin as part of the sensing architecture, the EIS devices offer an excellent pressure sensitivity (5 nF/kPa), an ultrafast mechanical response (in a sub-millisecond range), and a good long-term stability (>10,000) for both internal (body) and external (environment) mechanical stimuli. Demonstrations of its versatile application to wearable scenarios include measuring blood pressure pulsations, monitoring respiration rates, tracking muscle activities, and digitalizing hand palpation and gripping.

To accurately reproduce a sense of compliance in tactile displays, we must understand the physics of how the finger pad skin deforms. To this end, we developed a stereo imaging technique to visualize the skin through optically clear stimuli. The method achieves a 3-D spatial resolution of 60-120 microns and temporal resolution of 30 frames per second. With human subjects, we measured the skin’s deformation over a range of compliances (61-266 kPa), displacements (0-4 mm), and velocities (1- 15 mm/s). The method can differentiate patterns of skin deformation between compliances, as defined by new metrics such as surface penetration depth.

Numerous studies have been conducted to develop a tactile device for providing convincing tactile feedback. However, most of the devices are limited in portability, and restricted to delivering either texture information with vibration cues or contact orientation with force feedback. To the best of our knowledge, there has been no wearable tactile display, which can display texture information together with contact orientation. In this paper, we propose a ferro-fluid based tactile display, which is lightweight and wearable and can replicate convincing contact orientation together with texture information. New design principle of introducing ferro-fluid and minimizing moving actuator components and replacing them with permanent magnet, allows the device to be compact and increases its portability. This also enables it to provide both contact orientation and texture cues. Preliminary experimental evaluation for force profile on finger nail and finger tip has been carried out. In addition, experiments for curvature discrimination and cuing simultaneous orientation and vibrational information by using an experimental prototype have also been conducted.

There is much interest in using haptic feedback for training new skills or guiding human movement. However, the results of studies that have incorporated haptic guidance to train new skills are mixed, depending on task complexity and the method by which the haptic guidance is implemented. Subjects show dependency on the guidance forces and difficulty in discerning which aspects of the haptic feedback are related to the task dynamics and which are meant to convey task completion strategies. For these reasons, new methods to separate haptic cues for guidance from haptic feedback of task dynamics are needed. In this experiment, 30 subjects completed a trajectory following task using a wrist exoskeleton which also rendered task forces. To assist subjects, guidance cues were provided in one of three forms: (1) cutaneous forces from a wearable skin-stretch device on the ipsilateral forearm and (2) contralateral forearm, and (3) kinesthetic forces from a kinematically similar wrist exoskeleton operated by the contralateral arm. The efficacies of each guidance condition are compared by examining subject performance and learning rates. The results indicate that cutaneous guidance is nearly as effective as kinesthetic guidance, making it a practical and cost-effective alternative for spatially separated assistance.

This paper presents a novel sensory feedback device and an original method aiming to provide skin stretch feedback of three degree-of-freedom wrist movements, which are extension-flexion, radial-ulnar deviation and pronation-supination, to prosthetic hand users. In order to evaluate the performance of the device, a test setup is built. A user study with able-bodied participants is conducted. Confusion matrices are created from the experimental data, and the results are analyzed in terms of correct identification of the intended stimulations. Analysis shows that the proposed design together with the sensory feedback method is a viable option for proprioceptive feedback lacking in robotic prosthetic hands.

Touch is an essential method for communicating emotions between individuals. Humans use a variety of different gestures to convey these emotions, including squeezes, pats, and strokes. This paper presents a device for creating a continuous lateral motion on the arm to mimic a subset of the gestures used in social touch. The device is composed of a linear array of voice coil actuators that is embedded in a fabric sleeve. The voice coils are controlled to sequentially press into the user's arm to create the sensation of linear travel up the arm. We evaluate the device in a human-subject study to confirm that a linear lateral motion can be created using only normal force, and to determine the optimal actuation parameters for creating a continuous and pleasant sensation. The results of the study indicated that the voice coils should be controlled with a long duration for each indentation and a short delay between the onset of indentation between adjacent actuators to maximize both continuity and pleasantness.

In this paper, we present a new computational model for cable-conduit systems that describes interaction with non-passive environments. Unlike previous models, our model features bi-directional propagation of motion within the cable-conduit system. This allows for simulation of human-interacting systems where both the human and the robot have the capability to impose motion or force. Because of this feature, the developed model is applicable to a wide range of physical systems. The model is validated in a physical prototype through experiments involving physical interaction with a human subject. We show that our model accurately predicts behaviors observed in the experimental system.

This paper presents a model matching based controller design technique to render mass using a haptic interface. The main hindrance in stably rendering mass through the open loop impedance control is the noise added to the acceleration signal due to twice differentiation of the position signal. We show that using model matching framework we can stably and accurately render desired mass. In addition, we present that effects of inherent dynamics such as damping present in the system can be minimized using the model matching framework. Further, we perform experiments on a single degree of freedom haptic device to validate our claim.

Haptic rendering is typically done through either impedance or admittance devices, which exist at opposite ends of a spectrum from light direct-drive to non-backdrivable highly geared. The device hardware necessarily influences what the user will feel, particularly at high frequencies. While human motion is generally limited to 10Hz, virtual environments with contact transitions have the potential to excite a wider frequency spectrum. We employ the effective impedance decomposition to discuss the effects of hardware and controller selection outside of the rendering bandwidth. We also introduce an analysis of the admittance and impedance controllers with respect to sensitivity to load cell noise. Using a reconfigurable admittance/impedance device and a perceptual experiment, we explore these effects. We find that higher effective mass outside the rendering bandwidth leads to more confusion in discriminating springs in an admittance device than an impedance device.

In this paper, we describe an analytical model developed to investigate the performance limitations of admittance-based haptic interfaces. The model is used to investigate the effects that position control bandwidth and outer loop delay have on the stability and rendering range of an admittance-based interface. We show that the performance, as defined by both the minimum renderable mass and the rendering frequency range, is directly related to the closed-loop bandwidth of the inner-position loop and the amount of additional delay in the outer rendering loop. In addition, we show that the minimum renderable mass is directly proportional to the damping provided by the user which implies a stronger grip, with a higher damping, decreases the stable rendering region of the admittance-based haptics device as opposed to an impedance-based device where increased damping enhances stability Our results are validated using a one degree-of-freedom admittance-based device.

Haptic simulation systems allow human users to kinesthetically interact with virtual environment models through a robotic mechanism known as a haptic interface. The range of environment dynamics that can be stably rendered in a haptic simulation system is determined by a number of factors, including the method by which the virtual environment is implemented in discrete-time domain. Uncoupled stability, the stability of the system when it is not interfaced to any user, is regarded as one of the most stringent stability conditions for impedance-type interfaces. In this work, we analytically and experimentally investigate the system uncoupled stability conditions for various discrete implementation of damper-spring environments when position or velocity is sampled. The results point at different ranges of environment dynamics suitable for various applications.

Time delays in bilateral teleoperation systems can degrade the transparency and result in instability. For large delays, model predictive control or impedance reflective methods in which the operator interacts with the slave and environment model, can help overcome this issue. An important key to the success of these systems is to have the local virtual slave model synchronized with the slave. This is especially important when the slave gets into contact with hard environments, resulting in rapid contact force build up and slave chattering. In this paper, we propose a model-reference model-mediated teleoperation control architecture that provides enhanced synchronization between the slave and virtual slave models and keeps both the master and slave responsive. The proposed architecture is evaluated against through simulations and experiments carried out on a 1-DOF master-slave testbed.

In the design of a haptic device, it is difficult to achieve simultaneously high stiffness and low friction and inertia. In our previous research, we proposed a co-actuation method to overcome this difficulty. The method uses a physical constraint to simulate hard contact and allows the device move freely in free motion space by keeping a clearance between the physical constraint and the link of the device. A stiffness of 40N/mm and back-driving friction of less than 0.3N have been achieved in a co-actuation module of one degree-of-freedom (DOF) using an electromagnetic motor and gear reducer. However, the stiffness is not high sufficient (10N/mm) at the initial contact due to the backlash in the transmission. In this paper, we explore the possibility to use an ultrasonic motor (USM) for solving this problem. Compared with the electromagnetic motor used in the early design, the USM is able to generate a larger resistant torque without gearbox, at a fast rate. We develop a model to determine the maximum clearance between the physical constraint and the link. We verify the clearance model and the force feedback performance using a one DOF haptic device. The experimental results show that the device achieves a stiffness of 61.5N/mm and back-driving friction less than 0.4N, which implies that the USM is promising for achieving both high stiffness and low friction and inertia required by the haptic application.

A damped sinusoidal vibration is generally used to reproduce the sensation of tapping. However, the type of actuator generally used in the field of tactile display cannot produce very low-frequency vibrations, and thus cannot activate Merkel cells to produce the sensation of cutaneous pressure. In this study we propose a method that combines cathodic electrical stimulation, which produces a pressure-like sensation, with a mechanical damped sinusoidal vibration. Our experiment demonstrated that the cutaneous pressure sensation produced by cathodic electrical stimulation mostly affects the perception of softness, allowing our method to reproduce sensations of softness/hardness over a wider range than when using mechanical vibration alone. Most participants felt that the combination of these two stimulations provided a more realistic tapping sensation.

Tactile sensors are numerous and varied, and the data they provide has proven advantages in industrial and consumer products. Despite this fact, these sensors are not used to their full potential. This illustrates the need for low-cost, versatile tactile sensors. In this paper we introduce novel robotic fingertips that use low-cost components coupled with mechanical ingenuity in order to attain important and high-quality tactile data. Our robotic fingertips contain a rolling mechanism, and can sense the force applied to an object, the normal direction of the contact point, as well as parallel movement along the object's surface. We show three main uses for this fingertip. Firstly, we demonstrate how the fingertips can be used for the detection of soft or lightweight objects by applying extremely small forces to them. Secondly, we present a method of full-perimeter definition (location and normal direction) of rigid objects by tracing. Lastly, we explain how our sensors can be used to detect stiffness and stiffness anomalies in soft objects, such as organic tissue.

A study of the human sense of touch has attracted much interest for a long time. Various kinds of sensors have been studied with various technologies for the sensing of external stimuli. Tactile sensors using capacitive, piezoresistive, piezoelectric and pyroelectric mechanism have been suggested to sense pressure, vibration and temperature. Unfortunately, it is rare to find a studying results of tactile sensor related with a surface information detection in comparison with pressure and temperature sensing. The ability to sense a surface information is the important factor to distinguish surface feelings such as roughness and smoothness. In this paper, a sensor array using piezoelectricity is demonstrated with a sliding motion to sense surface information such as the pitch and width of surface features. The suggested sensor array has a high spatial resolution(500um) and detected the variable sliding speeds thanks to the excellent dynamic response of the piezoelectric material. The reconfigured surface information had high accuracy compared to actual one. The demonstrated sliding speed sensing, high spatial resolution and the pitch and width of surface features measurement capabilities allow the sensor to be utilized as an intelligent artificial tactile sensor.

This work presents the development of a novel thermal display for transient heat rendering in virtual environments. The design of the fingertip device focused on the low weight and compactness in order to achieve a high level of wearability and high performance in thermal rendering. The device is intended for integration with the Haptic Thimble device, in order to develop a novel interface for thermal and haptic rendering of virtual surfaces. An iterative Finite Elements (FE) procedure has been performed within the design process in order to numerically validate the final design of the thermal module. Finally, a thorough experimental characterization has been performed in order to evaluate the dynamic thermal capabilities of the prototype, considering also the finger contact response and the temperature uniformity of the plate in contact with the finger.

Inspired by needs for haptic support of large motions on a surface (in embodied conceptual learning, commercial design, and 2D virtual / augmented reality), we present the ballpoint drive. This novel approach circumvents conventional constraints by imposing a new one: motion restricted to rolling on an arbitrary two dimensional surface, and grounding forces generated through friction. We analyze the ballpoint's design considerations in the context of our framing applications, describe a first prototype and its performance, and assess its potential for further development.

We proposed a projection mapping interface enabling interaction with unreachable objects by overlaid graphical hands. A user's motion on a touch panel is amplified, and the user's virtual hand (VH) is graphically extended on a real scene. While the user manipulates VH by the user's right hand, haptic feedback is provided to the other hand in our system. The more fingers the haptic feedback is given, the stronger the sense of touching real object (STRO). In the demo, the user will experience STRO to the object which only VH touches.

This research explores using SynTouch’s tactile measurement system (Toccare) to record texture properties and novel haptic displays to replay them. The Toccare uses a biomimetic tactile sensor that explores surfaces to measure 15 dimensions of touch (Fishel, 2012). When inverting measured data to create haptic display functions, the realism of those rendered textures when compared to their physical counterparts was highly restricted by the capabilities of the display. A wide range of displays were explored and our best results were achieved when simulating various sizes of wire mesh on an electrostatic friction display (Meyer, 2013; Tanvas)

In this study, vibrotactile perception of a person in a video is estimated from the visual and audio information by using neural network. In the estimation, we limit the video scene to the back view of a tennis player rallying. But, our method can work regardless of the difference in some conditions such as locations, a player’s clothes, and sound environments. We use tennis videos taken in three locations for neural network learning of the relation between the video and measured acceleration of the racket grip. Then we show the grip sensation can be successfully estimated from an unknown video which was taken in a different location from learning. The quality of the produced vibrotactile sensation is also evaluated by a subject experiment. In the demonstration, user can experience the estimated (or real) vibrotactile sensation with our device imitating a tennis racket grip while watching the corresponding video.

The Biometric Campfire art project operates via the principle of differential biopotential signal amplification. The participants place their hands upon the campfire chairs’ domed silver / silver chloride surfaces and the detected electrical signals are amplified approximately 30,000 times. The detected signals are the participants’ Lead I electrocardiograms. This Lead is a standard measure during a typical medical electrocardiograph (ECG) recording. The amplified ECGs are directed to haptic (touch sensation) transducers within the chairs and illuminating light sources constituting the “campfire”. The visceral environment generated within the confines of the Biometric Campfire is fully determined by the participants’ measured ECGs.

We propose a data-driven method for realistic texture rendering on an electrovibration display. To compensate the nonlinear dynamics of an electrovibration display, we use nonlinear autoregressive with external input (NARX) neural networks as an inverse dynamics model of an electrovibration display. The neural networks are trained with lateral forces resulting from actuating the display with a pseudo-random binary signal (PRBS). The lateral forces collected from the textured surface with various scanning velocities and normal forces are fed into the neural network to generate the actuation signal for the display. The generated signal is interpolated by user’s scanning velocities and normal forces measured by an infrared-ray frame and a load cell in real time.

This work proposes a haptic-enabled immersive virtual reality system, capable of conveying the sensation of touching objects made of different materials by rendering pressure and texture stimuli through a moving platform and a vibrotactile motor. At the beginning, the virtual environment is composed of three objects made of different materials and placed on the table. A hand model is also rendered to mimic the user’s hand pose. After a familiarization phase, subjects are presented with a different environment, having only one object on the table and whose material is hidden. Users have to interact with it and guess its material.

In our study, we proposed a method that combines cathodic electrical stimulation, which produces a pressure-like sensation, with a mechanical damped sinusoidal vibration for expressing the sensation of tactile softness in the tapping process. Experiment showed that pressure sensation produced by cathodic stimulation mostly affects the perception of softness, allowing our method to reproduce sensations of softness/hardness over a wider range than when using mechanical vibration alone. In the demo, participants can experience the sensation produced by this combination of stimulation and compare it to the sensation produced by each of these stimulation.

Touchy is a novel approach to enhance images on touchscreens with haptic effects through purely visual cues. A symbolic cursor is introduced under the user's finger(s), which shape and motion are altered in order to express a variety of haptic properties : hardness/softness, roughness/smoothness, bumpiness/flatness, or stickiness/slipperiness. Because it is purely software, Touchy does not require additional hardware and is very easy to widespread. It is multitouch and several users can experiment independent pseudo-haptic effects simultaneoulsy. We propose to Haptic Symposium attendees to interact with a gallery of haptic images showcasing six different effects on a tactile tablet.

shapeShift is a compact, high-resolution (7 mm pitch), mobile tabletop shape display. We explore interactions enabled by 2D spatial manipulation and self-actuation of a tabletop shape display. On a passive platform, the user is able to freely move and spin the display as it renders spatially relevant content. We introduce use cases for rendering lateral I/O elements, exploring volumetric datasets, and grasping objects. On a self-actuated platform, shapeShift can display moving objects and provide both vertical and lateral kinesthetic feedback. We show its use as an encounter-type haptic device combined with a head-mounted display.

We propose a novel tactile sensing method for imaging pressure distribution on an object based on electromechanical boundary impedance tomography (EmBIT). The sensor consists of driving and proving layers which are electromechanically coupled according to the pressure and the system solves an optimization problem to find the electromechanical boundary impedance related to contact pressure. The potential at each electrode is used to find the electrical boundary condition between the layers. The proposed method can be customized for an object of arbitrary shape. In the demonstration, we show the imaging results using the sheet-type and finger-type sensors in real time.

We present a haptic texture authoring algorithm for synthesizing new virtual textures by manipulating affective properties of existing real textures. Two different spaces are established: ``affective space'' built from a series of psychophysical experiments and ``haptic model space'' built on features from tool-surface contact vibrations. Another space, called ``authoring space'' is formed by merging the two spaces, whereby, features of model space that were highly correlated with affective space become axes of the space. Thus, new texture signal corresponding to any point in authoring space can be synthesized by weighted interpolation of nearest real surfaces in perceptually correct manner.

We present a touch-sensitive tactile display based on an array of bubble-like actuators integrated with a microfluidic chip. Each bubble is addressed by a corresponding fluidic memory circuit - the fluidic equivalent of an electronic memory circuit – that stores binary information (1s and 0s) as high and low pressures. The bubbles and memory circuits are fabricated together in a thin microfluidic chip using large-scale integration and low-cost silicone materials. The fluidic memory circuits are arranged within the chip to realize an XY-addressable static memory device. A minimal number of electronic valves can control an exponentially larger number of memory circuits and corresponding bubbles. The resulting thin silicone chip is arranged on top of a touchscreen to enable co-located input and output functionality.

Stroke has been identified globally as the leading cause of disability. These disabilities hinder performance of activities of daily living (ADL) such as writing, using cutlery, knife, tooth brush etc. which require fine motor control of the patient's hand. In this research, we adopt haptics technology to rehabilitate ADL skills in a home-setting. Using a combination of haptics and virtual-reality, we present a system that imitates a massed-practice-therapy regime that benefits from the inherent advantages of implicit therapy. The current implementation involves three tasks: handling a spoon, writing and 3D trajectory tracking game for improving fine motor skills.

A miniature three degree-of-freedom actuator provides haptic stimulation via skin deformation on a user’s forearm. The device, which consists of a parallel delta-mechanism linkage driven by three servo motors, can produce skin stretch, normal force, and low frequency vibration. The 2 cm diameter footprint of the device enables multiple devices to be packed together with a high density, enabling an array of stimulation locations. The device was used to measure the directional skin stretch discriminability of human subjects on the forearm, and preliminary tests suggest that average angular error is approximately 27 degrees.

This demonstration will showcase our work in designing and creating the HUE tablet: a mobile platform capable of generating ultrasonic and electrostatic surface haptic sensations on a single surface. At 11.75” x 8.75”, this device is approximately the size of a typical tablet. Hands on demonstrations will convey to the user the broad ranges of haptic information achieved by leveraging these haptic modalities simultaneously. This will be done with a series of exercises that compare each individual modality’s effective haptic ranges to the one created by a hybridized haptic technique to access the efficacy of the hybridized approach.

In this study, we tested a novel system for manipulating the sensation of hardness and softness using a touch panel in augmented reality (AR) space. Many studies of touch panel-based haptic displays have developed methods for manipulating macro-roughness (shape), fine-roughness (texture), and friction. However, few studies have examined the sensation of softness. Here we used a film-winding mechanism to present tangential force opposite to the tangential displacement of the fingers, which functioned as a sufficient cue for the sensation of softness. We also combined this with a visual cue for softness which indicated visual deformation of the object.

This demonstration presents a new approach for the haptic augmentation of surfaces via interactive touch amplification and action-dependent processing. The system consists of a finger-worn device, including a cutaneous vibration sensor and an actuator for stimulating the skin; a method of tracking the finger using a projected touch surface; and associated electronics. When a user touches a surface area that has been assigned a function, the touch-elicited vibrations are processed and amplified, augmenting what is felt by the finger in real-time. This system can augment everyday surfaces with tactile feedback, and could prove relevant to applications in human computer interfaces.

We report an electroadhesive approach to controlling friction forces on sliding fingertips which is capable of producing vibrations across an exceedingly broad range of tactile, audible, and ultrasonic frequencies. Vibrations on the skin can be felt directly, and vibrations in the air can be heard emanating from the finger. Additionally, we report evidence from an investigation of the electrical dynamics of the system suggesting that an air gap at the skin/surface interface is primarily responsible for the induced electrostatic attraction underlying the electroadhesion effect. We developed an experimental apparatus capable of recording friction forces up to a frequency of 6 kHz, and used it to characterize two different electroadhesive systems, both of which exhibit flat force magnitude responses throughout the measurement range. These systems use custom electrical hardware to modulate a high frequency current and apply surprisingly low distortion, broadband forces to the skin. Recordings of skin vibrations with a laser Doppler vibrometer demonstrate the tactile capabilities of the system, while recordings of vibrations in the air with a MEMS microphone quantify the audible response and reveal the existence of ultrasonic forces applied to the skin via electronic friction modulation. Implications for surface haptic and audio-haptic displays are briefly discussed.

We describe a new haptic force feedback device capable of creating lateral shear force on a bare fingertip—the eShiver. The eShiver creates a net lateral force from in-plane oscillatory motion of a surface synchronized with a “friction switch” based on Johnsen-Rahbek electroadhesion. Using an artificial finger, a maximum net lateral force of ± 300 mN is achieved at 55 Hz lateral oscillation frequency, and net force is shown to be a function of velocity and applied voltage, as well as the phase between them. A second set of experiments is carried out on a human finger, and a lateral force of up to ± 450 mN is achieved at a lateral oscillation frequency of 1,000 Hz. This force is reached at a peak lateral surface velocity of 400 mm/s and a peak applied voltage of 400 V. We develop a simple lumped parameter model of the eShiver, and a time domain simulation of the artificial finger is shown to agree with the experimental results. Three distinct zones of operation are found, which predict the limitations of force generation and which may be used for optimization. The human finger is found to be similar to the artificial finger in its dependence on actuation parameters, suggesting that the same lumped parameter model may be applied, albeit with different parameters. Curiously, the friction force due to Johnsen-Rahbek electroadhesion is found to increase substantially over time as the finger remains in contact with the surface. Considerations for optimizing the performance of the eShiver are discussed.

In this paper we focused on the building a universal haptic texture models library and automatic assignment of haptic texture models to any given surface from the library based on image features. It is shown that a relationship exists between perceived haptic texture and its image features, and this relationship is effectively used for automatic haptic texture model assignment. An image feature space and a perceptual haptic texture space are defined, and the correlation between the two spaces is found. A universal haptic library was built, using 84 real life textured surfaces, by training a multi-class support vector machine with radial basis function kernel. The perceptual space was classified into perceptually similar clusters using K-means. Haptic texture models were assigned to new surfaces in a two step process; classification into a perceptually similar group using the trained multi-class support vector machine, and finding a unique match from within the group using binarized statistical image features. The system was evaluated using 21 new real life texture surfaces and an accuracy of 71.4 % was achieved in assigning haptic models to these surfaces.

Recent studies suggest that vibration of piano keys affect the perceived quality of the instrument, as well as the dynamic control and timing in piano playing. However, the time signals of piano key vibrations and its physical properties have not been analyzed and compared to the threshold of vibration sensation in a real-life playing situation yet. This study investigates piano key vibrations and explores the diversity of vibrations among different pianos with a laser Doppler vibrometer. A pianist was performing single keystrokes, note sequences, and a music piece excerpt on four concert grand pianos, five grand pianos, and two upright pianos. The measurements showed peak displacement levels up to 80 um and the frequency spectrum of the vibrations is dominated by frequencies lower than 500 Hz. Finally, a frequency weighting filter is introduced to show that vibration displacement time signals exceed the threshold of human vibration sensation for all evaluated instruments, when a note sequence is played in the bass to mid range with a single hand at forte level. The conducted experiments demonstrate that the vibration characteristics vary distinctively among the investigated pianos.

Haptic simulation systems, which typically implement virtual environments in the discrete-time domain, present an inherent trade-off between stability, sampling frequency and the range of implementable environment dynamics. Previous research has demonstrated the potential of analog feedback for expanding the range of environment dynamics that result in a stable haptic interaction. In this paper, the effect of various system parameters on the environment dynamic range is analytically and experimentally investigated in the sense of uncoupled stability. In addition, Multilayer Decomposition, which enables a simple analog PD controller to implement nonhomogeneous or multilayer virtual environment dynamics, will be presented and evaluated.

Refreshable displays for tactile graphics are typically composed of pins that have smaller diameters and spacing than standard braille dots. We investigated configurations of high-density pins to form braille text on such displays using non-refreshable stimuli produced with a 3D printer. Normal dot braille (diameter 1.5 mm) was compared to high-density dot braille (diameter 0.75 mm) wherein each normal dot was rendered by high-density simulated pins alone or in a cluster of pins configured in a diamond, X, or square; and to blobs that could result from covering normal braille and high-density multi-pin configurations with a thin membrane. Twelve blind participants read MNREAD sentences displayed in these conditions. For high-density simulated pins, single pins were as quickly and easily read as normal braille, but diamond, X, and square multi-pin configurations were slower and/or harder to read than normal braille. We therefore conclude that as long as center-to-center dot spacing and dot placement is maintained, the dot diameter may be open to variability for rendering braille on a high density tactile display.

We present a novel 3RRS wearable fingertip device for the rendering of stiffness. It is composed of a static upper body and a mobile end-effector. The upper body is located on the nail side of the finger, supporting three servo motors, and the mobile end-effector is in contact with the finger pulp. The two parts are connected by three articulated legs, actuated by the motors. The end-effector can move toward the user's fingertip and rotate it to simulate contacts with arbitrarily-oriented surfaces. Moreover, a vibrotactile motor placed below the end-effector conveys vibrations to the fingertip. The proposed device weights 25 g for 35×50×48 mm dimensions. To test the effectiveness of our wearable haptic device and its level of wearability, we carried out two experiments. The first experiment tested the capability of our device in differentiating stiffness information, while the second one focused on evaluating its applicability in an immersive virtual reality scenario. Results showed the effectiveness of the proposed wearable solution, with a JND for stiffness of 208.5 ± 17.2 N/m. Moreover, all subjects preferred the virtual interaction experience when provided with wearable cutaneous feedback, even if results also showed that subjects found our device still a bit difficult to use.

Although Augmented Reality (AR) has been around for almost five decades, only recently we have witnessed AR systems and applications entering in our everyday life. Representative examples of this technological revolution are the smartphone games “Pokémon GO” and “Ingress” or the Google Translate real-time sign interpretation app. Even if AR applications are already quite compelling and widespread, users are still not able to physically interact with the computer-generated reality. In this respect, wearable haptics can provide the compelling illusion of touching the superimposed virtual objects without constraining the motion or the workspace of the user. In this paper, we present the experimental evaluation of two wearable haptic interfaces for the fingers in three AR scenarios, enrolling 38 participants. In the first experiment, subjects were requested to write on a virtual board using a real chalk. The haptic devices provided the interaction forces between the chalk and the board. In the second experiment, subjects were asked to pick and place virtual and real objects. The haptic devices provided the interaction forces due to the weight of the virtual objects. In the third experiment, subjects were asked to balance a virtual sphere on a real cardboard. The haptic devices provided the interaction forces due to the weight of the virtual sphere rolling on the cardboard. Providing haptic feedback through the considered wearable device significantly improved the performance of all the considered tasks. Moreover, subjects significantly preferred conditions providing wearable haptic feedback.

Novel wearable tactile interfaces offer the possibility to simulate tactile interactions with virtual environments directly on our skin. But, unlike kinesthetic interfaces, for which haptic rendering is a well explored problem, they pose new questions about the formulation of the rendering problem. In this work, we propose a formulation of tactile rendering as an optimization problem, which is general for a large family of tactile interfaces. Based on an accurate simulation of contact between a finger model and the virtual environment, we pose tactile rendering as the optimization of the device configuration, such that the contact surface between the device and the actual finger matches as close as possible the contact surface in the virtual environment. We describe the optimization formulation in general terms, and we also demonstrate its implementation on a thimble-like wearable device. We validate the tactile rendering formulation by analyzing its force error, and we show that it outperforms other approaches.

Many tasks involve the fine manipulation of objects despite limited visual feedback. In such scenarios, tactile and proprioceptive feedback can be leveraged for task completion. We present an approach for real-time haptic perception and decision-making for a haptics-driven, functional contour-following task: the closure of a ziplock bag. This task is challenging for robots because the bag is deformable, transparent, and visually occluded by artificial fingertip sensors that are also compliant. A deep neural net classifier was trained to estimate the state of a zipper within a robot's pinch grasp. A Contextual Multi-Armed Bandit (C-MAB) reinforcement learning algorithm was implemented to maximize cumulative rewards by balancing exploration versus exploitation of the state-action space. The C-MAB learner outperformed a benchmark Q-learner by more efficiently exploring the state-action space while learning a hard-to-code task. The learned C-MAB policy was tested with novel ziplock bag scenarios and contours (wire, rope). Importantly, this work contributes to the development of reinforcement learning approaches that account for limited resources such as hardware life and researcher time. As robots are used to perform complex, physically interactive tasks in unstructured or unmodeled environments, it becomes important to develop methods that enable efficient and effective learning with physical testbeds.

This paper describes the properties of proprioceptive sensations induced by asymmetric vibration using a vibration speaker-type non-grounded haptic interface. We confirm that the vibration speaker generates a perceived force that pulls or pushes a user's hand in a particular direction when an asymmetric amplitude signal that is generated by inverting a part of a sine wave is input. In this paper, to verify the system with respect to various factors of force perception caused by asymmetric vibration, we conducted six experiments and the following results were obtained. (1) The force vector can be controlled by reversing the asymmetric waves. (2) By investigating the physical characteristics of the vibration, asymmetric vibration was confirmed. (3) The presentation of vibration in the shear direction on the finger pad is effective. (4) The point of subjective equality of the perceived force can be controlled by up to 0.43 N by changing the amplitude voltage of the input signals. (5) The minimum stimulation time required for force perception is 66.7 ms. (6) When the vibration is continuously presented for 40 to 50 s, the perceived force decreases because of adaptation. Hence, we confirmed that we can control both the direction and magnitude of the reaction force by changing the input signal of the vibration speaker.

We describe an instrument intended to study finger contacts under tangential dynamic loading. This type of loading is relevant to the natural conditions when touch is used to discriminate and identify the properties of the surfaces of objects-it is also crucial during object manipulation. The system comprises a high performance tribometer able to accurately record in vivo the components of the interfacial forces when a finger interacts with arbitrary surfaces which is combined with a high-speed, high-definition imaging apparatus. Broadband skin excitation reproducing the dynamic contact loads previously identified can be effected while imaging the contact through a transparent window, thus closely approximating the condition when the skin interacts with a non-transparent surface during sliding. As a preliminary example of the type of phenomenon that can be identified with this apparatus, we show that traction in the range from 10 to 1000 Hz tends to decrease faster with excitation frequency for dry fingers than for moist fingers.

This paper describes a haptic interface that has a uniform response over the entire human tactile frequency range. Structural mechanics makes it very difficult to implement articulated mechanical systems that can transmit high frequency signals. Here, we separated the frequency range into two frequency bands. The lower band is within the first structural mode of the corresponding haptic device while the higher one can be transmitted accurately by a fast actuator operating from conservation of momentum, that is, without reaction forces to the ground. To couple the two systems, we adopted a channel separation approach akin to that employed in the design of acoustic reproduction systems. The two channels are recombined at the tip of the device to give a uniform frequency response from DC to one kHz. In terms of mechanical design, the high-frequency transducer was embedded inside the tip of the main stage so that during operation, the human operator has only to interact with a single finger interface. In order to exemplify the type of application that would benefit from this kind of interface, we applied it to the haptic exploration with microscopic scales objects which are known to behave with very fast dynamics. The novel haptic interface was bilaterally coupled with a micromanipulation platform to demonstrate its capabilities. Operators could feel interaction forces arising from contact as well as those resulting from Brownian motion and could manoeuvre a micro bead in the absence of vision.

Recently, many studies examined electrostatic tactile feedback on touch screens to enrich interaction experience. However, it is unclear as to whether added tactile feedback during a sliding process increases the accuracy of pan gestures with velocity constraints. In this study, a custom-designed electrostatic tactile display was considered. Initially, the accuracy and efficiency of pan gestures were compared under two conditions, namely with and without electrostatic tactile feedback. This was followed by exploring the evolution of completion time (CT) with different indices of difficulties (ID). Experimental results with twelve participants indicated that the accuracy and completion time of pan gestures with added tactile feedback significantly exceeded those without tactile feedback. Furthermore, the relationship between CT and ID satisfied Fitts' Law with a correlation coefficient exceeding 0.9. Based on the findings, a “Tactile Fruit Sorting” game was designed, and subjective and objective evaluations were conducted. The results confirmed that the added tactile feedback enhanced both user performance and interest with respect to the game.

To present information using vibrotactile stimuli in wearable devices, it is fundamental to understand human performance of localizing vibrotactile cues across the skin surface. In this paper, we studied human ability to identify locations of multiple vibrotactile cues activated simultaneously on both arms. Two haptic bands were mounted in proximity to the elbow and shoulder joints on each arm, and two vibrotactile motors were mounted on each band to provide vibration cues to the dorsal and palmar side of the arm. The localization performance under four conditions were compared, with the number of the simultaneously activated cues varying from one to four in each condition. Experimental results illustrate that the rate of correct localization decreases linearly with the increase in the number of activated cues. It was 27.8% for three activated cues, and became even lower for four activated cues. An analysis of the correct rate and error patterns show that the layout of vibrotactile cues can have significant effects on the localization performance of multiple vibrotactile cues. These findings might provide guidelines for using vibrotactile cues to guide the simultaneous motion of multiple joints on both arms.

In many domains, it's important to understand the ways in which humans learn and develop new motor skills effectively and efficiently. For example, in dental operations, the ability to apply a weak force with a required tolerance is a fundamental skill to ensure diagnostic and treatment outcome, but acquiring such a skill is a challenge for novices. In this paper, we focus on motor memory for producing normally applied force by a hand-held probe and we compare the effects of two feedback methods on motor memory consolidation. Fourteen participants were randomly assigned to two groups: a Concurrent Group and a Delayed Group. Participants in the Concurrent Group were trained to apply a target force with concurrent visual feedback, while those in the Delayed Group were trained with delayed visual feedback. The task included two phases: a Training/Testing Phase, and a Retention Phase. The results indicated that participants in the Delayed Group obtained more effective learning outcomes and better retention effects. These findings provide a new perspective to explore the relationship between feedback methods and the cognitive process of motor skill learning, and open a new way to train motor skill using more effective methods than the traditional concurrent feedback approaches.

What are the technological bottlenecks in virtual assembly simulations with haptic feedback? To tackle this question, we present an evaluation study in which real feedback modalities are gradually replaced by synthetic ones. In particular, the effects of the following factors on the user performance and perception during virtual assemblies are analyzed: (i) a visual feedback system consisting of an nVisor head-mounted display, (ii) our haptic device HUG suited for unscaled upper-body movements, and (iii) our novel six-DoF constraint-based haptic rendering algorithm. Besides of that, the influence of (iv) real collision sounds is also examined to a lesser extent. The experimental assembly scenario consisted of three variations of peg-in-hole tasks which were performed by a total of N = 24 participants in a within-design study. The mentioned three synthetic factors (i)-(iii) gradually replaced in five degrees or steps the real feedback sources, ending up in completely virtual assembly simulations. For each of the degrees, three objective variables (completion time, collision forces, and muscular effort) and five subjective ratings (related to the perception of realism and the workload) were recorded and statistically analyzed. In order to explain subjective perception also with objective measures, reaction times of a secondary audio task performed in parallel with the assembly exercises were recorded, too. While previous works have mainly focused on differences of completion time between real and virtual manipulations, our results show how all of the mentioned twelve performance and perception indicators are influenced by each of the four varied feedback factors, building a multi-modality relationship function that maps our or similar systems and expected user responses. In general, the haptic feedback modality turned out to have the largest impact on the dependent variables, particularly the HUG interface, whereas audio cues seemed to be less significant. We quantify these previous and further qualitative statements within the domain defined by the used systems. Moreover, the relationship of our insights with related other work is discussed, and their projections are outlined.

Grasping and manipulating an object requires us to perceive its material compliance. Compliance is thought to be encoded by relationships of force, displacement and contact area at the finger pad. Prior work suggests that objects must be sufficiently deformed to become discriminable, but the utility of time-dependent cues has not been fully explored. The studies herein find that the availability of force-rate cues improve compliance discriminability so as to require less deformation of stimulus and finger pad. In particular, we tested the impact of controlling force-rate and displacement-rate cues in passive touch psychophysical experiments. An ink-based method to mark the finger pad was used to measure contact area per stimulus, simultaneously with displacement and force. Compliances spanned a range harder and softer than the finger pad. The results indicated harder compliances were discriminable at lower peak forces when the stimulus control mode was displacement-rate (0.5 N) compared to force-rate (1.3 N). That is, when displacement-rate was controlled to be equal between the two compliances, the resultant force-rate psychophysical cues could be more readily discriminated. In extending prior studies, while some magnitude of finger pad deformation may be sufficient for discriminability, temporal cues tied to force afford more efficient judgments.

Smart materials such as magnetorheological fluids (MRF) offer an interesting technology for use in haptic displays as changes in the magnetic field are rapid, reversible and controllable. These interfaces have been evaluated in a number of medical and surgical simulators where they can provide cues regarding the viscoelastic properties of tissues. The objective of the present set of experiments was first to determine whether a shape embedded in the MRF could be precisely localized and second whether ten shapes rendered in a MRF haptic display could be accurately identified. It was also of interest to determine how the information transfer associated with this type of haptic display compares to that achieved using other haptic channels of communication. The overall performance of participants at identifying the shapes rendered in the MRF was good with a mean score of 73% correct and an Information Transfer (IT) of 2.2 bits. Participants could also localize a rigid object in the display accurately. These findings indicate that this technology has potential for use in training manual palpation skills and in exploring haptic shape perception in dynamic environments.

This paper presents a system of two double-gimbal control moment gyroscopes (CMGs) for providing ungrounded kinesthetic haptic feedback. By spinning a second flywheel opposite the first, and rotating them through opposite trajectories, undesired gyroscopic effects can be eliminated, isolating a single torque axis. This produces a moment pulse proportional to the flywheel spin speed and rotation speed. Rotating the CMG gimbals quickly in one direction, then resetting them more slowly generates repeated torque pulses indicating a clear direction cue. We present the mathematical model for moments produced by this system and verify that the performance of our device matches this model. Using these asymmetric moment pulses, we provide haptic cues to participants in two studies. In the first study, users simply identify the direction of torque cues. In the second study, we use the torque pulses to guide users to target orientations. Performance in both studies shows that this system has the potential to provide useful guidance for applications where ungrounded haptic feedback is desired.

Multi-finger operations provide realistic and natural methods when interacting with remote or virtual environment. Hence, haptic devices with multi-finger input are highly desirable. MR (Magneto-rheological) actuators are preferable options in haptics, because they can produce larger passive torque and have larger torque-volume ratios than the conventional actuators. Among the existing haptic MR actuators, most of them are still bulky and heavy. If they were smaller and lighter, they would become more suitable for haptics. In this paper, a small-scale yet powerful MR actuator was designed to build a multi-finger interface for the 6 DOF haptic device. The compact structure was achieved by adopting the multi-disc configuration. Based on this configuration, the MR actuator can generate the maximum torque of 480 N.mm with dimensions of only 36 mm diameter and 18 mm height. Performance evaluation showed that it can exhibit a relatively high dynamic range and good response characteristics when compared with some other haptic MR actuators. The multi-finger interface is equipped with three MR actuators and can provide up to 8 N passive force to the thumb, index and middle fingers, respectively. An application example was used to demonstrate the effectiveness and potential of this new MR actuator based interface.

This paper describes the development of an encountered-type haptic interface that can generate the physical characteristics, such as shape and rigidity, of three-dimensional (3D) virtual objects using an array of newly developed non-expandable balloons. To alter the rigidity of each non-expandable balloon, the volume of air in it is controlled through a linear actuator and a pressure sensor based on Hooke's law. Furthermore, to change the volume of each balloon, its exposed surface area is controlled by using another linear actuator with a trumpet-shaped tube. A position control mechanism is constructed to display virtual objects using the balloons. The 3D position of each balloon is controlled using a flexible tube and a string. The performance of the system is tested and the results confirm the effectiveness of the proposed principle and interface.

Despite the fact that conventional haptic interfaces and rendering algorithms commonly approximate interactions with force only, the dynamic effects of even simple tasks, e.g., writing on a paper, involve both forces and torques. To extend previous algorithms as well as to investigate the effects of torque feedback on human roughness perception, we deployed a novel haptic platform with two probes, fingertip and penhandle. Three torque conditions were examined: 1) Slope Torque, which orients the probe perpendicular to the surface, 2) No Torque, where no active torque is provided by the device, and 3) Stiff Torque, where torque feedback is provided to keep the probe upright. A conventional magnitude estimation experiment was performed. The results indicated that both the torque signals and grasp type mediate human perception of virtual textures. Slope Torque led to greater perceived roughness when the fingertip was used, and the fingertip led to higher roughness ratings than the penhandle with the Slope Torque condition. The Slope Torque algorithm appears to be advantageous for generating rougher surfaces compared to the force-based algorithms which are typically limited by the system stability and actuator saturation.

We present PhysVib: a software solution on the mobile platform extending an open-source physics engine in a multi-rate rendering architecture for automatic vibrotactile feedback upon collision events. PhysVib runs concurrently with a physics engine at a low update rate and generates vibrotactile feedback commands at a high update rate based on the simulation results of the physics engine using an exponentially-decaying sinusoidal model. We demonstrate through a user study that this vibration model is more appropriate to our purpose in terms of perceptual quality than more complex models based on sound synthesis. We also evaluated the perceptual performance of PhysVib by comparing eight vibrotactile rendering methods. Experimental results suggested that PhysVib enables more realistic vibrotactile feedback than the other methods as to perceived similarity to the visual events. PhysVib is an effective solution for providing physically plausible vibrotactile responses while reducing application development time to great extent.

Electrovibration technology has the potential for seamless integration into ordinary smartphones and tablets to provide programmable haptic feedback. The aim of this work is to seek effective ways to improve 3D perception of visual objects rendered on an electrovibration display. Utilizing a gradient-based algorithm, we first investigated whether rendering only lateral frictional force on an electrovibration display improves 3D shape perception compared to doing the same using a force-feedback interface. We observed that although users do not naturally associate electrovibration patterns to geometrical shapes, they can map patterns to shapes with moderate accuracy if guidance or context is given. Motivated by this finding, we generalized the gradient-based rendering algorithm to estimate the surface gradient for any 3D mesh and added an edge detection algorithm to render sharp edges. Then, we evaluated the advantages of our algorithm in a user study and found that our algorithm can notably improve the performance of 3D shape recognition when visual information is limited.

Information provided by sensory systems is inherently ambiguous as to its source in the physical world. To arrive at a coherent representation, perception deploys heuristic rules and multimodal input, which potentially produce errors such as illusions. The current work uses these effects to create apparent tactile motion and illusory depth motion using sparse vibrotactile stimulation across the hands. Experiment 1 showed the effects of vibrotactile duration and temporal separation between the hands on the quality of perceived illusory linear motion. Experiment 2 indicated a compressed linear relation between the visual and tactile speeds, and established a linear function relating visual size to perceived tactile intensity at three durations. Experiment 3 introduced an “M-filter” algorithm that varies tactile stimulus amplitude by a parabolic function based on visual looming and receding. It demonstrated that the M-filters, accompanied by visual depth cues, can induce tactile motion in depth. Experiment 4 showed the M-filter algorithm is necessary to create tactile perception in depth, as opposed to apparent tactile motion. The current research has value for a basic understanding of haptic perception, as well as haptic applications that digitally generate perceptual representations of the distal world on small-sized devices in the space between the hands.

In this study, we investigated the effect of input voltage waveform on our haptic perception of electrovibration on touch screens. Through psychophysical experiments performed with eight subjects, we first measured the detection thresholds of electrovibration stimuli generated by sinusoidal and square voltages at various fundamental frequencies. We observed that the subjects were more sensitive to stimuli generated by square wave voltage than sinusoidal one for frequencies lower than 60 Hz. Using Matlab simulations, we showed that the sensation difference of waveforms in low fundamental frequencies occurred due to the frequency-dependent electrical properties of human skin and human tactile sensitivity. To validate our simulations, we conducted a second experiment with another group of eight subjects. We first actuated the touch screen at the threshold voltages estimated in the first experiment and then measured the contact force and acceleration acting on the index fingers of the subjects moving on the screen with a constant speed. We analyzed the collected data in the frequency domain using the human vibrotactile sensitivity curve. The results suggested that Pacinian channel was the primary psychophysical channel in the detection of the electrovibration stimuli caused by all the square-wave inputs tested in this study. We also observed that the measured force and acceleration data were affected by finger speed in a complex manner suggesting that it may also affect our haptic perception accordingly.

We investigated the effect of vibration feedback latency on material perception during a tapping interaction using a rod device. When a user taps a surface, the perception of the material can be modulated by providing a decaying sinusoidal vibration at the moment of contact. To achieve this haptic material augmentation on a touchscreen, a system that can measure the approach velocity and provide vibration with low latency is required. To this end, we developed a touchscreen system that is capable of measuring the approach velocity and providing vibration feedback via a rod device with latency of 0.1 ms. Using this system, we experimentally measured the human detection threshold of the vibration feedback latency adopting a psychophysical approach. We further investigated the effect of latency on the perception of the material using a subjective questionnaire. Results show that the threshold was around 5.5 ms and the latency made the user feel that the surface is soft. In addition, users reported bouncing and denting sensations induced by the latency.

The lack of haptic feedback in robotics-assisted surgery can result in tissue damage or accidental tool-tissue hits. This paper focuses on exploring the effect of haptic feedback via direct force reflection and visual presentation of force magnitudes on performance during suturing in robotics-assisted minimally invasive surgery (RAMIS). For this purpose, a haptics-enabled dual-arm master-slave teleoperation system capable of measuring tool-tissue interaction forces in all seven Degrees-of-Freedom (DOFs) was used. Two suturing tasks, tissue puncturing and knot-tightening, were chosen to assess user skills when suturing on phantom tissue. Sixteen subjects participated in the trials and their performance was evaluated from various points of view: force consistency, number of accidental hits with tissue, amount of tissue damage, quality of the suture knot, and the time required to accomplish the task. According to the results, visual force feedback was not very useful during the tissue puncturing task as different users needed different amounts of force depending on the penetration of the needle into the tissue. Direct force feedback, however, was more useful for this task to apply less force and to minimize the amount of damage to the tissue. Statistical results also reveal that both visual and direct force feedback were required for effective knot tightening: direct force feedback could reduce the number of accidental hits with the tissue and also the amount of tissue damage, while visual force feedback could help to securely tighten the suture knots and maintain force consistency among different trials/users. These results provide evidence of the importance of 7-DOF force reflection when performing complex tasks in a RAMIS setting.

We actively move our hands and eyes when exploring the external world and gaining information about object's attributes. Previous studies showing that how we touch might be related to how we felt led us to consider whether we could decode observers' subjective tactile experiences only by analyzing their exploratory movements without explicitly asking how they perceived. However, in those studies, explicit judgment tasks were performed about specific tactile attributes that were prearranged by experimenters. Here, we systematically investigated whether exploratory movements can explain tactile ratings even when participants do not need to judge any tactile attributes. While measuring both hand and eye movements, we asked participants to touch materials freely without judging any specific tactile attributes (free-touch task) or to evaluate one of four tactile attributes (roughness, hardness, slipperiness, and temperature). We found that tactile ratings in the judgment tasks correlated with exploratory movements even in the free-touch task and that eye movements as well as hand movements correlated with tactile ratings. These results might open up the possibility of decoding tactile experiences by exploratory movements.

Thermal displays have been applied in various haptic applications, from material simulation to interpersonal communication; however, there is insufficient knowledge about the temporal processing in human thermal sense to provide a knowledge basis for thermal display design. In this study, we investigated the physical-perceptual correspondence for dynamic thermal stimulation to shed a light on the temporal processing of human thermal sense. In the experiments, participants reported subjective timings of the temperature onset and temperature peak of continuous temperature changes applied to the thenar eminence. We found that the physical-perceptual correspondence was not consistent for warm and cold stimulations. For warm stimulation, the subjective experience always came after the corresponding physical event. On the other hand, for cold stimulation, while the subjective onset always lagged the physical onset, the subjective temperature peak preceded the physical temperature peak. We analyzed these results in the framework of linear systems theory. The results suggest that the senses of warmth and cold have distinct temporal filtering properties, with the sense of cold being more transient than the sense of warmth. These findings advance our knowledge regarding temporal processing in human thermal sense and serve as a basis for thermal display design.

Ultrasonic vibration is employed to modify the friction of a finger pad in way that induces haptic sensations. A combination of intermittent contact and squeeze film levitation has been previously proposed as the most probable mechanism. In this paper, in order to understand the underlying principles that govern friction modulation by intermittent contact, numerical models based on finite element (FE) analysis and also a spring-Coulombic slider are developed. The physical input parameters for the FE model are optimized by measuring the contact phase shift between a finger pad and a vibrating plate. The spring-slider model assists in the interpretation of the FE model and leads to the identification of a dimensionless group that allows the calculated coefficient of friction to be approximately superimposed onto an exponential function of the dimensionless group. Thus, it is possible to rationalize the computed relative reduction in friction being (i) dependent on the vibrational amplitude, frequency, and the intrinsic coefficient of friction of the device, and the reciprocal of the exploration velocity, and (ii) independent of the applied normal force, and the shear and extensional elastic moduli of the finger skin provided that intermittent contact is sufficiently well developed. Experimental validation of the modelling using real and artificial fingertips will be reported in part 2 of this work, which supports the current modelling.

Temperature perception is a highly nonlinear phenomenon with faster rates of change being perceived at much lower thresholds than slower rates. This paper presents a method that takes advantage of this nonlinear characteristic to generate a perception of continuous cooling even though the average temperature is not changing. The method uses multiple thermal actuators so that a few are cooling quickly while the rest of the actuators are heating slowly. The slowly-heating actuators are below the perceptual threshold temperature change and hence are not perceived, while the quickly-cooling actuators are above the perceptual temperature change, hence are perceived. As a result, a feeling of decreasing temperature was elicited, when in fact, there was no net change in the temperature of the skin. Three sets of judiciously designed experiments were conducted in this study, investigating the effects of actuator sizes, forearm measurement locations, patterns of actuator layout, and various heating/cooling time cycles. Our results showed that 19 out 21 participants perceived the continuous cooling effect as hypothesized. Our research indicates that the measurement location, heating/cooling cycle times, and arrangement of the actuators affect the perception of continuous cooling.

Interacting with physical objects through a tool elicits tactile and kinesthetic sensations that comprise your haptic impression of the object. These cues, however, are largely missing from interactions with virtual objects, yielding an unrealistic user experience. This article evaluates the realism of virtual surfaces rendered using haptic models constructed from data recorded during interactions with real surfaces. The models include three components: surface friction, tapping transients, and texture vibrations. We render the virtual surfaces on a SensAble Phantom Omni haptic interface augmented with a Tactile Labs Haptuator for vibration output. We conducted a human-subject study to assess the realism of these virtual surfaces and the importance of the three model components. Following a perceptual discrepancy paradigm, subjects compared each of 15 real surfaces to a full rendering of the same surface plus versions missing each model component. The realism improvement achieved by including friction, tapping, or texture in the rendering was found to directly relate to the intensity of the surface's property in that domain (slipperiness, hardness, or roughness). A subsequent analysis of forces and vibrations measured during interactions with virtual surfaces indicated that the Omni's inherent mechanical properties corrupted the user's haptic experience, decreasing realism of the virtual surface.

The multifaceted human sense of touch is fundamental to direct manipulation, but technical challenges prevent most teleoperation systems from providing even a single modality of haptic feedback, such as force feedback. This paper postulates that ungrounded grip-force, fingertip-contact-and-pressure, and high-frequency acceleration haptic feedback will improve human performance of a teleoperated pick-and-place task. Thirty subjects used a teleoperation system consisting of a haptic device worn on the subject's right hand, a remote PR2 humanoid robot, and a Vicon motion capture system to move an object to a target location. Each subject completed the pick-and-place task 10 times under each of the eight haptic conditions obtained by turning on and off grip-force feedback, contact feedback, and acceleration feedback. To understand how object stiffness affects the utility of the feedback, half of the subjects completed the task with a flexible plastic cup, and the others used a rigid plastic block. The results indicate that the addition of grip-force feedback with gain switching enables subjects to hold both the flexible and rigid objects more stably, and it also allowed subjects who manipulated the rigid block to hold the object more delicately and to better control the motion of the remote robot's hand. Contact feedback improved the ability of subjects who manipulated the flexible cup to move the robot's arm in space, but it deteriorated this ability for subjects who manipulated the rigid block. Contact feedback also caused subjects to hold the flexible cup less stably, but the rigid block more securely. Finally, adding acceleration feedback slightly improved the subject's performance when setting the object down, as originally hypothesized; interestingly, it also allowed subjects to feel vibrations produced by the robot's motion, causing them to be more careful when completing the task. This study supports the utility of grip-force and high-frequency acceleration feedback in teleoperation systems and motivates further improvements to fingertip-contact-and-pressure feedback.

Shape-changing interfaces are a category of device capable of altering their form in order to facilitate communication of information. In this work, we present a shape-changing device that has been designed for navigation assistance. `The Animotus' (previously, `The Haptic Sandwich' ), resembles a cube with an articulated upper half that is able to rotate and extend (translate) relative to the bottom half, which is fixed in the user's grasp. This rotation and extension, generally felt via the user's fingers, is used to represent heading and proximity to navigational targets. The device is intended to provide an alternative to screen or audio based interfaces for visually impaired, hearing impaired, deafblind, and sighted pedestrians. The motivation and design of the haptic device is presented, followed by the results of a navigation experiment that aimed to determine the role of each device DOF, in terms of facilitating guidance. An additional device, `The Haptic Taco', which modulated its volume in response to target proximity (negating directional feedback), was also compared. Results indicate that while the heading (rotational) DOF benefited motion efficiency, the proximity (translational) DOF benefited velocity. Combination of the two DOF improved overall performance. The volumetric Taco performed comparably to the Animotus' extension DOF.

This paper presents a haptic assistance strategy for teleoperation that makes a task and situation-specific compromise between improving tracking performance or human-machine interaction in partially structured environments via the scheduling of the parameters of an admittance controller. The proposed assistance strategy builds on decision-making models and combines one of them with impedance control techniques that are standard in bilateral teleoperation systems. Even though several decision-making models have been proposed in cognitive science, their application to assisted teleoperation and assisted robotics has hardly been explored yet. Experimental data supports the Drift-Diffusion model as a suitable scheduling strategy for haptic shared control, in which the assistance mechanism can be adapted via the parameters of reward functions. Guidelines to tune the decision making model are presented. The influence of the reward structure on the realized haptic assistances is evaluated in a user study and results are compared to the no assistance and human assistance case.

It is known that humans experience a kinesthetic illusion, like a pulling sensation in a particular direction, when asymmetric vibrations are presented. However, the perceptual mechanism of this illusion has not been well clarified. In this study, we have developed a device that can control the vibrations for quantitative evaluation of the pulling sensation induced by exerting asymmetric vibrations to user’s fingertip. As a result of an evaluation, it was confirmed that this device can generate acceleration of target asymmetric vibration even if frequency and participants are changed.

The display of compliance, or ‘softness,’ to the finger pad is important for replicating interactions such as holding another’s hand and identifying tissue irregularities. To contribute to work in this area, we designed a multi-chamber pneumatic actuator, fabricated of silicone elastomer with concentric rings at its surface coupled to a drum at its base. Using air pressure, different combinations of chambers can be inflated to variable degrees in real-time. In this way, attributes including the curvature, displacement, and spatial features of its surface as well as bulk compliance can be separately configured. Such capability is needed to decipher the cutaneous cues that drive perception. Precise cues remain unresolved, but likely tie to contact area as a function of force or displacement, over time. In a series of imaging and force measurement experiments, the actuator was evaluated in producing concave and convex surfaces, displacing upward, and inflating its drum. Spatial features from single hard spots to open holes were also produced. The next steps, leading to psychophysical experiments, are to further miniaturize and increase the number of surface rings, and integrate the actuator with a wearable fixture on the finger.

In this study, we tested a novel system for manipulating the sensation of hardness and softness using a touch panel in augmented reality (AR) space. Many studies of touch panel-based haptic displays have developed methods for manipulating macro-roughness (shape), fine-roughness (texture), and friction. However, few studies have examined the sensation of softness. Here we used a film-winding mechanism to present tangential force opposite to the tangential displacement of the fingers, which functioned as a sufficient cue for the sensation of softness. We combined this with a visual cue for softness, adding visual shadow to the display of the virtual image generated by a half-mirror, which indicated visual deformation of the object.

Human children typically experience their surroundings both visually and haptically, providing ample opportunities to learn rich cross-sensory associations. To thrive in human environments and interact with the real world, robots also need to build models of these cross-sensory associations; current advances in machine learning should make it possible to infer models from large amounts of data. We previously built a visuo-haptic sensing device, the Proton Pack, and are using it to collect a large database of matched multimodal data from tool-surface interactions. As a benchmark to compare with machine learning performance, we conducted a human subject study (n = 84) on estimating haptic surface properties (here: hardness, roughness, friction, and warmness) from images. Using a 100-surface subset of our database, we showed images to study participants and collected 5635 ratings of the four haptic properties, which we compared with ratings made by the Proton Pack operator and with physical data recorded using motion, force, and vibration sensors. Preliminary results indicate weak correlation between participant and operator ratings, but potential for matching up certain human ratings (particularly hardness and roughness) with features from the literature.

The core hypothesis of this ongoing research project is that co-designing haptic-feedback and forward-control methods for shared-control teleoperation will enable the operator to more readily understand the shared-control algorithm, better enabling him or her to work collaboratively with the shared-control technology.} This paper presents a novel method that can be used to co-design forward control and force feedback in unmanned aerial vehicle (UAV) teleoperation. In our method, a potential field is developed to quickly calculate the UAV's risk of collision online. We also create a simple proxy to represent the operator's confidence, using the swiftness with which the operator sends commands the to UAV. We use these two factors to generate both a scale factor for a position-control scheme and the magnitude of the force feedback to the operator. Currently, this methodology is being implemented and refined in a 2D-simulated environment. In the future, we will evaluate our methods with user study experiments using a real UAV in a 3D environment.

The security of telerobotic systems used to perform critical tasks is of paramount importance because compromising these systems can result in significant harm. In this paper, we attempt to address threats leading to illegitimate access to telerobotic devices. We conducted an experiment in which 32 users explored a virtual scene using a haptic device, while receiving almost no visual feedback. Our results show that haptic signals collected during a session can be successfully used to distinguish between users. As a result, telerobotic operators can be authenticated transparently throughout a session by relying on haptic measurements.

A continuous stroking sensation on the skin can convey messages or emotion cues to individuals. We seek to induce this sensation using a combination of illusory motion and lateral stroking in a single device. We present a device that provides discrete lateral skin-slip on the forearm by actuating five separate tactors, which independently provide lateral skin-slip in a timed sequence to induce a stroking sensation. We validated the device design and optimized the actuation parameters for pleasantness and continuity through a user study with 16 participants.

This paper describes the design of a novel, high-performance one degree-of-freedom hybrid haptic interface. The interface integrates a particle brake, a Series Elastic Actuator (SEA), and a small DC motor into a single actuator. Unlike existing hybrid designs, the one presented here does not suffer from a significant mismatch between passive and active torque capability. The paper describes the design of the three actuator components. The motivation for selecting a particle brake is discussed. The SEA design approach is contextualized through its integration into the hybrid actuator, and the ideal SEA spring stiffness for the actuator is explored. Finally, mini motor design and sizing considering SEA and particle brake performance is included. The interface design provides insight into the dynamics of the system as a whole and allows for additional future work in control and optimization.

Grounding of kinesthetic feedback against a user's hand can potentially increase the portability and wearability of a haptic device. However, the effects of different hand-grounding locations on the haptic experience of a user are hitherto unexplored. We present a haptic device design that provides kinesthetic feedback grounded against three different locations on a user hand --- the back of the hand, and the proximal and middle phalanges of the index finger. The proposed 2-degree-of-freedom haptic device is modular, light-weight, and wearable. In different grounding modes, the device can provide kinesthetic feedback along the axis of the index finger, and in the direction of its flexion and extension movements. The device will be used in a virtual environment based study to compare user performance in different hand-grounding modes.

The effect of electrical stimulation applied to the lower limb tendons on the sensation of virtual walking was investigated. The virtual walking, here, is a passive experience in which the user sees and acts virtually as some other person did to learn through his/her experience. A display device was developed to provide electrical stimulation to sensory nerves in the Achilles tendon and the tibialis anterior tendon. A kinesthetic stimulus and visual stimulus were simultaneously provided to a seated participant. We measured the sensations of walking and translational motion regarding three factors of electrical, kinesthetic and visual stimuli. The result obtained from eleven participants revealed that the three factors were significant in enhancing the sensation of virtual walking. The electrical stimulation for proprioception seemed effective to compensate the characteristics of passive kinesthetic playback of experiences.

In this work, vibrotactile signal felt by a person in a tennis video is automatically estimated from the visual and audio information by using neural network. The system is based on a similar concept to VibVid proposed by the authors. We believe that VibVid system can greatly reduce the effort to describe mathematical models to generate tactile information from diverse videos. In this paper, we try more general and difficult task than the previous research in order to examine the system robustness. We limit the video scene to the back view of a tennis player rallying, but other factors such as locations, player’s clothes, sound environments are arbitrary. We use tennis videos taken in three locations for neural network learning of the relation between the video and measured acceleration of the racket grip. Then we show the grip sensation can be successfully estimated from an unknown video which is taken in a different location from learning. In this case, we plan to examine how to recognize shots more accurately and reproduce high quality tactile sensations.

Brain-computer interfaces (BCIs) can measure movement-related neural activity and decode it into command signals that drive external devices. These command signals are usually biased with neural decoder noise. Haptic feedback is an emerging feedback modality for BCIs, and it is unknown how users execute tasks given haptic feedback driven by noisy command signals. We propose an experiment that measures task performance while a user is provided with non-collocated haptic feedback with varying magnitudes of noise. Our preliminary results point to performance degradation with increasing magnitude of noise. The results also suggest that users change their search strategy depending on the amount of noise. A formal user study is planned for future work.

This work focusses on single cell laterotactile reading for users who do not know Braille. In this method of presentation of words with a single actuator, letters that constitute the word come right under the primary reading finger of the user. We look at the accuracy and the speed of reading for blindfolded sighted users who read tactually by virtue of the memory they have of the shapes of letters. In testing with ten blind-folded sighted users, good results were obtained with an average accuracy of 90%.The average speed of reading was 6-7 WPM. These results will find their application in the design of laterotactile readers for late–blind users who do not know Braille.

Hand exoskeletons could potentially improve hand use after a stroke or other conditions that weaken the hand, but are typically obtrusive, and it is still unclear how to provide intuitive control. Here, we propose and pilot test with unimpaired subjects a grip force control strategy suitable for a minimalistic hand exoskeleton. The strategy is based on three experimental observations seen in studies conducted in our lab with unimpaired individuals and stroke survivors. First, using only a pinch grip, unimpaired people can achieve a substantial level of clinical hand function. Second, people with severe hand impairment after stroke have a surprisingly well-preserved ability to control isometric finger flexion force. Third, force generation is highly correlated between fingers after a stroke. These observations suggest the strategy of measuring the isometric flexion force produced by digits 3-5 (middle-pinky finger) against the palm to control the force of an exoskeleton assisting in pinch grip, while preserving haptic input. We implemented this “residual force control” strategy using the FINGER exoskeleton and found that unimpaired subjects could intuitively use this strategy to pick up an object and learn to amplify their grip force with this strategy. We are preparing to test this “grip amplifier” with people with stroke.

Reconstruction of the skilled human sensations and design of a related control system is important for robust control of the robots. We are developing an unscrewing robot with a comprehensive control system for the automated disassembly of electronic devices. Experiments involve screwing and unscrewing, and since humans typically have a broad range of screwing experiences and sensations throughout their lives, we conducted a series of experiments to find out these haptic patterns. Results show that people apply axial force to the screws to avoid screwdriver slippage (cam-outs), which is one of the key problems during screwing and unscrewing, and this axial force is proportional to the torque which is required for screwing, that depends on the type of the screw head. This knowledge will make the unscrewing control systems more robust and enable development of realistic simulators.

We present a method for interactive virtual fixtures definition for bilateral teleoperation. The proposed approach is based on the extraction of geometrical features (shapes and surfaces) from depth video cameras which are then displayed interactively to a human-operator through a graphical interface. The human-operator can select these automatically detected graphical features and associate virtual fixtures to them which are directly taken into account by the bilateral controller. The developed system enables on-fly definition of virtual fixtures of different types.

In minimally invasive surgery, indirect contact leads to impaired haptic feedback for the surgeon. In this study, considering the viscoelasticity of human tissues and organs, we assessed the effect of damping and spring constant on hardness perception. Dampers which exhibit both elastic property and viscous property were designed and manufactured using 3D printing technology. In the experiment, the ability to distinguish hardness was evaluated using a magnitude estimation method. Two different ways of holding the tool were used in a probing task. Force data were also analyzed to determine the role of force feedback relative to elasticity and viscosity. Preliminary results suggest that the major factor in hardness perception in this context is force application, followed by spring constant and damping coefficient. Implications for tool design will be considered in light of these results.

Much of three decades of haptic device invention is effectively lost to today's designers: dispersion across time, region, and discipline imposes an incalculable drag on innovation in this field. Our goal is to make historical haptic invention accessible through interactive navigation of a comprehensive library -- a Haptipedia -- of devices that have been annotated with designer-relevant metadata. To build this open resource, we will systematically mine the literature and engage the haptics community for expert annotation. In a multi-year broad-based initiative, we will empirically derive salient attributes of haptic devices, design an interactive visualization tool where device creators and repurposers can efficiently explore and search Haptipedia, and establish methods and tools to manually and algorithmically collect data from the haptics literature and our community of experts. This paper outlines progress in compiling an initial corpus of grounded force-feedback devices and their attributes, and it presents a concept sketch of the interface we envision.

Laparoscopic surgery is a modern surgical technique in which surgeons insert surgical tools through small incisions they make in the abdomen to perform a surgical procedure. Surgeons typically have only a limited 2D visual feedback based on a small camera that is inserted into the abdomen, and minimal haptic feedback that is transmitted through the rigid surgical tools. Our previous study showed that the phenomenon, Stochastic Resonance (SR), observed by the introduction of white noise at a subthreshold level, enhances surgeons' performance (Accuracy and Speed) in laparoscopic surgeries by increasing their sensitivity, through a significant increase in Signal-to-Noise Ratio (SNR), to haptic feedback. Our current research studies the after-effect of SR application for a better and more effective implementation of this technology in the design of medical devices, surgical instruments and laparoscopic tools.

We propose a novel palm-worn tactile display LinkGlide for presentation of the physical properties of the movable object in the hand, such as, weight, slippage, texture. LinkGlide consists of three 2-DoF LinkTouch devices distributed in parallel planes to produce the sliding force on the skin and the contact state at the palm. Two servo motors of each LinkTouch device control the planar position of the single contact point between palm and linkage, therefore, three dynamic contact points can be created. When we need to generate the sensation of object sliding downwards, all three contact points moves on the skin surface from upper side of the palm in the same direction. The force of at contact point is controlled using the pressure sensor and impedance control, thus, the desired pressure while sliding motion can be generated to represent object of different weight, and to simulate the object softness. The frequency at the end-effector is generated by the high-frequency oscillations of two motors, so the sensation of friction against the textured clothes can be simulated. In the paper, we describe the mechanism structure, kinematics, force analysis and design principle.

In this study, we propose analytic models for finite element method (FEM) analysis to estimate mechanoreceptor activity. In order to evaluate contact to fine surfaces, a crosssectional finger model with a high mesh resolution at the stratum corneum was applied to the analysis. The obtained von Mises stress at each mechanoreceptor position was converted into mechanoreceptor activity using mathematical firing models of the mechanoreceptors. First, we determined the coefficients in the mathematical firing models using the results for neural thresholds. Then, we verified responses of the modeled mechanoreceptors to lump stimulation. The obtained results indicate that the proposed models can simulate the activity of mechanoreceptors.

Our group recently developed a novel fMRI-compatible wrist robot, the MR-SoftWrist, to study the neural correlates of force-field adaptation. Here we present our first fMRI pilot study on four healthy subjects. Our results validate the MR-SoftWrist as a tool for investigating motor adaptation during wrist pointing movements executed during fMRI, and compare neural activations associated with motor control in three different dynamic conditions.

This paper describes the design and development of an electromagnetic system incorporated into a graphical display to provide haptic feedback forces on the tip of a stylus held near the display, according to the stylus position and virtual fixtures implemented in software. An array of magnetometer sensors is used to detect the position of the stylus tip, while a pair of box-shaped coils behind the display produce forces on the stylus in the plane of the display.

Haptic human interfaces are nowadays becoming more and more popular also thanks to their combined use with Virtual Reality & Augmented Reality (VR & AR). Although many research platforms explore hand interaction with virtual objects, the only feedback available in widespread commercial devices is the simple vibration. In this paper we introduce “FOLDAWAY-Touch” a novel portable kinesthetic haptic interface whose technology can be easily integrated in commercial devices. The key technological aspects are the origami inspired miniaturized parallel platform, that is in contact with the human finger, and the compact actuation and transmission that makes the device very portable. Another achievement of this paper is the use of FOLDAWAY-Touch for conducting experiments with users in order to investigate their capability to discriminate different level of stiffness.

Safety is often compromised when control is transitioned between a human and automation system. To explore what features of an automation system interface are most critical to safety, we explored four schemes for transitioning control in a driving simulator that featured a motorized steering wheel. The four schemes differ according to whether transitions are discrete or continuous and whether they include haptic communication between driver and automation system across the steering wheel, in the axis of control. The driving task required regular shifts of authority to avoid obstacles and conditionally included a secondary task. Preliminary results indicate that continuous transitions carry an advantage for secondary task performance and haptic communication aides in avoiding obstacles.

Emotions can be affected by a wide variety of environmental parameters. The effect of heat and cold has been studied just recently. It has been shown that heat can affect feelings by applying a thermal stimuli on the palm and acquiring self-reported responses. In this study, we evaluate the effect of thermal stimulation on emotions by applying thermal stimulation on two locations of the body and collecting data through self-report and physiological responses. The goal of this study is to understand if applying temperature to an individual’s skin can predispose them to a different emotion.

The paper focuses on the touch and weight sensitive robotic manipulator DroneArm for UAV, which allows grasping the objects from the warehouse shelves, and delivering them to the destination, even directly to the human hands. The robot can determine the identifier (bar code) and position of the object, estimate their weight and react on collision with human and obstacle. The experiments showed the stable behavior of the flying robot during the manipulator movement, high accuracy of the weight estimation, and robust collision detection.

In this paper, we present an experimental testbed and methodology for comparing the utility of kinesthetic and vibrotactile feedback of grip force and grip aperture in a myoelectric upper-limb prosthesis. The testbed features a cable-driven voluntary-closing 1-DoF prosthesis, a cable-driven elbow exoskeleton for rendering joint-torque feedback, and a vibrotactile actuation unit. The system is capable of measuring the stiffness of objects being grasped by the prosthesis terminal device and present this information to users as an extension moment about the elbow. The system is also capable of rendering grip force and grip aperture as vibration. To equally compare kinesthetic and vibrotactile feedback, a cross-modal matching study was conducted which indicated that the normalized exoskeleton force intensity should be higher than the normalized vibration intensity for the two modalities to be perceived the same. The results of this preliminary analysis will guide future investigations into the utility of kinesthetic and vibrotactile feedback, as well as various other haptic feedback modalities, in object recognition and discrimination tasks with upper-limb myoelectric prostheses.

Little is known about the shape and properties of the human finger during haptic interaction even though this knowledge is essential to control wearable finger devices and deliver realistic tactile feedback. This study explores a framework for four-dimensional scanning and modeling of finger-surface interactions, aiming to capture the motion and deformations of the entire finger with high resolution. The results show that when the fingertip is actively pressing a rigid surface, it undergoes lateral expansion of about 0.2 cm and proximal/distal bending of about 30°, deformations that cannot be captured by imaging of the contact area alone. This project constitutes a first step towards an accurate statistical model of the finger's behavior during haptic interaction.

Creating haptic interfaces capable of rendering the rich sensation needed for dexterous manipulation is crucial for the advancement of human-in-the-loop telerobotic systems (HiLTS). One limiting factor has been the absence of detailed knowledge of the effect of operator limb dynamics and haptic exploration dynamics on haptic perception. We propose to begin investigations of these effects with single-joint haptic exploration and feedback of physical and virtual environments. Here, we present our experimental apparatus, a 1-DoF rotational kinesthetic haptic device and electromyography (EMG) system, along with preliminary findings from our efforts to investigate the change in stiffness discrimination thresholds for differing exploration velocities. Result trends indicate a possible relationship between exploration velocity and discrimination thresholds, as well as a complex interaction between muscle activation, exploration velocity, and haptic feedback.

Magnetic resonance imaging (MRI) provides safe, high contrast images and is often the preferred diagnostic tool over ultrasound and computer tomography (CT). Despite its advantages, physicians do not perform surgical interventions, such as liver or breast biopsies, with real-time MRI guidance because they are unable to reach patients inside the MRI bore. To address this challenge, we present an MRI compatible teleoperator that enables needle manipulation in three degrees of freedom (3-DOF). The device provides haptic feedback through a low friction, high stiffness hydrostatic transmissions and is a step towards haptic transparency in real-time MRI guided interventions.

Sixteen subjects were trained with a user-driven interface to recognize sixteen tacton-phoneme pairs. To improve distinctiveness, tactons were created from vibrotactile stimuli that varied based on four forearm locations: dorsal/ventral, wrist/elbow; and four vibration patterns: continuous, short, fast, and frequency-modulated. To improve memorability, vibration patterns were assigned to similar-sounding phonemes. Subjects were asked to select the phoneme associated with each played tacton following a 30-minute training session consisting of interleaved learning and practice tests, during which they were free to choose the number of tactons and the difficulty of each practice test. Subjects who performed best on the final test chose to receive (i) more repetition during the practice tests and (ii) more difficult practice tests that resembled the final test. These two strategies gave successful learners greater opportunity for strengthening the association between tactons and phonemes.

The present study investigated the perceptual dimensions associated with manual key clicks, with the goal of developing realistic haptic key-click feedback signals for virtual keys. We first harvested eight adjective pairs for describing the haptic feel of button and key presses from native English speakers. We then conducted the main experiment where participants provided adjective ratings and grouping data for twenty-three buttons and keys. An MDS analysis of the grouping data led to either a 2-D or 3-D solution. By projecting adjective ratings onto the MDS solution spaces, we found the 2-D perceptual space to be an adequate representation of human perception of manual key clicks. The two perceptual dimensions are determined to be shallow-deep and rough-smooth. Future work will explore the physical parameters corresponding to the perceptual dimensions and ways to simulate realistic key clicks by designing feedback signals using the relevant parameters.

Pianists usually pay little attention to piano key vibrations but it was shown that they influence the perceived quality of an instrument and it was suggested that they might play a role for the precise timing and dynamic control in piano playing. To objectively measure and understand the influence of vibrotactile feedback in the pianist-piano interaction, we plan experiments with a digital hybrid grand piano - the Yamaha AvantGrand N3X - that simulates piano key vibrations with a rendering system. In this paper, we evaluate piano key vibrations of this instrument with a laser Doppler vibrometer and compare the vibrations to measurements with an acoustic grand piano. The peak levels of both instruments (13 to 35 um for the acoustic grand piano and 16 to 25 um for the AvantGrand) are comparable but the rendering system has limitations outside the frequency range from 150 to 400 Hz. Furthermore, the perceptibility of the vibrations at the left and right hand playing positions is investigated. Finally, the impact of the finger on the vibrations during different stages of a key press is analyzed and it is demonstrated that the finger influences the displacement levels and the spectral weighting of the piano key vibrations.

<p>Proprioception—the sense of one's body position and movement, without the aid of vision—plays a critical role in human motor control, allowing us to adeptly move our bodies through a high-dimensional task space. The relationship between joint space and task space with regard to proprioception has not been studied in the general population. This work begins to explore the relationship between proprioceptive acuity—the combination of accuracy and precision—in joint space and task space, focusing on the elbow, shoulder, and hand of the arm in single-joint (joint-space) and integrated multi-joint (task-space) active position-matching tests with a planar, robotic arm support. Our results reveal a strong correlation between joint-space proprioception at the shoulder and elbow and task-space proprioception at the hand. However, when joint-space proprioceptive error is propagated through a model of the arm's planar kinematics, it agrees poorly with the proprioceptive error measured explicitly in task space. Task-space proprioception exhibits greater accuracy than joint-space proprioception, as would be expected given the greater biological relevance of a planar reach compared to an isolated joint movement. Task-space and joint-space proprioception also differ in directional precision, exhibiting the greatest variance along nearly orthogonal axes, approximately aligned with the sagittal and frontal body planes. These findings have implications for the diagnosis of sensorimotor impairment and the development of movement therapies following neurological injury.</p>

The proprioceptive sense is an important component in the sensory system, providing position estimates for the parts of our body that are needed in order to guide behavior. While prior work has characterized proprioceptive localization of the upper limb as a whole, few studies have investigated the proprioceptive localization of individual digits, despite their central role in tactile exploration and fine manipulation. In this study, we asked subjects to report the perceived positions of the endpoints of each finger by means of a virtual reality and multi-finger tracking system, and we investigated the magnitude and variability of the proprioceptive localization errors that ensued. We found the errors to be large, averaging 4.3 cm per digit, even when the location of the wrist was disclosed to participants. This error magnitude is comparable to those that have been reported for whole limb localization, and represents a significant fraction of the length of an entire digit. Localization accuracy is greatest for the thumb and smallest for the little finger, reflecting in all cases, significant biases in perceived locations. The precision of localization was similar for each finger, with uncertainties on the order of several centimeters. The posture of the hand and specific location of the finger both affected errors. The specific error patterns for each subject were idiosyncratic, and the results suggest an underlying proprioceptive hand representation in which the fingers are distorted from the true shape of the hand. These results may have implications for understanding the role of position sensing in tactile exploration and fine manipulation, and suggest that an improved understanding of sensorimotor integration of proprioceptive information from the fingers is needed.

Our sensorimotor system estimates stiffness to form stiffness perception, such as for choosing a ripe fruit, and to generate actions, such as to adjust grip force to avoid slippage of a scalpel during surgery. We examined how temporal manipulation of the haptic and visual feedback affect stiffness perception and grip force adjustment during a stiffness discrimination task. We used delayed force feedback and delayed visual feedback to break the natural relations between these modalities when participants tried to choose the harder spring between pairs of springs. We found that visual delay caused participants to slightly overestimate stiffness while force feedback delay caused a mixed effect on perception; for some it caused underestimation and for some overestimation of stiffness. Interestingly and in contrast to previous findings without vision, we found that participants increased the magnitude of their applied grip force for all conditions. We propose a model that suggests that this increase was a result of coupling the grip force adjustment to their proprioceptive hand position, which was the only modality which we could not delay. Our findings shed light on how the sensorimotor system combines information from different sensory modalities for perception and action. These results are important for the design of improved teleoperation systems that suffer from unavoidable delays.

Thermal perception is important in the experience of touching real objects, and thermal display devices are of growing interest for applications in virtual reality, medicine, and wearable technologies. In this paper, we designed a new thermal display, and investigated the perception of spatially varying thermal stimuli, including the thermal grill illusion. The latter is a perceptual effect in which a burning sensation is elicited in response to touching a surface composed of spatially juxtaposed warm and cool areas. Using a computer controlled thermal display, we present experiments in which we measured temporal correlates of the perception of spatially inhomogeneous stimuli, or thermal grills. We assessed the intensity of responses elicited by thermal grill stimuli with different temperature settings, and measured the response time until the onset of burning sensations. We found that thermal grills elicited highly stereotyped responses. The experimental results also indicated that as the temperature difference increases, the intensity increases monotonically, while the response time decreases monotonically. Consequently, perceived intensity was inversely correlated with response time. Under current physiological explanations, responses to thermal stimuli depend on tissue heating, neural processing, and the spatial distribution (or juxtaposition) of surface temperatures. The results of this study could help to inform models accounting for these factors, enabling new applications of the thermal grill illusion.

We report on the characterization of two variable friction electroadhesive displays using careful electrical and electrochemical impedance measurements. We qualitatively and quantitatively examine the properties of the skin, body, surface coating, and various electrode interface impedances in isolation using different contact interface conditions and measurement types. A lumped series impedance model explains how all impedances are related during normal usage, and the linearity of this model is shown to be valid under certain assumptions, such as high applied frequencies or small applied currents. Speculation as to the physical mechanisms underlying each impedance element is also given. This analysis unambiguously verifies the existence of a previously hypothesized key electrical system parameter: the sliding interfacial impedance (or air gap impedance). This parameter represents the large increase (100-1000 percent) in overall electrical impedance observed when a finger is sliding versus when it is stationary. It is concluded that this impedance increase cannot be explained by other measured electrical impedance elements in the system and that it vanishes again when the finger comes to rest.

Touchscreen interfaces have become increasingly popular worldwide. However, few commercial touchscreens enable reactive tactile signals. We use lateral-force-based tactile feedback devices that employ electrostatic force. Our goal is to realize more realistic tactile sensation to the user. Previous researches evaluated by using multiple axes determined by the experimenter, however, intuitive classification of these sensation with onomatopoeic words has not been held. In this research, we focused on evaluating how the user feels about the electrostatic stimulation. Furthermore, we proposed a modeling of magnitude of tactile sensation, and discussed what kind of tactile sensation can be displayed with our system.

Electrovibration is a promising method used to generate tactile feedback on touch screens. Although the method has recently received considerable attention from tactile display developers, almost all efforts so far have focused on single touch applications. However, extending this technology to apply to multi-touch systems is required to benefit from the rapidly growing touch screen industry. Through the course of this study, we propose a method and present a tactile display prototype that creates multi-touch tactile feedback using electrostatic attraction. The method relies on applying high-voltage AC signals on certain orthogonal electrode lines resulting in perceivable changes of friction at the intersection points. Generated surface friction on the prototype is measured using a planar tribometer. Results reveal that multiple localized friction spots can be generated with the proposed method. A user study is also carried out to test the prototype in a multi-touch scenario where two fingers explore a virtual texture simultaneously. Additionally, quantitative and qualitative analyses demonstrate the feasibility of creating multi-touch haptic feedback on an electrostatic tactile display.

Due, in part, to the popularity of online shopping, there is considerable interest in enabling consumers to experience material touch via internet connected devices. While there have been several efforts to render texture via electrostatic tactile displays, the textures involved have typically consisted of synthetic patterns, such as shapes, shadings, or gradients of photographic textures. In this paper, we propose a data-driven algorithm for the haptic rendering of fabric textures on an electrostatic tactile display. We measure the friction force, normal force, and displacement during the swiping of a finger across real fabric using a new measurement apparatus introduced here. Using these measurements, we compute friction coefficients derived from the recorded frictional and normal forces. We then reproduce the friction coefficients by controlling the voltage applied to an electrostatic tactile display in order to render the tactile texture of the measured fabric. In order to evaluate this rendering method, we conducted a psychophysical experiment that assessed the visual and haptic similarity of ten real and simulated fabrics. The experimental results show that the virtual textures generated using this electrostatic rendering algorithm were perceptually similar to the corresponding real textures for all fabrics tested, underlining the promise of electrostatic tactile displays for material simulation.

Although diverse foot-based applications have been explored, foot-based menu selection is underexplored given its potential for low-fatigue secondary control input. Here, we are investigating whether the effect of adding haptic modalities can achieve higher performance in a menu selection task. We study the effect of auditory or vibrotactile feedback on selection performance in radial menus consisting of three, six and nine items. We compared no feedback to one auditory and two vibrotactile clicks, one across the foot, one localized to the movement direction. All feedback modalities allowed for rapid completion of menu selections and, while audio was generally preferred and our results suggest a superiority over haptics, the latter are still helpful in increasing selection accuracy. However, we argue that the difference is such that haptics could still be used with comparable performance in noisy environments or by users with auditory disabilities. Finally, we use an analysis of the number of attempts required to select the correct position, coupled with the number of errors, to make design recommendations for foot-based menus.

While many wearable tactile displays for the fingers, such as fingertip-type and glove-type displays, have been developed, their weight and size typically hinder the free movement of fingers, especially when considering the multi-finger scenario. We propose a method of presenting the haptics sensation of the fingertip on the forearm, not on the fingertip, to address this issue. A five-bar linkage mechanism was adopted to present a two-degree-of-freedom force. We conducted two experiments. In the first experiment, we presented a pressure sensation and a horizontal friction sensation perceived by the index finger to multiple sites of the forearm to search for a proper location of presentation, finding that the volar part of the wrist is optimal. On the basis of this result, we developed a device for the index finger and thumb, and conducted a second experiment to present the grasping force in a virtual reality environment. The realism of the experience in virtual reality was better when using the designed device than for no haptics cue or for vibration conditions.

Existing force feedback gloves mainly adopt rigid actuators such as electric motors and pneumatic cylinders, which have limitations including safety issues, heavyweight, and complex transmission mechanisms. In this paper, we introduce a light-weighted force-feedback glove using pneumatic-driven soft actuators. Based on the unilateral deformable features of the strain-limiting layer in the soft actuator, the dorsal-side mounting solution along with a light-weighted linkage mechanism is proposed to produce fingertip force feedback. We applied a pre-deformation of the soft actuator to enable the back drivability and the free space sensation. We then implemented a physical prototype of a two-fingered glove. Experimental results show that the glove could achieve considerable performance in free space with small frictional force (0.58N in maximum). While simulating the constrained space, the fingertip force reaches up to 2.1N. For the future work, we plan to improve the current solution to five fingers with finger position tracking and distributed tactile sensing on the palm.

Skin stretch haptic devices are well-suited for transmitting information through touch, a promising avenue in prosthetic research, addressing the lack of feedback in myoelectric designs. Rocker-based skin stretch devices have been proposed for sensory substitution and navigational feedback, but the designs vary in their geometry. Other works create torsional stretch, and utilize nonlinear mappings to enhance perception. This work investigates parameters of rocker geometry and mapping functions, and how they impact user perception. We hypothesize that perceptual changes are dependent on the choice of stretch increment sizes over the range of motion. The rocker geometry is varied with an offset between the rotational and geometric axes, and three rocker designs are evaluated during a targeting task implemented with a nonlinear or linear mapping. The rockers with no offset and a positive offset (wide) perform better than the negative offset (narrow) case, though the mapping method does not affect target accuracy.

We propose a new lateral force feedback device, the UltraShiver, which employs a combination of in-plane ultrasonic oscillation (around 30 kHz) and out-of-plane electroadhesion. It can achieve a strong active lateral force (400 mN) on the bare fingertip while operating silently. The lateral force is a function of pressing force, lateral vibration velocity, and electroadhesive voltage, as well as the relative phase between the velocity and voltage. In this paper, we perform experiments to investigate characteristics of the UltraShiver and their influence on lateral force.

Alongside questions of how to create haptic effects on displays via alternative hardware, recent work has explored rendering options with respect to haptic effects, i.e. when and where to provide haptic feedback. In particular, recent work by Zhang and Harrison for electrostatic haptic feedback noted that the optimal technique for haptic feedback during interaction is the Fill technique, where haptic effects are rendered at all times when a user's finger is within the bounds of the target. In this paper, we explore whether this result generalizes to an alternative haptic rendering technology that uses ultrasonic vibrations to create haptic sensations, a technique called the ``Squeeze Film Effect''. In contrast to prior work, our results indicate that positioning the haptic feedback as a discrete linear stimulus centred on the target provides an optimal trade-off between speed, accuracy, and user preference. We highlight the implications of this work to the generalizability of haptic feedback: Haptic feedback can improve time, errors, and user satisfaction during interaction, but only if the correct form of feedback is used for the specific haptic effect generated by the hardware.

The influence of the vibration frequency on the friction reduction of an ultrasonic haptic surface has been reported in the literature. The models predict that increasing the frequency of the vibration leads to higher friction reduction at constant vibration amplitude, but this has not been reported experimentally. In this paper, we study the friction reduction on a prototype which can vibrate at low (66kHz) and high (225kHz) frequency. By estimating the Point of Subjective Equivalence between a standard at low frequency with a sample at high frequency, we have found that high frequencies can indeed reduce the friction, with the advantage of much smaller vibration amplitude. Moreover, we show that the invariant between the two frequency conditions is not the vibration amplitude. Our conclusion is that the invariant could be the acceleration instead.

To date, Several focusing techniques have been proposed to realize localized stimulation on haptic interfaces: phased arrays of actuators with delayed excitation, or time reversal techniques which require a preliminary learning phase. Additionally, these techniques are sensitive to parameters variation and disturbances. Modal decomposition allows to realize arbitrary vibration fields congruent with the boundary conditions and in this paper, Modal decomposition is proposed to realize a desired vibration velocity field in order to have differentiated stimuli. The experimental results validate the ability of the method to approximate satisfactorily a desired reference form. The psychophysical evaluations show that users can differentiate and localize the stimulation while exploring a surface with two fingers.

Although we hardly interact with objects that are purely elastic or viscous, haptic perception studies of deformable objects are mostly limited to stiffness and damping. Psychophysical investigation of materials that show both elastic and viscous behavior (viscoelastic materials) is challenging due to their complex, time and rate dependent mechanical behavior. In this study, we provide a new insight into the investigation of human perception of viscoelasticity in the frequency domain. In the frequency domain, the force response of a viscoelastic material can be represented by its magnitude and phase angle. Using this framework, we estimated the point of subjective equality (PSE) of a Maxwell arm (a damper and a spring in series) to a damper and a spring using complex stiffness magnitude and phase angle in two sets of experiments. A damper and a spring are chosen for the comparisons since they actually represent the limit cases for a viscoelastic material. We first performed 2I-2AFC adaptive staircase experiments to investigate how the perceived magnitude of complex stiffness changes in a Maxwell arm for small and large values of time constant. Then, we performed 3I-2AFC adaptive staircase experiments to investigate how the PSE changes as a function of the phase angle in a Maxwell arm. The results of our study show that the magnitude of complex stiffness was underestimated due to the smaller phase lag (with respect to a damper's) between the sinusoidal displacement applied by the participants to the Maxwell arm and the force felt in their finger when the time constant was small, whereas no difference was observed for a large time constant. Moreover, we observed that the PSE values estimated for the lower bound of the phase angle were significantly closer to their actual limit (0 degree) than those of the upper bound to 90 degrees.

This study investigates the relationship between electroencephalogram (EEG) activity and motor control difficulty during physical interaction with an admittance controlled robot. Subjects performed a fine cooperative manipulation task, in which the motor control difficulty was manipulated by altering admittance dynamics. To quantify motor control difficulty, an interaction instability index is proposed based on the spectral information of interaction forces. Regression analysis is then performed to construct a model to estimate motor control difficulty from EEG spectral features. The results indicate the reliability of EEG signals as an indicator of motor control difficulty in pHRI.

Dynamic mechanical allodynia is an aggravating neuropathological condition in which light, physical touch leads to pain. Developing pharmaceutical agents to treat this condition requires extensive animal trials using a mouse model, and a laborious process of manually stroking inflicted mouse paws, with a brush or cotton swab, while recording responses to that stimulus. In this paper, we developed an autonomous testing mechanism to create repeatable stroking sensations for mice during dynamic allodynia testing. The chamber consists of a belt driven brush mechanism and light and dark cham- bers. Additionally, we conducted a human subjects study to determine the baseline variability in human-performed dynamic allodynia testing. Our tactile stoke display is capable of stroking a mouse paw between 1-5 mm/s with a repeatable force. In our human subject experiments, the user applied force ranged from 0.1-9.0 gF with a maximum standard deviation of 4.13 gF. In contrast, our device is capable of producing repeatable brush strokes at 0.69 gF (SD = 0.13 gF) and 1.78 gF (SD = 0.16 gF) for two brushes. Preliminary animal studies show that normal mice are not disturbed by the stroking sensation; however, mice afflicted with allodynia move away from it. On average the injured mice spent 90% of their time in a bright, adverse environment to avoid the brush, whereas normal mice only spent 40% of their time in the bright environment.

Abstract—Surgery is a challenging domain for motor skill acquisition, and compounding this difficulty is the often delayed and qualitative nature of feedback that is provided to trainees. In this paper, we explore the effectiveness of providing real-time feedback of movement smoothness, a characteristic associated with skilled and coordinated movement, via a vibrotactile cue. Subjects performed a mirror-tracing task that requires coordination and dexterity similar in nature to that required in endovascular surgery. Movement smoothness, measured by spectral arc length, a frequency-domain measure of movement smoothness, was encoded in a vibrotactile cue. Performance of the mirror tracing task with smoothness-based feedback was compared to position-based feedback (where the subject was alerted when they moved outside the path boundary) and to a no-feedback control condition. Although results of this pilot study failed to indicate a statistically significant effect of smoothness-based feedback on performance, subjects receiving smoothness-based feedback altered their task completion strategies to improve speed and accuracy, while those receiving position-based feedback or no feedback only improved in terms of increased accuracy. In tasks such as surgery where both speed and accuracy are vital to positive patient outcomes, the provision of smoothness-based feedback to the surgeon has the potential to positively influence performance.

Haplink is an open-source, 3-D printed haptic device for educational applications. Haplink can be used as a one-degree-of-freedom device and then be turned into a two-degree-of-freedom device such that students who have learned one-degree-of-freedom concepts can translate those concepts to two degrees of freedom. In Autumn 2017, students in an undergraduate class built one-degree-of-freedom versions of Haplink, expanded them into two-degree-of-freedom devices, and then customized them to create novel haptic applications. In this demonstration, we will present two new systems resulting from student-driven Haplink customization that validate the device's effectiveness for scaffolding concepts related to design, programming, control, and mechatronics.

This demonstration presents a mirror tracing task, designed to serve as a proxy for endovascular surgical procedures, and intended to facilitate study of the effectiveness of delivering performance feedback based on movement smoothness via a vibrotactile device. The task requires participants to navigate through a path using a Novint Falcon where the motion is inverted on both axes. The shape of the path is designed to emulate the navigational tasks required in endovascular surgical procedures. In the demo, participants will perform the mirror tracing task and receive vibration feedback via C-2 tactors that encode either position-based or smoothness-based performance feedback.

The immersive experience of VR and AR is often hindered by the lack of force feedback, especially when users want to grasp virtual objects. Indeed, commercial joysticks provide only a simple vibrational feedback while high-end haptic devices, capable of very accurate force rendering, are costly and bulky. FOLDAWAY touch leverages origami manufacturing to bring force feedback into users’ hands by shrinking high-end robotized mechanisms to the size of human fingers. In the demonstration, the user will wear VR goggles and will manipulate virtual objects while perceiving their stiffness through the handheld FOLDAWAY touch joystick.

It is known that humans perceive a sense of force, like a pulling sensation in a particular direction, when asymmetric vibrations are presented. However, the perceptual mechanism of this illusion has not been well clarified. We have developed a device that can control the vibrations for quantitative evaluation of the pulling sensation induced by exerting asymmetric vibrations to user’s fingertip. It was confirmed that this device can generate acceleration of target asymmetric vibration even if frequency and participants are changed.

We propose a new lateral force feedback device, the UltraShiver, which employs a combination of in-plane ultrasonic oscillation (around 30 kHz) and out-of-plane electroadhesion. The lateral force is a result of friction being greater when electroadhesion is turned on than when it is turned off. The direction and magnitude of the lateral force can be adjusted by varying the phase between the in-plane oscillation and the electroadhesion. The UltraShiver is a simple and robust device that should serve as the basis for a wide variety of bare finger force feedback applications, such as virtual shape rendering.

We proposed a ferro-fluid based tactile display, which is lightweight and portable, but can replicate convincing contact orientation and texture information. Numerous studies have been conducted to develop tactile displays for providing convincing tactile feedback. However, most of the displays were limited in portability and restricted to delivering either texture information with vibrational cues or contact orientation with force feedback. To the best of our knowledge, the proposed tactile display is the first wearable tactile display which can deliver texture information together with contact orientation, and still be lightweight and portable. Introducing ferro-fluid allows minimizing the moving actuator components and also eliminates any counter reactive force on the fingernail of the user which may distort the tactile sensation otherwise. The demo will showcase ferro-fluid based tactile display for curvature discrimination and texture cues.

We investigate the mechanism, design, modeling and fabrication of a scalable high resolution, low cost and lightweight refreshable 2.5D tactile pin array controlled by electrostatic adhesive brakes which are made by patterning interdigital electrodes on high dielectric constant thin films. We will present two high resolution brake modules which are 1.7 mm pitch with 0.8 mm width pins and 4 mm pitch with 1.58 mm width pins with a maximum measured dynamic loading force of 76.3 gf and static loading force of 28 gf on an individual pin and show a small demonstration of 4 *2 pin array.

Two audio frequencies played together cannot be perceptually matched by a single frequency in between. But how does this translate to haptics; can a texture composed of two added spatial frequencies be perceptually matched by a single spatial frequency? Our recent work suggests we can match the pitch of these two textures, even if they are still distinguishable overall. We use an ultrasonic friction modulation device to display a composite texture side-by-side with a single frequency texture. A GUI allows users to make the best match by adjusting intensity and frequency. An audio version is available as well.

While many wearable tactile displays for the fingers, such as fingertip-type and glove-type displays, have been developed, their weight and size typically hinder the free movement of fingers, especially when considering the multi-finger scenario. We propose a method of presenting the haptics sensation of the fingertip on the forearm, not on the fingertip, to address this issue. A five-bar linkage mechanism was adopted to present a two-degree-of-freedom force. We presented a pressure sensation and a horizontal friction sensation perceived by the index finger and thumb to forearm, and the realism of the grasping experience in virtual reality was better when using the designed device than for no haptics cue or for vibration conditions.

We present a novel untethered, grounded 2D force feedback mechanism, with applications in education and design. The ballpoint drive achieves grounding through rolling frictional contact on an arbitrary 2D surface: through a pen, users feel directional force applied by four actuators to a mouseball in the tip. The ballpoint can render virtual features (walls, springs, texture) and active guidance. We will demo two environments. Racecar: draw features (e.g., a road), then “drive” on the road, bouncing off the road edges and over speedbumps. Electrostatic Lab: place point charges, write an equation governing their attraction, then feel these forces.

Although the hands are the most natural interface to interact with the world, we are still not using them to interact with virtual worlds. CLAP takes a step in that direction providing a realistic haptics-ready simulation of soft-hand. User hand is tracked and his representation (coupled structure of FEM skin, articulated body, rigid bodies) is simulated in a virtual environment. The forces experimented by this representation are used to drive haptic/tactile devices. CLAP(mslab.es/clap) is easily integrated within real-time engines and customized drivers can be developed for any type of tactile, haptics or tracking device using the provided interface and SDK.

This demo proposes a haptic ring for immersive virtual reality interactive experiences. It is capable of conveying the sensation of touching objects by providing skin indentation and thermal cues through a moving platform and a Peltier cell, respectively. The user's hand is replicated in the virtual environment through a Leap Motion tracker. The user can interact with a can and set its temperature pushing two buttons: one lights a fire increasing the temperature and the other starts a nitrogen flow to decrease it. The Peltier cell comes in contact with the user's phalanx only when the can is touched.

This demonstration presents MISSIVE (Multi-modal Interface of Stretch, Squeeze, and Integrated Vibration Elements), a novel, wearable, and multi-modal haptic feedback system that allows for increased perceptual accuracy compared to uni-modal, vibrotactile arrays of comparable size. The device combines two original servo-driven actuators with commercially available vibrotactors that mount on the user’s upper-arm. Our multi-modal haptic cues are comprised of simultaneously rendered, yet perceptually distinct elements: radial squeezing, lateral skin-stretching, and cutaneous vibration. Experiments with human subjects demonstrate that our approach can increase perceptual accuracy compared to a uni-modal vibrotactile system of comparable size.

In this demo, users will wear a sleeve of voice coils on their arm to mimic gestures used in social touch through the illusion of lateral motion. The voice coils are controlled to sequentially press into the user’s arm to create the sensation of linear travel up the arm. The duration of each actuator’s indentation and the amount of delay between the onset of indentation between adjacent actuators affects the perceived continuity and pleasantness of the interaction. Tests with this device have shown that slow indentations (1-3 Hz) with short delays between actuators creates the most continuous illusory motion.

The Haply platform simplifies the development of haptic applications across various programming environments and device configurations. The platform, coupled with the haptic API (hAPI)[i], allows developers to easily experiment with various robotic setups using up to four independent actuators. The ecosystem currently provides simulation development support on Mac, PC, and Android tablets. The Haply development platform seeks to lower the barrier of entry in the creation of haptic applications, allowing developers to test out ideas quickly at a marginal development cost. [i] The hAPI was created as part of the Cyberhap project, an NSF funded collaboration with the University of British Columbia and Stanford University to explore haptic technology in STEM education

Designing haptics could be complex and time consuming but well-placed haptics can create a positive experience. To maximize design potential, haptic designers need to understand how touch makes people feel and impacts their motivation. Haptics must also match and complement the visual and audio elements to improve the user experience. This presentation shares best practices uncovered from neuroscience and behavioral studies on people’s reactions to haptics. We will present our findings on the way haptics can enhance the user experience and discuss tips for advanced haptic design, such as focus on key elements, highlight positive emotions, and emphasize movement and function.

We will demonstrate haptic interaction using a magnetic levitation device of a new design, which uses low-cost components and a compact array of 12 cylindrical coils to generate forces and torques in any direction on magnets embedded in a free-floating instrument handle grasped by the user. The device allows co-located graphics when a thin display monitor is placed directly on top of the coil array, underneath the user handle. The system can display shape, texture, friction, and elasticity of 3D objects in dynamic simulations, providing rigid-body haptic feedback with no friction or backlash, and minimal inertia.

This artistic project presents a vibrotactile “musical sculpture” in virtual reality. While wearing a spatially-tracked head-mounted display and custom 2-channel-tactor glove, participants can visually and haptically explore a virtual, permeable sculpture suspended in 3-dimensional space, comprised of dynamic, musical-like vibrations. As a general aim, the work proposes a more abstract approach to using vibrotactility in virtual environments for artistic ends. More specifically, as a product of research-practice, this piece is a proof-of-concept of the expressive spatial potential concerning two apparent motion illusions that arise from frequency beating and changing phase. A number of informal studies investigating these illusions informed the creative choices of this work.

We present a robotic setup for the acquisition of object surface material properties. Our setup is able to collect selected kinesthetic characteristics, such as the surface structure and weight, as well as tactile properties like the friction coefficient and the fine roughness characteristics of the object surface. Additionally, the setup captures the visual appearances of the object. The recorded multimodal sensor data provide all relevant information required to form a haptic model of a material sample. We then use this representation in a standard haptic rendering framework and display the virtual materials using an augmented Phantom Omni device. We conducted a subjective experiment which shows that its participants perceived and rated the rendered virtual materials as similar to the corresponding real materials in a direct comparison test where the users interact simultaneously with the real and the virtual material samples. An overall user rating of 4.3 out of 5.0 is achieved during the subjective experiment.

Touching an object with our fingers yields frictional forces that allow us to perceive and explore its texture, shape, and other features. While the relevance of these frictional forces to sensory and motor function in the hand is well established, the way that they reflect the shape and texture of touched objects is not well understood. This lack of knowledge currently constraints the development of haptic rendering algorithms that rely on reproducing frictional forces to mimic the sensation of touching real surfaces. To address this, we created a low order contact mechanics inspired model that accounted for: disconnection between the finger and high relief features of the surfaces, and differences of pressure applied in the contact regions. The output of our model was compared to experimental frictional force measurements of bare-fingers sliding on structured surfaces. The results show high similarity between the measurements and the model output. The model presented can be used to predict frictional forces if the geometry of a touched surface is known together with the coefficient of friction of the untextured surface.

This paper presents an improved approach to geometry-based haptic texture modeling and rendering. We adopt photometric stereo, one of the most accurate algorithms for 3D surface reconstruction, to increase the resolution of captured geometry profiles. This benefit of higher texture resolution can enhance the realism of rendered textures in terms of roughness. To this end, we have designed and constructed a dome-shaped lighting structure for use in the modeling using photometric stereo. With this apparatus and the photometric stereo algorithms, we can achieve very high texture modeling resolution in the order of 10 mm. We also identify the stiffness and friction of real materials using the Hunt-Crossley model and the Dahl model, respectively, for realistic texture rendering. A user study measuring the perceived similarity between real and virtual textures demonstrated that our system can achieve a reasonably high level of realism in rendering textured objects with high compliance or low friction.

With the introduction of variable friction displays, new possibilities have emerged in haptic texture rendering on flat surfaces. In this work, we propose a data-driven method for realistic texture rendering on an electrovibration display. We first describe a motorized linear tribometer we developed to collect lateral frictional forces from the textured surfaces under various scanning velocities and normal forces. We then propose an inverse dynamics model of the display to describe its output input relationship using nonlinear autoregressive with external input (NARX) neural networks. Forces resulting from applying a pseudo-random binary signal (PRBS) to the display are used to train each network under the given experimental condition. A comparison between the real and virtual forces in frequency domain shows promising results for recreating virtual textures similar to the real ones and also reveals the capabilities and limitations of the proposed method.

This paper proposes a novel method of enhancing the pseudo-haptic effect on the touch panel. In prior studies to invoke pseudo-haptics on the touch panel, there has been a problem stemming from the difference in position between the finger and the object. We added the virtual string on display which shows a connection between the finger and the object. The user studies showed that the presence of the virtual string was effective to invoke pseudo-haptics, and the setting of string breaks was key to enhance the effect of it.

The focus of this study was to investigate the ability to discriminate between melodic intervals of the equal tempered scale based solely on vibrotactile stimulation. In music, a melodic interval is the musical distance between two pitches, or notes, that are played sequentially. This paper tests the hypothesis that people can detect melodic intervals that are presented to different body sites such as the fingertip of the index finger of the non-dominant hand, as well as to the flank, the lateral region between the ribcage and the hip bone. Vibrotactile stimuli on the flank were displayed through voice coils of different diameters (∅ 13 mm and ∅ 25 mm respectively), while those at the finger were displayed with a ∅ 13 mm diameter voice coil. Six melodic intervals ranging from a minor second (∆f=100 cents) to a perfect fifth (∆f=700 cents) were compared to the reference interval of a perfect prime (∆f=0 cents) at a fundamental frequency (f0) of 65 Hz. Discrimination was significant for intervals as small as a major second (∆f=200 cents; i.e. 8Hz, or 12.3%), depending on the location of the body. Overall, results tend to suggest that larger intervals are less difficult to detect than smaller intervals.

This study explores people's ability to distinguish spatial complexity in tactile textures, with the eventual goal of reducing the necessary complexity of texture representation for surface display devices. To this end, we tested subjects' ability to perceptually match a reference texture containing two spatial frequency components by adjusting the frequency and amplitude of a single frequency. All textures consisted of spatially varied friction levels on a glass display screen, where friction was modulated via amplitude of ultrasonic vibrations. Resulting chosen single frequencies were systematic, and suggest subjects can identify a single frequency, or tactile pitch, falling somewhere between those of the reference texture. Subject-adjusted frequency is modeled as a function of the reference texture's frequency components and the ratio of their amplitudes.

How sensitive is the neck in distinguishing two points of contact? The current study was performed to gain insight about the spatial acuity of the neck in hopes of using this information in future research and development of tactile devices targeting this area of the body. Using a two-point orientation discrimination task, a relatively novel measure of spatial sensitivity, the threshold of eight locations around the circumference of the neck was measured. Results showed uniform sensitivity in seven of the eight locations, with a slightly lower threshold at the very front of the neck. These results indicate that the neck could be a viable location to effectively communicate tactile information in a uniform fashion.

Tactile distance perception typically requires the integration of tactile speed over time. In this study, we investigated tactile distance perception by comparing subjects’ ability to reproduce tactile distance by moving the hand, and compared this to their ability to estimate the magnitude of sliding distance. This is the first investigation of subjects’ ability to use motor reproduction to reproduce the same tactile distance that was felt during passive sliding contact. To clarify the role of vibrotactile information in sliding distance perception, we investigated tactile distance perception by manipulating tactile mechanical signals in two different ways: by adding mechanical noise to the finger, or by sensing and amplifying vibrotactile signals in the finger. In our experiment, subjects systematically underestimated tactile sliding distance. On average, they reproduced 55% of the actual distance. Responses on the magnitude estimation and the distance reproduction tasks were highly similar. Subjects estimated sliding distance to be significantly longer when tactile signals were manipulated compared to no vibrotactile manipulations, although mechanical noise impaired distance perception accuracy. These results suggest that further research is needed in order to clarify the effect of underestimation of tactile distance on fine motor tasks guided by touch.

The objective of the present research was to determine how variations in the signals generated by different vibrotactile actuators are perceived and which features are judged as being distinctive. For this purpose, three different types of actuator were used to generate signals that varied in amplitude, waveform and frequency. Participants were required to judge the degree of similarity-dissimilarity between pairs of stimuli. Multi-dimensional scaling (MDS) techniques were then used as an exploratory data analysis technique to create a spatial map that depicted the relations among the various vibrotactile signals. The first dimension that emerged from the MDS represented a continuum associated with transitions in the amplitudes of the signals, with a smooth sine wave pattern contrasting with the more abrupt transitions in square waves. This may be considered a smooth-rough dimension. The second dimension extracted from the data was more difficult to characterize in that each of the two clusters along this dimension involved signals of varying waveform and frequency. Further work will aim at defining the perceptual qualities of this dimension.

We characterize the detection thresholds of subjects holding a stylus with a precision grasp for six principal modes of vibrotactile display: three orthogonal force directions and three orthogonal torque directions. Subjects are far more sensitive to torque signals about the shaft of the stylus than to torque signals orthogonal to it, subjects are less sensitive at low frequencies to force signals parallel to the shaft than to force signals orthogonal to it, and the thresholds for force and torque signals applied orthogonal to the shaft can be approximately equated by considering the reaction moment felt at the grasp point.

We investigate the mechanism, design, modeling and fabrication of a scalable high resolution, low cost and lightweight refreshable 2.5D tactile pin array controlled by electrostatic adhesive brakes. By replacing linear actuators in motorized shape displays with a high voltage solid-state circuit that can be fabricated with printable electronics techniques, we can decrease the cost and complexity of such devices. Electrostatic adhesive brakes, made by patterning interdigital electrodes on high dielectric constant thin films, are used to hold metal pins' positions and provide contact force to the user's fingertip. We present designs of two high resolution brake modules which are 1.7 mm pitch with 0.8 mm width pins and 4 mm pitch with 1.58 mm width pins with a maximum measured dynamic loading force of 76.3 gf and static loading force of 28 gf on an individual pin (for the later size). A small demonstration of 4 x 2 pin array with a 4 mm pitch size within a row and 2.5 mm pitch size between the rows, using 1.58 mm width pins, was created. We also characterized the refresh time to be 37.5 ms for each brake, which enables refreshable actuated pin displays.

This work presents an exploratory user study of human-to-robot handovers. In particular, it examines how changes in a robot behaviour influence human participation and the overall interaction. With a 2x2x2 experimental design, we vary three basic factors and observe both the interaction position and forces. We find the robot's initial pose can inform the giver about the upcoming handover geometry and impact fluency and efficiency. Also we find variations in grasp method and retraction speed induce significantly different interaction forces. This effect may occur by changing the giver's perception of object safety and hence their release timing. Alternatively, it may stem from unnatural or mismatched robot movements. We determine that making the robot predictable is important: we observe a learning effect with forces declining over repeated trials. Simultaneously, the participants' self-reported discomfort with the robot decreases and perception of emotional warmth increases. Thus, we posit users are learning to predict the robot, becoming more familiar with its behaviours, and perhaps becoming more trusting of the robot's ability to safely receive the object. We find these results exciting as we believe a robot can become a trusted partner in collaborative tasks.

HapticDrone is our new approach to transform a drone into a force-reflecting haptic interface. While our earlier work proved the concept, this paper concretizes it by implementing accurate force control along with effective encountered-type stiffness and weight rendering for 1D interaction as a first step. To this end, generated force is identified with respect to drone's thrust command, allowing a precise control of force. Force control is combined with tracking of drone and hand, turning the HapticDrone into an ideal end-effector for the encountered-type haptics and making the system ready for more sophisticated physics simulation. Stiffness and weight of a virtual object is our first target of the simulation. Conventional stiffness and gravity simulation is merged into the encountered-type position control scheme, allowing a user to feel the softness and weight of an object. We further confirmed the feasibility by proving that the system fulfills the physical performance requirements commonly needed for an encountered-type haptic interface, i.e., force rendering bandwidth, accuracy, and tracking performance.

A midair haptic display that provides a cooling sensation is developed in this study. Although a noncontact heat display using infrared rays has been reported, such an infrared display cannot cool the skin surface. In this study, we generate and control a straight, thin, long, air flow driven by an ultrasound-phased array and transport cold air to a localized spot on the users’ skin. Since the air flow position and direction is electronically steerable, the display area can be controlled freely. A prototype of the proposed display was developed and the cooling performance was evaluated. We confirm that a small cool spot is created and shifted on a user’s forearm based on our observations of the temporal changes in the skin temperature distribution using thermography.

This paper presents a data-driven thermal rendering framework for highly realistic re-creation of the thermal dynamics of real objects in a virtual environment. We begin with a physics-based model for thermal rendering and then extend it to compute the desired temperature of a thermal display without identification of the thermal parameters of real objects. Our approach is data-driven: we collect the thermal responses between hand and object for many different combinations of the initial temperatures of hand and object and then interpolate several real thermal responses to render virtual thermal stimuli with the target initial temperatures. We also present perceptual evaluation results obtained in a discrimination experiment and a (preliminary) identification experiment, which both advocated the adequacy of our data-driven thermal rendering framework.

Cable haptic devices can produce a large set of forces while being inconspicuous. This paper outlines the design and the implementation of such a device as a desktop virtual environment. It's combined to a stereoscopic 3D display, visualized through a semi-mirror, below which the manipulation space is located. This configuration co-localizes, i.e superimposes, visual and haptic feedbacks for realistic user interaction. The constraint imposed by the reflective surface requires a specific configuration of the cable device. The process to design such a desktop sized virtual reality interface is explicitly exposed here. An over-constraint cable mechanism is proposed. The assembly of the complete co-localized visio-haptic interface is described, as well as a force computation method easy to implement.

The Stanford SHAPE Lab, directed by Prof. Sean Follmer, explores how we can interact with digital information in a more physical and tangible way. We develop advanced technologies in robotics, mechatronics, and sensing to create interactive, dynamic physical 3D displays and haptic interfaces that allow 3D information to be touched as well as seen. We are specifically interested in using these novel interfaces to support richer remote collaboration, computer aided design, education, and interfaces for people with visual impairments. In pursuit of these goals, we use a design process grounded in iterative prototyping and human centered design and look to create new understanding about human perception and interaction through controlled studies. Our research in Human Computer Interaction and Human Robot Interaction is currently directed in five areas: dynamic physical shape displays, wearable haptics for grasping in VR, ubiquitous robotic interfaces, mobile haptics, and soft actuation and sensing.

The Stanford Collaborative Haptics and Robotics in Medicine Lab, directed by Prof. Allison Okamura, designs and studies haptic systems using both analytical and experimental approaches. Haptics, the sense of touch, is crucial for human exploration and manipulation of the world. In medicine and rehabilitation, haptic interaction is often necessary for reasons of performance, safety, and user acceptance. Our research is devoted to developing the principles and tools needed to realize advanced robotic and human-machine systems capable of haptic interaction. We are particularly interested in teleoperation, virtual environments, and robotic manipulation. This research has applications in many areas, including robot-assisted surgery, simulation and training, rehabilitation, exploration of hazardous or remote environments, enabling technologies, manufacturing, design, mobile computing, and education. A major theme of our work is biomedical systems.

The Stanford Biomimetics and Dexterous Manipulation Lab, directed by Prof. Mark Cutkosky, studies bioinspired mechanical systems. Current areas of research include adhesions and applications, mobile manipulation, multi-limbed climbing, MR compatible tools, smart graspers for minimally invasive surgery and robot-assisted surgery, multi-modal robots, automotive haptics, tunable compliance and damping, and manufacturing and prototyping.

The Stanford Camarillo Lab, directed by Prof. David Camarillo, invents intelligent biomedical devices that advance science, enhance health, and reduce illness and disability. Current projects include: investigating the mechanism of concussion and characterizing the response of head-blows through imaging, blood, and other neurophysiological measurements; researching cell mechanics for regenerative medicine to develop a quantitative, noninvasive and early (day 1) measure of viability; and improving the usability and safety of the process of cardiac catheter ablation through new control methods, medical image guidance, and automation for robotic catheter procedures.

The Stanford Robotics Lab, directed by Prof. Oussama Khatib, is involved in research pertaining to robotic manipulation and control with a focus on interacting with humans in unstructured environments. In addition to developing safe, reactive and compliant autonomous behaviors to work in such environments, we also study human motion control and manipulation skills to develop complex interaction capabilities between robots and humans. A key aspect of this interaction is a rich interface that allows multi-modal communication either at a distance or while co-operating in the same space. Our recent successes include Ocean One, the first underwater humanoid robot, which was used by archeologists to recover artifacts from the Mediterranean seabed.