ISS 2024
Conference on Interactive Surfaces and Spaces (ISS 2024)

Powered by

Conference Publishing Consulting

Conference on Interactive Surfaces and Spaces (ISS 2024), October 27–30, 2024, Vancouver, BC, Canada

ISS 2024 – Preliminary Table of Contents

Contents - Abstracts - Authors

Frontmatter

Title Page

Message from the Chairs

Committees

Sponsors

Doctoral Symposium

A Physical Computing Workshop to Engage Girls from Low-Income Backgrounds
Katherine Vergara
(Pontificia Universidad Católica de Chile, Chile)
The persistent gender gap in computer science, especially among women from low-income backgrounds, continues to limit diversity and innovation within the technological sector. This underrepresentation also restricts access to career paths that can enhance social mobility, particularly for women in developing countries. Physical computing offers a hands-on approach that can improve programming skills and computational thinking through interaction with tangible hardware. This research focuses on developing a short physical computing workshop tailored to young girls from low-income communities. The study combines education, tangible interfaces, and coding. Through a series of classroom-based studies and laboratory experiments, this PhD work will assess the impact of the workshop on self-efficacy and learning in programming and computational thinking. The anticipated contributions of this research include insights into the effectiveness of tangible, user-friendly physical computing workshops in increasing engagement among underrepresented groups in computer science.

Enhancing Communication between Older Adults with Chronic Conditions and Healthcare Providers using Large Displays
Mahsa Sinaei Hamed
(Carleton University, Canada)
Our research aims to address three interconnected objectives. First, we investigated the challenges associated with designing, developing, and evaluating data visualizations for large displays by exploring expert insights into the design, development, and evaluation of data visualizations on LDs, revealing 12 challenges and proposing future research directions (Objective 1). Second, we explored the challenges older adults with multiple chronic conditions encounter when reviewing and sharing their health data with their healthcare providers during clinical visits through interviews and diary studies (Objective 2). Finally, using our findings from Objectives 1 and Objective 2, we aim to develop a data visualization tool on LDs to facilitate communication between older adults with multiple chronic conditions and healthcare providers (Objective 3). This approach seeks to bridge the gap between the technical challenges of designing, developing, and evaluating data visualizations on LDs and the practical communication needs of older adults with multiple chronic conditions. By addressing both aspects, we aim to design and develop an effective data visualization tool on LDs, which could result in overall better health outcomes for older adults with multiple chronic conditions.

Monocular Tracking of Passive Stylus on Passive Surface
Tavish M. Burnah
(Massey University, New Zealand)
This PhD project introduces a mixed reality passive stylus system designed for smartphones enabling digital ink creation on a surface. Traditional passive stylus systems face challenges in usability and accessibility. This research aims to overcome these limitations through a dual approach that integrates interactive design and robust machine learning, trained on extensive datasets. The research methodology is divided into phases of data collection, design, implementation and evaluation. The project develops an application to deliver an accessible digital tool viable for widespread use, particularly in developing regions.

Demonstrations

Spot Shadow: A System for Manipulating Shadows in Spatial Design
Saki Sakaguchi
(Tokyo Metropolitan University, Japan)
The design of the lighting environment is important for determining room specifications. Recently, systems that can change the lighting environment in a room by controlling the direction and intensity of light using a computer have been proposed. However, with such a method of controlling the light, it is easy to specify the size and position of the brightly illuminated area, but not the darkened area. In this study, we controlled shadows and proposed a method for controlling dark areas. We construct Spot Shadow, a system that can generate shadows of arbitrary sizes and shapes at positions specified by users. The prototypes of a tabletop system and a large system can create shadow generation areas of 1m × 0.9m and 7m × 4.3m, respectively. This study proposes a method for manipulating shadows and demonstrates new possibilities for creating interactive spaces using shadows.

PanoCoach: Enhancing Tactical Communication in Soccer with Virtual Reality (VR)
Andrew Kang, Hanspeter Pfister, and Tica Lin
(Harvard University, USA; Rice University, USA)
Soccer, as a dynamic team sport, requires seamless coordination and integration of tactical strategies across all players. Adapting to new tactical systems is a critical but often challenging aspect of soccer at all professional levels. Even the best players can struggle with this process, primarily due to the complexities of conveying and internalizing intricate tactical patterns. Traditional communication methods like whiteboards, on-field instructions, and video analysis often present significant difficulties in perceiving spatial relationships, anticipating team movements, and facilitating live conversation during training sessions. These challenges can lead to inconsistent interpretations of the coach’s tactics among players, regardless of their skill level. To bridge the gap between tactical communication and physical execution, we propose a mixed-reality telepresence solution, PanoCoach, designed to support multi-view tactical explanations during practice. Our concept involves a multi-screen setup combining a tablet for coaches to annotate and demonstrate concepts in both 2D and 3D views, alongside VR to immerse athletes in a first-person perspective, allowing them to experience a sense of presence during coaching. In our preliminary study, we prototyped the cross-device functionality to implement the key steps of our approach: Step 1, where the coach uses a tablet to provide clear and dynamic tactical instructions, Step 2, where players engage with these instructions through an immersive VR experience, and Step 3, where the coach tracks players' movements and provides real time feedback. User evaluation with coaches at City Football Group, Harvard Soccer and Rice Soccer suggests this mixed-reality telepresence approach holds promising potential for improving tactical understanding and communication. Based on these findings, we outline future directions and discuss the research needed to expand this approach beyond controlled indoor environments, such as locker rooms, leveraging telepresence to enhance tactical comprehension and simulated training.

Eye-Hand Movement of Objects in Near Space
Uta Wagner, Andreas Asferg Jacobsen, Tiare Feuchtner, Hans Gellersen, and Ken Pfeuffer
(Aarhus University, Denmark; University of Konstanz, Germany; Lancaster University, United Kingdom)
Hand-tracking in Extended Reality (XR) enables moving objects in near space with direct hand gestures, to pick, drag and drop objects in 3D. We explore the use of eye-tracking to reduce the effort involved in this interaction. As the eyes naturally look ahead to the target for a drag operation, the principal idea is to map the translation of the object in the image plane to gaze, such that the hand only needs to control the depth component of the operation. We demo several applications we build for 3D manipulations, including area selection, 3D path specification, and a "Bejeweled" inspired game, showing potential for effortless drag-and-drop actions in 3D space. This demonstration includes the study apparatus and the applications from a paper that will be presented at UIST'24 with the same title.

Have Fun with Math: Multimodal, Interactive, and Immersive Exploration of Wave Functions with 3D Models
Shamima Yasmin
(Eastern Washington University, USA)
Postsecondary STEM courses include substantial mathematics and algorithms. Students need motivation to dig deep into the topic. Multisensory modeling allows users to explore objects with multiple senses, i.e., audio, visual, and touch. If augmented with virtual reality (VR), the overall experience could be more immersive and enjoyable. This research investigated students’ experience with unimodal versus multimodal visualization and exploration of wave functions with and without VR integration. Students interactively explored wave functions, i.e., sine, cosine, sawtooth, square, and triangular waves, with 3D models to concretize understanding of wave parameters, i.e., frequency, amplitude, phase, and vertical shifts. Initial findings showed that students preferred the audio- visual exploration of wave functions over the visual-only version in VR-enhanced and non-VR platforms. Overall, VR enriched their experience while interacting with the data.

Workshop Summaries

Summary of the Workshop on Interactions for Supporting Explanations and Promoting Comprehension
Maryam Rezaie, Sydney K. Purdue, Anjali Khurana, Parmit K. Chilana, Melanie Tory, and Sheelagh Carpendale
(Simon Fraser University, Canada; Northeastern University, USA)
Interfaces and visualizations often challenge comprehension, especially as they grow in complexity. Traditional methods—relying on standard inputs like touch, mouse, and keyboard—fall short in addressing the nuanced demands for explainability in complex systems. This workshop explores innovative interaction strategies to enhance self-enabled comprehension, focusing on the development and refinement of new devices and input modalities. We aim to gather researchers, designers, and practitioners to exchange ideas and explore interaction techniques that promote clearer understanding and transparency across diverse applications. This collaborative effort seeks to advance interactive systems that improve explanation and user comprehension in digital environments.

Summary of the Workshop on Visual Methods and Analyzing Visual Data in Human Computer Interaction
Zezhong Wang, Samuel Huron, Miriam Sturdee, and Sheelagh Carpendale
(Simon Fraser University, Canada; Télécom Paris - Institut Polytechnique de Paris, France; University of St Andrews, United Kingdom)
Visual methods have become increasingly vital in Human Computer Interaction (HCI) research, particularly as we analyze and interpret the complex visual data that emerges from various interaction modalities. However, the methodologies for analyzing this visual data remain underdeveloped compared to textual data analysis. This workshop seeks to unite HCI researchers who work with visual data — such as hand sketches, photographs, physical artifacts, UI screenshots, videos, and information visualizations — to identify, name, and categorize methods for analyzing visual data in HCI.

Posters

Extracting Corneal Reflection of Screen by High-Speed Control of Polarization
Ayato Nakamura and Kentaro Takemura
(Tokai University, Japan)
Polarization has been considered to be a reference instead of near-infrared light sources in eye tracking because the light emitted from a liquid-crystal display is typically polarized. However, the degree of polarization depends on the display content. Thus, devising a novel method is crucial for stably extracting the display reflection from the corneal surface. Therefore, we propose an eye-tracking method that inserts a white background between the display contents using a high-speed display to extract the screen reflection on the cornea. A high-speed camera and polarization modulator are integrated to extract the polarized light emitted from a high-speed display, and then the point-of-gaze is estimated. We evaluated the accuracy of the estimated point-of-gaze under several conditions to compare the proposed method with conventional approaches. The results revealed that the proposed method improved eye gaze estimation.

A Two-Handed Ellipsoidal Device for Interactive Grasping and Gripping Rehabilitation Exercise
Bingjie Xu, Yijia An, Qinglei Bu, Yung C. Liang, and Jie Sun
(Suzhou Industrial Park Institute of Vocational Technology, China; Xi’an Jiaotong-Liverpool University, China; National University of Singapore, Singapore)

Interactive Game Carpet with Gait Analysis for Children’s Lower Limb Training
Yaxuan Liu, Yijia An, Keming Zhang, Martijn Ten Bhömer, Yung C. Liang, Qinglei Bu, Jie Sun, and Siyuan Chen
(National University of Singapore (Suzhou) Research Institute, China; Xi’an Jiaotong-Liverpool University, China; National University of Singapore, Singapore)

Exploring Effects of Interactive Virtual Reality Sensory Environment on Anxiety Reduction in Adolescents with Autism
Amaya E. Keys, Oyewole Oyekoya, and Daniel Chan
(Howard University, USA; CUNY Hunter College, USA; CUNY, USA)
Autism Spectrum Disorder (ASD) is a developmental disability often characterized by sensory processing difficulties that can lead to anxiety, particularly in children and adolescents. Previous research on virtual reality-based anxiety intervention tools focuses on using social skills training, exposure therapy, and meditative coaching to mitigate social and phobia related anxiety. However, minimal work has specifically evaluated the effects of virtual multi-sensory environments for people with ASD, often only testing feasibility. This pilot study aims to build on previous work by investigating how various auditory, visual, and interactive components contribute to user satisfaction and sensory-related anxiety reduction. The objective is to gain a better understanding of what features are significant towards developing a successful virtual anxiety intervention tool. Results suggest using interactive activities that promote fine motor skills can provide a healthy outlet for self-mediated stress relief. Future development aims to incorporate task-based activities, and enhance audio, visual, and lighting displays. The deployment of a full-scale study with a larger sample size and target participant pool is warranted to substantiate these initial findings.

Exploring the Impact of Size and Position on Visual Feedback Efficacy for Ballet Dancers
Hana Mizukami, Arinobu Niijima, Chanho Park, and Takefumi Ogawa
(University of Tokyo, Japan; NTT Corporation, Japan)
In balance training, such as ballet, observing one's posture in a mirror makes it easier to maintain balance. By using projectors, it is possible to show the user's posture from different angles, magnify specific body parts, and display the Center of Pressure (COP) trajectory in real time. This gives the user more visual feedback information than a mirror can provide, helping to improve balance. However, the appropriate projection position and size to enhance the effect of such visual feedback remains unclear. This study focuses on relevé in ballet and examines the effects of different types and positions of visual feedback on balance improvement. We conducted a user study and calculated balance metrics from COP data obtained from a balance board. The results indicate that only visual feedback projected directly at eye level in front of the user during relevé contributes to balance improvement. In contrast, visual feedback projected above eye level to the right did not show a clear effect on balance improvement.

Co-designing a Tangible Communication Device to Enrich Communication over Distance
Hannah Friederike Fischer, Anke Königschulte, Jana Koch, Serge Autexier, and Gesche Joost
(German Research Center for Artificial Intelligence, Germany; C&S Computer and Software, Germany)
The initial phase of innovative product design is marked by uncertainty and complexity. This paper examines the use of participatory workshops to navigate this phase within the ToCaro research project. The project aims to develop tactile and multisensory interfaces for remote communication to mitigate feelings of loneliness by promoting a sense of physical proximity. Fourteen co-design workshops were conducted with senior participants (age ≥ 65) to examine their communication behaviors, identify latent needs and evaluate physical sensations elicited by various materials and forms of interaction. The workshops included semi-structured interviews, sensory perception tests, interaction concept evaluations, and “quick-and-dirty” prototyping. This paper outlines the facilitators’ experiences, the challenges, and learnings. Results indicate that while participants exhibited varied levels of engagement, those with a perceived need for new communication devices contributed effectively to the creative process.

A Preliminary Design Space for Immersive Data Storytelling
Radhika Pankaj Jain, Kadek Ananta Satriadi, Adam Drogemuller, Ross Smith, and Andrew Cunningham
(University of South Australia, Australia; Monash University, Australia)
Immersive data storytelling is an emerging field that combines narrative visualisation and immersive analytics to engage an audience. While there are existing design spaces for narrative visualisation on 2D displays, there are no guidelines for creating immersive data stories, making it difficult for practitioners and researchers to explore this space. In this paper, we present a preliminary design space for immersive data storytelling that is informed by current practices and multi-disciplinary views. We interviewed multi-disciplinary experts, including museum designers, architects, and game designers, to understand how they communicate stories in physical spaces and immersive mediums. We applied inductive thematic analysis to the interview responses to inform the dimensions of the design space and analysed a systematic selection of publicly available immersive stories. In the end, we had 13 dimensions in 7 categories. We present insights into this design space as common practice or areas for future research.

Effects of Increasing Command Capacity of Spatial Memory Menus in Tablets
Sayeem Md. Abdullah and Md. Sami Uddin
(University of Regina, Canada)
Spatially-stable touch menus, like FastTap, leverage users’ spatial memory to enable rapid command selection on tablets. Although these spatial tablet interfaces can aid in developing spatial memory of commands having a small command set, it is, however, unknown whether spatial memory remains beneficial when the number of commands grows. Therefore, we carried out a study to investigate spatial learning in four different sizes of single-tab FastTap Menus: Small, Medium, Large, and Extra-Large, with 16, 30, 42, and 56 items, respectively. Results indicated that people do develop spatial memory in all menus; however, there is a negative correlation between command capacity and spatial memory development in tablets. We contribute new knowledge on spatial memory development in touch tablets that can enhance the design of future spatial memory-based tablet interfaces.

Balancing Autonomy: Investigating User-Controlled vs Automated Guidance Systems for Sequential Tasks
Robin Wiethüchter, Saikishore Kalloori, and David Lindlbauer
(ETH Zurich, Switzerland; Carnegie Mellon University, USA)
Self-guided tutorials are popular resources for learning new tasks, but they lack important aspects of in-person guidance like feedback or personalized explanations. Adaptive guidance systems aim to overcome this challenge by reacting to users' performance and expertise and adapting instructions accordingly. We aim to understand the users' preferred balance of automation and control, what representation of instructions they prefer, and how human experts give instructions to match users' needs. We contribute an experiment where users perform different virtual tasks, guided by instructions that are controlled by experts using a wizard-of-oz paradigm. We employ different levels of automation to control instructions and alter their level of detail and step granularity to match the user's needs. Results indicate that while users preferred automated systems for convenience and instant feedback, they appreciated a degree of manual control since they felt less rushed. Experts relied on factors such as expected expertise, hesitation, errors, and their understanding of the current task state as main triggers to adapt instructions.

Gamification of Food Selection and Nutrition Education in Virtual Reality
Caroline A. Klein, Oyewole Oyekoya, and Margrethe Horlyck-Romanovsky
(Vassar College, USA; CUNY Hunter College, USA; CUNY Brooklyn College, USA)
The increasing global prevalence of obesity and related health issues underscores the need for innovative dietary interventions. This paper explores the potential of combining gamification and Virtual Reality (VR) to promote healthier eating habits among young adults. By creating an interactive VR food environment with engaging game elements, we aim to assess the impact of gamified VR intervention on nutritional knowledge and attitudes. Preliminary results show an increase in nutritional understanding and awareness, though further research is necessary for statistical validation. This study suggests that VR-based gamified interventions could be a promising tool for nutrition education, behavior modification, and virtual food selection.

Enhancing Virtual Mobility for Blind and Low Vision Individuals: A Stationary Exploration Method
Hong Zhao and Oyewole Oyekoya
(CUNY Borough of Manhattan Community College, USA; CUNY Hunter College, USA)
Designing accessible locomotion methods for individuals who are blind or have low vision (BLV) is a complex challenge, particularly in mobile VR environments with limited interface options. In this paper, we propose a novel locomotion technique on mobile VR that enables users to control a virtual character's movement while staying stationary or within a small physical area. The technique utilizes the phone's gyroscope for movement control, while providing spatial audio and vibration feedback to enhance virtual exploration for BLV individuals. Our study examines how BLV individuals acquire spatial knowledge in mobile VR environments. A user study is conducted to assess the effectiveness of the proposed approach.

proc time: 0.56