Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Indoor Photovoltaic Interactive ..."
Indoor Photovoltaic Interactive Surfaces for Sustainable Smart Home Control: Gesture Design using Guessability and Production Methods
Nora Abdullah Almania, Sarah Yousef Alhouli, and Deepak Ranjan Sahoo
(Swansea University, Swansea, UK; Shaqra University, Shaqra, Saudi Arabia; Kuwait Institute for Scientific Research, Kuwait, Kuwait)
Indoor photovoltaic materials are novel low-cost light sensors that can be flexible, decorative, self-powered, and battery-free, and can be embedded in various surfaces throughout the home. They offer a unique opportunity for contextual control of multiple different devices in a smart home using guessable and favorite gestures. Currently available gesture vocabularies are survey-based and sensor-agnostic, but still require experimental validation. Therefore, we present experimentally generated and validated original gesture vocabularies using two user elicitation methods, the guessability and production methods, for such sensors. The capabilities of the sensor was used to prime participants for design thinking to multi-control smart home devices. We provide guidelines for designing gesture vocabularies using the two elicitation methods and report on their similarities and differences. The methodological findings and experimentally validated gesture sets would inform HCI researchers in the design of user-elicited interactions for such versatile light or electromagnetic field sensors and similar gesture-driven applications.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Indoor Photovoltaic Interactive ..."
Indoor Photovoltaic Interactive Surfaces for Sustainable Smart Home Control: Gesture Design using Guessability and Production Methods
Nora Abdullah Almania, Sarah Yousef Alhouli, and Deepak Ranjan Sahoo
(Swansea University, Swansea, UK; Shaqra University, Shaqra, Saudi Arabia; Kuwait Institute for Scientific Research, Kuwait, Kuwait)
Indoor photovoltaic materials are novel low-cost light sensors that can be flexible, decorative, self-powered, and battery-free, and can be embedded in various surfaces throughout the home. They offer a unique opportunity for contextual control of multiple different devices in a smart home using guessable and favorite gestures. Currently available gesture vocabularies are survey-based and sensor-agnostic, but still require experimental validation. Therefore, we present experimentally generated and validated original gesture vocabularies using two user elicitation methods, the guessability and production methods, for such sensors. The capabilities of the sensor was used to prime participants for design thinking to multi-control smart home devices. We provide guidelines for designing gesture vocabularies using the two elicitation methods and report on their similarities and differences. The methodological findings and experimentally validated gesture sets would inform HCI researchers in the design of user-elicited interactions for such versatile light or electromagnetic field sensors and similar gesture-driven applications.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MR-MDEs: Exploring the Integration ..."
MR-MDEs: Exploring the Integration of Mixed Reality into Multi-display Environments
Griffin J. Hurt, Talha Khan, Nicolás Matheo Kass, Anthony Tang, Edward Andrews, and Jacob Biehl
(University of Pittsburgh, Pittsburgh, USA; Grand Valley State University, USA)
Multi-display environments (MDEs) are applicable to both everyday and specialized tasks like cooking, appliance repair, surgery, and more. In these settings, displays are often affixed in a manner that prevent reorientation, forcing users to split their attention between multiple visual information sources. Mixed reality (MR) has the potential to transform these spaces by presenting information through virtual interfaces that are not limited by physical constraints. While MR has been explored for single-task work, its role in multi-task, information-dense environments remains relatively unexplored. Our work bridges this gap by investigating the impact of different display modalities (large screens, tablets, and MR) on performance and perception in these environments. Our study's findings demonstrate the capability for MR to integrate into these spaces, extending traditional display technology with no impact to performance, cognitive load, or situational awareness. The study also further illustrates the nuanced relationship between performance and preference in tools used to guide task work. We provide insights toward the eventual authentic integration of MR in MDEs.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Detecting Group Configurations ..."
Detecting Group Configurations in Shared Spaces: A Heuristic for Understanding Space Use Behavior
Andrew Xu, Yuezhong Xu, Jacob Biehl, and Adam J. Lee
(University of Pittsburgh, Pittsburgh, USA; University of Illinois at Chicago, Chicago, USA)
The rapid development and deployment of “smart space” technology in commercial and educational buildings has brought efficiencies and comfort to users. Yet, the “smart” is largely limited to knowledge gained from occupancy and activity data. These limited data explain what is happening in a space but nothing about how effectively the space is being used. In this work, we explore how to understand effective space use and how it may impact future smart spaces and their design. As part of this investigation, we design generalizable heuristics derived from observable human behaviors and space usage to detect group configurations in public spaces. The resultant system models similar qualities of a space that end-users will judge and measure themselves before use. The effectiveness of our system is validated against a crowdsourced study. Our system serves as a foundation for applications building upon improving space use efficacy.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MR-MDEs: Exploring the Integration ..."
MR-MDEs: Exploring the Integration of Mixed Reality into Multi-display Environments
Griffin J. Hurt, Talha Khan, Nicolás Matheo Kass, Anthony Tang, Edward Andrews, and Jacob Biehl
(University of Pittsburgh, Pittsburgh, USA; Grand Valley State University, USA)
Multi-display environments (MDEs) are applicable to both everyday and specialized tasks like cooking, appliance repair, surgery, and more. In these settings, displays are often affixed in a manner that prevent reorientation, forcing users to split their attention between multiple visual information sources. Mixed reality (MR) has the potential to transform these spaces by presenting information through virtual interfaces that are not limited by physical constraints. While MR has been explored for single-task work, its role in multi-task, information-dense environments remains relatively unexplored. Our work bridges this gap by investigating the impact of different display modalities (large screens, tablets, and MR) on performance and perception in these environments. Our study's findings demonstrate the capability for MR to integrate into these spaces, extending traditional display technology with no impact to performance, cognitive load, or situational awareness. The study also further illustrates the nuanced relationship between performance and preference in tools used to guide task work. We provide insights toward the eventual authentic integration of MR in MDEs.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Discovering the Potential ..."
Discovering the Potential of Living Room Objects as Alternative Smart Home Controllers: An Exploration of Secondary Affordances
Michael Chamunorwa, Heiko Müller, and Susanne Boll
(University of Oldenburg, Oldenburg, Germany)
Smart home appliances are becoming more popular, yet their user interfaces (UI) often lack integration into living spaces and daily practices. To facilitate more natural interactions, reduce clutter and promote better integration in our living spaces, we propose embedding controls into everyday objects. This paper outlines our approach in three steps: identifying suitable household objects through ethnographic research, gathering interaction ideas via user gesture elicitation, and evaluating the intuitiveness of conceptual interfaces through a Wizard-of-Oz study. Our findings suggest that an object's primary function has less influence on its potential as a smart home controller. Instead, its location, physical characteristics, and intuitive affordances play a more significant role in shaping its potential. Designers should, therefore, focus on making these affordances easily discoverable by concurrently considering the object's physical properties, function, and interaction context.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "WindowSpace: A Web-Based XR ..."
WindowSpace: A Web-Based XR Window Manager for Interacting with 2D Windows in Immersive 3D Space
Emeric Marguet, Marcel Borowski, Janus Bager Kristensen, Clemens Nylandsted Klokmose, and Niklas Elmqvist
(Ecole Polytechnique, Palaiseau, France; Aarhus University, Aarhus, Denmark)
Extended Reality (XR) head-mounted displays (HMDs) coupled with standard mouse and keyboard hardware present a unique opportunity to rethink traditional desktop productivity. In this paper, we propose WindowSpace, an XR-based window manager for placing 2D windows in immersive 3D space while using a traditional mouse and keyboard. WindowSpace is a web-based system supporting consumer-level HMD-AR that can display native 2D windows from standard operating systems using a native desktop streaming server based on a standards-compliant optimized WebRTC protocol. To facilitate interacting with these native applications, we investigate the design space for effectively controlling the cursor between windows arranged in immersive 3D space. We propose three pointing techniques in our design space: an off-screen cursor (EtherXR), a head-tracking based pointer (Face&Jump), and a novel technique based on switching screens by extending the mouse trajectory (BallisticXR). Our empirical study revealed that EtherXR performs slightly better overall; however, BallisticXR showed better accuracy. Notably, Face&Jump yielded slower and less accurate performance, suggesting that full gaze assistance may be necessary to support this approach effectively.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Mirror Me: Exploring Avatar ..."
Mirror Me: Exploring Avatar Self-Views in Multi-user Virtual Reality
Katja Krug, Daniel Immanuel Fink, Mats Ole Ellenberg, Anke V. Reinschluessel, Wolfgang Büschel, Tiare Feuchtner, and Raimund Dachselt
(Dresden University of Technology, Dresden, Germany; University of Konstanz, Konstanz, Germany; University of Stuttgart, Stuttgart, Germany)
This paper investigates avatar self-views in virtual reality (VR) and their impact on multi-user communication scenarios. Self-views in video conferencing have been shown to impact our self-perception and mental load, so we explore whether similar effects occur in VR, as personal and professional gatherings progressively move to virtual spaces with 3D avatars. We identify the following key design dimensions for self-representations in VR: spatiality, anchoring and size, and self-visibility. Based on these, we designed three variants (Remote Perspective View, Personal Mirror, and Miniature Avatar), which we compare to a baseline (No Additional Self-View) in a user study. Our analysis of sixteen dyads playing a word-guessing game requiring verbal and non-verbal communication (i.e., explaining and charades) in VR confirms that self-views are beneficial for communication scenarios that require expressive body language or facial expressions, as this allows monitoring the own avatar's animations. Further, the Miniature Avatar was overall preferred due to its spatiality, world-anchoring, and full self-visibility. Based on our results, we offer design recommendations for self-views covering the key design dimensions in VR.
Publisher's Version
Video
Info
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Studying Visual Evidence from ..."
Studying Visual Evidence from Using Physical Space to Think
Maryam Rezaie, Samuel Huron, Parnian Taghipour, Lien Quach, Victor Cheung, and Sheelagh Carpendale
(Simon Fraser University, Burnaby, Canada; Télécom Paris - Institut Polytechnique de Paris, Palaiseau, France; Simon Fraser University, Surrey, Canada)
This paper investigates how people use physical space, free from digital constraints, to organize thoughts and ideas. Such spatial arrangements commonly support sensemaking, idea generation, and data understanding. We collected over 300 publicly shared images capturing these activities collected from real-world contexts. Our visual analysis identifies a small set of recurring spatial patterns such as clusters, grids as well as the techniques people use to build and adapt them, such as color coding, directional lines, and selective rule-following. We discuss how these practices can inform the design of more expressive and adaptable digital spatially-aware tools that better support the diverse ways people think with space.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Discovering the Potential ..."
Discovering the Potential of Living Room Objects as Alternative Smart Home Controllers: An Exploration of Secondary Affordances
Michael Chamunorwa, Heiko Müller, and Susanne Boll
(University of Oldenburg, Oldenburg, Germany)
Smart home appliances are becoming more popular, yet their user interfaces (UI) often lack integration into living spaces and daily practices. To facilitate more natural interactions, reduce clutter and promote better integration in our living spaces, we propose embedding controls into everyday objects. This paper outlines our approach in three steps: identifying suitable household objects through ethnographic research, gathering interaction ideas via user gesture elicitation, and evaluating the intuitiveness of conceptual interfaces through a Wizard-of-Oz study. Our findings suggest that an object's primary function has less influence on its potential as a smart home controller. Instead, its location, physical characteristics, and intuitive affordances play a more significant role in shaping its potential. Designers should, therefore, focus on making these affordances easily discoverable by concurrently considering the object's physical properties, function, and interaction context.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Investigating Design Considerations ..."
Investigating Design Considerations for Supporting Remote Teaching and Learning of Physical Drawing
Minghui Chen, Simo Hosio, and Koji Yatani
(University of Tokyo, Tokyo, Japan; University of Oulu, Oulu, Finland)
Teaching and learning physical drawing remotely pose user-experience challenges. In an online physical drawing course, instructors must interact with students through a camera for demonstrations, supervision, communication, and annotations in real-time. However, these activities are not well supported by commonly used video conferencing services. We conduct a qualitative study with seven instructors and five students to understand their practices and challenges in teaching and learning of physical drawing in a remote setting. Our study identifies four design considerations that a system supporting remote teaching and learning of physical drawing should take into account. We then demonstrate the technical feasibility of these design considerations by building a working prototype called SharedCanvas. We also validate our design considerations through a user study with SharedCanvas, revealing how they better supported interactions between instructors and students. We conclude by discussing our work in light of related studies on online drawing education and present future research opportunities to improve remote teaching of drawing.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Studying Visual Evidence from ..."
Studying Visual Evidence from Using Physical Space to Think
Maryam Rezaie, Samuel Huron, Parnian Taghipour, Lien Quach, Victor Cheung, and Sheelagh Carpendale
(Simon Fraser University, Burnaby, Canada; Télécom Paris - Institut Polytechnique de Paris, Palaiseau, France; Simon Fraser University, Surrey, Canada)
This paper investigates how people use physical space, free from digital constraints, to organize thoughts and ideas. Such spatial arrangements commonly support sensemaking, idea generation, and data understanding. We collected over 300 publicly shared images capturing these activities collected from real-world contexts. Our visual analysis identifies a small set of recurring spatial patterns such as clusters, grids as well as the techniques people use to build and adapt them, such as color coding, directional lines, and selective rule-following. We discuss how these practices can inform the design of more expressive and adaptable digital spatially-aware tools that better support the diverse ways people think with space.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Mirror Me: Exploring Avatar ..."
Mirror Me: Exploring Avatar Self-Views in Multi-user Virtual Reality
Katja Krug, Daniel Immanuel Fink, Mats Ole Ellenberg, Anke V. Reinschluessel, Wolfgang Büschel, Tiare Feuchtner, and Raimund Dachselt
(Dresden University of Technology, Dresden, Germany; University of Konstanz, Konstanz, Germany; University of Stuttgart, Stuttgart, Germany)
This paper investigates avatar self-views in virtual reality (VR) and their impact on multi-user communication scenarios. Self-views in video conferencing have been shown to impact our self-perception and mental load, so we explore whether similar effects occur in VR, as personal and professional gatherings progressively move to virtual spaces with 3D avatars. We identify the following key design dimensions for self-representations in VR: spatiality, anchoring and size, and self-visibility. Based on these, we designed three variants (Remote Perspective View, Personal Mirror, and Miniature Avatar), which we compare to a baseline (No Additional Self-View) in a user study. Our analysis of sixteen dyads playing a word-guessing game requiring verbal and non-verbal communication (i.e., explaining and charades) in VR confirms that self-views are beneficial for communication scenarios that require expressive body language or facial expressions, as this allows monitoring the own avatar's animations. Further, the Miniature Avatar was overall preferred due to its spatiality, world-anchoring, and full self-visibility. Based on our results, we offer design recommendations for self-views covering the key design dimensions in VR.
Publisher's Version
Video
Info
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "CorrectMe: An Interactive ..."
CorrectMe: An Interactive Framework for Human-in-the-Loop Correction and Explanation of Object Detection Models
Yinuo Liu, Zhiyuan Wu, Xiaoju Dong, Shaoxiong Jiang, and Weijie Li
(Shanghai Jiao Tong University, Shanghai, China)
Object detection models, while achieving greater performance, often suffer from recurring errors such as misclassifications or missed detections. Existing explainable AI (XAI) tools primarily offer static, observation-based explanations and rarely support interactive correction or retraining, especially for non-expert users. To bridge this gap, we introduce CorrectMe, an interactive framework that integrates human-in-the-loop correction and explanation into object detection workflows. CorrectMe empowers users to iteratively explore, interpret, and rectify model errors through a unified interface featuring semantic embedding visualizations, saliency-based explanations, and natural language rationales. Users can revise predictions and incrementally retrain the model, streamlining the refinement process through lightweight updates rather than full-scale retraining or annotation. Through application scenarios and user studies, we demonstrate that CorrectMe enables more strategic corrections, improves model understanding, and lowers the barrier to practical refinement of object detection models.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Mirror Me: Exploring Avatar ..."
Mirror Me: Exploring Avatar Self-Views in Multi-user Virtual Reality
Katja Krug, Daniel Immanuel Fink, Mats Ole Ellenberg, Anke V. Reinschluessel, Wolfgang Büschel, Tiare Feuchtner, and Raimund Dachselt
(Dresden University of Technology, Dresden, Germany; University of Konstanz, Konstanz, Germany; University of Stuttgart, Stuttgart, Germany)
This paper investigates avatar self-views in virtual reality (VR) and their impact on multi-user communication scenarios. Self-views in video conferencing have been shown to impact our self-perception and mental load, so we explore whether similar effects occur in VR, as personal and professional gatherings progressively move to virtual spaces with 3D avatars. We identify the following key design dimensions for self-representations in VR: spatiality, anchoring and size, and self-visibility. Based on these, we designed three variants (Remote Perspective View, Personal Mirror, and Miniature Avatar), which we compare to a baseline (No Additional Self-View) in a user study. Our analysis of sixteen dyads playing a word-guessing game requiring verbal and non-verbal communication (i.e., explaining and charades) in VR confirms that self-views are beneficial for communication scenarios that require expressive body language or facial expressions, as this allows monitoring the own avatar's animations. Further, the Miniature Avatar was overall preferred due to its spatiality, world-anchoring, and full self-visibility. Based on our results, we offer design recommendations for self-views covering the key design dimensions in VR.
Publisher's Version
Video
Info
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "WindowSpace: A Web-Based XR ..."
WindowSpace: A Web-Based XR Window Manager for Interacting with 2D Windows in Immersive 3D Space
Emeric Marguet, Marcel Borowski, Janus Bager Kristensen, Clemens Nylandsted Klokmose, and Niklas Elmqvist
(Ecole Polytechnique, Palaiseau, France; Aarhus University, Aarhus, Denmark)
Extended Reality (XR) head-mounted displays (HMDs) coupled with standard mouse and keyboard hardware present a unique opportunity to rethink traditional desktop productivity. In this paper, we propose WindowSpace, an XR-based window manager for placing 2D windows in immersive 3D space while using a traditional mouse and keyboard. WindowSpace is a web-based system supporting consumer-level HMD-AR that can display native 2D windows from standard operating systems using a native desktop streaming server based on a standards-compliant optimized WebRTC protocol. To facilitate interacting with these native applications, we investigate the design space for effectively controlling the cursor between windows arranged in immersive 3D space. We propose three pointing techniques in our design space: an off-screen cursor (EtherXR), a head-tracking based pointer (Face&Jump), and a novel technique based on switching screens by extending the mouse trajectory (BallisticXR). Our empirical study revealed that EtherXR performs slightly better overall; however, BallisticXR showed better accuracy. Notably, Face&Jump yielded slower and less accurate performance, suggesting that full gaze assistance may be necessary to support this approach effectively.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "GazeMatch: Drawing on the ..."
GazeMatch: Drawing on the Benefits of Motion Matching Gaze Interfaces without Extensive Refactoring of Existing Public Display UIs
Diogo Lopes and Augusto Esteves
(Instituto Superior Técnico - University of Lisbon, Lisbon, Portugal; Interactive Technologies Institute - LARSyS, Lisbon, Portugal)
Touch controls on public displays, while ubiquitous, introduce public health and accessibility concerns. Alternative controls based on gaze or mid-air interaction techniques usually require a calibration process that hinders spontaneous interaction. Calibration-free gaze alternatives tend to rely on user interfaces (UIs) with moving elements, which require substantial changes to existing UIs, can hinder touch input, and contribute to an overwhelming and confusing user experience -- ultimately limiting adoption. This paper introduces GazeMatch, a novel interaction technique that combines techniques such as Pursuits and Actigaze to address these constraints. We present three user studies that help us define the technique's parameters (N=8), and compare it to state-of-the-art gaze techniques in an abstract selection task (N=12) and in a UI representative of the metro kiosks in New York City (N=5). With little practice, participants committed more errors with GazeMatch than with Google's Look to Speak in the abstract task (19.25% and 4.33%, respectively), but were considerably faster (3.66s and 7.58s, respectively) and ultimately reported preferring GazeMatch. In the representative task, and with no onboarding, participants reported higher discoverability for Pursuits but higher performance for GazeMatch.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MobileMuseum: Smart, Portable, ..."
MobileMuseum: Smart, Portable, and Borrowable Museum Displays to Explore Interaction Patterns and Public Engagement
Alaa Nousir, Lee Jones, Flora Lin, Tom Everrett, and Sara Nabil
(Queen’s University, Kingston, Canada; Ingenium - Canada’s Museums of Science and Innovation, Ottawa, Canada)
We explore how smart, portable, tangible, and borrowable museum displays can be used to expand museum outreach, and provide tools for measuring user engagement. Our prototype, MobileMuseum, is a portable museum display on wheels, housing digitally-fabricated replicas of artifacts from a national science and technology museum. To track interactions without cameras, we embedded photo and magnetic door sensors and RFID tags and antenna. Our deployment study was carried out for 2 months and included 4 different locations (2 recreational and 2 educational). Data collected from 6 sensors, 261 hours of field observations, 17 filled questionnaires, and 14 interviews helped us understand interaction patterns. Qualitative and quantitative findings revealed that self-monitored interactions encouraged deeper open-ended exploration. Social sharing increased among people in groups while lower foot traffic increased engagement duration. We present generalizable opportunities for self-monitored Interfaces that lends themselves to social circulation, ‘honeybee-effect’, ‘order effect’, and ‘leftovers’ from users.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Mirror Me: Exploring Avatar ..."
Mirror Me: Exploring Avatar Self-Views in Multi-user Virtual Reality
Katja Krug, Daniel Immanuel Fink, Mats Ole Ellenberg, Anke V. Reinschluessel, Wolfgang Büschel, Tiare Feuchtner, and Raimund Dachselt
(Dresden University of Technology, Dresden, Germany; University of Konstanz, Konstanz, Germany; University of Stuttgart, Stuttgart, Germany)
This paper investigates avatar self-views in virtual reality (VR) and their impact on multi-user communication scenarios. Self-views in video conferencing have been shown to impact our self-perception and mental load, so we explore whether similar effects occur in VR, as personal and professional gatherings progressively move to virtual spaces with 3D avatars. We identify the following key design dimensions for self-representations in VR: spatiality, anchoring and size, and self-visibility. Based on these, we designed three variants (Remote Perspective View, Personal Mirror, and Miniature Avatar), which we compare to a baseline (No Additional Self-View) in a user study. Our analysis of sixteen dyads playing a word-guessing game requiring verbal and non-verbal communication (i.e., explaining and charades) in VR confirms that self-views are beneficial for communication scenarios that require expressive body language or facial expressions, as this allows monitoring the own avatar's animations. Further, the Miniature Avatar was overall preferred due to its spatiality, world-anchoring, and full self-visibility. Based on our results, we offer design recommendations for self-views covering the key design dimensions in VR.
Publisher's Version
Video
Info
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Mirror Me: Exploring Avatar ..."
Mirror Me: Exploring Avatar Self-Views in Multi-user Virtual Reality
Katja Krug, Daniel Immanuel Fink, Mats Ole Ellenberg, Anke V. Reinschluessel, Wolfgang Büschel, Tiare Feuchtner, and Raimund Dachselt
(Dresden University of Technology, Dresden, Germany; University of Konstanz, Konstanz, Germany; University of Stuttgart, Stuttgart, Germany)
This paper investigates avatar self-views in virtual reality (VR) and their impact on multi-user communication scenarios. Self-views in video conferencing have been shown to impact our self-perception and mental load, so we explore whether similar effects occur in VR, as personal and professional gatherings progressively move to virtual spaces with 3D avatars. We identify the following key design dimensions for self-representations in VR: spatiality, anchoring and size, and self-visibility. Based on these, we designed three variants (Remote Perspective View, Personal Mirror, and Miniature Avatar), which we compare to a baseline (No Additional Self-View) in a user study. Our analysis of sixteen dyads playing a word-guessing game requiring verbal and non-verbal communication (i.e., explaining and charades) in VR confirms that self-views are beneficial for communication scenarios that require expressive body language or facial expressions, as this allows monitoring the own avatar's animations. Further, the Miniature Avatar was overall preferred due to its spatiality, world-anchoring, and full self-visibility. Based on our results, we offer design recommendations for self-views covering the key design dimensions in VR.
Publisher's Version
Video
Info
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "StringWall: A Low-Budget, ..."
StringWall: A Low-Budget, Projection-Based Tangible User Interface
Jana Hoffard, Shio Miyafuji, and Hideki Koike
(Institute of Science, Tokyo, Japan)
Efficient space division is increasingly important as space is decreasing for both apartments and offices, where the average person spends most of their time. Therefore, it is essential to explore additional ways to separate and cover space without preventing its accessibility, while also providing possibilities for personalization by the individual. To achieve this, we present StringWall, a tangible, projection-based user interface created from low-cost materials that can be integrated into any space or size. It consists of rubber strings tracked by a depth camera that can be adapted to the user's needs in terms of size and orientation. To evaluate our system, we conducted visibility tests for the projection and background under different conditions. Furthermore, we conducted two preliminary studies to explore the users' preferences in terms of different interaction methods and use cases, as well as the haptic feedback capabilities of our system. The results show that users found the system and proposed use cases engaging, and that the system is suitable for providing extended haptic feedback when physical restraints align with the user's expectations. We further provide a demonstration of two versions of a shelf prototype incorporating the StringWall and its interactions.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Investigating Design Considerations ..."
Investigating Design Considerations for Supporting Remote Teaching and Learning of Physical Drawing
Minghui Chen, Simo Hosio, and Koji Yatani
(University of Tokyo, Tokyo, Japan; University of Oulu, Oulu, Finland)
Teaching and learning physical drawing remotely pose user-experience challenges. In an online physical drawing course, instructors must interact with students through a camera for demonstrations, supervision, communication, and annotations in real-time. However, these activities are not well supported by commonly used video conferencing services. We conduct a qualitative study with seven instructors and five students to understand their practices and challenges in teaching and learning of physical drawing in a remote setting. Our study identifies four design considerations that a system supporting remote teaching and learning of physical drawing should take into account. We then demonstrate the technical feasibility of these design considerations by building a working prototype called SharedCanvas. We also validate our design considerations through a user study with SharedCanvas, revealing how they better supported interactions between instructors and students. We conclude by discussing our work in light of related studies on online drawing education and present future research opportunities to improve remote teaching of drawing.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Studying Visual Evidence from ..."
Studying Visual Evidence from Using Physical Space to Think
Maryam Rezaie, Samuel Huron, Parnian Taghipour, Lien Quach, Victor Cheung, and Sheelagh Carpendale
(Simon Fraser University, Burnaby, Canada; Télécom Paris - Institut Polytechnique de Paris, Palaiseau, France; Simon Fraser University, Surrey, Canada)
This paper investigates how people use physical space, free from digital constraints, to organize thoughts and ideas. Such spatial arrangements commonly support sensemaking, idea generation, and data understanding. We collected over 300 publicly shared images capturing these activities collected from real-world contexts. Our visual analysis identifies a small set of recurring spatial patterns such as clusters, grids as well as the techniques people use to build and adapt them, such as color coding, directional lines, and selective rule-following. We discuss how these practices can inform the design of more expressive and adaptable digital spatially-aware tools that better support the diverse ways people think with space.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MR-MDEs: Exploring the Integration ..."
MR-MDEs: Exploring the Integration of Mixed Reality into Multi-display Environments
Griffin J. Hurt, Talha Khan, Nicolás Matheo Kass, Anthony Tang, Edward Andrews, and Jacob Biehl
(University of Pittsburgh, Pittsburgh, USA; Grand Valley State University, USA)
Multi-display environments (MDEs) are applicable to both everyday and specialized tasks like cooking, appliance repair, surgery, and more. In these settings, displays are often affixed in a manner that prevent reorientation, forcing users to split their attention between multiple visual information sources. Mixed reality (MR) has the potential to transform these spaces by presenting information through virtual interfaces that are not limited by physical constraints. While MR has been explored for single-task work, its role in multi-task, information-dense environments remains relatively unexplored. Our work bridges this gap by investigating the impact of different display modalities (large screens, tablets, and MR) on performance and perception in these environments. Our study's findings demonstrate the capability for MR to integrate into these spaces, extending traditional display technology with no impact to performance, cognitive load, or situational awareness. The study also further illustrates the nuanced relationship between performance and preference in tools used to guide task work. We provide insights toward the eventual authentic integration of MR in MDEs.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "CorrectMe: An Interactive ..."
CorrectMe: An Interactive Framework for Human-in-the-Loop Correction and Explanation of Object Detection Models
Yinuo Liu, Zhiyuan Wu, Xiaoju Dong, Shaoxiong Jiang, and Weijie Li
(Shanghai Jiao Tong University, Shanghai, China)
Object detection models, while achieving greater performance, often suffer from recurring errors such as misclassifications or missed detections. Existing explainable AI (XAI) tools primarily offer static, observation-based explanations and rarely support interactive correction or retraining, especially for non-expert users. To bridge this gap, we introduce CorrectMe, an interactive framework that integrates human-in-the-loop correction and explanation into object detection workflows. CorrectMe empowers users to iteratively explore, interpret, and rectify model errors through a unified interface featuring semantic embedding visualizations, saliency-based explanations, and natural language rationales. Users can revise predictions and incrementally retrain the model, streamlining the refinement process through lightweight updates rather than full-scale retraining or annotation. Through application scenarios and user studies, we demonstrate that CorrectMe enables more strategic corrections, improves model understanding, and lowers the barrier to practical refinement of object detection models.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "EEG-Based Monitoring of Pilot ..."
EEG-Based Monitoring of Pilot Training: Transferring Full-Cap Representations to Headphone-Style Electrode Positions
Linyu Zheng, Yujing Mark Jiang, Zhuo Liu, and Weidong Yang
(Fudan University, Shanghai, China; AVIC General Huanan Aircraft Industry, Zhuhai, China)
Monitoring pilots’ mental states during training is important for ensuring safety and optimizing performance, yet existing full-cap EEG systems are impractical for operational use. In this work, we evaluate a headphone-style 9-channel electrode montage rather than a complete headphone EEG device, and propose a transfer learning framework to preserve performance under sparse coverage. Our two-stage approach first applies self-supervised pretraining on 64-channel full-cap EEG, and then adapts the learned representations to the headphone-style montage through a position correction module that accounts for electrode misplacement. We validate the framework in a flight simulator with 12 participants on two tasks: motor intention (left vs. right) and cognitive workload (low vs. high). Despite limited coverage, the headphone-style montage achieves within-subject accuracies of 85% for motor imagery and 89% for workload, compared to full-cap performance of 88% and 91%. Cross-subject accuracies reached 66% and 73%, demonstrating generalizability across users. Ablation analyses show that both self-supervised pretraining and position correction independently improve performance, and together provide a 12% boost in cross-subject decoding accuracy. These findings highlight the feasibility of headphone-style EEG montages for practical pilot monitoring, while clarifying methodological limitations and the need for future real-time and user-centered evaluation.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MobileMuseum: Smart, Portable, ..."
MobileMuseum: Smart, Portable, and Borrowable Museum Displays to Explore Interaction Patterns and Public Engagement
Alaa Nousir, Lee Jones, Flora Lin, Tom Everrett, and Sara Nabil
(Queen’s University, Kingston, Canada; Ingenium - Canada’s Museums of Science and Innovation, Ottawa, Canada)
We explore how smart, portable, tangible, and borrowable museum displays can be used to expand museum outreach, and provide tools for measuring user engagement. Our prototype, MobileMuseum, is a portable museum display on wheels, housing digitally-fabricated replicas of artifacts from a national science and technology museum. To track interactions without cameras, we embedded photo and magnetic door sensors and RFID tags and antenna. Our deployment study was carried out for 2 months and included 4 different locations (2 recreational and 2 educational). Data collected from 6 sensors, 261 hours of field observations, 17 filled questionnaires, and 14 interviews helped us understand interaction patterns. Qualitative and quantitative findings revealed that self-monitored interactions encouraged deeper open-ended exploration. Social sharing increased among people in groups while lower foot traffic increased engagement duration. We present generalizable opportunities for self-monitored Interfaces that lends themselves to social circulation, ‘honeybee-effect’, ‘order effect’, and ‘leftovers’ from users.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MR-MDEs: Exploring the Integration ..."
MR-MDEs: Exploring the Integration of Mixed Reality into Multi-display Environments
Griffin J. Hurt, Talha Khan, Nicolás Matheo Kass, Anthony Tang, Edward Andrews, and Jacob Biehl
(University of Pittsburgh, Pittsburgh, USA; Grand Valley State University, USA)
Multi-display environments (MDEs) are applicable to both everyday and specialized tasks like cooking, appliance repair, surgery, and more. In these settings, displays are often affixed in a manner that prevent reorientation, forcing users to split their attention between multiple visual information sources. Mixed reality (MR) has the potential to transform these spaces by presenting information through virtual interfaces that are not limited by physical constraints. While MR has been explored for single-task work, its role in multi-task, information-dense environments remains relatively unexplored. Our work bridges this gap by investigating the impact of different display modalities (large screens, tablets, and MR) on performance and perception in these environments. Our study's findings demonstrate the capability for MR to integrate into these spaces, extending traditional display technology with no impact to performance, cognitive load, or situational awareness. The study also further illustrates the nuanced relationship between performance and preference in tools used to guide task work. We provide insights toward the eventual authentic integration of MR in MDEs.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MR-MDEs: Exploring the Integration ..."
MR-MDEs: Exploring the Integration of Mixed Reality into Multi-display Environments
Griffin J. Hurt, Talha Khan, Nicolás Matheo Kass, Anthony Tang, Edward Andrews, and Jacob Biehl
(University of Pittsburgh, Pittsburgh, USA; Grand Valley State University, USA)
Multi-display environments (MDEs) are applicable to both everyday and specialized tasks like cooking, appliance repair, surgery, and more. In these settings, displays are often affixed in a manner that prevent reorientation, forcing users to split their attention between multiple visual information sources. Mixed reality (MR) has the potential to transform these spaces by presenting information through virtual interfaces that are not limited by physical constraints. While MR has been explored for single-task work, its role in multi-task, information-dense environments remains relatively unexplored. Our work bridges this gap by investigating the impact of different display modalities (large screens, tablets, and MR) on performance and perception in these environments. Our study's findings demonstrate the capability for MR to integrate into these spaces, extending traditional display technology with no impact to performance, cognitive load, or situational awareness. The study also further illustrates the nuanced relationship between performance and preference in tools used to guide task work. We provide insights toward the eventual authentic integration of MR in MDEs.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "WindowSpace: A Web-Based XR ..."
WindowSpace: A Web-Based XR Window Manager for Interacting with 2D Windows in Immersive 3D Space
Emeric Marguet, Marcel Borowski, Janus Bager Kristensen, Clemens Nylandsted Klokmose, and Niklas Elmqvist
(Ecole Polytechnique, Palaiseau, France; Aarhus University, Aarhus, Denmark)
Extended Reality (XR) head-mounted displays (HMDs) coupled with standard mouse and keyboard hardware present a unique opportunity to rethink traditional desktop productivity. In this paper, we propose WindowSpace, an XR-based window manager for placing 2D windows in immersive 3D space while using a traditional mouse and keyboard. WindowSpace is a web-based system supporting consumer-level HMD-AR that can display native 2D windows from standard operating systems using a native desktop streaming server based on a standards-compliant optimized WebRTC protocol. To facilitate interacting with these native applications, we investigate the design space for effectively controlling the cursor between windows arranged in immersive 3D space. We propose three pointing techniques in our design space: an off-screen cursor (EtherXR), a head-tracking based pointer (Face&Jump), and a novel technique based on switching screens by extending the mouse trajectory (BallisticXR). Our empirical study revealed that EtherXR performs slightly better overall; however, BallisticXR showed better accuracy. Notably, Face&Jump yielded slower and less accurate performance, suggesting that full gaze assistance may be necessary to support this approach effectively.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "StringWall: A Low-Budget, ..."
StringWall: A Low-Budget, Projection-Based Tangible User Interface
Jana Hoffard, Shio Miyafuji, and Hideki Koike
(Institute of Science, Tokyo, Japan)
Efficient space division is increasingly important as space is decreasing for both apartments and offices, where the average person spends most of their time. Therefore, it is essential to explore additional ways to separate and cover space without preventing its accessibility, while also providing possibilities for personalization by the individual. To achieve this, we present StringWall, a tangible, projection-based user interface created from low-cost materials that can be integrated into any space or size. It consists of rubber strings tracked by a depth camera that can be adapted to the user's needs in terms of size and orientation. To evaluate our system, we conducted visibility tests for the projection and background under different conditions. Furthermore, we conducted two preliminary studies to explore the users' preferences in terms of different interaction methods and use cases, as well as the haptic feedback capabilities of our system. The results show that users found the system and proposed use cases engaging, and that the system is suitable for providing extended haptic feedback when physical restraints align with the user's expectations. We further provide a demonstration of two versions of a shelf prototype incorporating the StringWall and its interactions.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "WindowSpace: A Web-Based XR ..."
WindowSpace: A Web-Based XR Window Manager for Interacting with 2D Windows in Immersive 3D Space
Emeric Marguet, Marcel Borowski, Janus Bager Kristensen, Clemens Nylandsted Klokmose, and Niklas Elmqvist
(Ecole Polytechnique, Palaiseau, France; Aarhus University, Aarhus, Denmark)
Extended Reality (XR) head-mounted displays (HMDs) coupled with standard mouse and keyboard hardware present a unique opportunity to rethink traditional desktop productivity. In this paper, we propose WindowSpace, an XR-based window manager for placing 2D windows in immersive 3D space while using a traditional mouse and keyboard. WindowSpace is a web-based system supporting consumer-level HMD-AR that can display native 2D windows from standard operating systems using a native desktop streaming server based on a standards-compliant optimized WebRTC protocol. To facilitate interacting with these native applications, we investigate the design space for effectively controlling the cursor between windows arranged in immersive 3D space. We propose three pointing techniques in our design space: an off-screen cursor (EtherXR), a head-tracking based pointer (Face&Jump), and a novel technique based on switching screens by extending the mouse trajectory (BallisticXR). Our empirical study revealed that EtherXR performs slightly better overall; however, BallisticXR showed better accuracy. Notably, Face&Jump yielded slower and less accurate performance, suggesting that full gaze assistance may be necessary to support this approach effectively.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Mirror Me: Exploring Avatar ..."
Mirror Me: Exploring Avatar Self-Views in Multi-user Virtual Reality
Katja Krug, Daniel Immanuel Fink, Mats Ole Ellenberg, Anke V. Reinschluessel, Wolfgang Büschel, Tiare Feuchtner, and Raimund Dachselt
(Dresden University of Technology, Dresden, Germany; University of Konstanz, Konstanz, Germany; University of Stuttgart, Stuttgart, Germany)
This paper investigates avatar self-views in virtual reality (VR) and their impact on multi-user communication scenarios. Self-views in video conferencing have been shown to impact our self-perception and mental load, so we explore whether similar effects occur in VR, as personal and professional gatherings progressively move to virtual spaces with 3D avatars. We identify the following key design dimensions for self-representations in VR: spatiality, anchoring and size, and self-visibility. Based on these, we designed three variants (Remote Perspective View, Personal Mirror, and Miniature Avatar), which we compare to a baseline (No Additional Self-View) in a user study. Our analysis of sixteen dyads playing a word-guessing game requiring verbal and non-verbal communication (i.e., explaining and charades) in VR confirms that self-views are beneficial for communication scenarios that require expressive body language or facial expressions, as this allows monitoring the own avatar's animations. Further, the Miniature Avatar was overall preferred due to its spatiality, world-anchoring, and full self-visibility. Based on our results, we offer design recommendations for self-views covering the key design dimensions in VR.
Publisher's Version
Video
Info
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Detecting Group Configurations ..."
Detecting Group Configurations in Shared Spaces: A Heuristic for Understanding Space Use Behavior
Andrew Xu, Yuezhong Xu, Jacob Biehl, and Adam J. Lee
(University of Pittsburgh, Pittsburgh, USA; University of Illinois at Chicago, Chicago, USA)
The rapid development and deployment of “smart space” technology in commercial and educational buildings has brought efficiencies and comfort to users. Yet, the “smart” is largely limited to knowledge gained from occupancy and activity data. These limited data explain what is happening in a space but nothing about how effectively the space is being used. In this work, we explore how to understand effective space use and how it may impact future smart spaces and their design. As part of this investigation, we design generalizable heuristics derived from observable human behaviors and space usage to detect group configurations in public spaces. The resultant system models similar qualities of a space that end-users will judge and measure themselves before use. The effectiveness of our system is validated against a crowdsourced study. Our system serves as a foundation for applications building upon improving space use efficacy.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "AutoPark: Adaptive Friction ..."
AutoPark: Adaptive Friction Mouse Facilitating Stable Multi-touch Gesture Input
Yohan Yun, Jisu Yim, and Geehyuk Lee
(KAIST, Daejeon, Republic of Korea)
Multi-touch mice have been proposed to integrate the rich input vocabulary of multi-touch gestures into traditional computer mice. However, incorporating multi-finger gestures into a mouse introduces a design conflict: Mice are optimized for low friction to support smooth cursor navigation, whereas touch gestures require high friction for stability. In this paper, we present AutoPark—the concept of a multi-touch mouse that dynamically adjusts its friction based on user intent. AutoPark mouse operates in a low-friction mode for cursor movement and shifts to a high-friction state for multi-touch interactions. Our first user study measured the shear forces during touch gestures, confirming that conventional friction levels of mousepads are insufficient to maintain gesture input stability on a mouse. We developed an AutoPark prototype and conducted controlled experiments, comparing it with both conventional low-friction and gesture-favored high-friction setups. The results demonstrate that AutoPark enhances gesture stability while maintaining comparable pointing performance in our study tasks, supporting a richer gesture input vocabulary within mouse-based GUI interactions.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "CorrectMe: An Interactive ..."
CorrectMe: An Interactive Framework for Human-in-the-Loop Correction and Explanation of Object Detection Models
Yinuo Liu, Zhiyuan Wu, Xiaoju Dong, Shaoxiong Jiang, and Weijie Li
(Shanghai Jiao Tong University, Shanghai, China)
Object detection models, while achieving greater performance, often suffer from recurring errors such as misclassifications or missed detections. Existing explainable AI (XAI) tools primarily offer static, observation-based explanations and rarely support interactive correction or retraining, especially for non-expert users. To bridge this gap, we introduce CorrectMe, an interactive framework that integrates human-in-the-loop correction and explanation into object detection workflows. CorrectMe empowers users to iteratively explore, interpret, and rectify model errors through a unified interface featuring semantic embedding visualizations, saliency-based explanations, and natural language rationales. Users can revise predictions and incrementally retrain the model, streamlining the refinement process through lightweight updates rather than full-scale retraining or annotation. Through application scenarios and user studies, we demonstrate that CorrectMe enables more strategic corrections, improves model understanding, and lowers the barrier to practical refinement of object detection models.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MobileMuseum: Smart, Portable, ..."
MobileMuseum: Smart, Portable, and Borrowable Museum Displays to Explore Interaction Patterns and Public Engagement
Alaa Nousir, Lee Jones, Flora Lin, Tom Everrett, and Sara Nabil
(Queen’s University, Kingston, Canada; Ingenium - Canada’s Museums of Science and Innovation, Ottawa, Canada)
We explore how smart, portable, tangible, and borrowable museum displays can be used to expand museum outreach, and provide tools for measuring user engagement. Our prototype, MobileMuseum, is a portable museum display on wheels, housing digitally-fabricated replicas of artifacts from a national science and technology museum. To track interactions without cameras, we embedded photo and magnetic door sensors and RFID tags and antenna. Our deployment study was carried out for 2 months and included 4 different locations (2 recreational and 2 educational). Data collected from 6 sensors, 261 hours of field observations, 17 filled questionnaires, and 14 interviews helped us understand interaction patterns. Qualitative and quantitative findings revealed that self-monitored interactions encouraged deeper open-ended exploration. Social sharing increased among people in groups while lower foot traffic increased engagement duration. We present generalizable opportunities for self-monitored Interfaces that lends themselves to social circulation, ‘honeybee-effect’, ‘order effect’, and ‘leftovers’ from users.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "CorrectMe: An Interactive ..."
CorrectMe: An Interactive Framework for Human-in-the-Loop Correction and Explanation of Object Detection Models
Yinuo Liu, Zhiyuan Wu, Xiaoju Dong, Shaoxiong Jiang, and Weijie Li
(Shanghai Jiao Tong University, Shanghai, China)
Object detection models, while achieving greater performance, often suffer from recurring errors such as misclassifications or missed detections. Existing explainable AI (XAI) tools primarily offer static, observation-based explanations and rarely support interactive correction or retraining, especially for non-expert users. To bridge this gap, we introduce CorrectMe, an interactive framework that integrates human-in-the-loop correction and explanation into object detection workflows. CorrectMe empowers users to iteratively explore, interpret, and rectify model errors through a unified interface featuring semantic embedding visualizations, saliency-based explanations, and natural language rationales. Users can revise predictions and incrementally retrain the model, streamlining the refinement process through lightweight updates rather than full-scale retraining or annotation. Through application scenarios and user studies, we demonstrate that CorrectMe enables more strategic corrections, improves model understanding, and lowers the barrier to practical refinement of object detection models.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "EEG-Based Monitoring of Pilot ..."
EEG-Based Monitoring of Pilot Training: Transferring Full-Cap Representations to Headphone-Style Electrode Positions
Linyu Zheng, Yujing Mark Jiang, Zhuo Liu, and Weidong Yang
(Fudan University, Shanghai, China; AVIC General Huanan Aircraft Industry, Zhuhai, China)
Monitoring pilots’ mental states during training is important for ensuring safety and optimizing performance, yet existing full-cap EEG systems are impractical for operational use. In this work, we evaluate a headphone-style 9-channel electrode montage rather than a complete headphone EEG device, and propose a transfer learning framework to preserve performance under sparse coverage. Our two-stage approach first applies self-supervised pretraining on 64-channel full-cap EEG, and then adapts the learned representations to the headphone-style montage through a position correction module that accounts for electrode misplacement. We validate the framework in a flight simulator with 12 participants on two tasks: motor intention (left vs. right) and cognitive workload (low vs. high). Despite limited coverage, the headphone-style montage achieves within-subject accuracies of 85% for motor imagery and 89% for workload, compared to full-cap performance of 88% and 91%. Cross-subject accuracies reached 66% and 73%, demonstrating generalizability across users. Ablation analyses show that both self-supervised pretraining and position correction independently improve performance, and together provide a 12% boost in cross-subject decoding accuracy. These findings highlight the feasibility of headphone-style EEG montages for practical pilot monitoring, while clarifying methodological limitations and the need for future real-time and user-centered evaluation.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "GazeMatch: Drawing on the ..."
GazeMatch: Drawing on the Benefits of Motion Matching Gaze Interfaces without Extensive Refactoring of Existing Public Display UIs
Diogo Lopes and Augusto Esteves
(Instituto Superior Técnico - University of Lisbon, Lisbon, Portugal; Interactive Technologies Institute - LARSyS, Lisbon, Portugal)
Touch controls on public displays, while ubiquitous, introduce public health and accessibility concerns. Alternative controls based on gaze or mid-air interaction techniques usually require a calibration process that hinders spontaneous interaction. Calibration-free gaze alternatives tend to rely on user interfaces (UIs) with moving elements, which require substantial changes to existing UIs, can hinder touch input, and contribute to an overwhelming and confusing user experience -- ultimately limiting adoption. This paper introduces GazeMatch, a novel interaction technique that combines techniques such as Pursuits and Actigaze to address these constraints. We present three user studies that help us define the technique's parameters (N=8), and compare it to state-of-the-art gaze techniques in an abstract selection task (N=12) and in a UI representative of the metro kiosks in New York City (N=5). With little practice, participants committed more errors with GazeMatch than with Google's Look to Speak in the abstract task (19.25% and 4.33%, respectively), but were considerably faster (3.66s and 7.58s, respectively) and ultimately reported preferring GazeMatch. In the representative task, and with no onboarding, participants reported higher discoverability for Pursuits but higher performance for GazeMatch.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "WindowSpace: A Web-Based XR ..."
WindowSpace: A Web-Based XR Window Manager for Interacting with 2D Windows in Immersive 3D Space
Emeric Marguet, Marcel Borowski, Janus Bager Kristensen, Clemens Nylandsted Klokmose, and Niklas Elmqvist
(Ecole Polytechnique, Palaiseau, France; Aarhus University, Aarhus, Denmark)
Extended Reality (XR) head-mounted displays (HMDs) coupled with standard mouse and keyboard hardware present a unique opportunity to rethink traditional desktop productivity. In this paper, we propose WindowSpace, an XR-based window manager for placing 2D windows in immersive 3D space while using a traditional mouse and keyboard. WindowSpace is a web-based system supporting consumer-level HMD-AR that can display native 2D windows from standard operating systems using a native desktop streaming server based on a standards-compliant optimized WebRTC protocol. To facilitate interacting with these native applications, we investigate the design space for effectively controlling the cursor between windows arranged in immersive 3D space. We propose three pointing techniques in our design space: an off-screen cursor (EtherXR), a head-tracking based pointer (Face&Jump), and a novel technique based on switching screens by extending the mouse trajectory (BallisticXR). Our empirical study revealed that EtherXR performs slightly better overall; however, BallisticXR showed better accuracy. Notably, Face&Jump yielded slower and less accurate performance, suggesting that full gaze assistance may be necessary to support this approach effectively.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "StringWall: A Low-Budget, ..."
StringWall: A Low-Budget, Projection-Based Tangible User Interface
Jana Hoffard, Shio Miyafuji, and Hideki Koike
(Institute of Science, Tokyo, Japan)
Efficient space division is increasingly important as space is decreasing for both apartments and offices, where the average person spends most of their time. Therefore, it is essential to explore additional ways to separate and cover space without preventing its accessibility, while also providing possibilities for personalization by the individual. To achieve this, we present StringWall, a tangible, projection-based user interface created from low-cost materials that can be integrated into any space or size. It consists of rubber strings tracked by a depth camera that can be adapted to the user's needs in terms of size and orientation. To evaluate our system, we conducted visibility tests for the projection and background under different conditions. Furthermore, we conducted two preliminary studies to explore the users' preferences in terms of different interaction methods and use cases, as well as the haptic feedback capabilities of our system. The results show that users found the system and proposed use cases engaging, and that the system is suitable for providing extended haptic feedback when physical restraints align with the user's expectations. We further provide a demonstration of two versions of a shelf prototype incorporating the StringWall and its interactions.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Discovering the Potential ..."
Discovering the Potential of Living Room Objects as Alternative Smart Home Controllers: An Exploration of Secondary Affordances
Michael Chamunorwa, Heiko Müller, and Susanne Boll
(University of Oldenburg, Oldenburg, Germany)
Smart home appliances are becoming more popular, yet their user interfaces (UI) often lack integration into living spaces and daily practices. To facilitate more natural interactions, reduce clutter and promote better integration in our living spaces, we propose embedding controls into everyday objects. This paper outlines our approach in three steps: identifying suitable household objects through ethnographic research, gathering interaction ideas via user gesture elicitation, and evaluating the intuitiveness of conceptual interfaces through a Wizard-of-Oz study. Our findings suggest that an object's primary function has less influence on its potential as a smart home controller. Instead, its location, physical characteristics, and intuitive affordances play a more significant role in shaping its potential. Designers should, therefore, focus on making these affordances easily discoverable by concurrently considering the object's physical properties, function, and interaction context.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MobileMuseum: Smart, Portable, ..."
MobileMuseum: Smart, Portable, and Borrowable Museum Displays to Explore Interaction Patterns and Public Engagement
Alaa Nousir, Lee Jones, Flora Lin, Tom Everrett, and Sara Nabil
(Queen’s University, Kingston, Canada; Ingenium - Canada’s Museums of Science and Innovation, Ottawa, Canada)
We explore how smart, portable, tangible, and borrowable museum displays can be used to expand museum outreach, and provide tools for measuring user engagement. Our prototype, MobileMuseum, is a portable museum display on wheels, housing digitally-fabricated replicas of artifacts from a national science and technology museum. To track interactions without cameras, we embedded photo and magnetic door sensors and RFID tags and antenna. Our deployment study was carried out for 2 months and included 4 different locations (2 recreational and 2 educational). Data collected from 6 sensors, 261 hours of field observations, 17 filled questionnaires, and 14 interviews helped us understand interaction patterns. Qualitative and quantitative findings revealed that self-monitored interactions encouraged deeper open-ended exploration. Social sharing increased among people in groups while lower foot traffic increased engagement duration. We present generalizable opportunities for self-monitored Interfaces that lends themselves to social circulation, ‘honeybee-effect’, ‘order effect’, and ‘leftovers’ from users.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MobileMuseum: Smart, Portable, ..."
MobileMuseum: Smart, Portable, and Borrowable Museum Displays to Explore Interaction Patterns and Public Engagement
Alaa Nousir, Lee Jones, Flora Lin, Tom Everrett, and Sara Nabil
(Queen’s University, Kingston, Canada; Ingenium - Canada’s Museums of Science and Innovation, Ottawa, Canada)
We explore how smart, portable, tangible, and borrowable museum displays can be used to expand museum outreach, and provide tools for measuring user engagement. Our prototype, MobileMuseum, is a portable museum display on wheels, housing digitally-fabricated replicas of artifacts from a national science and technology museum. To track interactions without cameras, we embedded photo and magnetic door sensors and RFID tags and antenna. Our deployment study was carried out for 2 months and included 4 different locations (2 recreational and 2 educational). Data collected from 6 sensors, 261 hours of field observations, 17 filled questionnaires, and 14 interviews helped us understand interaction patterns. Qualitative and quantitative findings revealed that self-monitored interactions encouraged deeper open-ended exploration. Social sharing increased among people in groups while lower foot traffic increased engagement duration. We present generalizable opportunities for self-monitored Interfaces that lends themselves to social circulation, ‘honeybee-effect’, ‘order effect’, and ‘leftovers’ from users.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Studying Visual Evidence from ..."
Studying Visual Evidence from Using Physical Space to Think
Maryam Rezaie, Samuel Huron, Parnian Taghipour, Lien Quach, Victor Cheung, and Sheelagh Carpendale
(Simon Fraser University, Burnaby, Canada; Télécom Paris - Institut Polytechnique de Paris, Palaiseau, France; Simon Fraser University, Surrey, Canada)
This paper investigates how people use physical space, free from digital constraints, to organize thoughts and ideas. Such spatial arrangements commonly support sensemaking, idea generation, and data understanding. We collected over 300 publicly shared images capturing these activities collected from real-world contexts. Our visual analysis identifies a small set of recurring spatial patterns such as clusters, grids as well as the techniques people use to build and adapt them, such as color coding, directional lines, and selective rule-following. We discuss how these practices can inform the design of more expressive and adaptable digital spatially-aware tools that better support the diverse ways people think with space.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Mirror Me: Exploring Avatar ..."
Mirror Me: Exploring Avatar Self-Views in Multi-user Virtual Reality
Katja Krug, Daniel Immanuel Fink, Mats Ole Ellenberg, Anke V. Reinschluessel, Wolfgang Büschel, Tiare Feuchtner, and Raimund Dachselt
(Dresden University of Technology, Dresden, Germany; University of Konstanz, Konstanz, Germany; University of Stuttgart, Stuttgart, Germany)
This paper investigates avatar self-views in virtual reality (VR) and their impact on multi-user communication scenarios. Self-views in video conferencing have been shown to impact our self-perception and mental load, so we explore whether similar effects occur in VR, as personal and professional gatherings progressively move to virtual spaces with 3D avatars. We identify the following key design dimensions for self-representations in VR: spatiality, anchoring and size, and self-visibility. Based on these, we designed three variants (Remote Perspective View, Personal Mirror, and Miniature Avatar), which we compare to a baseline (No Additional Self-View) in a user study. Our analysis of sixteen dyads playing a word-guessing game requiring verbal and non-verbal communication (i.e., explaining and charades) in VR confirms that self-views are beneficial for communication scenarios that require expressive body language or facial expressions, as this allows monitoring the own avatar's animations. Further, the Miniature Avatar was overall preferred due to its spatiality, world-anchoring, and full self-visibility. Based on our results, we offer design recommendations for self-views covering the key design dimensions in VR.
Publisher's Version
Video
Info
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Studying Visual Evidence from ..."
Studying Visual Evidence from Using Physical Space to Think
Maryam Rezaie, Samuel Huron, Parnian Taghipour, Lien Quach, Victor Cheung, and Sheelagh Carpendale
(Simon Fraser University, Burnaby, Canada; Télécom Paris - Institut Polytechnique de Paris, Palaiseau, France; Simon Fraser University, Surrey, Canada)
This paper investigates how people use physical space, free from digital constraints, to organize thoughts and ideas. Such spatial arrangements commonly support sensemaking, idea generation, and data understanding. We collected over 300 publicly shared images capturing these activities collected from real-world contexts. Our visual analysis identifies a small set of recurring spatial patterns such as clusters, grids as well as the techniques people use to build and adapt them, such as color coding, directional lines, and selective rule-following. We discuss how these practices can inform the design of more expressive and adaptable digital spatially-aware tools that better support the diverse ways people think with space.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Indoor Photovoltaic Interactive ..."
Indoor Photovoltaic Interactive Surfaces for Sustainable Smart Home Control: Gesture Design using Guessability and Production Methods
Nora Abdullah Almania, Sarah Yousef Alhouli, and Deepak Ranjan Sahoo
(Swansea University, Swansea, UK; Shaqra University, Shaqra, Saudi Arabia; Kuwait Institute for Scientific Research, Kuwait, Kuwait)
Indoor photovoltaic materials are novel low-cost light sensors that can be flexible, decorative, self-powered, and battery-free, and can be embedded in various surfaces throughout the home. They offer a unique opportunity for contextual control of multiple different devices in a smart home using guessable and favorite gestures. Currently available gesture vocabularies are survey-based and sensor-agnostic, but still require experimental validation. Therefore, we present experimentally generated and validated original gesture vocabularies using two user elicitation methods, the guessability and production methods, for such sensors. The capabilities of the sensor was used to prime participants for design thinking to multi-control smart home devices. We provide guidelines for designing gesture vocabularies using the two elicitation methods and report on their similarities and differences. The methodological findings and experimentally validated gesture sets would inform HCI researchers in the design of user-elicited interactions for such versatile light or electromagnetic field sensors and similar gesture-driven applications.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Designing Augmented Reality ..."
Designing Augmented Reality for Cyclists: How Text-Based Notification Placements Influence Attentional Tunneling and Cycling Experience
Linjia He, Matthew Siegenthaler, Lau Yiu Ho, Lucas Liu, Esther Bosch, Thomas Kosch, Barrett Ens, Sarah Goodwin, Benjamin Tag, and Don Samitha Elvitigala
(Monash University, Melbourne, Australia; Monash University, Clayton, Australia; German Aerospace Centre, Braunschweig, Germany; HU Berlin, Berlin, Germany; University of British Columbia at Okanagan, Kelowna, Canada; University of New South Wales, Sydney, Australia)
Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Studying Visual Evidence from ..."
Studying Visual Evidence from Using Physical Space to Think
Maryam Rezaie, Samuel Huron, Parnian Taghipour, Lien Quach, Victor Cheung, and Sheelagh Carpendale
(Simon Fraser University, Burnaby, Canada; Télécom Paris - Institut Polytechnique de Paris, Palaiseau, France; Simon Fraser University, Surrey, Canada)
This paper investigates how people use physical space, free from digital constraints, to organize thoughts and ideas. Such spatial arrangements commonly support sensemaking, idea generation, and data understanding. We collected over 300 publicly shared images capturing these activities collected from real-world contexts. Our visual analysis identifies a small set of recurring spatial patterns such as clusters, grids as well as the techniques people use to build and adapt them, such as color coding, directional lines, and selective rule-following. We discuss how these practices can inform the design of more expressive and adaptable digital spatially-aware tools that better support the diverse ways people think with space.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Transfer Effects of Long-Term ..."
Transfer Effects of Long-Term and High-Frequency Web Service Use on Target-Pointing Performance
Shota Yamanaka and Hiroaki Taguchi
(LY Corporation, Tokyo, Japan)
Prior work has shown that skills acquired through repeated computer-based tasks can transfer to similar tasks. While typical experiments span only up to several days, it remains unclear whether such transfer effects would be evident over longer learning periods. To address this question, we measured the target-tapping performance of 912 crowdworkers and analyzed its relationship with their usage logs of Yahoo! JAPAN services. Our analysis included data of unprecedented scale, featuring users holding Yahoo! JAPAN accounts for over 24 years, performing over 100,000 taps annually, or viewing over 180,000 pages annually (with at least ten users in each category). Results showed that a longer account age was correlated with improved tap performance, whereas a higher number of taps over the past year was negatively correlated with tap performance. More in-depth analyses also revealed that the performance trends varied depending on the specific services that were heavily used. These findings clarify the limitations of a naive assumption that long-term and heavy use of web services leads to better task performance through transfer effects.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "MR-MDEs: Exploring the Integration ..."
MR-MDEs: Exploring the Integration of Mixed Reality into Multi-display Environments
Griffin J. Hurt, Talha Khan, Nicolás Matheo Kass, Anthony Tang, Edward Andrews, and Jacob Biehl
(University of Pittsburgh, Pittsburgh, USA; Grand Valley State University, USA)
Multi-display environments (MDEs) are applicable to both everyday and specialized tasks like cooking, appliance repair, surgery, and more. In these settings, displays are often affixed in a manner that prevent reorientation, forcing users to split their attention between multiple visual information sources. Mixed reality (MR) has the potential to transform these spaces by presenting information through virtual interfaces that are not limited by physical constraints. While MR has been explored for single-task work, its role in multi-task, information-dense environments remains relatively unexplored. Our work bridges this gap by investigating the impact of different display modalities (large screens, tablets, and MR) on performance and perception in these environments. Our study's findings demonstrate the capability for MR to integrate into these spaces, extending traditional display technology with no impact to performance, cognitive load, or situational awareness. The study also further illustrates the nuanced relationship between performance and preference in tools used to guide task work. We provide insights toward the eventual authentic integration of MR in MDEs.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "CorrectMe: An Interactive ..."
CorrectMe: An Interactive Framework for Human-in-the-Loop Correction and Explanation of Object Detection Models
Yinuo Liu, Zhiyuan Wu, Xiaoju Dong, Shaoxiong Jiang, and Weijie Li
(Shanghai Jiao Tong University, Shanghai, China)
Object detection models, while achieving greater performance, often suffer from recurring errors such as misclassifications or missed detections. Existing explainable AI (XAI) tools primarily offer static, observation-based explanations and rarely support interactive correction or retraining, especially for non-expert users. To bridge this gap, we introduce CorrectMe, an interactive framework that integrates human-in-the-loop correction and explanation into object detection workflows. CorrectMe empowers users to iteratively explore, interpret, and rectify model errors through a unified interface featuring semantic embedding visualizations, saliency-based explanations, and natural language rationales. Users can revise predictions and incrementally retrain the model, streamlining the refinement process through lightweight updates rather than full-scale retraining or annotation. Through application scenarios and user studies, we demonstrate that CorrectMe enables more strategic corrections, improves model understanding, and lowers the barrier to practical refinement of object detection models.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Detecting Group Configurations ..."
Detecting Group Configurations in Shared Spaces: A Heuristic for Understanding Space Use Behavior
Andrew Xu, Yuezhong Xu, Jacob Biehl, and Adam J. Lee
(University of Pittsburgh, Pittsburgh, USA; University of Illinois at Chicago, Chicago, USA)
The rapid development and deployment of “smart space” technology in commercial and educational buildings has brought efficiencies and comfort to users. Yet, the “smart” is largely limited to knowledge gained from occupancy and activity data. These limited data explain what is happening in a space but nothing about how effectively the space is being used. In this work, we explore how to understand effective space use and how it may impact future smart spaces and their design. As part of this investigation, we design generalizable heuristics derived from observable human behaviors and space usage to detect group configurations in public spaces. The resultant system models similar qualities of a space that end-users will judge and measure themselves before use. The effectiveness of our system is validated against a crowdsourced study. Our system serves as a foundation for applications building upon improving space use efficacy.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Detecting Group Configurations ..."
Detecting Group Configurations in Shared Spaces: A Heuristic for Understanding Space Use Behavior
Andrew Xu, Yuezhong Xu, Jacob Biehl, and Adam J. Lee
(University of Pittsburgh, Pittsburgh, USA; University of Illinois at Chicago, Chicago, USA)
The rapid development and deployment of “smart space” technology in commercial and educational buildings has brought efficiencies and comfort to users. Yet, the “smart” is largely limited to knowledge gained from occupancy and activity data. These limited data explain what is happening in a space but nothing about how effectively the space is being used. In this work, we explore how to understand effective space use and how it may impact future smart spaces and their design. As part of this investigation, we design generalizable heuristics derived from observable human behaviors and space usage to detect group configurations in public spaces. The resultant system models similar qualities of a space that end-users will judge and measure themselves before use. The effectiveness of our system is validated against a crowdsourced study. Our system serves as a foundation for applications building upon improving space use efficacy.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Transfer Effects of Long-Term ..."
Transfer Effects of Long-Term and High-Frequency Web Service Use on Target-Pointing Performance
Shota Yamanaka and Hiroaki Taguchi
(LY Corporation, Tokyo, Japan)
Prior work has shown that skills acquired through repeated computer-based tasks can transfer to similar tasks. While typical experiments span only up to several days, it remains unclear whether such transfer effects would be evident over longer learning periods. To address this question, we measured the target-tapping performance of 912 crowdworkers and analyzed its relationship with their usage logs of Yahoo! JAPAN services. Our analysis included data of unprecedented scale, featuring users holding Yahoo! JAPAN accounts for over 24 years, performing over 100,000 taps annually, or viewing over 180,000 pages annually (with at least ten users in each category). Results showed that a longer account age was correlated with improved tap performance, whereas a higher number of taps over the past year was negatively correlated with tap performance. More in-depth analyses also revealed that the performance trends varied depending on the specific services that were heavily used. These findings clarify the limitations of a naive assumption that long-term and heavy use of web services leads to better task performance through transfer effects.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "EEG-Based Monitoring of Pilot ..."
EEG-Based Monitoring of Pilot Training: Transferring Full-Cap Representations to Headphone-Style Electrode Positions
Linyu Zheng, Yujing Mark Jiang, Zhuo Liu, and Weidong Yang
(Fudan University, Shanghai, China; AVIC General Huanan Aircraft Industry, Zhuhai, China)
Monitoring pilots’ mental states during training is important for ensuring safety and optimizing performance, yet existing full-cap EEG systems are impractical for operational use. In this work, we evaluate a headphone-style 9-channel electrode montage rather than a complete headphone EEG device, and propose a transfer learning framework to preserve performance under sparse coverage. Our two-stage approach first applies self-supervised pretraining on 64-channel full-cap EEG, and then adapts the learned representations to the headphone-style montage through a position correction module that accounts for electrode misplacement. We validate the framework in a flight simulator with 12 participants on two tasks: motor intention (left vs. right) and cognitive workload (low vs. high). Despite limited coverage, the headphone-style montage achieves within-subject accuracies of 85% for motor imagery and 89% for workload, compared to full-cap performance of 88% and 91%. Cross-subject accuracies reached 66% and 73%, demonstrating generalizability across users. Ablation analyses show that both self-supervised pretraining and position correction independently improve performance, and together provide a 12% boost in cross-subject decoding accuracy. These findings highlight the feasibility of headphone-style EEG montages for practical pilot monitoring, while clarifying methodological limitations and the need for future real-time and user-centered evaluation.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Investigating Design Considerations ..."
Investigating Design Considerations for Supporting Remote Teaching and Learning of Physical Drawing
Minghui Chen, Simo Hosio, and Koji Yatani
(University of Tokyo, Tokyo, Japan; University of Oulu, Oulu, Finland)
Teaching and learning physical drawing remotely pose user-experience challenges. In an online physical drawing course, instructors must interact with students through a camera for demonstrations, supervision, communication, and annotations in real-time. However, these activities are not well supported by commonly used video conferencing services. We conduct a qualitative study with seven instructors and five students to understand their practices and challenges in teaching and learning of physical drawing in a remote setting. Our study identifies four design considerations that a system supporting remote teaching and learning of physical drawing should take into account. We then demonstrate the technical feasibility of these design considerations by building a working prototype called SharedCanvas. We also validate our design considerations through a user study with SharedCanvas, revealing how they better supported interactions between instructors and students. We conclude by discussing our work in light of related studies on online drawing education and present future research opportunities to improve remote teaching of drawing.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "AutoPark: Adaptive Friction ..."
AutoPark: Adaptive Friction Mouse Facilitating Stable Multi-touch Gesture Input
Yohan Yun, Jisu Yim, and Geehyuk Lee
(KAIST, Daejeon, Republic of Korea)
Multi-touch mice have been proposed to integrate the rich input vocabulary of multi-touch gestures into traditional computer mice. However, incorporating multi-finger gestures into a mouse introduces a design conflict: Mice are optimized for low friction to support smooth cursor navigation, whereas touch gestures require high friction for stability. In this paper, we present AutoPark—the concept of a multi-touch mouse that dynamically adjusts its friction based on user intent. AutoPark mouse operates in a low-friction mode for cursor movement and shifts to a high-friction state for multi-touch interactions. Our first user study measured the shear forces during touch gestures, confirming that conventional friction levels of mousepads are insufficient to maintain gesture input stability on a mouse. We developed an AutoPark prototype and conducted controlled experiments, comparing it with both conventional low-friction and gesture-favored high-friction setups. The results demonstrate that AutoPark enhances gesture stability while maintaining comparable pointing performance in our study tasks, supporting a richer gesture input vocabulary within mouse-based GUI interactions.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "AutoPark: Adaptive Friction ..."
AutoPark: Adaptive Friction Mouse Facilitating Stable Multi-touch Gesture Input
Yohan Yun, Jisu Yim, and Geehyuk Lee
(KAIST, Daejeon, Republic of Korea)
Multi-touch mice have been proposed to integrate the rich input vocabulary of multi-touch gestures into traditional computer mice. However, incorporating multi-finger gestures into a mouse introduces a design conflict: Mice are optimized for low friction to support smooth cursor navigation, whereas touch gestures require high friction for stability. In this paper, we present AutoPark—the concept of a multi-touch mouse that dynamically adjusts its friction based on user intent. AutoPark mouse operates in a low-friction mode for cursor movement and shifts to a high-friction state for multi-touch interactions. Our first user study measured the shear forces during touch gestures, confirming that conventional friction levels of mousepads are insufficient to maintain gesture input stability on a mouse. We developed an AutoPark prototype and conducted controlled experiments, comparing it with both conventional low-friction and gesture-favored high-friction setups. The results demonstrate that AutoPark enhances gesture stability while maintaining comparable pointing performance in our study tasks, supporting a richer gesture input vocabulary within mouse-based GUI interactions.
Publisher's Version
Video
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "Characterizing Amplitude-Dominated ..."
Characterizing Amplitude-Dominated Performance: A Refinement of Fitts’ Law for Target Acquisition on Large-Format Touchscreens
Xinyong Zhang
(Renmin University of China, Beijing, China)
Large-format touchscreens have become commonplace in classrooms and meeting rooms, yet little is known about how their scale affects input behavior. Operating these displays engages more upper-limb joints—especially the shoulder—thereby altering movement dynamics. In a 65-inch touchscreen study, we found that conventional Fitts' law explains ≤ 80% of the variance in movement time, as performance is primarily driven by amplitude. To address the modeling bias resulting from this amplitude dominance, we refine the index of difficulty as IDx = log_2(A/(W + c) + 1). Rather than treating c as a mere fitting parameter, we interpret it as an intrinsic property of pointing dynamics: it quantifies the systematic deviation from the canonical speed-accuracy tradeoff implied by the "as quickly and accurately as possible" instruction. By calibrating c with the proposed anti-overfitting criteria, we improve model fits for both finger and pen input, raising R² to above 0.97. We present the calibration rules, interpret c across contexts, validate IDx robustness via leave-one-out cross-validation, and demonstrate its generalizability on data from prior studies—including a large tabletop experiment. Finally, we translate the findings into practical guidelines for UI and experimental design.
Publisher's Version
Proc. ACM Hum. Comput. Interact., vol. 9, issue 8: "EEG-Based Monitoring of Pilot ..."
EEG-Based Monitoring of Pilot Training: Transferring Full-Cap Representations to Headphone-Style Electrode Positions
Linyu Zheng, Yujing Mark Jiang, Zhuo Liu, and Weidong Yang
(Fudan University, Shanghai, China; AVIC General Huanan Aircraft Industry, Zhuhai, China)
Monitoring pilots’ mental states during training is important for ensuring safety and optimizing performance, yet existing full-cap EEG systems are impractical for operational use. In this work, we evaluate a headphone-style 9-channel electrode montage rather than a complete headphone EEG device, and propose a transfer learning framework to preserve performance under sparse coverage. Our two-stage approach first applies self-supervised pretraining on 64-channel full-cap EEG, and then adapts the learned representations to the headphone-style montage through a position correction module that accounts for electrode misplacement. We validate the framework in a flight simulator with 12 participants on two tasks: motor intention (left vs. right) and cognitive workload (low vs. high). Despite limited coverage, the headphone-style montage achieves within-subject accuracies of 85% for motor imagery and 89% for workload, compared to full-cap performance of 88% and 91%. Cross-subject accuracies reached 66% and 73%, demonstrating generalizability across users. Ablation analyses show that both self-supervised pretraining and position correction independently improve performance, and together provide a 12% boost in cross-subject decoding accuracy. These findings highlight the feasibility of headphone-style EEG montages for practical pilot monitoring, while clarifying methodological limitations and the need for future real-time and user-centered evaluation.
Publisher's Version