NUILab
NUILab · Research
Research
How do we make interaction with 3D and immersive systems natural, low-effort, and available to everyone?
We build and study natural user interfaces — gesture, speech, gaze, and touch — for augmented and virtual reality. The throughline is people: how they perceive, where their attention goes, and what they can do without a manual. Below are the six areas the lab is pushing now. Most projects sit at more than one.
Where we’re heading. The lab has always asked how people naturally interact with computers. As AI becomes part of that interface, we’re extending the question to human-centered AI: how people work alongside intelligent systems naturally, effectively, and safely — in the real world and in immersive ones. The six areas below are the throughline; we’re now pursuing each with intelligent systems in view.
Our north star: making the computer invisible“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”
— Mark WeiserEverything below serves that goal: letting people concentrate on the task, not the technology.
Unimodal & Multimodal Interaction
What people naturally do with gesture, speech, and gaze — one modality at a time or combined. We run elicitation and observational studies, then build the interaction techniques they point to.
XR Widgets & Displays
The building blocks of useful AR — notifications, timers, cues, labels — and what they cost. We measure how cueing, clutter, and automation bias shape attention, and design displays that inform without overload.
XR Training & Education
Immersive systems that teach. We optimize cognitive load in XR training and build immersive learning environments — and study how those experiences shape interest and retention in computing.
XR for Health & Wellbeing
XR aimed at wellbeing: virtual nature for stress reduction, nature contact for older adults, and clinical AR such as ergonomic feedback in surgery.
XR Accessibility
Interaction design for everyone — accessible XR interaction for people with disability, Deaf community led XR research, and more.
Human-Centered AI
As intelligent systems become the interface, we study how people work alongside them — intelligent user interfaces, what AI model degradation does to users, and the privacy, trust, and safety guarantees XR + AI must meet.
Funding
The lab's work is supported by more than $5.5M in external funding at CSU since 2018 — from NSF, ONR, DARPA, NIH, and foundation partners.
Active grants
- NSF I-USE · co-PI Collaborative Research: Improving Conceptual Understanding of Invisible Physics by Translating the Science of Learning into Virtual Reality Environments
- ONR DURIP · PI Training Optimization for US Navy and Marine Radio Operations to Assess Cognitive Load and Manage Extraneous Load
- Office of Naval Research · PI Assessing Cognitive Load and Managing Extraneous Load to Optimize Training
- NSF CAREER · PI CAREER: Microgesture and Multimodal Interaction Techniques for Augmented Reality
Completed grants (12)
- Office of Naval Research · PI Perceptual/Cognitive Aspects of Augmented Reality: Experimental Research and a Computational Model
- DARPA · PI Ego-Centric Emotion Recognition using Augmented Reality Headsets (I2O Postdoctoral Fellowship)
- ONR DURIP · PI WARFighting Performance: Augmented Reality Multi-Modal Interaction Techniques for JTAC and Battlefield Readiness
- NSF CRII · PI CRII: CHS: Understanding Gesture User Behavior in Augmented Reality Headsets
- NSF CCRI · PI CCRI Planning: Collaborative Research: Low-Latency for Augmented Reality Interactive Systems (LLARIS)
- ONR sub-award via VR Rehab, Inc. · PI Fused Augmented Realities with Synthetic Vision (FAR/SV) Systems for Ground Forces
- ONR via VR Rehab, Inc. · PI SAIPAN: Single Amphibious Integrated Precision Augmented Reality Navigation System (SBIR Phase I)
- NIH challenge award to BioMagic VR, Inc. · PI BioBraceVR: Bio-Interactive Device with Personalized Avatar Therapy for SUD (NIH-NIDA SUD Challenge)
- NSF SBIR via Polymer Braille Inc. · PI NSF SBIR Phase IIA: 2.5D Extensions to Braille-based User Interaction (sub-award)
- NSF Future of Work · co-PI (PI: Benjamin Clegg) FW-HTF-P: Optimizing Long-Term Human Performance in Future Work
- DARPA via DOD-ARMY · co-PI (PI: J. Ross Beveridge) Communication through Gestures, Expression, and Shared Perception (CWC)
- Florida Center for Cybersecurity (FC2) · co-PI (PI: Peter Clarke, FIU) Using a Cyberlearning Environment to Enhance Critical Cybersecurity Education