CAREER: Bridging the Digital Accessibility Gap in STEM Using Multisensory Haptic Platforms

This project investigates how to use new touch technologies, like touchscreens, to create graphics and simulations that can be felt, heard, and seen. Using readily available, low-cost systems, the principal investigator will investigate how to map visual information to touch and sound for students with visual impairments.

Full Description

Consider learning visual subjects such as math, engineering, or science without being able to see. Suddenly, the graphs, charts, and diagrams that provide a quick way to gather information are no longer effective. This is a challenge that students with visual impairments face in classrooms today as educational materials are most often presented electronically. The current way that individuals with visual impairments "read" graphics is through touch, feeling raised dots and patterns on paper that represent images. Creating these touch-based graphics requires extensive time and resources, and the output provides a static, hard-copy image. Lack of access to graphics in STEM subjects is one of the most pressing challenges currently facing individuals with visual impairments. This is a concern given the low representation of students with these disabilities in STEM fields and professions.

This project investigates how to use new touch technologies, like touchscreens, to create graphics and simulations that can be felt, heard, and seen. Using readily available, low-cost systems, the principal investigator will investigate how to map visual information to touch and sound. This research builds on prior research focused on representing the building blocks of graphics (points, lines, and shapes) nonvisually. In this project, the investigator will determine how to represent more challenging graphics such as charts, plots, and diagrams, nonvisually. The project will then explore the role of touch feedback in interactive simulations, which have moving elements that change with user input, making nonvisual access challenging. Finally, the projects extends the research to students with other disabilities, toward understanding the benefits and changes necessary for touch technologies to have broad impact. The project involves group and single-subject designs with approximately 65 students with visual impairments and focuses on the following outcomes of interest: students' graph literacy, percent correct on task assessments, time of exploration, response time, number of revisits to particular areas of the graphic, and number of switches between layers. Working closely with individuals with disabilities and their teachers, this work seeks to bridge the current graphical accessibility gap in STEM and raise awareness of universal design in technology use and development.

Publications:

Tennison J., Greenberg J., E. Moore, and Gorlewicz, J. L. Haptic paradigms for multimodal interactive simulations. Journal on Technology and Persons with Disabilities. CSUN Assistive Technology Conference, March 2021.

Gorlewicz, J. L., Tennison, J. L., Uesbeck, P. M., Richard, M. E., Palani, H. P., Stefik, A., Smith, D. W., &   Giudice, N. A. (2020). Design Guidelines and Recommendations for Multimodal, Touchscreen-based Graphics. ACM Transactions on Accessible Computing (TACCESS), 13(3), 1-30.

Tennison, J. L., Uesbeck, P. M., Giudice, N. A., Stefik, A., Smith, D. W., & Gorlewicz, J. L. (2020). Establishing Vibration-Based Tactile Line Profiles for Use in Multimodal Graphics. ACM Transactions on Applied Perception (TAP), 17(2), 1-14.

PROJECT KEYWORDS

Project Materials