TL;DR To enable collaboration between sighted and visually impaired individuals in STEM education. Drawxi uses haptic feedback enabled board for sketching simple diagrams. Plus, sighted users can use Drawxi canvas to view and edit diagrams remotely. Thus, visually impaired users can convey complex ideas in a hands-on way.
Location: Student Design Competition, CHI 2019, Scotland, UK.
Role: Product design, UX Research, Team facilitator.
Methods: Contextual Inquiry, Co-Design workshop, Wizard of Oz.
Tools: Sketch, After Effects, Blender.
CHI 2019 Student design competition was a great opportunity to collaborate with other students who are interested in accessibility! My 5 member team conducted user interviews with 20 visually impaired participants in the age group of 21 to 45 years. User interviews revealed that most visually impaired users chose not to pursue STEM education because it requires communicating complex ideas through pictures and drawings. It’s difficult to have positive learning experiences, and tools are not readily available for facilitating communication with instructors and sighted classmates.
My son often needs help with his homework. But it is hard because I cannot see the diagrams in his textbooks. So I end up asking a lot of questions before I can help!
My job requires me to make course material accessible in the University. But majority learning happens in the classroom. I wish students and instructors could create accessible content collectively.
How might we enable collaboration between visually impaired and sighted users with an affordable drawing tool?
Performing contextual inquiry made my team aware of the drawing habits of visually impaired participants. Their stories from the co-design workshop facilitated brainstorming and concept generation. So let's look at the insights from participatory design methods!
Visually impaired users combined information from multi-sensory interactions to form mental imagery.
Using the same tools enhances the discovery of diverse work practices in a hands-on way promoting empathy.
Drawing a closed shape is hard because retracing the starting point is difficult for visually impaired users.
Visually impaired users are more confident when using their fingers or hands because they recognize physical objects through touch.
Our team found there were too many decisions to make. I proposed using the methods from Michele and Jim McCarthy’s "The Core Protocols" to distill our research findings. We outlined key features that are most beneficial to the users;
A Drawxi board (a haptic touch surface) for creating drawings
A Drawxi Canvas (a web-based drawing platform) for remote collaboration.
The concept is inspired by the latest developments in electromagnetic haptic touch surfaces such as TeslaTouch, Senseg, and Ultrahaptics.
Our team created a video explaining how the proposed concept enables the communication of complex ideas through simple diagrams. The video showcases a visually impaired participant sharing her thoughts in a team meeting by using Drawxi. The proposed concept facilitates all team members reaching a common understanding, promoting fair collaboration and empathy.
Let's say Emma wants to clarify the concept of magnetic poles with her instructor. She connects the Drawxi board to her laptop using a USB connection.
Emma presses the rectangle button on the top left of the Drawxi board. This triggers basic shapes mode.
She double taps on the board to mark the starting point. Using the vibrating starting point as a reference, she draws the length and breadth of the rectangle.
Emma sends her instructor the URL and passcode for collaborating remotely. After authentication, her instructor can view the diagram on Drawxi Canvas.
Emma double taps on the Drawxi board to mark the starting point and triggers freehand drawing mode.
Using the vibrating starting point as a reference, Emma draws a label. The drawing ends when she lifts her finger.
Her instructor can view the label on Drawxi Canvas after a few seconds.
Emma's instructor updates the diagram by using the edit buttons to undo the changes and draws a new diagram.
Emma receives a notification after her instructor saves the changes on Drawxi canvas. Emma clicks on the OK button on the top to refresh the frame.
Through haptic feedback, Emma perceives the changes by using her fingers. By default Drawxi is in the explore mode for users to perceive the changes.
"Let's look at the design process that led to the proposed design!"
I took the lead on reading research papers relevant to mental models in communication, evaluative studies of several modalities, and experimental designs. We learned that visually impaired users relied on solutions that leveraged multi-sensory interactions to compensate for the visual deficit. But, the tradeoff between the effectiveness of a sensory interaction versus the cost of implementation is tricky.
Key finding: Most solutions are prohibitively expensive, difficult to carry, or not suitable for collaboration.
Mapping user scenario proved helpful to understand the best place to introduce a design change. Looking through the user's perspective demonstrated that lack of affordable, portable, and collaborative tools impede adequate communication. Thus, the generative capabilities of visually impaired users remain undiscovered, discouraging them from pursuing STEM education.
Current solutions are prohibitively expensive, hence the majority of the users find it difficult to buy them.
John wants to help his son with science homework.
John ends up asking many questions before he can provide any guidance.
By that time, his son is distracted and tries to procrastinate the homework.
Majority solutions are too bulky or require too many peripherals. Making them unfit for users who move frequently.
At the end of each class, Alan has to pack all his things before moving to the next class.
In case he is going alone, he has to figure out a way to reach his next class.
Then he has to unpack all his things before the instructor begins the lecture.
Most devices are made for single person use. But in education, multiple people in the learning process.
The instructor has shared all the required tactile class material with Laura.
During the class, the instructor makes some changes while explaining a doubt.
But, Laura could not follow the clarification because her tactile class material does not have the changes.
Key learning: Making quick changes to diagrams in a classroom environment is vital to STEM education. But visually impaired students miss out on those changes.
To better understand users drawing habits and workarounds, we conducted contextual inquiries with 2 participants. This revealed that users frequently rely on tactile artifacts to perceive graphical information. They have a structure based mental-model (3D) instead of a survey like mental-model (2D). Hence, graphical information should be available in 2D and 3D formats.
Line quality with pen-type tools is better, but users felt less confident due to lack of touch.
Users felt more confident while drawing with fingers because they had a better sense of space.
Drawing with pen and fingers was difficult for users with limited finger movements.
Users rely on custom workarounds to communicate complex ideas. For e.g. an excel sheet or drawing a stick figure.
Key finding: Drawing closed shapes is a problem because it is difficult to retrace the starting point. Visually impaired users perceive through their fingers. First, they touch the edges of a physical object and then move inwards to perceive the texture.
We conducted co-design workshops with 10 visually impaired participants. We kickstarted participant brainstorming with "What if" cards, then formed teams with both sighted and visually impaired participants. Each team used materials like clay, popsicle sticks, stickers, stencils, and tactile UI elements (created by laser cutting soft cardboard) to propose a design solution.
The concept uses a magnetic pen to raise the metal balls when users draw.
Pros: Tactile feedback, erasable.
Cons: Low accuracy of drawing, difficult to maintain.
Users will use a pressure sensor-enabled cover to interact with electronic devices for drawing.
Pros: Affordable, compatible with many devices.
Cons: not easy to maintain, fragile.
It has a panel on the left for drawing and a panel on the right for perceiving shapes via pin matrix.
Pros: Less confusing.
Cons: Poor use of real estate, restrictive.
It uses haptic feedback to help users draw and perceive drawings. Buttons on the top for left/right hand use.
Pros: More space for drawing, less confusion.
Cons: Requires many haptic actuators.
Key learning: Visually impaired users need effective tactile product semantics to use physical products. As the size of physical products is restricted, it is essential to scale a small physical real estate on a large software space.
Our team performed usability testing with three participants by creating prototypes via laser cutting soft cardboard material. My role was planning the test, and performing the usability study.
Participants preferred the (200 mm * 160 mm) size for the product. They felt they could easily carry it in their handbags or purses. Yet, it was big enough to find it amongst other school materials.
Participants used their non-dominant hand to map the drawing space. Hence, they preferred a drawing pad size compatible with their palm.
Participants felt less confident using a pen-type device while drawing. They preferred drawing with their fingers. This also enabled perceiving other product semantics on the device.
We tested various gridlines, dotted surfaces, and other tactile layouts. Participants preferred using a flat smooth surface, allowing free movement of fingers.
Key finding: Visually impaired users use the non-dominant hand to make sense of the space. Hence, using fingers to draw makes them more confident.
I learned that we can optimize meetings by keeping them short and goal-oriented. It is better to make decisions quickly and fail fast.
Participatory design is central for accessible designs. Designing with the user community eliminates faulty premises, ensuring confidence in the design. It is easy to develop empathy, trust, and cultivate humility.
Design is not a straightforward process, and it can be difficult to visualize the final results. It is vital to trust the process and keep moving ahead one step at a time.