Drawxi

Product Design

TL;DR To enable collaboration between sighted and visually impaired individuals in STEM education. Drawxi uses haptic feedback enabled board for sketching simple diagrams. Plus, sighted users can use Drawxi canvas to view and edit diagrams remotely. Thus, visually impaired users can convey complex ideas in a hands-on way.

Drawing of an visually impaired student visualizing several concepts in science, maths, technology, and engineering.

Location: Student Design Competition, CHI 2019, Scotland, UK.
Role: Product design, UX Research, Team facilitator.

Methods: Contextual Inquiry, Co-Design workshop, Wizard of Oz.
Tools: Sketch, After Effects, Blender.

Discovering user needs through user interviews

CHI 2019 Student design competition was a great opportunity to collaborate with other students who are interested in accessibility! My 5 member team conducted user interviews with 20 visually impaired participants in the age group of 21 to 45 years. User interviews revealed that most visually impaired users chose not to pursue STEM education because it requires communicating complex ideas through pictures and drawings. It’s difficult to have positive learning experiences, and tools are not readily available for facilitating communication with instructors and sighted classmates.

Image of a visually impaired participant walking on a sidewalk with a walking stick.
My son often needs help with his homework. But it is hard because I cannot see the diagrams in his textbooks. So I end up asking a lot of questions before I can help!

-Participant 1

Image of two visually impaired participants in their apartment talking to design students as part of user interviews.
My job requires me to make course material accessible in the University. But majority learning happens in the classroom. I wish students and instructors could create accessible content collectively.

-Participant 2

How might we enable collaboration between visually impaired and sighted users with an affordable drawing tool?
Key insights from contextual inquiry, co-design workshop, and usability testing

Performing contextual inquiry made my team aware of the drawing habits of visually impaired participants. Their stories from the co-design workshop facilitated brainstorming and concept generation. So let's look at the insights from participatory design methods!

Icon of multiple senses sending messages to the brain.
High multi-sensory perception

Visually impaired users combined information from multi-sensory interactions to form mental imagery.

Icon of software tools.
Tools influence empathy

Using the same tools enhances the discovery of diverse work practices in a hands-on way promoting empathy.

Icon of a hand with a stop sign
Pain points while drawing

Drawing a closed shape is hard because retracing the starting point is difficult for visually impaired users.

Icon of a hand touching braille.
Need for tactile feedback

Visually impaired users are more confident when using their fingers or hands because they recognize physical objects through touch.

Achieving deadline-ready design

Our team found there were too many decisions to make. I proposed using the methods from Michele and Jim McCarthy’s "The Core Protocols" to distill our research findings. We outlined key features that are most beneficial to the users;  

  • A Drawxi board (a haptic touch surface) for creating drawings

  • A Drawxi Canvas (a web-based drawing platform) for remote collaboration.

The concept is inspired by the latest developments in electromagnetic haptic touch surfaces such as TeslaTouch, Senseg, and Ultrahaptics.

Image showing Drawxi Board and Drawxi Canvas. Final concepts presented at CHI 2019.

Our team created a video explaining how the proposed concept enables the communication of complex ideas through simple diagrams. The video showcases a visually impaired participant sharing her thoughts in a  team meeting by using Drawxi. The proposed concept facilitates all team members reaching a common understanding, promoting fair collaboration and empathy.

Checkout the video
How it works?

Let's say Emma wants to clarify the concept of magnetic poles with her instructor. She connects the Drawxi board to her laptop using a USB connection.

Drawing basic shapes
Image showing person clicking rectangle button on the top left of the Drawxi Board.

Emma presses the rectangle button on the top left of the Drawxi board. This triggers basic shapes mode.

Image showing a person drawing length and breadth of an rectangle.

She double taps on the board to mark the starting point. Using the vibrating starting point as a reference, she draws the length and breadth of the rectangle.

Image showing rectangle being drawn on Drawxi canvas after user has drawn a reactangle on Drawxi Board.

Emma sends her instructor the URL and passcode for collaborating remotely. After authentication, her instructor can view the diagram on Drawxi Canvas.

Drawing freehand diagrams or labels
Image showing a person performing a double tap operation to trigger the freeform drawing mode.

Emma double taps on the Drawxi board to mark the starting point and triggers freehand drawing mode.

Image showing a person drawing a freeform drawing on Drawxi board.

Using the vibrating starting point as a reference, Emma draws a label. The drawing ends when she lifts her finger.

Image showing Freeform drawing being represented on Drawxi Canvas.

Her instructor can view the label on Drawxi Canvas after a few seconds.

Perceiving diagrams drawn by others
This image shows sighted participant editing a diagram on Drawxi Canvas.

Emma's instructor updates the diagram by using the edit buttons to undo the changes and draws a new diagram.

This image shows Drawxi board receiving a notification after drawing is edited on Drawxi Canvas.

Emma receives a notification after her instructor saves the changes on Drawxi canvas. Emma clicks on the OK button on the top to refresh the frame.

This image shows a person perceiving a diagram on Drawxi Board.

Through haptic feedback, Emma perceives the changes by using her fingers. By default Drawxi is in the explore mode for users to perceive the changes.

"Let's look at the design process that led to the proposed design!"
Performing competitive analysis of current solutions

I took the lead on reading research papers relevant to mental models in communication, evaluative studies of several modalities, and experimental designs. We learned that visually impaired users relied on solutions that leveraged multi-sensory interactions to compensate for the visual deficit. But, the tradeoff between the effectiveness of a sensory interaction versus the cost of implementation is tricky.

Feelif Creator is an adapted tablet that combines haptics (vibrations), sounds, and an embossed grid. It comes with several apps for drawing, games, communication, etc. Its costs $770 to $2755. Highly portable and collaborative. InTACT sketchpad and pen allow users to create tactile graphical drawings using erasable raised lines drawn on a tactile drawing sheet. It costs roughly about $250. But users cannot digitize the content and share it remotely. Hyperbraille is a pin-matrix display to perceive spatial structures and graphic symbols. It has a limited touch input from users but it costs approximately $50,000. Besides it is very bulky and difficult to carry around. It is a pin matrix display with the ability to set different heights for pins to enable the display of topographical maps and other graphical elements. Connectable through USB, USB-C, HDMI, etc. It costs $4000 to $5000. Highly portable but medium collaboration.
Icon of bulb to showcase key finding.

Key finding: Most solutions are prohibitively expensive, difficult to carry, or not suitable for collaboration.

Understanding problematic situations

Mapping user scenario proved helpful to understand the best place to introduce a design change. Looking through the user's perspective demonstrated that lack of affordable, portable, and collaborative tools impede adequate communication. Thus, the generative capabilities of visually impaired users remain undiscovered, discouraging them from pursuing STEM education.

Scenario 1 is the problem of affordability. A dollar icon represents the theme of the scenario.

Current solutions are prohibitively expensive, hence the majority of the users find it difficult to buy them.

Image of John's son asking help for homework where he is trying to understand budding in yeast.

John wants to help his son with science homework.

Image of John asking many questions for example, Tell me what do you see?, Are there things inside the diagram, Do you see anything growing?

John ends up asking many questions before he can provide any guidance.

Image of John's son getting distracted and suggesting he will ask his teacher at school. John wishes he could afford better tools to help his son with homework.

By that time, his son is distracted and tries to procrastinate the homework.

Scenario 2 is the problem of portability. The icon shows a person carrying an object and running.

Majority solutions are too bulky or require too many peripherals. Making them unfit for users who move frequently.

In this image Alan is hurriedly packing his stuff to switch classrooms.

At the end of each class, Alan has to pack all his things before moving to the next class.

In this image, Alan is walking fast in a corridor.

In case he is going alone, he has to figure out a way to reach his next class.

In this image, Alan is unpacking his bag to get ready for the next class.

Then he has to unpack all his things before the instructor begins the lecture.

Scenario 3 is based on collaboration. Icon of two hands forming a heart represents collaboration.

Most devices are made for single person use. But in education, multiple people in the learning process.

Laura is sitting in the classroom with all the tactile material she needs for the class.

The instructor has shared all the required tactile class material with Laura.

laura sitting the class when a students asks a doubt and the instructor makes some changes to the diagram explaining Photoelectric effect.

During the class, the instructor makes some changes while explaining a doubt.

Image of Laura feeling bad that she could not understand the instructors changes and she fears she needs to stay back after class to get extra help.

But, Laura could not follow the clarification because her tactile class material does not have the changes.

Icon for key findings.

Key learning: Making quick changes to diagrams in a classroom environment is vital to STEM education. But visually impaired students miss out on those changes.

Observing user mental models and workarounds

To better understand users drawing habits and workarounds, we conducted contextual inquiries with 2 participants. This revealed that users frequently rely on tactile artifacts to perceive graphical information. They have a structure based mental-model (3D) instead of a survey like mental-model (2D). Hence, graphical information should be available in 2D and 3D formats.

Contextual Inquiry mage showing a visually impaired participant drawing with a pen.
Drawing with a pen

Line quality with pen-type tools is better, but users felt less confident due to lack of touch.

Contextual Inquiry image showing a participant drawing with their fingers.
Drawing with fingers

Users felt more confident while drawing with fingers because they had a better sense of space.

Contextual Inquiry image showing a visually impaired participant who has difficulty drawing with fingers because of difficulty with motor skills.
Influence of Motor skills

Drawing with pen and fingers was difficult for users with limited finger movements.

Contextual Inquiry image showing a participant drawing a stick figure with a pen to convey an idea.
Current workarounds

Users rely on custom workarounds to communicate complex ideas. For e.g. an excel sheet or drawing a stick figure.

Icon of a bulb to showcase key finding.

Key finding: Drawing closed shapes is a problem because it is difficult to retrace the starting point. Visually impaired users perceive through their fingers. First, they touch the edges of a physical object and then move inwards to perceive the texture.

Generating concepts together

We conducted co-design workshops with 10 visually impaired participants. We kickstarted participant brainstorming with "What if" cards, then formed teams with both sighted and visually impaired participants. Each team used materials like clay, popsicle sticks, stickers, stencils, and tactile UI elements (created by laser cutting soft cardboard) to propose a design solution.

Image showing four visually impaired participants participating in Co-Design workshop where they are creating concepts with design students.
Image showing metal ball concept in which users use a magnetic tip enabled pen to draw. The pin attracts metal balls in circular slots. This allows visually impaired users to perceive diagrams.
Metal Ball concept

The concept uses a magnetic pen to raise the metal balls when users draw.
Pros: Tactile feedback, erasable.
Cons: Low accuracy of drawing, difficult to maintain.

This image shows a pressure sensor enabled phone or tablet cover that allows users to perceive the diagrams on the screen.
Phone/Tablet cover

Users will use a pressure sensor-enabled cover to interact with electronic devices for drawing.
Pros: Affordable, compatible with many devices.
Cons: not easy to maintain, fragile.

This image shows a concept with two panels. On the left panel users can draw diagrams on a smooth surface. On the right panel pin matrix enables users to perceive the diagram.
Two-panel drawing pad

It has a panel on the left for drawing and a panel on the right for perceiving shapes via pin matrix.
Pros: Less confusing.
Cons: Poor use of real estate, restrictive.

This image shows the drawing pad concept which uses smooth surface to enable users to draw and perceive graphical information through haptic feedback.
Plain drawing pad

It uses haptic feedback to help users draw and perceive drawings. Buttons on the top for left/right hand use.
Pros: More space for drawing, less confusion.
Cons: Requires many haptic actuators.

Icon of a bulb to signify key insights.

Key learning: Visually impaired users need effective tactile product semantics to use physical products. As the size of physical products is restricted, it is essential to scale a small physical real estate on a large software space.

Choosing a design and trying different iterations together

Our team performed usability testing with three participants by creating prototypes via laser cutting soft cardboard material. My role was planning the test, and performing the usability study.

An image showing different board sizes that were used during usability testing. Participants preferred the 200 mm * 160 mm size.
Ideal size of the board

Participants preferred the (200 mm * 160 mm) size for the product. They felt they could easily carry it in their handbags or purses. Yet, it was big enough to find it amongst other school materials.

Image showing Participant trying different gridlines and trying to draw with a pen.
Role of non-dominant hand

Participants used their non-dominant hand to map the drawing space. Hence, they preferred a drawing pad size compatible with their palm.

Image showing Participant trying different gridlines and trying to draw with a finger.
Fingers Vs Stylus

Participants felt less confident using a pen-type device while drawing. They preferred drawing with their fingers. This also enabled perceiving other product semantics on the device.

Image showing different gridlines that were tested during the usability testing.
Texture of the drawing surface

We tested various gridlines, dotted surfaces, and other tactile layouts. Participants preferred using a flat smooth surface, allowing free movement of fingers.

A icon of a bulb to signify key finding.

Key finding: Visually impaired users use the non-dominant hand to make sense of the space. Hence, using fingers to draw makes them more confident.

Reflections

I learned that we can optimize meetings by keeping them short and goal-oriented. It is better to make decisions quickly and fail fast.

Participatory design is central for accessible designs. Designing with the user community eliminates faulty premises, ensuring confidence in the design. It is easy to develop empathy, trust, and cultivate humility.  

Design is not a straightforward process, and it can be difficult to visualize the final results. It is vital to trust the process and keep moving ahead one step at a time.

Icon of a bulb representing reflections.