Can emerging technology be used to help with Stroke Recovery?

 
profile.png

Team Size

3

Duration

3 weeks

Role

Designer and Researcher

SKills

  1. Research
  2. Stakeholder Analysis
  3. Wire framing & prototyping
  4. Visual & Motion Design

TOOLS

  1. Cinema 4d
  2. Adobe After Effects
  3. Sketch
  4. Framer
 
 

THE CHALLENGE

Stroke is the leading causing of physical disability in the United States. Nearly 80,000 people suffer from a stroke annually causing serious, long-term impairments.

 

 

 

The Client

This project was created for The Comprehensive Stroke Center at Allegheny Health Network to develop a solution for stroke patients in any phase ( before, during or after) with the help of emerging technology.

 

 

 

OUR SOLUTION

Wabbit is an augmented reality mobile game designed for motor skill recovery of stroke patients.

 

Using Augmented reality and machine learning, Wabbit creates an exercise plan for the patients and converts these into game actions. Wabbit combines the soothing and playful environment of farming, uses augmented reality combined with ambient sounds to create an immersive experience and uses the game actions to help convert the benefits of physical therapy into engaging interactions. 

 
 

WATCH HOW IT WORKS

 
 
 

Discovery

STAKEHOLDER MAPPING

Since Stroke is such a broad subject, and neither of us came from a medical background, we started to make sense of the entire thing by creating a stakeholder mapping. This exercise helped us get an over-arching view of the entire problem space which we could then breakdown into tiny chunks for further analysis and spotting a problem. A further quick study on stroke research made us choose

  1. Stroke Centre
  2. Insurance
  3. Patient ( before, during and after stroke phase)
  4. Existing tools for these phases.

as areas to start indepth study.

 
Screen Shot 2018-03-31 at 1.02.23 AM.png
 
 

Discovery

FOCUSSED ANALYSIS & COMPETITIVE RESEARCH

A study of the chosen areas from the stakeholder maps led to the following insights. During the study, we did competitive analysis of existing products that help with stroke and also went over articles, educational videos and papers related to the subject. Out of this research we figured that we wanted to focus on the recovery phase. This decision was made based on the following insights:

Insights.png
 

conceptualization

ADDRESSING RESEARCH INSIGHTS

To apply all the insights from the research, we decided to do design sprints involving activities such as "Reverse assumptions and 20 questions" ( down left) and crazy eights ( down right) which helped us think very creatively of the ideas. It was out of thes crazy eight seeions that I first came up with the idea of "virtual gardening" which the team decided to take forward as a farming game.

Design sprint.png
insights_1.png
insights_2.png
 

Discovery

EXPERT INTERVIEWS

Dr. Eugene.png
 
Josbeno.png
 

Implementation

DESIGNING THE GAME

As a team with discussed the user flow and wireframes of the entire game and then moved onto making screens for a select task .The task we had chosen was to uproot beats on a farm. The chosen wireframes were then taken into Sketch for high fidelity prototypes, then into Cinema4d & After effects to mock the video and augmented reality section and then into Framer for creating an interactive clickthrough.

 
 An overview of the first user flow where we were trying to figure which are the most crucial screens to convey the idea of the application.

An overview of the first user flow where we were trying to figure which are the most crucial screens to convey the idea of the application.

Sketch for high fidelity screens

Screen Shot 2018-03-31 at 3.08.17 AM.png

Cinema4d + After Effects for augmented reality video

Screen Shot 2018-04-08 at 11.34.44 PM.png

framer for interactive prototype

framer.png
 
 

HOW WABBIT WORKS

ezgif.com-gif-maker (3).gif

The game opens with a list of tasks that it creates for the patient based on physical therapy exercises recommended for them.  The game starts with exercises suitable for all patients and modifies them with machine learning based on how the patient responds.

ezgif.com-gif-maker (4).gif

One handed interaction menus help patients navigate easily between tasks lined up for the day and select the one that they like.

 
ezgif.com-gif-maker (6).gif

A label accompanied by sound pops up to tell the patient on which farm the task is supposed to be done. Sound helps with locating the farm spatially. 

 
ezgif.com-gif-maker (7).gif

An assistive animation is over layed on top of the game action to help them practice the proper exercise form.

 
ezgif.com-gif-maker (5).gif

A task report is given to the patient at the end of the day making them feel like they accomplished so much by counting every small interaction. In case the patient is not able to do any of the given tasks, the game encourages them that "tomorrow is a new day!" and tells them what they can do and how it will help them.

 
recovery screen.png

Especially important is the progress report screen which shows the patient how their motor skills have been improving. This screen can play a major role is keeping patients with no caregivers or support motivated through the recovery period.

 
 
 
 

FUTURE

CREATING A ROAD MAP

The current version of Wabbit was designed as being a product that would use machine learning and help patients with a personalised physical therapy using game actions. In the immediate future, Wabbit would like to add specially manufactured objects as mentioned in the question above " how to measure muscle power". Some other things in the pipeline would be to make the product even more immersive with seamless IoT integration and to add social component of being able to play with other stroke patients and to get consultation with an actual doctor based on your gameplay.

 
Roadmap Final.png
 

Open questions

WHY AUGMENTED REALITY?

We had 2 main reasons for choosing Augmented Reality:

  1. AR is relatively new technology and hence helps create a novel, exciting experience for a patient. Something that he/she would be very excited to try everyday.
  2. AR allows people to use computer generated objects to interact with with the real world environment which allows for the exercise to be brought to the patients own house. It eliminates the use of traditional screen based gestures for game inputs, and provides a way in which the entire body can be used, if needed.

WHY FARMING?

Ten papers published since 2003 reported positive effects of gardening as a mental health intervention for service users, including reduced symptoms of depression and anxiety. We saw this as teh perfect environment for a stroke patient who needed a relaxing immersive enviroment. We also saw immense potential in coming up with various farm related tasks that involve all parts of the body.

 

WHY PORTRAIT ASPECT RATIO?

Unlike traditoinal games, AR requires a good use of perspective and depth. This is the reason why almost all existing AR games eg. Pokemon go are in the vertical format.

 

HOW IS MUSCLE POWER MEASURED?

Wabbit plans to combine the normal gameplay with specially manufactured objects such as squeeze balls, weights and therabands that can give the patients an even more immersive experience of actually holding a tangible object. These objects will also help us record muscle resistance and give the patients some actually resistance that is essential for some exercises.

 The same objects would help us in measuring muscle power and reporting it back to the patient for a holistic progress report.

The same objects would help us in measuring muscle power and reporting it back to the patient for a holistic progress report.

 In the first whiteboarding session, we came up with the idea of using objects to simulate the tangible experience.

In the first whiteboarding session, we came up with the idea of using objects to simulate the tangible experience.

 
 

IMPACT

SOME EXCITING NEWS

Wabbit was been chosen as one of the top 5 finalists amongst thousands of entries in the National Geographic Chasing Genius Challenge. 

Screen Shot 2018-03-17 at 3.47.42 PM.png
 

NEXT STEPS

If we get a chance to take Wabbit forward, the first thing we would like to figure if how the technical limitations that come with augmented reality, example depth perception, drifting etc. would affect the gameplay and the experience for a patient.

We would also like to get real data on how much difference the game actions are making on the patient's motor skills and whether we are able to measure them effectively using the specially manufactured objects. The object studies can be conducted my using placeholder squeeze balls and thera bands etc. and will not involve a high cost of testing.