Sudipto Sur, Chief Technology Officer at Reflexion Health07.12.19
It is surprising how resistant people are to change, particularly when it requires additional work. Even in cases where it is apparent that change is necessary to survive, humans often resist it. This is likely why the “gamification” of healthcare has caught the interest and attention of so many professionals. If only you could make being healthy fun, perhaps even slightly addictive, you could compel populations to help themselves. However, in healthcare, unlike gaming, it is not only important that a solution be fun, it must also be useful.
The Reflexion Health VERA (Virtual Exercise Rehabilitation Assistant) tele-rehabilitation system allows patients to perform physiotherapy at home in a monitored environment, providing guidance and keeping their clinicians updated on the progress they are making. While the ability of patients to recover at home brings various advantages, including allowing patients to avoid travel to and from clinics at a time when their mobility may be severely impaired, it also means that the solution must be compelling enough for patients to continue to use it day after day. This is where game mechanics come into play.
Motivating the ‘Player Patient’: Game Dynamics
People’s enjoyment of gaming comes from escaping reality, the desire for competence and control paired with frequent affirmation of that competence, the desire for independence or autonomy, and the desire for relating to others as provided by multiplayer games and the communities that support them. That the cost of failure is low is an additional attractor. In game design, these are largely considered intrinsic motivations related to interest and enjoyment in the “gaming” task itself. Most games also provide extrinsic motivations like points, badges, rewards, or levels. Gamification experts believe that although intrinsic motivations make a game absorbing, extrinsic motivations can be used to increase the intrinsic. Well-designed games consider both.
The epitome of the gaming environment is one that provides engagement without frustration. A challenge is one common way to provide engagement, but the shift from challenge to frustration can be a fine line, particularly in a situation where the user has acute or chronic disadvantages or disabilities. Similarly, providing the ability for exhibiting player agency is understood to be a cornerstone of superlative gaming mechanics. However, in a rehabilitation situation, “free” parameters are difficult. Exercises must be performed within a narrow range to be effective.
Incorporating game mechanics in our rehabilitation applications presents our patients with an experience that they will crave sufficiently to justify the investment of the patient in the application. When it is known that there is a finite term to the engagement, we must question what the means available are to drive the patient to feel that the investment has a payoff. Successful games often target addictiveness, but in our scenario a successful experience should result in faster recovery and therefore a shorter term of engagement.
Use of Game Dynamics in Our Application
There are principles we use to decide what dynamics of gaming we need to incorporate into our systems. One of our tenets is that patients should be motivated by the idea of getting better, but in a derivative way. People can relate more to a formulation of “getting better” in terms of a meaningful ability rather than the fixing of a physiological condition. For example, going through a knee replacement surgery and recovering the use of a limb is better articulated as the ability to go for a walk than focusing on the mechanics of the knee itself. Where possible, our applications use the idea of meaningful abilities for goal setting and tracking to set a purpose for the patient interacting with our application. In gaming, “purpose” is considered one of the four core drivers of intrinsic motivation. Some of the official, clinical surveys of course do not allow for such an approach, but there are other interactions where we utilize such an approach to drive patient engagement and adherence.
Another critical element that we leverage from gaming is the use of virtual three-dimensional (3D) representations on two-dimensional (2D) screens. The use of highly visual, virtual 3D environments as an interactive space has been pioneered by the computer gaming field. The gaming industry has developed the approaches, the technology, the techniques and the tools required to create and deploy high-fidelity virtual 3D spaces. This turns out to be a very natural fit for virtual physical rehabilitation that delivers high value for guidance as well as tracking. Exercises naturally exist in three dimensional spaces and require the language of three dimensions to describe and perform them. We leverage the ability provided by gaming to provide our patients with 3D renderings and imagery on screen. We can show them how to perform exercises from multiple viewpoints by using virtual moving cameras in the virtual 3D environment or by using virtual mirrors to provide a simultaneous multi-view representation of an exercise.
Similarly, our use of 3D data capture also has a gaming origin. The Microsoft Kinect we (and quite a few other medical applications) use, owes its origin to gaming. The price point at which this technology is accessible has been driven largely by what customers in the gaming world are willing to pay, but benefits us greatly as we try to make our product an affordable and accessible healthcare solution. As mentioned previously, because physical exercises are a 3D activity, we require an accurate source of data to track and monitor the performance of these exercises. A device like the Kinect fulfills this requirement admirably. Although it is not the most accurate system available, it is accurate enough for the task, simple to set up, provides a featureful programming interface and remains affordable.
The aspect of simplicity is worth discussing in greater detail. Imagine a senior setting up a 3D imaging system that requires multiple sensors laid out in an optimal pattern and calibrated by some means before being usable. Now imagine having to put several markers on their person, before they can be tracked by the cameras. Some parts of this could be automated to an extent, but overall it would not be practical or pragmatic. Compare this with the Kinect, which is a switch on and go system: single sensor, no calibration and no markers. Driven by the needs of the gaming community, the usability and utility of this type of system is tremendous in our space and demographics. Since the launch of the Kinect, other devices which have the same functionality have been launched that may provide the same ease of use and cost characteristics.
Additionally, the 3D data from the Kinect allows us to represent the patient in a virtual 3D exercise space on the screen and provide instantaneous visual feedback to them: an example of augmented reality. This interface is crucial because patients need to position themselves appropriately and transition between the 3D poses required to successfully perform an exercise.
Another concept we have borrowed from the gaming world is that of an avatar. Our rehabilitation assistant VERA has a namesake avatar who is a guide and partner in the patient’s rehabilitation experience. It is understood that avatars personalize learning by forming a relationship with learners. To be effective, the avatar must be designed carefully with the target audience in mind. It should be relatable but authoritative. In our scenario the goal of the avatar is to provide an empathetic interface for the complex software, building on the idea that patients experience it as a caring agent based on the human appearance. In practice this has worked out very well.
Divergence from the Gaming Approach
We have found many aspects of gaming useful to our approach, but elements of gaming related to user agency are more difficult to translate into healthcare applications. Recovery from serious health issues and medical procedures have prescriptions and processes that are mandated by the accepted standard of care. Deviations from these standards are not tolerated for good reasons, and indeed being at variance with them can result in violation of regulatory compliance. The path through an episode of care is typically regimented and for an FDA cleared system it is difficult to provide the means to a user to modify that path. Similarly, we find that too many options for patient agency can result in lower compliance by the patient. A patient workflow presented as a recipe, facilitated by an avatar has proven to be the best interface.
Also, there are limitations to the kind of community we can provide to patients. Our application suite creates a community between the patient and their caregivers with full access to data and real-time audio-visual interactions. However, given the nature of the data we deal with, it is imperative that we restrict that access to those in an official care team, and no one else. We must comply with the Health Insurance Portability and Accountability Act (HIPAA) security and privacy rules, which, in the U.S., establish the standards for protection of health information. Our applications and infrastructure are required to undergo rigorous testing for safety, security and integrity.
The quality and quantity of data collected by our systems are also very different from that performed by typical game software. We need to understand deeply the mechanics by which we collect data to use them appropriately and to create the most effective visualizations so that clinicians are empowered to make decisions based on what we present to them. Our approach to clinical visualization is very different from that used in games where it may be used to provide extrinsic motivations for players to continue playing. Clinical visualizations are driven by the need to be accurate and clear while also being concise, so they may be efficiently reviewed by clinicians and acted upon with minimal chance of error.
It is important to mention in this article the role of network communication. A surprisingly high percentage of problems are a result of poor network communications. For online games, network connectivity is also an important consideration with both bandwidth and latency being major factors. For our application, we are more concerned with bandwidth and stability. We go to extraordinary lengths to provide reliable connectivity for our patient kits because of the criticality we attach to not interrupting a rehabilitation program. In the event of a network disruption, we provide the ability to continue progress on the rehabilitation program. Patients can follow their therapy programs during network outages. Changes are synchronized on the restoration of communication. Data is uploaded using specialized systems which guarantee upload over intermittent and low bandwidth connections. Integrity and safety of the data is of paramount importance. Any data with privacy implications is encrypted at rest and during transmission. No such data is ever stored on the disk from memory without being encrypted. In our field, these aspects are as important as any gamification features we implement.
Conclusion
Gaming approaches and technologies provide a great set of tools that help us create systems that can be used effectively for the care of recovering patients. Games have dynamics which are specifically designed for engagement, some of which can be leveraged to make recovery for patients more convenient, motivating, and meaningful. However, a patient undergoing physical therapy for their recovery is driven by a different purpose, and has considerations that may differ from those of gamers. Further, there are requirements for developers driven by compliance to regulatory regimes which constrain medical applications. Compared to most computer games used for entertainment, the burden of testing for quality, safety, security and integrity is far larger for FDA cleared software: a necessary and serious, time-consuming and expensive undertaking. It is important to understand the target users and the medical environment before setting out to gamify healthcare applications. The best solutions pick the gamification ideas that harmonize well without jeopardizing the larger goal of doing what is best for the patient and their recovery while also understanding the needs of the clinical team that supports that recovery.
Sudipto Sur, Ph.D., is the chief technology officer at Reflexion Health where he leads efforts in technological development, innovation, and growth, serving as the key strategist in expanding the company’s platform to reach a larger, more diverse patient population. With a long history of pioneering solutions in robotics, applied computer vision, medical device development and diagnostics, Dr. Sur brings more than 20 years of engineering, technology and management experience to Reflexion Health.
The Reflexion Health VERA (Virtual Exercise Rehabilitation Assistant) tele-rehabilitation system allows patients to perform physiotherapy at home in a monitored environment, providing guidance and keeping their clinicians updated on the progress they are making. While the ability of patients to recover at home brings various advantages, including allowing patients to avoid travel to and from clinics at a time when their mobility may be severely impaired, it also means that the solution must be compelling enough for patients to continue to use it day after day. This is where game mechanics come into play.
Motivating the ‘Player Patient’: Game Dynamics
People’s enjoyment of gaming comes from escaping reality, the desire for competence and control paired with frequent affirmation of that competence, the desire for independence or autonomy, and the desire for relating to others as provided by multiplayer games and the communities that support them. That the cost of failure is low is an additional attractor. In game design, these are largely considered intrinsic motivations related to interest and enjoyment in the “gaming” task itself. Most games also provide extrinsic motivations like points, badges, rewards, or levels. Gamification experts believe that although intrinsic motivations make a game absorbing, extrinsic motivations can be used to increase the intrinsic. Well-designed games consider both.
The epitome of the gaming environment is one that provides engagement without frustration. A challenge is one common way to provide engagement, but the shift from challenge to frustration can be a fine line, particularly in a situation where the user has acute or chronic disadvantages or disabilities. Similarly, providing the ability for exhibiting player agency is understood to be a cornerstone of superlative gaming mechanics. However, in a rehabilitation situation, “free” parameters are difficult. Exercises must be performed within a narrow range to be effective.
Incorporating game mechanics in our rehabilitation applications presents our patients with an experience that they will crave sufficiently to justify the investment of the patient in the application. When it is known that there is a finite term to the engagement, we must question what the means available are to drive the patient to feel that the investment has a payoff. Successful games often target addictiveness, but in our scenario a successful experience should result in faster recovery and therefore a shorter term of engagement.
Use of Game Dynamics in Our Application
There are principles we use to decide what dynamics of gaming we need to incorporate into our systems. One of our tenets is that patients should be motivated by the idea of getting better, but in a derivative way. People can relate more to a formulation of “getting better” in terms of a meaningful ability rather than the fixing of a physiological condition. For example, going through a knee replacement surgery and recovering the use of a limb is better articulated as the ability to go for a walk than focusing on the mechanics of the knee itself. Where possible, our applications use the idea of meaningful abilities for goal setting and tracking to set a purpose for the patient interacting with our application. In gaming, “purpose” is considered one of the four core drivers of intrinsic motivation. Some of the official, clinical surveys of course do not allow for such an approach, but there are other interactions where we utilize such an approach to drive patient engagement and adherence.
Another critical element that we leverage from gaming is the use of virtual three-dimensional (3D) representations on two-dimensional (2D) screens. The use of highly visual, virtual 3D environments as an interactive space has been pioneered by the computer gaming field. The gaming industry has developed the approaches, the technology, the techniques and the tools required to create and deploy high-fidelity virtual 3D spaces. This turns out to be a very natural fit for virtual physical rehabilitation that delivers high value for guidance as well as tracking. Exercises naturally exist in three dimensional spaces and require the language of three dimensions to describe and perform them. We leverage the ability provided by gaming to provide our patients with 3D renderings and imagery on screen. We can show them how to perform exercises from multiple viewpoints by using virtual moving cameras in the virtual 3D environment or by using virtual mirrors to provide a simultaneous multi-view representation of an exercise.
Similarly, our use of 3D data capture also has a gaming origin. The Microsoft Kinect we (and quite a few other medical applications) use, owes its origin to gaming. The price point at which this technology is accessible has been driven largely by what customers in the gaming world are willing to pay, but benefits us greatly as we try to make our product an affordable and accessible healthcare solution. As mentioned previously, because physical exercises are a 3D activity, we require an accurate source of data to track and monitor the performance of these exercises. A device like the Kinect fulfills this requirement admirably. Although it is not the most accurate system available, it is accurate enough for the task, simple to set up, provides a featureful programming interface and remains affordable.
The aspect of simplicity is worth discussing in greater detail. Imagine a senior setting up a 3D imaging system that requires multiple sensors laid out in an optimal pattern and calibrated by some means before being usable. Now imagine having to put several markers on their person, before they can be tracked by the cameras. Some parts of this could be automated to an extent, but overall it would not be practical or pragmatic. Compare this with the Kinect, which is a switch on and go system: single sensor, no calibration and no markers. Driven by the needs of the gaming community, the usability and utility of this type of system is tremendous in our space and demographics. Since the launch of the Kinect, other devices which have the same functionality have been launched that may provide the same ease of use and cost characteristics.
Additionally, the 3D data from the Kinect allows us to represent the patient in a virtual 3D exercise space on the screen and provide instantaneous visual feedback to them: an example of augmented reality. This interface is crucial because patients need to position themselves appropriately and transition between the 3D poses required to successfully perform an exercise.
Another concept we have borrowed from the gaming world is that of an avatar. Our rehabilitation assistant VERA has a namesake avatar who is a guide and partner in the patient’s rehabilitation experience. It is understood that avatars personalize learning by forming a relationship with learners. To be effective, the avatar must be designed carefully with the target audience in mind. It should be relatable but authoritative. In our scenario the goal of the avatar is to provide an empathetic interface for the complex software, building on the idea that patients experience it as a caring agent based on the human appearance. In practice this has worked out very well.
Divergence from the Gaming Approach
We have found many aspects of gaming useful to our approach, but elements of gaming related to user agency are more difficult to translate into healthcare applications. Recovery from serious health issues and medical procedures have prescriptions and processes that are mandated by the accepted standard of care. Deviations from these standards are not tolerated for good reasons, and indeed being at variance with them can result in violation of regulatory compliance. The path through an episode of care is typically regimented and for an FDA cleared system it is difficult to provide the means to a user to modify that path. Similarly, we find that too many options for patient agency can result in lower compliance by the patient. A patient workflow presented as a recipe, facilitated by an avatar has proven to be the best interface.
Also, there are limitations to the kind of community we can provide to patients. Our application suite creates a community between the patient and their caregivers with full access to data and real-time audio-visual interactions. However, given the nature of the data we deal with, it is imperative that we restrict that access to those in an official care team, and no one else. We must comply with the Health Insurance Portability and Accountability Act (HIPAA) security and privacy rules, which, in the U.S., establish the standards for protection of health information. Our applications and infrastructure are required to undergo rigorous testing for safety, security and integrity.
The quality and quantity of data collected by our systems are also very different from that performed by typical game software. We need to understand deeply the mechanics by which we collect data to use them appropriately and to create the most effective visualizations so that clinicians are empowered to make decisions based on what we present to them. Our approach to clinical visualization is very different from that used in games where it may be used to provide extrinsic motivations for players to continue playing. Clinical visualizations are driven by the need to be accurate and clear while also being concise, so they may be efficiently reviewed by clinicians and acted upon with minimal chance of error.
It is important to mention in this article the role of network communication. A surprisingly high percentage of problems are a result of poor network communications. For online games, network connectivity is also an important consideration with both bandwidth and latency being major factors. For our application, we are more concerned with bandwidth and stability. We go to extraordinary lengths to provide reliable connectivity for our patient kits because of the criticality we attach to not interrupting a rehabilitation program. In the event of a network disruption, we provide the ability to continue progress on the rehabilitation program. Patients can follow their therapy programs during network outages. Changes are synchronized on the restoration of communication. Data is uploaded using specialized systems which guarantee upload over intermittent and low bandwidth connections. Integrity and safety of the data is of paramount importance. Any data with privacy implications is encrypted at rest and during transmission. No such data is ever stored on the disk from memory without being encrypted. In our field, these aspects are as important as any gamification features we implement.
Conclusion
Gaming approaches and technologies provide a great set of tools that help us create systems that can be used effectively for the care of recovering patients. Games have dynamics which are specifically designed for engagement, some of which can be leveraged to make recovery for patients more convenient, motivating, and meaningful. However, a patient undergoing physical therapy for their recovery is driven by a different purpose, and has considerations that may differ from those of gamers. Further, there are requirements for developers driven by compliance to regulatory regimes which constrain medical applications. Compared to most computer games used for entertainment, the burden of testing for quality, safety, security and integrity is far larger for FDA cleared software: a necessary and serious, time-consuming and expensive undertaking. It is important to understand the target users and the medical environment before setting out to gamify healthcare applications. The best solutions pick the gamification ideas that harmonize well without jeopardizing the larger goal of doing what is best for the patient and their recovery while also understanding the needs of the clinical team that supports that recovery.
Sudipto Sur, Ph.D., is the chief technology officer at Reflexion Health where he leads efforts in technological development, innovation, and growth, serving as the key strategist in expanding the company’s platform to reach a larger, more diverse patient population. With a long history of pioneering solutions in robotics, applied computer vision, medical device development and diagnostics, Dr. Sur brings more than 20 years of engineering, technology and management experience to Reflexion Health.
