Gesture based Serious Games for Physical Therapy

The Problem:

·  Physiotherapy exercises are boring.

·  Forcing young disabled children to perform exercises can be specially challenging.

·  Two thirds of the patients do not perform exercises correctly at home.

·  A therapist has no way of knowing if a patient performed exercises at home correctly or not.

·  There is no way for a therapist to correct a person performing exercise at home.

·  Recovery in many patients is slow and cannot be measured accurately by instruments currently in use.

The Solution:

·  We have developed an online flying game that can be played by a patient using hand and body gestures.

·  The gestures used to play the game are the ones required to perform physical therapy.

·  The therapist can customize the game with required gestures and difficulty level, depending on the current state of the patient.

·  Off the shelf multi-sensory environment will allow patients to play the games while the kinematic data will be available to the doctor for further analysis and deciding quality of improvement.

The Advantages:

·  Immersive and engaging game play removes boredom from exercise.

·  Difficulty level of the game can be changed automatically by the system or remotely by the therapist according to the recovery behavior of each patient.

·  Visual feedback allows the patient to correct the exercise immediately. Wrong movements do not score points.

·  Powerful gesture detection sensors record minute differences in joint movements and report full details of the exercise performed at home to the therapist.

·  Advanced analytics can provide detailed reports, graphs and charts regarding recovery.

·  Social media integration can help people recover quickly by sharing their experiences and supporting each other.

·  Big Data back-end can store data generated by millions of users simultaneously.

·  Computational Intelligence can be used to study and discover new recovery parameters and patterns based on Big Data analysis.

Value to the Kingdom of Saudi Arabia:

·  Nearly 2 million disabled people in Saudi Arabia can get benefit from this invention.

·  People living in distant areas of the country can get high quality medical services remotely.

·  Multilingual interface can be very helpful in dealing with patients from different nationalities.

Patents Related to this Demo:

1. A Synchronized Multimedia Engine for Therapy Recording, Playback, Annotation and Query in Big Data Environment

2. A web-based non-invasive way of detecting, tracking, recognizing and 3d visualizing live kinematic hand therapy data of hemiplegic patient

3. An electromyography (emg) signal and augmented reality based occupational hand therapy framework for people with hemiplegia

4. A gesture controlled mobile based camera-less smart home management solution for providing assisted living services to physically challenged individuals with low mobility

5. A non-invasive healthgis multimedia serious game therapy environment for providing ranges for motion of full-body joints to hemiplegic patient through forward and inverse kinematics

6. System and method for detecting and visualizing live kinetic and kinematic data for the musculoskeletal system

7. BCI (Brain Computer Interface) based VR (Virtual Reality) Therapy System for Paralyzed patients to recover muscle functions

8. Therapist-Assistant: Virtual Reality based Live Augmented Skeletal Therapy Management System

9. A serious game generator for disability persons based on patient disability level and/or therapist recommendation using non-invasive motion sensors

10. Gestpass: a multimodal gesture-based login system for providing assisted living services to physically challenged individuals

11. Mapping virtual 3d human model with the joint-motions of a disabled: a web-based 3d human model with natural user interface-based interaction capability for designing physical therapy

12. Immersive telepresence as occupational therapy for hemiplegic patients through VR technology

13. Physical Therapy with Mixed Reality