Systems, apparatus and methods for delivery and augmentation of behavior modification therapy and teaching

A system for providing a lesson to a user with an autism spectrum disorder to teach communication skills or behaviors, comprising:

• a sensor apparatus, the sensor apparatus generating an output indicative of a user's actions, the sensor apparatus including a visual sensor having a field of view directed toward the user and comprising a depth sensor operative to image and assess discreet objects, picture cards, or structures within a three-dimensional space with which the user appears to interact within the field of view, the sensor apparatus operative to assign a depth and size to the object, picture card, or structure, the sensor apparatus further including a motion tracking sensor, facial feature recognition, eye tracking, and voice recognition,
• memory for storing an individual user profile including at least a developmental assessment profile, a treatment plan, and progress data,
• an image display, and
• a processor and associated storage comprising a computing platform in communication with the sensor apparatus, the memory, and the image display, the computing platform including:
• (i) a personalization module in communication with the individual user profile and operative to identify a communication skill or behavioral lesson based on user need,
• (ii) a feedback module, and
• (iii) a prompting module in communication with the feedback module and the personalization module, and operative to select modes of interaction including modeling, gestural, positional, visual, and verbal modes of interaction;
• the computing platform operative to generate user feedback and reinforcement of a communication behavior or skill by:
• (a) selecting an application lesson based at least in part on (1) the developmental assessment profile and (2) application of the personalization module to customize the lesson based at least in part on a user's preference stored in the individual user profile, the application lesson adapted to elicit a desired user response to teach verbal and non-verbal communication skills or behaviors,
• (b) generating a display of an application lesson for presentation to the user to elicit the desired user response, the application lesson including picture cards having pictures thereon for communication,
• (c) displaying the application lesson to the user on the image display, including requesting the user to find one of the picture cards having a picture thereon for communication, and to hold the picture card to be identified;
• (d) receiving from the sensor apparatus a user response, the user response including holding the picture card for imaging by the sensor apparatus,
• (e) analyzing at the feedback module the user response to identify the picture on the picture card and to discern response characteristics of the user, including the user's eye gaze direction, facial expression, body orientation, verbal response, and response time,
• (f)comparing the user response with the desired user response, including identifying the picture on the picture card held by the user,
• (g) performing feedback analysis to determine the effect of the lesson, including determining the user's attentive status and initiating a specific action or lesson in response to identification of the picture on the picture card held by the user,
• (h) generating a prompt to regain the user's attention,
• (i) generating a second display on the image display for presentation to the user to elicit the desired user response;
• and
• (j) tracking which prompting method is more effective for the user by measuring and tracking reaction rate, response time, engagement level, emotional state and time to skill acquisition.