# US 2020/0402420 A1 · Computing technologies for diagnosis and therapy of language-related disorders

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402420 A1
- Document ID: US20200402420A1
- Status: Published application
- Kind: A1
- Application number: 17/014,946
- Filed: 2020-09-08
- Published: 2020-12-24
- Earliest claimed: 2013-10-31
- Assignees: Spring River Holdings LLC
- Applicants: Spring River Holdings LLC
- Inventors: Pau-San Haruta; Charisse Si-Fei Haruta; Kieran Bing-Fei Haruta
- Web page: https://patentgrep.com/patent/US20200402420A1

## CPC

- G09B 19/04: Speaking (with audible presentation of the material to be studied G09B5/04)
- A61B 5/16: Devices for psychotechnics (using teaching or educational appliances G09B1/00 - G09B7/00); Testing reaction times {; Devices for evaluating the psychological state}
- A61B 5/4088: {Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia}
- G09B 5/00: Electrically-operated educational appliances (working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F)
- G09B 7/00: Electrically-operated teaching apparatus or devices working with questions and answers (mechanically operated G09B3/00; computing arrangements G06F)
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 20/70: relating to mental therapies, e.g. psychological therapy or autogenous training
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

The present disclosure relates to computing technologies for diagnosis and therapy of language-related disorders. Such technologies enable computer-generated diagnosis and computer-generated therapy delivered over a network to at least one computing device. The diagnosis and therapy are customized for each patient through a comprehensive analysis of the patient's production and reception errors, as obtained from the patient over the network, together with a set of correct responses at each phase of evaluation and therapy.

## Claims

### Claim 1

A method of therapy, the method comprising:
receiving, by a server, over a network, in real-time, a user request from a client in communication with a camera and operated by a user having a face with a pair of eyes;
hosting, by the server, over the network, in real-time, responsive to the user request, a secure session with the client; and
during the secure session:
selecting, by the server, a first node containing a first type value in a first graph based on a first comparison of a first matrix specific to the user against a second matrix not specific to the user;
selecting, by the server, a second node containing a first audio task or a first image task in a second graph based on the first type value and the first comparison;
serving, by the server, over the network, the first audio task or the first image task from the second node to the client;
instructing, by the server, over the network, the client to activate the camera such that the camera captures a video depicting the face with the pair of eyes while the first audio task or the first image task is output via the client;
receiving, by the server, over the network, a user input and the video from the client responsive to the first audio task or the first image task being output via the client;
writing, by the server, the user input into the first matrix;
identifying, by the server, the pair of eyes in the video;
determining, by the server, an eye gaze value or a blinking rate based on the pair of eyes identified in the video;
writing, by the server, the eye gaze value or the blinking rate into the first matrix;
selecting, by the server, the first node or a third node containing a second type value in the first graph based on a second comparison of the user input and the eye gaze value or the blinking rate in the first matrix against the second matrix, wherein the second type value is different from the first type value, wherein the second comparison is after the first comparison;
selecting, by the server, the second node or a fourth node containing a second audio task or a second image task in the second graph based on the first type value or the second type value and the second comparison, wherein the second audio task or the second image task is respectively different from the first audio task or the first image task;
generating, by the server, a corrective or evaluative feedback message containing the second audio or the second image; and
serving, by the server, the corrective or evaluative feedback message to the client such that the corrective or evaluative feedback message is output to the user via the client.

### Claim 2 (depends on 1)

The method of claim 1, wherein the user has a pair of lips, and further comprising:
during the secure session:
instructing, by the server, over the network, the client to activate the camera such that the camera captures the video depicting the face with the pair of eyes and the pair of lips while the first audio task or the first image task is output via the client;
identifying, by the server, the pair of lips in the video;
determining, by the server, a lip movement value based on the pair of lips identified in the video;
writing, by the server, the lip movement value into the first matrix;
selecting, by the server, the first node or the third node containing the second type value in the first graph based on the second comparison of the user input, the lip movement value, and the eye gaze value or the blinking rate in the first matrix against the second matrix.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
during the secure session:
selecting, by the server, the first node or the third node containing the second type value in the first graph based on the second comparison of the user input, the lip movement value, and the eye gaze value in the first matrix against the second matrix.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
during the secure session:
selecting, by the server, the first node or the third node containing the second type value in the first graph based on the second comparison of the user input, the lip movement value, and the blinking rate in the first matrix against the second matrix.

### Claim 5 (depends on 2)

The method of claim 2, further comprising:
during the secure session:
selecting, by the server, the first node or the third node containing the second type value in the first graph based on the second comparison of the user input, the lip movement value, the eye gaze value, and the blinking rate in the first matrix against the second matrix.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
during the secure session:
serving, by the server, the corrective or evaluative feedback message to the client such that the corrective or evaluative feedback message is unobtrusively output to the user.

### Claim 7 (depends on 1)

The method of claim 1, wherein the client is a wearable.

### Claim 8 (depends on 7)

The method of claim 7, wherein the client is an eyewear unit or an optical head-mounted display (OHMD) having the camera.

### Claim 9 (depends on 1)

The method of claim 1, wherein the client is a phone or a tablet that hosts the camera, wherein the camera is a front-facing camera.

### Claim 10 (depends on 1)

The method of claim 1, wherein the client runs a browser through which the secure session takes place.

### Claim 12 (depends on 1)

The method of claim 1, wherein the client is in communication with a vibrator configured for a bone conduction to the user, and further comprising:
serving, by the server, the corrective or evaluative feedback message to the client such that the corrective or evaluative feedback message is output to the user via the client instructing the vibrator to output the corrective or evaluative feedback message via the bone conduction.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
during the secure session:
identifying, by the server, the face in the video;
determining, by the server, an emotion of the user based on the face being identified in the video;
writing, by the server, the emotion into the first matrix;
selecting, by the server, the first node or the third node containing the second type value in the first graph based on the second comparison of the user input, the emotion, and the eye gaze value or the blinking rate in the first matrix against the second matrix.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
during the secure session:
identifying, by the server, the face in the video;
extracting, by the server, a geometrical feature of the face;
producing, by the server, a temporal profile for each of a plurality of movements of the face in the video;
writing, by the server, the temporal profiles to the first matrix such that an iterative feedback loop is formed during the first comparison or the second comparison.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
during the secure session:
selecting, by the server, the first node or the third node containing the second type value in the first graph based on the second comparison of the user input and the eye gaze value in the first matrix against the second matrix.

### Claim 16 (depends on 1)

The method of claim 1, further comprising:
during the secure session:
selecting, by the server, the first node or a third node containing a second type value in the first graph based on the second comparison of the user input and the blinking rate in the first matrix against the second matrix.

### Claim 17 (depends on 1)

The method of claim 1, further comprising:
during the secure session:
selecting, by the server, the first node or a third node containing a second type value in the first graph based on the second comparison of the user input, the eye gaze value, and the blinking rate in the first matrix against the second matrix.

### Claim 18 (depends on 1)

The method of claim 1, further comprising:
during the secure session:
receiving, by the server, a pulse of the user from the client responsive to the first audio task or the first image task being output via the client, wherein the pulse is measured while the first audio task or the first image task is output via the client;
writing, by the server, the pulse into the first matrix;
selecting, by the server, the first node or the third node containing the second type value in the first graph based on the second comparison of the user input, the pulse, and the eye gaze value or the blinking rate in the first matrix against the second matrix.

### Claim 19 (depends on 1)

The method of claim 1, wherein the client is in communication with a videogame controller, wherein the user input is via the videogame controller.

### Claim 20 (depends on 1)

The method of claim 1, wherein the corrective or evaluative feedback message containing the second audio or the second image is informative of a symptom of a language-related disorder with which the user is diagnosed.
