# US 2024/0186009 A1 · Systems and methods of providing deep learning based neurocognitive impairment evaluation using extended reality

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0186009 A1
- Document ID: US20240186009A1
- Status: Published application
- Kind: A1
- Application number: 18/530,965
- Filed: 2023-12-06
- Published: 2024-06-06
- Earliest claimed: 2022-12-06
- Assignees: The Board of Trustees of the University of Illinois
- Applicants: The Board of Trustees of the University of Illinois
- Inventors: Inki Kim; Adam Cross
- Web page: https://patentgrep.com/patent/US20240186009A1

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G16H 20/70: relating to mental therapies, e.g. psychological therapy or autogenous training
- G16H 40/67: for remote operation
- G16H 50/30: for calculating health indices; for individual health risk assessment
- G16H 50/50: for simulation or modelling of medical disorders
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2210/41: Medical
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods of deep-learning neurocognitive impairment evaluation using extended reality. One system includes an electronic processor configured to generate an extended reality user environment (“XR-UX”) for display via a user device, the XR-UX including a first augmented object configured to change position within the XR-UX and a second augmented object configured to dynamically change position within the XR-UX based on a position of the user device. The electronic processor is also configured to control the first augmented object to traverse a path within the XR-UX. The electronic processor is also configured to receive motion data describing movement of the user device. The electronic processor is also configured to determine a medical classification based on the physical movement of the user device in relation to the path. The electronic processor is also configured to generate and transmit an assessment report for display, the assessment report including the medical classification.

## Claims

### Claim 1

A system for providing neurocognitive impairment simulation using extended reality, the system comprising:
one or more electronic processors configured to:
generate an extended reality user environment (“XR-UX”) for display via a user device, the XR-UX including a first augmented object configured to change position within the XR-UX and a second augmented object configured to dynamically change position within the XR-UX based on a position of the user device;
control the first augmented object to traverse a path within the XR-UX;
receive motion data describing physical movement of the user device;
determine a medical classification based on the physical movement of the user device in relation to the path; and
generate and transmit an assessment report for display, the assessment report including the medical classification.

### Claim 2 (depends on 1)

The system of claim 1, wherein the one or more electronic processors are configured to receive environment data from a sensor of the user device, wherein the environment data describes an external environment of the user device.

### Claim 3 (depends on 2)

The system of claim 2, wherein the one or more electronic processors are configured to control the first augmented object based on the environment data.

### Claim 4 (depends on 1)

The system of claim 1, wherein the one or more electronic processors are configured to dynamically adapt the path traversed by the first augmented object based on the physical movement of the user device.

### Claim 5 (depends on 1)

The system of claim 1, wherein the one or more electronic processors are configured to determine the medical classification using a classification model trained using machine learning.

### Claim 6 (depends on 1)

The system of claim 1, wherein the one or more electronic processors are configured to transmit the assessment report to another user device associated with a medical professional.

### Claim 7 (depends on 1)

The system of claim 1, wherein the one or more electronic processors are further configured to:
determine a performance metric based on the physical movement of the user device in relation to the path.

### Claim 8 (depends on 7)

The system of claim 7, wherein the one or more electronic processors are further configured to:
dynamically adapt the path traversed by the first augmented object based on the performance metric.

### Claim 9 (depends on 1)

The system of claim 1, wherein the medical classification is indicative of a neurocognitive impairment of a user of the user device.

### Claim 10 (depends on 1)

The system of claim 1, wherein the one or more electronic processors are configured to:
control the second augmented object within the XR-UX to dynamically change position based on the physical movement of the user device.

### Claim 11

A method for providing neurocognitive impairment simulation using extended reality, the method comprising:
generating, with one or more electronic processors, an extended reality user environment (“XR-UX”) for display via a user device, the XR-UX including a first augmented object and a second augmented object;
controlling, with the one or more electronic processors, the first augmented object to traverse a path within the XR-UX;
receiving, with the one or more electronic processors, motion data describing movement of the user device;
controlling, with the one or more electronic processors, the second augmented object within the XR-UX to dynamically change position based on the movement of the user device;
determining, with the one or more electronic processors, a medical classification based on the movement of the user device in relation to the path; and
generating and transmitting, with the one or more electronic processors, an assessment report for display, the assessment report including the medical classification.

### Claim 12 (depends on 11)

The method of claim 11, wherein generating the XR-UX includes:
receiving, from a sensor of the user device, environment data describing an external surrounding of the user device,
wherein the XR-UX is generated such that the XR-UX reflects the external surrounding of the user device.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
receiving environment data from a sensor of the user device, wherein the environment data describes an external environment of the user device; and
generating, based on the environment data, a scene mesh of the external environment of the user device,
wherein controlling the first augmented object to traverse the path includes controlling the first augmented object to traverse the path based on the scene mesh.

### Claim 14 (depends on 11)

The method of claim 11, further comprising:
determining, based on the movement of the user device in relation to the path, a subsequent path for the first augmented object;
controlling the first augmented object to traverse the subsequent path; and
receiving subsequent motion data describing subsequent movement of the user device,
wherein determining the medical classification includes determining the medical classification based on the subsequent movement of the user device in relation to the subsequent path.

### Claim 15 (depends on 14)

The method of claim 14, wherein determining the subsequent path for the first augmented object includes:
receiving, with the one or more electronic processors, environment data describing an external environment of the user device; and
determining the subsequent path for the first augmented object based on the environment data and the motion data.

### Claim 16

A non-transitory, computer-readable medium storing instructions that, when executed by an electronic processor, perform a set of functions, the set of functions comprising:
generating an extended reality user environment (“XR-UX”) for display via a user device, the XR-UX including a first augmented object configured to change position within the XR-UX and a second augmented object configured to dynamically change position within the XR-UX based on a position of the user device;
controlling the first augmented object to traverse a path within the XR-UX;
receiving motion data describing physical movement of the user device;
determining a medical classification based on the physical movement of the user device in relation to the path; and
generating and transmitting an assessment report for display, the assessment report including the medical classification.

### Claim 17 (depends on 16)

The computer-readable medium of claim 16, wherein controlling the first augmented object to traverse the path includes controlling the first augmented object to continuously change positions.

### Claim 18 (depends on 16)

The computer-readable medium of claim 16, wherein controlling the first augmented object to traverse the path includes controlling the first augmented object to move to a multi-dimensional position and upon reaching the multi-dimensional position, maintaining the multi-dimensional position until a subsequent change in position of the user device is detected.

### Claim 19 (depends on 11)

The computer-readable medium of claim 11, wherein controlling the first augmented object to traverse the path includes controlling the first augmented object to traverse a three-dimensional path.

### Claim 20 (depends on 11)

The computer-readable medium of claim 11, wherein controlling the first augmented object to traverse the path includes controlling a speed at which the first augmented object traverses the path.
