# US 2020/0138292 A1 · Non-Contact Breathing Activity Monitoring And Analyzing Through Thermal And CO2 Imaging

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0138292 A1
- Document ID: US20200138292A1
- Status: Published application
- Kind: A1
- Application number: 16/676,346
- Filed: 2019-11-06
- Published: 2020-05-07
- Earliest claimed: 2018-11-06
- Assignees: The Regents of the University of Colorado
- Applicants: The Regents of the University of Colorado, a body corporate
- Inventors: Min-Hyung Choi; Shane Transue; Ann Halbower
- Web page: https://patentgrep.com/patent/US20200138292A1

## CPC

- A61B 5/0878: {using temperature sensing means}
- A61B 5/0033: {Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room}
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/0816: {Measuring devices for examining respiratory frequency (measuring frequency of electric signals G01R23/00)}
- A61B 5/082: {Evaluation by breath analysis, e.g. determination of the chemical composition of exhaled breath (A61B5/083, A61B5/091 take precedence)}
- A61B 5/0873: {using optical means}
- A61B 5/091: Measuring volume of inspired or expired gases, e.g. to determine lung capacity
- A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles}
- G06T 5/10: using non-spatial domain filtering
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/11: Region-based segmentation
- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/50: for simulation or modelling of medical disorders
- A61B 5/01: Measuring temperature of body parts {; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue} (clinical contact thermometers G01K13/20)
- A61B 2560/0431: Portable apparatus, e.g. comprising a handle or case
- A61B 2562/0271: Thermal or temperature sensors
- A61B 2576/00: Medical imaging apparatus involving image processing or analysis (A61B1/00009, A61B6/52 and A61B8/52 take precedence)
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10048: Infrared image
- G06T 2207/20221: Image fusion; Image merging
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Various embodiments of the disclosed technology present a structural foundation for volumetric flow reconstructions for expiratory modeling enabled through multi-modal imaging for pulmonology. In some embodiments, this integrated multi-modal system includes infrared (IR) imaging, thermal imaging of carbon dioxide (CO2), depth imaging (D), and visible spectrum imaging. These multiple image modalities can be integrated into flow models of exhale behaviors enable the creation of three-dimensional volume reconstructions based on visualized CO2 distributions over time, formulating a four-dimensional exhale model which can be used to estimate various pulmonological traits (e.g., breathing rate, flow rate, exhale velocity, nose/mouth distribution, tidal volume estimation, and CO2 density distributions). Various embodiments also enable the accurate acquisition of numerous pulmonary metrics that are then stored within distributed systems for respiratory data analytics and feature extraction through deep learning embodiments.

## Claims

### Claim 1

A system for exhale gas imaging comprising:
an infrared thermal imaging device to perform a spectral band-pass filter tuned to limit incoming wavelengths to that relevant to the visualization of exhale gaseous flow;
an infrared imaging device that provides depth imaging;
a high resolution visible spectrum (RGB) imaging device; and
a computing device communicably coupled to the infrared thermal imaging device, the infrared imaging device and the high resolution visible spectrum (RGB) imaging device to perform fusion, respiratory analysis, and metric extraction.

### Claim 2 (depends on 1)

The system of claim 1, wherein the infrared thermal camera is tuned to a sub-interval of the infrared portion of the electromagnetic spectrum to identify a target infrared wavelength corresponding to a desired gas.

### Claim 3 (depends on 1)

This system of claim 1 wherein the computing device receives pixel intensity values from the infrared thermal imaging device and estimates measure fluid flow characteristics of the exhale behavior from the pixel intensity values.

### Claim 4 (depends on 3)

The system of claim 3, wherein the fluid flow characteristics are identified independently of exact pixel values and characterized by changes of apparently flow measurements to reconstruct 4D models of exhale episodes.

### Claim 5 (depends on 1)

The system of claim 1, wherein the computing device generates a 4D model composed of three-dimensional flow and volume information extracted through relative pixel measurements over time.

### Claim 6 (depends on 1)

The system of claim 1, wherein the computing device generates a representation of exhale behavior and measurements obtained independent of image content through the isolation and extraction of flow behaviors.

### Claim 7 (depends on 1)

The system of claim 1, wherein the computing device generates a localization and reconstruction of exhale flow behaviors by separating the exhale flow behaviors from identifiable objects within background images.

### Claim 8 (depends on 1)

The system of claim 1, wherein the computing device generates flow and volume reconstructions by processing a sequence of images that compose a video stream of thermal CO2, depth, and RGB images.

### Claim 9 (depends on 1)

The system of claim 1, wherein the thermal imaging device, infrared imaging device, and high resolution visible spectrum (RGB) imaging device are housed within a single housing that provides a targetable vision system.

### Claim 10 (depends on 9)

The system of claim 9, wherein an orientation and position of the single housing is defined through inverse kinematics within adjustable segments or through a ceiling mounted system that contains rotational freedom and an attached arm with adjustable linear movement.

### Claim 11 (depends on 1)

The system of claim 1, wherein the single housing includes a mobile or wearable form factor.

### Claim 12

A method for analyzing pulmonological function, the method comprising:
obtaining, using a first imaging device, a thermal image of an exhale flow of a subject; and
isolating pulmonological behavior of the subject by identifying and tracking exhale fluid dynamics exhibited by an intensity of pixels corresponding to CO2 gas flow behaviors collected by the first imaging device.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
identifying and tracking exhale behaviors from thermal objects surrounding the subject; and
constructing a three-dimensional (3D) representation of the exhale flow based on data collected via a depth camera that provides units to the volumetric measurements of the exhale flow.

### Claim 14 (depends on 12)

The method of claim 12, wherein pixel intensities from the first imaging device represent a coupled relationship between thermal energy and CO2 emissivity, and wherein the method further comprises localizing and extracting the exhale flow from a mixed signal that represents IR emission from thermal heat sources interfering with the exhale signal due to the combined infrared wavelengths detected from the background heat source and the energy of the exhaled CO2 region.

### Claim 15

A system comprising:
a thermal camera to take an image of an exhale of a patient;
a communications interface communicably coupled to the thermal camera to receive the image;
a processor;
a memory having instructions stored thereon that when executed by the processor cause the processor to generate a representation of the exhale of the patient.

### Claim 16 (depends on 15)

The system of claim 15, wherein the instructions when executed by the processor cause the processor to extract metrics for respiratory analysis.

### Claim 17 (depends on 17)

The system of claim 17, wherein the metrics include the reconstruction of exhale flows to obtain tidal volume estimates, the identification of the separation between nose and mouth exhale flows to measure nose/mouth distribution, and both the velocity and strength of exhale flows.

### Claim 18 (depends on 18)

The system of claim 18, wherein the thermal camera is a CO2 thermal imaging camera and wherein the instructions when executed by the processor cause the processor to create a 3D reconstruction of turbulent exhale flows through the use of the CO2 thermal imaging camera.

### Claim 19 (depends on 15)

The system of claim 15, wherein the instructions when executed by the processor create the 3D reconstruction by recording an exhale flow and extract flow characteristics and spatial metrics required to provide a direct measurement of the physiological phenomena.

### Claim 20 (depends on 15)

The system of claim 15, wherein the instructions when executed by the processor further generate an inverse thermal segmentation that separate and extracts CO2 exhale signatures observed in front of other thermal sources.
