# US 12,594,003 B2 · Device, system and method for determining respiratory information of a subject > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,594,003 B2 - Document ID: US12594003B2 - Status: Granted - Kind: B2 - Application number: 18/292,170 - Filed: 2022-07-19 - Published: 2024-08-29 - Granted: 2026-04-07 - Earliest claimed: 2021-07-26 - Assignees: Koninklijke Philips N.V. - Applicants: Koninklijke Philips N.V. - Inventors: Ilde Lorato; Gerard de Haan - Web page: https://patentgrep.com/patent/US12594003B2 ## CPC - A61B 5/015: {By temperature mapping of body part} - A61B 5/087: Measuring breath flow - A61B 5/01: Measuring temperature of body parts {; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue} (clinical contact thermometers G01K13/20) - A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)} - A61B 5/113: occurring during breathing - G06T 7/0012: {Biomedical image inspection} - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 2207/10048: Infrared image - G06T 2207/30004: Biomedical image processing - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present invention relates to a device, system and method for determining respiratory information of a subject. The device comprises an image input (21) configured to obtain thermal images of the subject, processing unit (22) configured to determine respiratory information of the subject from the thermal images, and an output (23) configured to output the determined respiratory information of the subject. The processing unit (22) is configured to determine the respiratory information of the subject by identifying in the obtained thermal images respiratory pixels and/or pixel groups that show temperature variations related to respiration, determining, from among the respiratory pixels and/or pixel groups, respiratory flow pixels and/or pixel groups that show respiratory flow, wherein one or more spatial filters are applied to the respiratory pixels and/or pixel groups to discriminate the respiratory flow pixels and/or pixel groups from respiratory motion pixels and/or pixel groups that show respiratory motion, and determining respiratory information of the subject from the respiratory flow pixels and/or pixel groups and/or the respiratory motion pixels and/or pixel groups. ## Claims ### Claim 1 A device for determining respiratory information of a subject, the device comprising: an image input that is configured to obtain thermal image data of the subject, the thermal image data including data associated with at least a portion of the subject's facial area and/or its surrounding; a processor configured to determine the respiratory information of the subject from the thermal image data; and an output that is configured to output the respiratory information of the subject, wherein the processor is configured to determine the respiratory information of the subject by: identifying, in the thermal image data, thermal image respiratory pixels and/or pixel groups indicative of a temperature variation related to respiration, determining, from among the thermal image respiratory pixels and/or pixel groups, respiratory flow pixels and/or pixel groups indicative of a respiratory flow, applying one or more spatial filters of the processor to the thermal image respiratory pixels and/or pixel groups to discriminate the respiratory flow pixels and/or pixel groups from respiratory motion pixels and/or pixel groups indicative of a respiratory motion, and determining the respiratory information of the subject from the respiratory flow pixels and/or pixel groups and/or the respiratory motion pixels and/or pixel groups. ### Claim 2 (depends on 1) The device as claimed in claim 1, wherein the processor is configured to apply at least two spatial filters of the one or more spatial filters to the thermal image respiratory pixels and/or pixel groups. ### Claim 3 (depends on 2) The device as claimed in claim 2, wherein the at least two spatial filters include at least two Gabor filters, having at least two different spatial filtering directions and/or one or more spatial frequencies. ### Claim 4 (depends on 1) The device as claimed in claim 1, wherein applying the one or more spatial filters of the processor to the thermal image respiratory pixels and/or pixel groups identifies the respiratory motion pixels and/or pixel groups, while pixels and/or pixel groups of the thermal image respiratory pixels and/or pixel groups that remain upon filtering are the respiratory flow pixels and/or pixel groups. ### Claim 5 (depends on 1) The device as claimed in claim 1, wherein the processor is configured to determine a respiration signal from the thermal image respiratory pixels and/or pixel groups and to eliminate or reduce contributions of the respiratory motion pixels and/or pixel groups to the respiration signal to obtain a respiratory flow signal as information on respiratory flow. ### Claim 6 (depends on 5) The device as claimed in claim 5, wherein the processor is configured to determine the respiration signal by averaging the thermal image respiratory pixels and/or pixel groups that individually exhibit a temporal period or pseudo-periodic variation of pixel value in a respiration range or by selecting a pixel and/or pixel group from the thermal image respiratory pixel and/or pixel groups that exhibits a temporal period or pseudo-periodic variation of pixel value in a respiration range. ### Claim 7 (depends on 1) The device as claimed in claim 1, wherein the processor is configured to: determine respiratory motion pixels and/or pixel groups by detecting, from among the thermal image respiratory pixels and/or pixel groups, pixels and/or pixel groups that represent edges or temperature gradients in the obtained thermal image data, and/or determine respiratory flow pixels and/or pixel groups by detecting, from among the thermal image respiratory pixels and/or pixel groups, pixels and/or pixel groups that do not represent edges or temperature gradients in the obtained thermal image data, and/or determine respiratory motion pixels and/or pixel groups as those respiratory pixels and/or pixel groups, wherein the spatial filtering of the thermal image changes intensity of a respiration signal and to determine respiratory flow pixels and/or pixel groups as those respiratory pixels and/or pixel groups, wherein the spatial filtering leaves the respiration signal substantially the same. ### Claim 8 (depends on 1) The device as claimed in claim 1, wherein the processor is configured to determine respiratory flow pixel or pixel groups by determining one or more features of a set of features of the obtained thermal image data, the set of features comprising: a pseudo-periodicity indicating the height of a spectrum peak of a respiration pixel's signal, a respiration rate cluster indicating cluster of pixels having similar frequencies, a gradient indicating one or more edges in thermal image data, a correlation indicating if a time domain signal of a pixel is correlated with a time domain signal of a respiratory pixel, and a covariance indicating the covariance between the time domain signal of the pixel and the time domain signal of the respiratory flow pixel. ### Claim 9 (depends on 8) The device as claimed in claim 8, wherein the processor is configured to determine a flow map by combining two or more of the features of the set of features and to apply the one or more spatial filters to the flow map to determine the respiratory flow pixels and/or pixel groups. ### Claim 10 (depends on 1) The device as claimed in claim 1, wherein the processor is configured to determine one or more of respiratory flow information, breathing effort information, obstructive apneas, central apneas, and mixed apneas. ### Claim 11 (depends on 10) The device as claimed in claim 10, wherein the one or more of respiratory flow information is determined by averaging time domain signals of the respiratory flow pixels and/or pixel groups. ### Claim 12 (depends on 1) A device as claimed in claim 1, wherein the processor is further configured to: detect respiratory motion pixels and/or pixel groups; and detect apnea of the subject based on the respiratory motion pixels and/or pixel groups and the respiratory flow pixels and/or pixel groups. ### Claim 13 (depends on 1) A device as claimed in claim 1, wherein the image input is further configured to obtain second image data of a second image of the subject obtained from emissions in the shorter wavelength range than the emissions from which the thermal image data is obtained, in a wavelength range between 400 and 2000 nm such as visible or near-infrared light, the second image data including data associated with at least part of the same region of the subject as the thermal image data, and wherein the processor is further configured to identify, in the second image, respiratory motion pixels and/or pixel groups and to use the respiratory motion pixels and/or pixel groups identified in the second image data and the thermal image respiratory pixels and/or pixel groups identified in the thermal image data to determine the respiratory information. ### Claim 14 (depends on 13) The device as claimed in claim 13, wherein the processor is further configured to detect which of the thermal image respiratory pixels and/or pixel groups determined in the thermal image data are absent in the second image data and to determine, from among the thermal image respiratory pixels and/or pixel groups that are present in the thermal image data but absent in the second image data, the respiratory flow pixels and/or pixel groups. ### Claim 15 A system for determining respiratory information of a subject, the system comprising: a thermal sensor configured to acquire thermal image data associated with the subject, the thermal image data including data associated with at least a part of the subject's facial area and/or its surrounding; a device comprising: an image input that is configured to obtain thermal image data via the thermal sensor, the thermal image data including data associated with at least a portion of the subject's facial area and/or its surrounding; a processor configured to determine the respiratory information of the subject from the thermal image data, wherein the processor is configured to determine the respiratory information of the subject by: identifying, in the thermal image data, thermal image respiratory pixels and/or pixel groups indicative of a temperature variation related to respiration, determining, from among the thermal image respiratory pixels and/or pixel groups, respiratory flow pixels and/or pixel groups indicative of a respiratory flow, applying one or more spatial filters of the processor to the thermal image respiratory pixels and/or pixel groups to discriminate the respiratory flow pixels and/or pixel groups from respiratory motion pixels and/or pixel groups indicative of a respiratory motion, and determining the respiratory information of the subject from the respiratory flow pixels and/or pixel groups and/or the respiratory motion pixels and/or pixel groups; and an output interface that is configured to output the respiratory information of the subject. ### Claim 16 (depends on 15) The system as claimed in claim 15, wherein the one or more spatial filters comprise at least two spatial filters. ### Claim 17 (depends on 16) The system as claimed in claim 16, wherein the at least two spatial filters include at least two Gabor filters, having at least two different spatial filtering directions and/or one or more spatial frequencies. ### Claim 18 (depends on 15) The system as claimed in claim 15, wherein applying the one or more spatial filters of the processor to the thermal image data respiratory pixels and/or pixel groups identifies the respiratory motion pixels and/or pixel groups while pixels and/or pixel groups of the respiratory pixels and/or pixel groups that remain upon filtering are the respiratory flow pixels and/or pixel groups. ### Claim 19 A method for determining respiratory information of a subject, the method comprising: obtaining thermal image data of the subject, the thermal image data associated with at least part of the subject's facial area and/or its surrounding; determining the respiratory information of the subject from the thermal image data; and outputting the respiratory information of the subject, wherein the respiratory information of the subject is determined by: identifying in the thermal image data, respiratory pixels and/or pixel groups indicating temperature variations related to respiration, determining, from among the thermal image data respiratory pixels and/or pixel groups, respiratory flow pixels and/or pixel groups indicative of respiratory flow; applying one or more spatial filters to the thermal image data respiratory pixels and/or pixel groups to discriminate the respiratory flow pixels and/or pixel groups from respiratory motion pixels and/or pixel groups that show respiratory motion, and determining the respiratory information of the subject from the respiratory flow pixels and/or pixel groups and/or the respiratory motion pixels and/or pixel groups. ### Claim 20 (depends on 19) A non-transitory computer readable medium comprising a computer code for causing a computer to carry out the steps of the method as claimed in claim 19.