# US 12,002,224 B2 · Apparatus and method for protecting against environmental hazards

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,002,224 B2
- Document ID: US12002224B2
- Status: Granted
- Kind: B2
- Application number: 17/215,319
- Filed: 2021-03-29
- Published: 2021-09-30
- Granted: 2024-06-04
- Earliest claimed: 2020-03-27
- Assignees: Hs Vk Idea Factory, LLC
- Applicants: Hs Vk Idea Factory, LLC
- Inventors: Veljko Krunic; Helen Stella
- Web page: https://patentgrep.com/patent/US12002224B2

## CPC

- G06T 7/248: {involving reference images or patches}
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 7/12: Edge-based segmentation
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G08B 21/0476: {Cameras to detect unsafe condition, e.g. video cameras}
- G06T 2207/20096: Interactive definition of curve of interest
- G06T 2207/30196: Human being; Person
- G06T 2207/30204: Marker
- G08B 21/245: {Reminder of hygiene compliance policies, e.g. of washing hands}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

The disclosures are directed to methods and apparatuses that allow for one or more of personal sanitizing/decontamination of a user, prevention of the touching of exposed parts of the user's body, tracking of surface sanitizing/decontamination status, tracking of the interpersonal distance, and sanitization of personal electronics (e.g., a mobile phone). In addition, data relating to a user's usage of the various components of the system may be tracked, and artificial intelligence (AI) techniques may be employed to mine for patterns of user behavior in the system.

## Claims

### Claim 1

An apparatus comprising:
a memory storing instructions; and
a processor communicatively coupled to the memory and configured to execute the instructions to:
receive, from at least one camera, a first image of a scene;
determine, based on the first image, a first location of at least one marker within the scene;
based on the first location of the at least one marker within the scene, generate boundary data characterizing a boundary of an object within the scene;
receive, from the at least one camera, a second image of the scene;
determine, based on the second image, a second location of the at least one marker within the scene;
modify the boundary data based on the second location of the at least one marker; and
track a movement of the object based on the modified boundary data.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the processor is configured to execute the instructions to determine that the tracked movement satisfies at least one predetermined movement based on movement data.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the processor is configured to execute the instructions to:
provide for displaying a first indication that the object is not cleaned when the at least one predetermined movement has not been satisfied; and
provide for displaying a second indication that the object is cleaned when the at least one predetermined movement has been satisfied.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein generating the boundary data comprises applying an image segmentation algorithm to the first image.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the processor is configured to execute the instructions to detect the at least one marker within the first image, wherein generating the boundary data is based on the at least one marker.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein generating the boundary data comprises:
determining an area of the first image that comprises the at least one marker; and
generating the boundary data based on combining the area with the boundary of the object.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein determining the area of the first image that comprises the at least one marker comprises:
generating at least one virtual marker at a location of the first image based on marker configuration data, the market configuration data characterizing the location of the first image in relation to the first location of the at least one marker; and
determining the area based on the location of the at least one virtual marker.

### Claim 8 (depends on 1)

The apparatus of claim 1 wherein the processor is configured to execute the instructions to:
determine that the tracked movement fails to satisfy at least one predetermined movement based on movement data; and
provide for display an indication that the object has not been cleaned.

### Claim 9 (depends on 1)

The apparatus of claim 1 wherein the processor is configured to execute the instructions to:
detect a surface in the first image;
determine at least a portion of the surface that coincides with the object; and
determine that the at least portion of the surface has been cleansed by the object.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein determining the at least portion of the surface that coincides with the object comprises determining that first pixels of the first image that correspond to the at least portion of the surface coincides with second pixels of the first image that correspond to the object.

### Claim 11

A method by at least one processor comprising:
receiving, from at least one camera, a first image of a scene;
determining, based on the first image, a first location of at least one marker within the scene;
based on the first location of the at least one marker within the scene, generating boundary data characterizing a boundary of an object within the scene;
receiving, from the at least one camera, a second image of the scene;
determining, based on the second image, a second location of the at least one marker within the scene;
modifying the boundary data based on the second location of the at least one marker; and
tracking a movement of the object based on the modified boundary data.

### Claim 12 (depends on 11)

The method of claim 11 comprising determining that the tracked movement satisfies at least one predetermined movement based on movement data.

### Claim 13 (depends on 12)

The method of claim 12 comprising:
providing for display a first indication that the object is not cleaned when the at least one predetermined movement has not been satisfied; and
providing for display a second indication that the object is cleaned when the at least one predetermined movement has been satisfied.

### Claim 14 (depends on 11)

The method of claim 11, wherein generating the boundary data comprises applying an image segmentation algorithm to the first image.

### Claim 15 (depends on 11)

The method of claim 11 comprising detecting the at least one marker within the first image, wherein generating the boundary data is based on the at least one marker.

### Claim 16

A non-transitory, computer readable medium comprising instructions stored thereon, wherein the instructions, when executed by at least one processor, cause the at least one process to perform operations comprising:
receiving, from at least one camera, an image of a scene;
determining, based on the first image, a first location of at least one marker within the scene;
based on the first location of the at least one marker within the scene, generating boundary data characterizing a boundary of an object within the scene;
receiving, from the at least one camera, a second image of the scene;
determining, based on the second image, a second location of the at least one marker within the scene;
modifying the boundary data based on the second location of the at least one marker; and
tracking a movement of the object based on the modified boundary data.

### Claim 17 (depends on 16)

The non-transitory, computer readable medium of claim 16, wherein the performed operations comprise determining that the tracked movement satisfies at least one predetermined movement based on movement data.

### Claim 18 (depends on 16)

The non-transitory, computer readable medium of claim 16, wherein the performed operations comprise:
providing for display a first indication that the object is not cleaned when the at least one predetermined movement has not been satisfied; and
providing for display a second indication that the object is cleaned when the at least one predetermined movement has been satisfied.

### Claim 19 (depends on 16)

The non-transitory, computer readable medium of claim 16, wherein the performed operations comprise applying an image segmentation algorithm to the first image to generate the boundary data.

### Claim 20 (depends on 16)

The non-transitory, computer readable medium of claim 16, wherein the performed operations comprise detecting the at least one marker within the first image, wherein generating the boundary data is based on the at least one marker.

### Claim 21

An apparatus comprising:
a memory storing instructions; and
a processor communicatively coupled to the memory and configured to execute the instructions to:
receive a first image and a second image from a camera directed to a sink;
apply a computer vision algorithm to the first image and, based on the application of the computer vision algorithm to the first image, determine a first location of a hand;
apply the computer vision algorithm to the second image and, based on the application of the computer vision algorithm to the second image determine a second location of the hand;
determine a hand movement is satisfied based on the first location of the hand, the second location of the hand, and hand movement data characterizing the hand movement;
generate hand status data characterizing whether the hand is sanitized based on the hand movement; and
store the hand status data in a data repository.

### Claim 22 (depends on 21)

The apparatus of claim 21, wherein determining the hand movement is satisfied comprises determining a first orientation of the hand based on the first image, a second orientation of the hand based on the second image, and comparing the first orientation of the hand to the second orientation of the hand.

### Claim 23

An apparatus comprising:
a memory storing instructions; and
a processor communicatively coupled to the memory and configured to execute the instructions to:
receive a first image of a room;
apply a computer vision algorithm to the first image and, based on applying the computer vision algorithm to the first image, determine a first location of a first object in the room;
receive a second image of the room;
apply the computer vision algorithm to the second image and, based on applying the computer vision algorithm to the second image, determine a second location of a second object in the room;
determine a distance between the first location of the first object and the second location of the second object;
compare the distance to a predetermined threshold and, based on the comparison, determine the distance is below the predetermined threshold;
based on determining that the distance is below the predetermined threshold, determine the first object has contaminated the second object;
based on determining that the first object has contaminated the second object, generate contamination graph data characterizing that the first object has contaminated the second object; and
store the contamination graph data in a database.

### Claim 24 (depends on 23)

The apparatus of claim 23, wherein determining the first object has contaminated the second object comprises determining the first object has touched the second object.

### Claim 25 (depends on 23)

The apparatus of claim 23, wherein the predetermined threshold is an interpersonal distance.

### Claim 26 (depends on 25)

The apparatus of claim 25, wherein the predetermined threshold is an airflow distance between the first object and the second object.