# US 2015/0146027 A1 · Strategies for triggering depth sensors and transmitting rgbd images in a cloud-based object recognition system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0146027 A1
- Document ID: US20150146027A1
- Status: Published application
- Kind: A1
- Application number: 14/461,000
- Filed: 2014-08-15
- Published: 2015-05-28
- Earliest claimed: 2013-11-27
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Mahesh Ramachandran; Bojan Vrcelj; Raghuraman Krishnamoorthi
- Web page: https://patentgrep.com/patent/US20150146027A1

## CPC

- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- G06T 7/50: Depth or shape recovery
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 23/611: where the recognised objects include parts of the human body
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30: Subject of image; Context of image processing
- G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, apparatus and methods for triggering a depth sensor and/or limiting bandwidth and/or maintaining privacy are presented. By limiting use of a depth sensor to times when an optical image alone is insufficient, mobile device power is saved. Furthermore, by reducing a size of an optical image to only the portion of the image needed to detect an object, bandwidth is saved and privacy is maintained by not communicating unneeded or undesired information.

## Claims

### Claim 1

A method in a mobile device for triggering a depth sensor, the method comprising:
capturing an image with an image sensing device;
sending at least some of the image to a server to search for an object within the image;
receiving an indication indicating the object was not found in the image with sufficient certainty;
enabling the depth sensor and capturing depth information;
converting the depth information into a depth map; and
sending at least some of the depth map to the server.

### Claim 2 (depends on 1)

The method of claim 1, further comprising disabling the depth sensor.

### Claim 3 (depends on 1)

The method of claim 1, wherein sending the at least some of the depth map to the server comprises:
parsing the depth map into a partial depth map based at least in part on the depth map; and
sending the partial depth map to the server.

### Claim 4 (depends on 1)

The method of claim 1, wherein sending the at least some of the depth map to the server comprises:
capturing a newer image; and
sending the newer image and the at least some of some of the depth map to the server.

### Claim 5 (depends on 1)

The method of claim 1, wherein the depth sensor comprises an infrared depth sensor.

### Claim 6

A mobile device for triggering a depth sensor, the mobile device comprising:
an image sensing device to capture an image;
a transmitter to send at least some of the image to a server to search for an object within the image;
a receiver to receive an indication indicating the object was not found with sufficient certainty;
the depth sensor to capture depth information; and
a processor coupled to the image sensing device, the transmitter, the receiver and the depth sensor, wherein the processor is configured to:
enable the depth sensor;
convert the depth information into a depth map; and
send at least some of the depth map to the server.

### Claim 7 (depends on 6)

The mobile device of claim 6, wherein the processor is further configured to disable the depth sensor.

### Claim 8 (depends on 6)

The mobile device of claim 6, wherein the processor is further configured to:
parse the depth map into a partial depth map based at least in part on the depth map; and
wherein the transmitter is further to send the partial depth map to the server.

### Claim 9 (depends on 6)

The mobile device of claim 6, wherein the image sensing device is further configured to:
capture a newer image; and
wherein the transmitter is further to send the newer image and the at least some of some of the depth map to the server.

### Claim 10 (depends on 6)

The mobile device of claim 6, wherein the depth sensor comprises an infrared depth sensor.

### Claim 11

A mobile device for triggering a depth sensor, the mobile device comprising:
means for capturing an image with an image sensing device;
means for sending at least some of the image to a server to search for an object within the image;
means for receiving an indication indicating the object was not found in the image with sufficient certainty;
means for enabling the depth sensor and capturing depth information;
means for converting the depth information into a depth map; and
means for sending at least some of the depth map to the server.

### Claim 12 (depends on 11)

The mobile device of claim 11, further comprising means for disabling the depth sensor.

### Claim 13 (depends on 11)

The mobile device of claim 11, wherein the means for sending the at least some of the depth map to the server comprises:
means for parsing the depth map into a partial depth map based at least in part on the depth map; and
means for sending the partial depth map to the server.

### Claim 14 (depends on 11)

The mobile device of claim 11, wherein means for sending the at least some of the depth map to the server comprises:
means for capturing a newer image; and
means for sending the newer image and the at least some of some of the depth map to the server.

### Claim 15 (depends on 11)

The mobile device of claim 11, wherein the depth sensor comprises an infrared depth sensor.

### Claim 16

A non-transient computer-readable storage medium, for a mobile device for triggering a depth sensor, including program code stored thereon, comprising program code for:
capturing an image with an image sensing device;
sending at least some of the image to a server to search for an object within the image;
receiving an indication indicating the object was not found in the image with sufficient certainty;
enabling the depth sensor and capturing depth information;
converting the depth information into a depth map; and
sending at least some of the depth map to the server.

### Claim 17 (depends on 16)

The non-transient computer-readable storage medium of claim 16, wherein the program code further comprises program code for disabling the depth sensor.

### Claim 18 (depends on 16)

The non-transient computer-readable storage medium of claim 16, wherein the program code for sending the at least some of the depth map to the server comprises program code for:
parsing the depth map into a partial depth map based at least in part on the depth map; and
sending the partial depth map to the server.

### Claim 19 (depends on 16)

The non-transient computer-readable storage medium of claim 16, wherein the program code for sending the at least some of the depth map to the server comprises:
capturing a newer image; and
sending the newer image and the at least some of some of the depth map to the server.

### Claim 20 (depends on 16)

The non-transient computer-readable storage medium of claim 16, wherein the depth sensor comprises an infrared depth sensor.
