# US 2020/0394436 A1 · Identification of misplaced objects using internet of things (iot) devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394436 A1
- Document ID: US20200394436A1
- Status: Published application
- Kind: A1
- Application number: 16/437,407
- Filed: 2019-06-11
- Published: 2020-12-17
- Earliest claimed: 2019-06-11
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Sarbajit K. Rakshit; Martin G. Keen; James E. Bostick; John M. Ganci, JR.
- Web page: https://patentgrep.com/patent/US20200394436A1

## CPC

- H04L 67/125: involving control of end-device applications over a network
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/5854: {using shape and object relationship}
- G06F 18/24: Classification techniques
- G06V 10/82: using neural networks
- G06V 10/95: {structured as a network, e.g. client-server architectures}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L 67/52: specially adapted for the location of the user terminal
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/306: User profiles
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A computer enabled method for locating a misplaced object that includes receiving at an object locating system a request defining an object to be located, the input data request being received audibly by a digital assistant. The method may include analyzing the request with the object locating system by classifying the object to be located with an image search of a corpus of images correlated to comparison objects; and receiving images from Internet of Thing devices (IoT) devices of search objects. The method further includes comparing the images correlated to comparison objects from the image search with images of the search objects to identify retrieval objects that match the object classified by the object locating system that analyzed the request. The location of the retrieval object is then identified.

## Claims

### Claim 1

A computer enabled method for locating a misplaced object comprising:
registering a user to an object locating system that collects data from the user, wherein the user can revoke permission to the object locating system at any time;
receiving at the object locating system a request defining an object to be located, the input data request being received audibly by a digital assistant;
analyzing the request with the object locating system by classifying the object to be located with an image search of a corpus of images correlated to comparison objects;
receiving images from Internet of Thing devices (IoT) devices of search objects;
comparing the images correlated to comparison objects from the image search with images of the search objects to identify retrieval objects that match the object classified by the object locating system that analyzed the request; and
identifying the location of the retrieval object.

### Claim 2 (depends on 1)

The computer implemented method of claim 1, wherein the input data request further comprises a possible location of the object.

### Claim 3 (depends on 1)

The computer implemented method of claim 1, wherein the digital assistant receives instructions from the user and provides an audible output to the user to an answer the input data request.

### Claim 4 (depends on 1)

The computer implemented method of claim 1, wherein the image search comprises searching images captured by the digital assistant, searching images captured by IoT devices, searching cloud image libraries, searching global image libraries or combinations thereof.

### Claim 5 (depends on 1)

The computer implemented method of claim 1, wherein the comparing of the images correlated to the comparison objects from the image search with images of the search objects to identify retrieval objects comprises a convolutional neural network.

### Claim 6 (depends on 1)

The computer implemented method of claim 1, further comprising the digital assistant audibly communicating the location of the classified object to the user.

### Claim 7 (depends on 5)

The computer implemented method of claim 5, wherein the digital assistant is provided commands to said audibly communicate the location of the classified object from the object locating system.

### Claim 8 (depends on 1)

The computer implemented method of claim 1, wherein the identifying the location of the retrieval object is provided by a signal from an Internet of Things (IoT) device that is proximate to the retrieval object.

### Claim 9 (depends on 8)

The computer implemented method of claim 8, wherein the signal may be provided by a sound or an application of light.

### Claim 10

A system for locating a misplaced object or item in a defined area using IoT (Internet of Things) identification comprising:
a request analyzer for receiving a request defining an object to be located, the request being forwarded by a digital assistant that was audibly given the request by a user;
a transceiver for receiving images from Internet of Thing devices (IoT) devices of search objects; and
a visual classifier including at least one hardware processor for executing a series of instructions for analyzing the request by classifying the object to be located with an image search of a corpus of images correlated to comparison objects, comparing the images correlated to comparison objects from the image search with images of the search objects to identify retrieval objects that match the object classified by the object locating system that analyzed the request, and identifying the location of the retrieval object.

### Claim 11 (depends on 10)

The system of claim 10, wherein the input data request further comprises a possible location of the object.

### Claim 12 (depends on 10)

The system of claim 10, wherein the transceiver further receives instructions from the digital assistant, wherein the transceiver provides an audible output to the user to answer the input data request.

### Claim 13 (depends on 10)

The system of claim 10, further comprising a classification corpus that the image search searches, the image search comprising searching images captured by the digital assistant, searching images captured by IoT devices, searching cloud image libraries, searching global image libraries or combinations thereof.

### Claim 14 (depends on 10)

The system of claim 10, wherein the comparing of the images correlated to the comparison objects from the image search with images of the search objects to identify retrieval objects comprises a convolutional neural network.

### Claim 15 (depends on 10)

The system of claim 10, wherein the identifying the location of the retrieval object is provided by a signal from an Internet of Things (IoT) device that is proximate to the retrieval object.

### Claim 16 (depends on 10)

The system of claim 10, wherein the signal may be provided by a sound or an application of light.

### Claim 17

A computer program product comprising a computer readable storage medium having computer readable program code embodied therein for locating a misplaced object or item in a defined area using IoT (Internet of Things) identification, the method comprising:
registering a user to an object locating system that collects data from the user, wherein the user can revoke permission to the object locating system at any time;
receiving at the object locating system a request defining an object to be located, the input data request being received audibly by a digital assistant;
analyzing the request with the object locating system by classifying the object to be located with an image search of a corpus of images correlated to comparison objects;
receiving images from Internet of Thing devices (IoT) devices of search objects; and
comparing the images correlated to comparison objects from the image search with images of the search objects to identify retrieval objects that match the object classified by the object locating system that analyzed the request; and
identifying the location of the retrieval object.

### Claim 18 (depends on 18)

The computer program product of claim 18, wherein the input data request further comprises a possible location of the object.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein the identifying the location of the retrieval object is provided by a signal from an Internet of Things (IoT) device that is proximate to the retrieval object.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the signal may be provided by a sound or an application of light.
