# US 11,113,485 B2 · Method to correlate an object with a localized tag

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,113,485 B2
- Document ID: US11113485B2
- Status: Granted
- Kind: B2
- Application number: 15/982,360
- Filed: 2018-05-17
- Published: 2019-11-21
- Granted: 2021-09-07
- Earliest claimed: 2018-05-17
- Assignees: Motorola Mobility LLC
- Applicants: Motorola Mobility LLC
- Inventors: Nikhil Ambha Madhusudhana; Vivek K. Tyagi; Sudhir C. Vissa
- Web page: https://patentgrep.com/patent/US11113485B2

## CPC

- H04W 4/029: Location-based management or tracking services
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06K 7/10386: {the interrogation device being of the portable or hand-handheld type, e.g. incorporated in ubiquitous hand-held devices such as PDA or mobile phone, or in the form of a portable dedicated RFID reader}
- G06T 1/0007: {Image acquisition}
- G06V 10/17: {using hand-held instruments}
- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method and data processing device for receiving at least one response signal corresponding to a respective tag. The method includes, identifying a directional location of the respective tag relative to the user device. The method includes activating a camera device to focus on one or more focal points in an area in the directional location of the respective tag. The method includes estimating a distance of the respective tag relative to the user device. The method includes capturing an image of an object located within the area in the directional location of the respective tag. The method includes correlating the directional location, distance of the respective tag, and image of the object to generate corresponding position information. The position information provides more precise point location of the object. The method includes outputting the corresponding position information to an electronic display.

## Claims

### Claim 1

A method comprising:
receiving, by a user device, at least one response signal from a tag antenna corresponding to a respective tag of one or more tags that exist in a space located within a detectable distance of the user device, each tag antenna producing a corresponding response signal in response to a trigger signal, the corresponding response signal having a unique tag identification (ID) of a respective tag modulated thereon;
in response to receiving the corresponding response signal from the respective tag within the space, identifying, by a processor, a directional location of the respective tag relative to the user device, wherein the identifying comprises: determining, for the respective RFID tag and object, (i) a general horizontal position within the space along a horizontal axis relative to the user device; and (ii) a general vertical position within the space along a vertical axis relative to the user device;
activating a camera device to focus on one or more focal points in an area in the directional location of the respective tag;
estimating, based on a signal strength and the directional location of the corresponding response signal, a distance of the respective tag relative to the user device;
capturing, by the camera device, an image of an object located within the area in the directional location of the respective tag;
in response to capturing the image, correlating the (i) directional location, (ii) distance of the respective tag, and (iii) image of the object to generate corresponding position information that provides a more precise point location of the object that corresponds to the respective tag within the space; and
outputting the corresponding position information of the object that corresponds to the respective tag.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
triggering a radio frequency identification (RFID) reader module of the user device to transmit, at a first frequency range, the trigger signal that propagates into the space to determine a presence of the respective tag of the one or more tags, wherein the respective tag is a respective RFID tag.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
providing, to a RFID tag and object mapping database, the position data of the respective RFID tag and object within the space;
initiating tracking of the respective RFID tag and object within the space;
monitoring one or more inertial measurement unit (IMU) sensors associated with the user device to identify device movement, wherein the one or more IMU sensors are one or more of an accelerometer, a gyroscope, and a magnetometer;
identifying movement of the RFID tag and object based on a change in a signal strength of the corresponding response signal and image data of the object; and
correlating a change in device movement with a change in signal strength of the corresponding response signal.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
determining when a change in signal strength is associated with device movement;
dynamically adjusting a signal strength value for the RFID tag when the change in signal strength of the corresponding response signal is in response to device movement;
generating updated position data of the RFID tag and object based, in part, on the adjusted signal strength value of the respective RFID tag, and in part on image data; and
updating the RFID tag and object mapping database to track movement of the respective RFID tag and object in the space, wherein the RFID tag and object mapping database recognizes a pattern of movement for the RFID tag and object and generates updated position data based on the pattern of movement.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining when the signal strength of the corresponding response signal of the respective RFID tag is no longer detectable within the space; and
in response to the signal strength of the corresponding response signal of the respective RFID tag no longer being detectable, initiating an operation from among: (i) updating the RFID tag and object mapping database, (ii) removing, from a graphical user interface, an image that correlates to the RFID tag and object, (iii) generating an audible notation that corresponds to repositioning of the object.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
identifying camera device parameters, comprising a focal length of a lens of the camera device, a field of view of the space, and a size of a preselected camera sensor;
determining more specific position data of the object within the space relative to the camera device, based on geometric relationships between the camera device parameters; and
providing, to the corresponding position map, the more specific position data of the object within the space.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
in response to receiving the corresponding response signal, demodulating the response signal to identify a unique RFID tag ID of the respective RFID tag, wherein the unique RFID tag ID is modulated on the corresponding response signal;
initiating, within a preconfigured object database, a search for information associated with an object assigned to the unique RFID tag ID;
outputting the corresponding position information of the object that corresponds to the respective tag as position data on a corresponding position map on a graphical user interface of an electronic display;
linking access to the information within the graphical user interface at or near the point location of the respective RFID tag along with a representation of a corresponding object; and
in response to selection of the representation of the object, triggering presentation of the information.

### Claim 8

A data processing device comprising:
a module that transmits a trigger signal, at a first frequency range, that propagates into a space to determine if one or more tags exist in the space within a detectable distance; and
a processor that is communicatively coupled to the module and which:
receives, by the module, at least one response signal from a tag antenna corresponding to a respective tag of one or more tags that exist in a space located within a detectable distance of the module, each tag antenna capable of producing a corresponding response signal in response to a trigger signal, the corresponding response signal having a unique tag identification (ID) of the respective tag modulated thereon;
in response to receiving the corresponding response signal from the respective tag within the space, identifies a directional location of the respective tag relative to the module, wherein to identify the directional location the processor:
determines, for the respective RFID tag and object, (i) a general horizontal position within the space along a horizontal axis relative to the user device (ii) a general vertical position within the space along a vertical axis relative to the user device;
activates a camera device to focus on one or more focal points in an area in the directional location of the respective tag;
estimates, based on a signal strength and the directional location of the corresponding response signal, a distance of the respective tag relative to the module;
captures, by the camera device, an image of an object located within the area in the directional location of the respective tag;
in response to capturing the image, correlates the (i) directional location, (ii) distance of the respective tag, and (iii) image of the object to generate corresponding position information that provides a more precise point location of the object that corresponds to the respective tag within the space; and
outputs the corresponding position information of the object that corresponds to the respective tag.

### Claim 9 (depends on 8)

The data processing device of claim 8, wherein the processor triggers the module to transmit the trigger signal that propagates into the space to determine a presence of the respective tag, wherein the module is a radio frequency identification (RFID) reader module and the respective tag is a respective RFID tag.

### Claim 10 (depends on 9)

The data processing device of claim 9, wherein the processor:
provides, to a RFID tag and object mapping database, the position data of the respective RFID tag and object within the space;
initiates tracking of the respective RFID tag and object within the space;
monitors one or more inertial measurement unit (IMU) sensors associated with the module to identify device movement, wherein the one or more IMU sensors are one or more of an accelerometer, a gyroscope, and a magnetometer;
identifies movement of the RFID tag and object based on a change in a signal strength of the corresponding response signal and image data of the object; and
correlates a change in device movement with a change in signal strength of the corresponding response signal.

### Claim 11 (depends on 10)

The data processing device of claim 10, wherein the processor:
determines when a change in signal strength is associated with device movement;
dynamically adjusts a signal strength value for the respective RFID tag when the change in signal strength of the corresponding response signal is in response to device movement;
generates updated position data of the respective RFID tag and object based, in part, on the adjusted signal strength value of the respective RFID tag, and in part on image data; and
updates the RFID tag and object mapping database to track movement of the respective RFID tag and object in the space, wherein the RFID tag and object mapping database recognizes a pattern of movement for the respective RFID tag and object and generates updated position data based on the pattern of movement.

### Claim 12 (depends on 10)

The data processing device of claim 10, wherein the processor:
determines when the signal strength of the corresponding response signal of the respective RFID tag is no longer detectable within the space; and
in response to the signal strength of the corresponding response signal of the respective RFID tag no longer being detectable, updates the RFID tag and object mapping database and initiates an operation from among: removing, from a graphical user interface, an image, that correlates to the RFID tag and object; and generating an audible notation that corresponds to repositioning of the object.

### Claim 13 (depends on 9)

The data processing device of claim 9, wherein the processor:
in response to receiving the corresponding response signal, demodulates the response signal to identify a unique RFID tag ID of the respective RFID tag, wherein the unique RFID tag ID is modulated on the corresponding response signal;
initiates, within a preconfigured object database, a search for information associated with an object assigned to the unique RFID tag ID;
outputs the corresponding position information of the object that corresponds to the respective tag as position data on a corresponding position map on a graphical user interface of an electronic display;
links access to the information within the graphical user interface at or near the point location of the respective RFID tag along with a representation of a corresponding object; and
in response to selection of the representation of the object, triggers presentation of the information.

### Claim 14 (depends on 8)

The data processing device of claim 8, wherein the processor:
identifies camera device parameters, comprising a focal length of a lens of the camera device, a field of view of the space, and a size of a preselected camera sensor;
determines more specific position data of the object within the space relative to the camera device, based on geometric relationships between the camera device parameters; and
provides, to the corresponding position map, the more specific position data of the object within the space.

### Claim 15

A computer program product comprising:
a computer readable storage device; and
program code on the computer readable storage device that when executed within a processor associated with a user device, the program code enables the user device to provide a functionality of:
receiving, by a user device, at least one response signal from a tag antenna corresponding to a tag of one or more tags that exist in a space located within a detectable distance of the user device, each tag antenna capable of producing a corresponding response signal in response to a trigger signal, the corresponding response signal having a unique tag identification (ID) of the respective tag modulated thereon;
in response to receiving the corresponding response signal from the respective tag within the space, identifying, by a processor, a directional location of the respective tag relative to the user device, wherein the user devices identifying the directional location comprises determining, for the respective RFID tag and object, (i) a general horizontal position within the space along a horizontal axis relative to the user device; and (ii) a general vertical position within the space along a vertical axis relative to the user device;
activating a camera device to focus on one or more focal points in an area in the directional location of the respective tag;
estimating, based on a signal strength and the directional location of the corresponding response signal, a distance of the respective tag relative to the user device;
capturing, by the camera device, an image of an object located within the area in the directional location of the respective tag;
correlating the (i) directional location, (ii) distance of the respective tag, and (iii) image of the object to generate corresponding position information that provides a more precise point location of the object that corresponds to the respective tag within the space; and
outputting the corresponding position information of the object that corresponds to the respective tag.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the program code further enables the user device to provide a functionality of:
triggering a respective radio frequency identification (RFID) reader module of the user device to transmit, at a first frequency range, the trigger signal that propagates into the space;
providing, to a RFID tag and object mapping database, the position data of the respective tag and object within the space, wherein the respective tag is a respective RFID tag;
initiating tracking of the respective RFID tag and object within the space;
monitoring one or more inertial measurement unit (IMU) sensors associated with the user device to identify device movement, wherein the one or more IMU sensors are one or more of an accelerometer, a gyroscope, and a magnetometer;
determining, by the sensors, a general horizontal position within the space along a horizontal axis relative to the user device of the respective RFID tag and object;
determining, by the sensors, a general vertical position within the space along a vertical axis relative to the user device of the respective RFID tag and object;
identifying movement of the respective RFID tag and object based on a change in a signal strength of the corresponding response signal and image data of the object; and
correlating a change in device movement with a change in signal strength of the corresponding response signal.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein the program code further enables the user device to provide a functionality of:
determining when a change in signal strength is associated with device movement;
dynamically adjusting a signal strength value for the respective RFID tag when the change in signal strength of the corresponding response signal is in response to device movement;
generating updated position data of the respective RFID tag and object based, in part, on the adjusted signal strength value of the respective RFID tag, and in part on image data;
updating the RFID tag and object mapping database to track movement of the respective RFID tag and object in the space, wherein the RFID tag and object mapping database recognizes a pattern of movement for the respective RFID tag and object and generates updated position data based on the pattern of movement;
determining when the signal strength of the corresponding response signal of the respective RFID tag is no longer detectable within the space; and
in response to the signal strength of the corresponding response signal of the respective RFID tag no longer being detectable, initiating an operation from among: (i) updating the RFID tag and object mapping database, (ii) removing, from a graphical user interface, an image, that correlates to the respective RFID tag and object, (iii) generating an audible notation that corresponds to repositioning of the object.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein the program code further enables the user device to provide a functionality of:
in response to receiving the corresponding response signal, demodulating the response signal to identify a unique RFID tag ID of the respective RFID tag, wherein the unique RFID tag ID is modulated on the corresponding response signal;
initiating, within a preconfigured object database, a search for information associated with an object assigned to the unique RFID tag ID;
outputting the corresponding position information of the object that corresponds to the respective tag as position data on a corresponding position map on a graphical user interface of an electronic display;
linking access to the information within the graphical user interface at or near the point location of the respective RFID tag along with a representation of a corresponding object; and
in response to selection of the representation of the object, presenting the information.