# US 2020/0401802 A1 · Augmented reality tagging of non-smart items > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401802 A1 - Document ID: US20200401802A1 - Status: Published application - Kind: A1 - Application number: 16/448,248 - Filed: 2019-06-21 - Published: 2020-12-24 - Earliest claimed: 2019-06-21 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Robert Huntington Grant; Zachary A. Silverstein; Vyacheslav Zheltonogov; Juan C. Lopez - Web page: https://patentgrep.com/patent/US20200401802A1 ## CPC - G06V 20/20: in augmented reality scenes - G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - H04W 4/021: Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences - H04W 4/029: Location-based management or tracking services - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract A computer-implemented system and method provide for a tagging user (TU) device that determines a first location of the TU device and receives, in the first location, a selection of a real-world object from a TU who views the object through the TU device. The TU device receives, from a TU, tagging information to attach to the object, and captures descriptive attributes of the object. The descriptive attributes and the tagging information associated with the first location are stored in a tagged object database. ## Claims ### Claim 1 A computer-implemented method for tagging and retrieving objects, the method comprising: in a tagging mode of operation and with an augmented reality (AR) tagging user (TU) device: determining a first location of the TU device; receiving, in the first location, a selection of a real-world object from a TU who views a first image of the object through the TU device; receiving, from the TU, tagging information to attach to the object; capturing descriptive attributes of the object by the TU device; and storing the descriptive attributes and the tagging information associated with the first location in a tagged object database (TODB); wherein the capturing of the descriptive attributes of the object by the TU device comprises, after the object has been identified: a) receiving image information from an imaging device of the TU device subsequent to the first image; and b) collecting additional object information or attributes about the object from the image information; the method further comprising storing the additional object information or attributes in the TODB. ### Claim 2 (depends on 1) The method of claim 1, wherein the descriptive attributes comprise the following attributes: semantic attributes, descriptor attributes, location attributes, and image attributes. ### Claim 3 (depends on 1) The method of claim 1, wherein the first location is a geofence defining a bounded area. ### Claim 4 (depends on 1) The method of claim 1, wherein the TU device is AR glasses. ### Claim 5 (canceled) ### Claim 6 (depends on 1) The method of claim 1, wherein the tagging information is selected from the group consisting of: message information, voice information, and video information. ### Claim 7 (depends on 1) The method of claim 1, further comprising accessing additional information associated with the object from external sources. ### Claim 8 (depends on 1) The method of claim 1, further comprising: in a reviewing mode of operation and with an AR reviewing user (RU) device: determining that the RU device is at the first location; determining that the object is at the first location; receiving the tagging information associated with the object that is stored in the TODB based on a retrieval trigger; and displaying, on a display of the RU device, information associated with the received tagging information. ### Claim 9 (depends on 8) The method of claim 8, wherein the retrieval trigger is entry by the RU device into the first location that is a geofence defining a bounded area. ### Claim 10 (depends on 8) The method of claim 8, wherein the retrieval trigger is a detected distance between the object and the RU device. ### Claim 11 (depends on 8) The method of claim 8, wherein the determining that the object is at the first location comprises: scanning the first location; locating the object as it is viewed through a display of the RU device; capturing the descriptive attributes of the object by the RU device; and matching the captured descriptive attributes of the object with the descriptive attributes of the object stored in the TODB that meets a predefined matching criteria. ### Claim 12 (depends on 8) The method of claim 8, further comprising: modifying the tagging information of the object stored in the TODB subsequent to the receiving of the tagging information. ### Claim 13 (depends on 8) The method of claim 8, wherein the TU device and the RU device are a same device. ### Claim 14 An augmented reality (AR) tagging system, comprising: a tagging user (TU) device processor of an augmented reality (AR) TU device configured to: determine a first location of the TU device; receive, in the first location, a selection of a real-world object from a TU who views a first image of the object through the TU device; receive, from the TU, tagging information to attach to the object; capture descriptive attributes of the object by the TU device; and store the descriptive attributes and the tagging information associated with the first location in a tagged object database (TODB); wherein the capture of the descriptive attributes of the object by the TU device comprises, after the object has been identified: a) receipt of image information from an imaging device of the TU device subsequent to the first image; and b) collection of additional object information or attributes about the object from the image information; the TU device being further configured to store the additional object information or attributes in the TODB. ### Claim 15 (depends on 14) The system according to claim 14, wherein: the descriptive attributes comprise the following attributes: semantic attributes, descriptor attributes, location attributes, and image attributes; the first location is a geofence defining a bounded area; the TU device is AR glasses worn by the TU; the capturing of the descriptive attributes of the object comprises receiving image information from an imaging device of the TU device; and the tagging information is selected from the group consisting of: message information, voice information, and video information. ### Claim 16 (depends on 14) The system according to claim 14, comprising: a reviewing user (RU) device processor of an augmented reality (AR) RU device configured to: determine that the RU device is at the first location; determine that the object is at the first location; receive the tagging information associated with the object that is stored in the TODB based on a retrieval trigger; and display, on a display of the RU device, information associated with the received tagging information. ### Claim 17 (depends on 16) The system according to claim 16, wherein the RU device processor is further configured to, in the determination that the object is at the first location: scan the first location; locate the object as it is viewed through a display of the RU device; capture the descriptive attributes of the object by the RU device; and match the captured descriptive attributes of the object with the descriptive attributes of the object stored in the TODB that meets a predefined matching criteria. ### Claim 18 (depends on 16) The system according to claim 16, wherein the RU device processor is further configured to modify the tagging information of the object stored in the TODB subsequent to the receiving of the tagging information. ### Claim 19 A computer program product for an augmented reality (AR) tagging system, the computer program product comprising a computer readable storage medium having computer-readable program code embodied therewith to, when executed on a tagging user (TU) device processor of the TU device: determine a first location of the TU device; receive, in the first location, a selection of a real-world object from a TU who views a first image of the object through the TU device; receive, from the TU, tagging information to attach to the object; capture descriptive attributes of the object by the TU device; and store the descriptive attributes and the tagging information associated with the first location in a tagged object database (TODB); wherein the capture of the descriptive attributes of the object by the TU device comprises, after the object has been identified: a) receipt of image information from an imaging device of the TU device subsequent to the first image; and b) collection of additional object information or attributes about the object from the image information; the program code being further configured for storing the additional object information or attributes in the TODB. ### Claim 20 (depends on 19) The computer program product of claim 19, wherein the computer-readable program code is further configured to, when executed on a reviewing user (RU) device processor of an augmented reality (AR) RU device: determine that the RU device is at the first location; determine that the object is at the first location; receive the tagging information associated with the object that is stored in the TODB based on a retrieval trigger; and display, on a display of the RU device, information associated with the received tagging information.