# US 11,113,524 B2 · Schemes for retrieving and associating content items with real-world objects using augmented reality and object recognition > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,524 B2 - Document ID: US11113524B2 - Status: Granted - Kind: B2 - Application number: 16/903,240 - Filed: 2020-06-16 - Published: 2020-10-01 - Granted: 2021-09-07 - Earliest claimed: 2014-09-29 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Glenn Black - Web page: https://patentgrep.com/patent/US11113524B2 ## CPC - G06V 20/20: in augmented reality scenes - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 11/60: Creating or editing images; Combining images with text - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06V 20/653: {by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces} - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - G06T 2219/004: Annotating, labelling - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method includes identifying a real-world object in a scene viewed by a camera of a user device, matching the real-world object with a tagged object based at least in part on image recognition and a sharing setting of the tagged object, the tagged object having been tagged with a content item, providing a notification to a user of the user device that the content item is associated with the real-world object, receiving a request from the user for the content item, and providing the content item to the user. A computer readable storage medium stores one or more computer programs, and an apparatus includes a processor-based device. ## Claims ### Claim 1 A method comprising: identifying a real-world object in a scene viewed by a camera of a user device; matching the real-world object with a tagged object based at least in part on image recognition and a sharing setting that has been assigned to the tagged object, the tagged object having been tagged with a content item that is distinct from the sharing setting; providing a notification to a user of the user device that the content item is associated with the real-world object; receiving a request from the user for the content item; and providing the content item to the user. ### Claim 2 (depends on 1) The method of claim 1, wherein the sharing setting comprises whether the tagged object will be matched only with an image of the tagged object or with an image of any object sharing one or more common attributes with the tagged object. ### Claim 3 (depends on 1) The method of claim 1, wherein the matching of the real-world object with the tagged object is further based on a location of the user device and/or a social network connection between the user and an author of the content item. ### Claim 4 (depends on 1) The method of claim 1, wherein the notification comprises one or more of a sound notification, a pop-up notification, a vibration, or an icon in a display of the scene viewed by the camera of the user device. ### Claim 5 (depends on 1) The method of claim 1, wherein the content item comprises one or more of a text comment, an image, an audio clip, or a video clip. ### Claim 6 An apparatus comprising: a processor-based device; and a non-transitory storage medium storing a set of computer readable instructions configured to cause the processor-based device to perform steps comprising: identifying a real-world object in a scene viewed by a camera of a user device; matching the real-world object with a tagged object based at least in part on image recognition and a sharing setting that has been assigned to the tagged object, the tagged object having been tagged with a content item that is distinct from the sharing setting; providing a notification to a user of the user device that the content item is associated with the real-world object; receiving a request from the user for the content item; and providing the content item to the user. ### Claim 7 (depends on 6) The apparatus of claim 6, wherein the sharing setting comprises whether the tagged object will be matched only with an image of the tagged object or with an image of any object sharing one or more common attributes with the tagged object. ### Claim 8 (depends on 6) The apparatus of claim 6, wherein the matching of the identified real-world object with the tagged object is further based on a location of the user device and/or a social network connection between the user and an author of the content item. ### Claim 9 (depends on 6) The apparatus of claim 6, wherein the notification comprises one or more of a sound notification, a pop-up notification, a vibration, or an icon in a display of the scene viewed by the camera of the user device. ### Claim 10 (depends on 6) The apparatus of claim 6, wherein the content item comprises one or more of a text comment, an image, an audio clip, or a video clip. ### Claim 11 A method comprising: receiving a selection of a real-world object as viewed by a camera of a first user device from a user; receiving a content item to tag to the real-world object; capturing one or more images of the real-world object with the camera; receiving a sharing setting from the user, wherein the sharing setting is distinct from the content item and comprises whether the real-world object will be matched only with an image of the real-world object or with an image of any object sharing one or more common attributes with the real-world object; and storing the content item, the one or more images of the real-world object, and the sharing setting on a server, wherein the content item is configured to be retrieved by a second user device viewing an object matching the real-world object. ### Claim 12 (depends on 11) The method of claim 11, wherein the content item comprises one or more of a text comment, an image, an audio clip, or a video clip. ### Claim 13 (depends on 11) The method of claim 11, wherein the sharing setting further comprises restrictions on who can view the content item. ### Claim 14 (depends on 11) The method of claim 11, wherein the sharing setting is entered by the user selecting from a list of attributes corresponding to the real-world object. ### Claim 15 (depends on 14) The method of claim 14, wherein the list of attributes is generated through object recognition performed on the one or more images of the real-world object. ### Claim 16 An apparatus comprising: a processor-based device; and a non-transitory storage medium storing a set of computer readable instructions configured to cause the processor-based device to perform steps comprising: receiving a selection of a real-world object as viewed by a camera of a first user device from a user; receiving a content item to tag to the real-world object; capturing one or more images of the real-world object with the camera; receiving a sharing setting from the user, wherein the sharing setting is distinct from the content item and comprises whether the real-world object will be matched only with an image of the real-world object or with an image of any object sharing one or more common attributes with the real-world object; and storing the content item, the one or more images of the real-world object, and the sharing setting on a server, wherein the content item is configured to be retrieved by a second user device viewing an object matching the real-world object. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the content item comprises one or more of a text comment, an image, an audio clip, or a video clip. ### Claim 18 (depends on 16) The apparatus of claim 16, wherein the sharing setting further comprises restrictions on who can view the content item. ### Claim 19 (depends on 16) The apparatus of claim 16, wherein the sharing setting is entered by the user selecting from a list of attributes corresponding to the real-world object. ### Claim 20 (depends on 19) The apparatus of claim 19, wherein the list of attributes is generated through automatic object recognition performed on the one or more images of the real-world object.