# US 2021/0241533 A1 · Method for providing augmented reality based on multi-user interaction with real objects and apparatus using the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241533 A1 - Document ID: US20210241533A1 - Status: Published application - Kind: A1 - Application number: 17/151,992 - Filed: 2021-01-19 - Published: 2021-08-05 - Earliest claimed: 2020-01-31 - Assignees: Electronics and Telecommunications Research Institute - Applicants: Electronics And Telecommunications Research Institute - Inventors: Byung-Kuk SEO - Web page: https://patentgrep.com/patent/US20210241533A1 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed herein are a method for providing augmented reality based on participation of multiple users using interaction with a real object and an apparatus for the same. The method is configured such that an augmented reality provision apparatus identifies a target real object on which visual processing is to be performed based on the interaction between a virtual object and a real object in an augmented reality area, delivers instance information corresponding to the target real object to at least one additional user included in the augmented reality area, provides a target real object image corresponding to the view of a user by performing the visual processing at an instance level corresponding to the target real object, and provides an augmented reality event resulting from the interaction so as to correspond to the view of the user. ## Claims ### Claim 1 A method for providing augmented reality, comprising: identifying, by an augmented reality provision apparatus, a target real object on which visual processing is to be performed based on interaction between a virtual object and a real object in an augmented reality area; delivering, by the augmented reality provision apparatus, instance information corresponding to the target real object to at least one additional user included in the augmented reality area; performing, by the augmented reality provision apparatus, the visual processing at an instance level corresponding to the target real object, thereby providing a target real object image corresponding to a view of a user; and providing, by the augmented reality provision apparatus, an augmented reality event resulting from the interaction so as to correspond to the view of the user. ### Claim 2 (depends on 1) The method of claim 1, wherein the visual processing is performed based on the target real object viewed from a viewpoint of each of the at least one additional user. ### Claim 3 (depends on 1) The method of claim 1, wherein the visual processing is performed so as to correspond to at least one of deformation of the real object, deletion thereof, and reconstruction thereof. ### Claim 4 (depends on 1) The method of claim 1, wherein the augmented reality event is generated differently for a view of each of the at least one additional user and is configured to provide augmented reality play information that is displayed so as to correspond to the view of each of the at least one additional user. ### Claim 5 (depends on 1) The method of claim 1, wherein the target real object image is displayed in a different form, corresponding to a view of each of the at least one additional user, so as to correspond to the visual processing. ### Claim 6 (depends on 1) The method of claim 1, wherein the target real object is identified by an augmented reality provision apparatus of the at least one additional user using an instance level of a 3D mesh reprojected based on the instance information. ### Claim 7 (depends on 3) The method of claim 3, wherein providing the target real object image is configured to perform the reconstruction of the real object based on 3D structural information pertaining to the target real object. ### Claim 8 An apparatus for providing augmented reality, comprising: a processor for identifying a target real object on which visual processing is to be performed based on interaction between a virtual object and a real object in an augmented reality area, delivering instance information corresponding to the target real object to at least one additional user included in the augmented reality area, providing a target real object image corresponding to a view of a user by performing the visual processing at an instance level corresponding to the target real object, and providing an augmented reality event resulting from the interaction so as to correspond to the view of the user; and memory for storing at least one of identification information corresponding to the target real object and the instance information. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the visual processing is performed based on the target real object viewed from a viewpoint of each of the at least one additional user. ### Claim 10 (depends on 8) The apparatus of claim 8, wherein the visual processing is performed so as to correspond to at least one of deformation of the real object, deletion thereof, and reconstruction thereof. ### Claim 11 (depends on 8) The apparatus of claim 8, wherein the augmented reality event is generated differently for a view of each of the at least one additional user and is configured to provide augmented reality play information that is displayed so as to correspond to the view of each of the at least one additional user. ### Claim 12 (depends on 8) The apparatus of claim 8, wherein the target real object image is displayed in a different form, corresponding to a view of each of the at least one additional user, so as to correspond to the visual processing. ### Claim 13 (depends on 8) The apparatus of claim 8, wherein the target real object is identified by an augmented reality provision apparatus of the at least one additional user using an instance level of a 3D mesh reprojected based on the instance information. ### Claim 14 (depends on 10) The apparatus of claim 10, wherein the processor performs the reconstruction of the real object based on 3D structural information pertaining to the target real object.