# US 11,442,268 B2 · Augmented reality gaming using virtual eyewear beams > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,268 B2 - Document ID: US11442268B2 - Status: Granted - Kind: B2 - Application number: 17/037,011 - Filed: 2020-09-29 - Published: 2022-03-31 - Granted: 2022-09-13 - Earliest claimed: 2020-09-29 - Assignees: Snap Inc. - Applicants: Ilteris Canberk; Jacob Knipfing - Inventors: Ilteris Canberk; Jacob Knipfing - Web page: https://patentgrep.com/patent/US11442268B2 ## CPC - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G02B 27/0172: {characterised by optical features} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity - A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) - A63F 13/219: for aiming at specific areas on the display, e.g. light-guns - A63F 13/26: having at least one additional display device, e.g. on the game controller or outside a game booth - A63F 13/655: by importing photos, e.g. of the player - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR ## Abstract Interactive augmented reality experiences with an eyewear device including a virtual eyewear beam. The user can direct the virtual beam by orienting the eyewear device or the user's eye gaze or both. The eyewear device may detect the direction of an opponent's eyewear device or eye gaze of both. The eyewear device may calculate a score based on hits of the virtual beam of the user and the opponent on respective target areas such as the other player's head or face. ## Claims ### Claim 1 An augmented reality system comprising: a position detection system; a camera system; a display system; an eye tracking system; an eyewear device comprising the position detection system, the display system, the eye tracking system, a processor, and a memory; and programming in the memory, wherein execution of the programming by the processor configures the eyewear device to perform functions, including functions to: register, with the position detection system, a first target position of a first target in the physical environment and an eyewear device position of the eyewear device; capture at least one image of the first target with the camera system of the eyewear device; identify at least one portion of the first target in the at least one image; monitor at least one user position, the user position comprising a position of at least one eye of the user of the eyewear device, or a position of the eyewear device of the user; calculate at least one virtual beam extending from the at least one user position; present at least one virtual image on the display of the eyewear device reflecting the direction of the at least one virtual beam; and determine whether the at least one virtual beam intersect at least one portion of the first target within virtual image on the display of the eyewear device and display in the display system an indication that the at least one virtual beam intersects the at least one portion of the first target within the virtual image. ### Claim 2 (depends on 1) The system of claim 1, wherein the at least one portion of the first target includes the face, the head or the body of the first target. ### Claim 3 (depends on 1) The system of claim 1, further comprising functions to: determine a score based on whether one or more of the calculated beams intersect the first target. ### Claim 4 (depends on 1) The system of claim 1, further comprising functions to: monitor at least one of the first target position, the first target position comprising a position of at least one eye of the first target, or a position of the eyewear device of the first target; and calculate at least one virtual beam extending from the first target position. ### Claim 5 (depends on 1) The system of claim 1, further comprising functions to: receive data of at least one of the first target position, the first target position comprising at least one eye of the first target, or a position of the eyewear device of the first target; and calculate at least one virtual beam extending from the first target position. ### Claim 6 (depends on 1) The system of claim 1, wherein the camera system is configured to capture sequences of frames of video data, wherein each frame of video data comprises depth information for a plurality of pixels. ### Claim 7 (depends on 1) The system of claim 1, wherein the display system comprises: a see-through display supported by the eyewear device to present the at least one virtual beam. ### Claim 8 An augmented reality method for use with an eyewear device having a position detection system, a camera system, and a display system, the method comprising: registering, with the position detection system, a first target position of a first target in the physical environment and an eyewear device position of the eyewear device; capturing at least one image of the first target with the eyewear device; identifying at least one portion of the first target in the at least one image; monitoring at least one user position, the user position comprising a position of at least one eye of the user of the eyewear device, or a position of the eyewear device of the user; calculating at least one virtual beam extending from the at least one user position; presenting a virtual image on the display of the eyewear device reflecting the direction of the at least one calculated beam; and determining whether the at least one virtual beam intersects at least one portion of the first target within virtual image on the display of the eyewear device and displaying in the display system an indication that the at least one virtual beam intersects the at least one portion of the first target within the virtual image. ### Claim 9 (depends on 8) The augmented reality method of claim 8, wherein the at least one portion of the first target includes the face, the head or the body of the first target. ### Claim 10 (depends on 8) The augmented reality method of claim 8, further comprising: determining a score based on whether one or more of the calculated beams intersect the first target. ### Claim 11 (depends on 8) The augmented reality method of claim 8, further comprising: monitoring at least one of the first target position, the first target position comprising a position of at least one eye of the first target, or a position of the eyewear device of the first target; and calculating at least one virtual beam extending from the first target position. ### Claim 12 (depends on 8) The augmented reality method of claim 8, further comprising: receiving data of at least one of the first target position, the first target position comprising a position of at least one eye of the first target, or a position of the eyewear device of the first target; and calculating at least one virtual beam extending from the first target position. ### Claim 13 (depends on 8) The augmented reality method of claim 8, wherein the camera system is configured to capture sequences of frames of video data, wherein each frame of video data comprises depth information for a plurality of pixels. ### Claim 14 (depends on 8) The augmented reality method of claim 8, wherein the display system comprises: a see-through display supported by the eyewear device to present the at least one virtual beam. ### Claim 15 A non-transitory computer-readable medium storing program code which, when executed, is operative to cause an electronic processor to perform the steps of: registering, with a position detection system of an eyewear device, a first target position of a first target in the physical environment and an eyewear device position of the eyewear device; capturing at least one image of the first target with a camera system of the eyewear device; identifying at least one portion of the first target in the at least one image; monitoring at least one user position, the user position comprising a position of at least one eye of the user of the eyewear device, or a position of the eyewear device of the user; calculating at least one virtual beam extending from the at least one user position; presenting a virtual image on the display of the eyewear device reflecting the direction of the at least one calculated beam; and determining whether the at least one virtual beams intersect at least one portion of the first target within virtual image on the display of the eyewear device and displaying in the display system an indication that the at least one virtual beam intersects the at least one portion of the first target within the virtual image. ### Claim 16 (depends on 15) The non-transitory computer-readable medium storing program code of claim 15, wherein the at least one portion of the first target includes the face, the head or the body of the first target. ### Claim 17 (depends on 15) The non-transitory computer-readable medium storing program code of claim 15, wherein the program code, when executed, is operative to cause an electronic processor to perform the further steps of: determining a score based on whether one or more of the calculated beams intersect the first target. ### Claim 18 (depends on 15) The non-transitory computer-readable medium storing program code of claim 15, wherein the program code, when executed, is operative to cause an electronic processor to perform the further steps of: monitoring at least one of the first target position, the first target position comprising a position of at least one eye of the first target, or a position of the eyewear device of the first target; and calculating at least one virtual beam extending from the at least one first target position. ### Claim 19 (depends on 15) The non-transitory computer-readable medium storing program code of claim 15, wherein the program code, when executed, is operative to cause an electronic processor to perform the further steps of: receiving data of at least one first target position, the first target position comprising a position of at least one eye of the first target, or a position of the eyewear device of the first target; and calculating at least one virtual beam extending from the first target position. ### Claim 20 (depends on 15) The non-transitory computer-readable medium storing program code of claim 15, wherein the program code, when executed, is operative to cause the electronic processor to perform the further steps of: capturing sequences of frames of video data with the camera system, wherein each frame of video data comprises depth information for a plurality of pixels.