# US 12,182,326 B2 · Apparatus, methods and computer programs for controlling image augmentations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,326 B2 - Document ID: US12182326B2 - Status: Granted - Kind: B2 - Application number: 18/495,913 - Filed: 2023-10-27 - Published: 2024-05-02 - Granted: 2024-12-31 - Earliest claimed: 2023-10-27 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Marja Pauliina Salmimaa; Toni Johan Jarvenpaa; Arto Juhani Lehtiniemi - Web page: https://patentgrep.com/patent/US12182326B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)} - G02B 27/017: {Head mounted} - G02B 27/0172: {characterised by optical features} - G02B 27/0179: {Display position adjusting means not related to the information to be displayed} - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G02B 2027/0185: {Displaying image at variable distance} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract Examples of the disclosure enable image augmentations to be used with images of users wearing head mounted items. In examples of the disclosure an apparatus can be configured to use gaze tracking to determine a location of a user's pupil wherein the user is wearing a head mounted item. The apparatus can also be configured to enable display of an indication of the location of the user's pupil so that the indication is visible on a surface of the head mounted item. The indication is configured to be used for positioning at least one image augmentation for a captured image comprising the user. ## Claims ### Claim 1 An apparatus, comprising means for: at least one processor; and at least one memory storing instructions that, when executed with the at least one processor, cause the apparatus to perform: using gaze tracking to determine a location of a user's pupil wherein the user is wearing a head mounted item; and enabling display of an indication of the location of the user's pupil so that the indication is visible on a surface of the head mounted item and wherein the indication is configured to be used for positioning at least one image augmentation for a captured image comprising the user. ### Claim 2 (depends on 1) An apparatus as claimed in claim 1 wherein the at least one image augmentation comprises a graphical overlay and wherein the instructions, when executed with the at least one processor, cause the apparatus to superimpose the graphical overlay over at least part of the image comprising the user. ### Claim 3 (depends on 1) An apparatus as claimed in claim 1 wherein the instructions, when executed with the at least one processor, incorporate at least part of the indication as part of the at least one image augmentation. ### Claim 4 (depends on 1) An apparatus as claimed in claim 1 wherein the instructions, when executed with the at least one processor, cause the apparatus to display the indication of the location of the user's pupil on a shutter of the head mounted item. ### Claim 5 (depends on 1) An apparatus as claimed in claim 1 wherein the indication of the location of the user's pupil comprises outcoupled light from the head mounted item. ### Claim 6 (depends on 5) An apparatus as claimed in claim 5 wherein at least part of an image displayed with the head mounted item is adapted to enable the outcoupled light to be used to identify the position of the user's pupil. ### Claim 7 (depends on 1) An apparatus as claimed in claim 1 wherein the instructions, when executed with the at least one processor, enable an exchange of information with an electronic device being used to capture the image comprising the user wherein the information comprises information relating to at least one of: relative positions of the electronic device and the head mounted item, or relative orientations of the electronic device and the head mounted item. ### Claim 8 (depends on 1) An apparatus as claimed in claim 1 wherein the head mounted item comprises an augmented reality headset. ### Claim 9 (depends on 1) A head mounted item comprising an apparatus as claimed in claim 1. ### Claim 10 A method comprising: using gaze tracking to determine a location of a user's pupil wherein the user is wearing a head mounted item; and enabling display of an indication of the location of the user's pupil so that the indication is visible on a surface of the head mounted item and wherein the indication is configured to be used for positioning at least one image augmentation for a captured image comprising the user. ### Claim 11 (depends on 10) A non-transitory program storage device readable with an apparatus, tangibly embodying a program of instructions executable with the apparatus for performing the method of claim 10. ### Claim 12 An apparatus, comprising: at least one processor; and at least one memory storing instructions that, when executed with the at least one processor, cause the apparatus to perform: detecting an indication of a location of a user's pupil wherein the user is wearing a head mounted item and the indication is displayed so that it is visible on a surface of the head mounted item; capturing one or more images comprising the user; and using the indication of the location of the user's pupil to position at least one image augmentation for at least one captured image comprising the user. ### Claim 13 (depends on 12) An apparatus as claimed in claim 12 wherein the instructions, when executed with the at least one processor, cause the apparatus to remove the indication of the location of the user's pupil from respective ones of the captured one or more images comprising the user. ### Claim 14 A method comprising: detecting an indication of a location of a user's pupil wherein the user is wearing a head mounted item and the indication is displayed so that it is visible on a surface of the head mounted item; capturing one or more images comprising the user; and using the indication of the location of the user's pupil to position at least one image augmentation for at least one captured image comprising the user. ### Claim 15 (depends on 14) A non-transitory program storage device readable with an apparatus, tangibly embodying a program of instructions executable with the apparatus for performing the method of claim 14.