# US 12,217,218 B2 · Rendering augmented reality content based on post-processing of application content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,218 B2 - Document ID: US12217218B2 - Status: Granted - Kind: B2 - Application number: 17/962,795 - Filed: 2022-10-10 - Published: 2024-04-11 - Granted: 2025-02-04 - Earliest claimed: 2022-10-10 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Ramprasad Sedouram - Web page: https://patentgrep.com/patent/US12217218B2 ## CPC - G02B 27/0172: {characterised by optical features} - G06Q 10/1097: {using calendar-based scheduling for task assignment} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G06F 9/453: {Help systems} - G06T 11/00: Two-dimensional [2D] image generation - G06V 20/20: in augmented reality scenes - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/0141: {characterised by the informative content of the display} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06V 40/161: {Detection; Localisation; Normalisation} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Implementations relate to an automated assistant that provides augmented reality content, via a display interface of computerized glasses, resulting from post-processing of application content. The application content can be identified based on prior interactions between a user and one or more applications, and the application content can be processed to determine objects, and/or object classifications, that may be associated with the application content. When the user is wearing the computerized glasses, and the object is detected within a field of view of the computerized glasses, the automated assistant can cause certain content to be rendered at the display interface of the computerized glasses. In some implementations, the content can be generated to supplement, and/or be different from, existing content that the user may have already accessed, in furtherance of preventing duplicative usage of applications and/or preserving computational resources. ## Claims ### Claim 1 A method implemented by one or more processors, the method comprising: determining, based on sensor data generated by one or more sensors of computerized glasses being worn by a user, that a particular physical object is present in a field of view of the computerized glasses; generating, based on the sensor data, an entry that associates the particular physical object with a classification of physical objects; subsequent to generating the entry that associates the particular physical object with the classification of physical objects: determining, based on contextual data associated with the user, that the user is accessing, via a separate computing device or the computerized glasses, application content that is associated with the classification of physical objects; determining, subsequent to the user accessing the application content that is associated with the particular physical object, that the particular physical object, or another physical object having the classification of physical objects, is within the field of view of the computerized glasses within a duration of time specified by the user for receiving notifications corresponding to the classification of physical objects; and causing, based on the contextual data and the particular physical object or other physical object being within the field of view of the computerized glasses, an image associated with the application content to be displayed at a display interface of the computerized glasses. ### Claim 2 (depends on 1) The method of claim 1, wherein determining that the particular physical object or the other physical object is within the field of view of the computerized glasses includes: determining that the particular physical object, or the other physical object, is within the field of view of the computerized glasses within a threshold duration of time from the user accessing the application content. ### Claim 3 (depends on 1) The method of claim 1, wherein determining the application content that is associated with the classification of physical objects includes: determining, during an interaction between the user and the application and based on the contextual data associated with the user, that the user is accessing, via the separate computing device or the computerized glasses, the application content that is of the application and that is associated with the classification of physical objects, and wherein the image to be displayed at the display interface of the computerized glasses is different from the application content. ### Claim 4 (depends on 1) The method of claim 1, wherein the image is selectable via an interface of the computerized glasses, and a selection of the image by the user causes the application content to be edited via the computerized glasses or the separate computing device. ### Claim 5 (depends on 1) The method of claim 1, wherein the duration of time specified by the user includes one or more times of a day. ### Claim 6 (depends on 1) The method of claim 1, wherein the duration of time specified by the user includes one or more days of a week. ### Claim 7 (depends on 1) The method of claim 1, wherein the duration of time specified by the user includes one or more times of one or more days of a week. ### Claim 8 Computerized glasses comprising: one or more sensors; a display interface; memory storing instructions; and one or more processors operable to execute the instructions to: determine, based on sensor data generated by one or more of the sensors, that a particular physical object is present in a field of view of the computerized glasses; generate, based on the sensor data, an entry that associates the particular physical object with a classification of physical objects; subsequent to generating the entry that associates the particular physical object with the classification of physical objects: determine, based on contextual data associated with the user, that the user is accessing, via a separate computing device or the computerized glasses, application content that is associated with the classification of physical objects; determine, subsequent to the user accessing the application content that is associated with the particular physical object, that the particular physical object, or another physical object having the classification of physical objects, is within the field of view of the computerized glasses within a duration of time specified by the user for receiving notifications corresponding to the classification of physical objects; and cause, based on the contextual data and the particular physical object or other physical object being within the field of view of the computerized glasses, an image associated with the application content to be displayed at the display interface of the computerized glasses. ### Claim 9 (depends on 8) The computerized glasses of claim 8, wherein in determining that the particular physical object or the other physical object is within the field of view of the computerized glasses one or more of the processors are to: determine that the particular physical object, or the other physical object, is within the field of view of the computerized glasses within a threshold duration of time from the user accessing the application content. ### Claim 10 (depends on 8) The computerized glasses of claim 8, wherein in determining the application content that is associated with the classification of physical objects one or more of the processors are to: determine, during an interaction between the user and the application and based on the contextual data associated with the user, that the user is accessing, via the separate computing device or the computerized glasses, the application content that is of the application and that is associated with the classification of physical objects, and wherein the image to be displayed at the display interface of the computerized glasses is different from the application content. ### Claim 11 (depends on 8) The computerized glasses of claim 8, wherein the image is selectable via an interface of the computerized glasses, and a selection of the image by the user causes the application content to be edited via the computerized glasses or the separate computing device. ### Claim 12 (depends on 8) The computerized glasses of claim 8, wherein the duration of time specified by the user includes one or more times of a day. ### Claim 13 (depends on 8) The computerized glasses of claim 8, wherein the duration of time specified by the user includes one or more days of a week. ### Claim 14 (depends on 8) The computerized glasses of claim 8, wherein the duration of time specified by the user includes one or more times of one or more days of a week. ### Claim 15 A non-transitory computer-readable medium comprising: memory storing instructions; and one or more processors operable to execute the instructions to: determine, based on sensor data generated by one or more sensors of computerized glasses being worn by a user, that a particular physical object is present in a field of view of the computerized glasses; generate, based on the sensor data, an entry that associates the particular physical object with a classification of physical objects; subsequent to generating the entry that associates the particular physical object with the classification of physical objects: determine, based on contextual data associated with the user, that the user is accessing, via a separate computing device or the computerized glasses, application content that is associated with the classification of physical objects; determine, subsequent to the user accessing the application content that is associated with the particular physical object, that the particular physical object, or another physical object having the classification of physical objects, is within the field of view of the computerized glasses within a duration of time specified by the user for receiving notifications corresponding to the classification of physical objects; and cause, based on the contextual data and the particular physical object or other physical object being within the field of view of the computerized glasses, an image associated with the application content to be displayed at a display interface of the computerized glasses. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein in determining that the particular physical object or the other physical object is within the field of view of the computerized glasses one or more of the processors are to execute the instructions to: determining that the particular physical object, or the other physical object, is within the field of view of the computerized glasses within a threshold duration of time from the user accessing the application content. ### Claim 17 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein in determining the application content that is associated with the classification of physical objects one or more of the processors are to execute the instructions to: determining, during an interaction between the user and the application and based on the contextual data associated with the user, that the user is accessing, via the separate computing device or the computerized glasses, the application content that is of the application and that is associated with the classification of physical objects, and wherein the image to be displayed at the display interface of the computerized glasses is different from the application content. ### Claim 18 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the image is selectable via an interface of the computerized glasses, and a selection of the image by the user causes the application content to be edited via the computerized glasses or the separate computing device. ### Claim 19 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the duration of time specified by the user includes one or more times of a day. ### Claim 20 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the duration of time specified by the user includes one or more days of a week.