# US 11,308,697 B2 · Virtual reality based selective automation adoption > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,697 B2 - Document ID: US11308697B2 - Status: Granted - Kind: B2 - Application number: 16/569,363 - Filed: 2019-09-12 - Published: 2021-03-18 - Granted: 2022-04-19 - Earliest claimed: 2019-09-12 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Sarbajit K. Rakshit; Krishna Koley; Disha Maitra; Ananya Pal - Web page: https://patentgrep.com/patent/US11308697B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - A63F 13/65: automatically by game devices or servers from real world data, e.g. measurement in live racing competition - G05B 19/042: using digital processors (G05B19/05 takes precedence) - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06K 19/077: Constructional details, e.g. mounting of circuits in the carrier - G06N 20/00: Machine learning - H04L 12/282: {based on user interaction within the home (receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204)} - H04L 12/283: {Processing of data at an internetworking point of a home automation network} - G06F 1/163: {Wearable computers, e.g. on a belt} - H04N 21/466: Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)} - H04N 21/4781: {Games} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract One example of a computer-implemented method comprises analyzing data received from an automation control device to identify a plurality of automation devices communicatively coupled to the automation control device; determining a functionality of at least one of the plurality of automation devices not utilized by the user; presenting a gamification scenario associated with the determined functionality to a user in a virtual reality environment; monitoring interaction of the user with the gamification scenario; and enabling use of the determined functionality based on the monitored interaction of the user with the gamification scenario. ## Claims ### Claim 1 A computer-implemented method comprising: analyzing data received from an automation control device to identify a plurality of automation devices communicatively coupled to the automation control device; determining a functionality of at least one of the plurality of automation devices not utilized by a user and available to the user; presenting a gamification scenario associated with the determined functionality to the user in a virtual reality environment, wherein the virtual reality environment provides haptic feedback to the user, and the gamification scenario is configured to show the user how to identify and use the determined functionality; monitoring interaction of the user with the gamification scenario; and enabling use of the determined functionality based on the monitored interaction of the user with the gamification scenario. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the functionality not utilized by the user comprises: retrieving functionality data for each of the automation devices from a device provider server; identifying one or more historical uses by the user of the plurality of automation devices based on the data received form the automation control device; and comparing the one or more historical uses of the plurality of automation devices with the functionality data for each of the automation devices to determine the functionality not utilized by the user. ### Claim 3 (depends on 1) The method of claim 1, wherein presenting the gamification scenario includes retrieving the gamification scenario from a device provider server for the at least one automation device. ### Claim 4 (depends on 1) The method of claim 1, wherein the determined functionality is a functionality enabled by two or more of the plurality of automation devices together. ### Claim 5 (depends on 1) The method of claim 1, wherein enabling use of the determined functionality comprises communicating with the automation control device to update a profile associated with the user based on the monitored interaction of the user with the gamification scenario. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the functionality not utilized by the user comprises: retrieving functionality data for each of the automation devices from a device provider server; comparing the functionality data to a user profile of the user containing user preferences; and identifying differences between the user preferences and the functionality data based on the comparison to identify the functionality not utilized by the user. ### Claim 7 (depends on 1) The method of claim 1, wherein presenting the gamification scenario includes generating the gamification scenario based on the determined functionality and on an age of the user indicated in a user profile of the user. ### Claim 8 A virtual reality device comprising: a display device configured to present a virtual reality environment to a user; an interface; and a processor communicatively coupled to the interface and the display device, wherein the processor is configured to: analyze data received from an automation control device via the interface to identify a plurality of automation devices communicatively coupled to the automation control device; determine a functionality of at least one of the plurality of automation devices not utilized by a user and available to the user; present a gamification scenario associated with the determined functionality to the user in a virtual reality environment displayed in the display device, wherein the virtual reality environment provides haptic feedback to the user, and the gamification scenario is configured to show the user to identify and use the determined functionality; monitoring interaction of the user with the gamification scenario; and enable use of the determined functionality based on the monitored interaction of the user with the gamification scenario. ### Claim 9 (depends on 8) The virtual reality device of claim 8, wherein the processor is configured to determine the functionality not utilized by the user by: retrieving functionality data for each of the automation devices from a device provider server via the interface; identifying one or more historical uses by the user of the plurality of automation devices based on the data received form the automation control device; and comparing the one or more historical uses of the plurality of automation devices with the functionality data for each of the automation devices to determine the functionality not utilized by the user. ### Claim 10 (depends on 8) The virtual reality device of claim 8, wherein the processor is configured to retrieve the gamification scenario from a device provider server via the interface for the at least one automation device. ### Claim 11 (depends on 8) The virtual reality device of claim 8, wherein the determined functionality is a functionality enabled by two or more of the plurality of automation devices together. ### Claim 12 (depends on 8) The virtual reality device of claim 8, wherein the processor is configured to enable use of the determined functionality by communicating with the automation control device to update a profile associated with the user based on the monitored interaction of the user with the gamification scenario. ### Claim 13 (depends on 8) The virtual reality device of claim 8, wherein the processor is configured to determine the functionality not utilized by the user by: retrieving functionality data for each of the automation devices from a device provider server via the interface; comparing the functionality data to a user profile of the user containing user preferences; and identifying differences between the user preferences and the functionality data based on the comparison to identify the functionality not utilized by the user. ### Claim 14 (depends on 8) The virtual reality device of claim 8, wherein the processor is configured to generate the gamification scenario based on the determined functionality and on an age of the user indicated in a user profile of the user. ### Claim 15 A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed by a processor, causes the processor to: analyze data received from an automation control device via the interface to identify a plurality of automation devices communicatively coupled to the automation control device; determine a functionality of at least one of the plurality of automation devices not utilized by a user, and available to the user; present a gamification scenario associated with the determined functionality to a user in the virtual reality environment displayed in the display device, wherein the virtual reality environment provides haptic feedback to the user, and the gamification scenario is configured to show the user to identify and use the determined functionality; monitoring interaction of the user with the gamification scenario; and enable use of the determined functionality based on the monitored interaction of the user with the gamification scenario. ### Claim 16 (depends on 15) The computer program product of claim 15, wherein the program instructions are further configured to cause the processor to determine the functionality not utilized by the user by: retrieving functionality data for each of the automation devices from a device provider server via the interface; identifying one or more historical uses by the user of the plurality of automation devices based on the data received form the automation control device; and comparing the one or more historical uses of the plurality of automation devices with the functionality data for each of the automation devices to determine the functionality not utilized by the user. ### Claim 17 (depends on 15) The computer program product of claim 15, wherein the program instructions are further configured to cause the processor to retrieve the gamification scenario from a device provider server via the interface for the at least one automation device. ### Claim 18 (depends on 15) The computer program product of claim 15, wherein the program instructions are further configured to cause the processor to enable use of the determined functionality by communicating with the automation control device to update a profile associated with the user based on the monitored interaction of the user with the gamification scenario. ### Claim 19 (depends on 15) The computer program product of claim 15, wherein the program instructions are further configured to cause the processor to determine the functionality not utilized by the user by: retrieving functionality data for each of the automation devices from a device provider server via the interface; comparing the functionality data to a user profile of the user containing user preferences; and identifying differences between the user preferences and the functionality data based on the comparison to identify the functionality not utilized by the user. ### Claim 20 (depends on 15) The computer program product of claim 15, wherein the program instructions are further configured to cause the processor to generate the gamification scenario based on the determined functionality and on an age of the user indicated in a user profile of the user.