# US 12,602,633 B2 · Data center guide creation and cost estimation for augmented reality headsets > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,633 B2 - Document ID: US12602633B2 - Status: Granted - Kind: B2 - Application number: 18/450,442 - Filed: 2023-08-16 - Published: 2025-02-20 - Granted: 2026-04-14 - Earliest claimed: 2023-08-16 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Sowjanya Rao; Rodolfo Lopez; Aastha Khatri; Prakash Ramasamy - Web page: https://patentgrep.com/patent/US12602633B2 ## CPC - G06Q 10/063112: {Skill-based matching of a person or a group to a task} - G06Q 10/0633: Workflow analysis - G06Q 10/20: Administration of product repair or maintenance - G06T 11/60: Creating or editing images; Combining images with text - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method, computer program product and computer system to evaluate maintenance performance in a data center is provided. A processor provides an augmented reality device to a user performing a maintenance procedure, where the augmented reality device displays a guidance script to the user while performing the maintenance procedure. A processor determines an initial base cost associated with the maintenance procedure. A processor retrieves user activity data regarding the performance of the maintenance procedure. A processor determines an activity cost of the user performing the procedure. A processor, in response to the activity cost varying a threshold amount from the initial base cost, adjusts the initial base cost of the maintenance procedure based on the activity data of the user regarding the performance of the maintenance procedure. ## Claims ### Claim 1 A computer-implemented method comprising: causing, by one or more processors, an augmented reality device to guide a user through performing a maintenance procedure to a device, wherein the augmented reality device displays: (i) a step of the maintenance procedure and (ii) an overlay highlighting an area of the device corresponding to the step of the maintenance procedure, to the user while performing the maintenance procedure; determining, by one or more processors, an initial base cost associated with the maintenance procedure; monitoring, by one or more processors, user activity data, in real time, regarding performance of the maintenance procedure, the user activity data including a length of time to complete the step of the maintenance procedure; receiving, by one or more processors, feedback from the user indicating that the overlay highlighting the area of the device corresponding to the step of the maintenance procedure is incorrectly located; collating, by one or more processors, the feedback from the user with additional feedback, wherein the additional feedback also indicates that the overlay highlighting the area of the device corresponding to the step of the maintenance procedure is incorrectly located; determining, by one or more processors, a correct position for the overlay utilizing a machine learning algorithm; responsive to the feedback and the additional feedback, changing, by one or more processors, a position of the overlay to the correct position; determining, by one or more processors, an activity cost of the user performing the maintenance procedure; and responsive to the activity cost varying above a threshold amount from the initial base cost, automatically adjusting, by one or more processors, the initial base cost of the maintenance procedure based on the user activity data regarding the performance of the maintenance procedure. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the activity cost is determined based on a machine learning algorithm. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein the machine learning algorithm is trained based on additional activity data from other performances of the maintenance procedure. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, the computer-implemented method further comprising: evaluating, by one or more processors, the maintenance procedure with the machine learning algorithm; and determining, by one or more processors, a new order of tasks for the maintenance procedure based on the evaluation of the maintenance procedure by the machine learning algorithm. ### Claim 5 (depends on 3) The computer-implemented method of claim 3, the computer-implemented method further comprising: evaluating, by one or more processors, ongoing activity data with the machine learning algorithm, wherein the ongoing activity data is gathered from an ongoing maintenance procedure; and responsive to the ongoing activity data indicating variance from an expected cost of the ongoing maintenance procedure, identifying, by one or more processors, a new user to perform the ongoing maintenance procedure. ### Claim 6 (depends on 3) The computer-implemented method of claim 3, wherein the machine learning algorithm is trained to minimize time spent performing the maintenance procedure. ### Claim 7 (depends on 3) The computer-implemented method of claim 3, wherein the machine learning algorithm is trained to maximize compliance to requirements of the maintenance procedure. ### Claim 8 A computer program product comprising: one or more computer-readable storage devices; and program instructions stored on the one or more computer-readable storage devices to perform operations comprising: causing an augmented reality device to guide a user through performing a maintenance procedure to a device, wherein the augmented reality device displays: (i) a step of the maintenance procedure and (ii) an overlay highlighting an area of the device corresponding to the step of the maintenance procedure, to the user while performing the maintenance procedure; determining an initial base cost associated with the maintenance procedure; monitoring user activity data, in real time, regarding performance of the maintenance procedure, the user activity data including a length of time to complete the step of the maintenance procedure; receiving feedback from the user indicating that the overlay highlighting the area of the device corresponding to the step of the maintenance procedure is incorrectly located; collating the feedback from the user with additional feedback, wherein the additional feedback also indicates that the overlay highlighting the area of the device corresponding to the step of the maintenance procedure is incorrectly located; determining a correct position for the overlay utilizing a machine learning algorithm; responsive to the feedback and the additional feedback, changing a position of the overlay to the correct position; determining an activity cost of the user performing the maintenance procedure; and responsive to the activity cost varying above a threshold amount from the initial base cost, automatically adjusting the initial base cost of the maintenance procedure based on the user activity data regarding the performance of the maintenance procedure. ### Claim 9 (depends on 8) The computer program product of claim 8, wherein the activity cost is determined based on a machine learning algorithm. ### Claim 10 (depends on 9) The computer program product of claim 9, wherein the machine learning algorithm is trained based on additional activity data from other performances of the maintenance procedure. ### Claim 11 (depends on 10) The computer program product of claim 10, wherein the operations further comprise: evaluating the maintenance procedure with the machine learning algorithm; and determining a new order of tasks for the maintenance procedure based on the evaluation of the maintenance procedure by the machine learning algorithm. ### Claim 12 (depends on 10) The computer program product of claim 10, wherein the operations further comprise: evaluating ongoing activity data with the machine learning algorithm, wherein the ongoing activity data is gathered from an ongoing maintenance procedure; and responsive to the ongoing activity data indicating variance from an expected cost of the ongoing maintenance procedure, identifying a new user to perform the ongoing maintenance procedure. ### Claim 13 A computer system comprising: a processor set; one or more computer readable storage media; and program instructions stored on the one or more computer readable storage media to cause the processor set to perform operations comprising: causing an augmented reality device to guide a user through performing a maintenance procedure to a device, wherein the augmented reality device displays: (i) a step of the maintenance procedure and (ii) an overlay highlighting an area of the device corresponding to the step of the maintenance procedure, to the user while performing the maintenance procedure; determining an initial base cost associated with the maintenance procedure; monitoring user activity data, in real time, regarding performance of the maintenance procedure, the user activity data including a length of time to complete the step of the maintenance procedure; receiving feedback from the user indicating that the overlay highlighting the area of the device corresponding to the step of the maintenance procedure is incorrectly located; collating the feedback from the user with additional feedback, wherein the additional feedback also indicates that the overlay highlighting the area of the device corresponding to the step of the maintenance procedure is incorrectly located; determining a correct position for the overlay utilizing a machine learning algorithm; responsive to the feedback and the additional feedback, changing a position of the overlay to the correct position; determining an activity cost of the user performing the maintenance procedure; and responsive to the activity cost varying above a threshold amount from the initial base cost, automatically adjusting the initial base cost of the maintenance procedure based on the user activity data regarding the performance of the maintenance procedure. ### Claim 14 (depends on 13) The computer system of claim 13, wherein the activity cost is determined based on a machine learning algorithm. ### Claim 15 (depends on 14) The computer system of claim 14, wherein the machine learning algorithm is trained based on additional activity data from other performances of the maintenance procedure. ### Claim 16 (depends on 15) The computer system of claim 15, wherein the operations further comprise: evaluating the maintenance procedure with the machine learning algorithm; and determining a new order of tasks for the maintenance procedure based on the evaluation of the maintenance procedure by the machine learning algorithm. ### Claim 17 (depends on 15) The computer system of claim 15, wherein the operations further comprise: evaluating ongoing activity data with the machine learning algorithm, wherein the ongoing activity data is gathered from an ongoing maintenance procedure; and responsive to the ongoing activity data indicating variance from an expected cost of the ongoing maintenance procedure, identifying a new user to perform the ongoing maintenance procedure. ### Claim 18 (depends on 15) The computer system of claim 15, wherein the machine learning algorithm is trained to minimize time spent of the maintenance procedure. ### Claim 19 (depends on 1) The computer-implemented method of claim 1, further comprising: receiving, by one or more processors, non-structured text that includes steps necessary to perform a maintenance routine for the device; extracting, by one or more processors, an imperative statement contained in the non-structured text; comparing, by one or more processors, an object in the imperative statement to a schematic of the device; and generating, by one or more processors, the step of the maintenance procedure from the non-structured text and the schematic of the device. ### Claim 20 (depends on 1) The computer-implemented method of claim 1, wherein determining the activity cost of the user performing the maintenance procedure is based on the monitored user activity data and cost factors associated with a predetermined skill of the user.