# US 10,455,188 B2 · Correlating UI with CPU stacks for profiling sessions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,455,188 B2 - Document ID: US10455188B2 - Status: Granted - Kind: B2 - Application number: 15/356,180 - Filed: 2016-11-18 - Published: 2018-05-24 - Granted: 2019-10-22 - Earliest claimed: 2016-11-18 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Nikolaus L. Karpinsky - Web page: https://patentgrep.com/patent/US10455188B2 ## CPC - H04N 5/775: between a recording apparatus and a television receiver - G06F 11/3466: {Performance evaluation by tracing or monitoring} - G06F 11/36: Prevention of errors by analysis, debugging or testing of software - G06F 11/3636: {by tracing the execution of the program} - G06F 11/3698: Environments for analysis, debugging or testing of software - G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel - H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract Methods, systems, apparatuses, and computer program products are provided for correlating a user interface (UI) of an executing process with a profiling trace generated for the process. A profiling trace of a process is captured during execution of the process. A window handle associated with the process is determined. Video of a displayed window having the window handle is captured simultaneous with the capturing of the profiling trace. The displayed window displays a user interface (e.g., a graphical user interface) of the process. The captured profiling trace and the video are stored in a profiling trace object, and may be analyzed together to diagnose any problems/issues with the process. ## Claims ### Claim 1 A method in a computing device, comprising: capturing a profiling trace of a process during execution of the process; determining a window handle associated with the process; capturing video of a displayed window having the window handle during said capturing a profiling trace, said capturing video of the displayed window comprising: detecting a lack of activity in the process; and pausing said capturing video in response to said detecting the lack of activity in the process; storing the captured profiling trace and the video in a profiling trace object; providing a user interface enabling a user to mark a start time and a stop time in the video corresponding to a time period of interest; and displaying the profiling trace corresponding to the time period of interest. ### Claim 2 (depends on 1) The method of claim 1, wherein said capturing video comprises: capturing video of the displayed window having the window handle during said capturing a profiling trace, the displayed window displaying a user interface of the process. ### Claim 3 (depends on 1) The method of claim 1, wherein said determining comprises: determining a root window handle associated with the process, the root window handle being a handle for a displayed root window that includes a sub-window that displays a user interface of the process. ### Claim 4 (depends on 1) The method of claim 1, wherein said capturing a profiling trace comprises: emitting an event detected during operation of the process. ### Claim 5 (depends on 1) The method of claim 1, further comprising: correlating the profiling trace and the captured video. ### Claim 6 (depends on 1) The method of claim 1, further comprising: analyzing the process by enabling a user to play the video to determine a time period of interest for the process in the video; and enabling the user to select for examination the portion of the profiling trace corresponding to the time period of interest. ### Claim 7 (depends on 1) The method of claim 1, further comprising: resuming said capturing video in response to detecting a resumption of activity in the process. ### Claim 8 A computing device, comprising: at least one processor circuit; and at least one memory that stores instructions configured to be executed by the at least one processor circuit, the instructions configured to perform operations for correlating a user interface of a process with a profiling information for the process, the operations comprising: capturing a profiling trace of a process during execution of the process; determining a window handle associated with the process; capturing video of a displayed window having the window handle during said capturing a profiling trace, said capturing video comprising: detecting a lack of activity in the process; and pausing said capturing video in response to said detecting the lack of activity in the process; storing the captured profiling trace and the video in a profiling trace object; providing a user interface enabling a user to mark a start time and a stop time in the video corresponding to a time period of interest; and displaying the profiling trace corresponding to the time period of interest. ### Claim 9 (depends on 8) The computing device of claim 8, wherein said capturing video comprises: capturing video of the displayed window having the window handle during said capturing a profiling trace, the displayed window displaying a user interface of the process. ### Claim 10 (depends on 8) The computing device of claim 8, wherein said determining comprises: determining a root window handle associated with the process, the root window handle being a handle for a displayed root window that includes a sub-window that displays a user interface of the process. ### Claim 11 (depends on 8) The computing device of claim 8, wherein said capturing a profiling trace comprises: emitting an event detected during operation of the process. ### Claim 12 (depends on 8) The computing device of claim 8, wherein the operations further comprise: correlating the profiling trace and the captured video. ### Claim 13 (depends on 8) The computing device of claim 8, wherein the operations further comprise: analyzing the process by enabling a user to play the video to determine a time period of interest for the process in the video; and enabling the user to select for examination the portion of the profiling trace corresponding to the time period of interest. ### Claim 14 (depends on 8) The computing device of claim 8, further comprising: resuming said capturing video in response to detecting a resumption of activity in the process. ### Claim 15 A system in a computing device, comprising: a profile trace capturer configured to: capture a profiling trace of a process during execution of the process, and detect a lack of activity in the process; a window handle determiner configured to determine a window handle associated with the process; a video capturer configured to: capture video of a displayed window having the window handle during capture of the profiling trace, and pause said capture of video in response to the detection of the lack of activity in the process; a captured data packager configured to store the captured profiling trace and the video in a profiling trace object; a video player configured to enable a user to mark a start time and a stop time in the video corresponding to a time period of interest; and a profiling trace displayer configured to display the profiling trace corresponding to the time period of interest. ### Claim 16 (depends on 15) The system of claim 15, wherein the displayed window displays a user interface of the process. ### Claim 17 (depends on 15) The system of claim 15, wherein the window handle determiner is configured to determine a root window handle associated with the process, the root window handle being a handle for a displayed root window that includes a sub-window that displays a user interface of the process. ### Claim 18 (depends on 15) The system of claim 15, wherein the profile trace capturer is configured to emit an event detected during operation of the process. ### Claim 19 (depends on 15) The system of claim 15, further comprising: a profile analyzer configured to analyze the process, the profile analyzer configured to enable a user to play the video to determine a time period of interest for the process in the video, and enable the user to select for examination the portion of the profiling trace corresponding to the time period of interest. ### Claim 20 (depends on 15) The system of claim 15, wherein the video capturer is configured to the capture of video in response to a detection of a resumption of activity in the process.