# US 2019/0172499 A1 · Recording user activity on a computer > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0172499 A1 - Document ID: US20190172499A1 - Status: Published application - Kind: A1 - Application number: 15/833,169 - Filed: 2017-12-06 - Published: 2019-06-06 - Earliest claimed: 2017-12-06 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: MICHAL BODZIONY; ARTUR M. GRUSZECKI; TOMASZ KAZALSKI; RAFAL KORCZYK; TOMASZ SEKMAN - Web page: https://patentgrep.com/patent/US20190172499A1 ## CPC - G06F 16/74: Browsing; Visualisation therefor (end-user interfaces for requesting or interacting with video content, e.g. video on demand interfaces or electronic programme guides, H04N21/472) - G06F 8/427: {Parsing} - G06F 16/70: of video data - G06F 16/95: Retrieval from the web - G06F 16/9577: {Optimising the visualization of content, e.g. distillation of HTML documents} - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - G11B 27/3036: {Time code signal} - G11B 27/36: Monitoring, i.e. supervising the progress of recording or reproducing {(for digital recording G11B20/00 and s.gr.; for monitoring, testing or measuring of TV recorders of the type covered by H04N5/76 and subgroups, see H04N17/06)} - G06F 40/143: Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract The present disclosure relates to a method for recording user activity on a computer. The method includes creating a video file for a video sequence of a user interface of an application of the computer. Video frames may be recorded in the video file, wherein a video frame comprises data indicative of the visual representation of the user interface and corresponding timestamp, wherein the data of the video frame is parsable. ## Claims ### Claim 1 A method for recording user activity on a computer, the method comprising: creating a video file for a video sequence of an application delivered through a browser interface of web browser on the computer; recording, via functionality of the web browser, video frames in the video file, wherein a video frame comprises data indicative of a visual representation of the browser interface and content of the browser interface as displayed on a display of the computer and a corresponding timestamp, and wherein the data of the visual representation of the video frame is generated using a http response; capturing at least part of http traffic of the application that is destined for the web browser; and generating the data from the captured http traffic that corresponds to the visual representation. ### Claim 2 (canceled) ### Claim 3 (depends on 1) The method of claim 1, further comprising: generating a Document Object Model, DOM, structure of captured http traffics, wherein the data comprises the DOM structure. ### Claim 4 (depends on 1) The method of claim 1, further comprising: generating a Document Object Model, DOM, structure of a source code of the browser interface, wherein the data comprises the DOM structure. ### Claim 5 (depends on 1) The method of claim 1, further comprising: parsing a source code of the browser interface, wherein the data comprises the parsed source code corresponding to the visual representation. ### Claim 6 (depends on 1) The method of claim 1, further comprising receiving a text search request, searching the data of the video frames for identifying the video frames having the requested text, and providing the identified video frames. ### Claim 7 (depends on 1) The method of claim 1, wherein the recording of a current video frame comprises: in response to determining that a current visual representation is different from the visual representation of a previously stored video frame, modifying the previously stored video frame to reflect a difference between the visual representations, and storing the modified previously stored video frame as the current video frame. ### Claim 8 (depends on 1) The method of claim 1, the video file comprising a header comprising an indication of data type of the data of the video frames, a payload comprising the video frames and an index comprising a mapping of time values and an indication of the video frames. ### Claim 9 (depends on 8) The method of claim 8, further comprising receiving a predefined time value and using the index for identifying the video frame corresponding to the time value, and using the identified video frame for providing a video subsequence of the video sequence. ### Claim 10 (depends on 1) The method of claim 1, further comprising sorting in association with each video frame and audio frame indicative of the visual representation of the browser interface. ### Claim 11 (depends on 10) The method of claim 10, the audio frame comprising ambient voice of the computer recorded at the timestamp. ### Claim 12 (depends on 1) The method of claim 1, the application being a web application involving a web browser. ### Claim 13 (depends on 12) The method of claim 12, further comprising storing the video as a history content of a web browser of the web application. ### Claim 14 (depends on 1) The method of claim 1, further comprising decoding the video file by rendering the data of each of the video frames in accordance with a predefined frame rate. ### Claim 15 (depends on 1) The method of claim 1, further comprising inserting a digital signature in the video file. ### Claim 16 (depends on 1) The method of claim 1, further comprising storing the video file in a container having one of AVI or MKV formats. ### Claim 17 (depends on 1) The method of claim 1, further comprising using a player for transforming or rendering the content of the frames into subsequent images of the video sequence. ### Claim 18 (depends on 1) A computer program product comprising a computer-readable storage device having computer-readable program code embodied therewith, the computer-readable program code configured to implement the steps of the method according to claim 1. ### Claim 19 A computer system for recording user activity on the computer system, being configured for: creating a video file for a video sequence of an application delivered through a browser interface of web browser on the computer; recording, via functionality of the web browser, video frames in the video file, wherein a video frame comprises data indicative of a visual representation of the browser interface and content of the browser interface as displayed on a display of the computer and a corresponding timestamp, wherein the data of the visual representation of the video frame is generated using a http response; capturing at least part of http traffic of the application that is destined for the web browser; and generating the data from the captured http traffic that corresponds to the visual representation.