# US 11,307,891 B2 · Virtual reality application data processing method, computer device and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,891 B2 - Document ID: US11307891B2 - Status: Granted - Kind: B2 - Application number: 16/508,483 - Filed: 2019-07-11 - Published: 2019-10-31 - Granted: 2022-04-19 - Earliest claimed: 2017-05-09 - Assignees: Tencent Technology (Shenzhen) Company Limited - Applicants: Tencent Technology (Shenzhen) Company Limited - Inventors: Yang Zhou; Long fang Lin - Web page: https://patentgrep.com/patent/US11307891B2 ## CPC - A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use - G06F 9/485: {Task life-cycle, e.g. stopping, restarting, resuming execution (G06F9/4881 takes precedence)} - A63F 13/48: Starting a game, e.g. activating a game device or waiting for other players to join a multiplayer session - A63F 13/493: Resuming a game, e.g. after pausing, malfunction or power failure - A63F 13/497: Partially or entirely replaying previous game actions - A63F 13/5252: using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 9/451: Execution arrangements for user interfaces - G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)} - G06T 11/00: Two-dimensional [2D] image generation - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06V 20/20: in augmented reality scenes - A63F 13/77: involving data related to game devices or game servers, e.g. configuration data, software version or amount of memory - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method of processing data in a virtual reality (VR) application is provided. The VR application is restored to an initial running state in response to receiving a playback start instruction. Manipulation data that are transmitted to a runtime library to be pre-captured and stored are obtained, the manipulation data having a time sequence and being used to control running of the VR application. The manipulation data are imported into the runtime library based on the time sequence. The imported manipulation data are transferred to the VR application in the initial running state by using the runtime library and performing playback processing based on the transferred manipulation data. ## Claims ### Claim 1 A method of processing data in a virtual reality (VR) application, performed by a computing device comprising at least one processor, comprising: restoring, by at least one of the at least one processor, the VR application to an initial running state in response to receiving a playback start instruction; obtaining, by at least one of the at least one processor, manipulation data that are transmitted to a runtime library to be pre-captured and stored, the manipulation data having a time sequence and being used to control running of the VR application; importing, by at least one of the at least one processor, the manipulation data into the runtime library based on the time sequence; and transferring, by at least one of the at least one processor, the imported manipulation data to the VR application in the initial running state by using the runtime library and performing playback processing based on the transferred manipulation data, wherein the method further comprises: obtaining, by at least one of the at least one processor, a playback picture that is generated by the VR application based on the manipulation data, the playback picture being a picture of a first-party view of angle based on a picture drawing parameter value transferred by the VR application to a drawing function in the VR application; changing, by at least one of the at least one processor, the picture drawing parameter value based on a drawing parameter value conversion relationship of the first-party view of angle and a third-party view of angle; invoking, by at least one of the at least one processor, the drawing function, to draw a picture of the third-party view of angle based on the changed picture drawing parameter value; and displaying, by at least one of the at least one processor, the picture of the third-party view of angle, to superimpose the picture of the third-party view of angle onto the playback picture. ### Claim 2 (depends on 1) The method according to claim 1, further comprising: registering, by at least one of the at least one processor, a callback function with the runtime library; and invoking, by at least one of the at least one processor, the callback function in response to receiving the manipulation data, to capture and store the manipulation data. ### Claim 3 (depends on 1) The method according to claim 1, wherein the importing comprises: in response to determining that a playback manner being a first playback manner, shortening, by at least one of the at least one processor, a record time interval between manipulation data, neighboring based on the time sequence, based on an acceleration import multiple corresponding to the first playback manner; and importing, by at least one of the at least one processor, the manipulation data sequentially into the runtime library based on the time sequence and the shortened record time interval. ### Claim 4 (depends on 1) The method according to claim 1, further comprising: in response to determining that a playback manner being a second playback manner, obtaining, by at least one of the at least one processor, a time node corresponding to the manipulation data; and importing, by at least one of the at least one processor, the manipulation data sequentially into the runtime library based on the obtained time node. ### Claim 5 (depends on 3) The method according to claim 3, further comprising: determining, by at least one of the at least one processor, that the playback manner is the first playback manner based on a one-to-one mapping relationship between virtual three-dimensional coordinates in the VR application and physical three-dimensional coordinates in physical space. ### Claim 6 (depends on 4) The method according to claim 4, further comprising: determining, by at least one of the at least one processor, that the playback manner is the second playback manner in response to virtual three-dimensional coordinates in the VR application and physical three-dimensional coordinates in physical space not satisfying a one-to-one mapping relationship. ### Claim 7 (depends on 1) The method according to claim 1, wherein the displaying comprises: performing, by at least one of the at least one processor, blur processing on the playback picture; and displaying, by at least one of the at least one processor, the playback picture on which the blur processing is performed. ### Claim 8 (depends on 7) The method according to claim 7, wherein the picture of the third-party view of angle is superimposed onto the playback picture on which the blur processing is performed. ### Claim 9 (depends on 1) The method according to claim 1, further comprising: in response to receiving a playback stop instruction, stopping, by at least one of the at least one processor, importing the manipulation data into the runtime library, to end the playback processing; receiving, by at least one of the at least one processor by using the runtime library, instant manipulation data, and transferring the instant manipulation data to the VR application; and generating, by at least one of the at least one processor, a VR picture by using the VR application based on the instant manipulation data, and displaying the VR picture. ### Claim 10 A computer device, comprising: at least one memory operable to store program code; and at least one processor operable to read the program code and operate as instructed by the program code, the program code comprising: restoring code configured to cause at least one of the at least one processor to restore the VR application to an initial running state in response to receiving a playback start instruction; first obtaining code configured to cause at least one of the at least one processor to obtain manipulation data that are transmitted to a runtime library to be pre-captured and stored, the manipulation data having a time sequence and being used to control running of the VR application; first importing code configured to cause at least one of the at least one processor to import the manipulation data into the runtime library based on the time sequence; and transferring code configured to cause at least one of the at least one processor to transfer the imported manipulation data to the VR application in the initial running state by using the runtime library and performing playback processing based on the transferred manipulation data, wherein the program code further comprises: playback picture obtaining code configured to cause at least one of the at least one processor to obtain a playback picture that is generated by the VR application based on the manipulation data, the playback picture being a picture of a first-party view of angle based on a picture drawing parameter value transferred by the VR application to a drawing function in the VR application; changing code configured to cause at least one of the at least one processor to change the picture drawing parameter value based on a drawing parameter value conversion relationship of the first-party view of angle and a third-party view of angle; first invoking code configured to cause at least one of the at least one processor to invoke the drawing function, to draw a picture of the third-party view of angle based on the changed picture drawing parameter value; and displaying code configured to cause at least one of the at least one processor to display the picture of the third-party view of angle, to superimpose the picture of the third-party view of angle onto the playback picture. ### Claim 11 (depends on 10) The computer device according to claim 10, wherein the program code further comprises: registering code configured to cause at least one of the at least one processor to register a callback function with the runtime library; and second invoking code configured to cause at least one of the at least one processor to invoke the callback function in response to receiving the manipulation data, to capture and store the manipulation data. ### Claim 12 (depends on 10) The computer device according to claim 10, wherein the first importing code comprises: shortening code configured to cause at least one of the at least one processor to, in response to determining that a playback manner being a first playback manner, shorten a record time interval between manipulation data, neighboring based on the time sequence, based on an acceleration import multiple corresponding to the first playback manner; and importing code configured to cause at least one of the at least one processor to import the manipulation data sequentially into the runtime library based on the time sequence and the shortened record time interval. ### Claim 13 (depends on 10) The computer device according to claim 10, wherein the program code further comprises: second obtaining code configured to cause at least one of the at least one processor to obtain, in response to determining that a playback manner being a second playback manner, a time node corresponding to the manipulation data; and second importing code configured to cause at least one of the at least one processor to import the manipulation data sequentially into the runtime library based on the obtained time node. ### Claim 14 (depends on 10) The computer device according to claim 10, wherein the displaying code is configured to cause at least one of the at least one processor to perform blur processing on the playback picture, and display the playback picture on which the blur processing is performed. ### Claim 15 (depends on 14) The computer device according to claim 14, wherein the picture of the third-party view of angle is superimposed onto the playback picture on which the blur processing is performed. ### Claim 16 (depends on 10) The computer device according to claim 10, wherein the program code further comprises: stopping code configured to cause at least one of the at least one processor to, in response to receiving a playback stop instruction, stop importing the manipulation data into the runtime library, to end the playback processing; receiving code configured to cause at least one of the at least one processor to, by using the runtime library, receive instant manipulation data, and transfer the instant manipulation data to the VR application; and second generating code configured to cause at least one of the at least one processor to generate a VR picture by using the VR application based on the instant manipulation data, and display the VR picture. ### Claim 17 (depends on 1) The method according to claim 1, wherein the obtaining the picture drawing parameter value comprises: obtaining, by at least one of the at least one processor, the picture drawing parameter value transferred by the VR application to the drawing function by using a hook function injected into the drawing function, and wherein the invoking comprises: invoking, by at least one of the at least one processor, the drawing function by using the hook function injected into the drawing function, to draw the picture of the third-party view of angle based on the changed picture drawing parameter value. ### Claim 18 A non-transitory storage medium storing a computer-readable instruction, which is executable by at least one processor to perform: restoring the VR application to an initial running state in response to receiving a playback start instruction; obtaining manipulation data that are transmitted to a runtime library to be pre-captured and stored, the manipulation data having a time sequence and being used to control running of the VR application; importing the manipulation data into the runtime library according to the time sequence; and transferring the imported manipulation data to the VR application in the initial running state by using the runtime library and performing playback processing based on the transferred manipulation data, wherein the computer-readable instruction is executable by the at least one processor to further perform: obtaining a playback picture that is generated by the VR application based on the manipulation data, the playback picture being a picture of a first-party view of angle based on a picture drawing parameter value transferred by the VR application to a drawing function in the VR application; changing the picture drawing parameter value based on a drawing parameter value conversion relationship of the first-party view of angle and a third-party view of angle; invoking the drawing function in the VR application, to obtain a picture of the third-party view of angle by changing the picture drawing parameter; and displaying the picture of the third-party view of angle, to superimpose the picture of the third-party view of angle onto the playback picture.