# US 10,313,746 B2 · Server, client and video processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,313,746 B2 - Document ID: US10313746B2 - Status: Granted - Kind: B2 - Application number: 14/892,194 - Filed: 2013-12-10 - Published: 2016-04-21 - Granted: 2019-06-04 - Earliest claimed: 2013-05-20 - Assignees: Beijing Zhigu Rui Tuo Tech Co., Ltd - Applicants: Beijing Zhigu Rui Tuo Tech Co., Ltd - Inventors: Lei Pan; Hongjiang Zhang; Kuifei Yu - Web page: https://patentgrep.com/patent/US10313746B2 ## CPC - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - H04L 65/60: Network streaming of media packets - H04L 65/612: for unicast - H04L 65/762: {at the source (reformatting of additional data in video distribution servers H04N21/2355)} - H04N 21/226: {Characteristics of the server or} Internal components of the server - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} - H04N 21/858: Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A method includes: a server sending at least one piece of video related information to at least one client, where an image of a frame in a video corresponding to the video related information is formed of multiple sub-regions; the client receiving a video and the video related information sent by the server; selecting a sub-region; analyzing the video according to the sub-region and generating corresponding video analysis information; returning the video analysis information to the server; the server receiving the video analysis information returned by the at least one client; and adjusting the video related information according to the video analysis information. The server and client include modules for implementing functions in the method. Processing cost of the video server is reduced and the efficiency and accuracy of video analysis is improved. ## Claims ### Claim 1 A method, comprising: sending, by a system comprising a processor, a video and at least one piece of video related information to at least one client, wherein an image of a frame in the video is formed of multiple sub-regions; receiving at least one piece of video analysis information returned by the at least one client, wherein the at least one piece of video analysis information comprises identity information of an object in a sub-region of the multiple sub-regions; and adjusting the at least one piece of video related information according to the at least one piece of video analysis information; wherein the adjusting the at least one piece of video related information comprises performing a first action selected from a first group of actions, the first group of actions comprising: adding the identity information of the object to the at least one piece of video related information, modifying the identity information of the object in the at least one piece of video related information, and deleting the identity information of the object from the at least one piece of video related information. ### Claim 2 (depends on 1) The method of claim 1, wherein the multiple sub-regions are distributed in the form of grids. ### Claim 3 (depends on 1) The method of claim 1, wherein sizes of the multiple sub-regions are set according to an image feature of the video. ### Claim 4 (depends on 3) The method of claim 3, wherein the image feature comprises resolution of the image or scene information of the image. ### Claim 5 (depends on 1) The method of claim 1, wherein the identity information of the object comprises adaptation-completed information. ### Claim 6 (depends on 1) The method of claim 1, wherein the at least one piece of video analysis information further comprises hyperlink information of the object in the sub-region of the multiple sub-regions. ### Claim 7 (depends on 6) The method of claim 6, wherein the adjusting the at least one piece of video related information according to the at least one piece of video analysis information comprises performing a second action selected from a second group of actions, the second group of actions comprising: adding the hyperlink information of the object to the at least one piece of video related information, modifying the hyperlink information of the object in the at least one piece of video related information, and deleting the hyperlink information of the object from the at least one piece of video related information. ### Claim 8 (depends on 1) The method of claim 1, wherein the at least one piece of video analysis information comprises sub-region setting optimization information. ### Claim 9 (depends on 8) The method of claim 8, wherein the adjusting the at least one piece of video related information according to the at least one piece of video analysis information comprises: adjusting settings of the sub-region of the multiple sub-regions according to the sub-region setting optimization information. ### Claim 10 (depends on 1) The method of claim 1, wherein the at least one piece of video related information comprises sub-region completion index information. ### Claim 11 (depends on 10) The method of claim 10, wherein the adjusting the at least one piece of video related information according to the at least one piece of video analysis information comprises: updating the sub-region completion index information according to the at least one piece of video analysis information. ### Claim 12 (depends on 1) The method of claim 1, wherein: the receiving the at least one piece of video analysis information returned by the at least one client comprises receiving the at least one piece of video analysis information returned by multiple clients of the at least one client, and the method further comprises: before the adjusting of the at least one piece of video related information according to the at least one piece of video analysis information, integrating the at least one piece of video analysis information in response to the receiving the at least one piece of video analysis returned by the multiple clients. ### Claim 13 (depends on 1) The method of claim 1, further comprising: before the adjusting of the at least one piece of video related information according to the at least one piece of video analysis information: selecting at least one sub-region of the multiple sub-regions; performing video analysis on the at least one sub-region; and generating at least one piece of corresponding video analysis information. ### Claim 14 (depends on 13) The method of claim 13, further comprising: before the adjusting of the at least one piece of video related information according to the at least one piece of video analysis information, integrating the at least one piece of video analysis information returned by the at least one client and the at least one piece of video analysis information generated by a server. ### Claim 15 A method, comprising: receiving, by a system comprising a processor, a video and at least one piece of video related information sent by a server, wherein an image of a frame in the video is formed of multiple sub-regions; selecting at least one sub-region of the multiple sub-regions, resulting in at least one selected sub-region; performing video analysis on the at least one selected sub-region; generating at least one piece of video analysis information as a result of the performing the video analysis, wherein the at least one piece of video analysis information comprises identity information of an object in a sub-region of the multiple sub-regions; and returning the at least one piece of video analysis information to the server, to initiate adjusting of the at least one piece of video related information according to the at least one piece of video analysis information, wherein the adjusting of the at least one piece of video related information according to the at least one piece of video analysis information comprises performing an action selected from a group of actions, the group of actions comprising: adding the identity information of the object to the at least one piece of video related information, modifying the identity information of the object in the at least one piece of video related information, and deleting the identity information of the object from the at least one piece of video related information. ### Claim 16 (depends on 15) The method of claim 15, wherein the selecting the at least one sub-region comprises: selecting the at least one sub-region according to an attention determined for a user. ### Claim 17 (depends on 15) The method of claim 15, wherein the selecting the at least one sub-region comprises: selecting the at least one sub-region according to feature information of a user. ### Claim 18 (depends on 15) The method of claim 15, wherein the selecting the at least one sub-region comprises: selecting the at least one sub-region according to completion index information in the at least one piece of video related information. ### Claim 19 (depends on 15) The method of claim 15, wherein the selecting the at least one sub-region comprises: selecting the at least one sub-region according to motion prediction of associated pixel groups in the video. ### Claim 20 (depends on 15) The method of claim 15, wherein the performing the video analysis on the sub-region, or the generating the at least one piece of corresponding video analysis information comprise: generating the identity information of the object identified in the sub-region. ### Claim 21 (depends on 15) The method of claim 15, wherein the identity information of the object comprises adaptation-completed information. ### Claim 22 (depends on 15) The method of claim 15, wherein the at least one piece of video related information comprises adaptation-completed information. ### Claim 23 (depends on 22) The method of claim 22, wherein the generating the identity information of the object comprises performing mode adaptation on the object in the sub-region, according to the adaptation-completed information, and updating the adaptation-completed information. ### Claim 24 (depends on 15) The method of claim 15, further comprising: generating hyperlink information corresponding to the object. ### Claim 25 (depends on 15) The method of claim 15, wherein the at least one piece of video related information comprises hyperlink information of the object in the video. ### Claim 26 (depends on 15) The method of claim 15, further comprising storing hyperlink information corresponding to the object. ### Claim 27 (depends on 15) The method of claim 15, wherein the performing the video analysis on the sub-region, or the generating the at least one piece of corresponding video analysis information comprise: generating sub-region setting optimization information. ### Claim 28 (depends on 27) The method of claim 27, wherein the generating the sub-region setting optimization information comprises: generating the sub-region setting optimization information according to an interaction with a user. ### Claim 29 (depends on 27) The method of claim 27, wherein the generating the sub-region setting optimization information comprises: generating the sub-region setting optimization information according to a shape feature of associated pixel groups in the image of the frame of the video. ### Claim 30 (depends on 27) The method of claim 27, wherein the generating the sub-region setting optimization information comprises: generating the sub-region setting optimization information according to an inter-frame relation of the video. ### Claim 31 A server, comprising: a memory that stores executable modules; and a processor, coupled to the memory, that executes the executable modules to perform operations, the executable modules comprising: a server input/output module used for sending at least one piece of video related information to at least one client, wherein an image of a frame in a video corresponding to the at least one piece of video related information is formed of multiple sub-regions, and for receiving at least one piece of video analysis information returned by the at least one client, wherein the at least one piece of video analysis information comprises identity information of an object in a sub-region of the multiple sub-regions; and a video related information processing module used for adjusting the at least one piece of video related information according to the at least one piece of video analysis information, wherein the adjusting the at least one piece of video related information according to the at least one piece of video analysis information comprises initiating an action selected from a group of actions comprising: adding the identity information of the object to the at least one piece of video related information, modifying the identity information of the object in the at least one piece of video related information, and deleting the identity information of the object from the at least one piece of video related information. ### Claim 32 (depends on 31) The server of claim 31, wherein the server input/output module is further used for receiving at least one piece of video analysis information returned by multiple clients, and wherein the video related information processing module comprises an integration processing unit used for integrating the at least one piece of video analysis information returned by the at least one client. ### Claim 33 (depends on 31) The server of claim 31, the executable modules further comprising: a server sub-region selection module used for selecting at least one sub-region of the multiple sub-regions; and a server video analysis module used for generating corresponding at least one piece of video analysis information in response to performing video analysis on the at least one sub-region. ### Claim 34 (depends on 33) The server of claim 33, wherein the video related information processing module comprises: an integration processing unit used for integrating the at least one piece of video analysis information returned by the at least one client and at least one piece of video analysis information generated by the server. ### Claim 35 A client device, comprising: a memory that stores executable modules; and a processor, coupled to the memory, that executes the executable modules to perform operations of the client device, the executable modules comprising: a client input/output module used for receiving a video and at least one piece of video related information that are sent by a server, wherein an image of a frame in the video is formed by multiple sub-regions; a client sub-region selection module used for selecting a selected sub-region from at least one sub-region of the multiple sub-regions; and a client video analysis module used for performing video analysis on the selected sub-region, and generating corresponding at least one piece of video analysis information, wherein the at least one piece of video analysis information comprises identity information of an object in a sub-region of the multiple sub-regions, wherein the client input/output module is further used for returning the at least one piece of video analysis information to the server, so as to cause the server to perform an adjustment of the at least one piece of video related information according to the at least one piece of video analysis information; wherein the adjustment of the at least one piece of video related information according to the at least one piece of video analysis information comprises an action selected from a group of actions comprising: adding the identity information of the object to the at least one piece of video related information, modifying the identity information of the object in the at least one piece of video related information, and deleting the identity information of the object from the at least one piece of video related information. ### Claim 36 (depends on 35) The client device of claim 35, wherein the client sub-region selection module comprises: an attention-based selection unit used for selecting the selected sub-region according to attention of a user. ### Claim 37 (depends on 36) The client device of claim 36, wherein the client sub-region selection module comprises: a feature information-based selection unit used for selecting the selected sub-region according to feature information of a user. ### Claim 38 (depends on 37) The client device of claim 37, wherein the client sub-region selection module comprises: a motion prediction-based selection unit used for selecting the selected sub-region according to motion prediction of associated pixel groups in the video. ### Claim 39 (depends on 37) The client device of claim 37, wherein the client sub-region selection module comprises: a completion-based selection unit used for determining whether to select the selected sub-region according to completion index information in the at least one piece of video related information. ### Claim 40 (depends on 37) The client device of claim 37, wherein the client video analysis module comprises: an identification unit used for identifying an object in the selected sub-region, and generating identity information of the object. ### Claim 41 (depends on 40) The client device of claim 40, wherein the identity information comprises adaptation-completed information. ### Claim 42 (depends on 40) The client device of claim 40, wherein the at least one piece of video related information comprises adaptation-completed information. ### Claim 43 (depends on 42) The client device of claim 42, wherein the identification unit performs mode adaptation on the object in the selected sub-region according to the adaptation-completed information, and updates the adaptation-completed information. ### Claim 44 (depends on 40) The client device of claim 40, wherein the client video analysis module further comprises: a hyperlink generating unit used for generating hyperlink information corresponding to the object. ### Claim 45 (depends on 37) The client device of claim 37, wherein the at least one piece of video related information comprises hyperlink information of the object in the video. ### Claim 46 (depends on 44) The client device of claim 44, further comprising: a storage module used for storing the hyperlink information. ### Claim 47 (depends on 37) The client device of claim 37, wherein the client video analysis module comprises: an optimization information generating unit used for generating sub-region setting optimization information. ### Claim 48 (depends on 47) The client device of claim 47, wherein the optimization information generating unit generates the sub-region setting optimization information according to an interaction with the user. ### Claim 49 (depends on 47) The client device of claim 47, wherein the optimization information generating unit generates the sub-region setting optimization information according to a shape feature of associated pixel groups in the image of the frame of the video. ### Claim 50 (depends on 47) The client device of claim 47, wherein the optimization information generating unit generates the sub-region setting optimization information according to an inter-frame relation of the video. ### Claim 51 A computer readable storage device comprising executable instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising: sending a video and at least one piece of video related information to a client device, wherein an image of a frame in the video is formed of multiple sub-regions; receiving at least one piece of video analysis information returned by the client device wherein the at least one piece of video analysis information comprises identity information of an object in a sub-region of the multiple sub-regions; and adjusting the at least one piece of video related information according to the at least one piece of video analysis information, wherein the adjusting the at least one piece of video related information according to the at least one piece of video analysis information comprises: adding the identity information of the object to the at least one piece of video related information, modifying the identity information of the object in the at least one piece of video related information, or deleting the identity information of the object from the at least one piece of video related information. ### Claim 52 A computer readable storage device comprising executable instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising: receiving a video and at least one piece of video related information sent by a server, wherein an image of a frame in the video comprises multiple sub-regions; selecting a selected sub-region of the multiple sub-regions; performing video analysis on the selected sub-region; generating at least one piece of video analysis information as a result of the performing the video analysis, wherein the at least one piece of video analysis information comprises identity information of an object in a sub-region of the multiple sub-regions; and returning the at least one piece of video analysis information to the server, so as to cause the server to adjust the at least one piece of video related information according to the at least one piece of video analysis information; wherein adjusting the at least one piece of video related information according to the at least one piece of video analysis information comprises at least one of: adding the identity information of the object to the at least one piece of video related information, modifying the identity information of the object in the at least one piece of video related information, or deleting the identity information of the object from the at least one piece of video related information. ### Claim 53 (depends on 35) The client device of claim 35, the executable modules further comprising: a display module used for displaying the video. ### Claim 54 (depends on 31) The server of claim 31, wherein the video related information processing module further comprises a sub-region completion index information processing unit used for updating sub-region completion index information according to the video analysis information.