# US 12,114,022 B2 · Identification of decreased object detectability for a video stream > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,114,022 B2 - Document ID: US12114022B2 - Status: Granted - Kind: B2 - Application number: 17/285,509 - Filed: 2018-10-17 - Published: 2021-12-09 - Granted: 2024-10-08 - Earliest claimed: 2018-10-17 - Assignees: Telefonaktiebolaget LM Ericsson (publ) - Applicants: Telefonaktiebolaget LM Ericsson (publ) - Inventors: Volodya Grancharov; Manish Sonal - Web page: https://patentgrep.com/patent/US12114022B2 ## CPC - H04N 19/89: involving methods or arrangements for detection of transmission errors at the decoder - G06F 18/2193: {based on specific statistical tests} - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 7/00: Image analysis - G06T 7/136: involving thresholding - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/54: relating to texture - G06V 10/56: relating to colour - G06V 10/764: using classification, e.g. of video objects - G06V 10/82: using neural networks - G06V 10/993: {Evaluation of the quality of the acquired pattern} - G06V 20/20: in augmented reality scenes - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06T 2207/10016: Video; Image sequence - G06T 2207/20076: Probabilistic image processing - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30196: Human being; Person - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract There is provided mechanisms for identifying decreasing object detectability for a video stream. A method is performed by a controller. The method includes receiving the video stream over a communication channel. The method includes receiving first visual object detection (VOD) data of the video stream. The first VOD data has been obtained from the video stream before is sent over the communication channel towards the controller. The method includes obtaining second VOD data of the video stream. The second VOD data is obtained from the video stream after is received by the controller. The method includes determining, by comparing the first VOD data to the second VOD data, whether any object present in the first VOD data has a decreased object detectability in the second VOD data, thereby identifying the decreasing object detectability for the video stream. The method includes issuing a notification when the decreasing object detectability has been identified. ## Claims ### Claim 1 A method for identifying decreasing object detectability for a video stream, the method being performed by a controller, the method comprising: receiving the video stream over a communication channel; receiving first visual object detection, VOD, data of the video stream, wherein the first VOD data has been obtained from the video stream before being sent over the communication channel towards the controller, wherein the first VOD data includes a first detection probability value; obtaining second VOD data of the video stream, wherein the second VOD data is obtained from the video stream after being received by the controller, wherein the controller receives a subset of all first VOD data obtained from the video stream before being sent over the communication channel, wherein the subset is selected based on a bandwidth limitation of the communication channel, wherein the second VOD data includes a second detection probability value; determining, by comparing the first VOD data to the second VOD data, whether at least one object present in the first VOD data has a decreased object detectability in the second VOD data, thereby identifying the decreasing object detectability for the video stream, wherein comparing the first VOD data to the second VOD data includes utilizing one or more of the first detection probability value and the second detection probability value to determine a detectability level for the at least one object and wherein the identifying the decreasing object detectability includes determining that the detectability level is below a detectability threshold value; and issuing a notification when the decreasing object detectability has been identified. ### Claim 2 (depends on 1) The method according to claim 1, wherein said determining comprises: matching objects in the first VOD data with objects in the second VOD data. ### Claim 3 (depends on 1) The method according to claim 1, wherein said determining comprises: determining that a detectability level for said any object in the second VOD data is below a detectability threshold value. ### Claim 4 (depends on 3) The method according to claim 3, wherein the detectability level is determined from a detection probability for said any object in the first VOD data and a detection probability for said any object in the second VOD data. ### Claim 5 (depends on 1) The method according to claim 1, wherein said determining comprises: determining that a texture level for a region of interest in a video frame of the video stream obtained over the communication channel is below a texture threshold value. ### Claim 6 (depends on 5) The method according to claim 5, wherein the texture level is determined from a measure of texture of the region of interest in a video frame of the video stream before being sent over the communication channel towards the controller and a measure of texture of the region of interest in a video frame of the video stream obtained over the communication channel. ### Claim 7 (depends on 1) The method according to claim 1, wherein each of the first VOD data and the second VOD data is frame-wise obtained from video frames in the video stream. ### Claim 8 (depends on 7) The method according to claim 7, wherein when comparing the first VOD data to the second VOD data, the first VOD data of a given video frame is compared to the second VOD data of same frame number as well as to the second VOD data in video frames surrounding the video frame of said same frame number. ### Claim 9 (depends on 1) The method according to claim 1, wherein each object in the first VOD data and each object in the second VOD data is represented by a respective tuple, each tuple comprising: an object type field, a detection probability field, a bounding box field, and a video frame number field. ### Claim 10 (depends on 1) The method according to claim 1, wherein each of the first VOD data and the second VOD data is application specific VOD data. ### Claim 11 (depends on 10) The method according to claim 10, wherein the application specific VOD data relates to a remote vehicle driving application or a remote surgery application. ### Claim 12 (depends on 1) The method according to claim 1, wherein the communication channel is a lossy communication channel. ### Claim 13 A controller for identifying decreasing object detectability for a video stream, the controller comprising: a receive module configured to receive the video stream over a communication channel; a receive module configured to receive first visual object detection, VOD, data of the video stream, wherein the first VOD data has been obtained from the video stream before being sent over the communication channel towards the controller, wherein the first VOD data includes a first detection probability value; an obtain module configured to obtain second VOD data of the video stream, wherein the second VOD data is obtained from the video stream after being received by the controller, wherein the controller receives a subset of all first VOD data obtained from the video stream before being sent over the communication channel, wherein the subset is selected based on a bandwidth limitation of the communication channel, wherein the second VOD data includes a second detection probability value; a determine module configured to determine, by comparing the first VOD data to the second VOD data, whether any object present in the first VOD data has a decreased object detectability in the second VOD data, thereby identifying the decreasing object detectability for the video stream, wherein comparing the first VOD data to the second VOD data includes utilizing one or more of the first detection probability value and the second detection probability value to determine a detectability level for the at least one object and wherein the identifying the decreasing object detectability includes determining that the detectability level is below a detectability threshold value; and an issue module configured to issue a notification when the decreasing object detectability has been identified. ### Claim 14 (depends on 13) A system for identifying decreasing object detectability for a video stream, the system comprising the controller according to claim 13, wherein the video stream originates from a video capturing system, wherein the first VOD data has been obtained by the controller from the video stream as captured at the video capturing system, and wherein the system further comprises the video capturing system. ### Claim 15 A controller for identifying decreasing object detectability for a video stream, the controller comprising processing circuitry, the processing circuitry being configured to cause the controller to: receive the video stream over a communication channel; receive first visual object detection, VOD, data of the video stream, wherein the first VOD data has been obtained from the video stream before being sent over the communication channel towards the controller, wherein the first VOD data includes a first detection probability value; obtain second VOD data of the video stream, wherein the second VOD data is obtained from the video stream after being received by the controller, wherein the controller receives a subset of all first VOD data obtained from the video stream before being sent over the communication channel, wherein the subset is selected based on a bandwidth limitation of the communication channel, wherein the second VOD data includes a second detection probability value; determine, by comparing the first VOD data to the second VOD data, whether any object present in the first VOD data has a decreased object detectability in the second VOD data, thereby identifying the decreasing object detectability for the video stream, wherein comparing the first VOD data to the second VOD data includes utilizing one or more of the first detection probability value and the second detection probability value to determine a detectability level for the at least one object and wherein the identifying the decreasing object detectability includes determining that the detectability level is below a detectability threshold value; and issue a notification when the decreasing object detectability has been identified. ### Claim 16 (depends on 15) The controller according to claim 15, wherein the video stream originates from a video capturing system, and wherein the first VOD data has been obtained from the video stream as captured at the video capturing system. ### Claim 17 (depends on 15) The controller according to claim 15, wherein the second VOD data is obtained from the video stream as to be sent to a screen for visualization. ### Claim 18 (depends on 17) The controller according to claim 17, wherein the notification is provided as a bounding box surrounding a region of interest in a video frame of the video stream to be sent to the screen for visualization. ### Claim 19 (depends on 18) The controller according to claim 18, wherein the region of interest is defined by said any object present in the first VOD data.