# US 12,192,263 B2 · Systems and methods for quality of experience computation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,263 B2 - Document ID: US12192263B2 - Status: Granted - Kind: B2 - Application number: 17/890,683 - Filed: 2022-08-18 - Published: 2024-02-22 - Granted: 2025-01-07 - Earliest claimed: 2022-08-18 - Assignees: Adeia Guides Inc. - Applicants: Adeia Guides, Inc. - Inventors: Zhu Li; Tao Chen - Web page: https://patentgrep.com/patent/US12192263B2 ## CPC - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - H04L 65/80: Responding to QoS - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/09: Supervised learning - G06N 3/092: Reinforcement learning - G06N 20/00: Machine learning - H04L 65/75: Media network packet handling - H04L 65/764: {at the destination (reformatting of additional data in video clients H04N21/4355)} - H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The system trains a machine learning model using a loss function, with a part that penalizes overall signal loss, and a second part of the loss function that penalizes texture loss. The system computes a first neural feature of a first media frame stored by a media server using the trained machine learning model. The system causes a client device to receive a second media frame as a part of a media stream from the media server where the second frame is a modified version of the first media frame. The system causes the client to compute a second neural feature of the second media frame using the trained machine learning model, and compute a QoE metric based on the first neural feature and the second neural feature. The system receives the QoE metric, and uses it to modify at least one parameter of the media stream. ## Claims ### Claim 1 A method comprising: training a first machine learning model to generate a neural feature for an inputted media frame using a loss function, wherein a first part of the loss function penalizes overall signal loss and a second part of the loss function penalizes texture loss, wherein the first machine learning model comprises weights; computing a first neural feature of a first media frame stored by a media server using the trained first machine learning model; causing a client device to receive the first neural feature; causing the client device to receive a second media frame as a part of a media stream from the media server, wherein the second media frame is a degraded version of the first media frame, wherein the client device is configured to: receive the weights of the first machine learning model; compute a second neural feature of the second media frame using a second trained machine learning model, wherein the second machine learning model is configured with the received weights of the first machine learning model; compute a Quality of Experience (QoE) metric based at least on the first neural feature and the second neural feature; receiving the QoE metric from the client device; and modifying, based on the QoE metric, at least one parameter of the media stream transmitted by the media server. ### Claim 2 (depends on 1) The method of claim 1, wherein computing the QoE metric based on the first neural feature and the second neural feature comprises: inputting the first neural feature and the second neural feature as inputs into a third trained machine learning model trained to receive two neural features as input and to output at least one of a peak signal to noise ratio measure, a structural similarity index measure, a video multi-method assessment fusion measure, a mean opinion score measure, a just noticeable difference measure, or a visual information fidelity measure. ### Claim 3 (depends on 1) The method of claim 1, wherein the first neural feature is transmitted to the client device as part of a supplemental enhancement information for the media stream. ### Claim 4 (depends on 1) The method of claim 1, wherein the trained first machine learning model and the second machine learning model comprise convolutional auto encoders. ### Claim 5 (depends on 4) The method of claim 4, wherein arithmetic coding is applied to the first neural feature prior to transmission to the client device. ### Claim 6 (depends on 1) The method of claim 1, wherein the loss function comprises a third part that penalizes bit rate of the neural feature. ### Claim 7 (depends on 1) The method of claim 1, wherein the modifying at least one parameter of the media stream based on the QoE metric comprises switching the media stream to another server. ### Claim 8 (depends on 1) The method of claim 1, wherein the modifying at least one parameter of the media stream based on the QoE metric comprises adjusting bit rate of the media stream. ### Claim 9 (depends on 1) The method of claim 1, wherein the modifying at least one parameter of the media stream based on the QoE metric comprises modifying the storage location of frames of the media stream. ### Claim 10 (depends on 1) The method of claim 1, wherein the first neural feature and the second media frame are transmitted to the client by the media server. ### Claim 11 A system comprising: control circuitry configured to: train a first machine learning model to generate a neural feature for an inputted media frame using a loss function, wherein a first part of the loss function penalizes overall signal loss and a second part of the loss function penalizes texture loss, wherein the first machine learning model comprises weights; compute a first neural feature of a first media frame stored by a media server using the trained first machine learning model; cause the client device to receive the first neural feature; cause a client device to receive a second media frame as a part of a media stream from the media server, wherein the second media frame is a degraded version of the first media frame, wherein the client device is configured to: receive the weights of the first machine learning model; compute a second neural feature of the second media frame using a second trained machine learning model, wherein the second machine learning model is configured with the received weights of the first machine learning model; compute a Quality of Experience (QoE) metric based at least on the first neural feature and the second neural feature; and an input/output circuitry configured to: receive the QoE metric; and wherein the control circuitry is further configured to: modify, based on the QoE metric, at least one parameter of the media stream transmitted by the media server. ### Claim 12 (depends on 11) The system of claim 11, wherein the QoE metric is computed based on the first neural feature and the second neural feature by: inputting the first neural feature and the second neural feature as inputs into a third trained machine learning model trained to receive two neural features as input and to output at least one of a peak signal to noise ratio measure, a structural similarity index measure, a video multi-method assessment fusion measure, a mean opinion score measure, a just noticeable difference measure, or a visual information fidelity measure. ### Claim 13 (depends on 11) The system of claim 11, wherein the Input/Output circuitry is further configured to: transmit the first neural feature to the client device as part of a supplemental enhancement information for the media stream. ### Claim 14 (depends on 11) The system of claim 11, wherein the trained first machine learning model and the second machine learning model are convolutional auto encoders. ### Claim 15 (depends on 14) The system of claim 14, wherein the control circuitry is further configured to apply arithmetic coding to first neural feature prior to transmission to the client device. ### Claim 16 (depends on 11) The system of claim 11, wherein the loss function comprises a third part that penalizes bit rate of the neural feature. ### Claim 17 (depends on 11) The system of claim 11, wherein the control circuitry is configured to modify at least one parameter of the media stream based on the QoE metric by switching the media stream to another server.