# US 11,310,553 B2 · Changing resource utilization associated with a media object based on an engagement score > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,310,553 B2 - Document ID: US11310553B2 - Status: Granted - Kind: B2 - Application number: 17/234,435 - Filed: 2021-04-19 - Published: 2021-12-23 - Granted: 2022-04-19 - Earliest claimed: 2020-06-19 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Shruti Singhal; Yasser Saby Hafez Rihan; Jonathan Iain Wight; Yinxuan Shi Moore - Web page: https://patentgrep.com/patent/US11310553B2 ## CPC - H04N 21/42201: {biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user (input arrangements for interaction with the human body based on nervous system activity detection G06F3/015)} - H04N 21/42653: {for processing graphics} - H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)} - H04N 21/42202: {environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes} - H04N 21/4318: {by altering the content in the rendering process, e.g. blanking, blurring or masking an image region (image enhancement or restoration in general G06T5/00)} - H04N 21/440245: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/44204: {Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched (monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31)} - H04N 21/4621: {Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen (arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10)} - H04N 21/4667: {Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections} - H04N 21/47205: {for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally} - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N 21/816: {involving special video data, e.g 3D video} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In accordance with some implementations, a method is performed at an electronic device with one or more processors, a non-transitory memory, and a display. The method includes displaying, on the display, a representation of a first portion of a media object, wherein the first portion of the media object is associated with a first resource utilization value. The first resource utilization value characterizes a utilization of a respective resource by the electronic device. The method includes determining an engagement score that characterizes a level of user engagement with respect to the representation of the first portion of the media object. The method includes changing the utilization of the respective resource from the first resource utilization value to a second resource utilization value based on a function of the engagement score. The second resource utilization value is associated with a second portion of the media object. ## Claims ### Claim 1 A method comprising: at an electronic device with one or more processors, a non-transitory memory, one or more input devices, and a display: displaying, on the display, a representation of a first portion of a media object, wherein the first portion of the media object is associated with a first resource utilization value, and wherein the first resource utilization value characterizes a utilization of a respective resource by the electronic device; detecting, via the one or more input devices, a positional change input, wherein the positional change input is indicative of the electronic device changing from a first position to a second position; in response to detecting the positional change input, determining an engagement score that characterizes a level of user engagement with respect to the representation of the first portion of the media object, wherein: the engagement score corresponds to a first value according to a determination that the representation of the first portion of the media object is located at a first location on the display that is less than a threshold distance from the center of the display; the engagement score corresponds to a second value that is less than the first value according to a determination that the representation of the first portion of the media object is located at a second location on the display that is not less than the threshold distance from the center of the display; and the engagement score corresponds to a nominal value according to a determination that the representation of the first portion of the media object is not located on the display; and changing the utilization of the respective resource from the first resource utilization value to a second resource utilization value based on the engagement score, wherein the second resource utilization value is associated with a second portion of the media object. ### Claim 2 (depends on 1) The method of claim 1, further comprising: registering a change to the engagement score, wherein changing the utilization of the respective resource from the first resource utilization value to the second resource utilization value is in response to determining that the change to the engagement score satisfies a change criterion. ### Claim 3 (depends on 2) The method of claim 2, wherein: when the change to the engagement score corresponds to an increase of the engagement score, the second resource utilization value is higher than the first resource utilization value; and when the change to the engagement score corresponds to a decrease of the engagement score, the second resource utilization value is lower than the first resource utilization value. ### Claim 4 (depends on 1) The method of claim 1, wherein the respective resource corresponds to a rendering resource, wherein the first resource utilization value characterizes the electronic device rendering the first portion of the media object, and wherein the second resource utilization value characterizes the electronic device rendering the second portion of the media object. ### Claim 5 (depends on 4) The method of claim 4, further comprising: rendering, according to the first resource utilization value, the first portion of the media object in order to generate the representation of the first portion of the media object; and rendering, according to the second resource utilization value, the second portion of the media object in order to generate a representation of the second portion of the media object. ### Claim 6 (depends on 5) The method of claim 5, wherein the representation of the first portion of the media object is characterized by a first video resolution, and wherein the representation of the second portion of the media object is characterized by a second video resolution that is different from the first video resolution. ### Claim 7 (depends on 1) The method of claim 1, wherein the respective resource corresponds to a communication link resource, wherein the first resource utilization value characterizes the electronic device obtaining the first portion of the media object, and wherein the second resource utilization value characterizes the electronic device obtaining the second portion of the media object. ### Claim 8 (depends on 7) The method of claim 7, further comprising: obtaining, according to the first resource utilization value, the first portion of the media object from a content provisioning system at a first output rate; and obtaining, according to the second resource utilization value, the second portion of the media from the content provisioning system at a second output rate that is different from the first output rate. ### Claim 9 (depends on 8) The method of claim 8, wherein the content provisioning system is integrated in the electronic device. ### Claim 10 (depends on 8) The method of claim 8, wherein the content provisioning system corresponds to a content delivery server that is separate from the electronic device. ### Claim 11 (depends on 10) The method of claim 10, wherein the electronic device corresponds to an adaptive bit rate (ABR) enabled device, and wherein changing the utilization of the respective resource from the first resource utilization value to the second resource utilization value includes transmitting, to the content delivery server, a request for the second portion of the media object. ### Claim 12 (depends on 10) The method of claim 10, wherein the content delivery server is included in a content delivery network (CDN). ### Claim 13 (depends on 1) The method of claim 1, wherein the electronic device includes an eye tracker sensor that provides eye tracking data associated with the user, and wherein determining the engagement score is based on a function of the eye tracking data. ### Claim 14 (depends on 13) The method of claim 13, wherein the representation of the first portion of the media object is displayed at a first location on the display, the method further comprising: identifying a gaze position based on a function of the eye tracking data; in accordance with a determination that the gaze position is less than a threshold distance from the first location, setting the engagement score to a first value; and in accordance with a determination that the gaze position is not less than the threshold distance from the first location, setting the engagement score to a second value that is less than first value. ### Claim 15 (depends on 1) The method of claim 1, wherein the electronic device includes a depth sensor that provides depth data with respect to the first portion of the media object, and wherein determining the engagement score is based on a function of the depth data. ### Claim 16 (depends on 1) The method of claim 1, further comprising: in response to determining that the engagement score falls below a threshold: ceasing to display the representation of the first portion of the media object; and playing, via a speaker integrated in the electronic device, an audio representation of the second portion of the media object, wherein the audio representation corresponds to spatial audio that is played based on a function of the engagement score. ### Claim 17 An electronic device comprising: one or more processors; a non-transitory memory; one or more input devices; a display; and one or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for: displaying, on the display, a representation of a first portion of a media object, wherein the first portion of the media object is associated with a first resource utilization value, and wherein the first resource utilization value characterizes a utilization of a respective resource by the electronic device; detecting, via the one or more input devices, a positional change input, wherein the positional change input is indicative of the electronic device changing from a first position to a second position; in response to detecting the positional change input, determining an engagement score that characterizes a level of user engagement with respect to the representation of the first portion of the media object, wherein: the engagement score corresponds to a first value according to a determination that the representation of the first portion of the media object is located at a first location on the display that is less than a threshold distance from the center of the display; the engagement score corresponds to a second value that is less than the first value according to a determination that the representation of the first portion of the media object is located at a second location on the display that is not less than the threshold distance from the center of the display; and the engagement score corresponds to a nominal value according to a determination that the representation of the first portion of the media object is not located on the display; and changing the utilization of the respective resource from the first resource utilization value to a second resource utilization value based on the engagement score, wherein the second resource utilization value is associated with a second portion of the media object. ### Claim 18 (depends on 17) The electronic device of claim 17, wherein the respective resource corresponds to a rendering resource, wherein the first resource utilization value characterizes the electronic device rendering the first portion of the media object, and wherein the second resource utilization value characterizes the electronic device rendering the second portion of the media object. ### Claim 19 (depends on 17) The electronic device of claim 17, wherein the respective resource corresponds to a communication link resource, wherein the first resource utilization value characterizes the electronic device obtaining the first portion of the media object, and wherein the second resource utilization value characterizes the electronic device obtaining the second portion of the media object. ### Claim 20 (depends on 17) The electronic device of claim 17, wherein the electronic device corresponds to an adaptive bit rate (ABR) enabled device, and wherein changing the utilization of the respective resource from the first resource utilization value to the second resource utilization value includes transmitting, to the content delivery server, a request for the second portion of the media object. ### Claim 21 A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with one or more processors, one or more input devices, and a display, cause the electronic device to: display, on the display, a representation of a first portion of a media object, wherein the first portion of the media object is associated with a first resource utilization value, and wherein the first resource utilization value characterizes a utilization of a respective resource by the electronic device; detect, via the one or more input devices, a positional change input, wherein the positional change input is indicative of the electronic device changing from a first position to a second position; in response to detecting the positional change input, determine an engagement score that characterizes a level of user engagement with respect to the representation of the first portion of the media object, wherein: the engagement score corresponds to a first value according to a determination that the representation of the first portion of the media object is located at a first location on the display that is less than a threshold distance from the center of the display; the engagement score corresponds to a second value that is less than the first value according to a determination that the representation of the first portion of the media object is located at a second location on the display that is not less than the threshold distance from the center of the display; and the engagement score corresponds to a nominal value according to a determination that the representation of the first portion of the media object is not located on the display; and change the utilization of the respective resource from the first resource utilization value to a second resource utilization value based on the engagement score, wherein the second resource utilization value is associated with a second portion of the media object. ### Claim 22 (depends on 21) The non-transitory computer readable storage medium of claim 21, wherein the respective resource corresponds to a rendering resource, wherein the first resource utilization value characterizes the electronic device rendering the first portion of the media object, and wherein the second resource utilization value characterizes the electronic device rendering the second portion of the media object. ### Claim 23 (depends on 21) The non-transitory computer readable storage medium of claim 21, wherein the respective resource corresponds to a communication link resource, wherein the first resource utilization value characterizes the electronic device obtaining the first portion of the media object, and wherein the second resource utilization value characterizes the electronic device obtaining the second portion of the media object. ### Claim 24 (depends on 21) The non-transitory computer readable storage medium of claim 21, wherein the electronic device corresponds to an adaptive bit rate (ABR) enabled device, and wherein changing the utilization of the respective resource from the first resource utilization value to the second resource utilization value includes transmitting, to the content delivery server, a request for the second portion of the media object.