# US 10,310,928 B1 · Dynamic selection of multimedia segments using input quality metrics > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,310,928 B1 - Document ID: US10310928B1 - Status: Granted - Kind: B1 - Application number: 15/469,856 - Filed: 2017-03-27 - Granted: 2019-06-04 - Earliest claimed: 2017-03-27 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Ryan Paul Hegar; Khawaja Salman Shams - Web page: https://patentgrep.com/patent/US10310928B1 ## CPC - G06F 11/0784: {Routing of error reports, e.g. with a specific transmission path or data flow} - H04L 43/10: Active monitoring, e.g. heartbeat, ping or trace-route - G06F 3/0635: {by changing the path, e.g. traffic rerouting, path reconfiguration} - G06F 11/0709: {in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems} - G06F 11/0754: {by exceeding limits} - H04L 43/0823: Errors, e.g. transmission errors - H04L 43/16: Threshold monitoring - G06F 11/2035: {without idle spare hardware} - G06F 11/2048: {where the redundant components share neither address space nor persistent storage} - G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)} - G06F 11/3409: {for performance assessment} - G06F 2201/81: Threshold - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Multimedia input may be received by two or more transcoders. A primary transcoder may transcode the multimedia input and generate a first multimedia output segment having a first segment quality metric based at least in part on errors detected as part of the transcoding process. A second transcoder may transcode the multimedia input and generate a redundant second multimedia output segment encoded with a segment quality metric. The segment quality metrics may be encoded in a manifest file associated with each particular segment. A multimedia delivery platform may receive redundant segments from two or more transcoders and select, among the redundant segments, a segment to provide to a downstream entity based at least in part on the quality metric of the segments. ## Claims ### Claim 1 A system, comprising: one or more processors; memory that stores computer-executable instructions that, if executed, cause the system to: obtain a first multimedia stream comprising of a first plurality of multimedia segments in an encoding format and including segment quality metrics based at least in part on whether transcoding errors were detected during transcoding of segments of the first plurality; obtain a second multimedia stream comprising of a second plurality of multimedia segments in the same encoding format and including segment quality metrics based at least in part on whether transcoding errors were detected during transcoding of segments of the second plurality; generate a multimedia output stream comprising a third plurality of multimedia segments by selecting multimedia segments from either the first plurality or the second plurality, wherein a segment of the third plurality is selected based at least in part on comparing a first segment quality metrics of a particular segment of the first plurality and a second segment quality metric associated with a segment of the second plurality that corresponds to the particular segment of the first plurality, wherein the third plurality comprises a first segment of the first plurality and a second segment of the second plurality; and provide the multimedia output stream to another system. ### Claim 2 (depends on 1) The system of claim 1, wherein: at least one multimedia segment of the third plurality comprises a plurality of frames having frame quality metrics based at least in part on whether transcoding errors were detected during transcoding of the plurality of frames; and the at least one multimedia segment of the third plurality has an associated segment quality metric that is based at least in part on the frame quality metrics of at least some of the plurality of frames. ### Claim 3 (depends on 1) The system of claim 1, wherein a segment of the first plurality has an associated manifest that includes the segment quality metric of the segment. ### Claim 4 (depends on 3) The system of claim 3, wherein: the memory that stores the computer-executable instructions that, if executed, further causes the system to: obtain a multimedia segment associated with the selected segment of the third plurality; compare a third segment quality metric associated with the obtained multimedia segment with a fourth segment quality metric associated with the selected segment of the third plurality; and based at least in part on the comparing, modify the multimedia output stream by at least replacing the selected segment of the third plurality with the obtained segment; and the multimedia output stream is made available after the selected segment is replaced by the obtained segment. ### Claim 5 A computer-implemented method, comprising: obtaining, from a first transcoder, a first multimedia segment of a first plurality of multimedia segments associated with a multimedia input, wherein a first segment quality metric based at least in part on one or more errors detected by the first transcoder is associated with the first multimedia segment; obtaining, from a second transcoder, a second transcoded multimedia segment of a second plurality of multimedia segments associated with the multimedia input, wherein a second segment quality metric based at least in part on one or more errors detected by the second transcoder is associated with the second multimedia segment, and further wherein the first multimedia segment and the second multimedia segment share a same encoding format; selecting, between the first multimedia segment and the second multimedia segment, an output segment based at least in part on the first segment quality metric and the second segment quality metric; and generating a multimedia output stream comprising a third plurality of multimedia segments and based at least in part on the output segment, wherein the multimedia output stream comprises multimedia segments from both the first plurality and the second plurality. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein the first multimedia segment comprises a plurality of frames having frame quality metrics based at least in part on whether transcoding errors were detected during transcoding of the plurality of frames; and the first multimedia segment has an associated segment quality metric that is based at least in part on the frame quality metrics of at least some of the plurality of frames. ### Claim 7 (depends on 5) The computer-implemented method of claim 5, wherein a manifest is associated with the first multimedia segment, the manifest comprising information that includes: the first segment quality metric; and the one or more errors detected by the first transcoder. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein the one or more errors includes: an input data error; a frame error; a decoding error; or a volume error. ### Claim 9 (depends on 5) The computer-implemented method of claim 5, wherein the method further comprises: obtaining another multimedia segment associated with the output segment; comparing a third segment quality metric associated with the another multimedia segment with a fourth segment quality metric associated with the output segment; and based at least in part on the comparing, modifying the multimedia output stream by at least replacing the output segment with the another segment; and wherein the multimedia output stream is made available after the selected segment is replaced by the obtained segment. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, wherein: selecting the output segment occurs after waiting a predetermined period of time; and the another multimedia segment is obtained after the predetermined period of time. ### Claim 11 (depends on 5) The computer-implemented method of claim 5, wherein the first transcoder and the second transcoder are located in different locations. ### Claim 12 (depends on 5) The computer-implemented method of claim 5, wherein selecting the output segment includes selecting which of the first multimedia segment and the second multimedia segment has a higher segment quality metric. ### Claim 13 A non-transitory computer-readable storage medium comprising executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to: obtain, from a first transcoder, a first multimedia segment of a first plurality of multimedia segments associated with a multimedia input, wherein a first segment quality metric based at least in part on one or more errors detected by the first transcoder is associated with the first multimedia segment; obtain, from a second transcoder, a second transcoded multimedia segment of a second plurality of multimedia segments associated with the multimedia input, wherein a second segment quality metric based at least in part on one or more errors detected by the second transcoder is associated with the second multimedia segment, and further wherein the first multimedia segment and the second multimedia segment share a same encoding format; select, between the first multimedia segment and the second multimedia segment, an output segment based at least in part on the first segment quality metric and the second segment quality metric; and generate a multimedia output stream comprising a third plurality of multimedia segments and based at least in part on the output segment, wherein the multimedia output stream comprises multimedia segments from both the first plurality and the second plurality. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the instructions to select the output segment include instructions that, as a result of being executed by the one or more processors, cause the system to select the output segment with a predetermined period of time. ### Claim 15 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the first segment quality metric is calculated based at least in part on determining whether a transcoding error was detected as part of transcoding the first multimedia segment. ### Claim 16 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the instructions to select the output segment include instructions that, as a result of execution by the one or more processors, cause the computer system to compare at least one of the first segment quality metric and the second segment quality metric against a threshold value. ### Claim 17 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the first multimedia segment comprises a plurality of frames, one or more frames of the plurality of frames including metadata indicating a transcoding success or a transcoding error. ### Claim 18 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein the first segment quality metric is calculated based at least in part on frame quality metrics associated with the frames of the plurality, wherein the frame quality metrics are calculated based at least in part on the metadata included in the frames. ### Claim 19 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the instructions to select the output segment include instructions that, as a result of execution by the one or more processors, cause the computer system to select a highest segment quality metric among two or more segment quality metrics. ### Claim 20 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the instructions to select the output segment include instructions that, as a result of execution by the one or more processors, cause the computer system to: obtain an identifier associated with the first multimedia segment; and obtain the first segment quality based at least in part on the identifier.