Segmented parallel encoding with frame-aware, variable-size chunking

A distributed computing system for transcoding content in parallel on a plurality of machines coupled via a computer network, the system comprising:

• a first machine programmed to assist in transcoding and running a first transcoding process, the first machine comprising circuitry forming one or more processors and memory holding computer program instructions to execute on the one or more processors, the execution of the computer program instructions providing the first transcoding process;
• a second machine programmed to assist in transcoding and running a second transcoding process, the second machine comprising circuitry forming one or more processors and memory holding computer program instructions to execute on the one or more processors, the execution of the computer program instructions providing the second transcoding process;
• a management machine to manage and coordinate transcoding processes, the management machine comprising circuitry forming one or more processors and memory holding computer program instructions to execute on the one or more processors;
• wherein, upon execution of the instructions, the management machine receives a request to perform a transcoding process, the request comprising a request to convert a file from a first version to a second version, the first version being an original version of the file and the second version being an output file of the requested transcoding process;
• wherein, upon execution of the instructions, the management server receives the first version of the file, the file comprising a plurality of video frames organized into a plurality of groups-of-pictures (GoPs), each of the plurality of GoPs including at least an I-frame and at least one of:
• a B frame and a P frame;
• wherein the received first version of the file comprises GoPs with a first size, size being expressed in number of frames in each GoP, and the requested transcoding process comprises conversion of the first version of the file to a second version of the file that comprises GoPs with a second size, the second size being smaller than the first size;
• wherein, upon execution of the instructions, the management server compares the first size and the second size, and performs mathematical division to determine that the first size cannot be evenly divided into the second size, and upon said determination:
• creates a first segment from the received first version of the file, the first segment including all frames in a given GoP from the first version of the file, and a subset of frames from a GoP immediately following the given GoP in the first version of the file,
• sends the first segment to the first machine,
• creates a second segment from the received first version of the file, the second segment including all frames in another given GoP from the first version of the file, and a subset of frames from a GoP immediately following the another given GoP in the first version of the file,
• sends the second segment to the second machine;
• the first machine comprising the one or more processors and the memory thereof, the memory holding computer program instructions that, upon execution by the one or more processors, cause the first machine to execute the first transcoding process to receive the first segment from the management machine, and from the first segment create a first plurality of GoPs with the second size, whereby increasing transcoding speed relative to the number of encoders and reducing the encoding time;
• the second machine comprising the one or more processors and the memory thereof, the memory holding computer program instructions that, upon execution by the one or more processors, cause the second machine to execute the second transcoding process to:
• receive the second segment from the management machine, and from the second segment create a second plurality of GoPs with the second size, whereby increasing transcoding speed relative to the number of encoders and reducing the encoding time.