# US 2018/0322369 A1 · Combining independent solutions to an image or video processing task > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322369 A1 - Document ID: US20180322369A1 - Status: Published application - Kind: A1 - Application number: 15/973,184 - Filed: 2018-05-07 - Published: 2018-11-08 - Earliest claimed: 2016-06-08 - Assignees: GoPro, Inc. - Applicants: GoPro, Inc. - Inventors: Balineedu Chowdary Adsumilli; Ryan Lustig - Web page: https://patentgrep.com/patent/US20180322369A1 ## CPC - G06N 3/126: Evolutionary algorithms, e.g. genetic algorithms or genetic programming - G06F 18/2111: by using evolutionary computational techniques, e.g. genetic algorithms - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06F 18/285: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system} - G06N 20/00: Machine learning - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06V 10/771: Feature selection, e.g. selecting representative features from a multi-dimensional feature space - G06V 10/776: Validation; Performance evaluation - G06V 10/87: {using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system} - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06V 40/161: {Detection; Localisation; Normalisation} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An algorithm for performing an image or video processing task is generated that may be used to combine a plurality of different independent solutions to the image or video processing task in an optimized manner. A plurality of base algorithms may be applied to a training set of images or video and a first generation of different combining algorithms may be applied to combine the respective solutions from each of the respective base algorithms into respective combined solutions. The respective combined solutions may be evaluated to generate respective fitness scores representing measures of how well the plurality of different combining algorithms each perform the image or video processing task. The algorithms may be iteratively updated to generate an optimized combining algorithm that may be applied to an input image or video. ## Claims ### Claim at position 1 1-23. (canceled) ### Claim 24 A method for combining solutions for an image or video processing task, the method comprising: independently applying a plurality of algorithms to a set of input images or video, each of the plurality of algorithms independently generating respective algorithm results to the image or video processing task, the respective algorithm results comprising a first algorithm result and a second algorithm result; combining the first algorithm result with the second algorithm result, subsequent to applying one or more of a plurality of modification operations to at least one of the first algorithm result and the second algorithm result, into a first combined solution; evaluating respective combined solutions, the respective combined solutions comprising the first combined solution, in order to determine a best combined solution for the respective combined solutions, the best combined solution meeting a predefined optimization criterion; and using the best combined solution in order to perform a background/foreground segmentation task for the set of input images or video. ### Claim 25 (depends on 24) The method of claim 24, wherein the independently applying of the plurality of algorithms comprises independently applying two or more of: (1) a Gaussian Mixture Model (GMM) algorithm, (2) a K-Nearest Neighbor (K-NN) algorithm, and (3) a Self-Balanced Sensitivity Segmenter (SuB SENSE) algorithm. ### Claim 26 (depends on 25) The method of claim 25, wherein the applying of the one or more of the plurality of modification operations comprises applying at least one of: (1) a median filter (MF) operation, (2) an erosion (ERO) operation, and (3) a dilation (DIL) operation. ### Claim 27 (depends on 26) The method of claim 26, wherein the combining of the first algorithm result with the second algorithm result comprises combining using one of: (1) a majority vote (MV) operation, (2) a logical OR (OR) operation, and (3) a logical AND (AND) operation. ### Claim 28 (depends on 27) The method of claim 27, further comprising: independently applying the plurality of algorithms to the set of input images or video, each of the plurality of algorithms independently generating first respective algorithm results to the image or video processing task; and combining the first respective algorithm results into a combined solution to the image or video processing task. ### Claim 29 (depends on 28) The method of claim 28, further comprising: applying one or more of the plurality of modification operations to the combined solution thereby generating a modified result; applying a first algorithm of the plurality of algorithms to the set of input images or video thereby generating a third algorithm result; applying one or more of the plurality of modification operations to the third algorithm result thereby generating a first modified result; and combining the modified result with the first modified result thereby generating a second combined result. ### Claim 30 (depends on 29) The method of claim 29, further comprising: applying a second algorithm of the plurality of algorithms to the set of input images or video thereby generating a fourth algorithm result, the second algorithm differing from the first algorithm; applying one or more of the plurality of modification operations to the fourth algorithm result thereby generating a second modified result; and combining the second combined result with the second modified result thereby generating a third combined result. ### Claim 31 (depends on 30) The method of claim 30, further comprising applying one or more of the plurality of modification operations to the first combined solution, thereby generating a third modified result. ### Claim 32 (depends on 31) The method of claim 31, further comprising combining the third modified result with the third combined result, the combining of the third modified result with the third combined result resulting in the best combined solution for the respective combined solution the best combined solution meeting the predefined optimization criterion. ### Claim 33 A non-transitory computer-readable storage apparatus comprising a storage medium having instructions stored thereon, the instructions for combining independent solutions for an image or video processing task, the instructions, when executed by a processor apparatus, being configured to: apply a plurality of algorithms to a set of input images or video, each of the plurality of algorithms independently generating respective algorithm results to the image or video processing task, the respective algorithm results comprising a first algorithm result and a second algorithm result; combine the first algorithm result with the second algorithm result, subsequent to application of one or more of a plurality of modification operations to at least one of the first algorithm result and the second algorithm result, into a first combined solution; evaluate respective combined solutions, the respective combined solutions comprising the first combined solution, in order to determine a best combined solution for the respective combined solutions, the best combined solution meeting a predefined optimization criterion; and use the best combined solution in order to perform a background/foreground segmentation task for the set of input images or video. ### Claim 34 (depends on 33) The non-transitory computer-readable storage apparatus of claim 33, wherein the application of the plurality of algorithms comprises an independent application of two or more of: (1) a Gaussian Mixture Model (GMM) algorithm, (2) a K-Nearest Neighbor (K-NN) algorithm, and (3) a Self-Balanced Sensitivity Segmenter (SuB SENSE) algorithm. ### Claim 35 (depends on 34) The non-transitory computer-readable storage apparatus of claim 34, wherein the application of the one or more of a plurality of modification operations comprises application of at least one of: (1) a median filter (MF) operation, (2) an erosion (ERO) operation, and (3) a dilation (DIL) operation. ### Claim 36 (depends on 35) The non-transitory computer-readable storage apparatus of claim 35, wherein the combination of the first algorithm result with the second algorithm result comprises combination using one of: (1) a majority vote (MV) operation, (2) a logical OR (OR) operation, and (3) a logical AND (AND) operation. ### Claim 37 (depends on 36) The non-transitory computer-readable storage apparatus of claim 36, wherein the instructions, when executed by the processor apparatus, being further configured to: independently apply the plurality of algorithms to the set of input images or video, each of the plurality of algorithms independently generating first respective algorithm results to the image or video processing task; and combine the first respective algorithm results into a combined solution to the image or video processing task. ### Claim 38 (depends on 37) The non-transitory computer-readable storage apparatus of claim 37, wherein the instructions, when executed by the processor apparatus, being further configured to: apply one or more of the plurality of modification operations to the combined solution thereby resulting in generation of a modified result; apply a first algorithm of the plurality of algorithms to the set of input images or video thereby resulting in generation of a third algorithm result; apply one or more of the plurality of modification operations to the third algorithm result thereby resulting in generation of a first modified result; and combine the modified result with the first modified result thereby resulting in generation of a second combined result. ### Claim 39 (depends on 38) The non-transitory computer-readable storage apparatus of claim 38, wherein the instructions, when executed by the processor apparatus, being further configured to: apply a second algorithm of the plurality of algorithms to the set of input images or video thereby resulting in generation of a fourth algorithm result, the second algorithm differing from the first algorithm; apply one or more of the plurality of modification operations to the fourth algorithm result thereby resulting in generation of a second modified result; and combine the second combined result with the second modified result thereby resulting in generation of a third combined result. ### Claim 40 (depends on 39) The non-transitory computer-readable storage apparatus of claim 39, wherein the instructions, when executed by the processor apparatus, being further configured to: apply one or more of the plurality of modification operations to the first combined solution, thereby resulting in generation of a third modified result. ### Claim 41 (depends on 40) The non-transitory computer-readable storage apparatus of claim 40, wherein the instructions, when executed by the processor apparatus, being further configured to: combine the third modified result with the third combined result, the combination of the third modified result with the third combined result resultant in the best combined solution for the respective combined solution, the best combined solution meeting the predefined optimization criterion. ### Claim 42 A computer system, comprising: a processor apparatus; and a non-transitory computer-readable storage apparatus comprising a storage medium having instructions stored thereon, the instructions for combining independent solutions for an image or video processing task, the instructions, when executed by the processor apparatus, being configured to: apply a plurality of algorithms to a set of input images or video, each of the plurality of algorithms independently generating respective algorithm results to the image or video processing task, the respective algorithm results comprising a first algorithm result and a second algorithm result; combine the first algorithm result with the second algorithm result, subsequent to application of one or more of a plurality of modification operations to at least one of the first algorithm result and the second algorithm result, into a first combined solution; evaluate respective combined solutions, the respective combined solutions comprising the first combined solution, in order to determine a best combined solution for the respective combined solutions, the best combined solution meeting a predefined optimization criterion; and use the best combined solution in order to perform a background/foreground segmentation task for the set of input images or video. ### Claim 43 (depends on 42) The computer system of claim 42, wherein: the application of the plurality of algorithms comprises an independent application of two or more of: (1) a Gaussian Mixture Model (GMM) algorithm, (2) a K-Nearest Neighbor (K-NN) algorithm, and (3) a Self-Balanced Sensitivity Segmenter (SuB SENSE) algorithm; the application of the one or more of a plurality of modification operations comprises application of at least one of: (1) a median filter (MF) operation, (2) an erosion (ERO) operation, and (3) a dilation (DIL) operation; and the combination of the first algorithm result with the second algorithm result comprises combination using one of: (1) a majority vote (MV) operation, (2) a logical OR (OR) operation, and (3) a logical AND (AND) operation.