# US 12,217,501 B2 · Identification apparatus, object identification method, learning apparatus, learning method, and recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,501 B2 - Document ID: US12217501B2 - Status: Granted - Kind: B2 - Application number: 17/783,429 - Filed: 2019-12-24 - Published: 2022-12-08 - Granted: 2025-02-04 - Earliest claimed: 2019-12-24 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Hideaki Sato; Katsumi Kikuchi; Kyota Higa - Web page: https://patentgrep.com/patent/US12217501B2 ## CPC - G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking - G06V 20/17: taken from planes or by drones - G06T 7/00: Image analysis - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06T 2207/10032: Satellite or aerial image; Remote sensing - G06T 2207/20081: Training; Learning - G06T 2207/30232: Surveillance - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract In an object identification apparatus, a foreground extraction unit performs a foreground extraction with respect to input images, and generates a foreground extraction result. A state extraction unit extracts a state for each foreground based on the foreground extraction result. An identification model selection unit selects one or more identification models based on the extracted state for each foreground by using a selection model. An identification unit identifies a moving object included in the input images using the selected identification model. ## Claims ### Claim 1 An object identification apparatus comprising: a first memory storing instructions; and one or more first processors configured to execute the instructions to: perform foreground extraction with respect to input images, and generate a foreground extraction result; extract a state of a foreground of each input image based on the foreground extraction result, wherein the state includes a movement trajectory feature, an area variation feature, and an appearance feature of the foreground; select one or more identification models based on the extracted state by using a selection model, wherein the selection model receives the state and outputs a likelihood to select each identification model, and wherein the selection model is trained using correct answer data having a value indicating the identification model to be used to identify a foreground of each of training input images; and identify a moving object included in the input images by using the selected one or more identification models. ### Claim 2 (depends on 1) The object identification apparatus according to claim 1, wherein the first processor generates the foreground extraction result with respect to a predetermined number of frames of each of the input images; and the first processor extracts the state of the foreground of each input image using the foreground extraction result concerning the predetermined number of frames. ### Claim 3 (depends on 2) The object identification apparatus according to claim 2, wherein the first processor extracts an appearance feature of the foreground of each input image included in the predetermined frames, and outputs the state of the foreground of each input image. ### Claim 4 (depends on 3) The object identification apparatus according to claim 3, wherein the first processor extracts respective foreground areas corresponding to a same foreground from the predetermined number of frames, and outputs, as the appearance feature, a feature vector extracted from the predetermined number of frames. ### Claim 5 (depends on 2) The object identification apparatus according to claim 2, wherein the first processor extracts the movement trajectory feature of a same foreground through the predetermined number of frames, and outputs the state of the foreground of each input image. ### Claim 6 (depends on 5) The object identification apparatus according to claim 5, wherein the first processor extracts the movement trajectory feature including coordinates and a movement amount of the same foreground through the predetermined number of frames, and outputs the extracted movement trajectory feature as the state for the foreground of each input image. ### Claim 7 (depends on 1) The object identification apparatus according to claim 1, wherein the first processor extracts the area variation feature of a same foreground from each of the predetermined number of frames, and outputs the state for the foreground of each input image. ### Claim 8 (depends on 1) The object identification apparatus according to claim 1, wherein the selection model is trained to output the likelihood for selecting the identification model for the state for the foreground of each input image. ### Claim 9 (depends on 1) The object identification apparatus according to claim 1, wherein the first processor specifies one or more features which the selected identification model uses, from among features included in the state of the foreground of each input image, and identifies the moving object based on the one or more features. ### Claim 10 (depends on 1) A learning apparatus for use in conjunction with the object identification apparatus according to claim 1, the learning apparatus comprising: a second memory storing instructions; and one or more second processors configured to execute the instructions to: perform the foreground extraction with respect to the training input images, and generate a foreground extraction result; extract a state for the foreground of each training input image by using the foreground extraction result; and train the selection model for selecting the one or more identification models of an object based on the state for the foreground of each training input image and the correct answer data prepared in advance. ### Claim 11 (depends on 1) The object identification apparatus according to claim 1, wherein the movement trajectory feature includes coordinates of the foreground for each of a plurality of frames of each input image and a movement amount of the foreground among adjacent frames. ### Claim 12 (depends on 1) The object identification apparatus according to claim 1, wherein the area variation feature indicates a variation ratio for an area of the foreground among adjacent frames of each input image. ### Claim 13 An object identification method performed by a computer and comprising: by performing foreground extraction with respect to input images, and generating a foreground extraction result; extracting a state of a foreground of each input image based on the foreground extraction result, wherein the state includes a movement trajectory feature, an area variation feature, and an appearance feature of the foreground; selecting one or more identification models based on the extracted state by using a selection model, wherein the selection model receives the state and outputs a likelihood to select each identification model, and wherein the selection model is trained using correct answer data having a value indicating the identification model to be used to identify a foreground of each of training input images; and identifying a moving object included in the input images by using the selected one or more identification models. ### Claim 14 (depends on 13) A learning method for use in conjunction with the object identification method according to claim 13, the learning method performed by a different computer and comprising: performing the foreground extraction with respect to the training input images, and generate a foreground extraction result; extracting a state for the foreground of each training input image by using the foreground extraction result; and training the selection model for selecting the one or more identification models of an object based on the state for the foreground of each training input image and the correct answer data prepared in advance. ### Claim 15 A non-transitory computer-readable recording medium storing a program executable by a computer to perform an object identification process comprising: by performing foreground extraction with respect to input images, and generating a foreground extraction result; extracting a state of a foreground of each input image based on the foreground extraction result, wherein the state includes a movement trajectory feature, an area variation feature, and an appearance feature of the foreground; selecting one or more identification models based on the extracted state by using a selection model, wherein the selection model receives the state and outputs a likelihood to select each identification model, and wherein the selection model is trained using correct answer data having a value indicating the identification model to be used to identify a foreground of each of training input images; and identifying a moving object included in the input images by using the selected one or more identification models. ### Claim 16 (depends on 15) A non-transitory computer-readable recording medium storing a program executable by a different computer to perform a learning process in conjunction with the object identification process according to claim 15, the learning process comprising: performing the foreground extraction with respect to the training input images, and generate a foreground extraction result; extracting a state for the foreground of each training input image by using the foreground extraction result; and training the selection model for selecting the one or more identification models of an object based on the state for the foreground of each training input image and the correct answer data prepared in advance.