Neural network-based key point training apparatus and method

A neural network-based key point training apparatus comprising:

• a key point model trained to extract key points from an input image;
• and
• an image reconstruction model trained to reconstruct the input image with the key points output by the key point model as the input,
• wherein the key point model comprises:
• an extraction module trained to extract a feature tensor from the input image;
• and
• a prediction module trained to predict a key point image based on the feature tensor extracted by the extraction module;
• and
• the image reconstruction model is trained to reconstruct the input image based on the key point image output by the key point model,
• wherein the key point image is an image indicating whether or not each pixel corresponds to the key point in an image space corresponding to the input image as a probability value,
• wherein an objective function Lprediction of the key point model is expressed through the following equation:
• Lprediction=−Σ{ptarget*xi,yi)log(p(xi,yi))+(1−ptarget(xi,yi))log(1−p(xi,yi))} (Equation)
• where, p(xi,yi):
• probability value of whether or not pixel p(xi, yi) is key point;
• p(xi,yi)=probability distribution (P(F(xi,yi);
• δ));
• P:
• neural network constituting key point model;
• F(xi,yi):
• feature tensor of pixel (xi, yi);
• δ:
• parameter of key point model;
• and
• ptarget(xi,yi):
• value of whether or not pixel (xi,yi) of the input image is labelled key point,
• wherein optimized parameters of the key point model and the image reconstruction model are calculated through the following equation:
• δ*,η*=argminδ,η(αLprediction+βLreconstruction) (Equation)
• where, δ*:
• optimized parameter of key point model;
• η*:
• optimized parameter of image reconstruction model;
• α:
• weight of objective function of key point model;
• β:
• weight of objective function of image reconstruction model;
• Lreconstruction:
• objective function of the image reconstruction model, Lreconstruction=∥I−H(P(I;δ);η)∥;
• I:
• input image;
• H:
• neural network constituting image reconstruction model;
• and
• η:
• parameters of image reconstruction model.