Method and device for displaying explanation of reference numeral in patent drawing image using artificial intelligence technology based machine learning

An integrating server for placing at least one technical term of at least one reference sign in at least one drawing around the at least one reference sign, comprising:

• a communication part for, on condition that i) a first state in which an OCR device has acquired at least one drawing image file corresponding to at least one drawing identifier related to at least one patent identifier, has recognized at least one reference sign in at least one drawing from the drawing image file, and has generated an information set on the reference sign in the drawing, including location information on the reference sign in the drawing, and that ii) a second state in which a reference sign and technical term recognizing device has parsed a specification text string including the reference sign and its corresponding technical term, has recognized the reference sign in a specification and its corresponding technical term included in the specification text string, and has generated each information set, for each patent identifier, on mapping between the reference sign in the specification and its corresponding technical term, have been performed, acquiring (1) the information set on the mapping between the reference sign in the specification and its corresponding technical term about the patent identifier and (2) the information set on the reference sign in the drawing about the drawing identifier;
• and
• a processor for performing processes of (I) searching for a specific free space, among entire free space, on the drawing image, where the technical term of the reference sign is to be displayed, by using (i) the location information on the reference sign in the drawing included in the information set on the reference sign in the drawing and (ii) term information on the reference sign mapped onto the reference sign in the specification included in the information set on the mapping between the reference sign in the specification and its corresponding technical term, and (II) generating an information set on placement including location information to be used for displaying the technical term of the reference sign on the drawing image by using the specific free space;
• wherein the reference sign in the drawing and the reference sign in the specification are text strings determined as identical to each other, and
• wherein the reference sign in the drawing is recognized in order to achieve the first state by performing steps of:
• (i1) sampling over collected image data including each image corresponding to each character;
• (i2) dividing the sampled image data into n folders wherein n is a natural number larger than 2;
• (i3) learning a learning model using a part of the sampled image data in n−1 folders as training data, and performing a cross validation process of acquiring at least one test error n times with a part of the sampled image data in the n folders excluding said n−1 folders, as test data, to thereby optimize at least one hyper parameter;
• and
• (i4) learning at least one parameter of the learning model with the optimized hyper parameter.