# US 12,002,182 B2 · Methods and apparatuses for processing image, methods and apparatuses for training image recognition network and methods and apparatuses for recognizing image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,002,182 B2 - Document ID: US12002182B2 - Status: Granted - Kind: B2 - Application number: 17/490,754 - Filed: 2021-09-30 - Published: 2022-05-05 - Granted: 2024-06-04 - Earliest claimed: 2020-10-29 - Assignees: Beijing Zhongxiangying Technology Co., Ltd.; Boe Technology Group Co., Ltd. - Applicants: Beijing Zhongxiangying Technology Co., Ltd.; BOE Technology Group Co., Ltd. - Inventors: Xiaohui Zhao; Wangqiang He; Libo Zhang; Dong Chai; Hong Wang - Web page: https://patentgrep.com/patent/US12002182B2 ## CPC - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - G06T 3/147: using affine transformations - G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 7/0004: {Industrial image inspection} - G06T 11/60: Creating or editing images; Combining images with text - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present disclosure relates to methods and apparatuses for processing an image, training an image recognition network and recognizing an image. The method of processing an image includes: obtaining a plurality of original images from an original image set, where at least one of the plurality of original images includes an annotation area; obtaining at least one first image by splicing the plurality of original images; for each of the at least one first image, adjusting a shape and/or size of the first image based on the plurality of original images to form a second image; obtaining respective positions of the at least one annotation area in the second image by converting respective positions of the at least one annotation area in the plurality of original images. ## Claims ### Claim 1 A method of processing an image, comprising: obtaining a plurality of original images from an original image set, wherein at least one of the plurality of original images comprises an annotation area; obtaining at least one first image by splicing the plurality of original images; for each of the at least one first image, adjusting a shape and/or size of the first image based on the plurality of original images to form a second image; obtaining respective positions of the at least one annotation area in the second image by converting respective positions of the at least one annotation area in the plurality of original images; wherein adjusting the shape and/or size of the first image based on the plurality of original images to form the second image comprises: determining a scaling ratio of an annotation area in the plurality of original images and an annotation area in the second image according to a parameter of an image acquisition device corresponding to the plurality of original images and a parameter of an image acquisition device corresponding to the second image; scaling the first image based on the scaling ratio to form a scaled first image; and adjusting a shape and/or size of the scaled first image to be consistent with those of the plurality of original images to form the second image; wherein obtaining respective positions of the at least one annotation area in the second image by converting respective positions of the at least one annotation area in the plurality of original images comprises: for each of the at least one annotation area, obtaining coordinates of the annotation area in the first image by converting coordinates of the annotation area in the plurality of original images; obtaining coordinates of the annotation area in the scaled first image by converting the coordinates of the annotation area in the first image; and obtaining coordinates of the annotation area in the second image by converting the coordinates of the annotation area in the scaled first image; wherein obtaining the coordinates of the annotation area in the first image by converting the coordinates of the annotation area in the plurality of original images comprises: in response to spaces corresponding to the plurality of original images being unoverlapped, obtaining coordinates of the annotation area in a first coordinate system of the first image by converting the coordinates of the annotation area in the plurality of original images, wherein, the first coordinate system takes a center of the first image as an origin; in response to spaces corresponding to two or more of the plurality of original images being overlapped, obtaining coordinates of the annotation area in a second coordinate system of the first image by converting the coordinates of the annotation area in the plurality of original images, wherein, the second coordinate system takes a center of duplicate area as an origin. ### Claim 2 (depends on 1) The method according to claim 1, wherein the original image set comprises a plurality of process subsets, each of the process subsets comprises a plurality of type nodes, each of the type nodes comprises a multitude of original images, and annotation areas of original images in each of the type nodes are annotated with labels corresponding to the type node; and obtaining the plurality of original images from the original image set comprises: obtaining the plurality of original images from a same type node in a same process subset, or obtaining the plurality of original images from different type nodes in a same process subset, or obtaining the plurality of original images from same type of type nodes in different process subsets. ### Claim 3 (depends on 2) The method according to claim 2, wherein original images in one type node of each of the process subsets are unannotated images. ### Claim 4 (depends on 1) The method according to claim 1, wherein obtaining the plurality of original images from the original image set comprises: obtaining N2 original images, wherein N is an integer greater than or equal to 2. ### Claim 5 (depends on 4) The method according to claim 4, wherein obtaining the at least one first image by splicing the plurality of original images comprises: in response to spaces corresponding to the plurality of original images being unoverlapped, forming the at least one first image by arranging the plurality of original images into N rows and N columns and splicing; in response to spaces corresponding to two or more of the plurality of original images being overlapped, forming the at least one first image by superposing duplicate areas of the two or more of the plurality of original images. ### Claim 6 (depends on 1) A method of training an image recognition network, comprising: training the image recognition network using an image training set, wherein images in the image training set are processed using the method of processing an image according to claim 1. ### Claim 7 (depends on 6) A method of recognizing an image, comprising: recognizing an image by using an image recognition network, wherein the image recognition network is trained using the training method according to claim 6. ### Claim 8 (depends on 1) The method according to claim 1, wherein adjusting the shape and/or size of the scaled first image to be consistent with those of the plurality of original images to form the second image comprises: padding the scaled first image and/or cropping the scaled first image to form the second image in a shape and size consistent with the plurality of original images. ### Claim 9 (depends on 1) The method according to claim 1, wherein obtaining the coordinates of the annotation area in the second image by converting the coordinates of the annotation area in the scaled first image comprises: obtaining coordinates of the annotation area in a third coordinate system by converting coordinates of the annotation area in the first coordinate system scaled or the second coordinate system scaled, wherein the third coordinate system takes an upper left corner of the second image as an origin. ### Claim 10 A device comprising: a memory, and a processor, wherein the memory is configured to store computer instructions executable on the processor, and the processor is configured to execute the computer instructions to implement: obtaining a plurality of original images from an original image set, wherein at least one of the plurality of original images comprises an annotation area; obtaining at least one first image by splicing the plurality of original images; for each of the at least one first image, adjusting a shape and/or size of the first image based on the plurality of original images to form a second image; obtaining respective positions of the at least one annotation area in the second image by converting respective positions of the at least one annotation area in the plurality of original images; wherein adjusting the shape and/or size of the first image based on the plurality of original images to form the second image comprises: determining a scaling ratio of an annotation area in the plurality of original images and an annotation area in the second image according to a parameter of an image acquisition device corresponding to the plurality of original images and a parameter of an image acquisition device corresponding to the second image; scaling the first image based on the scaling ratio to form a scaled first image; and adjusting a shape and/or size of the scaled first image to be consistent with those of the plurality of original images to form the second image; wherein obtaining respective positions of the at least one annotation area in the second image by converting respective positions of the at least one annotation area in the plurality of original images comprises: for each of the at least one annotation area, obtaining coordinates of the annotation area in the first image by converting coordinates of the annotation area in the plurality of original images; obtaining coordinates of the annotation area in the scaled first image by converting the coordinates of the annotation area in the first image; and obtaining coordinates of the annotation area in the second image by converting the coordinates of the annotation area in the scaled first image; wherein obtaining the coordinates of the annotation area in the first image by converting the coordinates of the annotation area in the plurality of original images comprises: in response to spaces corresponding to the plurality of original images being unoverlapped, obtaining coordinates of the annotation area in a first coordinate system of the first image by converting the coordinates of the annotation area in the plurality of original images, wherein, the first coordinate system takes a center of the first image as an origin; in response to spaces corresponding to two or more of the plurality of original images being overlapped, obtaining coordinates of the annotation area in a second coordinate system of the first image by converting the coordinates of the annotation area in the plurality of original images, wherein, the second coordinate system takes a center of duplicate area as an origin. ### Claim 11 (depends on 10) The device according to claim 10, wherein the original image set comprises a plurality of process subsets, each of the process subsets comprises a plurality of type nodes, each of the type nodes comprises a multitude of original images, and annotation areas of original images in each of the type nodes are annotated with labels corresponding to the type node; and obtaining the plurality of original images from the original image set comprises: obtaining the plurality of original images from a same type node in a same process subset, or obtaining the plurality of original images from different type nodes in a same process subset, or obtaining the plurality of original images from same type of type nodes in different process subsets. ### Claim 12 (depends on 10) The device according to claim 10, wherein obtaining the plurality of original images from the original image set comprises: obtaining N2 original images, wherein N is an integer greater than or equal to 2. ### Claim 13 (depends on 12) The device according to claim 12, wherein obtaining the at least one first image by splicing the plurality of original images comprises: in response to spaces corresponding to the plurality of original images being unoverlapped, forming the at least one first image by arranging the plurality of original images into N rows and N columns and splicing; in response to spaces corresponding to two or more of the plurality of original images being overlapped, forming the at least one first image by superposing duplicate areas of the two or more of the plurality of original images. ### Claim 14 (depends on 10) The device according to claim 10, wherein adjusting the shape and/or size of the scaled first image to be consistent with those of the plurality of original images to form the second image comprises: padding the scaled first image and/or cropping the scaled first image to form the second image in a shape and size consistent with the plurality of original images. ### Claim 15 A non-transitory computer readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor to implement: obtaining a plurality of original images from an original image set, wherein at least one of the plurality of original images comprises an annotation area; obtaining at least one first image by splicing the plurality of original images; for each of the at least one first image, adjusting a shape and/or size of the first image based on the plurality of original images to form a second image; obtaining respective positions of the at least one annotation area in the second image by converting respective positions of the at least one annotation area in the plurality of original images; wherein adjusting the shape and/or size of the first image based on the plurality of original images to form the second image comprises: determining a scaling ratio of an annotation area in the plurality of original images and an annotation area in the second image according to a parameter of an image acquisition device corresponding to the plurality of original images and a parameter of an image acquisition device corresponding to the second image; scaling the first image based on the scaling ratio to form a scaled first image; and adjusting a shape and/or size of the scaled first image to be consistent with those of the plurality of original images to form the second image; wherein obtaining respective positions of the at least one annotation area in the second image by converting respective positions of the at least one annotation area in the plurality of original images comprises: for each of the at least one annotation area, obtaining coordinates of the annotation area in the first image by converting coordinates of the annotation area in the plurality of original images; obtaining coordinates of the annotation area in the scaled first image by converting the coordinates of the annotation area in the first image; and obtaining coordinates of the annotation area in the second image by converting the coordinates of the annotation area in the scaled first image; wherein obtaining the coordinates of the annotation area in the first image by converting the coordinates of the annotation area in the plurality of original images comprises: in response to spaces corresponding to the plurality of original images being unoverlapped, obtaining coordinates of the annotation area in a first coordinate system of the first image by converting the coordinates of the annotation area in the plurality of original images, wherein, the first coordinate system takes a center of the first image as an origin; in response to spaces corresponding to two or more of the plurality of original images being overlapped, obtaining coordinates of the annotation area in a second coordinate system of the first image by converting the coordinates of the annotation area in the plurality of original images, wherein, the second coordinate system takes a center of duplicate area as an origin.