# US 2021/0279528 A1 · Systems and methods for fine tuning image classification neural networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279528 A1 - Document ID: US20210279528A1 - Status: Published application - Kind: A1 - Application number: 16/807,825 - Filed: 2020-03-03 - Published: 2021-09-09 - Earliest claimed: 2020-03-03 - Assignees: Assa Abloy AB - Applicants: Assa Abloy AB - Inventors: Kapil Sachdeva; Sylvain Jacques Prevost - Web page: https://patentgrep.com/patent/US20210279528A1 ## CPC - G06V 10/7784: {based on feedback from supervisors} - G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade} - G06F 18/2155: {characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling} - G06F 18/2193: {based on specific statistical tests} - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/047: Probabilistic or stochastic networks - G06N 3/08: Learning methods - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/09: Supervised learning - G06V 10/993: {Evaluation of the quality of the acquired pattern} - G06V 40/172: {Classification, e.g. identification} - G07C 9/37: using biometric data, e.g. fingerprints, iris scans or voice recognition - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE ## Abstract An authentication engine, residing at one or more computing machines, receives, from a vision device comprising one or more cameras, a probe image. The authentication engine generates, using a trained facial classification neural engine, one or more first labels for a person depicted in the probe image and a probability for at least one of the one or more first labels. The authentication engine determines that the probability is within a predefined low accuracy range. The authentication engine generates, using a supporting engine, a second label for the person depicted in the probe image. The supporting engine operates independently of the trained facial classification neural engine. The authentication engine further trains the facial classification neural engine based on the second label. ## Claims ### Claim 1 A system comprising: processing circuitry; and a memory storing instructions which, when executed by the processing circuitry, cause the processing circuitry to perform operations comprising: receiving, from a vision device comprising one or more cameras, a probe image; generating, using a trained facial classification neural engine, one or more first labels for a person depicted in the probe image and a probability for at least one of the one or more first labels; determining that the probability is within a predefined low accuracy range; generating, using a supporting engine, a second label for the person depicted in the probe image, wherein the supporting engine operates independently of the trained facial classification neural engine; further training the facial classification neural engine based on the second label. ### Claim 2 (depends on 1) The system of claim 1, the operations further comprising: using the further trained facial classification neural engine to identify one or more persons in visual data from the vision device; and based on the identified one or more persons in the visual data, controlling access to a physical location or an electronic resource. ### Claim 3 (depends on 1) The system of claim 1, wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises: generating the second label based on an identity card or token provided by the person or based on a user identifier and password entered by the person. ### Claim 4 (depends on 1) The system of claim 1, wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises: generating the second label based on a combination of weak authentication factors, the weak authentication factors comprising one or more of: a height, a weight and a gait. ### Claim 5 (depends on 1) The system of claim 1, wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises: verifying, via at least one client computing device, a correct identification for the person depicted in the probe image. ### Claim 6 (depends on 5) The system of claim 5, wherein verifying the correct identification comprises: providing, for display at the at least one client computing device, the probe image and a plurality of possible identifications for the person; and receiving, from the at least one client device, a selection of one of the possible identifications as the correct identification. ### Claim 7 (depends on 5) The system of claim 5, wherein the at least one client computing device comprises an administrator client computing device and N employee client computing devices, wherein N is a positive integer greater than or equal to two, wherein verifying the correct identification comprises: providing the probe image to at least a portion of the N employee client computing devices; upon receiving, from at least M employee client computing devices, a consistent identification of the person: verifying that the consistent identification is the correct identification, wherein M is a positive integer between half of N and N; and upon failing to receive, from the at least M employee client computing devices, the consistent identification of the person: providing the probe image to the administrator client computing device for verifying the correct identification via the administrator client computing device. ### Claim 8 (depends on 7) The system of claim 7, wherein the employee client computing devices are selected based on a corporate department or an office geographic location of at least one of the plurality of possible identifications. ### Claim 9 (depends on 1) The system of claim 1, wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises: providing the probe image to a training dataset for a semi-supervised learning facial classification engine; training the semi-supervised learning facial classification engine using the training dataset; generating, using the semi-supervised learning facial classification engine, the second label for the person depicted in the probe image and a probability value for the second label; and adjusting the trained facial classification neural engine based on the trained semi-supervised learning facial classification engine. ### Claim 10 (depends on 9) The system of claim 9, wherein providing the probe image to the training dataset for the semi-supervised learning facial classification engine is in response to determining that a quality of the probe image exceeds a quality threshold. ### Claim 11 (depends on 10) The system of claim 10, wherein the quality of the probe image is computed using a quality measuring neural engine. ### Claim 12 (depends on 10) The system of claim 10, wherein the quality of the probe image comprises a blurriness of the probe image. ### Claim 13 (depends on 9) The system of claim 9, wherein generating, using the supporting engine, the second label for the person depicted in the probe image further comprises: determining that the probability value for the second label is below a probability threshold; and in response to the probability value for the second label being below the probability threshold: verifying, via at least one client computing device, a correct identification for the person depicted in the probe image. ### Claim 14 (depends on 1) The system of claim 1, wherein the probe image is one of a plurality of images that track the person, the plurality of images being received from the vision device, wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises: determining, using the trained facial classification neural engine, that at least a threshold number of the plurality of images have a specified identification with a probability within a predefined high accuracy range; and determining that the probe image has the specified identification based on the at least the threshold number of the plurality of images having the specified identification. ### Claim 15 (depends on 14) The system of claim 14, the operations further comprising identifying the plurality of images that track the person based on timestamps associated with the plurality of images and a physical position of the person within a space depicted in the plurality of images. ### Claim 16 A non-transitory machine-readable medium storing instructions which, when executed by processing circuitry of one or more computing machines, cause the processing circuity to perform operations comprising: receiving, from a vision device comprising one or more cameras, a probe image; generating, using a trained facial classification neural engine, one or more first labels for a person depicted in the probe image and a probability for at least one of the one or more first labels; determining that the probability is within a predefined low accuracy range; generating, using a supporting engine, a second label for the person depicted in the probe image, wherein the supporting engine operates independently of the trained facial classification neural engine; further training the facial classification neural engine based on the second label. ### Claim 17 (depends on 16) The machine-readable medium of claim 16, the operations further comprising: using the further trained facial classification neural engine to identify one or more persons in visual data from the vision device; and based on the identified one or more persons in the visual data, controlling access to a physical location or an electronic resource. ### Claim 18 (depends on 16) The machine-readable medium of claim 16, wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises: providing the probe image to a training dataset for a semi-supervised learning facial classification engine; training the semi-supervised learning facial classification engine using the training dataset; generating, using the semi-supervised learning facial classification engine, the second label for the person depicted in the probe image and a probability value for the second label; and adjusting the trained facial classification neural engine based on the trained semi-supervised learning facial classification engine. ### Claim 19 (depends on 16) The machine-readable medium of claim 16, wherein the probe image is one of a plurality of images that track the person, the plurality of images being received from the vision device, wherein generating, using the supporting engine, the second label for the person depicted in the probe image comprises: determining, using the trained facial classification neural engine, that at least a threshold number of the plurality of images have a specified identification with a probability within a predefined high accuracy range; and determining that the probe image has the specified identification based on the at least the threshold number of the plurality of images having the specified identification. ### Claim 20 A method comprising: receiving, from a vision device comprising one or more cameras, a probe image; generating, using a trained facial classification neural engine, one or more first labels for a person depicted in the probe image and a probability for at least one of the one or more first labels; determining that the probability is within a predefined low accuracy range; generating, using a supporting engine, a second label for the person depicted in the probe image, wherein the supporting engine operates independently of the trained facial classification neural engine; further training the facial classification neural engine based on the second label.