# US 2021/0241443 A1 · Information processing system, image capturing apparatus, information processing apparatus, control methods therefor, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241443 A1 - Document ID: US20210241443A1 - Status: Published application - Kind: A1 - Application number: 17/162,943 - Filed: 2021-01-29 - Published: 2021-08-05 - Earliest claimed: 2020-02-03 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Ryo Takahashi - Web page: https://patentgrep.com/patent/US20210241443A1 ## CPC - G06V 10/764: using classification, e.g. of video objects - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G06V 10/82: using neural networks - G06V 20/30: in albums, collections or shared content, e.g. social network photos or video - G06V 40/174: {Facial expression recognition} - G06V 40/175: {Static expression} - H04N 23/60: Control of cameras or camera modules - H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image} - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes - G06T 2207/20081: Training; Learning - G06T 2207/30168: Image quality inspection - G06T 2207/30201: Face - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An information processing system comprises an image capturing apparatus and an information processing apparatus, the image capturing apparatus includes an image capturing device; a first evaluation unit configured to perform first evaluation on a captured image; and a first transmission unit configured to transmit the captured image to the information processing apparatus, and the information processing apparatus includes a reception unit configured to receive the captured image; a second evaluation unit configured to perform second evaluation on the captured image; and a second transmission unit configured to transmit an evaluation result to the image capturing apparatus, the image capturing apparatus further including a sorting unit configured to receive the evaluation result of the second evaluation, and sort the captured image using an evaluation results of the first evaluation and the second evaluation. ## Claims ### Claim 1 An information processing system comprising an image capturing apparatus and an information processing apparatus capable of communicating with each other, wherein the image capturing apparatus includes: an image capturing device configured to capture an object image; and at least one processor or circuit configured to function as: a first evaluation unit configured to perform first evaluation on a captured image captured by the image capturing device; and a first transmission unit configured to transmit the captured image to the information processing apparatus, wherein the information processing apparatus includes: at least one processor or circuit configured to function as: a reception unit configured to receive the captured image transmitted by the first transmission unit; a second evaluation unit configured to perform second evaluation on the captured image; and a second transmission unit configured to transmit an evaluation result of the second evaluation to the image capturing apparatus, and wherein the at least one processor or circuit of the image capturing apparatus is configured to further function as a sorting unit configured to receive the evaluation result of the second evaluation, and sort the captured image using an evaluation result of the first evaluation and the evaluation result of the second evaluation. ### Claim 2 (depends on 1) The information processing system according to claim 1, wherein the sorting unit decides whether to keep or discard the captured image. ### Claim 3 (depends on 1) The information processing system according to claim 1, wherein the evaluation performed by the first evaluation unit is evaluation relating to at least quality of focus. ### Claim 4 (depends on 1) The information processing system according to claim 1, wherein the evaluation performed by the second evaluation unit is evaluation relating to at least quality of composition or facial expression. ### Claim 5 (depends on 1) The information processing system according to claim 1, wherein a size of an image used in the evaluation by the first evaluation unit is larger than a size of an image used in the evaluation by the second evaluation unit. ### Claim 6 (depends on 1) The information processing system according to claim 1, wherein the at least one processor or circuit of the image capturing apparatus is configured to further function as a selection unit configured to select a viewpoint from which a user places emphasis in the evaluation of the captured image, and wherein the sorting unit sorts the captured image according to a content of the selection by the selection unit. ### Claim 7 (depends on 6) The information processing system according to claim 6, wherein, in a case where the selection unit performs a selection according to which the user places emphasis on composition or facial expression, the sorting unit sorts the captured image with emphasis placed on the evaluation result of the second evaluation. ### Claim 8 (depends on 6) The information processing system according to claim 6, wherein, in a case where the selection unit performs a selection according to which the user places emphasis on focus, the sorting unit sorts the captured image with emphasis placed on the evaluation result of the first evaluation. ### Claim 9 (depends on 2) The information processing system according to claim 2, wherein the sorting unit sorts the captured image whose evaluation result of the first evaluation and evaluation result of the second evaluation are both greater than or equal to a predetermined value as an image to be kept. ### Claim 10 (depends on 2) the information processing system according to claim 2, wherein the sorting unit adds a transmission scheduled tag to the captured image sorted as an image to be kept. ### Claim 11 (depends on 1) the information processing system according to claim 1, wherein the evaluation by the second evaluation unit is performed using a learning model specialized in at least determination of facial expression. ### Claim 12 (depends on 1) the information processing system according to claim 1, wherein the first transmission unit transmits the captured image whose evaluation result of the first evaluation unit is greater than or equal to a predetermined threshold value to the information processing apparatus. ### Claim 13 An image capturing apparatus capable of communicating with an information processing apparatus, comprising: an image capturing device configured to capture an object image; and at least one processor or circuit configured to function as: an evaluation unit configured to perform first evaluation on a captured image captured by the image capturing device; a transmission unit configured to transmit the captured image to the information processing apparatus; a reception unit configured to receive an evaluation result of second evaluation performed on the captured image by the information processing apparatus; and a sorting unit configured to sort the captured image, using an evaluation result of the first evaluation and the received evaluation result of the second evaluation. ### Claim 14 (depends on 13) The image capturing apparatus according to claim 13, wherein the sorting unit decides whether to keep or discard the captured image. ### Claim 15 (depends on 13) The image capturing apparatus according to claim 13, wherein the first evaluation is evaluation relating to at least quality of focus. ### Claim 16 (depends on 13) The image capturing apparatus according to claim 13, wherein the second evaluation is evaluation relating to at least quality of composition or facial expression. ### Claim 17 (depends on 13) The image capturing apparatus according to claim 13, wherein a size of an image used in the first evaluation is larger than a size of an image used in the second evaluation. ### Claim 18 (depends on 13) The image capturing apparatus according to claim 13, wherein the at least one processor or circuit is configured to further function as a selection unit configured to select a viewpoint from which a user places emphasis in the evaluation of the captured image, and wherein the sorting unit sorts the captured image according to a content of the selection by the selection unit. ### Claim 19 (depends on 18) The image capturing apparatus according to claim 18, wherein, in a case where the selection unit performs a selection according to which the user places emphasis on composition or facial expression, the sorting unit sorts the captured image with emphasis placed on the evaluation result of the second evaluation. ### Claim 20 (depends on 18) The image capturing apparatus according to claim 18, wherein, in a case where the selection unit performs a selection according to which the user places emphasis on focus, the sorting unit sorts the captured image with emphasis placed on the evaluation result of the first evaluation. ### Claim 21 (depends on 14) The image capturing apparatus according to claim 14, wherein the sorting unit sorts the captured image whose evaluation result of the first evaluation and evaluation result of the second evaluation are both greater than or equal to a predetermined value as an image to be kept. ### Claim 22 (depends on 14) The image capturing apparatus according to claim 14, wherein the sorting unit adds a transmission scheduled tag to the captured image sorted as an image to be kept. ### Claim 23 (depends on 13) The image capturing apparatus according to claim 13, wherein the transmission unit transmits the captured image whose evaluation result of the first evaluation unit is greater than or equal to a predetermined threshold value to the information processing apparatus. ### Claim 24 An information processing apparatus capable of communicating with an image capturing apparatus, comprising: at least one processor or circuit configured to function as: a reception unit configured to receive a captured image captured by the image capturing apparatus and transmitted to the information processing apparatus; a second evaluation unit configured to perform second evaluation that differs from first evaluation by the image capturing apparatus on the captured image; and a transmission unit configured to transmit an evaluation result of the second evaluation to the image capturing apparatus. ### Claim 25 (depends on 24) The information processing apparatus according to claim 24, wherein the first evaluation is evaluation relating to at least quality of focus, and the second evaluation is evaluation relating to at least quality of composition or facial expression. ### Claim 26 (depends on 24) The information processing apparatus according to claim 24, wherein a size of an image used in the first evaluation is larger than a size of an image used in the second evaluation. ### Claim 27 (depends on 24) The information processing apparatus according to claim 24, wherein the second evaluation is performed using a learning model specialized in at least determination of facial expression. ### Claim 28 A method of controlling an image capturing apparatus including an image capturing device for capturing an object image and capable of communicating with an information processing apparatus, the method comprising: performing first evaluation on a captured image captured by the image capturing device; transmitting the captured image to the information processing apparatus; receiving an evaluation result of second evaluation performed on the captured image by the information processing apparatus; and sorting the captured image, using an evaluation result of the first evaluation and the received evaluation result of the second evaluation. ### Claim 29 A method of controlling an information processing apparatus capable of communicating with an image capturing apparatus, the method comprising: receiving a captured image captured by the image capturing apparatus and transmitted to the information processing apparatus: performing second evaluation that differs from first evaluation by the image capturing apparatus on the captured image; and transmitting an evaluation result of the second evaluation to the image capturing apparatus. ### Claim 30 A non-transitory computer readable storage medium storing a program causing a computer to execute a method of controlling an image capturing apparatus including an image capturing device for capturing an object image and capable of communicating with an information processing apparatus, the method comprising: performing first evaluation on a captured image captured by the image capturing device; transmitting the captured image to the information processing apparatus; receiving an evaluation result of second evaluation performed on the captured image by the information processing apparatus; and sorting the captured image, using an evaluation result of the first evaluation and the received evaluation result of the second evaluation. ### Claim 31 A non-transitory computer readable storage medium storing a program causing a computer to execute a method of controlling an information processing apparatus capable of communicating with an image capturing apparatus, the method comprising: receiving a captured image captured by the image capturing apparatus and transmitted to the information processing apparatus; performing second evaluation that differs from first evaluation by the image capturing apparatus on the captured image; and transmitting an evaluation result of the second evaluation to the image capturing apparatus.