# US 2023/0410486 A1 · Information processing apparatus, information processing method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0410486 A1 - Document ID: US20230410486A1 - Status: Published application - Kind: A1 - Application number: 18/252,219 - Filed: 2021-11-04 - Published: 2023-12-21 - Earliest claimed: 2020-11-17 - Assignees: Sony Group Corporation - Applicants: Sony Group Corporation - Inventors: GUIFEN TIAN - Web page: https://patentgrep.com/patent/US20230410486A1 ## CPC - G06V 10/776: Validation; Performance evaluation - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - B60W 40/02: related to ambient conditions - G06T 7/00: Image analysis - G06V 10/778: Active pattern-learning, e.g. online learning of image or video features - G06V 10/82: using neural networks - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract The present technology relates to an information processing apparatus, an information processing method, and a program that enable efficient relearning of a recognition model. An information processing apparatus includes: a collection timing control unit configured to control a timing to collect a learning image candidate that is an image to be a candidate for a learning image to be used in relearning of a recognition model; and a learning image collection unit configured to select the learning image from among the learning image candidates that have been collected, on the basis of at least one of a feature of the learning image candidate or a similarity to the learning image that has been accumulated. The present technology can be applied to, for example, a system that controls automated driving. ## Claims ### Claim 1 An information processing apparatus comprising: a collection timing control unit configured to control a timing to collect a learning image candidate that is an image to be a candidate for a learning image to be used in relearning of a recognition model; and a learning image collection unit configured to select the learning image from among the learning image candidates that have been collected, on a basis of at least one of a feature of the learning image candidate or a similarity to the learning image that has been accumulated. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the recognition model is used to recognize a predetermined recognition target around a vehicle, and the learning image collection unit selects the learning image from among the learning image candidates including an image obtained by capturing an image of surroundings of the vehicle by an image sensor installed in the vehicle. ### Claim 3 (depends on 2) The information processing apparatus according to claim 2, wherein the collection timing control unit controls a timing to collect the learning image candidate on a basis of at least one of a place or an environment in which the vehicle is traveling. ### Claim 4 (depends on 3) The information processing apparatus according to claim 3, wherein the collection timing control unit performs control to collect the learning image candidate in at least one of a place where the learning image candidate has not been collected, a vicinity of a newly installed construction site, or a vicinity of a place where an accident of a vehicle including a system similar to a vehicle control system provided in the vehicle has occurred. ### Claim 5 (depends on 2) The information processing apparatus according to claim 2, wherein the collection timing control unit performs control to collect the learning image candidate when reliability of a recognition result by the recognition model has decreased while the vehicle is traveling. ### Claim 6 (depends on 2) The information processing apparatus according to claim 2, wherein the collection timing control unit performs control to collect the learning image candidate when at least one of a change of the image sensor installed in the vehicle or a change of an installation position of the image sensor occurs. ### Claim 7 (depends on 2) The information processing apparatus according to claim 2, wherein when the vehicle receives an image from outside, the collection timing control unit performs control to collect the received image as the learning image candidate. ### Claim 8 (depends on 1) The information processing apparatus according to claim 1, wherein the learning image collection unit selects the learning image from among the learning image candidates including at least one of a backlight region, a shadow, a reflector, a region in which a similar pattern is repeated, a construction site, an accident site, rain, snow, smog, or haze. ### Claim 9 (depends on 1) The information processing apparatus according to claim 1, further comprising: a verification image collection unit configured to select the verification image from among verification image candidates that are images to be a candidate for the verification image to be used for verification of the recognition model, on a basis of similarity to the verification image that has been accumulated. ### Claim 10 (depends on 9) The information processing apparatus according to claim 9, further comprising: a learning unit configured to relearn the recognition model by using the learning image that has been collected; and a recognition model update control unit configured to control update of the recognition model on a basis of a result of comparison between: recognition accuracy of a first recognition for the verification image, the first recognition model being the recognition model before relearning; and recognition accuracy of a second recognition model for the verification image, the second recognition model being the recognition model obtained by relearning. ### Claim 11 (depends on 10) The information processing apparatus according to claim 10, wherein on a basis of reliability of a recognition result of the first recognition model for the verification image, the verification image collection unit classifies the verification image into a high-reliability verification image having high reliability or a low-reliability verification image having low reliability, and the recognition model update control unit updates the first recognition model to the second recognition model in a case where recognition accuracy of the second recognition model for the high-reliability verification image has not decreased as compared with recognition accuracy of the first recognition model for the high-reliability verification image, and recognition accuracy of the second recognition model for the low-reliability verification image has been improved as compared with recognition accuracy of the first recognition model for the low-reliability verification image. ### Claim 12 (depends on 9) The information processing apparatus according to claim 9, wherein the recognition model recognizes a predetermined recognition target for every pixel of an input image and estimates reliability of a recognition result, and the verification image collection unit extracts a region to be used for the verification image in the verification image candidate, on a basis of a result of comparison between: reliability of a recognition result for every pixel of the verification image candidate by the recognition model; and a threshold value that is dynamically set. ### Claim 13 (depends on 12) The information processing apparatus according to claim 12, further comprising: a threshold value setting unit configured to learn the threshold value by using a loss function obtained by adding a loss component of the threshold value to a loss function to be used for learning the recognition model. ### Claim 14 (depends on 12) The information processing apparatus according to claim 12, further comprising: a threshold value setting unit configured to set the threshold value, on a basis of a recognition result for an input image by the recognition model and a recognition result for the input image by software for a benchmark test for recognizing a recognition target same as a recognition target of the recognition model. ### Claim 15 (depends on 12) The information processing apparatus according to claim 12, further comprising: a recognition model learning unit configured to relearn the recognition model by using a loss function including the reliability. ### Claim 16 (depends on 1) The information processing apparatus according to claim 1, further comprising: a recognition unit configured to recognize a predetermined recognition target by using the recognition model and estimate reliability of a recognition result. ### Claim 17 (depends on 16) The information processing apparatus according to claim 16, wherein the recognition unit estimates the reliability by taking statistics with a recognition result by another recognition model. ### Claim 18 (depends on 1) The information processing apparatus according to claim 1, further comprising: a learning unit configured to relearn the recognition model by using the learning image that has been collected. ### Claim 19 An information processing method comprising, by an information processing apparatus: controlling a timing to collect a learning image candidate that is an image to be a candidate for a learning image to be used in relearning of a recognition model; and selecting the learning image from among the learning image candidates that have been collected, on a basis of at least one of a feature of the learning image candidate or a similarity to the learning image that has been accumulated. ### Claim 20 A program for causing a computer to execute processing comprising: controlling a timing to collect a learning image candidate that is an image to be a candidate for a learning image to be used in relearning of a recognition model; and selecting the learning image from among the learning image candidates that have been collected, on a basis of at least one of a feature of the learning image candidate or a similarity to the learning image that has been accumulated.