# US 12,183,083 B2 · Information processing apparatus, information processing method, and information processing program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,083 B2 - Document ID: US12183083B2 - Status: Granted - Kind: B2 - Application number: 18/198,022 - Filed: 2023-05-16 - Published: 2023-09-14 - Granted: 2024-12-31 - Earliest claimed: 2012-09-28 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Ryoma Oami - Web page: https://patentgrep.com/patent/US12183083B2 ## CPC - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06F 16/7335: {Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate (end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728)} - G06F 16/78: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually - H04N 5/91: Television signal processing therefor - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract To efficiency search for an object associated with a sensed event, an information processing apparatus includes a sensor that analyzes a captured video and senses whether a predetermined event has occurred, a determining unit that determines a type of an object to be used as query information based on a type of the event in response to sensing of the event occurrence, and a generator that detects the object of the determined type from the video and generates the query information based on the detected object. ## Claims ### Claim 1 An information processing system comprising: one or more memories storing: time difference information designating a time period of video data at least one before and after luggage abandonment has occurred as a predetermined event; and program instructions; and one or more processors configured to execute the program instructions to, each of a plurality of times starting with a first time: analyze a captured video and sense occurrence of the predetermined event; detect an object relating to the predetermined event from the captured video, using a capturing time which is calculated based on a time of the occurrence of the predetermined event and the time difference information, wherein the time difference information that is used the first time the object is detected is predetermined time information, and the time different information that is used each time the object is detect other than the first time is the time difference information as has been updated a preceding time; track the detected object; extract a feature of the detected object as query information; search the captured video using the query information to identify images in the captured video that include the extracted feature; output the identified images; learn, from a result of detection and tracking of the object, a time difference between the time of the occurrence of the predetermined event and a time when the feature is extracted; and update the time difference information using the learned time difference, such that the time different as has been updated is used in detecting the object a next time. ### Claim 2 (depends on 1) The information processing system according to claim 1, wherein the one or more memories store the time difference information that designates a plurality of specified time differences for the predetermined event, the plurality of specified time differences being longer than a time difference between the time of the occurrence of the predetermined event and a time when the luggage abandonment is detected; and the one or more processors extract the feature in a plurality of frames at capturing times calculated using the specified time differences, respectively. ### Claim 3 (depends on 1) The information processing system according to claim 1, wherein the one or more processors determine that the detected object is the luggage that was abandoned if the detected object is at rest for a predetermined time. ### Claim 4 (depends on 1) The information processing system according to claim 1, wherein the one or more processors determine that the detected object is the person who abandoned the luggage, as a person nearest to the abandoned luggage at a specified time difference after the luggage abandonment that is longer than a time difference between the time of the occurrence of the predetermined event and a time when the luggage abandonment is detected. ### Claim 5 (depends on 1) The information processing system according to claim 1, wherein the one or more processors learn the time difference using a reliability of the feature. ### Claim 6 A method performed by a computer and comprising: storing time difference information designating a time period of video data at least one before and after luggage abandonment has occurred as a predetermined event; each of a plurality of times starting with a first time: analyzing a captured video and sense occurrence of the predetermined event; detecting an object relating to the predetermined event from the captured video, using a capturing time which is calculated based on a time of the occurrence of the predetermined event and the time difference information, wherein the time difference information that is used the first time the object is detected is predetermined time information, and the time different information that is used each time the object is detect other than the first time is the time difference information as has been updated a preceding time; tracking the detected object; extracting a feature of the detected object as query information; searching the captured video using the query information to identify images in the captured video that include the extracted feature; outputting the identified images; learning, from a result of detection and tracking of the object, a time difference between the time of the occurrence of the predetermined event and a time when the feature is extracted; and updating the time difference information using the learned time difference, such that the time different as has been updated is used in detecting the object a next time. ### Claim 7 (depends on 6) The method according to claim 6, further comprising storing the time difference information that designates a plurality of specified time differences for the predetermined event, the plurality of specified time differences being longer than a time difference between the time of the occurrence of the predetermined event and a time when the luggage abandonment is detected; and extracting the feature in a plurality of frames at capturing times calculated using the specified time differences, respectively. ### Claim 8 (depends on 6) The method according to claim 6, further comprising determining that the detected object is the luggage that was abandoned if the detected object is at rest for a predetermined time. ### Claim 9 (depends on 6) The method according to claim 6, further comprising determining that the detected object is the person who abandoned the luggage, as a person nearest to the abandoned luggage at a specified time difference after the luggage abandonment that is longer than a time difference between the time of the occurrence of the predetermined event and a time when the luggage abandonment is detected. ### Claim 10 (depends on 6) The method according to claim 6, further comprising learning the time difference using a reliability of the feature. ### Claim 11 A non-transitory computer readable medium storing a program causing a computer to execute a process comprising: storing time difference information designating a time period of video data at least one before and after luggage abandonment has occurred as a predetermined event; each of a plurality of times starting with a first time: analyzing a captured video and sense occurrence of the predetermined event; detecting an object relating to the predetermined event from the captured video, using a capturing time which is calculated based on a time of the occurrence of the predetermined event and the time difference information, wherein the time difference information that is used the first time the object is detected is predetermined time information, and the time different information that is used each time the object is detect other than the first time is the time difference information as has been updated a preceding time; tracking the detected object; extracting a feature of the detected object as query information; searching the captured video using the query information to identify images in the captured video that include the extracted feature; outputting the identified images; learning, from a result of detection and tracking of the object, a time difference between the time of the occurrence of the predetermined event and a time when the feature is extracted; and updating the time difference information using the learned time difference, such that the time different as has been updated is used in detecting the object a next time. ### Claim 12 (depends on 11) The non-transitory computer readable medium according to claim 11, wherein the process further comprises: storing the time difference information that designates a plurality of specified time differences for the predetermined event, the plurality of specified time differences being longer than a time difference between the time of the occurrence of the predetermined event and a time when the luggage abandonment is detected; and extracting the feature in a plurality of frames at capturing times calculated using the specified time differences, respectively. ### Claim 13 (depends on 11) The non-transitory computer readable medium according to claim 11, wherein the process further comprises determining that the detected object is the luggage that was abandoned if the detected object is at rest for a predetermined time. ### Claim 14 (depends on 11) The non-transitory computer readable medium according to claim 11, wherein the process further comprises determining that the detected object is the person who abandoned the luggage, as a person nearest to the abandoned luggage at a specified time difference after the luggage abandonment that is longer than a time difference between the time of the occurrence of the predetermined event and a time when the luggage abandonment is detected. ### Claim 15 (depends on 11) The non-transitory computer readable medium according to claim 11, wherein the process further comprises learning the time difference using a reliability of the feature.