# US 12,118,741 B2 · Three-dimensional person behavior estimation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,741 B2 - Document ID: US12118741B2 - Status: Granted - Kind: B2 - Application number: 17/617,431 - Filed: 2019-06-13 - Published: 2022-08-04 - Granted: 2024-10-15 - Earliest claimed: 2019-06-13 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Jianquan Liu; Junnan Li - Web page: https://patentgrep.com/patent/US12118741B2 ## CPC - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/82: using neural networks - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - G08B 13/196: using television cameras - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS ## Abstract The present invention provides a processing apparatus (20) including a first generation unit (22) that generates, from a plurality of time-series images, three-dimensional feature information indicating a time change of a feature in each position in each of the plurality of images, a second generation unit (23) that generates person position information indicating a position in which a person is present in each of the plurality of images, and an estimation unit (24) that estimates person behavior indicated by the plurality of images, based on the time change of the feature indicated by the three-dimensional feature information in the position in which the person is present being indicated by the person position information. ## Claims ### Claim 1 A processing apparatus comprising: at least one memory storing one or more instructions; and at least one processor configured to execute the one or more instructions to: generate, from a plurality of time-series images, three-dimensional feature information indicating a time change of a feature in each position in-each of the plurality of time-series images by inputting the plurality of time-series images to a CNN (Convolutional Neural Network), the CNN receiving input of the plurality of time-series images and outputting the three-dimensional feature information; generate person position information indicating a position in which a person is present in each of the plurality of time-series images; adjust the three-dimensional feature information by changing a value in a position in which no person is present to a predetermined value based on the person position information; and estimate person behavior indicated by the plurality of time-series images, based on the adjusted three-dimensional feature information indicating the predetermined value in positions in which no person is present and indicating the time change of the feature in positions in which the person is present. ### Claim 2 (depends on 1) The processing apparatus according to claim 1, wherein the at least one processor configured is to execute the one or more instructions to: generate the three-dimensional feature information, based on a 3D convolutional neural network (CNN); and generate the person position information, based on a deep learning network of object recognition. ### Claim 3 (depends on 1) The processing apparatus according to claim 1, wherein, the at least one processor is configured to execute the one or more instructions to generate, in case in which a plurality of persons are present in the image, the person position information indicating a position in which each of the plurality of persons is present. ### Claim 4 A processing method performed by a computer and comprising: generating, from a plurality of time-series images, three-dimensional feature information indicating a time change of a feature in each position in-each of the plurality of time-series images by inputting the plurality of time-series images to a CNN (Convolutional Neural Network), the CNN receiving input of the plurality of time-series images and outputting the three-dimensional feature information; generating person position information indicating a position in which a person is present in each of the plurality of time-series images; adjusting the three-dimensional feature information by changing a value in a position in which no person is present to a predetermined value based on the person position information; and estimating person behavior indicated by the plurality of time-series images, based on the adjusted three-dimensional feature information indicating the predetermined value in positions in which no person is present and indicating the time change of the feature in positions in which the person is present. ### Claim 5 A non-transitory storage medium storing a program causing a computer to: generate, from a plurality of time-series images, three-dimensional feature information indicating a time change of a feature in each position in-each of the plurality of time-series images by inputting the plurality of time-series images to a CNN (Convolutional Neural Network), the CNN receiving input of the plurality of time-series images and outputting the three-dimensional feature information; generate person position information indicating a position in which a person is present in each of the plurality of time-series images; adjust the three-dimensional feature information by changing a value in a position in which no person is present to a predetermined value based on the person position information; and estimate person behavior indicated by the plurality of time-series images, based on the adjusted three-dimensional feature information indicating the predetermined value in positions in which no person is present and indicating the time change of the feature in positions in which the person is present. ### Claim 6 (depends on 4) The processing method according to claim 4, wherein the computer generates the three-dimensional feature information based on a 3D convolutional neural network (CNN), and generates the person position information based on a deep learning network of object recognition. ### Claim 7 (depends on 4) The processing method according to claim 4, wherein the computer generates, in case in which a plurality of persons are present in the image, the person position information indicating a position in which each of the plurality of persons is present. ### Claim 8 (depends on 5) The non-transitory storage medium according to claim 5, wherein the program causes the computer to: generate the three-dimensional feature information, based on a 3D convolutional neural network (CNN); and generate the person position information, based on a deep learning network of object recognition. ### Claim 9 (depends on 5) The non-transitory storage medium according to claim 5, wherein the program causes the computer to: generate, in case in which a plurality of persons are present in the image, the person position information indicating a position in which each of the plurality of persons is present.