# US 11,113,515 B2 · Information processing device and information processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,515 B2 - Document ID: US11113515B2 - Status: Granted - Kind: B2 - Application number: 16/099,498 - Filed: 2017-02-24 - Published: 2019-04-11 - Granted: 2021-09-07 - Earliest claimed: 2016-05-17 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Sota Matsuzawa - Web page: https://patentgrep.com/patent/US11113515B2 ## CPC - G06V 40/23: {Recognition of whole body movements, e.g. for sport training} - A63B 24/0003: {Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis} - G01P 15/00: Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration - G06F 18/22: Matching criteria, e.g. proximity measures - G06V 40/25: {Recognition of walking or running movements, e.g. gait recognition} - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H) - G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G}) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Provided is an information processing device including a sensor control unit that acquires distance measuring sensor data detected by a plurality of distance measuring sensors and a data processing unit that obtains action recognition results of one or a plurality of users by analyzing the distance measuring sensor data. The information processing device further includes an output control unit that controls output based on the action recognition result. ## Claims ### Claim 1 An information processing device, comprising: a plurality of distance measuring sensors configured to detect distance measuring sensor data; and a processor configured to: acquire the detected distance measuring sensor data; calculate, based on the acquired distance measuring sensor data, a first distance between a first distance measuring sensor of the plurality of distance measuring sensors and a second distance measuring sensor of the plurality of distance measuring sensors; obtain an action recognition result of a user based on a period of time in which the calculated first distance is constant, wherein the period of time is greater than a specific period of time; calculate a score as a degree of similarity between the action recognition result and a specific action of the user; and control output of the calculated score. ### Claim 2 (depends on 1) The information processing device according to claim 1, wherein the processor is further configured to: calculate a second distance between a third distance measuring sensor of the plurality of distance measuring sensors and a fourth distance measuring sensor of the plurality of distance measuring sensors; and obtain the action recognition result of the user based on the calculated first distance and the calculated second distance. ### Claim 3 (depends on 1) The information processing device according to claim 1, further comprising at least one inertial sensor configured to detect inertial sensor data, wherein the processor is further configured to: acquire the detected inertial sensor data; analyze the acquired inertial sensor data; and obtain the action recognition result of the user based on the analyzed inertial sensor data. ### Claim 4 (depends on 3) The information processing device according to claim 3, wherein in a case where the acquired inertial sensor data satisfies a specific condition, the processor is further configured to obtain the action recognition result of the user based on a second distance between the first distance measuring sensor of the plurality of distance measuring sensors and a third distance measuring sensor of the plurality of distance measuring sensors. ### Claim 5 (depends on 3) The information processing device according to claim 3, wherein in a case where the acquired inertial sensor data satisfies a specific condition, the processor is further configured to obtain the action recognition result of the user based on a second distance between the first distance measuring sensor of the plurality of distance measuring sensors and each of the second distance measuring sensor of the plurality of distance measuring sensors, a third distance measuring sensor of the plurality of distance measuring sensors, and a fourth distance measuring sensor of the plurality of distance measuring sensors. ### Claim 6 (depends on 3) The information processing device according to claim 3, wherein the processor is further configured to obtain the action recognition result of the user based on a second distance between the first distance measuring sensor of the plurality of distance measuring sensors and a third distance measuring sensor of the plurality of distance measuring sensors, and the acquired inertial sensor data. ### Claim 7 (depends on 3) The information processing device according to claim 3, further comprising a plurality of proximity sensors configured to detect proximity sensor data, wherein the processor is further configured to: acquire the detected proximity sensor data; analyze the acquired proximity sensor data; and obtain the action recognition result of the user based on the analyzed inertial sensor data and the analyzed proximity sensor data. ### Claim 8 (depends on 1) The information processing device according to claim 1, further comprising at least one inertial sensor configured to detect inertial sensor data, wherein the plurality of distance measuring sensors is attachable to the user. ### Claim 9 (depends on 8) The information processing device according to claim 8, wherein the first distance measuring sensor of the plurality of distance measuring sensors is attachable to a right foot of the user, a third distance measuring sensor of the plurality of distance measuring sensors is attachable to a left foot of the user, and in a case where the inertial sensor data satisfies a specific condition, the processor is further configured to acquire a second distance between the first distance measuring sensor and the third distance measuring sensor, as a step size of the user. ### Claim 10 (depends on 8) The information processing device according to claim 8, wherein the first distance measuring sensor of the plurality of distance measuring sensors is attachable to one of a hand of the user, a wrist of the user, or an arm of the user, the second distance measuring sensor of the plurality of distance measuring sensors, a third distance measuring sensor of the plurality of distance measuring sensors, and a fourth distance measuring sensor of the plurality of distance measuring sensors are attachable to a body of the user, in a case where the inertial sensor data satisfies a specific condition, the processor is further configured to estimate a track of a golf club swung by the user, and the estimation of the track of the golf club is based on a second distance between the first distance measuring sensor and each of the second distance measuring sensor, the third distance measuring sensor, and the fourth distance measuring sensor. ### Claim 11 (depends on 8) The information processing device according to claim 8, wherein the first distance measuring sensor of the plurality of distance measuring sensors is attachable to one of a hand of the user, a wrist of the user, or an arm of the user, a third distance measuring sensor of the plurality of distance measuring sensors is attachable to one of a foot of the user or a board associated with the user, and in a case where the inertial sensor data satisfies a specific condition, the processor is further configured to determine a trick of snowboarding based on a second distance between the first distance measuring sensor and the third distance measuring sensor. ### Claim 12 (depends on 8) The information processing device according to claim 8, wherein the first distance measuring sensor of the plurality of distance measuring sensors is attachable to a foot of the user, a third distance measuring sensor of the plurality of distance measuring sensors is attachable to a body of the user, and in a case where the inertial sensor data satisfies a specific condition, the processor is further configured to estimate a kick performance of a ball based on a second distance between the first distance measuring sensor and the third distance measuring sensor. ### Claim 13 (depends on 1) The information processing device according to claim 1, further comprising at least one inertial sensor configured to detect inertial sensor data, wherein the plurality of distance measuring sensors is attachable to a plurality of users. ### Claim 14 (depends on 13) The information processing device according to claim 13, wherein the first distance measuring sensor of the plurality of distance measuring sensors is attachable to a first user of the plurality of users, a third distance measuring sensor of the plurality of distance measuring sensors is attachable to a second user of the plurality of users, and in a case where the inertial sensor data satisfies a first condition and a second distance between the first distance measuring sensor and the third distance measuring sensor satisfies a second condition, the processor is further configured to obtain an action as the action recognition result based on the first condition and the second condition. ### Claim 15 (depends on 1) The information processing device according to claim 1, wherein the plurality of distance measuring sensors is attachable to the user and an object related to the user. ### Claim 16 (depends on 15) The information processing device according to claim 15, further comprising at least one inertial sensor configured to detect inertial sensor data, wherein the first distance measuring sensor of the plurality of distance measuring sensors is attachable to a foot of the user, a third distance measuring sensor of the plurality of distance measuring sensors is attachable to a ball, and in a case where the inertial sensor data satisfies a specific condition, the processor is further configured to determine a speed of the ball based on a second distance between the first distance measuring sensor and the third distance measuring sensor. ### Claim 17 An information processing method, comprising: detecting, by a plurality of distance measuring sensors, distance measuring sensor data; acquiring, by a processor, the detected distance measuring sensor data; calculating, by the processor based on the acquired distance measuring sensor data, a distance between a first distance measuring sensor of the plurality of distance measuring sensors and a second distance measuring sensor of the plurality of distance measuring sensors; obtaining, by the processor, an action recognition result of a user based on a period of time in which the calculated distance is constant, wherein the period of time is greater than a specific period of time; calculating, by the processor, a score as a degree of similarity between the action recognition result and a specific action of the user; and controlling, by the processor, output of the calculated score. ### Claim 18 A non-transitory computer-readable medium having stored thereon computer-executable instructions which, when executed by a processor, cause the processor to execute operations, the operations comprising: acquiring distance measuring sensor data detected by a plurality of distance measuring sensors; calculating, based on the acquired distance measuring sensor data, a distance between a first distance measuring sensor of the plurality of distance measuring sensors and a second distance measuring sensor of the plurality of distance measuring sensors; obtaining an action recognition result of a user based on a period of time in which the calculated distance is constant, wherein the period of time is greater than a specific period of time; calculating a score as a degree of similarity between the action recognition result and a specific action of the user; and controlling output of the calculated score.