# US 12,109,455 B2 · Data-driven assistance for users involved in physical activities > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,109,455 B2 - Document ID: US12109455B2 - Status: Granted - Kind: B2 - Application number: 17/391,789 - Filed: 2021-08-02 - Published: 2023-02-02 - Granted: 2024-10-08 - Earliest claimed: 2021-08-02 - Assignees: Sony Group Corporation - Applicants: Sony Group Corporation - Inventors: Sri Ranganadh Thatha - Web page: https://patentgrep.com/patent/US12109455B2 ## CPC - A63B 24/0062: {Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance} - G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising - A61B 5/1112: {Global tracking of patients, e.g. by using GPS} - A61B 5/1114: {Tracking parts of the body} - A63B 24/0075: {Means for generating exercise programmes or schemes, e.g. computerized virtual trainer, e.g. using expert databases} - A63B 71/0622: {Visual, audio or audio-visual systems for entertaining, instructing or motivating the user} - G01P 13/00: Indicating or recording presence, absence, or direction, of movement (electric switches H01H; counting moving objects G06M7/00) - G01P 15/18: in two or more dimensions - G06N 20/20: Ensemble learning - A63B 2024/0068: {Comparison to target or threshold, previous performance or not real time comparison to other individuals} - A63B 2071/0647: {Visualisation of executed movements} - A63B 2071/065: {Visualisation of specific exercise parameters} - A63B 2230/42: rate - A63B 2230/62: posture - A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G}) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract An electronic device and method for data-driven assistance for users involved in physical activities is provided. The electronic device receives first sensor data associated with a movement pattern of one or more parts of a body of a user and receives first information associated with a location where the user performs the physical activity. The electronic device determines one or more first indicators which are likely to have affected the user or the performance of the user in the physical activity. Thereafter, the electronic device generates presentation data based on an application of a first machine learning model on the determined one or more first indicators and the received first information. The generated presentation data includes one or more improvement suggestions for the user in relation to the physical activity. The electronic device controls a display device to display the presentation data. ## Claims ### Claim 1 An electronic device, comprising: circuitry configured to: receive a first sensor data associated with a movement pattern of one or more parts of a body of a user, wherein the first sensor data corresponds to a duration in which the user performs a physical activity; receive a first information associated with a location where the user performs the physical activity, wherein the received first information comprises at least one of the location of the user, a ground elevation of the location, weather information associated with the location, and terrain information associated with the location; determine, based on the received first sensor data, one or more first indicators which are likely to have affected the user or performance of the user in the physical activity; generate presentation data based on application of a first machine learning model on the determined one or more first indicators and the received first information, wherein the generated presentation data comprises one or more improvement suggestions for the user in relation to the physical activity; and control a display device to display the presentation data. ### Claim 2 (depends on 1) The electronic device according to claim 1, wherein the circuitry is further configured to receive second sensor data comprising one or more biological markers of the body, and wherein the presentation data is generated based on further application of the first machine learning model on the received second sensor data. ### Claim 3 (depends on 1) The electronic device according to claim 1, wherein the circuitry is further configured to receive second information associated with the performance of the user in the physical activity, and wherein the presentation data is generated based on further application of the first machine learning model on the received second information. ### Claim 4 (depends on 1) The electronic device according to claim 1, wherein the first sensor data comprises first accelerometer data associated with the movement pattern of an arm portion of the body. ### Claim 5 (depends on 4) The electronic device according to claim 4, wherein the circuitry is further configured to: generate motion detection data associated with the movement pattern of the arm portion, based on the first sensor data; filter the generated motion detection data by removal of a first set of outlier samples from the generated motion detection data; and determine the one or more first indicators by application of a motion classifier on the filtered motion detection data. ### Claim 6 (depends on 5) The electronic device according to claim 5, wherein the one or more improvement suggestions are associated with a swing movement of the arm portion. ### Claim 7 (depends on 1) The electronic device according to claim 1, wherein the received first sensor data comprises second accelerometer data associated with the movement pattern of a foot portion of the body. ### Claim 8 (depends on 7) The electronic device according to claim 7, wherein the circuitry is configured to apply one or more signal processing operations on the second accelerometer data to determine the one or more first indicators comprising landing measurements associated with the foot portion and a running stride of the user, and wherein the one or more improvement suggestions are associated with the movement pattern of the foot portion. ### Claim 9 (depends on 1) The electronic device according to claim 1, wherein the circuitry is further configured to: receive breathing data associated with the user, wherein the breathing data corresponds to the duration in which the user performs the physical activity; process the received breathing data to generate breath detection data; filter the generated breath detection data by removal of a second set of outlier samples from the generated breath detection data; and determine one or more second indicators comprising a breathing pattern of the user, based on the filtered breath detection data. ### Claim 10 (depends on 9) The electronic device according to claim 9, wherein the circuitry is further configured to receive second sensor data comprising one or more biological markers of the body, and wherein the presentation data is generated based on further application of the first machine learning model on the determined one or more second indicators and the received second sensor data. ### Claim 11 (depends on 10) The electronic device according to claim 10, wherein the one or more improvement suggestions are associated with the breathing pattern of the user. ### Claim 12 (depends on 1) The electronic device according to claim 1, wherein the circuitry is configured to: control a set of image sensors to capture a set of images of the user, wherein the set of images is captured within the duration in which the user performs the physical activity; determine skeletal joints of the user in at least one of the captured set of images; and apply a posture classifier on the determined skeletal joints to determine one or more third indicators comprising a first body posture of the user. ### Claim 13 (depends on 12) The electronic device according to claim 12, wherein the presentation data is generated based on further application of the first machine learning model on the determined one or more third indicators, and wherein the one or more improvement suggestions are associated with the first body posture. ### Claim 14 (depends on 1) The electronic device according to claim 1, wherein the circuitry is configured to: apply a second machine learning model on the determined one or more first indicators and the received first information to generate a classification result; and determine an experience level of the user in relation to the physical activity, based on the classification result, wherein the determined experience level is one of an amateur level athlete, an intermediate level athlete, or an expert level athlete, and the presentation data is generated further based on the determined experience level. ### Claim 15 An electronic device, comprising: circuitry configured to: receive a first sensor data associated with a movement pattern of one or more parts of a body of a user, wherein the first sensor data corresponds to a duration in which the user performs a physical activity; receive a first information associated with a location where the user performs the physical activity: determine, based on the received first sensor data, one or more first indicators which are likely to have affected the user or performance of the user in the physical activity; generate presentation data based on application of a first machine learning model on the determined one or more first indicators and the received first information, wherein the generated presentation data comprises one or more improvement suggestions for the user in relation to the physical activity and further comprises visual data which compares the one or more first indicators with known indicators of expert athletes; and control a display device to display the presentation data. ### Claim 16 (depends on 1) The electronic device according to claim 1, wherein the circuitry is configured to: determine one or more injuries which are likely to be caused by the movement pattern or the performance of the physical activity, wherein the one or more injuries are determined based on analysis of the determined one or more first indicators, and the presentation data further comprises information which informs the user about the one or more injuries and recommendations to avoid the one or more injuries. ### Claim 17 A method, comprising: receiving a first sensor data associated with a movement pattern of body of a user or one or more parts of the body, wherein the first sensor data corresponds to a duration in which the user performs a physical activity; receiving a first information associated with a location where the user performs the physical activity, wherein the received first information comprises at least one of the location of the user, a ground elevation of the location, weather information associated with the location, and terrain information associated with the location; determining, based on the received first sensor data, one or more first indicators which are likely to have affected the user or performance of the user in the physical activity; generating presentation data based on application of a first machine learning model on the determined one or more first indicators and the received first information, wherein the generated presentation data comprises one or more improvement suggestions for the user in relation to the physical activity; and controlling a display device to display the presentation data. ### Claim 18 A non-transitory computer-readable medium having stored thereon, computer-executable instructions that when executed by an electronic device, causes electronic device to execute operations, the operations comprising: receiving a first sensor data associated with a movement pattern of body of a user or one or more parts of the body, wherein the first sensor data corresponds to a duration in which the user performs a physical activity; receiving a first information associated with a location where the user performs the physical activity, wherein the received first information comprises at least one of the location of the user, a ground elevation of the location, weather information associated with the location, and terrain information associated with the location; determining, based on the received first sensor data, one or more first indicators which are likely to have affected the user or performance of the user in the physical activity; generating presentation data based on application of a first machine learning model on the determined one or more first indicators and the received first information, wherein the generated presentation data comprises one or more improvement suggestions for the user in relation to the physical activity; and controlling a display device to display the presentation data.