# US 2021/0279554 A1 · Method and apparatus for monitoring physical activity > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279554 A1 - Document ID: US20210279554A1 - Status: Published application - Kind: A1 - Application number: 17/187,230 - Filed: 2021-02-26 - Published: 2021-09-09 - Earliest claimed: 2020-03-03 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Nabil Ibtehaz - Web page: https://patentgrep.com/patent/US20210279554A1 ## CPC - G06N 3/045: Combinations of networks - A61B 5/1118: {Determining activity level} - A61B 5/1123: {Discriminating type of movement, e.g. walking or running (A61B5/1116, A61B5/112 take precedence)} - A61B 5/6817: {Ear canal} - A61B 5/6898: {Portable consumer electronic devices, e.g. music players, telephones, tablet computers} - A61B 5/7267: {involving training the classification device} - A61B 5/7278: {Artificial waveform generation or derivation, e.g. synthesizing signals from measured signals} - G06F 1/1684: {Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675} - G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection} - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising - G16H 40/67: for remote operation - G06F 1/163: {Wearable computers, e.g. on a belt} - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/047: Probabilistic or stochastic networks - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 20/20: Ensemble learning - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - H04W 88/02: Terminal devices - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract An apparatus for monitoring a physical activity includes: a memory storing one or more instructions; and at least one processor configured to execute the one or more instructions stored in the memory to: obtain first type sensor data from a first type wearable sensor; obtain second type sensor data from a second type wearable sensor; and identify the physical activity of a user by using at least one artificial intelligence learning model, the first type sensor data, and the second type sensor data. ## Claims ### Claim 1 An apparatus for monitoring a physical activity, the apparatus comprising: a memory storing one or more instructions; and at least one processor configured to execute the one or more instructions stored in the memory, to: obtain first type sensor data from a first type wearable sensor; obtain second type sensor data from a second type wearable sensor; and identify the physical activity of a user by using at least one artificial intelligence learning model, the first type sensor data, and the second type sensor data. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the at least one artificial intelligence learning model includes a first artificial intelligence learning model and a second artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to: identify the physical activity of the user by inputting the first type sensor data into the first artificial intelligence learning model; and verify the identified physical activity by inputting the second type sensor data into the second artificial intelligence learning model. ### Claim 3 (depends on 2) The apparatus of claim 2, wherein the at least one processor is further configured to execute the one or more instructions to verify the identified physical activity by inputting the second type sensor data into the second artificial intelligence learning model, when the physical activity of the user identified based on the first type sensor data is one of a plurality of pre-defined physical activities. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the at least one artificial intelligence learning model includes a first artificial intelligence learning model and a second artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to: identify a first physical activity of the user by inputting the first type sensor data into the first artificial intelligence learning model; identify a second physical activity of the user by inputting the second type sensor data into the second artificial intelligence learning model; and determine whether the first physical activity corresponds to the second physical activity correspond. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the at least one artificial intelligence learning model includes a first artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to identify the physical activity of the user by inputting the first type sensor data and the second type sensor data into the first artificial intelligence learning model. ### Claim 6 (depends on 5) The apparatus of claim 5, wherein, when a current value of the second type sensor data is not obtained, the at least one processor is further configured to execute the one or more instructions to identify the physical activity of the user by inputting, into the at least one artificial intelligence learning model, an estimated value of the second type sensor data, the estimated value being estimated based on at least one of a previous value of the second type sensor data or the first type sensor data. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the first type wearable sensor includes a motion sensor, and the first type sensor data includes motion sensor data obtained from the motion sensor, and the second type wearable sensor includes a biomedical sensor, and the second type sensor data includes biomedical sensor data obtained from the biomedical sensor. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the first type wearable sensor includes a first motion sensor worn on a first body part of the user, and the first type sensor data includes first body part motion sensor data obtained from the first motion sensor worn on the first body part of the user, and the second type wearable sensor includes a second motion sensor worn on a second body part of the user, and the second type sensor data includes second body part motion sensor data obtained from the second motion sensor worn on the second body part of the user. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the at least one artificial intelligence learning model includes a first artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to identify a whole body physical activity of the user by inputting the first body part motion sensor data and the second body part motion sensor data into the first artificial intelligence learning model. ### Claim 10 (depends on 8) The apparatus of claim 8, wherein the first type wearable sensor includes a smartphone motion sensor included in a smartphone, and the first type sensor data includes smartphone motion sensor data obtained from the smartphone motion sensor, the at least one artificial intelligence learning model includes a first artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to identify a body part, on which the smartphone is worn, based on the smartphone motion sensor data, by using the first artificial intelligence learning model. ### Claim 11 (depends on 8) The apparatus of claim 8, wherein the first body part and the second body part do not include a torso, the at least one artificial intelligence learning model includes a first artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to identify a motion of the torso of the user based on the first body part motion sensor data and the second body part motion sensor data, by using the first artificial intelligence learning model. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the first type wearable sensor includes a first side motion sensor of the first body part at a first side of the first body part, and the first type sensor data includes first side motion sensor data of the first body part obtained from the first side motion sensor of the first body part, the second type wearable sensor includes a second side motion sensor of the second body part at a second side of the second body part, and the second type sensor data includes second side motion sensor data of the second body part obtained from the second side motion sensor of the second body part, wherein the second side is opposite to the first side, and the at least one processor is further configured to execute the one or more instructions to identify the motion of the torso of the user based on the first side motion sensor data of the first body part and the second side motion sensor data of the second body part, by using the first artificial intelligence learning model. ### Claim 13 (depends on 8) The apparatus of claim 8, wherein the second type wearable sensor includes an earphone motion sensor included in an earphone, and the second type sensor data includes earphone motion sensor data obtained from the earphone motion sensor. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the earphone motion sensor includes a left earphone motion sensor included in a left earphone portion and a right earphone motion sensor included in a right earphone portion, and the earphone motion sensor data includes left earphone motion sensor data obtained from the left earphone motion sensor and right earphone motion sensor data obtained from the right earphone motion sensor. ### Claim 15 (depends on 13) The apparatus of claim 13, wherein the first type wearable sensor includes a first side motion sensor at a first side of the first body part, and the first type sensor data includes first side motion sensor data obtained from the first side motion sensor, the at least one artificial intelligence learning model includes a first artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to identify a motion of a second side of the first body part based on the first side motion sensor data and the earphone motion sensor data, by using the first artificial intelligence learning model, wherein the second side is opposite to the first side. ### Claim 16 (depends on 13) The apparatus of claim 13, wherein the at least one processor is further configured to execute the one or more instructions to determine vertical symmetry of the physical activity of the user based on the earphone motion sensor data. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the first type wearable sensor includes a one-sided motion sensor at a side of the first body part, and the first type sensor data includes one-sided motion sensor data obtained from the one-sided motion sensor, the at least one artificial intelligence learning model includes a first artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to: identify the physical activity of the user by inputting the one-sided motion sensor data into the first artificial intelligence learning model; and verify the identified physical activity of the user based on the determined vertical symmetry. ### Claim 18 (depends on 16) The apparatus of claim 16, wherein the first type wearable sensor includes a one-sided motion sensor at a side of the second body part, and the first type sensor data includes one-sided motion sensor data obtained from the one-sided motion sensor, the at least one artificial intelligence learning model includes a first artificial intelligence learning model, and the at least one processor is further configured to execute the one or more instructions to identify the physical activity of the user by inputting the determined vertical symmetry and the one-sided motion sensor data into the first artificial intelligence learning model. ### Claim 19 An operating method of an apparatus for monitoring a physical activity, the operating method comprising: obtaining first type sensor data from a first type wearable sensor; obtaining second type sensor data from a second type wearable sensor; and identifying the physical activity of a user by using at least one artificial intelligence learning model, the first type sensor data, and the second type sensor data. ### Claim 20 (depends on 19) The operating method of claim 19, wherein the at least one artificial intelligence learning model includes a first artificial intelligence learning model and a second artificial intelligence learning model, and the operating method further comprises: identifying the physical activity of the user by inputting the first type sensor data into the first artificial intelligence learning model; and verifying the identified physical activity by inputting the second type sensor data into the second artificial intelligence learning model, when the physical activity of the user identified based on the first type sensor data is one of a plurality of pre-defined physical activities.