# US 11,304,656 B2 · Wearable device for medication adherence monitoring > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,304,656 B2 - Document ID: US11304656B2 - Status: Granted - Kind: B2 - Application number: 17/358,116 - Filed: 2021-06-25 - Published: 2022-02-24 - Granted: 2022-04-19 - Earliest claimed: 2020-08-19 - Assignees: Inhandplus Inc. - Applicants: Inhandplus Inc. - Inventors: Hwiwon Lee; Nam Eok Kim - Web page: https://patentgrep.com/patent/US11304656B2 ## CPC - A61B 5/002: {Monitoring the patient using a local or closed circuit, e.g. in a room or building (A61B5/0017 takes precedence)} - A61B 5/4833: {Assessment of subject's compliance to treatment} - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - A61B 5/681: {Wristwatch-type devices} - A61B 5/6824: {Arm or wrist} - A61B 5/7285: {for synchronizing or triggering a physiological measurement or image acquisition with a physiological event or waveform, e.g. an ECG signal} - G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade} - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/047: Probabilistic or stochastic networks - G06N 3/0475: Generative networks - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/251: {involving models} - G06V 10/17: {using hand-held instruments} - G06V 10/82: using neural networks - G06V 10/95: {structured as a network, e.g. client-server architectures} - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06V 20/66: Trinkets, e.g. shirt buttons or jewellery items (recognising microscopic objects G06V20/69) - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - G08C 17/02: using a radio link - G16H 20/10: relating to drugs or medications, e.g. for ensuring correct administration to patients - G16H 20/13: delivered from dispensers - G16H 40/63: for local operation - G16H 40/67: for remote operation - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - H04N 5/765: Interface circuits between an apparatus for recording and another apparatus - H04N 5/77: between a recording apparatus and a television camera - H04N 23/65: Control of camera operation in relation to power supply - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - H04Q 9/00: Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom - H04W 76/10: Connection setup - A61B 2560/0242: adapted to measure environmental factors, e.g. temperature, pollution - A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches - G06N 3/08: Learning methods - G06T 2207/10016: Video; Image sequence - G06T 2207/20081: Training; Learning - G06T 2207/30196: Human being; Person - G06V 2201/03: Recognition of patterns in medical or anatomical images - G16H 70/40: relating to drugs, e.g. their side effects or intended usage - H04Q 2209/43: using wireless personal area networks [WPAN], e.g. 802.15, 802.15.1, 802.15.4, Bluetooth® or Zigbee® - H04W 64/006: {with additional information processing, e.g. for direction or speed determination} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08C: TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS (fluid pressure transmitting systems F15B; mechanical means for transferring the output of a sensing member into a different variable G01D5/00; mechanical control systems G05G) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04Q: SELECTING (switches, relays, selectors H01H; wireless communication networks H04W) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract A monitoring system for obtaining a video related to medication adherence of a user, includes: a wireless communication device comprising a motion sensor, an ambient light sensor, a first transceiver transmitting data to an external device, and a first controller configured to control the motion sensor, the ambient light sensor, and the first transceiver, the wireless communication device having an attaching portion for being attached to an object containing a medication; and a wearable device including a camera, a second transceiver receiving a signal from the first transceiver, and a second controller configured to obtain video data through the camera based on the signal received through the second transceiver. ## Claims ### Claim 1 A monitoring system for obtaining a video related to medication adherence of a user, the monitoring system comprising: a wireless communication device comprising a motion sensor, an ambient light sensor, a first transceiver transmitting data to an external device, and a first controller configured to control the motion sensor, the ambient light sensor, and the first transceiver, the wireless communication device having an attaching portion for being attached to an object containing a medication; a wearable device comprising a camera, a second transceiver receiving a signal from the first transceiver, and a second controller configured to obtain video data through the camera based on the signal received through the second transceiver; wherein the first controller of the wireless communication device is configured to: obtain, by using the motion sensor, a motion value corresponding to a movement of the object, obtain, by using the ambient light sensor, an ambient light value corresponding to ambient light of the object, generate activation data for inducing activation of the camera of the wearable device, based on the motion value and the ambient light value, and provide, by using the first transceiver, the activation data to the wearable device, and wherein the second controller of the wearable device obtains the video data by activating the camera based on the received activation data generated from the wireless communication device based on the movement and the ambient light of the object. ### Claim 2 (depends on 1) The monitoring system of claim 1, wherein the wearable device initiates a pairing process when receiving an advertising signal from the wireless communication device, wherein the wearable device and the wireless communication device are paired as they share link data; and wherein the first controller provides the activation data to the second controller after the wearable device and the wireless communication device are paired. ### Claim 3 (depends on 1) The monitoring system of claim 1, wherein the wireless communication device is configured to operate in a sleep state or a standby state, wherein the first controller does not obtain a sensor value from the motion sensor and the ambient light sensor in the sleep state and the first controller obtains the sensor value from at least one of the motion sensor or the ambient light sensor in the standby state, and wherein the wireless communication device operates from the sleep state to the standby state when being paired with the wearable device. ### Claim 4 (depends on 3) The monitoring system of claim 3, wherein the first controller obtains the motion value by using the motion sensor in the standby state. ### Claim 5 (depends on 4) The monitoring system of claim 4, wherein the wireless communication device operates in the sleep state, the standby state, or the awake state, wherein the first controller controls the wireless communication device to operate in the awake state when the first controller determines that the motion value obtained in the standby state is equal to or greater than a motion threshold value, and wherein the first controller, in the awake state, generates the activation data and provides the activation data to the second controller when the ambient light value obtained by the ambient light sensor is equal to or greater than an ambient light threshold value. ### Claim 6 (depends on 2) The monitoring system of claim 2, wherein the wireless communication device is configured to operate in a sleep state or a standby state, wherein the motion sensor obtains the motion value per a first time gap in the sleep state, and wherein the motion sensor obtains the motion value per a second time gap, which is shorter than the first time gap, in the standby state. ### Claim 7 (depends on 2) The monitoring system of claim 2, wherein the wireless communication device is configured to operate in a sleep state or a standby state, wherein the ambient light sensor obtains the ambient light value per a first time gap in the sleep state, and wherein the ambient light sensor obtains the ambient light value per a second time gap, which is shorter than the first time gap, in the standby state. ### Claim 8 (depends on 1) The monitoring system of claim 1, wherein the first controller generates the activation data when the motion value obtained by using the motion sensor is equal to or greater than a predetermined motion threshold value and the ambient light value obtained by using the ambient light sensor is equal to or greater than a predetermined ambient light threshold value. ### Claim 9 (depends on 1) The monitoring system of claim 1, wherein the activation data is included in an advertising signal that the wireless communication device transmits to the wearable device. ### Claim 10 (depends on 1) The monitoring system of claim 1, wherein the first controller provides unique identification data of the wireless communication device to the second controller, wherein an imaging time is set based on the unique identification data of the wireless communication device, and wherein the second controller activates the camera based on the activation data to take a video for an imaging time. ### Claim 11 (depends on 10) The monitoring system of claim 10, wherein a signal that the second controller receives from the first transceiver includes a first signal received before a first time point when the wireless communication device and the wearable device are paired and a second signal received after the first time point, and wherein the second controller obtains the activation data from the second signal. ### Claim 12 (depends on 11) The monitoring system of claim 11, wherein the second controller obtains the unique identification data from the second signal. ### Claim 13 (depends on 1) The monitoring system of claim 1, further comprising a server obtaining the video data from the wearable device and determining whether the user has taken medication by using the video data. ### Claim 14 (depends on 13) The monitoring system of claim 13, wherein the wearable device activates the camera for an imaging time and transmits the video data to the server after a predetermined waiting time from when the camera is deactivated. ### Claim 15 (depends on 1) The monitoring system of claim 1, wherein the wearable device takes a video for a predetermined imaging time when obtaining a first activation data from the wireless communication device, and wherein the wearable device does not take a video even when obtaining a second activation data from the wireless communication device during the imaging time. ### Claim 16 (depends on 1) The monitoring system of claim 1, wherein the wearable device takes a video for a predetermined imaging time when obtaining a first activation data from the wireless communication device, wherein the wearable device further takes a video for an extra time when obtaining a second activation data from the wireless communication device during the imaging time, and wherein the extra time is shorter than the imaging time. ### Claim 17 (depends on 1) The monitoring system of claim 1, wherein the wearable device includes an output module and provides an alarm to the user through the output module while the video data is obtained as the camera being activated. ### Claim 18 (depends on 13) The monitoring system of claim 13, wherein the server determines whether the user has taken medication by using a medication monitoring model trained as outputting data related to whether the user has taken medication when the video data is inputted. ### Claim 19 (depends on 18) The monitoring system of claim 18, wherein the medication monitoring model includes: a detection model receiving image frames of the video data as input and outputting a probability value indicating whether the object related to the medication adherence is included in the video data, and a confirmation model receiving input data reflecting the probability value and outputting a class value indicating whether the user has taken the medication. ### Claim 20 The wireless communication device configured to transmit data to an external device for inducing the external device including a camera to initiate taking a video, the wireless communication device comprising: a transceiver; a motion sensor; an ambient light sensor; a controller; a housing having an inner space where the transceiver, the motion sensor, the ambient light sensor, and the controller are placed; and an attachment region implemented outside of the housing, the wireless communication device configured to attach to an object containing medication through the attachment region; wherein a controller is configured to: transmit the data to the external device through the transceiver; obtain a motion value reflecting a movement of the object by using the motion sensor, obtain an ambient light value reflecting an ambient environment of the object by using the ambient light sensor, and provide activation data based on the motion value and the ambient light value, the activation data indicating the camera of the external device to be activated, wherein the controller is further configured to: determine whether the ambient light value is equal to or greater than an ambient light threshold value when the motion value is equal to or greater than a motion threshold value, and generate the activation data and provide the activation data to the external device when the ambient light value is equal to or greater than the ambient light threshold value, and wherein the controller does not generate the activation data when the ambient light value is less than the ambient light threshold value even though the motion value is equal to or greater than the motion threshold value. ### Claim 21 (depends on 20) The wireless communication device of claim 20, wherein the controller and the motion sensor communicate in a full duplex manner, and wherein the controller and the ambient light sensor communicate in a half duplex manner. ### Claim 22 (depends on 20) The wireless communication device of claim 20, further comprising: a light sensing region for the ambient light sensor receiving light, the light sensing region not being overlapped with the attachment region. ### Claim 23 (depends on 22) The wireless communication device of claim 22, wherein the housing has a shape of cylinder, wherein the attachment region is located on a lower surface of the housing while the light sensing region is located on an upper surface of the housing.