# US 2019/0379964 A1 · Earphones with activity controlled output > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0379964 A1 - Document ID: US20190379964A1 - Status: Published application - Kind: A1 - Application number: 16/416,480 - Filed: 2019-05-20 - Published: 2019-12-12 - Earliest claimed: 2014-11-21 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Sammy PERGAMENT; Victor PÅSSE; Iman HABIB; Jesper JOHANSSON; Amir ADLOUNI - Web page: https://patentgrep.com/patent/US20190379964A1 ## CPC - H04R 1/1041: {Mechanical or electronic switches, or control elements (switches in general H01H)} - A61B 5/0205: Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition - A61B 5/1118: {Determining activity level} - A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety} - A61B 5/4809: {Sleep detection, i.e. determining whether a subject is asleep or not} - A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles} - A61B 5/6815: {Ear} - A61B 5/7225: {Details of analogue processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation (input circuits for detecting, measuring, or recording bioelectric or biomagnetic signals A61B5/30; specific diagnostic methods using bioelectric or biomagnetic signals A61B5/316)} - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - H04R 1/1025: Accumulators specially adapted for earpieces; Arrangements specially adapted for charging thereof - A61B 5/02416: {using photoplethysmograph signals, e.g. generated by infrared radiation (A61B5/14552 takes precedence)} - A61B 2560/0209: adapted for power saving - A61B 2562/0204: Acoustic sensors - A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches - H04R 2201/107: Monophonic and stereophonic headphones with microphone for two-way hands free communication - H04R 2430/01: Aspects of volume control, not necessarily automatic, in sound systems - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The present disclosure relates to an earphone apparatus and method. The earphone apparatus includes an earpiece, including a speaker, configured for arrangement relative to a user's ear for listening to audio from the speaker, a bio-sensor, a motion sensor, and a controller configured to determine an activity state of the user based on aggregated sensor data from the bio-sensor and the motion sensor over time, and to control audio output to the user based on the determined activity state. ## Claims ### Claim 1 An earphone system, comprising: an earpiece, including a speaker and a bio-sensor, configured for arrangement relative to a user's ear for listening to audio from the speaker; a motion sensor configured to detect motion of the user; and one or more processors configured to determine an activity state of the user based on a combined consideration of obtained bio-signal data from the bio-sensor and obtained motion data from the motion sensor, and control a user sensory output to the user based on the determined activity state of the user. ### Claim 2 (depends on 1) The earphone system of claim 1, wherein, for the determining of the activity state of the user, the one or more processors are configured to determine a preliminary activity state of the user based on the bio-signal data from the bio-sensor, and confirm the preliminary activity state based on the motion data from the motion sensor, to determine the activity state of the user. ### Claim 3 (depends on 2) The earphone system of claim 2, wherein, for the determining of the activity state of the user, the one or more processors are configured to determine the activity state of the user based on learned activity states of the user over time. ### Claim 4 (depends on 1) The earphone system of claim 1, further comprising a microphone, wherein, to determine the activity state of the user, the one or more processors are configured to determine a preliminary activity state of the user based on the bio-signal data and the motion data, and confirm the preliminary activity state based on a determination whether the preliminary activity state of the user correlates to an activity state represented by captured ambient sounds by the microphone. ### Claim 5 (depends on 4) The earphone system of claim 4, wherein, for the determining of the activity state of the user, the one or more processors are configured to determine the activity state of the user based on learned activity states of the user over time. ### Claim 6 (depends on 4) The earphone system of claim 4, wherein the bio-sensor and the one or more processors are configured within the earpiece, and the microphone is connected to the earpiece or configured within the earpiece. ### Claim 7 (depends on 1) The earphone system of claim 1, wherein the determined activity state of the user is a learned activity state. ### Claim 8 (depends on 7) The earphone system of claim 7, wherein the one or more processors are configured within the earpiece, and wherein the motion sensor is configured in the earpiece, another earpiece of the earphone system, or a media device of the earphone system separate from respective one or more earpieces of the earphone system. ### Claim 9 (depends on 8) The earphone system of claim 8, wherein the sensory output is output audio, and wherein the one or more processors are configured to control output of the output audio dependent on the determined activity state of the user, including control of an adjustment of the output audio to match a determined low activity when the determined activity state is a predetermined low activity state and an adjustment of the output audio to match a determined high activity when the determined activity state is a predetermined high activity state. ### Claim 10 (depends on 7) The earphone system of claim 7, wherein the one or more processors are configured in a media device of the earphone system separate from respective one or more earpieces of the earphone system. ### Claim 11 (depends on 10) The earphone system of claim 10, wherein the motion sensor is configured in the earpiece and the earphone system further comprises another motion sensor configured in another earpiece or separate media device of the earphone system, and the determining of the activity state further considers other obtained motion data from the other motion sensor. ### Claim 12 (depends on 10) The earphone system of claim 10, wherein the motion sensor is configured in the earpiece or another earpiece of the earphone system, and wherein, for the control of the user sensory output, the one or more other processors further control an adjustment of audio media output to the user, as the user sensory data, to match a determined low activity when the determined activity state is a predetermined low activity state and an adjustment of the audio media output to the user to match a determined high activity when the determined activity state is a predetermined high activity state. ### Claim 13 (depends on 10) The earphone system of claim 10, wherein the media device provides the earpiece audio media, as the controlled output user sensory data, wirelessly for output to the user using the speaker. ### Claim 14 (depends on 10) The earphone system of claim 10, wherein the determined activity state is determined from among plural learned activity states, including one or more learned activity states of the user over time. ### Claim 15 (depends on 10) The earphone system of claim 10, wherein the motion sensor is configured in the media device. ### Claim 16 (depends on 15) The earphone system of claim 15, wherein, for the control of the user sensory output, the one or more other processors further control an adjustment of audio media output to the user, as the user sensory data, to match a determined low activity when the determined activity state is a predetermined low activity state and an adjustment of the audio media output to the user to match a determined high activity when the determined activity state is a predetermined high activity state. ### Claim 17 (depends on 15) The earphone system of claim 15, wherein the media device provides the earpiece audio media, as the controlled output user sensory data, wirelessly for output to the user using the speaker. ### Claim 18 (depends on 15) The earphone system of claim 15, wherein the determined activity state is determined from among plural learned activity states, including one or more learned activity states of the user over time. ### Claim 19 (depends on 1) The earphone system of claim 1, wherein the motion sensor and the one or more processors are configured in a media device of the earphone system separate from respective one or more earpieces of the earphone system, the media device includes a user interface, and the control of the user sensory output to the user based on the determined activity state is dependent on which exercise activity state, among plural exercise activity states, is determined to be the activity state. ### Claim 20 (depends on 19) The earphone system of claim 19, wherein the plural exercise activity states include one or more learned exercise activity states. ### Claim 21 (depends on 1) The earphone system of claim 1, wherein, when the motion data from the motion sensor and the bio-signal data from the bio-sensor indicate the user is resting, the one or more processors determine whether the activity state is a sleeping state trained based on detected movements over time by the motion sensor that are below a predetermined threshold, the trained sleeping state being updated over time. ### Claim 22 (depends on 1) The earphone system of claim 1, wherein the one or more processors determine where the motion data from the motion sensor and the bio-signal data from the bio-sensor falls on a determined sleep-to-workout scale based on detected changes in movements of the user, to determine the activity state of the user. ### Claim 23 (depends on 1) The earphone system of claim 1, wherein the one or more processors are configured to determine between plural associated sub-states of the determined activity state based on detected changes in heart rate within a range of heart rates indicated by the bio-signal data of the bio-sensor and corresponding to the determined activity state, and to selectively control the user sensory output based on the determination between the plural sub-states. ### Claim 24 (depends on 23) The earphone system of claim 23, wherein, when the determined activity state is a workout state, the plural associated sub-states include two or more of warming up, running, lifting weights, bicycling, climbing, dancing, jumping, doing yoga, and stretching sub-states. ### Claim 25 (depends on 23) The earphone system of claim 23, wherein the one or more processors consider between the plural associated sub-states of the determined activity state based on location information regarding a location of the user. ### Claim 26 (depends on 25) The earphone system of claim 25, wherein the one or more processors are configured within the earpiece and the one or more processors determine the location of the user. ### Claim 27 (depends on 26) The earphone system of claim 26, wherein the one or more processors are configured within a media device of the earphone system and separate from respective one or more earpieces of the earphone system, and the one or more processors determine the location of the user. ### Claim 28 (depends on 23) The earphone system of claim 23, wherein the determination between the plural sub-states is based on a determined progression between the plural sub-states. ### Claim 29 (depends on 1) The earphone system of claim 1, further comprising a remote control unit, separate from respective one or more earpieces of the earphone system, comprising another motion sensor and a user control interface for user selective further control of the user sensory output, wherein the motion sensor is configured in the earpiece, and wherein the determining of the activity state further considers other obtained motion data from the other motion sensor. ### Claim 30 (depends on 1) The earphone system of claim 1, wherein the motion sensor is an acceleration sensor and the bio-sensor is a heart rate sensor. ### Claim 31 (depends on 1) The earphone system of claim 1, wherein the earpiece further comprises a communication module and wirelessly connects with a media device to receive audio output, corresponding to the user sensory output, using the communication module. ### Claim 32 (depends on 31) The earphone system of claim 31, wherein the one or more processors are configured in the earpiece and are further configured to transmit control data to the media device, based on the determined activity state, for controlling of the media device to provide the earpiece the audio output. ### Claim 33 (depends on 1) The earphone system of claim 1, further comprising a power harvesting component to perform a trickle energy capture, to power the motion sensor and the bio-sensor, using a wired connection to a media device providing media to the earphone system, wherein the one or more processors are further configured to selectively control respective periods of time the motion sensor and the bio-sensor are controlled to respectively measure motion and bio-signals, and wherein the power harvesting component selectively performs between the trickle energy capturing, with a storage of captured energy in a local power storage of the earpiece, during power off times of the bio-sensor of the respective periods of time and a supplying of the stored captured energy from the local power storage to the bio-sensor during power on times of the bio-sensor of the respective periods of time. ### Claim 34 (depends on 1) The earphone system of claim 1, wherein the one or more processors are further configured to filter the bio-signal data from the bio-sensor using the motion data from the motion sensor, included in the earpiece, and/or motion data from another motion sensor of the earphone system. ### Claim 35 (depends on 1) The earphone system of claim 1, wherein the one or more processors are further configured to monitor the motion data of the motion sensor, included in the earpiece, and/or other motion data from another motion sensor of the earphone system for detected indications that the user is inputting a touch or tap command, and to not rely on, for the determining of the activity state of the user, the motion data of the motion sensor and/or the other motion data of the other motion sensor that are determined to correspond to the detected indications that the user is inputting the touch or tap command. ### Claim 36 (depends on 35) The earphone system of claim 35, wherein the one or more processors are configured to determine whether the touch or tap is a command to control, with respect to the user sensory output, a pausing or resuming of audio output, to control a change in a song, genre, or playlist in the audio output, or to control an activation of a voice control. ### Claim 37 (depends on 1) The earphone system of claim 1, further comprising a remote control unit, separate from respective one or more earpieces of the earphone system, including the one or more processors and a user control interface for user selective further control of the user sensory output, wherein the respective one or more earpieces and the remote control unit are wiredly connected. ### Claim 38 (depends on 37) The earphone system of claim 37, wherein the remote control unit further includes a power harvesting component to perform a trickle energy capture, to power the motion sensor and the bio-sensor, using a wired connection between the earphone system and a media device providing media to the earphone system, the one or more processors are further configured to selectively control respective periods of time the motion sensor and the bio-sensor are controlled to respectively measure motion and bio-signals, and the power harvesting component selectively operates between the trickle energy capturing, with a storage of captured energy in a local power storage of any of the respective one or more earpieces and the remote control unit, during power off times of the bio-sensor of the respective periods of time and a supplying of the stored captured energy from the local power storage to the bio-sensor during power on times of the bio-sensor of the respective periods of time. ### Claim 39 (depends on 1) The earphone system of claim 1, wherein the controlling of the user sensory output includes a controlling, based on the determined activity state of the user, of audio output, including one of stopping, pausing, playing, increasing volume, decreasing volume, matching a beat per minute, a cadence, rhythm, predetermined user selection, genre, and select provision of a predefined user playlist or specific track selection. ### Claim 40 (depends on 1) The earphone system of claim 1, wherein the one or more processors are further configured to obtain cadence sensor data, and to determine the activity state further based on the obtained cadence sensor data. ### Claim 41 (depends on 1) The earphone system of claim 1, wherein, for the determining of the activity state of the user, the one or more processors are configured to determine the activity state of the user based on one or more user set activity states of the user with respect to user select bio-signal data and motion data. ### Claim 42 (depends on 1) The earphone system of claim 1, wherein the one or more processors are further configured to determine the activity state from among plural user preset activity states set through a user interface of the earphone system. ### Claim 43 An earphone method of an earphone system having earpieces configured for respective arrangements relative to a user's ears for listening to audio, the method comprising: controlling respective measuring of bio-signals and motion changes of a user by the earpieces; determining a learned activity state of the user based on a combined consideration of obtained bio-signal data from the bio-sensor and obtained motion data from the motion sensor; and controlling a provision of user sensory output to the user based on the determined activity state.