# US 2021/0240286 A1 · Stylus pen, terminal device set and controlling method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240286 A1 - Document ID: US20210240286A1 - Status: Published application - Kind: A1 - Application number: 17/064,835 - Filed: 2020-10-07 - Published: 2021-08-05 - Earliest claimed: 2020-01-30 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Seongho BYEON; Pilseob KANG; Daesung AN; Dongeon KIM; Hyunjun KIM; Chanhee LEE; Sungho LEE - Web page: https://patentgrep.com/patent/US20210240286A1 ## CPC - G06F 3/03545: {Pens or stylus} - G06F 3/0383: {Signal control means within the pointing device} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - H04R 1/028: {associated with devices performing functions other than acoustics, e.g. electric candles} - H04R 1/083: {Special constructions of mouthpieces} - G06F 2200/1632: Pen holder integrated in the computer - G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer - H04R 2420/07: Applications of wireless loudspeakers or wireless microphones - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract A stylus pen and a controlling method thereof are provided. A stylus pen housed in and withdrawn from a terminal device includes a user inputter configured to receive a contact command of a user, a microphone configured to receive a voice command of a user, a communicator configured to transmit the received voice command to the terminal device, and a processor configured to control the user inputter, the microphone, and the communicator, and the processor may, based on the stylus pen being withdrawn from the terminal device, and a first contact command being received through the user inputter, activate the microphone which is in a deactivated state to convert the stylus pen to a main voice input device. ## Claims ### Claim 1 A stylus pen housed in and withdrawn from a terminal device, comprising: a user inputter configured to receive a contact command of a user; a microphone configured to receive a voice command of a user; a communicator configured to transmit the received voice command to the terminal device; and a processor configured to control the user inputter, the microphone, and the communicator, wherein the processor is further configured to: based on the stylus pen being withdrawn from the terminal device, and a first contact command being received through the user inputter, activate the microphone which is in a deactivated state to convert the stylus pen to a main voice input device. ### Claim 2 (depends on 1) The stylus pen of claim 1, wherein the processor is further configured to, based on receiving a second contact command through the user inputter, control the communicator to transmit application execution control data for executing an application of the terminal device to the terminal device. ### Claim 3 (depends on 2) The stylus pen of claim 2, wherein the second contact command is a command that is input for greater than or equal to a predetermined first time, and wherein the processor is further configured to: based on the second contact command being input for greater than or equal to a predetermined first time, control the communicator to transmit first application execution control data for executing a first application of the terminal device to the terminal device, and based on the second contact command being input for greater than or equal to a second time that is longer than the predetermined first time, control the communicator to transmit the second application execution control data for executing a second application of the terminal device to the terminal device. ### Claim 4 (depends on 2) The stylus pen of claim 2, wherein the user inputter comprises: a first button receiving a first contact command; and a second button configured to be housed in or withdrawn from the stylus pen, wherein the processor is further configured to: based on the second button being inserted in the stylus pen, control the communicator to transmit, to the terminal device, first application execution control data for executing a first application of the terminal device, and based on the second button being withdrawn to an outside of the stylus pen, control the communicator to transmit, to the terminal device, second application execution control data for executing a second application of the terminal device. ### Claim 5 (depends on 2) The stylus pen of claim 2, wherein the processor is further configured to: based on an application being executed in the terminal device being a voice assistant service application, control the communication to transmit, to the terminal device, control data relative to the voice assistant application based on the input voice command of a user, and based on the application being executed in the terminal device being a camera application, control the communicator to transmit, to the terminal device, control data relative to the camera application based on the input voice command of the user. ### Claim 6 (depends on 2) The stylus pen of claim 2, wherein the processor is further configured to, based on receiving the second contact command again through the user inputter, control the communicator to transmit, to the terminal device, application termination data for terminating the application of the terminal device. ### Claim 7 (depends on 1) The stylus pen of claim 1, further comprising: an outputter configured to output at least one of status information, contact command information, or the voice command information of the stylus pen. ### Claim 8 (depends on 7) The stylus pen of claim 7, wherein the outputter comprises at least one of a speaker, a light emitting diode (LED), a display, or a motor. ### Claim 9 (depends on 1) The stylus pen of claim 1, wherein the processor is further configured to, based on receiving the first contact command again through the user inputter, deactivate the activated microphone. ### Claim 10 A method of controlling a stylus pen housed in and withdrawn from a terminal device, the method comprising: after the stylus pen is withdrawn from the terminal device, receiving a first contact command through a user inputter; and converting the stylus pen to a main voice input device by activating a microphone in a deactivated state based on the received first contact command. ### Claim 11 (depends on 10) The method of claim 10, further comprising: receiving a second contact command through the user inputter; and transmitting, to the terminal device, application execution control data for executing an application of the terminal device based on the input second contact command. ### Claim 12 (depends on 11) The method of claim 11, wherein the second contact command is a command that is input for greater than or equal to a predetermined first time, and wherein the transmitting the application execution control data to the terminal device comprises: transmitting first application execution control data for executing a first application of the terminal device to the terminal device, and based on the second contact command being input for greater than or equal to a second time that is longer than the predetermined first time, transmitting the second application execution control data for executing a second application of the terminal device to the terminal device. ### Claim 13 (depends on 11) The method of claim 11, wherein the transmitting the application execution control data to the terminal device comprises: based on the second button included in the user inputter being inserted in the stylus pen, transmitting, to the terminal device, first application execution control data for executing a first application of the terminal device, and based on the second button being withdrawn to an outside of the stylus pen, transmitting, to the terminal device, second application execution control data for executing a second application of the terminal device. ### Claim 14 (depends on 11) The method of claim 11, further comprising: transmitting application control data for controlling an application of the terminal device to the terminal device, wherein the transmitting the application control data to the terminal device comprises: based on the application being executed in the terminal device being a voice assistant service application, transmitting, to the terminal device, control data relative to the voice assistant application based on the input voice command of a user, and based on the application being executed in the terminal device being a camera application, transmitting, to the terminal device, control data relative to the camera application based on the received voice command of the user. ### Claim 15 (depends on 11) The method of claim 11, further comprising: receiving the second contact command again through the user inputter; and transmitting, to the terminal device, application termination data for terminating the application of the terminal device based on the second contact command input again. ### Claim 16 (depends on 10) The method of claim 10, further comprising: outputting at least one of status information, contact command information, or the voice command information of the stylus pen. ### Claim 17 (depends on 10) The method of claim 10, further comprising: receiving the first contact command again through the user inputter; and deactivating the activated microphone based on the first contact command input again. ### Claim 18 A terminal device set comprising: a stylus pen housed in and withdrawn from a main terminal device, configured to receive a contact command of a user and transmit the received voice command to the main terminal device; and a main terminal device configured to perform an operation corresponding to the received voice command, wherein the stylus pen is further configured to: based on being withdrawn from the main terminal device and receiving a first contact command from the user, activate a microphone in a deactivated state to convert to a main voice input device, and based on receiving a second contact command, transmit application execution control data for executing an application of the main terminal device to the main terminal device, and wherein the main terminal device is further configured to: execute an application corresponding to the received application execution control data, and output a command set used in the executed application. ### Claim 19 (depends on 18) The terminal device set of claim 18, wherein the main terminal device is further configured to: calculate a distance with the stylus pen based on at least one of a data transmission time received from the stylus pen, response time, a time difference between a time at which a same voice of a user is input to the main terminal device and a time at which the same voice of the user is input to the stylus pen, respectively, and control output based on the calculated distance. ### Claim 20 (depends on 19) The terminal device set of claim 19, wherein the main terminal device is further configured to: based on the output being an output to display a command set, control a size of display of the command set displayed in accordance with the calculated distance, and based on the output being an audio output, control a magnitude of the audio volume in accordance with the calculated distance.