# US 10,310,638 B2 · Mobile terminal displaying writing information generated by pen device based on voice received via microphone and control method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,310,638 B2 - Document ID: US10310638B2 - Status: Granted - Kind: B2 - Application number: 15/451,329 - Filed: 2017-03-06 - Published: 2018-02-22 - Granted: 2019-06-04 - Earliest claimed: 2016-08-22 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Eunhye Kim; Taehoon Cho; Minkyoung Shin - Web page: https://patentgrep.com/patent/US10310638B2 ## CPC - G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry - G06F 3/03545: {Pens or stylus} - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 2203/04105: Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position - G06F 2203/04106: Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection - G06F 2203/04108: Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction - G06F 2203/04807: Pen manipulated menu - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A mobile terminal includes: a display; a communication module configured to perform communication with a pen device having a first end and a second end; and a controller configured to: control the display and the communication module; recognize that the pen device operates in a first mode when the first end of the pen device is not in contact with the display for a predetermined time; and cause the display to display writing information when the pen device operates in the first mode, wherein the writing information is generated by the pen device on a surface other than a surface of the display. ## Claims ### Claim 1 A mobile terminal comprising: a microphone; a display; a communication module configured to perform communication with a pen device having a first end and a second end; and a controller configured to: control the microphone, the display, and the communication module; cause the display to display a window including specific text; recognize that the pen device operates in a first mode when the first end of the pen device is not in contact with the display for a predetermined time; recognize a specific voice received via the microphone while the window including the specific text is displayed in the first mode, the specific voice corresponding to the specific text; and cause the display to display writing information at a position corresponding to the specific text in the window in response to the recognized specific voice when the writing information is generated by the pen device on a surface other than a surface of the display in the first mode. ### Claim 2 (depends on 1) The mobile terminal of claim 1, wherein, while the pen device operates in the first mode, the controller is further configured to: cause the display to display a black screen; and cause the display to display the writing information on the black screen. ### Claim 3 (depends on 1) The mobile terminal of claim 1, wherein the controller is further configured to cause the display to display the writing information in a post-it form when the first end is in contact with the display such that the writing information is displayed in at least one post-it area of the display. ### Claim 4 (depends on 1) The mobile terminal of claim 1, wherein the surface comprises a display area of another mobile terminal or any type of surface. ### Claim 5 (depends on 1) The mobile terminal of claim 1, wherein the controller is further configured to: determine an angle between the pen device and the mobile terminal set by the pen device; and detect a memorization write mode and an eraser mode of the pen device, which are included in the first mode, based on the determined angle. ### Claim 6 (depends on 5) The mobile terminal of claim 5, wherein: the memorization write mode is a mode in which a portion of the displayed writing information is covered by the second end of the pen device when the angle is within a first angle range; and the eraser mode is a mode in which the displayed writing information is erased by the second end of the pen device when the angle is within a second angle range. ### Claim 7 (depends on 6) The mobile terminal of claim 6, wherein the controller is further configured to: recognize a touch applied by the second end of the pen device at a portion of the writing information; and cause the display to display a black box at the touched portion in response to the touch. ### Claim 8 (depends on 7) The mobile terminal of claim 7, wherein the controller is further configured to delete the black box when a writing generated by the second end corresponds to the covered portion after the black box is touched by the second end of the pen device for a predetermined time. ### Claim 9 (depends on 1) The mobile terminal of claim 1, wherein the controller is further configured to change an attribute of a writing window displayed on the display based on a user pattern by a user of the pen device. ### Claim 10 (depends on 1) The mobile terminal of claim 1, wherein: an operation mode of the pen device includes the first mode and a second mode; the first mode is a mode for performing an operation except for a writing function of the pen device; and the second mode is a mode in which a writing is input via the display by the first end of the pen device. ### Claim 11 A method for operating a mobile terminal, the method comprising: displaying a window including specific text on a display of the mobile terminal; recognizing that a pen device operates in a first mode when a first end of the pen device is not in contact with the display for a predetermined time; recognizing a specific voice received via a microphone while the window including the specific text is displayed in the first mode, the specific voice corresponding to the specific text; and displaying writing information at a position corresponding to the specific text in the window in response to the recognized specific voice when the writing information is generated by the pen device on a surface other than a surface of the display in the first mode. ### Claim 12 (depends on 11) The method of claim 11, wherein when the pen device operates in the first mode, the method further comprises: changing a screen of the display to a black screen; and displaying the writing information on the black screen. ### Claim 13 (depends on 11) The method of claim 11, further comprising displaying the writing information in a post-it form when the first end is in contact with the display such that the writing information is displayed in at least one post-it area of the display. ### Claim 14 (depends on 11) The method of claim 11, wherein the surface comprises a display area of another mobile terminal or any type of surface. ### Claim 15 (depends on 11) The method of claim 11, further comprising: determining an angle between the pen device and the mobile terminal set by the pen device; and detecting a memorization write mode and an eraser mode of the pen device, which are included in the first mode, based on the determined angle. ### Claim 16 (depends on 15) The method of claim 15, wherein: the memorization write mode is a mode in which a portion of the displayed writing information is covered by a second end of the pen device when the angle is within a first angle range; and the eraser mode is a mode in which the displayed writing information is erased by the second end of the pen device when the angle is within a second angle range. ### Claim 17 (depends on 16) The method of claim 16, further comprising: recognizing a touch applied by the second end of the pen device at a portion of the writing information; and displaying a black box at the touched portion in response to the touch. ### Claim 18 (depends on 17) The method of claim 17, further comprising deleting the black box when a writing generated by the second end corresponds to the covered portion after the black box is touched by the second end of the pen device for a predetermined time. ### Claim 19 (depends on 11) The method of claim 11, further comprising changing an attribute of a writing window displayed on the display based on a user pattern by a user of the pen device. ### Claim 20 (depends on 11) The method of claim 11, wherein: an operation mode of the pen device includes the first mode and a second mode; the first mode is a mode for performing an operation except for a writing function of the pen device; and the second mode is a mode in which a writing is input via the display by the first end of the pen device.