# US 2018/0025248 A1 · Handwriting recognition method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025248 A1
- Document ID: US20180025248A1
- Status: Published application
- Kind: A1
- Application number: 15/550,974
- Filed: 2016-02-11
- Published: 2018-01-25
- Earliest claimed: 2015-02-12
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Yuxian SHAN; Chuanxia LIU; Youxin CHEN
- Web page: https://patentgrep.com/patent/US20180025248A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/03545: {Pens or stylus}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06V 10/235: {based on user input or interaction}
- G06V 30/1423: {the instrument generating sequences of position coordinates corresponding to handwriting (preprocessing or recognising digital ink G06V30/32)}
- G06V 30/347: {Sampling; Contour coding; Stroke extraction}
- G06V 30/373: {using a special pattern or subpattern alphabet}
- G06V 20/20: in augmented reality scenes
- G06V 2201/01: Solutions for problems related to non-uniform document background
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of generating handwriting information about handwriting of a user includes determining a first writing focus and a second writing focus; sequentially shooting a first local writing area, which is within a predetermined range from a first writing focus, and a second local writing area, which is within a predetermined range from a second writing focus; obtaining first handwriting from the first local writing area and second handwriting from the second local writing area; combining the first handwriting with the second handwriting; and generating the handwriting information, based on a result of the combining.

## Claims

### Claim 1

A device for generating handwriting information about handwriting of a user, the device comprising:
a user interface configured to receive a user input;
a signal obtainer configured to sequentially shoot a first local writing area, which is within a predetermined range from a first writing focus, and a second local writing area, which is within a predetermined range from a second writing focus; and
a controller configured to determine the first writing focus and the second writing focus, obtain first handwriting from the first local writing area, obtain second handwriting from the second local writing area, combine the first handwriting with the second handwriting, and generate handwriting information, based on a result of the combining.

### Claim 2 (depends on 1)

The device of claim 1, wherein
the signal obtainer traces a visual focus of the user, and
the controller determines the first writing focus and the second writing focus, based on the traced visual focus.

### Claim 3 (depends on 1)

The device of claim 1, wherein
the user interface receives information about a color of handwriting from the user, and
the controller obtains the first handwriting from the first local writing area and the second handwriting from the second local writing area, based on information about the color of the handwriting.

### Claim 4 (depends on 1)

The device of claim 1, wherein the controller compares the first handwriting with the second handwriting, determines changed handwriting between a captured image frame of the first local writing area and a captured image frame of the second local writing area, based on a result of the comparing, and combines the changed handwriting, based on a result of the determining.

### Claim 5 (depends on 1)

The device of claim 1, wherein
the handwriting information comprises at least one of time sequence information of local handwriting and layout information,
the handwriting information is stored in at least one format from among a document format, an image format, and a video format, and
the layout information comprises line break information and identifier information of at least one of a deleting identifier, a replacement identifier, a repeat characteristic identifier, an insertion identifier, a position adjusting identifier, and an emphasis identifier.

### Claim 6 (depends on 1)

The device of claim 1, wherein
the signal obtainer obtains audio data generated while the first local writing area and the second local writing area are being sequentially shot, and
the controller generates the handwriting information such that the obtained audio data corresponds to the first handwriting and the second handwriting.

### Claim 7 (depends on 6)

The device of claim 6, wherein the controller generates the handwriting information such that audio data received between a shooting time of the first local area and a shooting time of the second local area corresponds to changed handwriting between a captured image frame of the first local writing area and a captured image frame of the second local writing area.

### Claim 8 (depends on 4)

The device of claim 4, wherein the controller controls the signal obtainer to determine whether a pause signal is detected, stop the handwriting obtaining and the handwriting combining, based on a result of the determining, and perform monitoring.

### Claim 9 (depends on 8)

The device of claim 8, wherein the controller determines whether a pause signal is received, based on whether the changed handwriting between the captured image frame of the first local writing area and the captured image frame of the second local writing area or the tip of the writing tool is detected, whether a changing angle of the head posture of the user exceeds a predetermined value, whether a distance between the visual focus of the user and the writing focus exceeds a predetermined value, and whether a pause instruction was received.

### Claim 10 (depends on 1)

The device of claim 1, wherein
the controller applies an enhancement process to the obtained first handwriting and the obtained second handwriting,
the enhancement process is applicable when light brightness or definition during shooting does not meet a pre-defined requirement, when a thickness of the obtained first or second handwriting is smaller than a pre-defined value, or when a difference between a color of handwriting in a captured image and a background color is smaller than a predefined value, and
the enhancement process removes a stroke whose confidence evaluation value is lower than a threshold from among at least one stroke included in the first handwriting and the second handwriting.

### Claim 11

A method of generating handwriting information about handwriting of a user, the method comprising:
determining a first writing focus and a second writing focus;
sequentially shooting a first local writing area, which is within a predetermined range from a first writing focus, and a second local writing area, which is within a predetermined range from a second writing focus;
obtaining first handwriting from the first local writing area and second handwriting from the second local writing area;
combining the first handwriting with the second handwriting; and
generating the handwriting information, based on a result of the combining.

### Claim 12 (depends on 11)

The method of claim 11, wherein the determining of the first writing focus and the second writing focus comprises:
tracing a tip of a writing tool;
determining a position of the tip of the writing tool; and
determining the first writing focus and the second writing focus, based on the determined position of the tip of the writing tool.

### Claim 13 (depends on 11)

The method of claim 11, wherein the obtaining of the first handwriting from the first local writing area and the second handwriting from the second local writing area comprises obtaining the first handwriting from the first local writing area and the second handwriting from the second local writing area, based on a difference between a handwriting color and a color of a writing background.

### Claim 14 (depends on 11)

The method of claim 11, further comprising:
determining whether an ending signal is detected; and
ending the handwriting obtaining, the handwriting combining, and the generating of the handwriting information, based on a result of the determining.

### Claim 15 (depends on 11)

A computer-readable recording medium having recorded thereon a program, which, when executed by a computer, performs the method of claim 11.