# US 2016/0253090 A1 · Method and system for ink data generation, ink data rendering, ink data manipulation and ink data communication

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0253090 A1
- Document ID: US20160253090A1
- Status: Published application
- Kind: A1
- Application number: 15/099,311
- Filed: 2016-04-14
- Published: 2016-09-01
- Earliest claimed: 2013-11-19
- Assignees: Wacom Co., Ltd.
- Applicants: Wacom Co., Ltd.
- Inventors: Branimir Angelov; Yoshitaka Nakayama
- Web page: https://patentgrep.com/patent/US20160253090A1

## CPC

- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/0354: with detection of two-dimensional [2D] relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F 3/03545: {Pens or stylus}
- G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06F 3/0412: {Digitisers structurally integrated in a display}
- G06F 3/0414: {using force sensing means to determine a position}
- G06F 3/044: by capacitive means
- G06F 3/0447: {Position sensing using the local deformation of sensor cells}
- G06Q 10/101: Collaborative creation, e.g. joint development of products or services
- G06T 11/23: using straight lines or curves
- G06V 30/1423: {the instrument generating sequences of position coordinates corresponding to handwriting (preprocessing or recognising digital ink G06V30/32)}
- G06V 30/347: {Sampling; Contour coding; Stroke extraction}
- G06F 2203/0383: Remote input, i.e. interface arrangements in which the signals generated by a pointing device are transmitted to a PC at a remote location, e.g. to a PC in a LAN
- G06F 2203/04105: Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
- G06F 2203/04106: Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method implemented by a transmission device to communicate with multiple reception devices that respectively share a drawing area with the transmission device is provided. The transmission device transmits to the multiple reception devices vector-data ink data representative of traces of input operation detected by an input sensor of the transmission device. The method includes: (a) an ink data generation step of generating fragmented data of a stroke object, wherein the stroke object contains multiple point objects to represent a trace formed by a pointer, the fragmented data being generated per defined unit T, and generating a drawing style object; (b) a message formation step of generating messages including the drawing style object and the fragmented data; and (c) a transmission step of transmitting the messages.

## Claims

### Claim 1

A method implemented by a transmission device to communicate with multiple reception devices that respectively share a drawing area with the transmission device, wherein the transmission device transmits to the multiple reception devices vector-data ink data representative of traces of input operation detected by an input sensor of the transmission device, the method comprising:
(a) an ink data generation step that comprises:
(i) continuously inputting pen even data (INPUT 1) generated according to movement of a pointer, and generating fragmented data of a stroke object, wherein the stroke object contains multiple point objects to represent a trace of said movement of the pointer, the fragmented data being generated per defined unit T, and
(ii) generating a drawing style object, which defines a rendering configuration to be applied to the whole of said trace of the stroke object based on context information (INPUT 2), at a pen down time corresponding to generation of the pen event data at a beginning point of said trace;
(b) a message formation step that comprises:
(i) generating a first message that includes the drawing style object, and
(ii) generating one or more second messages subsequent to the first message, the one or more second messages including the fragmented data to be rendered with the rendering configuration defined by the drawing style object in the first message; and
(c) a transmission step of transmitting the first message including the drawing style object and the one or more second messages in sequence according to a defined communications protocol, wherein
the transmission step starts transmitting the first message at the pen down time.

### Claim 2 (depends on 1)

The method of claim 1, wherein the defined unit T is selected from a group consisting of a unit time and a unit data size.

### Claim 3 (depends on 1)

The method of claim 1, wherein said rendering configuration of the trace of the movement of the pointer is selected from a group consisting of a color of the trace, opacity of the trace, transparency of the trace, width of the trace, and texture of the trace.

### Claim 4 (depends on 3)

The method of claim 3, wherein in step (b)(i) the first message includes the drawing style object and at least a first fragmented data, and in step (b)(ii) the second message includes the fragmented data generated subsequent to the at least first fragmented data included in the first message and does not include the drawing style object.

### Claim 5 (depends on 1)

The method of claim 1, wherein step (a)(ii) of generating the drawing style object is triggered by a pen down event which comprises inputting of the pen event data at the beginning point of said trace.

### Claim 6 (depends on 1)

The method of claim 1, wherein the ink data generation step of (a) further comprises (iii) generating a metadata object at a pen up time corresponding to generation of the pen event data at an ending point of said trace, and the message formation step of (b) further comprises (iii) including the metadata object in a last one of the one or more second messages.

### Claim 7 (depends on 6)

The method of claim 6, wherein the first message including the drawing style object is configured to request updating of the drawing area.

### Claim 8 (depends on 7)

The method of claim 7, wherein the message formation step of (b) further comprises (iv) when receiving a message indicating that said request for updating of the drawing area was rejected, aborting transmission of the one or more second messages.

### Claim 9 (depends on 1)

The method of claim 1, wherein the defined unit T is smaller than a unit corresponding to said trace and is greater than a minimum unit that can be processed to be drawn a curve segment by interpolating point objects included in the fragmented data.

### Claim 10 (depends on 1)

The method of claim 1, further comprising (d) a negotiation step, to be performed prior to the message formation step of (b), the negotiation step of (d) comprising determining a communications parameter value, which becomes a basis for determining said defined unit T of the fragmented data.

### Claim 11 (depends on 10)

The method of claim 10, wherein said communications parameter value determines whether said defined unit T of the fragmented data equals, or is smaller than, a unit corresponding to said trace.

### Claim 12 (depends on 11)

The method of claim 11, wherein the message formation step of (b) comprises forming the first and second messages using a first format when said communications parameter value determines said defined unit T of the fragmented data to be smaller than the unit corresponding to said trace, and forming the first and second messages using a second format different from the first format when said communications parameter value determines said defined unit T of the fragmented data to be equal to the unit corresponding to said trace.

### Claim 13 (depends on 12)

The method of claim 12, wherein said first format includes a field that indicates whether a message includes an ending point of said trace represented by said stroke object, while said second format does not include such field.

### Claim 14 (depends on 12)

The method of claim 12, wherein said defined unit T that is smaller than the unit corresponding to said trace is selected from a group consisting of: (a) a unit based on one of the multiple point objects that form the stroke object, (b) a unit based on a byte that is smaller than a minimum unit defined by the communications protocol, and (c) a unit of time equal to or less than 100 msec.

### Claim 15 (depends on 10)

The method of claim 10, further comprising (e) a negotiation step, to be performed prior to the message formation step of (b), the negotiation step of (e) comprising determining a communications parameter value, and the transmission step of (c) comprises transmitting the first message and the one or more second messages using a communication packet of the defined communications protocol specified by said communications parameter value.

### Claim 16 (depends on 15)

The method of claim 15, wherein the communications parameter value specifies one of a first protocol, which includes a function of detecting loss of the communication packet, and a second protocol, which does not include the function of detecting loss of the communication packet, and the message formation step of (b) comprises switching between generating the first and second messages using the first protocol and generating the first and second messages using the second protocol, based on said communications parameter value.

### Claim 17 (depends on 16)

The method of claim 16, wherein the message formation step of (b) comprises, when the second protocol is used, generating each of the first and second messages that includes a sequence ID to define an order of the first and second messages, and when the first protocol is used, generating each of the first and second messages without including a sequence ID.

### Claim 18 (depends on 17)

The method of claim 17, further comprising (g) a message retransmission step of retransmitting a message whose sequence ID is indicated as being lost and its subsequent messages.

### Claim 19 (depends on 18)

The method of claim 18, further comprising (h) a drawing area sharing step, to be performed prior to the message formation step of (b), the drawing area sharing step comprising transmitting latest information regarding the drawing area.

### Claim 20 (depends on 19)

The method of claim 19, wherein the drawing area sharing step of (h) comprises generating the latest information regarding the drawing area using a first encoding method, and the message formation step of (b) comprises generating the first and second messages using a second encoding method having a smaller compression rate than the first encoding method.

### Claim 21 (depends on 20)

The method of claim 20, wherein the message formation step of (b) further comprises determining a stroke ID that uniquely identifies the stroke object that is being generated amongst multiple stroke objects present within the drawing area, and including said stroke ID in at least the first message.

### Claim 22 (depends on 21)

The method of claim 21, wherein the message formation step of (b) further comprises including a device ID of the transmission device in at least the first message.

### Claim 23 (depends on 19)

The method of claim 19, wherein the message formation step of (b) further comprises including reproduction timing information, which indicates a time interval at which the fragmented data of the stroke object are to be reproduced by the reception device, in said stroke object.

### Claim 24 (depends on 19)

The method of claim 19, wherein the ink data generation step of (a) comprises generating, based on said context information at the beginning point of said trace, variability information that indicates whether at least one of a width and a color of said trace varies according to said movement of the pointer, and the message formation step of (b) comprises including said variability information in the first message or a first of the one or more second messages.

### Claim 25 (depends on 24)

The method of claim 24, wherein said width extends perpendicularly to a direction of said trace, and said variability information is a parameter that defines a range of the width in terms of at least one of a minimum width value and a maximum width value.

### Claim 26 (depends on 19)

The method of claim 19, further comprising (i) a sharing step, to be performed prior to the ink data generation step of (a), the sharing step of (i) comprising transmitting a set of drawing parameters of said drawing style object to the multiple reception devices, and the ink data generation step of (a) comprises generating the drawing style object using an identifier of the set of the drawing parameters.

### Claim 27 (depends on 26)

The method of claim 26, wherein the set of drawing parameters defines a set of pen-tip attributes applicable in said drawing area.

### Claim 28 (depends on 27)

The method of claim 27, wherein the set of drawing parameters includes drawing resolution information that specifies accuracy of drawing coordinate positions, and the step of ink data generation of (a) comprises generating drawing coordinate positions for the point objects to represent said trace using said drawing resolution information.

### Claim 29 (depends on 28)

The method of claim 28, wherein the step of ink data generation of (a) comprises scaling a vector from a first pen coordinate position of the pen event data corresponding to said beginning point of said trace to a second pen coordinate position of the pen event data corresponding to an ending point of said trace, using said drawing resolution information.

### Claim 30 (depends on 29)

The method of claim 29, wherein said sharing step of (i) comprises transmitting a drawing coordinate position of an origin of the drawing area, and said ink data generation step of (a) comprises encoding a drawing coordinate position corresponding to said beginning point of said trace relative to said origin.

### Claim 31 (depends on 30)

The method of claim 30, wherein the sharing step of (i) comprises transmitting multiple sets of drawing parameters of said drawing style object to the multiple reception devices, and subsequently transmitting an identifier of a selected one of the multiple sets of drawing parameters to the multiple reception devices.

### Claim 32 (depends on 1)

The method of claim 1, wherein the defined unit T is defined by an application that uses the ink data.

### Claim 33 (depends on 32)

The method of claim 32, wherein when the application uses audio in synchronization with the ink data, the defined unit is smaller than a unit corresponding to said trace when the ink data is transmitted with audio.

### Claim 34 (depends on 32)

The method of claim 32, wherein when the application is used with a recognition engine that forms the ink data in a unit based on semantics, the defined unit T is greater than a unit corresponding to said trace.

### Claim 35 (depends on 32)

The method of claim 32, further comprising the step of varying the frequency with which to include absolute coordinates in the point objects, wherein the point objects normally include relative coordinates, according to a type of protocol to be used for transmission.

### Claim 36 (depends on 32)

The method of claim 32, further comprising the step of including an absolute value in a beginning point object of each of the one or more second messages and including an absolute value in a non-beginning point object of each of the one or more second messages.
