# US 2019/0057266 A1 · Stroke extraction in free space

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0057266 A1
- Document ID: US20190057266A1
- Status: Published application
- Kind: A1
- Application number: 16/166,202
- Filed: 2018-10-22
- Published: 2019-02-21
- Earliest claimed: 2015-11-30
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Eli M. Dow; Thomas D. Fitzsimmons; Tynan J. Garrett; Emily M. Metruck
- Web page: https://patentgrep.com/patent/US20190057266A1

## CPC

- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 40/126: Character encoding
- G06V 30/2276: {with probabilistic networks, e.g. hidden Markov models}
- G06V 30/36: {Matching; Classification}
- G06V 30/10: Character recognition
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An approach for extracting strokes in a free space environment is described. Boundaries are displayed in a free space environment describing at least one two-dimensional surface area. One or more language movements are extracted from the free space environment by a paired ring device and transmitted as images for processing. Haptic feedback is provided to the paired ring device in response to detecting at least one language movement occurring outside of at least one two-dimensional surface area. At least one extracted language movement is input into a character training model.

## Claims

### Claim 1

A method for stroke extraction in free space utilizing a paired ring device and a paired display device connected to the paired display device, the method comprising:
displaying, by one or more computer processors, on the paired display device, a modified free space environment including one or more bounded surface areas, with the bounded surface areas including an area of two-dimensional space framed by a boundary within the free space environment;
receiving, by the one or more computer processors, one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from one or more language movements recorded by the paired ring device utilizing a magnitude value and a direction value of proper acceleration of the one or more language movements, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines;
determining, by the one or more processors, at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas;
responsive to determining that at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas, providing, by the one or more processors, haptic feedback to the paired ring device; and
inputting, by the one or more computer processors, the one or more images into a character training model.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
mapping, by the one or more computer processors, the one or more images into one or more characters in the character training model, wherein mapping includes determining whether the one or more images indicate at least one of the one or more characters in the character training model utilizing a pre-determined range of motion of the paired ring device during a time period of movement in free space, where the pre-determined range of motion creates a specific image.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
responsive to a determination that the one or more images indicate at least one of the one or more characters in the character training model, transcribing, by the one or more computer processors, the at least one of the one or more characters into a digital document, wherein the digital document is a visual augmentation for displaying in a heads up display of the paired display device.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
determining, by the one or more computer processors, a representation of the at least one of the one or more characters as a written language or a gesture output derived from a series of strokes in free space; and
encoding, by the one or more computer processors, the at least one of the one or more characters into a visual augmentation.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
the paired ring device includes an accelerometer and a gyroscope for monitoring the one or more language movements; and
the one or more language movements include one or more of the following: (i) a change in a magnitude value and a direction value of proper acceleration of the paired ring device, (ii) an orientation of the paired ring device, (iii) a position of the paired ring device, (iv) a location of the paired ring device, (v) a change in the orientation of the paired ring device, (vi) a change in the position of the paired ring device, and (vii) a change in the location of the paired ring device in the modified free space environment.

### Claim 6 (depends on 1)

The method of claim 1, wherein the paired ring device is sized to be worn on a human finger digit.

### Claim 7

A computer program product (CPP) for stroke extraction in free space, the CPP comprising:
a paired ring device;
a paired display device connected to the paired display device;
a computer readable storage medium; and
program instructions stored on the computer readable storage medium, with the program instructions including instructions for causing one or more computer processors to perform operations including the following:
displaying, by one or more computer processors, on the paired display device, a modified free space environment including one or more bounded surface areas, with the bounded surface areas including an area of two-dimensional space framed by a boundary within the free space environment,
receiving, by the one or more computer processors, one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from one or more language movements recorded by the paired ring device utilizing a magnitude value and a direction value of proper acceleration of the one or more language movements, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines, determining, by the one or more processors, at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas,
responsive to determining that at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas, providing, by the one or more processors, haptic feedback to the paired ring device, and
inputting, by the one or more computer processors, the one or more images into a character training model.

### Claim 8 (depends on 7)

The CPP of claim 7, wherein the program instructions further includes instructions for causing the one or more processors to perform the following operations:
mapping, by the one or more computer processors, the one or more images into one or more characters in the character training model, wherein mapping includes determining whether the one or more images indicate at least one of the one or more characters in the character training model utilizing a pre-determined range of motion of the paired ring device during a time period of movement in free space, where the pre-determined range of motion creates a specific image.

### Claim 9 (depends on 8)

The CPP of claim 8, wherein the program instructions further includes instructions for causing the one or more processors to perform the following operations:
responsive to a determination that the one or more images indicate at least one of the one or more characters in the character training model, transcribing, by the one or more computer processors, the at least one of the one or more characters into a digital document, wherein the digital document is a visual augmentation for displaying in a heads up display of the paired display device.

### Claim 10 (depends on 9)

The CPP of claim 9, wherein the program instructions further includes instructions for causing the one or more processors to perform the following operations:
determining, by the one or more computer processors, a representation of the at least one or more characters as a written language or a gesture output derived from a series of strokes in free space; and
encoding, by the one or more computer processors, the at least one or more characters into a visual augmentation.

### Claim 11 (depends on 7)

The CPP of claim 7, wherein:
the paired ring device includes an accelerometer and a gyroscope for monitoring the one or more language movements; and
the one or more language movements include one or more of the following: (i) a change in a magnitude value and a direction value of proper acceleration of the paired ring device, (ii) an orientation of the paired ring device, (iii) a position of the paired ring device, (iv) a location of the paired ring device, (v) a change in the orientation of the paired ring device, (vi) a change in the position of the paired ring device, and (vii) a change in the location of the paired ring device in the modified free space environment.

### Claim 12 (depends on 7)

The CPP of claim 7, wherein the paired ring device is sized to be worn on a human finger digit.

### Claim 13

A computer system (CS) for stroke extraction in free space, the CS comprising:
a paired ring device;
a paired display device connected to the paired display device;
one or more processors;
a computer readable storage medium; and
program instructions stored on the computer readable storage medium, with the program instructions including instructions for causing the processor(s) set to perform operations including the following:
displaying, by one or more computer processors, on the paired display device, a modified free space environment including one or more bounded surface areas, with the bounded surface areas including an area of two-dimensional space framed by a boundary within the free space environment,
receiving, by the one or more computer processors, one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from one or more language movements recorded by the paired ring device utilizing a magnitude value and a direction value of proper acceleration of the one or more language movements, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines,
determining, by the one or more processors, at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas,
responsive to determining that at least one of the one or more language movements crossed at least one side of at least one bounded surface area of the one or more bounded surface areas, providing, by the one or more processors, haptic feedback to the paired ring device, and
inputting, by the one or more computer processors, the one or more images into a character training model.

### Claim 14 (depends on 13)

The CS of claim 13, wherein the program instructions further includes instructions for causing the one or more processors to perform the following operations:
mapping, by the one or more computer processors, the one or more images into one or more characters in the character training model, wherein mapping includes determining whether the one or more images indicate at least one of the one or more characters in the character training model utilizing a pre-determined range of motion of the paired ring device during a time period of movement in free space, where the pre-determined range of motion creates a specific image.

### Claim 15 (depends on 14)

The CS of claim 14, wherein the program instructions further includes instructions for causing the one or more processors to perform the following operations:
responsive to a determination that the one or more images indicate at least one of the one or more characters in the character training model, transcribing, by the one or more computer processors, the at least one of the one or more characters into a digital document, wherein the digital document is a visual augmentation for displaying in a heads up display of the paired display device.

### Claim 16 (depends on 15)

The CS of claim 15, wherein the program instructions further includes instructions for causing the one or more processors to perform the following operations:
determining, by the one or more computer processors, a representation of the at least one or more characters as a written language or a gesture output derived from a series of strokes in free space; and
encoding, by the one or more computer processors, the at least one or more characters into a visual augmentation.

### Claim 17 (depends on 13)

The CS of claim 13, wherein:
the paired ring device includes an accelerometer and a gyroscope for monitoring the one or more language movements; and
the one or more language movements include one or more of the following: (i) a change in a magnitude value and a direction value of proper acceleration of the paired ring device, (ii) an orientation of the paired ring device, (iii) a position of the paired ring device, (iv) a location of the paired ring device, (v) a change in the orientation of the paired ring device, (vi) a change in the position of the paired ring device, and (vii) a change in the location of the paired ring device in the modified free space environment.

### Claim 18 (depends on 13)

The CS of claim 13, wherein the paired ring device is sized to be worn on a human finger digit.
