# US 2017/0285850 A1 · Developing contextual information from an image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0285850 A1
- Document ID: US20170285850A1
- Status: Published application
- Kind: A1
- Application number: 15/509,805
- Filed: 2014-09-12
- Published: 2017-10-05
- Earliest claimed: 2014-09-12
- Assignees: Hewlett-Packard Development Company, L.P.
- Applicants: Hewlett-Packard Development Company, L.P.
- Inventors: Immanuel Amo; Diogo Lima; Nicholas P Lyons; Arman Alimian
- Web page: https://patentgrep.com/patent/US20170285850A1

## CPC

- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)}
- G03B 17/54: with projector
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G03B 2206/00: Systems for exchange of information between different pieces of apparatus, e.g. for exchanging trimming information, for photo finishing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)

## Abstract

In an example implementation according to aspects of the present disclosure, a method may include capturing data entered on a touch sensitive mat or on an object physically disposed on the touch sensitive mat. The method further includes extracting the data from the captured image, and developing contextual information from the data extracted from the captured image. The method further includes projecting the contextual information onto the touch sensitive mat or onto the object.

## Claims

### Claim 1

A method comprising:
capturing an image of data entered on a touch sensitive mat or on dry object physically disposed on the touch sensitive mat;
extracting the data from the captured image;
developing contextual information from the data extracted from the captured image; and
projecting the contextual information onto the touch sensitive mat or onto the object.

### Claim 2 (depends on 1)

The method of claim 1, comprising:
collecting metadata associated with the extracted data, wherein the metadata comprises positioning of the extracted data on the touch sensitive mat or on the object.

### Claim 3 (depends on 2)

The method of claim 2, wherein the contextual information is developed based on a selection of the data made by a user on the touch sensitive mat or on the object.

### Claim 4 (depends on 3)

The method of claim 3, wherein a positioning of the selection of the data on the touch sensitive mat or on the object corresponds to the metadata associated with the extracted data.

### Claim 5 (depends on 1)

The method claim 1, wherein extracting the data comprises:
extracting characters from the captured image; and
converting the extracted characters into machine-readable characters.

### Claim 6 (depends on 5)

The method of claim 5, wherein developing the contextual information comprises selecting keywords from the machine-readable characters for developing the contextual information.

### Claim 7 (depends on 5)

The method of claim 5, wherein the contextual information is developed based on natural language processing algorithms.

### Claim 8 (depends on 1)

The method of claim 1, wherein projecting the contextual formation comprises projecting visual cues associated with the contextual information, for a user to choose and interact with the visual cues to present the contextual information when needed.

### Claim 9

A system, comprising:
a support structure including a base, an upright member extending upward from the base, and a cantilevered top extending outward from the upright member and including a fold mirror and a plurality of sensors;
a projector assembly attachable to the upright member;
a computing device attachable to the projector assembly; and
a touch sensitive mat communicatively coupled to the computing device, on to which the projector is to project an image, wherein the projector is to project the image upward to reflect off the mirror and on to the mat, and wherein the computing device is to cause:
the plurality of sensors to capture an image of data on an object physically disposed on the mat;
the computing device to extract data from the captured image:
the computing device to develop contextual information from the data extracted from the captured image; and
the projector assembly to project the contextual information onto the mat or onto the object.

### Claim 10 (depends on 9)

The system of claim 9, wherein the computing device is to collect metadata associated with the extracted data, wherein the metadata comprises positioning of the extracted data on the touch sensitive mat or on the object.

### Claim 11 (depends on 9)

The system of claim 9, wherein the contextual information is developed based on a selection of the data made by a user on the touch sensitive mat or on the object.

### Claim 12 (depends on 11)

The system of claim 11, wherein the selection of the data made by the user on the mat is to be detected by the mat and the selection of the data made by the user on the object is to be detected by the plurality of sensors.

### Claim 13

A non-transitory machine-readable storage medium comprising instructions executable by a processing resource of a computing system comprising a touch sensitive mat, a projector assembly to project images on the mat, and a plurality of sensors disposed above and pointed at the mat, the instructions executable to:
capture an image of data entered on the mat or on an object physically disposed on the mat;
extract the data from the captured image;
develop contextual information from the data extracted from the captured image; and
project the contextual information onto the matt or onto the object.

### Claim 14 (depends on 13)

The storage medium of claim 13, wherein the instructions executable to extract the data comprises:
extracting characters from the captured image; and
converting the extracted characters into machine-readable characters.

### Claim 15 (depends on 14)

The storage medium of claim 14, wherein the instructions executable to develop the contextual information comprises selecting keywords from the machine-readable characters for developing the contextual information.
