# US 2016/0259766 A1 · Ink experience for images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0259766 A1
- Document ID: US20160259766A1
- Status: Published application
- Kind: A1
- Application number: 14/998,325
- Filed: 2015-12-24
- Published: 2016-09-08
- Earliest claimed: 2015-03-08
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Sergei Ivanov; Panos C. Panay; Ales Holecek; Ali Akgun; Maxim Kudryavtsev
- Web page: https://patentgrep.com/patent/US20160259766A1

## CPC

- G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information}
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 40/169: Annotation, e.g. comment data or footnotes
- G06F 40/171: by use of digital ink
- G06V 30/226: of cursive writing
- G06V 30/36: {Matching; Classification}
- G06T 11/60: Creating or editing images; Combining images with text
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Techniques for an ink experience with images are discussed herein. In various implementations, an image is displayed via an image management application for viewing and/or editing images. In conjunction with interaction scenarios provided via the application, an inking mode for adding inked annotations to the image is enabled. Input to apply one or more inked annotations to the image is obtained, such as via finger touches on a touchscreen, drawing with a stylus, camera-based gestures, or other natural input mechanisms. Responsive to obtaining the input, data blocks corresponding to the one or more inked annotations are appended to an image file as additional data blocks for the image.

## Claims

### Claim 1

A system comprising:
a display; and
one or more processors in communication with the display, the one or more processors configured to:
display an image on the display via a viewing application;
enable an inking mode for adding inked annotations to the image, the inking mode supporting addition of annotations to at least front side and back side representations of the image;
obtain input to apply one or more inked annotations to the image; and
append data blocks corresponding to the one or more inked annotations as additional data blocks for the image.

### Claim 2 (depends on 1)

The system as described in claim 1, wherein the data blocks corresponding to the one or more inked annotations includes indications of locations for the inked annotations.

### Claim 3 (depends on 1)

The system as described in claim 1, wherein the viewing application is configured to animate transitions between the front side and back side representations of the image.

### Claim 4 (depends on 1)

The system as described in claim 1, wherein the inking mode further supports association of one or more note elements with the image and creation of inked annotations on the note elements.

### Claim 5 (depends on 1)

The system as described in claim 1, wherein the inked annotations are stored a vector objects that enable repositioning within the image, resizing, recoloring, and deletion of the inked annotations.

### Claim 6 (depends on 1)

The system as described in claim 1, wherein the original image is not altered by inked annotations added to the image.

### Claim 7 (depends on 1)

The system as described in claim 1, wherein ink data blocks for the inked annotations are appended to image data for the image as input of the ink content occurs.

### Claim 8 (depends on 1)

The system as described in claim 1, wherein the one or more processors are further configured to:
enable inking for different views of the image including one or more of the image body, margins, a back side of the image, or note pages added to the image;
provide navigation controls to switch back and forth between the different view;
render visual transitions responsive to navigation between the views; and
provide a toggle control to selectively show or hide inked annotations added to the different views.

### Claim 9 (depends on 1)

The system as described in claim 1, wherein the one or more processors are further configured to:
apply handwriting recognition to inked annotations added to the image;
extract keywords corresponding to the inked annotations; and
store the keywords as metadata for the image.

### Claim 10 (depends on 9)

The system as described in claim 9, wherein the one or more processors are further configured to:
enable keyword based photo management operations using the extracted keywords including one or a combination of searching, filtering, categorizing, and sorting.

### Claim 11 (depends on 9)

The system as described in claim 9, wherein the one or more processors are further configured to:
prior to storing the keywords, expose recognized keywords via a visual element to facilitate review and confirmation of the keywords, wherein the storing of the keywords as metadata for the image occurs responsive to confirmation via the visual element.

### Claim 12 (depends on 11)

The system as described in claim 11, wherein the one or more processors are further configured to:
provide via the visual element functionality to edit the recognized keywords, accept the keywords, or reject the keywords.

### Claim 13 (depends on 1)

The system as described in claim 1, wherein inked annotations are saved automatically without an explicit selection of a save command.

### Claim 14 (depends on 1)

The system as described in claim 1, wherein the inked annotations are usable by applications that support inking features and do not interfere with handling of images by applications that do not support inking features.

### Claim 15 (depends on 1)

The system as described in claim 1, wherein rendering of the inked annotations is bypassed in thumbnail views and other views of the image that result in illegible representations of the inked annotations.

### Claim 16

A computer-implemented method, comprising:
displaying an image via a viewing application;
enabling an inking mode for adding inked annotations to the image including enabling inking for different views of the image including representations of the image body, margins, a back side of the image, and note pages added to the image;
obtaining input to apply one or more inked annotations to the image; and
appending data blocks corresponding to the one or more inked annotations as additional data blocks for the image.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, further comprising:
applying handwriting recognition to inked annotations added to the image displayed via the viewing application;
extracting keywords corresponding to the inked annotations;
storing the keywords as metadata for the image; and
enabling keyword based photo management operations using the extracted keywords including one or a combination of searching, filtering, categorizing, and sorting.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, further comprising:
applying the handwriting recognition to recognize keywords corresponding to the inked annotations in real-time as the input to add the inked annotations is obtained;
exposing recognized keywords via a visual element to facilitate review and confirmation of the keywords; and
providing via the visual element functionality to edit the recognized keywords, accept the keywords, and reject the keywords.

### Claim 19 (depends on 16)

The computer-implemented method of claim 16, further comprising:
providing navigation controls to switch back and forth between the different views; and
rendering visual transitions responsive to navigation between the views.

### Claim 20

One or more computer-readable storage media storing computer-executable instructions that, responsive to execution by a computing device, cause the computing device to perform operations comprising:
exposing an image as an inkable canvas via a viewer that supports inking;
enabling inking for different views of the image including one or more of the image body, margins, a back side of the image, or note pages added to the image;
providing navigation controls to switch back and forth between the different views;
rendering visual transitions responsive to navigation between the views; and
providing a toggle control to selectively show or hide inked annotations added to the different views.
