# US 2013/0086472 A1 · Interactive techniques for rectifying and scrolling photos of documents in a 3d viewport

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0086472 A1
- Document ID: US20130086472A1
- Status: Published application
- Kind: A1
- Application number: 13/252,573
- Filed: 2011-10-04
- Published: 2013-04-04
- Earliest claimed: 2011-10-04
- Assignees: Fuji Xerox Co., Ltd.
- Inventors: Patrick CHIU
- Web page: https://patentgrep.com/patent/US20130086472A1

## CPC

- H04N 1/3872: {Repositioning or masking}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods for providing a user interface to help users rectify and scroll photos of documents. Systems and methods are based on an approach employing a 3D viewport with multi-touch gestures along with scroll bars. Scrolling a rotated photo in a 3D space exhibits some strange and unexpected behaviors; wherein systems and methods solve these problems using trigonometry in 3D and Newton's Method while being fast enough to support real-time interaction.

## Claims

### Claim 1

A method of manipulating an image containing a document, comprising:
creating a three dimensional viewport for the image;
upon receipt of a gesture for rectifying the image, rectifying the image according to the gesture;
wherein the gesture is received directly on the image within the three dimensional viewport.

### Claim 2 (depends on 1)

The method of claim 1, further comprising rendering the rectified image into a two dimensional image and displaying the image on a two dimensional viewport.

### Claim 3 (depends on 1)

The method of claim 1, wherein if the gesture for the rectifying is a command for rotating or stretching the image, performing the rotating or stretching according to the gesture.

### Claim 4 (depends on 1)

The method of claim 1, wherein if the gesture for the rectifying is a command for scrolling the image, determining if the scrolling is along a rotated axis of the image.

### Claim 5 (depends on 4)

The method of claim 4, wherein if the scrolling is along the rotated axis of the image, scrolling the image according to the gesture and correcting for skew by utilizing a numerical algorithm to solve for a geometric transform.

### Claim 6 (depends on 4)

The method if claim 4, wherein if the scrolling is not along the rotated axis of the image, scrolling the image according to the gesture and correcting for stretching of the image based on the scroll.

### Claim 7

A method of manipulating an image of a document, comprising:
creating a three dimensional viewport for the image of the document;
receiving a command;
wherein if the command is for scrolling the image, scrolling the image according to the gesture and correcting for geometric distortion by utilizing a numerical algorithm to solve for a geometric transform during the scrolling.

### Claim 8 (depends on 7)

The method of claim 7, wherein when the command is for dragging the image, dragging the image according to the gesture and correcting for geometric distortion during the dragging.

### Claim 9 (depends on 7)

The method of claim 7, wherein the correcting for geometric distortion comprises correcting for skew by applying a numerical algorithm to solve for a geometric transform when the scrolling is along a rotated axis of the image.

### Claim 10 (depends on 7)

The method of claim 7, wherein the correcting for geometric distortion comprises correcting for stretching of the image based on the scroll when the scrolling is orthogonal to the rotated axis of the image.

### Claim 11 (depends on 7)

The method of claim 7, wherein the correcting for geometric distortion comprises correcting for stretching and correcting for skew when the scrolling is not orthogonal or along the rotated axis.

### Claim 12 (depends on 11)

The method of claim 11, wherein the correcting for skew comprises utilizing Newton's Method.

### Claim 13

An apparatus, comprising:
a touch display operable to display an image of a document in a three dimensional viewport;
a manipulation module, wherein upon receipt of a gesture directly on the image displayed within the three dimensional viewport on the touch display, the manipulation module manipulates the image according to the gesture; and
a correction module correcting the image for geometric distortion caused by a scroll, drag or rotation of the image resulting from the manipulation of the image.

### Claim 14 (depends on 13)

The apparatus of claim 13, further comprising a rendering module for rendering the rectified image into a two dimensional image and
wherein the touch display is operable to display the two dimensional image in a two dimensional viewport.

### Claim 15 (depends on 13)

The apparatus of claim 13,
wherein the correction module corrects for geometric distortion by correcting for stretching of the image based on the scroll when the scroll is orthogonal to a rotated axis of the image.

### Claim 16 (depends on 13)

The apparatus of claim 13,
wherein the correction module corrects for geometric distortion by applying a numerical algorithm to solve for a geometric transform when the scroll is along a rotated axis of the image to correct for skew.

### Claim 17 (depends on 13)

The apparatus of claim 13, wherein the correction module corrects for geometric distortion by correcting for stretching and correcting for skew when the scrolling is not orthogonal or along the rotated axis.

### Claim 18 (depends on 13)

The apparatus of claim 13, wherein the manipulation module manipulates the image by rotating the image upon receipt of the gesture comprising a multi-touch gesture, the multi-touch gesture comprising a first touch directly on the image for holding the image in place and a second touch rotating the image.

### Claim 19 (depends on 13)

The apparatus of claim 13, wherein the manipulation module manipulates the image by scaling the image along a dimension upon receipt of the gesture comprising a multi-touch gesture, the multi-touch gesture comprising a four touches directly on the image for stretching the image to scale the image along a dimension.
