# US 2014/0155121 A1 · Portable terminal apparatus, cooperative image processing system, method for displaying screen data, and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0155121 A1
- Document ID: US20140155121A1
- Status: Published application
- Kind: A1
- Application number: 14/091,745
- Filed: 2013-11-27
- Published: 2014-06-05
- Earliest claimed: 2012-11-30
- Assignees: Konica Minolta, Inc.
- Applicants: Konica Minolta, Inc.
- Inventors: Kenya HABA
- Web page: https://patentgrep.com/patent/US20140155121A1

## CPC

- G06F 3/1218: {Reducing or saving of used resources, e.g. avoiding waste of consumables or improving usage of hardware resources}
- G06F 3/1253: {Configuration of print job parameters, e.g. using UI at the client}
- G06F 3/1268: {Job submission, e.g. submitting print job order or request not the print data itself}
- G06F 3/1292: {Mobile client, e.g. wireless printing}
- G06F 9/452: {Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131)}
- H04N 1/00129: {with a display device, e.g. CRT or LCD monitor (with a television apparatus H04N1/00283; arrangements for control of display devices G09G)}
- H04N 1/00411: {the display also being used for user input, e.g. touch screen}
- H04N 2201/0027: Adapting to communicate with plural different types of apparatus
- H04N 2201/0031: where the still picture apparatus controls another apparatus
- H04N 2201/0075: by a user operated remote control device, e.g. receiving instructions from a user via a computer terminal or mobile telephone handset
- H04N 2201/0094: Multifunctional device, i.e. a device capable of all of reading, reproducing, copying, facsimile transception, file transception
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A portable terminal apparatus comprising: a display for a remote screen of the image processing apparatus; an acquisition portion for acquiring information indicating whether an area in the remote screen is a first area which is set for editorial processing by the portable terminal apparatus or a second area which is set for editorial processing by the image processing apparatus; a determination portion for determining whether a gesture operation is an operation performed on the first area or on the second area; and a control portion for performing the editorial processing with screen data of the first area if the operation was performed on the first area, and for transmitting information instructing the image processing apparatus to perform the editorial processing of screen data of the second area and acquiring the editorial processed screen data, if the operation was performed on the second image area.

## Claims

### Claim 1

A portable terminal apparatus comprising:
a wireless communication portion connectable with an image processing apparatus via wireless communications;
a receiving portion for receiving screen data of the image processing apparatus, while being wirelessly connected with the image processing apparatus by the wireless communication portion;
a display portion for displaying a remote screen based on the screen data received by the receiving portion;
a touch operation portion provided on the display portion;
an acquisition portion for acquiring attribute information indicating whether an area in the remote screen displayed in the display portion on the basis of the screen data is a first screen area which is set for editorial processing to be performed by the portable terminal apparatus or a second screen area which is set for editorial processing to be performed by the image processing apparatus, when a gesture operation instructing editorial processing of screen data is performed on the remote screen;
a storage portion for storing the screen data received by the receiving portion and the acquired attribute information;
a determination portion for determining whether a gesture operation instructing editorial processing of screen data performed by a user via the touch operation portion on the remote screen displayed in the display portion is an operation performed on the first screen area or an operation performed an operation performed on the second screen area; and
a control portion for performing the editorial processing with the screen data of the first screen area stored in the storage portion and displaying the editorial processed screen data on the display screen if the determination portion determines that the operation was performed on the first screen area, and for transmitting information instructing the image processing apparatus to perform the editorial processing of the screen data of the second screen area and acquiring the editorial processed screen data from the image processing apparatus to display it in the display portion, if the determination portion determines that the operation was performed on the second image area.

### Claim 2 (depends on 1)

The portable terminal apparatus according to claim 1, comprising an information acquisition portion for acquiring information on a wireless communication state with the image processing apparatus, wherein
when a wireless communication state based on the information acquired by the information acquisition portion exceeds predetermined level, the control portion transmits to the image processing apparatus information instructing editorial processing of the screen data of the first screen area even if the operation instructing the editorial processing of the screen data was performed on the first screen area and acquires the editorial processed screen data from the image processing apparatus to display it in the display portion.

### Claim 3

A portable terminal apparatus comprising:
a wireless communication portion connectable with an image processing apparatus via wireless communications;
a receiving portion for receiving screen data of the image processing apparatus from the image processing apparatus, while being wirelessly connected with the image processing apparatus by the wireless communication portion;
an information acquisition portion for acquiring information on a wireless communication state with the image processing apparatus;
a display portion for displaying a remote screen based on the screen data received by the receiving portion;
a touch operation portion provided on the display portion;
a storage portion storing the screen data received by the receiving portion; and
a control portion for transmitting information instructing editorial processing of screen data to the image processing apparatus and acquiring the editorial processed screen data from the image processing apparatus to display it in the display portion when a gesture operation instructing editorial processing of screen data is performed by a user via the touch operation portion on the remote screen displayed in the display portion and when a wireless communication state based on the information acquired by the information acquisition portion exceeds predetermined level, and for performing the editorial processing with the screen data stored in the storage portion and displaying the editorial processed screen data in the display portion when the communication state acquired by the information acquisition portion is below the predetermined level.

### Claim 4 (depends on 3)

The portable terminal apparatus according to claim 3 wherein responding to continuous movement of a finger in a gesture operation, the control portion repeatedly performs transmission of information instructing editorial processing of screen data to the image processing apparatus, acquisition of the editorial processed screen data from the image processing apparatus, and display in the display portion when the communication state exceeds the predetermined level, and repeatedly performs editorial processing with the screen data stored in the storage portion and display of the editorial processed screen data in the display portion when the communication state is below the predetermined level.

### Claim 5 (depends on 1)

The portable terminal apparatus according to claim 1 wherein the control portion transmits information on updating to the image processing apparatus when a gesture operation instructing updating to a next screen on the remote screen is performed by the user, and displays a screen based on next screen data in the display portion when the control portion acquires updated next screen data from the image processing apparatus.

### Claim 6 (depends on 5)

The portable terminal apparatus according to claim 5 wherein it is determined that the gesture operation is the operation instructing updating to the next screen when the gesture operation performed by the user is a one-touch operation, and it is determined that the gesture operation is the operation instructing the editorial processing of the screen data when the gesture operation is a two-touch operation.

### Claim 7 (depends on 6)

The portable terminal apparatus according to claim 6 wherein a gesture operation by the two-touch operation is a pinch-in operation or a pinch-out operation.

### Claim 8 (depends on 1)

The portable terminal apparatus according to claim 1 wherein the editorial processing of the screen data is scaling or rotation of the screen data.

### Claim 9 (depends on 1)

The portable terminal apparatus according to claim 1 wherein the information instructing the editorial processing of the screen data is coordinate data on a touch position on the remote screen.

### Claim 10 (depends on 5)

The portable terminal apparatus according to claim 5 wherein the information on the updating is the coordinate data on the touch position on the remote screen.

### Claim 11 (depends on 10)

The portable terminal apparatus according to claim 10, wherein
the receiving portion further receives information on screen size of the display portion of the image processing apparatus, and
the control portion converts the coordinate data on the touch position into coordinate data corresponding to the display portion of the image processing apparatus and transmits it to the image processing apparatus, when the received screen size of the display portion of the image processing apparatus differs from screen size of the display portion of the portable terminal apparatus.

### Claim 12

A method for displaying screen data to be performed by a portable terminal apparatus connectable with an image processing apparatus via wireless communications, the method for displaying screen data comprising:
receiving screen data on the image processing apparatus, while being wirelessly connected with the image processing apparatus;
displaying a remote screen based on the received screen data in a display portion;
acquiring attribute information indicating whether an area in the remote screen displayed in the display portion on the basis of screen data is a first screen area which is set for editorial processing to be performed by the portable terminal apparatus or a second screen area which is set for editorial processing to be performed by the image processing apparatus, when a gesture operation instructing editorial processing of screen data is performed on the remote screen;
storing the received screen data and the acquired attribute information in a storage portion;
determining whether a gesture operation instructing editorial processing of screen data performed by a user on the remote screen displayed in the display portion via a touch operation portion provided in the display portion is an operation performed on the first screen area or an operation performed on the second screen area; and
controlling so as to perform editorial processing with the screen data of the first screen area stored in the storage portion and display the editorial processed screen data in the display portion when it is determined that the operation was performed on the first screen area, and so as to transmit information instructing the image processing apparatus to perform editorial processing of the screen data of the second screen area and acquire the editorial processed screen data from the image processing apparatus to display it in the display portion when it is determined that the operation was performed on the second screen area.

### Claim 13

A method for displaying screen data to be performed by a portable terminal apparatus connectable with an image processing apparatus via wireless communications, the method for displaying screen data comprising:
receiving screen data on the image processing apparatus from the image processing apparatus, while being wirelessly connected with the image processing apparatus;
information acquisition of acquiring information on a wireless communication state with the image processing apparatus;
displaying a remote screen based on the received screen data in a display portion;
storing the received screen data in a storage portion; and
controlling so as to transmit information instructing editorial processing of screen data to the image processing apparatus and acquire the editorial processed screen data from the image processing apparatus to display it in the display portion when a gesture operation instructing editorial processing of screen data is performed by a user via a touch operation portion on the remote screen displayed in the display portion and when the wireless communication state based on the information acquired in the information acquisition exceeds predetermined level, and so as to perform editorial processing with the screen data stored in the storage portion and display the editorial processed screen data in the display portion when the wireless communication state based on the information acquired in the information acquisition is below the predetermined level.

### Claim 14

A non-transitory computer-readable recording medium in which is recorded a screen data display program for causing a computer of a portable terminal apparatus connectable with an image processing apparatus via wireless communications to perform:
receiving screen data of the image processing apparatus, while being wirelessly connected with the image processing apparatus;
displaying a remote screen based on the screen data received in the receiving in a display portion;
acquiring attribute information indicating whether an area in the remote screen displayed in the display portion on the basis of the screen data is a first screen area which is set for editorial processing to be performed by the portable terminal apparatus or a second screen area which is set for editorial processing to be performed by the image processing apparatus, when a gesture operation instructing editorial processing of screen data is performed on the remote screen;
storing the screen data received in the receiving and the acquired attribute information in a storage portion;
determination of determining whether a gesture operation instructing editorial processing of screen data performed by a user on the remote screen displayed in the display portion via a touch operation portion provided in the display portion is an operation performed on the first screen area or an operation performed on the second screen area; and
controlling so as to perform editorial processing with the screen data of the first screen area stored in the storage portion and display the editorial processed screen data in the display portion when it is determined in the determination that the operation was performed on the first screen area, and so as to transmit information instructing the image processing apparatus to perform editorial processing of the screen data of the second screen area and acquire the editorial processed screen data from the image processing apparatus to display it in the display portion when it is determined that the operation was performed on the second screen area.

### Claim 15

A non-transitory computer-readable recording medium in which is recorded a screen data display program for causing a computer of a portable terminal apparatus connectable with an image processing apparatus via wireless communications to perform:
information acquisition of acquiring information on a wireless communication state with the image processing apparatus;
receiving screen data of the image processing apparatus from the image processing apparatus;
displaying a remote screen based on the received screen data in a display portion;
storing the received screen data in a storage portion; and
controlling so as to transmit information instructing editorial processing of screen data to the image processing apparatus and acquire the editorial processed screen data from the image processing apparatus to display it in the display portion when a gesture operation instructing editorial processing of screen data is performed by a user via a touch operation portion on the remote screen displayed in the display portion and when the wireless communication state based on the information acquired in the information acquisition exceeds predetermined level, and so as to perform editorial processing with the screen data stored in the storage portion and display the editorial processed screen data in the display portion when the wireless communication state based on the information acquired in the information acquisition is below the predetermined level.

### Claim 16

A cooperative image processing system comprising a portable terminal apparatus and an image processing apparatus which are mutually connected via wireless communications,
the portable terminal apparatus comprising:
a wireless communication portion connectable with the image processing apparatus via wireless communications;
a receiving portion for receiving screen data of the image processing apparatus, while being wirelessly connected with the image processing apparatus;
a display portion for displaying a remote screen based on the screen data received by the receiving portion;
a touch operation portion provided on the display portion;
an acquisition portion for acquiring attribute information indicating whether an area in the remote screen displayed in the display portion on the basis of the screen data is a first screen area which is set for editorial processing to be performed by the portable terminal apparatus or a second screen area which is set for editorial processing to be performed by the image processing apparatus, when a gesture operation instructing editorial processing of screen data is performed on the remote screen;
a storage portion for storing the screen data received by the receiving portion and the acquired attribute information;
a determination portion for determining whether a gesture operation instructing editorial processing of screen data performed by a user via the touch operation portion on the remote screen displayed in the display portion is an operation performed on the first screen area or an operation performed on the second screen area; and
a control portion for performing the editorial processing with the screen data of the first screen area stored in the storage portion and displaying the editorial processed screen data on the display portion if the determination portion determines that the operation was performed on the first screen area, and for transmitting information instructing the image processing apparatus to perform the editorial processing of screen data of the second screen area and acquiring the editorial processed screen data from the image processing apparatus to display it in the display portion, if the determination portion determines that the operation was performed on the second image area, and
the image processing apparatus comprising:
a receiving portion for receiving information instructing editorial processing of screen data of the second screen area from the portable terminal apparatus;
an editorial processing portion for editorial processing screen data on the basis of the information instructing the editorial processing of screen data received by the receiving portion; and
a transmission portion for transmitting screen data to the portable terminal apparatus and transmitting the editorial processed screen data to the portable terminal apparatus when the editorial processing portion editorial processed screen data.

### Claim 17

A cooperative image processing system comprising a portable terminal apparatus and an image processing apparatus which are mutually connected via wireless communications,
the portable terminal apparatus comprising:
a wireless communication portion connectable with the image processing apparatus via wireless communications;
a receiving portion for receiving screen data of the image processing apparatus from the image processing apparatus, while being wirelessly connected with the image processing apparatus;
an information acquisition portion for acquiring information on a wireless communication state with the image processing apparatus;
a display portion for displaying a remote screen based on the screen data received by the receiving portion;
a touch operation portion provided on the display portion;
a storage portion for storing the screen data received by the receiving portion; and
a control portion for transmitting information instructing editorial processing of screen data to the image processing apparatus and acquiring the editorial processed screen data from the image processing apparatus to display it in the display portion when a gesture operation instructing editorial processing of screen data is performed by a user via the touch operation portion on the remote screen displayed in the display portion and when a wireless communication state based on the information acquired by the information acquisition portion exceeds predetermined level, and for performing the editorial processing with the screen data stored in the storage portion and displaying the editorial processed screen data in the display portion when the communication state acquired by the information acquisition portion is below the predetermined level,
the image processing apparatus comprising:
a receiving portion for receiving information instructing editorial processing of the image data of the second screen area from the portable terminal apparatus;
an editorial processing portion for editorial processing screen data on the basis of the information instructing the editorial processing of screen data received by the receiving portion; and
a transmission portion for transmitting the screen data to the portable terminal apparatus and transmitting the editorial processed screen data to the portable terminal apparatus when the editorial processing portion editorial processed screen data.