# US 2015/0091830 A1 · Non-transitory storage medium storing instructions executable by communication terminal, communication terminal, and processing apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0091830 A1
- Document ID: US20150091830A1
- Status: Published application
- Kind: A1
- Application number: 14/493,805
- Filed: 2014-09-23
- Published: 2015-04-02
- Earliest claimed: 2013-09-30
- Assignees: Brother Kogyo Kabushiki Kaisha
- Applicants: Brother Kogyo Kabushiki Kaisha
- Inventors: Sadaaki MIYAZAKI
- Web page: https://patentgrep.com/patent/US20150091830A1

## CPC

- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0416: {Control or interface arrangements specially adapted for digitisers}
- G06F 2203/04104: Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A non-transitory storage medium stores instructions executable by a communication terminal. The communication terminal includes: a display; an input device for detecting an instructed point where an instruction is provided with an input object; and a communication device for communicating with a processing apparatus. When executed by the computer, the instructions cause the communication terminal to: receive, from the processing apparatus, instruction image data for displaying an instruction image for instructing a processing to be executed by the processing apparatus; display the instruction image on the display based on the received instruction image data; and transmit, to the processing apparatus, an input event on the input device and coordinate information about the instructed point on the display which is detected by the input device in a state in which the instruction image is displayed.

## Claims

### Claim 1

A non-transitory storage medium storing a plurality of instructions executable by a computer of a communication terminal,
the communication terminal comprising: a display; an input device configured to detect an instructed point which is a point on the display and at which an instruction is provided with an input object; and a communication device configured to communicate with a processing apparatus,
when executed by the computer, the plurality of instructions causing the communication terminal to perform:
receiving instruction image data from the processing apparatus, the instruction image data being for displaying, on the display, an instruction image for instructing a processing to be executed by the processing apparatus;
displaying the instruction image on the display based on the received instruction image data;
identifying an input event on the input device in response to an instruction provided on the input device with the input object in a state in which the instruction image is displayed; and
transmitting, to the processing apparatus, the identified input event and coordinate information about the instructed point on the display which is detected by the input device in a state in which the instruction image is displayed.

### Claim 2 (depends on 1)

The non-transitory storage medium according to claim 1, wherein when executed by the computer, the plurality of instructions cause the communication terminal to perform:
making a request of the processing apparatus for transmission of the instruction image data; and
receiving the instruction image data transmitted from the processing apparatus in response to the request of the processing apparatus for transmission of the instruction image data.

### Claim 3 (depends on 1)

The non-transitory storage medium according to claim 1, wherein when executed by the computer, the plurality of instructions cause the communication terminal to perform:
receiving identification information for identifying the instruction image data, from the processing apparatus when the instruction image data is received from the processing apparatus; and
transmitting, to the processing apparatus, identification information about instruction image data representative of an instruction image currently displayed on the display.

### Claim 4 (depends on 3)

The non-transitory storage medium according to claim 3,
wherein when executed by the computer, the plurality of instructions cause the communication terminal to perform:
receiving, from the processing apparatus, a difference notification indicating that the identification information about the instruction image data representative of the instruction image currently displayed on the display of the communication terminal differs from identification information about image data representative of an image currently displayed on the processing apparatus; and
displaying, on the display, an image indicating that the identification information about the instruction image data representative of the instruction image currently displayed on the display of the communication terminal differs from the identification information about the image data representative of the image currently displayed on the processing apparatus, in response to the reception of the difference notification.

### Claim 5 (depends on 3)

The non-transitory storage medium according to claim 3,
wherein when executed by the computer, the plurality of instructions cause the communication terminal to perform:
receiving, from the processing apparatus, a difference notification indicating that the identification information about the instruction image data representative of the instruction image currently displayed on the display of the communication terminal differs from identification information about image data representative of an image currently displayed on the processing apparatus; and
requesting the processing apparatus to transmit the image data representative of the image currently displayed on the processing apparatus, in response to the reception of the difference notification.

### Claim 6 (depends on 1)

The non-transitory storage medium according to claim 1, wherein when executed by the computer, the plurality of instructions cause the communication terminal not to transmit, to the processing apparatus, the input event on the input device which is performed within at least a period extending from a request of the processing apparatus for transmission of the instruction image data, to the reception of the instruction image data transmitted in response to request.

### Claim 7 (depends on 2)

The non-transitory storage medium according to claim 2, wherein when executed by the computer, the plurality of instructions cause the communication terminal to perform:
receiving a request instruction which is an instruction transmitted from the processing apparatus to the communication terminal and which instructs the communication terminal to request the instruction image data from the processing apparatus; and
making the request of the processing apparatus for transmission of the instruction image data, in response to the reception of the request instruction.

### Claim 8 (depends on 7)

The non-transitory storage medium according to claim 7, wherein when executed by the computer, the plurality of instructions cause the communication terminal to perform:
displaying, on the display, a screen inquiring whether the request of the processing apparatus for transmission of the instruction image data is to be allowed, when making the request of the processing apparatus for transmission of the instruction image data, in response to the reception of the request instruction; and
making the request of the processing apparatus for transmission of the instruction image data, when the request of the processing apparatus for transmission of the instruction image data is allowed.

### Claim 9 (depends on 1)

The non-transitory storage medium according to claim 1,
wherein the processing apparatus comprises: an apparatus-side display; and an apparatus-side input device configured to detect an instructed point which is a point on the apparatus-side display and at which an instruction is provided with an input object, and
wherein the instruction image to be displayed based on the received instruction image data is identical to an image to be displayed on the apparatus-side display.

### Claim 10 (depends on 2)

The non-transitory storage medium according to claim 2, wherein the request of the processing apparatus for transmission of the instruction image data is made based on transmission of the input event to the processing apparatus.

### Claim 11 (depends on 10)

The non-transitory storage medium according to claim 10,
wherein the input event comprises a choice of a point on the display and a confirmation of the choice of the point, and
wherein the request of the processing apparatus for transmission of the instruction image data is made based on transmission of the confirmation of the choice of the point to the processing apparatus.

### Claim 12 (depends on 11)

The non-transitory storage medium according to claim 11,
wherein the processing apparatus comprises: an apparatus-side display; and an apparatus-side input device configured to detect an instructed point which is a point on the apparatus-side display and at which an instruction is provided with an input object,
wherein the instruction image to be displayed based on the received instruction image data is identical to an image to be displayed on the apparatus-side display, and
wherein when executed by the computer, the plurality of instructions cause the communication terminal to wait for making the request of the processing apparatus for transmission of the instruction image data, for a predetermined period after the transmission of the confirmation of the choice of the point to the processing apparatus.

### Claim 13 (depends on 12)

The non-transitory storage medium according to claim 12, wherein when executed by the computer, the plurality of instructions cause the communication terminal not to transmit the input event having occurred within the predetermined period, to the processing apparatus.

### Claim 14 (depends on 2)

The non-transitory storage medium according to claim 2,
wherein the processing apparatus comprises: an apparatus-side display; and an apparatus-side input device configured to detect an instructed point which is a point on the apparatus-side display and at which an instruction is provided with an input object,
wherein the instruction image to be displayed based on the received instruction image data is identical to an image to be displayed on the apparatus-side display, and
wherein when executed by the computer, the plurality of instructions cause the communication terminal to perform:
receiving, from the processing apparatus, a notification indicating that an object on the image displayed on the apparatus-side display is to be displaced; and
making, based on the reception of the notification, the request of the processing apparatus for transmission of the instruction image data.

### Claim 15 (depends on 14)

The non-transitory storage medium according to claim 14, wherein when executed by the computer, the plurality of instructions cause the communication terminal to perform:
making the request of the processing apparatus for transmission of the instruction image data, based on an occurrence of the input event on the input device; and
when the notification is received, not making the request of the processing apparatus for transmission of the instruction image data based on an occurrence of the input event.

### Claim 16 (depends on 14)

The non-transitory storage medium according to claim 14, wherein when executed by the computer, the plurality of instructions cause the communication terminal not to make the request of the processing apparatus for transmission of the instruction image data, when an object which is located on an image displayed on the display and on which the instructed point on the display is located is displaced.

### Claim 17 (depends on 14)

The non-transitory storage medium according to claim 14, wherein when executed by the computer, the plurality of instructions cause the communication terminal to make the request of the processing apparatus for transmission of the instruction image data, when an animation indicating a degree of progress of a processing executed by the processing apparatus, as an object to be displaced on an image displayed on the display, is displayed on the display.

### Claim 18 (depends on 14)

The non-transitory storage medium according to claim 14, wherein when executed by the computer, the plurality of instructions cause the communication terminal to display an image indicating that the input event on the input device is not accepted, on the display based on the reception of the notification.

### Claim 19

A communication terminal, comprising:
a display;
an input device configured to detect an instructed point which is a point on the display and at which an instruction is provided with an input object;
a communication device configured to communicate with a processing apparatus; and
a controller configured to:
receive instruction image data from the processing apparatus, the instruction image data being for displaying, on the display, an instruction image for instructing a processing to be executed by the processing apparatus;
display the instruction image on the display based on the received instruction image data;
identify an input event on the input device in response to an instruction provided on the input device with the input object in a state in which the instruction image is displayed; and
transmit, to the processing apparatus, the identified input event and coordinate information about the instructed point on the display which is detected by the input device in a state in which the instruction image is displayed,
the controller being configured to execute a control using at least one of a software processing and a hardware processing.

### Claim 20 (depends on 19)

The communication terminal according to claim 19, wherein the controller is configured to:
make a request of the processing apparatus for transmission of the instruction image data; and
receive the instruction image data transmitted from the processing apparatus in response to the request of the processing apparatus for transmission of the instruction image data.

### Claim 21

A processing apparatus controllable by a communication terminal,
the communication terminal comprising: a display; an input device configured to detect an instructed point which is a point on the display and at which an instruction is provided with an input object; and a communication device configured to communicate with the processing apparatus,
the processing apparatus comprising a controller configured to:
transmit instruction image data to the communication terminal, the instruction image data being for displaying, on the display, an instruction image for instructing a processing to be executed by the processing apparatus;
receive, from the communication terminal, an input event on the input device and coordinate information about the instructed point on the display which is detected by the input device; and
execute a processing based on the received input event and coordinate information.

### Claim 22 (depends on 21)

The processing apparatus according to claim 21, wherein the controller is configured to:
transmit identification information for identifying the instruction image data, to the communication terminal when transmitting the instruction image data;
acquire, from the communication terminal, identification information about instruction image data representative of an instruction image currently displayed on the display of the communication terminal;
determine whether the acquired identification information coincides with identification information about image data representative of an image currently displayed on the processing apparatus; and
execute a processing based on the received input event when the acquired identification information coincides with the identification information about the image data representative of the image currently displayed on the processing apparatus.

### Claim 23 (depends on 22)

The processing apparatus according to claim 22, wherein the controller is configured to transmit, to the communication terminal, information indicating that the acquired identification information does not coincide with the identification information about the image data representative of the image currently displayed on the processing apparatus, when the acquired identification information does not coincide with the identification information about the image data representative of the image currently displayed on the processing apparatus.

### Claim 24 (depends on 21)

The processing apparatus according to claim 21, further comprising an apparatus-side display; and an apparatus-side input device configured to detect an instructed point which is a point on the apparatus-side display and at which an instruction is provided with an input object,
wherein the controller is configured to:
receive, from the communication terminal, a request of the processing apparatus for transmission of the instruction image data; and
transmit image data representative of an image currently displayed on the apparatus-side display, to the communication terminal, based on the reception of the request of the processing apparatus for transmission of the instruction image data.

### Claim 25 (depends on 24)

The processing apparatus according to claim 24, further comprising a storage device,
wherein the controller is configured to, when an image displayed on the apparatus-side display is switched in a state in which an image based on image data having not been transmitted to the communication terminal is displayed on the apparatus-side display, store, into the storage device, the image data representative of the image displayed before the image is switched.

### Claim 26 (depends on 25)

The processing apparatus according to claim 25, wherein the controller is configured to, based on a reception of the request of the transmission of the instruction image data in a state in which the image data is stored in the storage device before the switch, transmit the image data stored before the switch, to the communication terminal.

### Claim 27 (depends on 21)

The processing apparatus according to claim 21, further comprising an apparatus-side display; and an apparatus-side input device configured to detect an instructed point which is a point on the apparatus-side display and at which an instruction is provided with an input object,
wherein the controller is configured to transmit, to the communication terminal, image data representative of an image currently displayed on the apparatus-side display.

### Claim 28 (depends on 21)

The processing apparatus according to claim 21, wherein the controller is configured to:
receive a request of the processing apparatus for transmission of the instruction image data; and
transmit the instruction image data to the communication terminal in response to the received request of the processing apparatus for transmission of the instruction image data.

### Claim 29 (depends on 28)

The processing apparatus according to claim 28, further comprising an apparatus-side display; and an apparatus-side input device configured to detect an instructed point which is a point on the apparatus-side display and at which an instruction is provided with an input object,
wherein an image to be displayed on the apparatus-side display comprises an object which is displaceable, and
wherein the controller is configured to:
transmit, to the communication terminal, image data representative of an image currently displayed on the apparatus-side display, based on the reception of the request of the processing apparatus for transmission of the instruction image data;
when the request of the processing apparatus for transmission of the instruction image data is received from the communication terminal in a state in which an object on an image currently displayed on the apparatus-side display is being displaced, wait for the transmission of the instruction image data until the displacement of the object is completed;
upon completion of the displacement, transmit, to the communication terminal, image data representative of an image displayed at the completion of the displacement.

### Claim 30 (depends on 28)

The processing apparatus according to claim 28, further comprising an apparatus-side display; and an apparatus-side input device configured to detect an instructed point which is a point on the apparatus-side display and at which an instruction is provided with an input object,
wherein an image to be displayed on the apparatus-side display comprises an object which is displaceable, and
wherein the controller is configured to:
transmit image data representative of an image currently displayed on the apparatus-side display, to the communication terminal based on the reception of the request of the processing apparatus for transmission of the instruction image data; and
transmit, to the communication terminal, a notification indicating that an image currently displayed on the apparatus-side display contains an animation image, based on a fact that the image currently displayed on the apparatus-side display contains the animation image.
