# US 2020/0005099 A1 · Display control system and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0005099 A1
- Document ID: US20200005099A1
- Status: Published application
- Kind: A1
- Application number: 16/538,267
- Filed: 2019-08-12
- Published: 2020-01-02
- Earliest claimed: 2012-12-21
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: Kazuhiro Watanabe; Yoichiro Sako; Akira Tange; Kazuyuki Sakoda; Kohei Asada; Takatoshi Nakamura; Mitsuru Takehara; Yasunori Kamada; Takayasu Kon; Kazunori Hayashi; Hiroyuki Hanaya; Yuki Koga; Tomoya Onuma
- Web page: https://patentgrep.com/patent/US20200005099A1

## CPC

- G06V 40/161: {Detection; Localisation; Normalisation}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/044: by capacitive means
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/60: Analysis of geometric attributes
- G06V 40/113: {Recognition of static hand signs}
- G06V 40/25: {Recognition of walking or running movements, e.g. gait recognition}
- H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

There is provided a display control system including a plurality of display units, an imaging unit configured to capture a subject, a predictor configured to predict an action of the subject according to a captured image captured by the imaging unit, a guide image generator configured to generate a guide image that guides the subject according to a prediction result from the predictor, and a display controller configured to, on the basis of the prediction result from the predictor, select a display unit capable of displaying an image at a position corresponding to the subject from the plurality of display units, and to control the selected display unit to display the guide image at the position corresponding to the subject.

## Claims

### Claim 1

(canceled)

### Claim 2

A system comprising:
an action history information acquirer for acquiring information of user motion around a table;
a guide image generator for generating one or more images suggesting an action of the user based on the acquired information; and
a display controller for controlling display of the one or more images on the table.

### Claim 3 (depends on 2)

The system according to claim 2, further comprising a predictor for generating one or more predicted actions of the user according the information of user motion.

### Claim 4 (depends on 3)

The system according to claim 3, wherein the guide image generator generates the one or more images based on, at least, one of the predicted actions.

### Claim 5 (depends on 2)

The system according to claim 2, further comprising a learning unit for learning one or more patterns of items placed on the table, and wherein the guide image generator generates the one or more images based on, at least, one of the patterns.

### Claim 6 (depends on 5)

The system according to claim 5, wherein the one or more patterns comprises a pattern of dishes.

### Claim 7 (depends on 5)

The system according to claim 5, wherein the one or more patterns comprises a pattern of cutlery.

### Claim 8 (depends on 2)

The system according to claim 2, further comprising one or more imaging units for generating the information of user motion.

### Claim 9 (depends on 2)

The system according to claim 2, further comprising one or more displays for displaying the one or more images on the table.

### Claim 10 (depends on 2)

The system according to claim 2, wherein the one or more images indicates cutlery corresponding to a dish on the table.

### Claim 11

A display control method comprising:
controlling acquiring information of user motion around a table;
controlling generation of one or more images suggesting an action of the user based on the acquired information; and
controlling displaying the one or more images on the table.

### Claim 12 (depends on 11)

The method according to claim 11, further comprising controlling generation of one or more predicted actions of the user according to the information of user motion, and wherein controlling generation of one or more images comprises controlling generation of one or more images based on, at least, one of the predicted actions.

### Claim 13 (depends on 11)

The method according to according to claim 11, further comprising imaging the user to generate the information of user motion.

### Claim 14 (depends on 11)

The method according to claim 11, further comprising displaying the one or more images on the table.

### Claim 15

A display control method comprising:
controlling acquiring information of user motion around a table and previously learned action patterns of dishes brought out in a restaurant and cutlery corresponding to such dishes;
controlling generation of one or more guide images based on the information of user motion, the one or more guide images suggesting an action of the user associated with at least one of providing dishes or providing service; and
controlling displaying the one or more guide images on the table.

### Claim 16 (depends on 15)

The method according to claim 15, wherein the step of controlling generation of one or more guide images comprises generating one or more predicted actions of the user according to the information of user motion.

### Claim 17 (depends on 15)

The method according to according to claim 15, further comprising imaging the user to generate the information of user motion.

### Claim 18 (depends on 15)

The method according to claim 15, further comprising displaying the one or more images on the table.

### Claim 19

A non-transitory computer-readable medium having stored thereon a computer-readable program for implementing a display control method comprising:
controlling acquiring information of user motion around a table;
controlling generation of one or more images suggesting an action of the user based on the acquired information; and
controlling displaying the one or more images on the table.

### Claim 20

A non-transitory computer-readable medium having stored thereon a computer-readable program for implementing a display control method comprising:
controlling acquiring information of user motion around a table and previously learned action patterns of dishes brought out in a restaurant and cutlery corresponding to such dishes;
controlling generation of one or more guide images based on the information of user motion, the one or more guided images suggesting an action of the user associated with at least one of providing dishes or providing service; and
controlling displaying the one or more guide images on the table.
