# US 11,907,413 B2 · Play kitchen system using smart device based on virtual augmented reality and short range wireless communication

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,907,413 B2
- Document ID: US11907413B2
- Status: Granted
- Kind: B2
- Application number: 17/045,177
- Filed: 2019-04-02
- Published: 2021-06-24
- Granted: 2024-02-20
- Earliest claimed: 2018-04-05
- Applicants: Ftcubic Co., Ltd.
- Inventors: Se Jung Kim
- Web page: https://patentgrep.com/patent/US11907413B2

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/35: Details of game servers
- A63F 13/655: by importing photos, e.g. of the player
- A63F 13/90: Constructional details or arrangements of video game devices not provided for in groups A63F13/20 or A63F13/25, e.g. housing, wiring, connections or cabinets
- A63H 33/3055: {Ovens, or other cooking means}
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06K 7/10297: {arrangements for handling protocols designed for non-contact record carriers such as RFIDs NFCs, e.g. ISO/IEC 14443 and 18092 (cryptographic protocols H04L9/00; network security protocols H04L63/00; real-time communication protocols in data switching networks H04L65/00; network protocols for data switching network services H04L67/00)}
- G06K 7/1417: {2D bar codes}
- G06K 19/06037: {multi-dimensional coding}
- G06K 19/07758: {arrangements for adhering the record carrier to further objects or living beings, functioning as an identification tag}
- G06Q 10/06: Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06T 7/50: Depth or shape recovery
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 10/40: Extraction of image or video features
- G06V 20/20: in augmented reality scenes
- G09B 5/06: with both visual and audible presentation of the material to be studied
- G09B 19/003: {Repetitive work cycles; Sequence of movements}
- G09B 19/24: Use of tools
- H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04R 1/028: {associated with devices performing functions other than acoustics, e.g. electric candles}
- H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
- A63H 2200/00: Computerized interactive toys, e.g. dolls
- H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- H04R 2499/11: Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
- H04R 2499/15: Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63H: TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

An embodiment of the present invention comprises, a play kitchen toy body in which toy cooking tools form one set, a playhouse set, which is a set of toy tools to be used for cooking in a play kitchen by using the play kitchen toy body, a user terminal by which a user uses a play kitchen through the play kitchen toy body and the playhouse set, and in which a play kitchen assistant application is installed, the application extracting a unique toy number of the toy tools of the playhouse set so as to transmit same together with a play kitchen type to a play kitchen assistant server, and receiving, from the play kitchen assistant server, play kitchen assistant content matched to the unique toy number and outputting same, the play kitchen assistant server extracting play kitchen guide content matched to the play kitchen type and the unique toy number received from the user terminal so as to provide same to the user terminal, and a mirror coupled to a toy sink included in the play kitchen toy body, wherein the user terminal can photograph a play kitchen toy reflected through the mirror so as to extract a unique toy number of the play kitchen toy detected from a captured mirror image.

## Claims

### Claim 1

A kitchen play system using a smart device based on virtual reality and short-range wireless communication, the kitchen play system comprising:
a kitchen play toy main body forming a set of toy cooking tools;
a playing house set of toy props used in kitchen play on the kitchen play toy main body;
a user terminal used by a user of the kitchen play using the kitchen play toy main body and the playing house set, a kitchen play helper app installed on the user terminal, the kitchen play helper app extracting a toy unique number of a toy prop of the playing house set, transmitting the extracted toy unique number, along with a kind of the kitchen play, to a kitchen play helper server, receiving a kitchen play helper content matching the toy unique number from the kitchen play helper server, and outputting the kitchen play helper content;
the kitchen play helper server extracting a kitchen play guide content matching the kind of the kitchen play and the toy unique number received from the user terminal and providing the extracted kitchen play guide content to the user terminal; and
a mirror coupled to a toy sink included in the kitchen play toy main body, wherein the user terminal captures a kitchen play toy reflected by the mirror and extracts a toy unique number of the kitchen play toy detected from the captured mirror image, wherein the kitchen play helper server includes:
a server communication unit communicating with the user terminal;
a kitchen play helper content storage unit storing the kitchen play helper content matching the toy unique number and the kind of the kitchen play; and
a kitchen play helper content providing unit extracting a kitchen play guide content matching the toy unique number and the kind of the kitchen play received from the user terminal and providing the extracted kitchen play guide content to the user terminal, wherein the kitchen play helper content storage unit includes:
a kitchen play order guide content storage DB storing, per toy unique number, a kitchen play order guide content which needs to be performed next in order according to the kind of the kitchen play; and
a kitchen play phrase/retry content storage DB storing, per toy unique number, the kitchen play phrase/retry content to phrase or instruct to retry the kitchen play according to the state of the kitchen play toy according to the kind of the kitchen play.

### Claim 2 (depends on 1)

The kitchen play system of claim 1, wherein the mirror is hinged to the toy sink to allow an angle of the reflection to be adjusted towards a toy cutting board placed on the toy sink.

### Claim 3 (depends on 1)

The kitchen play system of claim 1, wherein the kitchen play helper content includes one or more of:
a kitchen play order guide content to guide kitchen play which needsto be performed next in order when any one kitchen play toy is detected; and
a kitchen play phrase/retry content to phrase or instruct to retry the kitchen play according to a stat: of the kitchen play toy detected.

### Claim 4 (depends on 1)

The kitchen play system of claim 1, wherein the kitchen play helper content includes any, one or more of a still image, a video, a virtual reality graphic, an augmented reality graphic, or a guidance voice.

### Claim 5 (depends on 1)

The kitchen play system of claim 1, wherein the kitchen play helper content storage unit assigns a combination of the kitchen play order guide content and the kitchen play phraselretry content, different per toy unique number, according to the kind of the kitchen play.

### Claim 6 (depends on 1)

The kitchen play system of claim 1, wherein the kitchen play toy main body includes:
a default toy main body including a default toy to allow a user to manually enjoy the kitchen play; and
an upgrade toy kit including an upgrade toy to allow the user to enjoy the kitchen play by device control, wherein the upgrade toy kit includes a short-range wireless communication module performing wireless communication with the user terminal via short-range wireless communication.

### Claim 7 (depends on 6)

The kitchen play system of claim 6, wherein the upgrade toy wirelessly transmits a user device control value resultant from control by the user, along with the toy unique number of the upgrade toy, to the user terminal, and wherein the user terminal wirelessly transmits the user device control value and the toy unique number of the upgrade toy received from the upgrade toy to the kitchen play helper server.

### Claim 8 (depends on 7)

The kitchen play system of claim 7, wherein the kitchen play helper content storage unit includes an upgrade toy kit default storage DB in which a device control default value is assigned to each toy unique number of upgrade toy depending on the kind of the kitchen play, and wherein the kitchen play helper content providing unit compares the user device control value received from the user terminal with the device control default value stored in the upgrade toy kit default value storage DB and provides the kitchen play helper content.

### Claim 9 (depends on 8)

The kitchen play system of claim 8, wherein the kitchen play helper content providing unit compares the user device control value received from the user terminal with the toy unique numberof the upgrade toy stored in the upgrade toy kit default value storage DB and, when the values differ from each other, notifies the user terminal that device control has been done wrong.

### Claim 10 (depends on 1)

The kitchen play system of claim 1, wherein the user terminal includes:
a terminal communication unit communicating with the kitchen play helper server;
a terminal speaker outputting the kitchen play helper content as an audio;
a terminal display unit displaying the kitchen play helper content;
a toy unique number extracting unit extracting the toy unique number of the kitchen play toy used in the kitchen play performed via the kitchen play toy main body using any one of a toy shape feature point, a QR code attached to the toy, and an NEC tag attached to the toy; and
a terminal controller to drive the kitchen play helper app transmitting the extracted toy unique number to the kitchen play helper server via the terminal communication unit, receiving the kitchen play helper content matching the toy unique number from the kitchen play helper server via the terminal communication unit, and outputting the kitchen play helper content.

### Claim 11 (depends on 10)

The kitchen play system of claim 10, wherein upon extracting the toy unique number using the toy shape feature point, the user terminal includes:
a camera recording the toy of the kitchen play performed via the kitchen play toy main body; and
a toy shape feature point database storing the toy unique number assigned per toy shape feature point, and wherein the toy unique number extracting unit extracts the toy shape feature point of the kitchen play toy used in the kitchen play from the kitchen play image recorded via the camera and extracts the toy unique number assigned to the extracted toy shape feature point from the toy shape feature point database.

### Claim 12 (depends on 10)

The kitchen play system of claim 10, upon extracting the toy unique number using the QR code, the user terminal includes:
a camera capturing a QR code attached to the toy of the kitchen play performed via the kitchen play toy main body; and
a QR code database storing the toy unique number assigned per QR identification information, wherein the toy unique number extracting unit reads QR identification information of the QR code attached to the kitchen play toy captured via the camera and extracts the toy unique number assigned to the read QR identification information from the QR code database.

### Claim 13 (depends on 10)

The kitchen play system of claim 10, wherein upon extracting the toy unique number using the NFC tag, the user terminal includes:
an NFC reader performing NFC communication with the NFC tag attached to the kitchen play toy and reading NFC identification information; and
an NFC tag database storing the toy unique number assigned per NFC identification information, and wherein the toy unique number extracting unit extracts the toy unique number assigned to the NFC identification information read via the NFC reader from the NFC tag database.