Back to results
US 2026/0195953 A1Published application
Action control system and information processing system
AssigneeSoftBank Group Corp.
InventorMasayoshi SON
Life of this patent
Estimating the term…
Abstract
An avatar is caused to perform an action appropriate for an action of a user. In an action control system, in a case in which an action of the avatar includes dreaming and the action determination unit determines dreaming, as the action of the avatar, the action determination unit creates an original event obtained by combining multiple pieces of event data among pieces of data in history data.
Claims20
A networked speech processing system comprising:
- a client terminal including:
- a microphone interface configured to capture audio input representing user speech,
- an audio codec configured to encode the captured audio input into compressed audio data according to a speech coding standard,
- a network interface configured to:
- establish a network session with a remote server over a packet network,
- encapsulate the compressed audio data into network packets, and
- transmit the network packets to the remote server,
- a speaker interface configured to output audio feedback to the user, and
- a display configured to render graphical content;
- the remote server communicatively coupled to the client terminal via the packet network, the remote server including:
- a server network interface configured to:
- receive the network packets from the client terminal, and
- extract the compressed audio data from the received network packets, an audio decoder configured to decode the compressed audio data to generate decoded audio samples, and
- processing circuitry configured to:
- process the decoded audio samples to generate a phoneme sequence, map the phoneme sequence to text data representing the user speech,
- analyze the text data to identify grammatical structures and vocabulary complexity,
- compute a language proficiency estimate based on the identified grammatical structures and vocabulary complexity,
- generate feedback content including assessment of the language proficiency estimate,
- generate avatar action parameters for presenting the feedback content via an animated avatar, and
- transmit the avatar action parameters to the client terminal;
- and
- wherein the client terminal is further configured to:
- receive the avatar action parameters from the remote server,
- render the animated avatar on the display, and
- output synthesized speech corresponding to the feedback content via the speaker interface.
22 more elements
A networked speech processing apparatus comprising:
- a wearable client device including:
- a microphone configured to capture audio waveforms representing speech of a user,
- an audio encoder implementing a speech compression algorithm configured to generate encoded speech packets from the captured audio waveforms,
- a wireless network interface configured to transmit the encoded speech packets over a wireless communication link to a server system,
- a speaker, and
- a display panel;
- the server system including:
- a network interface configured to receive the encoded speech packets,
- a speech decoder configured to decode the encoded speech packets into decoded speech samples,
- a speech recognition processor configured to convert the decoded speech samples into text data,
- a proficiency analysis module implemented by processing circuitry, the proficiency analysis module configured to:
- analyze the text data to determine grammatical accuracy,
- analyze the text data to determine vocabulary complexity,
- compute a proficiency score based on the grammatical accuracy and vocabulary complexity, and
- store the proficiency score in association with a user identifier,
- an avatar controller configured to:
- generate synthesized speech content at a difficulty level corresponding to the proficiency score, and
- generate avatar animation instructions for rendering an animated character on the display panel, and
- a transmitter configured to transmit the synthesized speech content and avatar animation instructions to the wearable client device;
- and
- wherein the wearable client device is configured to render the animated character on the display panel and output the synthesized speech content via the speaker.
18 more elements
A method for networked speech processing with language proficiency estimation, the method comprising:
- capturing, by a microphone of a client terminal, audio input representing user speech;
- encoding, by an audio codec, the captured audio input into compressed audio data according to a speech coding standard;
- transmitting, via a network interface, network packets containing the compressed audio data to a remote server over a packet network;
- receiving, by the remote server, the network packets and extracting the compressed audio data;
- decoding the compressed audio data to generate decoded audio samples;
- processing the decoded audio samples to generate text data representing the user speech;
- analyzing the text data to identify grammatical structures and vocabulary complexity;
- computing a language proficiency estimate based on the identified grammatical structures and vocabulary complexity;
- generating feedback content including assessment of the language proficiency estimate;
- generating avatar action parameters for presenting the feedback content via an animated avatar;
- transmitting the avatar action parameters to the client terminal;
- rendering, by the client terminal, the animated avatar on a display;
- and
- outputting, by the client terminal, synthesized speech corresponding to the feedback content via a speaker.
11 more elements
Dependencies are inferred from the claim text.