System, support method, server apparatus and communication program
AssigneesDaikin Industries, Ltd., Fairy Devices Inc.
InventorsMiyu Nishizawa, Masato Fujino
Life of this patent
Estimating the term…
Abstract
A system for converting speech data into text data includes one or more control units that determine whether or not the speech data being acquired can be converted into the text data, or whether or not a quality of the converted text data after conversion of the speech data being acquired deteriorates, and output a determination result.
Claims10
A system comprising:
- a first terminal;
- a second terminal physically distinct from and spatially separated from the first terminal;
- and
- a server apparatus configured to perform a bidirectional communication between the first terminal and the second terminal, wherein:
- the first terminal includes:
- a speech acquisition device configured to acquire audio data including speech data and environmental sound in real-time,
- a first processor configured to detect a state of the environmental sound and a state of the speech acquisition device and output a detection result indicating the state of the environmental sound and the state of the speech acquisition device, and
- a first communication device configured to transmit the acquired audio data and the detection result to the server apparatus, the server apparatus includes:
- a second communication device communicable with the first terminal and the second terminal and configured to receive the audio data and the detection result from the first terminal, and
- a second processor configured to:
- convert the speech data of the received audio data into text data in real-time;
- determine, before a communication with the first terminal ends, whether or not the speech data being acquired during the communication with the first terminal can be converted into the text data;
- determine that a quality of the text data after the conversion deteriorates due to an influence of the environmental sound in a case where a magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal is greater than or equal to a predetermined threshold value, or in a case where a ratio between a magnitude of the speech data and the magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal satisfies a predetermined condition;
- determine that the speech data acquired during the communication with the first terminal cannot be converted into the text data due to a malfunction of the speech acquisition device included in the first terminal in a case where both a magnitude of the speech data being acquired and a magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal are less than a predetermined threshold value;
- determine that the speech data acquired during the communication with the first terminal cannot be converted into the text data due to a deterioration of a communication state of the communication with the first terminal in a case where the audio data is not transmitted from the first terminal for a certain period of time and the communication with the first terminal is determined as being disconnected;
- and
- transmit a display instruction based on a determination result indicating that the quality of the text data after the conversion deteriorates, and a determination result indicating that the speech data cannot be converted into the text data to the second terminal via the second communication device,
- the display instruction includes a display content on the state of the environmental sound or the state of the speech acquisition device and a reason that the quality of the text data after the conversion deteriorates and a reason that the speech data cannot be converted into the text data, and
- the second terminal includes a third communication device communicable with the server apparatus, a display device, and a third processor configured to display the determination result of the server apparatus and the reason for the determination result on the display device in response to the display instruction received by the third communication device from the server apparatus.
A server apparatus comprising:
- a communication device configured to perform a bidirectional communication between a first terminal and a second terminal physically distinct from and spatially separated from the first terminal;
- a memory configured to store a program;
- and
- a processor configured to execute the program stored in the memory and perform a process including:
- converting speech data included in audio data received from the first terminal via the communication device into text data, the audio data further including environmental sound at the first terminal;
- determining, before a communication with the first terminal ends, whether or not the speech data being acquired during the communication with the first terminal can be converted into the text data based on a detection result received from the first terminal via the communication device, the detection result indicating a state of the environmental sound at the first terminal and a state of a speech acquisition device that acquires the audio data at the first terminal;
- determining that a quality of the text data after the conversion deteriorates due to an influence of the environmental sound in a case where a magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal is greater than or equal to a predetermined threshold value, or in a case where a ratio between a magnitude of the speech data and the magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal satisfies a predetermined condition;
- determining that the speech data acquired during the communication with the first terminal cannot be converted into the text data due to a malfunction of the speech acquisition device at the first terminal in a case where both a magnitude of the speech data being acquired and a magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal are less than a predetermined threshold value;
- determining that the speech data acquired during the communication with the first terminal cannot be converted into the text data due to a deterioration of a communication state of the communication with the first terminal in a case where the audio data is not transmitted from the first terminal for a certain period of time and the communication with the first terminal is determined as being disconnected;
- and
- transmitting a display instruction based on a determination result indicating that the quality of the text data after the conversion deteriorates, and a determination result indicating that the speech data cannot be converted into the text data to the second terminal via the communication device,
- wherein the display instruction includes a display content on the state of the environmental sound or the state of the speech recognition device and a reason that the quality of the text data after the conversion deteriorates and a reason that the speech data cannot be converted into the text data, thereby causing the second terminal to display the determination result and the reason for the determination result in response to the display instruction.
A non-transitory computer-readable storage medium having stored therein a program which, when executed by a computer of a server apparatus, causes the computer to perform a process including:
- performing a bidirectional communication between a first terminal and a second terminal physically distinct from and spatially separated from the first terminal via a communication device of the server apparatus;
- converting speech data included in audio data received from the first terminal via the communication device into text data, the audio data further including environmental sound at the first terminal;
- determining, before a communication with the first terminal ends, whether or not the speech data being acquired during the communication with in the first terminal can be converted into the text data based on a detection result received from the first terminal via the communication device, the detection result indicating a state of the environmental sound at the first terminal and a state of a speech acquisition device that acquires the audio data at the first terminal;
- determining that a quality of the text data after the conversion deteriorates due to an influence of the environmental sound in a case where a magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal is greater than or equal to a predetermined threshold value, or in a case where a ratio between a magnitude of the speech data and the magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal satisfies a predetermined condition;
- determining that the speech data acquired during the communication with the first terminal cannot be converted into the text data due to a malfunction of the speech acquisition device at the first terminal in a case where both a magnitude of the speech data being acquired and a magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal are less than a predetermined threshold value;
- determining that the speech data acquired during the communication with the first terminal cannot be converted into the text data due to a deterioration of a communication state of the communication with the first terminal in a case where the audio data is not transmitted from the first terminal for a certain period of time and the communication with the first terminal is determined as being disconnected;
- and
- transmitting a display instruction based on a determination result indicating that the quality of the text data after the conversion deteriorates, and a determination result indicating that the speech data cannot be converted into the text data to the second terminal via the communication device,
- wherein the display instruction includes a display content on the state of the environmental sound or the state of the speech recognition device and a reason that the quality of the text data after the conversion deteriorates and a reason that the speech data cannot be converted into the text data, thereby causing the second terminal to display the determination result and the reason for the determination result in response to the display instruction.
Dependencies are inferred from the claim text.