# US 11,445,265 B2 · Artificial intelligence device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,265 B2 - Document ID: US11445265B2 - Status: Granted - Kind: B2 - Application number: 16/657,733 - Filed: 2019-10-18 - Published: 2020-02-20 - Granted: 2022-09-13 - Earliest claimed: 2019-09-20 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Taeho Lee; Boseop Kim - Web page: https://patentgrep.com/patent/US11445265B2 ## CPC - H04N 21/4884: {for displaying subtitles} - G06V 10/764: using classification, e.g. of video objects - G06V 10/82: using neural networks - G10L 15/16: using artificial neural networks - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/24: Speech recognition using non-acoustical features - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - H04N 21/4398: {involving reformatting operations of audio signals (details of audio signal transcoding G10L19/173)} - H04N 21/4666: {using neural networks, e.g. processing the feedback provided by the user} - G10L 2015/226: {using non-speech characteristics} - G10L 2015/228: {of application context} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract An AI device is provided. The AI device includes a content output interface to output video data contained in content and voice data contained in the content, and a processor to control the content output interface to acquire a voice recognition result by providing, to a voice recognition model, content extraction information including at least one of video information acquired from the video data in the content or tag information of the content and the voice data, and control the content output interface to output the voice recognition result. ## Claims ### Claim 1 An artificial intelligence (AI) device comprising: a content output interface configured to output video data contained in content and voice data contained in the content; and a processor configured to: acquire a voice recognition result by providing, to a voice recognition model, content extraction information including at least one of video information acquired from the video data in the content or tag information of the content and the voice data, and control the content output interface to output the voice recognition result, wherein the voice recognition model includes a recurrent neural network (RNN), and wherein the processor is configured to: set, for the RNN, an initial hidden state corresponding to the tag information of the content, and provide the voice data to the RNN having the initial hidden state, to acquire the voice recognition result corresponding to the voice data. ### Claim 2 An artificial intelligence (AI) device comprising: a content output interface configured to output video data contained in content and voice data contained in the content; and a processor configured to: acquire a voice recognition result by providing, to a voice recognition model, content extraction information including at least one of video information acquired from the video data in the content or tag information of the content and the voice data, and control the content output interface to output the voice recognition result, wherein the voice recognition model includes a recurrent neural network (RNN), and wherein the processor is configured to: set, for the RNN, an initial hidden state corresponding to the tag information of the content and video information extracted from a first scene of the content; and provide first voice data, which is output from the first scene, to the RNN having the initial hidden state, to acquire a voice recognition result corresponding to the first voice data. ### Claim 3 (depends on 1) The AI device of claim 1, wherein the video information includes: at least one of an object in the video data, a text in the video data, or description information of a scene. ### Claim 4 (depends on 1) The AI device of claim 1, wherein the tag information included in the content extraction information includes: at least one of a title of the content, a subject of the content, or description of the content. ### Claim 5 (depends on 1) The AI device of claim 1, wherein the RNN is: trained by using first training voice data corresponding to first content extraction information and a language labeled on the first training voice data, in a state that a first initial hidden state corresponding to the first content extraction information is set; and trained by using second training voice data corresponding to second content extraction information and a language labeled on the second training voice data, in a state that a second initial hidden state corresponding to the second content extraction information is set. ### Claim 6 An operating method of an artificial intelligence (AI) device, the operating method comprising: acquiring content extraction information including at least one of video information acquired from video data contained in content or tag information of the content; acquiring a voice recognition result by providing, to a voice recognition model, the content extraction information and voice data contained in the content; and outputting the video data contained in the content, the voice data contained in the content, and the voice recognition result, wherein the voice recognition model includes a recurrent neural network (RNN), and wherein the acquiring of the voice recognition result includes: setting, for the RNN, an initial hidden state corresponding to the tag information of the content; and providing the voice data to the RNN having the initial hidden state, to acquire the voice recognition result corresponding to the voice data. ### Claim 7 An operating method of an artificial intelligence (AI) device, the operating method comprising: acquiring content extraction information including at least one of video information acquired from video data contained in content or tag information of the content; acquiring a voice recognition result by providing, to a voice recognition model, the content extraction information and voice data contained in the content; and outputting the video data contained in the content, the voice data contained in the content, and the voice recognition result, wherein the voice recognition model includes a recurrent neural network (RNN), and wherein the acquiring of the voice recognition result includes: setting, for the RNN, an initial hidden state corresponding to the tag information of the content and video information extracted from a first scene of the content; and providing first voice data, which is output from the first scene, to the RNN having the initial hidden state, to acquire a voice recognition result corresponding to the first voice data. ### Claim 8 (depends on 6) The operating method of claim 6, wherein the video information includes: at least one of an object in the video data, a text in the video data, or description information of a scene. ### Claim 9 (depends on 6) The operating method of claim 6, wherein the tag information included in the content extraction information includes: at least one of a title of the content, a subject of the content, or description of the content. ### Claim 10 (depends on 6) The operating method of claim 6, wherein the RNN is: trained by using first training voice data corresponding to first content extraction information and a language labeled on the first training voice data, in a state that a first initial hidden state corresponding to the first content extraction information is set; and trained by using second training voice data corresponding to second content extraction information and a language labeled on the second training voice data, in a state that a second initial hidden state corresponding to the second content extraction information is set.