# US 2025/0149029 A1 · Artificial intelligence assistance for an audio, video and control system using room environment contextualization and oral command inferencing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0149029 A1
- Document ID: US20250149029A1
- Status: Published application
- Kind: A1
- Application number: 18/927,569
- Filed: 2024-10-25
- Published: 2025-05-08
- Earliest claimed: 2023-11-07
- Assignees: Qsc, LLC
- Applicants: Qsc, LLC
- Inventors: Derek Wearin Lieb; Christopher Jaynes; Matthew Skogmo; Thomas Bauer
- Web page: https://patentgrep.com/patent/US20250149029A1

## CPC

- G10L 15/183: using context dependencies, e.g. language models
- G10L 15/063: {Training}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An audio, video and control (“AVC”) operating system is implemented on an AVC processing core coupled to one or more peripheral devices. Using a large learning model (“LLM”) module, the AVC system detects audio signals obtained from a user and infers oral commands from the audio signals. Thereafter, one or more actions corresponding to the oral commands are performed on the peripheral devices and/or the AVC processing core. In another embodiment, the AVC system obtains contextual awareness data of a room environment in which the AVC operating system functions. Thereafter, based upon the contextual awareness data, the system performs actions on the peripheral devices or AVC processing core.

## Claims

### Claim 1

A computer-implemented method, comprising:
implementing an audio, video and control (“AVC”) operating system on an AVC processing core communicably coupled to one or more peripheral devices, the AVC processing core being configured to manage and control functionality of the peripheral devices,
wherein the AVC operating system is adapted to perform actions using oral commands detected within audio signals, the oral commands corresponding to the actions;
detecting, using a large language model (“LLM”) module communicably coupled to the AVC processing core, one or more audio signals obtained from a user;
inferring, using the LLM module, one or more oral commands from the audio signals; and
performing actions on the peripheral devices or AVC processing core which correspond to the oral commands.

### Claim 2 (depends on 1)

The computer-implemented claim as defined in claim 1, wherein inferring the oral commands comprises:
analyzing, using the LLM, the audio signals to determine which oral commands the audio signals most closely match; and
performing actions on the peripheral devices or AVC processing core which correspond to the matched oral commands.

### Claim 3 (depends on 1)

The computer-implemented method as defined in claim 1, wherein the LLM module executes a preconfigured command set to perform actions on the peripheral devices.

### Claim 4 (depends on 1)

The computer-implemented method as defined in claim 1, wherein the LLM module executes a taught command set to perform actions on the peripheral devices, the taught command set being taught by the user.

### Claim 5 (depends on 4)

The computer-implemented method as defined in claim 4, wherein the taught command set is obtained from the user via a web interface.

### Claim 6 (depends on 4)

The computer-implemented method as defined in claim 4, wherein the taught command set is obtained from the user via a listening device.

### Claim 7 (depends on 4)

The computer-implemented method as defined in claim 4, wherein the AVC processing core communicates the taught command set to one or more secondary AVC processing cores, thereby teaching the secondary AVC processing cores the taught command set.

### Claim 8 (depends on 1)

The computer-implemented method as defined in claim 1, wherein the LLM module is accessed from a cloud service, local network service, or on the AVC processing core.

### Claim 9

A system, comprising:
one or more peripheral devices; and
an audio, video and control (“AVC”) processing core communicably coupled to the peripheral devices, the AVC processing core having an AVC operating system executable thereon to manage and control functionality of the peripheral devices,
wherein the AVC operating system is adapted to perform actions using oral commands detected within audio signals, the oral commands corresponding to the actions,
wherein the AVC operating system is configured to perform operations comprising:
detecting, using a large language model (“LLM”) module communicably coupled to the AVC processing core, one or more audio signals obtained from a user;
inferring, using the LLM module, one or more oral commands from the audio signals; and
performing actions on the peripheral devices or AVC processing core which correspond to the oral commands.

### Claim 10 (depends on 9)

The system as defined in claim 9, wherein inferring the oral commands comprises:
analyzing, using the LLM, the audio signals to determine which oral commands the audio signals most closely match; and
performing actions on the peripheral devices or AVC processing core which correspond to the matched oral commands.

### Claim 11 (depends on 9)

The system as defined in claim 9, wherein the LLM module executes a preconfigured command set to perform actions on the peripheral devices.

### Claim 12 (depends on 9)

The system as defined in claim 9, wherein the LLM module executes a taught command set to perform actions on the peripheral devices, the taught command set being taught by the user.

### Claim 13 (depends on 12)

The system as defined in claim 12, wherein the taught command set is obtained from the user via a web interface.

### Claim 14 (depends on 12)

The system as defined in claim 12, wherein the taught command set is obtained from the user via a listening device.

### Claim 15 (depends on 12)

The system as defined in claim 12, wherein the AVC processing core communicates the taught command set to one or more secondary AVC processing cores, thereby teaching the secondary AVC processing cores the taught command set.

### Claim 16 (depends on 9)

The system as defined in claim 9, wherein the LLM module is accessed from a cloud service, local network service, or on the AVC processing core.

### Claim 17

A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations comprising:
implementing an audio, video and control (“AVC”) operating system on an AVC processing core communicably coupled to one or more peripheral devices, the AVC processing core being configured to manage and control functionality of the peripheral devices,
wherein the AVC operating system is adapted to perform actions using oral commands detected within audio signals, the oral commands corresponding to the actions,
wherein the AVC operating system is configured to perform operations comprising:
detecting, using a large language model (“LLM”) module communicably coupled to the AVC processing core, one or more audio signals obtained from a user;
inferring, using the LLM module, one or more oral commands from the audio signals; and
performing actions on the peripheral devices or AVC processing core which correspond to the oral commands.

### Claim 18 (depends on 17)

The computer-readable storage medium as defined in claim 17, wherein inferring the oral commands comprises:
analyzing, using the LLM, the audio signals to determine which oral commands the audio signals most closely match; and
performing actions on the peripheral devices or AVC processing core which correspond to the matched oral commands.

### Claim 19 (depends on 17)

The computer-readable storage medium as defined in claim 17, wherein the LLM module executes a preconfigured command set to perform actions on the peripheral devices.

### Claim 20 (depends on 17)

The computer-readable storage medium as defined in claim 17, wherein the LLM module executes a taught command set to perform actions on the peripheral devices, the taught command set being taught by the user.

### Claim 21 (depends on 20)

The computer-readable storage medium as defined in claim 20, wherein the taught command set is obtained from the user via a web interface.

### Claim 22 (depends on 20)

The computer-readable storage medium as defined in claim 20, wherein the taught command set is obtained from the user via a listening device.

### Claim 23 (depends on 20)

The computer-readable storage medium as defined in claim 20, wherein the AVC processing core communicates the taught command set to one or more secondary AVC processing cores, thereby teaching the secondary AVC processing cores the taught command set.

### Claim 24 (depends on 20)

The computer-readable storage medium as defined in claim 20, wherein the LLM module is accessed from a cloud service, local network service, or on the AVC processing core.

### Claim 25

A computer-implemented method, comprising:
implementing an audio, video and control (“AVC”) operating system on an AVC processing core communicably coupled to one or more peripheral devices, the AVC processing core being configured to manage and control functionality of the peripheral devices,
obtaining contextual awareness data of a room environment in which the AVC operating system functions; and
based upon the contextual awareness data, performing actions on the peripheral devices or AVC processing core.

### Claim 26 (depends on 25)

The computer-implemented method as defined in claim 25, wherein the contextual awareness data is video data.

### Claim 27 (depends on 25)

The computer-implemented method as defined in claim 25, wherein the contextual awareness data is audio data.

### Claim 28 (depends on 25)

The computer-implemented method as defined in claim 25, wherein the contextual awareness data is textual data.

### Claim 29 (depends on 28)

The computer-implemented method as defined in claim 28, wherein the textual data is obtained from the user via a user interface.

### Claim 30 (depends on 25)

The computer-implemented method as defined in claim 25, wherein the contextual awareness data is status data of the peripheral devices.

### Claim 31 (depends on 25)

The computer-implemented method as defined in claim 25, wherein the contextual awareness data is geo-spatial data of the room environment.

### Claim 32

A system, comprising:
one or more peripheral devices; and
an audio, video and control (“AVC”) processing core communicably coupled to the peripheral devices, the AVC processing core having an AVC operating system executable thereon to manage and control functionality of the peripheral devices,
wherein the AVC operating system is adapted to perform operations comprising:
obtaining contextual awareness data of a room environment in which the AVC operating system functions; and
based upon the contextual awareness data, performing actions on the peripheral devices or AVC processing core.

### Claim 33 (depends on 32)

The system as defined in claim 32, wherein the contextual awareness data is video data.

### Claim 34 (depends on 32)

The system as defined in claim 32, wherein the contextual awareness data is audio data.

### Claim 35 (depends on 32)

The system as defined in claim 32, wherein the contextual awareness data is textual data.

### Claim 36 (depends on 35)

The system as defined in claim 35, wherein the textual data is obtained from the user via a user interface.

### Claim 37 (depends on 32)

The system as defined in claim 32, wherein the contextual awareness data is status data of the peripheral devices.

### Claim 38 (depends on 32)

The system as defined in claim 32, wherein the contextual awareness data is geo-spatial data of the room environment.

### Claim 39

A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations comprising:
implementing an audio, video and control (“AVC”) operating system on an AVC processing core communicably coupled to one or more peripheral devices, the AVC processing core being configured to manage and control functionality of the peripheral devices,
obtaining contextual awareness data of a room environment in which the AVC operating system functions; and
based upon the contextual awareness data, performing actions on the peripheral devices or AVC processing core.

### Claim 40 (depends on 39)

The computer-readable storage medium as defined in claim 39, wherein the contextual awareness data is video data.

### Claim 41 (depends on 39)

The computer-readable storage medium as defined in claim 39, wherein the contextual awareness data is audio data.

### Claim 42 (depends on 39)

The computer-readable storage medium as defined in claim 39, wherein the contextual awareness data is textual data.

### Claim 43 (depends on 42)

The computer-readable storage medium as defined in claim 42, wherein the textual data is obtained from the user via a user interface.

### Claim 44 (depends on 39)

The computer-readable storage medium as defined in claim 39, wherein the contextual awareness data is status data of the peripheral devices.

### Claim 45 (depends on 39)

The computer-readable storage medium as defined in claim 39, wherein the contextual awareness data is geo-spatial data of the room environment.
