# US 12,014,750 B2 · Audio analysis of body worn camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,014,750 B2 - Document ID: US12014750B2 - Status: Granted - Kind: B2 - Application number: 18/180,652 - Filed: 2023-03-08 - Published: 2023-07-13 - Granted: 2024-06-18 - Earliest claimed: 2020-12-16 - Assignees: Truleo, Inc. - Applicants: Truleo, Inc. - Inventors: Tejas Shastry; Svyatoslav Vergun; Colin Brochtrup; Matthew Goldey - Web page: https://patentgrep.com/patent/US12014750B2 ## CPC - G06F 40/35: Discourse or dialogue representation - G10L 25/57: for processing of video signals - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 17/00: Speaker identification or verification techniques - G10L 25/87: Detection of discrete points within a voice signal - G06F 40/253: Grammatical analysis; Style critique - G10L 17/26: Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Machine natural language processing to analyze language in apparatus, systems, and methods of using are provided. Audio from camera footage can be transcribed in one exemplary method includes extracting at least one audio segment from a body camera video track, detecting voice activity to identify starting and ending timestamps of voice, transcribing the at least one audio segment to identify and separate audio of at least a first speaker, and scoring the audio of the first speaker to identify interactions of interest. Audio could be analyzed and scored to record verbal performance, respectfulness, wellness, etc. and speakers from the audio can be detected. ## Claims ### Claim 1 A method of using machine natural language processing to analyze language in transcribed camera footage comprising: extracting at least one audio segment from a body camera video track; detecting voice activity to identify starting and ending timestamps of voice; transcribing the at least one audio segment; identifying audio of at least one speaker; separating the audio of the at least one speaker; scoring the audio of the at least one speaker after separation to identify interactions of interest; wherein a voice detection model is used to analyze the at least one audio segment to identify the starting and ending timestamps of voice; and, wherein each word in the at least one audio segment is assigned a start and stop time. ### Claim 2 (depends on 1) The method of claim 1 wherein the at least one speaker is a figure of authority, including one of a: police officer, emergency technician, guard, soldier, doctor, or first responder. ### Claim 3 (depends on 2) The method of claim 2 wherein the interactions of interest include whether the figure of authority is escalating or de-escalating a situation. ### Claim 4 (depends on 2) The method of claim 2 wherein the interactions of interest include whether the figure of authority is using respectful language or negative language. ### Claim 5 (depends on 2) The method of claim 2 wherein the scoring includes analyzing for word disfluencies or filler words to analyze speaker confidence. ### Claim 6 (depends on 2) The method of claim 2 wherein the figure of authority is identified based on voice quality. ### Claim 7 (depends on 6) The method of claim 6 wherein the transcribing identifies whether audio of at least an other speaker is included on the at least one audio segment. ### Claim 8 (depends on 1) The method of claim 1 wherein the method further includes: identifying events that may have occurred in the body camera video track based on language cues in the at least one audio segment. ### Claim 9 (depends on 8) The method of claim 8 wherein the method further includes: compressing the body camera video track based on the events. ### Claim 10 (depends on 1) The method of claim 1 wherein the method is performed in real-time. ### Claim 11 (depends on 1) The method of claim 1 wherein an intent classification model utilizes a deep-learned transformer architecture. ### Claim 12 (depends on 11) The method of claim 11 wherein a sentiment analysis is performed to output labels with probabilities, with the labels comprising either a “positive” label or a “negative” label. ### Claim 13 (depends on 12) The method of claim 12 wherein the sentiment analysis further comprises analyzing the at least one audio segment for word disfluencies. ### Claim 14 (depends on 13) The method of claim 13 wherein the sentiment analysis further comprises giving weights that correlate with positive or negative sentiment. ### Claim 15 (depends on 11) The method of claim 11 wherein a sentiment analysis is performed to output labels with probabilities, with the labels comprising either a “positive” label or a “negative” label. ### Claim 16 (depends on 2) The method of claim 2 wherein the at least one speaker includes a first police officer wearing a body camera from which the body camera video track was extracted and a second police officer; wherein the second police officer is producing a second audio that is included on the at least one audio segment; wherein the method identifies the first police officer as the at least one speaker; and wherein the method separates the audio of the first police officer as the at least one speaker. ### Claim 17 (depends on 1) The method of claim 1 wherein, if a speaker change occurs, the at least one audio segment is re-segmented to include segments of only the at least one speaker. ### Claim 18 (depends on 1) The method of claim 1 wherein the voice detection model is based on audio analysis and does not rely on visual data to determine the start and stop time. ### Claim 19 (depends on 18) The method of claim 18 wherein, if a speaker change occurs, the at least one audio segment is re-segmented to include segments of only the at least one speaker.