# US 10,375,237 B1 · Virtual communications assessment system in a multimedia environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,237 B1 - Document ID: US10375237B1 - Status: Granted - Kind: B1 - Application number: 15/702,518 - Filed: 2017-09-12 - Granted: 2019-08-06 - Earliest claimed: 2016-09-12 - Assignees: Verint Americas Inc. - Applicants: Verint Americas Inc. - Inventors: Jamie Richard Williams; Marc Calahan; Robert Barnes; Murali Nareddy - Web page: https://patentgrep.com/patent/US10375237B1 ## CPC - H04M 3/42221: {Conversation recording systems (at the subscriber's set H04M1/656)} - G06F 16/41: Indexing; Data structures therefor; Storage structures - G06F 16/61: Indexing; Data structures therefor; Storage structures - G06F 16/71: Indexing; Data structures therefor; Storage structures - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - G06V 40/168: {Feature extraction; Face representation} - G06V 40/172: {Classification, e.g. identification} - H04L 67/1036: Load balancing of requests to servers for services different from user content provisioning, e.g. load balancing across domain name servers - H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching - H04M 3/42314: {in private branch exchanges} - H04M 3/51: Centralised call answering arrangements requiring operator intervention {, e.g. call or contact centers for telemarketing} - H04M 3/5175: {Call or contact centers supervision arrangements} - H04M 3/5183: {Call or contact centers with computer-telephony arrangements} - H04M 3/5191: {interacting with the Internet} - H04M 7/0051: {where the data service is a multimedia messaging service} - H04L 65/1066: Session management - H04L 65/1104: Session initiation protocol [SIP] - H04M 3/50: Centralised arrangements for answering calls; Centralised arrangements for recording messages for absent or busy subscribers (H04M3/487 takes precedence; centralised dictation systems H04M11/10); {Centralised arrangements for recording messages} - H04M 2203/558: Databases - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system for data recording across a network includes a session border controller connecting incoming data from the network to an endpoint recorder. A load balancer is connected to the network between the session border controller and the endpoint and receives the incoming data from the session border controller, wherein the load balancer comprises computer memory and a processor configured to parse the incoming data into video data and audio data according to identification protocols accessible by the processor from the computer memory. A recording apparatus includes recording memory that receives the incoming data from the load balancer, stores a duplicate version of the incoming data in the recording memory, and connects the incoming data to the endpoint. ## Claims ### Claim 1 A recording system comprising: at least one agent computer configured to accommodate bi-directional communications over a network between the at least one agent computer and a remote computer, the at least one agent computer connected to a processor and memory configured with an archiving selection module; a first recording apparatus in data communication over the network with the at least one agent computer, the first recording apparatus configured to implement instructions received from the archiving selection module to: selectively gather screen shots of the at least one agent computer during the bi-directional communications and store the screen shots with a respective time stamp; selectively gather video data generated at the at least one agent computer during the bi-directional communications and store the video data with a video time stamp; selectively gather audio data generated at the at least one agent computer during the bi-directional communications and store the audio data with an audio time stamp; an archival system connected to the network for storing the screen shots, the video data, and the audio data after assembling a consolidated communication according to the time stamps as adjusted for drift errors. ### Claim 2 (depends on 1) A system according to claim 1, wherein the archiving selection module comprises computer implemented commands stored therein to effect the selective gathering according to a randomizing rule, a line of business rule, or by locating a third party tag within the bi-directional communications. ### Claim 3 (depends on 1) A system according to claim 1, wherein the first recording apparatus is positioned in a first private branch exchange local to the at least one agent computer, the system further comprising a second recorder in data communication with a customer computer in a second private branch exchange, the second recorder selectively gathering remote audio, remote video, and remote screen shots from the customer computer according to corresponding instructions from the archiving selection module. ### Claim 4 (depends on 1) A system according to claim 1, wherein the archiving selection module is stored in a data center between the at least one agent computer and the customer computer. ### Claim 5 (depends on 1) A system according to claim 1, wherein the at least one agent computer is in communication with a load balancer comprising computer memory and a processor configured to parse the bi-directional communications into screen data, video data and audio data according to identification protocols accessible to the processor. ### Claim 6 (depends on 1) A system according to claim 1, further comprising a respective portion of memory in the first recorder for audio data, video data, and screen shot data. ### Claim 7 (depends on 1) A system according to claim 1, further comprising respective recorders for audio data, video data, and screen shot data. ### Claim 8 (depends on 1) A system according to claim 1, wherein the archival system comprises a consolidation module to pair audio data and screen shot data and store video data separately. ### Claim 9 (depends on 1) A system according to claim 1, wherein the archival system operates according to rules based commands for down sampling for long term storage an incoming data stream according to content of audio data, video data, and screen shot data. ### Claim 10 A recording system comprising: at least one agent computer configured to accommodate bi-directional communications over a network between the at least one agent computer and at least one customer computer, wherein the bi-directional communications traverse a data center server; a plurality of recording apparatuses connected over the network to the data center server, the at least one agent computer and the at least one customer computer; a load balancer connecting the recording apparatuses to the at least one agent computer and the at least one customer computer and receiving data streams representing respective portions of the bi-directional communications, the load balancer directing the portions of the bi-directional communications to a selected recording apparatus for storing recordings of the portions to recording wherein the portions comprise screen shots, video data, and audio data; a topology processor tracking a respective status of the bi-directional communications between pairs of the at least one agent computer and at least one customer computer and storing topology states in a topology memory bank, wherein the topology states comprise, for respective portions of the bi-directional communications, a data stream origination address, a data stream endpoint address, and a at least one time stamp; and an archival system receiving the recordings according to archival rules accessing the topology states, the archival system connected to the network for storing the screen shots, the video data, and the audio data after assembling a consolidated communication according to the time stamp as adjusted for drift errors. ### Claim 11 (depends on 10) A recording system according to claim 10, wherein said topology states comprise time based records of the data streams originating from the agent computer or the customer computer. ### Claim 12 (depends on 10) A recording system according to claim 10, further comprising computer implemented instructions within the archival rules to identify one computer as an agent computer or a customer computer for storing recording streams according to the respective data stream origination address, data stream endpoint address, and time stamp for the identified one computer. ### Claim 13 (depends on 10) A recording system according to claim 10, further comprising computer implemented instructions stored in the archival rules instructing the topology processor and load balancer to identify a selected delta signal from a selected data stream originating from or received by the identified one computer. ### Claim 14 A system for scoring an agent interacting with customers by electronic means, comprising: an analytics server comprising a processor and a memory coupled to the processor; a remote agent workstation comprising respective applications operating thereon and in bi-directional data communication with the analytics server over a network; a customer camera connected to a customer facing computer on the network such that a customer using the customer facing computer during the bi-directional data communication is within a customer camera field of view; an agent interaction camera connected to the remote agent workstation and the network such that an agent using the remote agent workstation during the bi-directional data communication is within an agent interaction camera field of view; wherein the analytics server, the remote agent workstation, and the customer facing computer are configured with respective memory storing computer implemented instructions to perform a method comprising the steps of: store the bi-directional data communications with a plurality of time stamps adjusted for drift errors; authenticating the agent at the remote agent workstation; scoring the agent according to score parameters stored in the memory of the analytics server by comparing the score parameters to customer audio data, customer video data, agent interaction audio data, and agent interaction video data. ### Claim 15 (depends on 14) A system according to claim 14, wherein the analytics server is configured to: selectively gather screen shots of the at least one agent computer during the bi-directional communications and, in real time, parse a portion of the screen shot for analysis; selectively gather video data generated at the at least one agent computer and, in real time, parse a portion of the video data for analysis; selectively gather audio data generated at the at least one agent computer and in real time, parse a portion of the audio data for analysis. ### Claim 16 (depends on 14) A system according to claim 14, configured to authenticate the agent by one or more of real time speech analytics performed on the portion of the audio data and biometric data gathered from the portion of the video data. ### Claim 17 (depends on 16) A system according to claim 16, wherein the analytics server compares the score parameters to at least one of the portions of the agent video data sufficient to evaluate virtual eye contact between the agent and the customer. ### Claim 18 (depends on 16) A system according to claim 16, further comprising: a customer security camera connected to the network and having a first geographical area within the customer security camera field of view; and an agent security camera connected to the network and positioned to encompass an agent workspace and a second geographical area within the agent security camera field of view, wherein the system is configured to: selectively gather screen shots of either the agent computer or the customer computer during the bi-directional communications and, in real time, parse a portion of the screen shot for analysis; selectively gather video data generated at the at least one agent computer or the customer computer and, in real time, parse a portion of the video data for analysis; selectively gather audio data generated at either the agent computer or the customer computer and in real time, parse a portion of the audio data for analysis; apply an audio data time stamp to the portion of the audio data.