# US 2019/0377953 A1 · Network switching appliance, process and system for performing visual analytics for a streaming video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0377953 A1 - Document ID: US20190377953A1 - Status: Published application - Kind: A1 - Application number: 16/431,818 - Filed: 2019-06-05 - Published: 2019-12-12 - Earliest claimed: 2018-06-06 - Assignees: Seventh Sense Artificial Intelligence Pvt Ltd - Applicants: Seventh Sense Artificial Intelligence Pvt Ltd - Inventors: VARUN CHATTERJI - Web page: https://patentgrep.com/patent/US20190377953A1 ## CPC - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - H04N 21/4666: {using neural networks, e.g. processing the feedback provided by the user} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - H04N 21/26603: {for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques} - G06V 20/44: {Event detection} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The network switching appliance that receives, compresses, decompresses and analyses any raw video stream emitting from image acquiring device such as a surveillance camera using Artificial Intelligence (AI). The network switching device is a low cost, low power device that is capable of receiving a transmitted video stream, performing image detection/recognition tasks, and transmitting an annotated video stream along with meta-data attached to the specific video stream and images to understand the visual content of the video stream. ## Claims ### Claim 1 A network switching appliance, comprising: a secure device having one or more ports to receive a raw video stream from one or more image capturing devices; a stream input module to receive the raw video stream from the decryption module in form of a compressed raw video stream for processing; a stream parsing module to decompress the compressed raw video stream and produce an uncompressed raw video stream; a secure video stream analytics system embedded in the secure device to locally process and annotate the raw video stream to produce an annotated video stream with a meta data attached to it; a stream output module to caches output frames in a buffer for final compression and transmission; a stream compression module that reads frames from a stream output module and compresses them to a format suitable for communication or local storage; and a local storage device to store the annotated video stream to be retrieved for inspection or stored in a database when connected via an internet. ### Claim 2 (depends on 1) The network switching appliance of claim 1, further comprising; a stream encryption module to secure the annotated video stream using an encryption key; and a stream decryption module to manage a decryption key in a trusted platform module and to verify authenticity of the network switching appliance and expose the raw video stream for local processing and annotation. ### Claim 3 (canceled) ### Claim 4 (depends on 3) The network switching appliance of claim 3, further comprising; an entity detection module to pass frames from one or more streams through a trained convolutional neural network executing on a convolutional neural network processing hardware to obtain regions of interest in the frames; an entity alignment module to process regions of interest obtained from an entity detection module and apply any affine transformations to them to make them upright by passing the regions of interest through a trained convolutional neural network executing on the convolutional neural network processing hardware to obtain keypoints/landmark locations; and an entity representation module to generate vector representations from aligned regions of interest obtained from an entity alignment module by passing the aligned regions of interest through a trained convolutional neural network executing on the convolutional neural network processing hardware to obtain high dimensional vector representations. ### Claim 5 (depends on 4) The network switching appliance of claim 4, further comprising; an entity query module to generate a concatenated vector representations from multiple models and find similar vectors by querying a single index in a database present on the local storage. ### Claim 6 (depends on 5) The network switching appliance of claim 5, further comprising; an entity annotation module to mark/draw entity metadata information on input frames. ### Claim 7 (depends on 6) The network switching appliance of claim 6, further comprising; a social graphing module to construct and update local social graphs stored in the local storage and to transmit/sync them to a remote storage for global merge when the appliance can connect to the remote storage. ### Claim 8 A process of using a network switching appliance for a streaming video analytics, comprising; capturing a raw video stream from an image device through a port located at the network switching appliance; receiving from a stream input module the raw video stream from the stream decryption module in form of a compressed raw video stream for processing; decompressing the compressed raw video stream and produce an uncompressed raw video stream using a stream parsing module; and performing a multiple object detections simultaneously using a convolutional neural network algorithm using a trained model executing on a convolutional neural network processing hardware and using the entity detection module; annotating the raw video stream using a secure video stream analytics system embedded in the appliance to locally process and produce an annotated video stream with a meta data attached to it; and retrieving the annotated video stream for inspection or storing in a database when connected via an internet. ### Claim 9 (depends on 8) The process of claim 8, further comprising; securing the annotated video stream using an encryption key residing in a trusted platform module; and verifying authenticity using a decryption key residing in the trusted platform module to verify authenticity of the network switching appliance and expose the raw video stream for local processing and annotation. ### Claim 10 (canceled) ### Claim 11 (depends on 9) The process of claim 9, further comprising; detecting of a key point and landmark for an image alignment using the entity alignment module and using the detected key points to make detections upright. ### Claim 12 (depends on 11) The process of claim 11, further comprising; generation of high dimensional vector representations using the entity representation module. ### Claim 13 (depends on 12) The process of claim 12, further comprising; querying concatenated vector representations using the entity query module and generating annotated video stream frames using entity annotation module. ### Claim 14 (depends on 13) The process of claim 13, further comprising; constructing a social graph using the social graphing module and using the social graphing algorithm shown in FIG. 10. ### Claim 15 (depends on 14) The process of claim 14, further comprising; buffering/caching video stream frames via the stream output module, compressing the processed and annotated video stream frames for storage or downstream communication via the stream compression module, and finally, encrypting the stream for storage or downstream consumption using the stream encryption module.