# US 2018/0025020 A1 · System and method for contextually enriching a concept database

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025020 A1
- Document ID: US20180025020A1
- Status: Published application
- Kind: A1
- Application number: 15/722,608
- Filed: 2017-10-02
- Published: 2018-01-25
- Earliest claimed: 2005-10-26
- Assignees: Cortica, Ltd.
- Applicants: Cortica, Ltd.
- Inventors: Adam HAREL; Igal RAICHELGAUZ; Karina ODINAEV; Yehoshua Y ZEEVI
- Web page: https://patentgrep.com/patent/US20180025020A1

## CPC

- G06F 16/41: Indexing; Data structures therefor; Storage structures
- G06F 3/048: Interaction techniques based on graphical user interfaces [GUI]
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 16/14: Details of searching files based on file metadata
- G06F 16/152: {using file content signatures, e.g. hash values}
- G06F 16/172: Caching, prefetching or hoarding of files
- G06F 16/1748: {De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641)}
- G06F 16/2228: {Indexing structures}
- G06F 16/284: {Relational databases}
- G06F 16/285: {Clustering or classification}
- G06F 16/35: Clustering; Classification
- G06F 16/40: of multimedia data, e.g. slideshows comprising image and additional audio data (retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70)
- G06F 16/43: Querying
- G06F 16/433: {using audio data}
- G06F 16/434: {using image data, e.g. images, photos, pictures taken by a user}
- G06F 16/435: Filtering based on additional data, e.g. user or group profiles
- G06F 16/438: Presentation of query results
- G06F 16/4393: {Multimedia presentations, e.g. slide shows, multimedia albums}
- G06F 16/45: Clustering; Classification
- G06F 16/48: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F 16/483: using metadata automatically derived from the content
- G06F 16/487: using geographical or spatial information, e.g. location
- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 16/683: using metadata automatically derived from the content
- G06F 16/685: {using automatically derived transcript of audio data, e.g. lyrics (speech recognition G10L15/00)}
- G06F 16/739: {in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames}
- G06F 16/783: using metadata automatically derived from the content
- G06F 16/7834: {using audio features}
- G06F 16/7844: {using original textual content or text extracted from visual content or transcript of audio data}
- G06F 16/7847: {using low-level visual features of the video content}
- G06F 16/904: Browsing; Visualisation therefor (for navigating the web G06F16/954; browsing optimisation for the web G06F16/957)
- G06F 16/951: Indexing; Web crawling techniques
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06F 16/9558: {Details of hyperlinks; Management of linked annotations}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 40/134: Hyperlinking
- G06N 3/045: Combinations of networks
- G06N 3/048: Activation functions
- G06N 3/063: using electronic means
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 5/02: Knowledge representation; Symbolic representation
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 5/025: Extracting rules from data
- G06N 5/04: Inference or reasoning models
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06Q 30/0201: Market modelling; Market analysis; Collecting market data
- G06Q 30/0246: {Traffic}
- G06Q 30/0261: {based on user location}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/48: {Matching video sequences}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G09B 19/0092: {Nutrition}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 25/51: for comparison or discrimination
- H04H 20/103: {Transmitter-side switching}
- H04H 20/26: Arrangements for switching distribution systems
- H04H 20/93: which locates resources of other pieces of information, e.g. URL [Uniform Resource Locator]
- H04H 60/33: Arrangements for monitoring the users' behaviour or opinions
- H04H 60/37: for identifying segments of broadcast information, e.g. scenes or extracting programme ID
- H04H 60/46: for recognising users' preferences
- H04H 60/49: for identifying locations
- H04H 60/56: Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
- H04H 60/58: of audio {(determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00)}
- H04H 60/59: of video {(recognising characters or patterns in general G06F18/00, G06V20/00)}
- H04H 60/66: for using the result on distributors' side
- H04H 60/71: using meteorological information
- H04L 51/18: Commands or executable codes
- H04L 51/42: Mailbox-related aspects, e.g. synchronisation of mailboxes
- H04L 65/765: {intermediate}
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04L 67/306: User profiles
- H04L 67/535: {Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34)}
- H04L 67/63: Routing a service request depending on the request content or context
- H04N 7/17318: {Direct or substantially direct transmission and handling of requests}
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/25891: {being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)}
- H04N 21/26603: {for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques}
- H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)}
- H04N 21/278: Content descriptor database or directory service for end-user access {(details of content or meta data based information retrieval of video data in video databases G06F16/739)}
- H04N 21/466: Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)}
- H04N 21/8106: {involving special audio data, e.g. different tracks for different languages}
- G06V 2201/10: Recognition assisted with metadata
- G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
- H04H 2201/90: characterised by the use of signatures
- Y10S 707/99943: Generating database or data structure, e.g. via user interface
- Y10S 707/99948: Application of database or data structure, e.g. distributed, multimedia, or image
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

A system and method for contextually enriching a concept database. The method includes determining, based on at least one signature of a first multimedia data element (MMDE) and signatures of a plurality of third concepts stored in the concept database, at least one matching first concept among the plurality of third concepts; generating a reduced representation of the first MMDE; comparing the reduced representation of the first MMDE to signatures representing a plurality of second MMDEs to determine a plurality of matching MMDEs among the plurality of second MMDEs; generating, based on the reduced representation of the first MMDE and the signatures representing the plurality of matching MMDEs, a second concept; and generating at least one context based on the second concept and the plurality of third concepts, wherein each context includes at least one common pattern among the second concept and at least one of the plurality of third concepts.

## Claims

### Claim 1

A method for contextually enriching a concept database, comprising:
determining, based on at least one signature of a first multimedia data element (MMDE) and signatures of a plurality of third concepts stored in the concept database, at least one first concept among the plurality of third concepts, wherein each of the at least one first concept matches at least a portion of the at least one signature of the first MMDE;
generating a reduced representation of the first MMDE, wherein the reduced representation does not represent the at least a portion of the first MMDE matching the at least one first concept;
comparing the reduced representation of the first MMDE to signatures representing a plurality of second MMDEs to determine a plurality of matching MMDEs among the plurality of second MMDEs;
generating, based on the reduced representation of the first MMDE and the signatures representing the plurality of matching MMDEs, a second concept; and
generating at least one context based on the second concept and the plurality of third concepts, wherein each context includes at least one common pattern among the second concept and at least one of the plurality of third concepts.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the at least one context further comprises:
analyzing associations among the second concept and the plurality of third concepts.

### Claim 3 (depends on 2)

The method of claim 2, wherein analyzing the associations further comprises:
correlating the second concept with each of the third concepts.

### Claim 4 (depends on 3)

The method of claim 3, wherein the correlation includes comparing the second concept and one of the plurality of third concepts with respect to at least one of: ratio of sizes, ratio of angles, ratio of scale, location, and orientation.

### Claim 5 (depends on 1)

The method of claim 1, wherein each context is generated based on a matrix of co-occurrences among the second concept and the plurality of third concepts.

### Claim 6 (depends on 5)

The method of claim 5, wherein each context is generated with respect to threshold number of co-occurrences in the matrix.

### Claim 7 (depends on 1)

The method of claim 1, wherein each MMDE is at least one of: an image, a graphic, a video stream, a video clip, a video frame, a photograph, and an image of signals.

### Claim 8 (depends on 1)

The method of claim 1, wherein each concept is a collection of signatures and metadata representing the concept.

### Claim 9 (depends on 1)

The method of claim 1, wherein each signature is generated by a signature generator system, wherein the signature generator system includes a plurality of at least statistically independent computational cores, wherein the properties of each core are set independently of the properties of each other core.

### Claim 10

A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to perform a process, the process comprising:
determining, based on at least one signature of a first multimedia data element (MMDE) and signatures of a plurality of third concepts stored in the concept database, at least one first concept among the plurality of third concepts, wherein each of the at least one first concept matches at least a portion of the at least one signature of the first MMDE;
generating a reduced representation of the first MMDE, wherein the reduced representation does not represent the at least a portion of the first MMDE matching the at least one first concept;
comparing the reduced representation of the first MMDE to signatures representing a plurality of second MMDEs to determine a plurality of matching MMDEs among the plurality of second MMDEs;
generating, based on the reduced representation of the first MMDE and the signatures representing the plurality of matching MMDEs, a second concept; and
generating at least one context based on the second concept and the plurality of third concepts, wherein each context includes at least one common pattern among the second concept and at least one of the plurality of third concepts.

### Claim 11

A system for enriching a concept database, comprising:
a processing circuitry; and
a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:
determine, based on at least one signature of a first multimedia data element (MMDE) and signatures of a plurality of third concepts stored in the concept database, at least one first concept among the plurality of third concepts, wherein each of the at least one first concept matches at least a portion of the at least one signature of the first MMDE;
generate a reduced representation of the first MMDE, wherein the reduced representation does not represent the at least a portion of the first MMDE matching the at least one first concept;
compare the reduced representation of the first MMDE to signatures representing a plurality of second MMDEs to determine a plurality of matching MMDEs among the plurality of second MMDEs;
generate, based on the reduced representation of the first MMDE and the signatures representing the plurality of matching MMDEs, a second concept; and
generate at least one context based on the second concept and the plurality of third concepts, wherein each context includes at least one common pattern among the second concept and at least one of the plurality of third concepts.

### Claim 12 (depends on 11)

The system of claim 11, wherein the system is further configured to:
analyze associations among the second concept and the plurality of third concepts.

### Claim 13 (depends on 12)

The system of claim 12, wherein the system is further configured to:
correlate the second concept with each of the at least one of the plurality of third concepts.

### Claim 14 (depends on 13)

The system of claim 13, wherein the correlation includes comparing the second concept and one of the plurality of third concepts with respect to at least one of: ratio of sizes, ratio of angles, ratio of scale, location, and orientation.

### Claim 15 (depends on 11)

The system of claim 11, wherein each context is generated based on a matrix of co-occurrences among the second concept and the plurality of third concepts.

### Claim 16 (depends on 15)

The system of claim 15, wherein each context is generated with respect to threshold number of co-occurrences in the matrix.

### Claim 17 (depends on 11)

The system of claim 11, wherein each MMDE is at least one of: an image, a graphic, a video stream, a video clip, a video frame, a photograph, and an image of signals.

### Claim 18 (depends on 11)

The system of claim 11, wherein each concept is a collection of signatures and metadata representing the concept.

### Claim 19 (depends on 11)

The system of claim 11, wherein each signature is generated by a signature generator system, wherein the signature generator system includes a plurality of at least statistically independent computational cores, wherein the properties of each core are set independently of the properties of each other core.

### Claim 20 (depends on 19)

The system of claim 19, further comprising:
the signature generator system.
