# US 2024/0205038 A1 · Personalized navigable meeting summary generator

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0205038 A1
- Document ID: US20240205038A1
- Status: Published application
- Kind: A1
- Application number: 18/067,499
- Filed: 2022-12-16
- Published: 2024-06-20
- Earliest claimed: 2022-12-16
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Dikla DOTAN-COHEN; Inbal Maimon; Ido Mordechai Priness
- Web page: https://patentgrep.com/patent/US20240205038A1

## CPC

- G06F 16/41: Indexing; Data structures therefor; Storage structures
- H04L 12/1831: {Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status}
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 16/44: Browsing; Visualisation therefor
- G06F 16/447: {Temporal browsing, e.g. timeline}
- G06F 16/483: using metadata automatically derived from the content
- G06F 16/489: {using time information}
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- G06F 3/048: Interaction techniques based on graphical user interfaces [GUI]
- G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- H04L 12/1822: {Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Technology is disclosed for programmatically providing structured and personalized meeting content that both (1) summarizes aspects of a meeting recording and (2) is used to navigate to a section of the meeting recording. To do so, the disclosure provides technologies to determine sections of the meeting recording. For each section of the meeting recording, a section representation and a meeting structure for presenting the section representation are determined. The section representation includes an image and a text summary of the corresponding section of the meeting recording. Additionally, the disclosure provides technologies to automatically generate a personalized navigable meeting recording (PNMR) user interface (UI) based on the meeting structure, the PNMR UI comprising an indication of each section based on the section representation. Selection of a section representation causes playback of the meeting recording to change to the section associated with the section representation.

## Claims

### Claim 1

A computer system comprising:
at least one processor; and
computer memory having computer-readable instructions embodied thereon, that, when executed by the at least one processor, perform operations comprising:
accessing meeting data associated with a meeting recording;
determining a meeting feature from the meeting data, where the meeting feature is generated without user interaction;
determining a plurality of sections of the meeting recording based on the meeting feature;
for each section of the meeting recording:
determining a section representation based on at least the meeting feature; and
determining a meeting structure; and
automatically generating a personalized navigable meeting recording (PNMR) user interface (UI) based on the meeting structure, the PNMR UI comprising an indication of each section based on the section representation.

### Claim 2 (depends on 1)

The system of claim 1, wherein the section representation comprises, for at least one section of the plurality of sections, at least one of: (1) an image or (2) a text summary, wherein the image is generated based on visual content occurring during the at least one section, and wherein the summary is generated based on at least one of the meeting feature, an audio transcript of the at least one section, and the visual content occurring during the at least one section.

### Claim 3 (depends on 2)

The system of claim 2, wherein the operations comprise, in response to receiving a user input to the image or the text summary for a respective indication, changing playback of the meeting recording on the PNMR UI to a section of the meeting recording corresponding to a respective section representation of the meeting recording associated with the respective indication.

### Claim 4 (depends on 2)

The system of claim 2, wherein the text summary comprises a description of a speaker associated with the at least one section and that is relevant to a user, a topic associated with the at least one section and that is relevant to the user, a transcript of speech associated with the at least one section, and metadata about a format in which the at least one section is presented.

### Claim 5 (depends on 1)

The system of claim 1, wherein determining the section representation comprises:
accessing user data;
based on the user data, determining a user feature; and
based on the meeting feature and the user feature, generating an image and a text summary that are each relevant to a particular user, wherein the PNMR UI comprises a first graphical user interface (GUI) element indicative of the image and a second GUI element indicative of the text summary.

### Claim 6 (depends on 1)

The system of claim 1, wherein the meeting structure comprises at least one of: a hierarchical logical organization, a chronological logical organization, a speaker-indexed logical organization, a topic-indexed logical organization, a relevance ranked logical organization.

### Claim 7 (depends on 1)

The system of claim 1, wherein determining the meeting structure comprises:
determining, based on a first meeting feature and a first user feature associated with a first section of the meeting recording, a first relevance weight to a user profile;
determining, based on a second meeting feature and a second user feature associated with a second section of the meeting recording, a second relevance weight to the user profile; and
based on the first relevance weight being greater than the second relevance weight, generating the meeting structure with the first section ranked higher than the second section.

### Claim 8 (depends on 7)

The system of claim 7, wherein based on the first section being ranked higher than the second section, the meeting structure for the first section and the second section comprises at least one of: the first section being listed above the second section, the first section being automatically played back before the second section, the first section being a grouping in a hierarchy and the second section being a subgroup of the grouping of the first section in the hierarchy, or the first section being displayed as more relevant than the second section.

### Claim 9 (depends on 1)

The system of claim 1, wherein at least one section of the plurality of sections is determined by:
determining a contiguous portion of the meeting recording having a common meeting feature comprising an indication of: a speaker, a topic, an audio content, a visual content, a relevance weight, a media type, or a meeting attendee;
determining a start time of the contiguous portion of the meeting recording that corresponds to a first change of the common meeting feature;
determining an end time of the contiguous portion of the meeting recording that corresponds to a second change of the common meeting feature; and
determining the at least one section of the meeting recording as the contiguous portion of the meeting recording from the start time to the end time.

### Claim 10 (depends on 1)

The system of claim 1, wherein the operations comprise:
accessing user data; and
based on the user data, determining a user feature indicative of at least one of a role of a particular user, a level of user engagement of the particular user with a meeting corresponding to the meeting recording, or a relevance of a topic to the particular user, wherein the meeting feature comprises a meeting feature specific to content of the meeting recording, and wherein at least one of the section representation and the meeting structure is determined based on the user feature and the meeting feature.

### Claim 11 (depends on 1)

The system of claim 1, wherein the meeting feature comprises at least one of a topic of the meeting recording, a type of meeting recording, an identity of a speaker in the meeting recording, a relationship of the speaker to a viewer, a duration of the meeting recording, a duration of pauses in the meeting recording, a transition from a first speaker to a second speaker different from the first speaker, a meeting type, a media type, or contextual metadata expressed as data features indicative of meeting invitees, meeting attendees, or a type of meeting.

### Claim 12

A computerized method, comprising:
receiving, from a computing device, a user request to access a meeting recording;
subsequent to receiving the user request, presenting a personalized navigable meeting recording (PNMR) user interface (UI) comprising a player region and a section representation region separate from the player region, the section representation region comprising a plurality of indications corresponding to a plurality of section representations of a plurality of sections of the meeting recording and arranged based on a meeting structure determined based on a meeting feature, each indication of the plurality of indications comprising at least one of: (1) an image or (2) a text summary, wherein the meeting feature is determined without an interaction by the user;
receiving, via the PNMR UI, a user input to a first indication corresponding to a first section representation of the plurality of section meeting section representations; and
based on the user input and the first indication, changing content on the player region to correspond to a first section of the meeting recording corresponding to the first section representation.

### Claim 13 (depends on 12)

The computerized method of claim 12, wherein receiving the user input comprises:
receiving the user input selecting a portion of the text summary associated with the first section;
determining a topic of the selected portion of the text summary;
accessing the first section of the plurality of sections of the meeting recording;
determining a subsection of the first section associated with the topic; and
changing content on the player region to the subsection of the first section based on the selected portion of the text summary and the subsection being associated with the topic.

### Claim 14 (depends on 12)

The computerized method of claim 12, wherein the image is generated based on visual content occurring during the first section, and wherein the summary is generated based on the meeting feature.

### Claim 15 (depends on 12)

The computerized method of claim 12, wherein determining the section representation comprises:
accessing user data;
based on the user data, determining a role of a particular user; and
based on the meeting feature and the role of the particular user, generating the image and the text summary corresponding to the role of the particular user, wherein the PNMR UI comprises a first graphical user interface (GUI) element indicative of the image and a second GUI element indicative of the text summary.

### Claim 16 (depends on 12)

The computerized method of claim 12, wherein the meeting structure comprises at least one of: a hierarchical logical organization, a chronological logical organization, a speaker-indexed logical organization, a topic-indexed logical organization, a relevance ranked logical organization, and wherein determining the meeting structure comprises:
determining, based on a first meeting feature and a first user feature associated with the first section of the meeting recording, a first relevance weight to a user profile;
determining, based on a second meeting feature and a second user feature associated with a second section of the meeting recording, a second relevance weight to a user profile; and
based on the first relevance weight being greater than the second relevance weight, generating the meeting structure with the first section ranked higher than the second section.

### Claim 17

Computer storage device having computer-executable instructions embodied thereon, that, when executed by at least one computer processor, cause computing operations to be performed, comprising:
accessing meeting data associated with a meeting recording;
determining a plurality of meeting features from the meeting data, wherein the plurality of meeting features are determined without an input from a user;
determining (1) a first section of a plurality of sections of the meeting recording associated with a first meeting feature of the plurality of meeting features and (2) a second section the plurality of sections of the meeting recording associated to a second meeting feature of the plurality of meeting features;
for each of the first and second sections of the meeting recording:
determining a section representation based on the first meeting feature and the second meeting feature, respectively, wherein each section representation comprises at least one of: (1) an image generated based on visual content occurring during the corresponding section or (2) a text summary corresponding to the corresponding section and generated based on the corresponding meeting feature; and
determining a meeting structure; and
automatically generating, for the user, a personalized navigable meeting recording (PNMR) user interface (UI) based on the meeting structure.

### Claim 18 (depends on 17)

The computer storage device of claim 17, wherein associating the first meeting feature to the first section comprises:
determining a start time and an end time associated with the first meeting feature;
determining whether the start time and the end time associated with the first meeting feature overlaps with a start time and an end time associated with the first section; and
based on a level of overlap between (1) the start time and the end time associated with the first meeting feature and (2) the start time and the end time associated with the first section, correlating the first meeting feature with the first section.

### Claim 19 (depends on 18)

The computer storage device of claim 18, wherein the meeting structure comprises at least one of: a hierarchical logical organization, a chronological logical organization, a speaker-indexed logical organization, a topic-indexed logical organization, a relevance ranked logical organization, and wherein determining the meeting structure comprises:
determining, based on the first meeting feature associated with the first section of the meeting recording, a first relevance weight and a first user feature associated with a user profile;
determining, based on the second meeting feature and a second user feature associated with the second section of the meeting recording, a second relevance weight to the user profile; and
based on the first relevance weight being greater than the second relevance weight, generating the meeting structure with the first section ranked higher than the second section.

### Claim 20

(canceled)

### Claim 21 (depends on 1)

The system of claim 1, wherein the operations comprise:
determining plurality of meeting features from the meeting data;
determining a first user feature and a second user feature based on user data, where the first user feature is associated with a first user profile and the second user feature is associated with a second user profile;
determine a first weight and a second weight associated with a section of the plurality of sections based on a classification or the plurality of meeting features and the first user feature and the second user features; and
generating a first PNMR UI and a second PNMR UI, where the first PNMR UI includes the section based on the first weight and the second PNMR UI does not include the second based on the second weight.
