# US 10,768,885 B1 · Video conference with shared whiteboard and recording

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,768,885 B1
- Document ID: US10768885B1
- Status: Granted
- Kind: B1
- Application number: 16/392,383
- Filed: 2019-04-23
- Granted: 2020-09-08
- Earliest claimed: 2019-04-23
- Assignees: Study Social, Inc.
- Applicants: Study Social, Inc.
- Inventors: Ethan Fieldman
- Web page: https://patentgrep.com/patent/US10768885B1

## CPC

- H04N 7/155: {involving storage of or access to video conference sessions (tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831)}
- G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay}
- G06F 9/451: Execution arrangements for user interfaces
- G06F 9/452: {Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131)}
- G06F 9/455: Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06T 11/23: using straight lines or curves
- G06T 11/60: Creating or editing images; Combining images with text
- H04L 12/1822: {Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission}
- H04L 12/1831: {Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status}
- H04L 65/1089: by adding media; by removing media
- H04L 65/4015: {where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference}
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- H04L 67/146: Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G09B 5/10: all student stations being capable of presenting the same information simultaneously (G09B5/14 takes precedence)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

One or more users log in to a video conference with a shared, online electronic whiteboard and draw upon a shared whiteboard. Each user has a video and audio stream that are sent to a recording server. Data describing each drawing action upon the shared whiteboard is sent to a database server and each action is then distributed to all users to update their whiteboards. Screenshots of the drawing actions are created by emulating a browser and inputting data describing each action from the database. Alternatively, screenshots are taken by each user computer (after each action or periodically) and sent to the recording server. Audiovisual manipulation software is used to create a whiteboard video from the screenshots are then combined with the audio and video streams to create a resulting video of the entire session. A video clip is created simply by sending begin and end timestamps during the session.

## Claims

### Claim 1

A method of creating a video of an electronic whiteboard, said method comprising:
executing code to emulate a computer browser on a computer;
loading a representation of an electronic whiteboard into said emulated browser on said computer;
loading a plurality of data sets from a database into said emulated browser in sequence and rendering a corresponding drawing action from each of said data sets on said electronic whiteboard, each of said data sets not being an image, a screenshot or a video of a drawing action;
after each of said data sets is loaded, said computer taking a screenshot of said electronic whiteboard and storing said screenshot, wherein each of said data sets includes a timestamp of when said corresponding drawing action was stored in said database, each of said stored screenshots also including the timestamp corresponding to the data set used to create said screenshot; and
importing said stored screenshots into audiovisual software in sequence and exporting a whiteboard video based upon said stored screenshots.

### Claim 2 (depends on 1)

A method as recited in claim 1 further comprising:
storing said data sets into said database as received from a plurality of computing devices, each of said computing devices displaying the same shared whiteboard.

### Claim 3 (depends on 1)

A method as recited in claim 1 wherein said representation of said electronic whiteboard includes a video box region arranged to include a video or image of at least one user.

### Claim 4 (depends on 1)

A method as recited in claim 1 further comprising:
importing an audio file of a user recorded on a computing device that produced said data sets into said audiovisual software; and
combining said audio file with said whiteboard video to create a resulting video of a whiteboard session such that audio of said audio file is synchronized with said drawing actions of said whiteboard video and exporting said resulting video.

### Claim 5 (depends on 4)

A method as recited in claim 4 further comprising:
importing a video file of a user recorded on said computing device into said audiovisual software; and
combining said video file with said resulting video such that images of said video file are synchronized with said drawing actions of said whiteboard video and exporting a new resulting video.

### Claim 6 (depends on 1)

A method as recited in claim 1 further comprising:
loading a begin timestamp from said database that originated from a computing device that produced said data sets into said audiovisual software;
loading an end timestamp from said database that originated from said computing device into said audiovisual software; and
clipping said whiteboard video using said begin and end timestamps to create a whiteboard video clip.

### Claim 7

A method of creating a video of an electronic whiteboard, said method comprising:
displaying an electronic whiteboard in a first browser of a first computing device;
receiving a plurality of drawing actions from a user of said first computing device and rendering said drawing actions in sequence on said electronic whiteboard;
after an end of each of said drawing actions, detecting said end of said each drawing action, and when said end is detected, said first browser taking a screenshot of said electronic whiteboard, wherein said end of said each drawing action acts as a signal to instruct said first browser to take said screenshot;
sending said screenshots taken with said first browser from said first computing device to a recording computer over a network connection, wherein said screenshots are discrete images and are not part of a video stream, and storing said screenshots taken with said first browser into a screenshot database; and
importing said stored screenshots from said screenshot database into audiovisual software in sequence, creating a whiteboard video from said stored screenshots on said recording computer and exporting said whiteboard video.

### Claim 8 (depends on 7)

A method as recited in claim 7 further comprising:
storing along with each of said stored screenshots a timestamp corresponding to the end of each of said drawing actions, wherein said creating a whiteboard video is based upon said timestamps and uses a preselected frame rate of said whiteboard video.

### Claim 9 (depends on 7)

A method as recited in claim 7 further comprising:
displaying said electronic whiteboard in a second browser of a second computing device, said electronic whiteboard being shared between said first and second computing devices;
receiving a plurality of drawing actions from a user of said second computing device and rendering said drawing actions received from said user of said second computing device in sequence on said electronic whiteboard on said second computing device; and
after an end of each of said drawing actions, said second browser of said second computing device taking a screenshot of said electronic whiteboard and storing said screenshot taken with said second browser into said screenshot database.

### Claim 10 (depends on 7)

A method as recited in claim 7 wherein said electronic whiteboard includes a video box region that includes a video or image of at least one user.

### Claim 11 (depends on 7)

A method as recited in claim 7 further comprising:
importing an audio file of a user recorded on said first computing device during said drawing actions into said audiovisual software; and
combining said audio file with said whiteboard video to create a resulting video of a session on said electronic whiteboard such that audio of said audio file is synchronized with said drawing actions of said whiteboard video and exporting said resulting video.

### Claim 12 (depends on 7)

A method as recited in claim 7 further comprising:
importing an audio file and a video file of a user recorded on said first computing device during said drawing actions into said audiovisual software; and
combining said audio file and said video file with said whiteboard video to create a resulting video of a session on said electronic whiteboard such that audio of said audio file and images of said video file are synchronized with said drawing actions of said whiteboard video and exporting a resulting video.

### Claim 13 (depends on 7)

A method as recited in claim 7 further comprising:
receiving a first selection on said electronic whiteboard from said user indicating a begin timestamp and sending said begin timestamp to said screenshot database;
receiving a second selection on said electronic whiteboard from said user indicating an end timestamp and sending said end timestamp to said screenshot database; and
importing said begin and end timestamps into said audiovisual software and clipping said whiteboard video using said begin and end timestamps to create a whiteboard video clip.

### Claim 14 (depends on 7)

A method as recited in claim 7 wherein each of said screenshots is only taken after said end of said each drawing action.

### Claim 15

A method of creating a video of an electronic whiteboard, said method comprising:
displaying an electronic whiteboard in a first browser of a first computing device;
receiving a plurality of drawing actions from a user of said first computing device and rendering said drawing actions in sequence on said electronic whiteboard;
said first browser periodically taking a screenshot of said electronic whiteboard at a first defined interval, wherein said first defined interval is a fixed amount of time;
sending said screenshots taken with said first browser from said first computing device to a recording computer over a network connection, wherein said screenshots are discrete images and are not part of a video stream, and storing said screenshots taken with said first browser into a screenshot database;
storing along with each of said stored screenshots a timestamp corresponding to the time when said screenshot was taken; and
importing said stored screenshots from said screenshot database into audiovisual software in sequence, creating a whiteboard video from said stored screenshots on said recording computer and exporting said whiteboard video, wherein said creating a whiteboard video is based upon said timestamps and a preselected frame rate of said whiteboard video.

### Claim 16 (depends on 15)

A method as recited in claim 15 further comprising:
displaying said electronic whiteboard in a second browser of a second computing device, said electronic whiteboard being shared between said first and second computing devices;
receiving a plurality of drawing actions from a user of said second computing device and rendering said drawing actions received from said user of said second computing device in sequence on said electronic whiteboard on said second computing device; and
said second browser of said second computing device periodically taking a screenshot of said electronic whiteboard at a second defined interval and storing said screenshot taken with said second browser into said screenshot database.

### Claim 17 (depends on 16)

A method as recited in claim 16 wherein said first and second defined intervals are the same.

### Claim 18 (depends on 15)

A method as recited in claim 15 further comprising:
importing an audio file of a user recorded on said first computing device during said drawing actions into said audiovisual software; and
combining said audio file with said whiteboard video to create a resulting video of a session on said electronic whiteboard such that audio of said audio file is synchronized with said drawing actions of said whiteboard video and exporting said resulting video.

### Claim 19 (depends on 15)

A method as recited in claim 15 further comprising:
importing an audio file and a video file of a user recorded on said first computing device during said drawing actions into said audiovisual software; and
combining said audio file and said video file with said whiteboard video to create a resulting video of a session on said electronic whiteboard such that audio of said audio file and images of said video file are synchronized with said drawing actions of said whiteboard video and exporting a resulting video.

### Claim 20 (depends on 15)

A method as recited in claim 15 further comprising:
receiving a first selection on said electronic whiteboard from said user indicating a begin timestamp and sending said begin timestamp to said screenshot database;
receiving a second selection on said electronic whiteboard from said user indicating an end timestamp and sending said end timestamp to said screenshot database; and
importing said begin and end timestamps into said audiovisual software and clipping said whiteboard video using said begin and end timestamps to create a whiteboard video clip.