# US 8,751,933 B2 · Video and audio waveform user interface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,751,933 B2
- Document ID: US8751933B2
- Status: Granted
- Kind: B2
- Application number: 12/872,778
- Filed: 2010-08-31
- Published: 2012-03-01
- Granted: 2014-06-10
- Earliest claimed: 2010-08-31
- Assignees: Apple Inc.
- Applicants: Laurent Perrodin; Randy Ubillos; Richard Townhill; Judson Coplan; Mike Mitchell; Jer Granucci; Keith Salvucci
- Inventors: Laurent Perrodin; Randy Ubillos; Richard Townhill; Judson Coplan; Mike Mitchell; Jer Granucci; Keith Salvucci
- Web page: https://patentgrep.com/patent/US8751933B2

## CPC

- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- G11B 27/034: on discs (G11B27/036, G11B27/038 take precedence)
- G11B 27/038: Cross-faders therefor
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)

## Abstract

Among other things, methods, systems and computer program products are disclosed for presenting a user interface for manipulation of audio waveforms associated with video content. In one aspect, a method can be performed by one or more processes executing on a computer system. Images representing video frames in a first region of a user interface can be displayed. An audio waveform representing audio associated with at least a portion of a video frame can be displayed. At least three audio waveform portions including (i) a first portion of the audio waveform that exceeds a first predetermined threshold, (ii) a second portion of the audio waveform below the first predetermined threshold and above a second predetermined threshold, and (iii) a third portion of the audio waveform below the second predetermined threshold can be graphically distinguished.

## Claims

### Claim 1

A method performed by one or more processes executing on a computer system, the method comprising:
displaying images representing video frames in a first region of a user interface;
displaying an audio waveform representing audio associated with at least a portion of a video frame; and
graphically distinguishing at least three audio waveform portions including (i) a first portion of the audio waveform that exceeds a first predetermined threshold, (ii) a second portion of the audio waveform below the first predetermined threshold and above a second predetermined threshold, and (iii) a third portion of the audio waveform below the second predetermined threshold;
wherein an audio volume range defined by the first predetermined threshold and the second predetermined threshold is a preferred audio volume range.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first predetermined threshold is substantially 0 decibels and the second predetermined threshold is substantially −6 decibels.

### Claim 3 (depends on 1)

The method of claim 1, wherein graphically distinguishing further comprises:
representing the first portion of the audio waveform in a first color, the second portion of the audio waveform in a second, different, color, and the third portion of the audio waveform in a third, different, color.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
receiving a user input indicating a change in volume level for at least a portion of the audio waveform; and
changing an appearance of the audio waveform in response to the received user input;
wherein changing the appearance of the audio waveform includes positioning a segment of the audio waveform between the first predetermined threshold and the second predetermined threshold and applying the second color to the segment, wherein the segment had not previously been positioned between the first and second predetermined thresholds.

### Claim 5

A method performed by one or more processes executing on a computer system, the method comprising:
displaying images representing video frames in a first region of a user interface;
displaying an audio waveform representing audio associated with at least a portion of the video frames, the audio waveform being associated with a first volume setting;
receiving input selecting a portion of the audio waveform, wherein the input indicates left and right bounds for the selected portion, and the selected portion is less than all of the audio waveform;
automatically determining left and right fade points corresponding to the selected portion;
displaying, in conjunction with the audio waveform, the left and right bounds of the selected portion, and the determined left and right fade points;
receiving a volume change to a second volume setting for the selected portion of the audio waveform;
fading, in response to the volume setting change, a portion of the audio waveform located between the left fade point and the left bound from the first volume setting to the second volume setting; and
fading, in response to the volume setting change, a portion of the audio waveform located between the right bound and the right fade point from the second volume setting to the first volume setting.

### Claim 6 (depends on 5)

The method of claim 5, wherein the duration between a fade point and the corresponding bound is a predetermined percentage of the duration between the left bound and the right bound.

### Claim 7 (depends on 6)

The method of claim 6, wherein the predetermined percentage is 10 percent.

### Claim 8 (depends on 6)

The method of claim 6, further comprising:
receiving input indicating a change in position of the right bound; and
changing the position of the right bound, the left fade point, and the right fade point in response to the received input, so that the duration between the left fade point and the left bound is the predetermined percentage of the duration between the left bound and the new position for the right bound and the duration between the right fade point and the right bound is the predetermined percentage of the duration between the left bound and the new position for the right bound.

### Claim 9 (depends on 5)

The method of claim 5, further comprising:
receiving input indicating a change in position of the right bound; and
changing the position of the right bound and the right fade point in response to the received input.

### Claim 10 (depends on 5)

The method of claim 5, wherein the left fade point is a set duration from the left bound and the right fade point is a set duration from the right bound.

### Claim 11 (depends on 5)

The method of claim 5, wherein fading comprises applying a linear fade.

### Claim 12 (depends on 5)

The method of claim 5, wherein fading comprises applying a user selected fade.

### Claim 13 (depends on 5)

The method of claim 5, further comprising:
displaying a visual representation of the fading for at least one of the portion of the audio waveform located between the left fade point and the left bound, and the portion of the audio waveform located between the right bound and the right fade point.

### Claim 14 (depends on 5)

The method of claim 5, wherein the received input is entered using a single cursor motion.

### Claim 15

A method performed by one or more processes executing on a computer system, the method comprising:
accessing a video sequence comprising images;
displaying a visual representation of the video sequence comprising one or more still images extracted from the video sequence, where each still image is associated with a segment of the video sequence;
presenting a first audio waveform representing a first audio signal in conjunction with at least a portion of the video sequence;
receiving input indicating an audio effect to be associated with a segment of the video sequence;
applying the indicated audio effect to a first portion of the first audio waveform corresponding to the segment;
changing a position of the first video sequence segment within the video sequence in response to a second user input, wherein the position of the first portion of the first audio waveform associated with the first video sequence is not changed; and
applying the indicated audio effect to a second portion of the first audio waveform that is associated with the first video sequence segment after the position of the first video sequence segment has been changed; wherein the first portion of the first audio waveform is different from the second portion of the first audio waveform.

### Claim 16 (depends on 15)

The method of claim 15, wherein the indicated audio effect is removed from the first portion of the first audio waveform responsive to the changing of position of the first video sequence segment.

### Claim 17 (depends on 15)

The method of claim 15, further comprising:
presenting a second audio waveform representing a second audio signal associated with the video sequence, wherein a first portion of the second audio waveform is associated with the first video sequence segment; and
changing a position of the first portion of the second audio waveform responsive to the changing of position of the first video sequence segment such that the first portion of the second audio waveform is aligned with the first video sequence segment after the position of the first video sequence segment has been changed.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
applying the indicated audio effect to the first portion of the second audio waveform.

### Claim 19 (depends on 15)

The method of claim 15, wherein the indicated audio effect is a ducking effect and a visual representation of the ducking effect is applied to the second portion of the first audio waveform.

### Claim 20

A system configured to facilitate the graphic distinguishing of portions of an audio waveform, the system comprising:
a display device configured to display images representing video frames in a first region of a user interface and an audio waveform representing audio associated with at least a portion of a video frame; and
a computer system executing one or more processes to perform operations comprising:
determining that a first portion of the audio waveform exceeds a first predetermined threshold;
determining that a second portion of the audio waveform is positioned below the first predetermined threshold and above a second predetermined threshold;
determining that a third portion of the audio waveform is positioned below the second predetermined threshold; and
graphically distinguishing the first, second, and third portions of the audio waveform;
wherein an audio volume range defined by the first predetermined threshold and the second predetermined threshold is a preferred audio volume range.

### Claim 21 (depends on 20)

The system of claim 20, wherein graphically distinguishing includes representing the first portion of the audio waveform in a first color, the second portion of the audio waveform in a second, different, color, and the third portion of the audio waveform in a third, different, color.

### Claim 22 (depends on 21)

The system of claim 21, further comprising:
an input device for receiving a user input indicating a change in volume level for at least a portion of the audio waveform;
wherein the computer system is configured to change an appearance of the audio waveform in response to the received user input; and
wherein changing the appearance of the audio waveform includes positioning a segment of the audio waveform between the first predetermined threshold and the second predetermined threshold and applying the second color to the segment, wherein the segment had not previously been positioned between the first and second predetermined thresholds.

### Claim 23

A computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
displaying images representing video frames in a first region of a user interface;
displaying an audio waveform representing audio associated with at least a portion of a video frame; and
graphically distinguishing at least three audio waveform portions including (i) a first portion of the audio waveform that exceeds a first predetermined threshold, (ii) a second portion of the audio waveform below the first predetermined threshold and above a second predetermined threshold, and (iii) a third portion of the audio waveform below the second predetermined threshold;
wherein an audio volume range defined by the first predetermined threshold and the second predetermined threshold is a preferred audio volume range.

### Claim 24 (depends on 23)

The computer storage medium of claim 23, wherein the first predetermined threshold is substantially 0 decibels and the second predetermined threshold is substantially −6 decibels.

### Claim 25

A system configured to facilitate the automatic identification and display of fade points for an audio waveform, the system comprising:
a display device configured to display images representing video frames in a first region of a user interface and an audio waveform representing audio associated with at least a portion of the video frames, the audio waveform being associated with a first volume setting;
a first input device for user input indicating a selection of a portion of the audio waveform, wherein the input indicates left and right bounds for the selected portion, and the selected portion is less than all of the audio waveform;
a fade point determination module configured to automatically determine left and right fade points corresponding to the selected portion;
a second input device for receiving user input indicating a volume change to a second volume setting for the selected portion of the audio waveform; and
a fading module configured to fade a portion of the audio waveform located between the left fade point and the left bound from the first volume setting to the second volume setting in response to the volume setting change, and fade a portion of the audio waveform located between the right bound and the right fade point from the second volume setting to the first volume setting in response to the volume setting change;
wherein the display device is further configured to display, in conjunction with the audio waveform, the left and right bounds of the selected portion, and the left and right fade points.

### Claim 26 (depends on 25)

The system of claim 25, wherein the duration between a fade point and the corresponding bound is a predetermined percentage of the duration between the left bound and the right bound.

### Claim 27 (depends on 26)

The system of claim 26, wherein the predetermined percentage is 10 percent.

### Claim 28 (depends on 25)

The system of claim 25, wherein the display device is further configured to display a visual representation of the fading for at least one of the portion of the audio waveform located between the left fade point and the left bound, and the portion of the audio waveform located between the right bound and the right fade point.

### Claim 29

A computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
displaying images representing video frames in a first region of a user interface;
displaying an audio waveform representing audio associated with at least a portion of the video frames, the audio waveform being associated with a first volume setting;
receiving input selecting a portion of the audio waveform, wherein the input indicates left and right bounds for the selected portion, and the selected portion is less than all of the audio waveform;
automatically determining left and right fade points corresponding to the selected portion;
displaying, in conjunction with the audio waveform, the left and right bounds of the selected portion, and the determined left and right fade points;
receiving a volume change to a second volume setting for the selected portion of the audio waveform;
fading, in response to the volume setting change, a portion of the audio waveform located between the left fade point and the left bound from the first volume setting to the second volume setting; and
fading, in response to the volume setting change, a portion of the audio waveform located between the right bound and the right fade point from the second volume setting to the first volume setting.

### Claim 30 (depends on 29)

The medium of claim 29, wherein the program further comprises instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
receiving input indicating a change in position of the right bound; and
changing the position of the right bound and the right fade point in response to the received input.

### Claim 31

A system configured to facilitate the application of audio effects to audio waveforms, the system comprising:
a display device configured to display a visual representation of a video sequence comprising one or more still images extracted from the video sequence, where each still image is associated with a segment of the video sequence; and further configured to display a first audio waveform representing a first audio signal in conjunction with at least a portion of the video sequence;
a first input device for receiving a user input indicating an audio effect to be associated with a segment of the video sequence;
an audio effect application module configured to apply the indicated audio effect to a first portion of the first audio waveform corresponding to the segment;
a second input device for receiving a second user input indicating a change of position of the first video sequence segment within the video sequence;
a computer system executing one or more processes to perform operations comprising:
changing a position of the first video sequence segment within the video sequence in response to the second user input, wherein the position of the first portion of the first audio waveform associated with the first video sequence is not changed; and
applying the indicated audio effect to a second portion of the first audio waveform that is associated with the first video sequence segment after the position of the first video sequence segment has been changed; wherein the first portion of the first audio waveform is different from the second portion of the first audio waveform.

### Claim 32 (depends on 31)

The system of claim 31, wherein the indicated audio effect is removed from the first portion of the first audio waveform responsive to the changing of position of the first video sequence segment.

### Claim 33 (depends on 31)

The system of claim 31, wherein the indicated audio effect is a ducking effect and a visual representation of the ducking effect is applied to the second portion of the first audio waveform.

### Claim 34

A computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
accessing a video sequence comprising images;
displaying a visual representation of the video sequence comprising one or more still images extracted from the video sequence, where each still image is associated with a segment of the video sequence;
presenting a first audio waveform representing a first audio signal in conjunction with at least a portion of the video sequence;
receiving input indicating an audio effect to be associated with a segment of the video sequence;
applying the indicated audio effect to a first portion of the first audio waveform corresponding to the segment;
changing a position of the first video sequence segment within the video sequence in response to a second user input, wherein the position of the first portion of the first audio waveform associated with the first video sequence is not changed; and
applying the indicated audio effect to a second portion of the first audio waveform that is associated with the first video sequence segment after the position of the first video sequence segment has been changed; wherein the first portion of the first audio waveform is different from the second portion of the first audio waveform.

### Claim 35 (depends on 34)

The medium of claim 34, wherein the indicated audio effect is removed from the first portion of the first audio waveform responsive to the changing of position of the first video sequence segment.

### Claim 36 (depends on 34)

The medium of claim 34, wherein the program further comprises instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
presenting a second audio waveform representing a second audio signal associated with the video sequence, wherein a first portion of the second audio waveform is associated with the first video sequence segment; and
changing a position of the first portion of the second audio waveform responsive to the changing of position of the first video sequence segment such that the first portion of the second audio waveform is aligned with the first video sequence segment after the position of the first video sequence segment has been changed.