# US 2024/0194172 A1 · Context-aware input gestures for music creation applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0194172 A1
- Document ID: US20240194172A1
- Status: Published application
- Kind: A1
- Application number: 18/062,711
- Filed: 2022-12-07
- Published: 2024-06-13
- Earliest claimed: 2022-12-07
- Assignees: Avid Technology, Inc.
- Applicants: Avid Technology, Inc.
- Inventors: Sylvain Girard; Joseph Plazak
- Web page: https://patentgrep.com/patent/US20240194172A1

## CPC

- G10H 1/0025: {Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece (automatically producing a series of tones G10H1/26)}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G10H 1/0008: {Associated control or indicating means}
- G10H 2220/121: for graphical editing of a musical score, staff or tablature
- G10H 2220/241: on touchscreens, i.e. keys, frets, strings, tablature or staff displayed on a touchscreen display for note input purposes
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Music creation applications enable users to enter new objects into a musical project using gestures mediated by input maps that are adapted to the musical context of the object entry location. The musical context may be defined by an explicit aspect of the musical project or implicitly by notes or intervals already entered into the project. The adaptation of the input maps may include spatial scaling to provide greater space to objects favored by the context. The context-adapted input maps may be displayed and used to guide user input, such as with gestures on a touch screen. In some implementations, input-sensitive regions of input maps are scaled in accordance with the musical context without being displayed. The methods apply to musical object entry via pop-up input maps and virtual instrument displays.

## Claims

### Claim 1

A method of enabling a user of a music creation application to enter data into a musical project, the method comprising:
retrieving a current musical context of a location in the musical project at which a new object is to be placed;
generating an input map of a plurality of musical objects within a graphical user interface of the music creation application, wherein at least one of a layout and object content of the input map is based at least in part on the current musical context;
enabling a user of the music creation application to select from the input map the new object to be entered at the location in the musical project; and
in response to a selection by the user of the new object from the input map, entering the selected new object at the location within the musical project.

### Claim 2 (depends on 1)

The method of claim 1, wherein a spatial scaling of the input map is based on the current musical context, such that objects favored by the current musical context are allocated more space in the input map than objects that are not favored by the current musical context.

### Claim 3 (depends on 2)

The method of claim 2, wherein the spatial scaling of the input map is a spatial scaling of input-sensitive areas of the input map.

### Claim 4 (depends on 1)

The method of claim 1, further comprising displaying the input map within the graphical user interface of the music creation application, and wherein the input map comprises a virtual keyboard instrument, and the spatial scaling scales an area sensitive to at least one of a touch gesture and a drag gesture such that keys of the virtual keyboard representing notes that are favored by the current musical context are allocated a larger sensitive area than keys representing notes that are not favored by the current musical context.

### Claim 5 (depends on 1)

The method of claim 1, further comprising displaying the input map within the graphical user interface of the music creation application, and wherein the input map comprises a virtual keyboard instrument, and keys of the virtual keyboard are displayed with a graphical feature indicative of the current musical context.

### Claim 6 (depends on 1)

The method of claim 1, further comprising displaying the input map within the graphical user interface of the music creation application, and wherein the input map is displayed with a default selection based on the current musical context.

### Claim 7 (depends on 6)

The method of claim 6, wherein the new object is a note, and the default selection is a pitch of the note.

### Claim 8 (depends on 1)

The method of claim 1, wherein the current musical context is explicitly associated with the musical project.

### Claim 9 (depends on 1)

The method of claim 1, wherein the current musical context is determined by the music creation application by analyzing at least a portion of the musical project that has already been entered.

### Claim 10 (depends on 1)

The method of claim 1, wherein the current musical context is a key of the musical project.

### Claim 11 (depends on 1)

The method of claim 1, wherein the current musical context is a chord.

### Claim 12 (depends on 1)

The method of claim 1, wherein the current musical context is based on diatonic intervals of notes that are already present within the musical project.

### Claim 13 (depends on 1)

The method of claim 1, wherein the current musical context is based on pitches of notes that are already present within the musical project.

### Claim 14 (depends on 1)

The method of claim 1, wherein the current musical context is based on objects present within a span of less than one of 3, 4, 5, and 6 measures of the musical project temporally preceding the location in the musical project at which the new object is to be placed.

### Claim 15 (depends on 1)

The method of claim 1, wherein the user selection of the new object to be entered comprises:
a selection of a first aspect of the new object from a first input map, wherein the first input map includes:
a plurality of options for the first aspect of the new object, wherein the at least one of the layout and object content of the first input map is based on a first aspect of the current musical context; and
a selection of a second aspect of the new object from a second input map, wherein the second input map includes:
a plurality of options for the second aspect of the new object, and wherein at least one of the layout and object content of the second input map is based on a second aspect of the current musical context.

### Claim 16 (depends on 15)

The method of claim 15, wherein the first aspect of the new object is a note duration, and the second aspect of the new object is a note pitch.

### Claim 17 (depends on 1)

The method of claim 1, further comprising enabling the user to select from a second input map a modifier of the selected new musical object, wherein the second input map is based at least in part on the current musical context.

### Claim 18 (depends on 1)

The method of claim 1, wherein the selection of the new object to be entered into the musical project involves a gesture by the user having a starting point on the input map, a drag movement within the input map, and a release point within the input map, and wherein an effect of at least one of the starting point, drag movement, and release point is based in part on the current musical context.

### Claim 19 (depends on 1)

The method of claim 1, wherein the selection of the new object to be entered into the musical project involves a gesture by the user that ends with a release point within the input map, and wherein the music creation application performs error correction at the release point based in part on the current musical context.

### Claim 20 (depends on 1)

The method of claim 1, wherein the user selection of the new object from the input map includes a drag motion from an initial selection within the input map to a final selection within the input map, and wherein the music creation application generates haptic feedback during the drag motion that favors user selection of objects favored by the current musical context.

### Claim 21 (depends on 1)

The method of claim 1, wherein the music creation application determines the current musical context based in part on a prior object entry behavior of the user.

### Claim 22

A computer program product comprising:
a non-transitory computer-readable medium with computer-readable instructions encoded thereon, wherein the computer-readable instructions, when processed by a processing device instruct the processing device to perform a method of enabling a user of a music creation application to enter data into a musical project, the method comprising:
retrieving a current musical context of a location in the musical project at which a new object is to be placed;
generating an input map of a plurality of musical objects within a graphical user interface of the music creation application, wherein at least one of a layout and object content of the input map is based at least in part on the current musical context;
enabling a user of the music creation application to select from the input map the new object to be entered at the location in the musical project; and
in response to a selection by the user of the new object from the input map, entering the selected new object at the location within the musical project.

### Claim 23

A system comprising:
a memory for storing computer-readable instructions; and
a processor connected to the memory, wherein the processor, when executing the computer-readable instructions, causes the system to perform a method of enabling a user of a music creation application to enter data into a musical project, the method comprising, the method comprising:
retrieving a current musical context of a location in the musical project at which a new object is to be placed;
generating an input map of a plurality of musical objects within a graphical user interface of the music creation application, wherein at least one of a layout and object content of the input map is based at least in part on the current musical context;
enabling a user of the music creation application to select from the input map the new object to be entered at the location in the musical project; and
in response to a selection by the user of the new object from the input map, entering the selected new object at the location within the musical project.
