# US 2021/0064231 A1 · System, method and user-interaction console for controlling at least one software application

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0064231 A1
- Document ID: US20210064231A1
- Status: Published application
- Kind: A1
- Application number: 16/556,275
- Filed: 2019-08-30
- Published: 2021-03-04
- Earliest claimed: 2019-08-30
- Assignees: LoupeDeck Oy
- Applicants: LoupeDeck Oy
- Inventors: Pauli Seppinen; Karo Holmberg; Vasily Balagurov; Jussi Mäkinen; Ville Rahikka; Pauli Immonen
- Web page: https://patentgrep.com/patent/US20210064231A1

## CPC

- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G06F 3/02: Input arrangements using manually operated switches, e.g. using keyboards or dials
- G06F 3/0219: {Special purpose keyboards}
- G06F 3/0362: with detection of one-dimensional [1D] translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
- G06F 3/0393: {Accessories for touch pads or touch screens, e.g. mechanical guides added to touch screens for drawing straight lines, hard keys overlaying touch screens or touch pads}
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G06F 2203/04809: Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system, method and a user-interaction console for controlling at least one software application includes at least one user-interaction console and a processor coupled in communication with the at least one user-interaction console. The user-interaction console includes a plurality of physical user-interaction controllers, wherein a given physical user-interaction controller is associated with at least one context-based command pertaining to the at least one software application; a first touch-sensitive display; and a mechanical grid frame arranged on top of at least a portion of the first touch-sensitive display, wherein the mechanical grid frame defines a plurality of sub-portions in at least a portion of the first touch-sensitive display.

## Claims

### Claim 1

A system for controlling at least one software application, the system comprising:
at least one user-interaction console, the user-interaction console comprising:
a plurality of physical user-interaction controllers, wherein a given physical user-interaction controller is associated with at least one context-based command pertaining to the at least one software application;
a first touch-sensitive display that, in operation, displays a first graphical information; and
a mechanical grid frame arranged on top of at least a portion of the first touch-sensitive display, wherein the mechanical grid frame defines a plurality of sub-portions in said portion of the first touch-sensitive display; and
a processor coupled in communication with the at least one user-interaction console, wherein the processor is configured to execute a service software application to:
obtain contextual information comprising operations performed by the at least one software application;
configure the at least one user-interaction console, based on the contextual information, to designate operations that each physical user-interaction controller performs in the at least one software application upon receiving a context based command from a user;
display the first graphical information at the first touch-sensitive display;
enable, via the at least one user-interaction console, a user to provide the at least one context-based command;
convert the at least one context-based command into a required format that is understandable by the at least one software application; and
transmit the at least one context-based command having the required format to the at least one software application to control the at least one software application.

### Claim 2 (depends on 1)

The system of claim 1, wherein the user-interaction console further comprises a second touch-sensitive display that, in operation, displays a second graphical information.

### Claim 3 (depends on 2)

The system of claim 2, wherein the processor is configured to execute the service software application to display the second graphical information at the second touch-sensitive display.

### Claim 4 (depends on 1)

The system of claim 1 wherein the processor is configured to execute the service software application to provide the user with a user interface to enable the user to manually configure the at least one user-interaction console, based on the contextual information.

### Claim 5 (depends on 1)

The system of claim 1, wherein the service software application comprises at least one context-based software plugin, the processor being configured to execute the at least one context-based software plugin to:
receive the at least one context-based command having a first format; and
convert the at least one context-based command having the first format into the at least one context-based command having the required format.

### Claim 6 (depends on 1)

The system of claim 1, wherein at least one of the user-interaction console or the processor further comprises an audio-based user-interaction module, and wherein the processor is configured to execute the service software application to:
enable, via the audio-based user-interaction module, the user to provide a configuration audio input; and
configure the at least one user-interaction console, based on the configuration audio input.

### Claim 7

A method for controlling at least one software application via at least one user-interaction console, the user-interaction console comprising a plurality of physical user-interaction controllers, a first touch-sensitive display, and a mechanical grid frame arranged on top of at least a portion of the first touch-sensitive display, the method comprising:
obtaining contextual information comprising operations performed by the at least one software application;
configuring the at least one user-interaction console, based on the contextual information, to designate operations that each physical user-interaction controller performs in the at least one software application upon receiving a context based command from a user;
displaying a first graphical information at the first touch-sensitive display;
enabling, via the at least one user-interaction console, a user to provide at least one context-based command;
converting the at least one context-based command into a required format that is understandable by the at least one software application; and
transmitting the at least one context-based command having the required format to the at least one software application to control the at least one software application.

### Claim 8

A user-interaction console for controlling at least one software application, the user-interaction console comprising:
a plurality of physical user-interaction controllers, wherein a given physical user-interaction controller is associated with at least one context-based command pertaining to the at least one software application;
a first touch-sensitive display that, in operation, displays a first graphical information;
a mechanical grid frame arranged on top of at least a portion of the first touch-sensitive display, wherein the mechanical grid frame defines a plurality of sub-portions in said portion of the first touch-sensitive display; and
a processor coupled in communication with the at least one user-interaction console, wherein the processor is configured to execute a service software application to:
obtain contextual information comprising operations performed by the at least one software application;
configure the at least one user-interaction console, based on the contextual information, to designate operations that each physical user-interaction controller performs in the at least one software application upon receiving a context based command from a user.

### Claim 9 (depends on 8)

The user-interaction console of claim 8, further comprising a second touch-sensitive display that, in operation, displays a second graphical information.

### Claim 10 (depends on 8)

The user-interaction console of claim 8, wherein the mechanical grid frame comprises a plurality of equisized grid sections.

### Claim 11 (depends on 8)

The user-interaction console of claim 8, wherein the mechanical grid frame is detachably arranged on the first touch-sensitive display.

### Claim 12 (depends on 8)

The user-interaction console of claim 8, wherein the plurality of physical user-interaction controllers comprise at least one button and at least one dial.

### Claim 13 (depends on 8)

The user-interaction console of claim 8, further comprising an audio-based user-interaction module configured to obtain an audio input from a user.

### Claim 14 (depends on 8)

The user-interaction console of claim 8, wherein the first graphical information is arranged to indicate at least one of:
the at least one context-based command associated with the plurality of physical user-interaction controllers;
at least one icon, wherein the at least one icon is associated with at least one workspace page of the at least one software application;
a digital media associated with the at least one software application;
a virtual keyboard.

### Claim 15 (depends on 9)

The user-interaction console of claim 9, wherein the second graphical information is arranged to indicate at least one of:
a list of parameters pertaining to the at least one software application;
a list of the plurality of physical user-interaction controllers;
information pertaining to the at least one software application;
a widget or an application executed by the processor.
