# US 2026/0086707 A1 · Methods and systems for multimodal dragging interactions with virtual objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0086707 A1
- Document ID: US20260086707A1
- Status: Published application
- Kind: A1
- Application number: 19/407,092
- Filed: 2025-12-03
- Published: 2026-03-26
- Earliest claimed: 2023-06-15
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Mohi REZA; Che YAN; Soheil KIANZAD; Wei LI
- Web page: https://patentgrep.com/patent/US20260086707A1

## CPC

- G06F 3/0486: Drag-and-drop
- G06F 3/03545: {Pens or stylus}
- G06F 3/048: Interaction techniques based on graphical user interfaces [GUI]
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 2015/223: {Execution procedure of a spoken command}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

There are provided methods and systems for multimodal dragging interactions with virtual objects. In examples, dragging interactions may be assisted by audio input in the form of voice commands. In response to detecting that a dragging gesture has been initiated, voice recognition is enabled. In examples, one or more voice commands for instructing a modification to a virtual object during a dragging gesture is received. A modification action for modifying the virtual object is determined, based on the dragging gesture and the voice command. In response to detecting a completion of the dragging gesture, the virtual object, modified using the one or more modification actions, is placed at the destination. The disclosed methods and systems may enable improved UI interaction with virtual objects, by enabling the modification of virtual objects during dragging gestures.

## Claims

### Claim 1

A computer implemented method comprising:
in response to detecting an initiation of a dragging gesture for moving a virtual object from a displayed source location within a graphical user interface (GUI) to a displayed destination location within the GUI, enabling voice recognition;
receiving a voice command for instructing a modification to the virtual object;
determining one or more modification actions for modifying the virtual object, based on the dragging gesture and the voice command; and
in response to detecting a completion of the dragging gesture, placing the virtual object, modified using the one or more modification actions, at the displayed destination location.

### Claim 2 (depends on 1)

The method of claim 1, wherein the GUI includes one or more interactive elements, each of the one or more interactive elements being associated with a respective selectable modification action for modifying the virtual object.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining the one or more modification actions comprises:
determining at least one of the one or more interactive elements were traversed by a dragging path of the dragging gesture; and
for each of the traversed portal elements:
activating the interactive element.

### Claim 4 (depends on 2)

The method of claim 2, wherein determining the one or more modification actions comprises:
determining that the voice command corresponds to a modification action associated with at least a corresponding one of the one or more interactive elements; and
for each of the at least corresponding one of the one or more interactive elements:
activating the portal element.

### Claim 5 (depends on 3)

The method of claim 3, further comprising:
for each activated interactive element of the one or more interactive elements:
altering an appearance of the activated interactive element.

### Claim 6 (depends on 3)

The method of claim 3, further comprising, prior to activating the interactive element:
altering an appearance of the interactive element to include a parent element representing a category of modification actions and at least one child element representing at least one of the one or more modification actions.

### Claim 7 (depends on 2)

The method of claim 2, wherein the one or more interactive elements are arranged in an interactive element menu, the interactive element menu being displayed at a fixed position on a display of an electronic device.

### Claim 8 (depends on 2)

The method of claim 2, wherein the one or more interactive elements are arranged in an interactive element menu, the interactive element menu being dynamically positioned on a display of an electronic device based on a displayed location of the source.

### Claim 9 (depends on 2)

The method of claim 2, wherein the one or more interactive elements are arranged in an interactive element menu, the interactive element menu being dynamically positioned on a display of an electronic device based on the displayed location of the destination.

### Claim 10 (depends on 1)

The method of claim 1, wherein enabling voice recognition includes activating a microphone for receiving a speech signal.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
in response to detecting the completion of the dragging gesture, deactivating the microphone.

### Claim 12 (depends on 1)

The method of claim 1, wherein the dragging gesture is representative of a movement of one of:
a pointer within the GUI;
a digital pen or stylus in contact with a touch sensitive surface of a display of an electronic device; or
a finger in contact with the touch sensitive surface of the display of the electronic device.

### Claim 13

A system comprising:
one or more processors; and
a memory storing machine-executable instructions which, when executed by the processor device, cause the system to:
in response to detecting an initiation of a dragging gesture for moving a virtual object from a displayed source location within a graphical user interface (GUI) to a displayed destination location within the GUI, enable voice recognition;
receive a voice command for instructing a modification to the virtual object;
determine one or more modification actions for modifying the virtual object, based on the dragging gesture and the voice command; and
in response to detecting a completion of the dragging gesture, place the virtual object, modified using the one or more modification actions, at the displayed destination location.

### Claim 14 (depends on 13)

The system of claim 13, wherein the GUI includes one or more interactive elements, each of the one or more interactive elements being associated with a respective selectable modification action for modifying the virtual object.

### Claim 15 (depends on 14)

The method of claim 14, wherein the machine-executable instructions, when executed by the one or more processors to determine the one or more modification actions, further cause the system to:
determine at least one of the one or more interactive elements were traversed by a dragging path of the dragging gesture; and
for each of the traversed portal elements:
activate the interactive element.

### Claim 16 (depends on 14)

The system of claim 14, wherein the machine-executable instructions, when executed by the one or more processors to determine the one or more modification actions, further cause the system to:
determine that the voice command corresponds to a modification action associated with at least a corresponding one of the one or more interactive elements; and
for each of the at least corresponding one of the one or more interactive elements:
activate the portal element.

### Claim 17 (depends on 15)

The system of claim 15, wherein the machine-executable instructions, when executed by the one or more processors, further cause the system to:
for each activated interactive element of the one or more interactive elements:
alter an appearance of the activated interactive element.

### Claim 18 (depends on 15)

The system of claim 15, wherein the machine-executable instructions, when executed by the one or more processors, further cause the system to:
prior to activating the interactive element:
alter an appearance of the interactive element to include a parent element representing a category of modification actions and at least one child element representing at least one of the one or more modification actions.

### Claim 19 (depends on 13)

The system of claim 13, wherein the dragging gesture is representative of a movement of one of:
a pointer within the GUI;
a digital pen or stylus in contact with a touch sensitive surface of a display of an electronic device; or
a finger in contact with the touch sensitive surface of the display of the electronic device.

### Claim 20

A non-transitory computer-readable medium having machine-executable instructions stored thereon which, when executed by a processor of a device, cause the device to:
in response to detecting an initiation of a dragging gesture for moving a virtual object from a displayed source location within a graphical user interface (GUI) to a displayed destination location within the GUI, enable voice recognition;
receive a voice command for instructing a modification to the virtual object;
determine one or more modification actions for modifying the virtual object, based on the dragging gesture and the voice command; and
in response to detecting a completion of the dragging gesture, place the virtual object, modified using the one or more modification actions, at the displayed destination location.
