# US 2025/0147594 A1 · System, Method, and Program for Realizing User Interface Based on Finger Identification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0147594 A1
- Document ID: US20250147594A1
- Status: Published application
- Kind: A1
- Application number: 18/730,482
- Filed: 2023-01-19
- Published: 2025-05-08
- Earliest claimed: 2022-01-19
- Assignees: Metamatiks PTE. LTD.
- Applicants: Metamatiks PTE. LTD.
- Inventors: Yohei INOUE
- Web page: https://patentgrep.com/patent/US20250147594A1

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)}
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0485: Scrolling or panning
- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
- G06F 2203/04104: Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
- G06F 2203/04803: Split screen, i.e. subdividing the display area or the window area into separate subareas
- G06F 2203/04806: Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
- G06F 2203/04808: Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Provided are a system, method, and program for realizing an intuitive and efficient user interface based on the identification of each finger of a user. Actions are shown using instantly recognizable icons, labels, or both, and the user can recognize the function to be performed by displaying instantly recognizable icons, labels, or both on the fingertips of virtual hands representing fingers identified by a camera or other means. It allows many types of direct actions on an object of an action, with the number of types being more than the number of fingers by mode switching.

## Claims

### Claim at position 1

-20. (canceled)

### Claim 21

A method for providing a user interface to a user, comprising steps of:
identifying a plurality of fingers of the user;
assigning a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and
switching functions assigned to a second finger and a third finger of the identified plurality of fingers to different functions in response to a touch action or a hover action by the first finger;
wherein at least one of the second finger and the third finger is different from the first finger; and
wherein when the object selection function is not assigned to the first finger, returning a function of the second finger or the third finger to its pre-switching function in response to release of the touch action or the hover action by the first finger.

### Claim 22 (depends on 21)

The method according to claim 21, wherein when the object selection function is assigned to the first finger, a selected state of the object is maintained even after the touch action or the hover action on the object is released.

### Claim 23 (depends on 21)

The method according to claim 21, wherein one or more functions assigned to the second finger include at least one of a command processing function, a parameter control function, and an object selection function.

### Claim 24 (depends on 23)

The method according to claim 23,
wherein one or more functions assigned to the second finger after switching include a parameter control function; and
wherein one or more function assigned to the third finger after switching include a function of changing the parameter control function.

### Claim 25 (depends on 23)

The method according to claim 23,
wherein one or more functions assigned to the second finger after switching include a parameter control function;
the method further includes assigning a function of changing the parameter control function to a finger different from the second finger in response to a touch action by the second finger.

### Claim 26 (depends on 23)

The method according to claim 23, wherein one or more functions assigned to the second finger after switching include at least one of a command processing function, an editing function, a search function, a saving function, a copying function, a calculation function, a sending function, and any combination thereof.

### Claim 27 (depends on 26)

The method according to claim 26, wherein the editing function includes at least one of moving, aligning, rotating, image editing, and adding text.

### Claim 28 (depends on 23)

The method according to claim 23, wherein the second finger is the same finger as the first finger.

### Claim 29 (depends on 21)

The method according to claim 21, further comprising a step of returning a function of the second finger or the third finger to a pre-switching function in response to release of the touch action or the hover action by the first finger within or below a predetermined time when an object selection function is not assigned to the first finger.

### Claim 30 (depends on 21)

The method according to claim 21, further comprising a step of displaying an icon or a label representing a function assigned to each of the identified fingers on a display used by the user along with a virtual hand representing the identified plurality of fingers.

### Claim 31 (depends on 30)

The method according to claim 30, wherein the label is displayed only when the finger associated with the label is positioned above a predetermined height from a desk used by the user.

### Claim 32 (depends on 30)

The method according to claim 30, wherein when the user's finger is outside a screen area of the display, the icon or the label is displayed at the edge of the screen.

### Claim 33

A non-transitory computer readable medium storing a program that when executed causes a computer to perform a method for providing a user interface to a user, the method comprising steps of:
identifying a plurality of fingers of the user;
assigning a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and
switching functions assigned to a second finger and a third finger of the identified plurality of fingers to different functions in response to a touch action or a hover action by the first finger;
wherein at least one of the second finger and the third finger is different from the first finger; and
wherein when the object selection function is not assigned to the first finger, returning a function of the second finger or the third finger to its pre-switching function in response to release of the touch action or the hover action by the first finger.

### Claim 34

An apparatus for providing a user interface to a user, configured to:
identify a plurality of fingers of the user;
assign a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and
switch functions assigned to a second finger and a third finger of the identified plurality of fingers to different functions in response to a touch action or a hover action by the first finger;
wherein at least one of the second finger and the third finger is different from the first finger; and
wherein when the object selection function is not assigned to the first finger, a function of the second finger or the third finger is returned to its pre-switching function in response to release of the touch action or the hover action by the first finger.

### Claim 35 (depends on 22)

The method according to claim 22, wherein one or more functions assigned to the second finger include at least one of a command processing function, a parameter control function, and an object selection function.
