# US 12,216,813 B2 · Merging multimodal multiuser interactions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,216,813 B2
- Document ID: US12216813B2
- Status: Granted
- Kind: B2
- Application number: 18/325,916
- Filed: 2023-05-30
- Published: 2023-11-30
- Granted: 2025-02-04
- Earliest claimed: 2022-05-31
- Assignees: Orange
- Applicants: Orange
- Inventors: Chantal Guionnet; Hélène Joucla
- Web page: https://patentgrep.com/patent/US12216813B2

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G06Q 10/101: Collaborative creation, e.g. joint development of products or services
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A processing method for a request for a digital service in an interactive environment, where the method includes: generating an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually, is only indicative of one simple request and where the enriched request triggers sending at least one command.

## Claims

### Claim 1

A processing method for a request for a digital service in an interactive environment, where the method comprises:
generating an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually, is only indicative of one simple request and where the enriched request triggers sending at least one command; and
detecting that the received signals originate from distinct users.

### Claim 2 (depends on 1)

The method according to claim 1, where the method comprises an interpretation of the received signals determining the enriched request on the basis of at least one of the following elements:
simple requests extracted from the received signals;
for each of the received signals, users originating the received signals; and
the result of a detection that the received signals originate from distinct users.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the method comprises identification, for each received signal, of a user originating the received signal.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the at least one of the signals comprises a user identifier and generating the enriched request comprises at least one of the following:
generating the enriched request based on the interpretation of the received signals and the associated user identifiers; and
generating the enriched request by adapting, according to user identifiers, the enriched request obtained by interpretation of the received signals.

### Claim 5

A processing method for a request for a digital service in an interactive environment, where the method comprises:
generating an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually, is only indicative of one simple request and where the enriched request triggers sending at least one command,
wherein, when at least one of the signals comprises contextual information, the generation of the enriched request adapts the enriched request based on an interpretation of the contextual information.

### Claim 6

A processing method for a request for a digital service in an interactive environment, where the method comprises:
generating an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually, is only indicative of one simple request and where the enriched request triggers sending at least one command, further comprising:
time-stamping the moments at least of beginning reception of the signals;
interpreting time gaps between moments of beginning respective reception of the signals; and
adapting the enriched request based on the interpretation of these time gaps.

### Claim 7

A processing method for a request for a digital service in an interactive environment, where the method comprises:
generating an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually. is only indicative of one simple request and where the enriched request triggers sending at least one command,
wherein at least one part of the received signals may come from a plurality of input interfaces distributed over various sites.

### Claim 8 (depends on 7)

The method according to claim 7, further comprising:
time-stamping the moments at least of beginning reception of the signals;
correcting the moments of beginning respective reception of the signals as a function of the latency between the sites;
interpreting corrected time gaps between corrected time-stamped moments; and
adapting the enriched request depending on the interpretation of the corrected time gaps.

### Claim 9 (depends on 1)

A processing circuit comprising a processor and a memory, the memory storing program code instructions of a computer program for executing the processing method according to claim 1 when the computer program is executed by the processor.

### Claim 10

A processing device for a request for a digital service in an interactive environment, where the processing device comprises:
a generator of an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually, is only indicative of one simple request and where the enriched request triggers sending at least one command; and
a detector that the received signals originate from distinct users.

### Claim 11 (depends on 10)

The processing device according to claim 10, wherein the processing device comprises an analyzer able to interpret the received signals determining the enriched request on the basis of at least one of the following elements:
simple requests extracted from the received signals; and
for each of the received signals, users originating the received signals.

### Claim 12 (depends on 11)

The processing device according to claim 11, wherein the processing device comprises a user identifier able to identify, for each received signal, a user originating the received signal.

### Claim 13 (depends on 10)

The processing device according to claim 10, wherein the processing device is implemented in at least one device among the following:
a terminal; and
a connected object manager able to command at least one object connected to a communication network by means of at least one command corresponding to the enriched request destined to at least one connected object.

### Claim 14

A processing device for a request for a digital service in an interactive environment, where the processing device comprises:
a generator of an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually, is only indicative of one simple request and where the enriched request triggers sending at least one command,
wherein, when at least one of the signals comprises contextual information, the generator of the enriched request adapts the enriched request based on an interpretation of the contextual information.

### Claim 15

A processing device for a request for a digital service in an interactive environment, where the processing device comprises:
a generator of an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually, is only indicative of one simple request and where the enriched request triggers sending at least one command;
a time-stamp module for time-stamping the moments at least of beginning reception of the signals;
an interpretation module for interpreting time gaps between moments of beginning respective reception of the signals; and
an adaptor module for adapting the enriched request based on the interpretation of these time gaps.

### Claim 16

A processing device for a request for a digital service in an interactive environment, where the processing device comprises:
a generator of an enriched request based on an interpretation of signals received from a multimodal set of human-machine interactions resulting from human-machine interactions coming from coordinated actions of a plurality of distinct users in the interactive environment, where each human-machine interaction, considered individually, is only indicative of one simple request and where the enriched request triggers sending at least one command,
wherein at least one part of the received signals can come from a plurality of input interfaces distributed over various sites.