# US 2020/0348765 A1 · Mutual interactivity between mobile devices based on position and orientation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0348765 A1
- Document ID: US20200348765A1
- Status: Published application
- Kind: A1
- Application number: 16/765,120
- Filed: 2018-12-03
- Published: 2020-11-05
- Earliest claimed: 2017-12-07
- Assignees: Elbit Systems Ltd.
- Applicants: Elbit Systems Ltd.
- Inventors: Eugene Gurovich; Israel Unterman; Yoav Ophir; Yoav Prager
- Web page: https://patentgrep.com/patent/US20200348765A1

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 1/1684: {Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675}
- G06F 1/1686: {the I/O peripheral being an integrated camera}
- G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer}
- G06F 1/1698: {the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone (details of antennas disposed inside a computer H01Q1/2266)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method for initiating a gesture-based mutual interaction scheme between a first and second mobile device, comprising: associating a gesture-based mutual interaction scheme between the first and second mobile device that associates a position scheme with a respective action, where the position scheme relates to any of: an absolute or relative orientation, and an absolute or relative trajectory; acquiring a first position property of the first mobile device and a second position property of the second mobile device; determining that each of the first and second position properties comply with the position scheme; triggering an execution of an action on the second mobile device, where the action is associated with the position scheme that the first position property complies with; where the second mobile device conditions the execution of the action triggered by the first mobile device on the compliance of the second position property with the position scheme.

## Claims

### Claim 1

A method for initiating at least one gesture-based mutual interaction scheme between a first mobile device and at least a second mobile device, said method comprising the procedures of:
associating at least one gesture-based mutual interaction scheme between said first mobile device and said at least second mobile device, said gesture-based mutual interaction scheme associating at least one position scheme with at least one respective action,
said at least one position scheme relating to at least one of: an absolute orientation, a relative orientation, an absolute trajectory, and a relative trajectory;
acquiring a first position property of said first mobile device and a second position property of said at least second mobile device;
determining that each of said first acquired position property and said second acquired position property comply with at least one of said at least one position scheme of said mutual interaction scheme;
triggering an execution of one of said at least one respective action on said at least second mobile device, said respective action triggered on said second mobile device associated with said at least one position scheme with which said position property of said first mobile device complies, in accordance with said gesture-based mutual interaction scheme; and
conditioning by said at least second mobile device said execution of said one of said at least one respective actions triggered by said first mobile device on said determined compliance of said second position property of said at least second mobile device.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising executing on said first mobile device, one of said one or more respective actions in response to said determined compliance of said acquired position property of said first mobile device.

### Claim 3 (depends on 1)

The method according to claim 1, wherein said acquired position property of said first mobile device relates to at least one of: an absolute orientation, a relative orientation, an absolute trajectory, and a relative trajectory of said first mobile device.

### Claim 4 (depends on 1)

The method according to claim 1, wherein said acquiring said position property of said first mobile device, and said determining that said acquired position property of said first mobile device complies with one of said at least one position scheme of said gesture-based mutual interaction scheme is performed by said first mobile device.

### Claim 5

(canceled)

### Claim 6

(canceled)

### Claim 7 (depends on 1)

The method according to claim 1, said acquisition of said position property of said at least second mobile device, and said determining of said compliance of said position property of said at least second mobile device is further performed by said first mobile device.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising:
triggering an execution of one of said one or more respective actions on said first mobile device, said respective action triggered on said first mobile device associated with said at least one position scheme with which said position property of said at least second mobile device complies, in accordance with said gesture-based mutual interaction scheme.

### Claim 9

(canceled)

### Claim 10 (depends on 1)

The method according to claim 1, wherein one of said one or more respective actions comprises indicating said determined compliance of said acquired position property of said first mobile device.

### Claim 11

(canceled)

### Claim 12

(canceled)

### Claim 13 (depends on 1)

The method according to claim 1,
wherein said position scheme further relates to at least one of an absolute position and a relative position, and
wherein said acquired first position property of said first mobile device and said acquired second position property of said at least second mobile device correspond to a distance between said first mobile device and said at least second mobile device,
wherein said at least one position scheme of said gesture-based mutual interaction scheme correspond to said acquired distance,
wherein determining further comprises determining that said acquired distance complies with said position scheme of said mutual interaction scheme corresponding to said acquired distance.

### Claim 14

(canceled)

### Claim 15 (depends on 1)

The method according to claim 1,
wherein said first position property of said first mobile device corresponds to a distance and orientation of said first mobile device with respect to an audio receiver, and
wherein said second position property of said at least second mobile device corresponds to a distance and orientation of said at least second mobile device with respect to said audio receiver,
wherein said at least one position scheme of said gesture-based mutual interaction scheme corresponds to an audio range with respect to said audio receiver,
wherein determining further comprises determining that said position property of said first mobile device and said position property of said at least second mobile device all comply with said audio range,
the method further comprising:
synchronizing an internal clock of said at least second mobile device with an internal clock of said first mobile device,
computing a phase shift for each of said at least second mobile device such that an audio file simultaneously transmitted by said first mobile device and each of said at least second mobile device constructively interferes at said audio receiver, and
rendering said audio file by said first device, and rendering said audio file by each of said at least second mobile device according to each respective phase shift.

### Claim 16

(canceled)

### Claim 17 (depends on 1)

The method according to claim 1, further comprising mutually notifying each of said first mobile device and said at least second mobile device of said respective compliances of said acquired position properties of said first mobile device and said at least second mobile device with said at least one position scheme of said gesture-based mutual interaction scheme,
wherein executing said respective action comprises invoking a multi-perspective imaging application at each of said first mobile device and said at least second mobile device,
wherein acquiring said respective position properties of said first mobile device and said at least second mobile device comprises capturing an image of an object with any of: a camera configured with said first mobile device and a camera configured with said at least second mobile device,
wherein said acquired position properties of said first mobile device and said at least second acquired position property are with respect to said object,
wherein said at least one position scheme of said gesture-based mutual interaction scheme with which said position property of said first mobile device complies, corresponds to an optical range of said camera configured with said first mobile device with respect to said object, and
wherein said at least one position scheme of said gesture-based mutual interaction scheme with which said position property of said at least second mobile device complies, corresponds to an optical range of said camera configured with said at least second mobile device with respect to said object.

### Claim 18

(canceled)

### Claim 19

(canceled)

### Claim 20 (depends on 1)

The method according to claim 1, further comprising:
depositing a visible trace,
wherein acquiring said first position property comprises acquiring a trajectory corresponding to said visible trace,
wherein determining that said acquired first position property complies with at least one of said at least one position scheme of said gesture-based mutual interaction scheme comprises determining that said acquired trajectory corresponds to said deposited visible trace, and
wherein triggering said at least second mobile device to execute said associated action comprises triggering said at least second mobile device to display a bit map corresponding to said visible trace.

### Claim 21

(canceled)

### Claim 22 (depends on 1)

The method according to claim 1, wherein said triggered one of said at least one respective action on said at least second mobile device comprises exchanging a security credential between said first mobile device and said at least second mobile device.

### Claim 23

A mutually interactive system, comprising:
a first mobile device comprising:
an inertial measurement unit configured to acquire a first position property of said first mobile device,
a transceiver; and
at least a second mobile device, each comprising:
an inertial measurement unit configured to acquire a second position property of each said at least second mobile device
a transceiver configured to communicate with said transceiver of said first mobile device,
said first and said at least a second mobile devices associated with at least one gesture-based mutual interaction scheme associating at least one position scheme with at least one respective action,
said at least one position scheme relating to at least one of: an absolute orientation, a relative orientation, an absolute trajectory, and a relative trajectory,
wherein said first mobile device is configured to:
determine that said acquired position property of said first mobile device complies with one of said at least one position schemes of said gesture-based mutual interaction scheme,
trigger, via said respective transceivers of first mobile device and said at least second mobile device, said processor of said at least second mobile device to execute one of said at least one respective action, said triggered action associated with said one of said at least one position schemes with which said first acquired position property complies, and
wherein said at least second mobile device is configured to
determine that said second acquired position property of said second mobile device complies with one of said at least one position schemes of said gesture-based mutual interaction scheme, and
condition an execution of said triggered action by said at least second mobile device of said at one of said at least one respective actions triggered by said first mobile device on said determined compliance of said second position property of said at least second mobile device.

### Claim 24

(canceled)

### Claim 25

(canceled)

### Claim 26 (depends on 23)

The system according to claim 23, wherein said respective acquired position property of said first mobile device and said at least second mobile device relates to at least one of: an absolute orientation, a relative orientation, an absolute trajectory, and a relative trajectory of said respective first mobile device and said at least second mobile device.

### Claim 27 (depends on 23)

The system according to claim 23, wherein any of said first mobile device and said second mobile device further comprise a camera configured to acquire any of: said first position property of said first mobile device, and said second position property of said at least second mobile device, and
wherein said first mobile device is further configured to determine that said acquired position property of said at least second mobile device complies with at least one of said at least one position schemes of said gesture-based mutual interaction scheme, and
wherein said at least second mobile device is further configured to determine that said acquired position property of said first mobile device complies with at least one of said at least one position schemes of said gesture-based mutual interaction scheme.

### Claim 28 (depends on 23)

The system according to claim 23, wherein said at least second mobile device is further configured to trigger an execution of one of said one or more respective actions on said first mobile device, said respective action triggered on said first mobile device associated with said at least one position scheme with which said second position property of said least second mobile device complies, in accordance with said gesture-based mutual interaction scheme.

### Claim 29 (depends on 23)

The system according to claim 23, wherein one of said one or more respective actions comprises indicating said determined compliance of said acquired first position property of said first mobile device.

### Claim 30

(canceled)

### Claim 31

(canceled)

### Claim 32

(canceled)

### Claim 33

(canceled)

### Claim 34 (depends on 23)

The system according to claim 23,
wherein said first mobile device is operative as an audio transmitter and further comprises a speaker, and
wherein said at least second mobile device is operative as an audio receiver and further comprises a speaker,
wherein said at least one position scheme of said gesture-based mutual interaction scheme corresponds to an audio range with respect to an audio receiver,
wherein said first acquired position property of said first mobile device corresponds to a distance and orientation of said first mobile device with respect to said audio receiver, and
wherein said first mobile device is further configured to:
receive at least said second position property corresponding to a distance and orientation of said at least second mobile device with respect to said first mobile device,
determine that said acquired first position property and said at least second position property both comply with said audio range respective of said audio receiver,
synchronizing an internal clock of said at least second mobile device with an internal clock of said first mobile device,
computing a phase shift for each of said at least second mobile device such that an audio file simultaneously transmitted by said first mobile device and each of said at least second mobile device constructively interferes at said audio receiver, and
rendering said audio file by said first device, and rendering said audio file by each of said at least second mobile device according to each respective phase shift.

### Claim 35 (depends on 23)

The system according to claim 23,
wherein said acquired position properties of said first mobile device and said at least second mobile device are with respect to an object,
wherein said at least one position scheme of said gesture-based mutual interaction scheme with which said first position property of said first mobile device complies, corresponds to an optical range of said camera configured with said first mobile device with respect to said object, and
wherein said at least one position scheme of said gesture-based mutual interaction scheme with which said second position property of said at least second mobile device complies, corresponds to an optical range of said camera configured with said at least second mobile device with respect to said object,
wherein said at least one respective action configured to be executed by said at least second mobile device is a multi-perspective imaging application, and
wherein said first mobile device is configured to execute said multi-perspective imaging application.

### Claim 36 (depends on 35)

The system of claim 35, further comprising:
an image processor,
wherein executing said multi-perspective imaging application comprises:
capturing at least one image of said object,
simultaneous with capturing said image, acquiring associated metadata, said associated metadata comprising position and orientation properties associated with said captured image, and a time stamp associated with said captured image, and
providing said captured image and said associated metadata to said image processor,
wherein said image processor is configured to:
receive from said first mobile device and said at least second mobile device, said multiple captured images with said associated metadata, and
process said multiple captured images using said associated metadata to perform any of: creating a panoramic image of said object, creating a multi-perspective image of said object, and tracking said object, and
provide said result of said processing step to any of said first and said at least second mobile device.

### Claim 37 (depends on 23)

The system according to claim 23,
wherein said first mobile device is configured deposit a visible trace, and
wherein said at least second mobile device is configured to: display a bit map corresponding to said visible trace at a user interface of said at least second mobile device, and store said bit map at a memory of said at least second mobile device,
wherein determining that said acquired first position property of said first mobile device complies with at least one of said at least one position scheme of said gesture-based mutual interaction scheme comprises determining that said acquired first position property of said first mobile device corresponds to said deposited visible trace, and
wherein triggering said at least second mobile device to execute said associated action comprises triggering said at least second mobile device to display said bit map corresponding to said visible trace.

### Claim 38 (depends on 23)

The system according to claim 23, wherein said triggered one of said at least one respective action on said at least second mobile device comprises exchanging a security credential between said first mobile device and said at least second mobile device.

### Claim 39

A method for initiating at least one gesture-based mutual interaction scheme between a mobile device and a computer, said method comprising the procedures of:
associating at least one gesture-based mutual interaction scheme between said mobile device and said computer,
said gesture-based mutual interaction scheme associating at least one position scheme with at least one respective action,
said at least one position scheme relating to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of said mobile device;
acquiring a position property of said mobile device;
determining that said acquired position property of said mobile device complies with at least one of said at least one position scheme of said gesture-based mutual interaction scheme; and
triggering an execution of one of said at least one respective action on a display of said computer, said respective action triggered on said display associated with said at least one position scheme with which said position property of said mobile device complies, in accordance with said gesture-based mutual interaction scheme,
wherein said at least one respective action comprises virtually coupling said mobile device with a three dimensional object displayed on said display, and manipulating a rendition of said three dimensional object on said display to correspond to said acquired position property of said mobile device.

### Claim 40

(canceled)
