# US 12,217,622 B2 · Direction presentation apparatus, direction presentation method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,622 B2
- Document ID: US12217622B2
- Status: Granted
- Kind: B2
- Application number: 17/617,972
- Filed: 2019-06-12
- Published: 2022-08-11
- Granted: 2025-02-04
- Earliest claimed: 2019-06-12
- Assignees: Nippon Telegraph and Telephone Corporation
- Applicants: Nippon Telegraph and Telephone Corporation
- Inventors: Hiroaki Gomi; Ryoma Tanase
- Web page: https://patentgrep.com/patent/US12217622B2

## CPC

- G09B 21/003: {using tactile presentation of the information, e.g. Braille displays}
- A61H 3/061: {with electronic detecting or guiding means}
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- A61H 2003/063: {with tactile perception}
- A61H 2201/0153: hand-held
- A61H 2201/5092: Optical sensor
- G01C 21/08: involving use of the magnetic field of the earth
- G01C 21/20: Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence)
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- A61H: PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY (electrotherapy, magnetotherapy, radiation therapy or ultrasound therapy A61N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)

## Abstract

To provide a technique for presenting a guided person with a direction in consideration of a relative orientation of a force sense presentation device with respect to an orientation of a camera. A direction presentation apparatus includes an instructed direction calculating unit which calculates, using a first angle in an external coordinate system representing an orientation of a camera that takes an image in front of a guided person and a second angle in the external coordinate system representing an orientation of a force sense presentation device that presents the guided person with a predetermined direction, an instructed force sense vector (hereinafter, referred to as a second instructed force sense vector) in a force sense presentation device coordinate system representing a direction to be instructed to the force sense presentation device from an instructed force sense vector (hereinafter, referred to as a first instructed force sense vector) in a camera coordinate system representing the direction to be instructed to the force sense presentation device.

## Claims

### Claim 1

A direction presentation apparatus, comprising circuitry configured to:
calculate, using a first angle and a second angle, a second instructed force sense vector from a first instructed force sense vector,
wherein
the first angle is in an external coordinate system and represents an orientation of a camera that takes an image in front of a guided person,
the second angle is in the external coordinate system and represents an orientation of a force sense presentation device that presents the guided person with a predetermined direction,
the external coordinates refer to coordinates capable of representing north, south, east and west,
the second instructed force sense vector represents a vector representing a direction input to the force sense presentation device in a force sense presentation device coordinate system,
the first instructed force sense vector represents a vector corresponding to the predetermined direction in a camera coordinate system,
the force sense presentation device coordinates refer to coordinates capable of representing the direction with respect to the force sense presentation device's orientation, and
the camera coordinates refer to coordinates capable of representing the direction with respect to the camera's orientation.

### Claim 2 (depends on 1)

The direction presentation apparatus according to claim 1, wherein
when θcw denotes the first angle, θdw denotes the second angle, fc denotes the first instructed force sense vector, and fd denotes the second instructed force sense vector,
the circuitry further configured to calculate the second instructed force sense vector fd by the following equations
f
d
=
d
P
c
·
f
c
d
P
c
=
[
cos
⁢
θ
d
c
sin
⁢
θ
d
c
-
sin
⁢
θ
d
c
cos
⁢
θ
d
c
]
⁢
θ
d
c
=
θ
d
w
-
θ
c
w
.

### Claim 3

A direction presentation apparatus, comprising circuitry configured to:
calculate a second instructed force sense vector from a first instructed force sense vector, using a first vector, a second vector, a third vector, a fourth vector, a first angle, and a second angle, wherein
the first vector represents an azimuth vector in a camera coordinate system and corresponds to a first azimuth in an external coordinate system,
the camera coordinates refer to coordinates capable of representing the direction with respect to the camera's orientation,
the external coordinates refer to coordinates capable of representing north, south, east and west,
the second vector represents an azimuth vector in a force sense presentation device coordinate system and corresponds to a second azimuth in the external coordinate system,
the force sense presentation device coordinates refer to coordinates capable of representing the direction with respect to the force sense presentation device's orientation,
the third vector is in the camera coordinate system and represents an orientation of gravitational force being applied to the camera,
the fourth vector is in the force sense presentation device coordinate system and represents an orientation of gravitational force being applied to the force sense presentation device,
the first angle is in the external coordinate system and represents an orientation of the camera,
the second angle is in the external coordinate system and represents an orientation of the force sense presentation device,
the second instructed force sense vector represents a vector representing a direction input to the force sense presentation device in the force sense presentation device coordinate system, and
the first instructed force sense vector represents a vector corresponding to a predetermined direction presented to a guided person in the camera coordinate system.

### Claim 4 (depends on 3)

The direction presentation apparatus according to claim 3, wherein
when rc denotes a compound matrix constructed from the first vector, the third vector, and a cross product vector of the first vector and the third vector, rd denotes a compound matrix constructed from the second vector, the fourth vector, and a cross product vector of the second vector and the fourth vector, θcw denotes the first angle, θdw denotes the second angle, fc denotes the first instructed force sense vector, and fd denotes the second instructed force sense vector,
the circuitry further configured to calculate the second instructed force sense vector fd by the following equations when amounts of change in acceleration based on the third vector and the fourth vector are within a predetermined range indicating that the amounts of change in acceleration are small
dRc=rd·(rc)−1
fd=dRc·fc,
but otherwise the circuitry is further configured to calculate the second instructed force sense vector fd by the following equations
f
d
=
d
P
c
·
f
c
⁢
d
P
c
=
[
cos
⁢
θ
d
c
sin
⁢
θ
d
c
-
sin
⁢
θ
d
c
cos
⁢
θ
d
c
]
⁢
θ
d
c
=
θ
d
w
-
θ
c
w
.

### Claim 5

A direction presentation method, comprising:
an instructed direction calculating step of calculating a second instructed force sense vector from a first instructed force sense vector, using a first angle and a second angle, wherein
the first angle is in an external coordinate system and represents an orientation of a camera that takes an image in front of a guided person,
the second angle is in the external coordinate system and represents an orientation of a force sense presentation device that presents the guided person with a predetermined direction,
the external coordinates refer to coordinates capable of representing north, south, east and west,
the second instructed force sense vector represents a vector representing a direction input to the force sense presentation device in a force sense presentation device coordinate system,
the first instructed force sense vector represents a vector corresponding to the predetermined direction in a camera coordinate system,
the force sense presentation device coordinates refer to coordinates capable of representing the direction with respect to the force sense presentation device's orientation, and
the camera coordinates refer to coordinates capable of representing the direction with respect to the camera's orientation.

### Claim 6

A direction presentation method, comprising:
an instructed direction calculating step of calculating a second instructed force sense vector from a first instructed force sense vector, using a first vector, a second vector, a third vector, a fourth vector, a first angle, and a second angle, wherein
the first vector represents an azimuth vector in a camera coordinate system and corresponds to a first azimuth in an external coordinate system,
the camera coordinates refer to coordinates capable of representing the direction with respect to the camera's orientation,
the external coordinates refer to coordinates capable of representing north, south, east and west,
the second vector represents an azimuth vector in a force sense presentation device coordinate system and corresponds to a second azimuth in the external coordinate system,
the force sense presentation device coordinates refer to coordinates capable of representing the direction with respect to the force sense presentation device's orientation,
the third vector is in the camera coordinate system and represents an orientation of gravitational force being applied to the camera,
the fourth vector is in the force sense presentation device coordinate system and represents an orientation of gravitational force being applied to the force sense presentation device,
the first angle is in the external coordinate system and represents an orientation of the camera,
the second angle is in the external coordinate system and represents an orientation of the force sense presentation device,
the second instructed force sense vector represents a vector representing a direction input to the force sense presentation device in the force sense presentation device coordinate system, and
the first instructed force sense vector represents a vector corresponding to a predetermined direction presented to a guided person in the camera coordinate.

### Claim 7 (depends on 1)

A non-transitory computer-readable storage medium which stores a program for causing a computer to function as the direction presentation apparatus according to claim 1.

### Claim 8 (depends on 3)

A non-transitory computer-readable storage medium which stores a program for causing a computer to function as the direction presentation apparatus according to claim 3.