# US 2024/0257649 A1 · Information processing device, information processing method, program, and mobile device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0257649 A1
- Document ID: US20240257649A1
- Status: Published application
- Kind: A1
- Application number: 18/633,670
- Filed: 2024-04-12
- Published: 2024-08-01
- Earliest claimed: 2018-09-04
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Masato NISHIO; Yuhei YABE
- Web page: https://patentgrep.com/patent/US20240257649A1

## CPC

- G08G 5/21: located onboard the aircraft
- B64C 13/16: actuated automatically, e.g. responsive to gust detectors
- B64C 39/024: {of the remote controlled vehicle type, i.e. RPV}
- G05D 1/0016: {characterised by the operator's input device (input arrangements for computing systems in general G06F3/00)}
- G05D 1/1064: {specially adapted for avoiding collisions with other aircraft}
- G05D 1/223: Command input arrangements on the remote controller, e.g. joysticks or touch screens
- G05D 1/693: for avoiding collisions between vehicles
- G06T 7/73: using feature-based methods
- G06V 10/243: {by compensating for image skew or non-uniform image deformations}
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06V 20/13: Satellite images
- G06V 20/17: taken from planes or by drones
- G08G 5/53: for cruising
- G08G 5/55: for a single aircraft
- G08G 5/723: {from the aircraft}
- G08G 5/80: Anti-collision systems
- H04N 5/38: Transmitter circuitry {for the transmission of television signals according to analogue transmission standards} (H04N5/14 takes precedence)
- H04N 23/10: for generating image signals from different wavelengths
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- B64U 2101/30: for imaging, photography or videography
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- B64U 2201/20: Remote controls
- G03B 15/006: {Apparatus mounted on flying objects}
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30204: Marker
- G06T 2207/30261: Obstacle
- G08G 5/57: for unmanned aircraft
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- B64C: AEROPLANES; HELICOPTERS
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)

## Abstract

It is intended to allow an orientation of a mobile device observed in imaging data to be more appropriately specified. There is provided an information processing device including an acquisition unit that acquires the imaging data and a direction specification unit that analyzes the imaging data to specify the orientation of the mobile device observed in the imaging data.

## Claims

### Claim 1

An information processing method to be implemented by a first mobile device, the method comprising:
acquiring imaging data;
analyzing the imaging data to specify an orientation of a second mobile device observed in the imaging data;
analyzing the imaging data to specify the position of the second mobile device observed in the imaging data;
predicting, based on the orientation and the position, the orientation or the position of the second mobile device at a given future time point; and
controlling, based on a result of predicting the orientation or the position of the second mobile device, movement of the first mobile device.

### Claim 2 (depends on 1)

The information processing method according to claim 1, wherein
specifying, as the orientation of the second mobile device, at least one of a direction in which the second mobile device is moving, a direction in which the second mobile device is movable, and a direction in which the second mobile device is expected to move.

### Claim 3 (depends on 1)

The information processing method according to claim 1, wherein
specifying, as the orientation of the second mobile device, at least one of a two-dimensional direction and a three-dimensional direction in which the second mobile device is oriented.

### Claim 4 (depends on 1)

The information processing method according to claim 1, wherein
specifying the orientation of the second mobile device based on a feature of the second mobile device, the feature being observed in the imaging data.

### Claim 5 (depends on 1)

The information processing method according to claim 1, wherein
specifying the orientation of the second mobile device based on light emitted from the second mobile device, the light being observed in the imaging data.

### Claim 6 (depends on 1)

The information processing method according to claim 1, wherein
specifying the orientation of the second mobile device based on an environment around the second mobile device, the environment being observed in the imaging data.

### Claim 7 (depends on 1)

The information processing method according to claim 1, wherein
specifying the orientation of the second mobile device based on a marker added to the second mobile device, the marker being observed in the imaging data.

### Claim 8 (depends on 1)

The information processing method according to claim 1, wherein
the first mobile device is a movable device including a camera that generates the imaging data.

### Claim 9 (depends on 8)

The information processing method according to claim 8, wherein
the camera that generates the imaging data is provided in an external device serving as a second movable device.

### Claim 10 (depends on 8)

The information processing method according to claim 8, wherein
recognizing that the specification of the orientation of the second mobile device is interrupted and controls the position to be moved forward/rearward, leftward/rightward, upward/downward, or a combination thereof.

### Claim 11 (depends on 1)

The information processing method according to claim 1, wherein
controlling, based on the result of predicting the orientation or the position of the second mobile device, the movement of the first mobile device to avoid a collision between the second mobile device and another object.

### Claim 12 (depends on 1)

The information processing method according to claim 1, wherein
at least one of the first mobile device or the second mobile device is an automobile.

### Claim 13 (depends on 5)

The information processing method according to claim 5, wherein
the light emitted from the second mobile device is a headlight of the second mobile device.

### Claim 14

A first mobile device comprising:
circuitry configured to
acquire imaging data,
analyze the imaging data to specify an orientation of a second mobile device observed in the imaging data,
analyze the imaging data to specify the position of the second mobile device observed in the imaging data,
predict, based on the orientation and the position, the orientation or the position of the second mobile device at a given future time point, and
control, based on a result of predicting the orientation or the position of the second mobile device, movement of the first mobile device.

### Claim 15 (depends on 14)

The first mobile device according to claim 14, wherein
the circuitry is configured to specify the orientation of the second mobile device based on light emitted from the second mobile device, the light being observed in the imaging data.

### Claim 16 (depends on 15)

The first mobile device according to claim 15, wherein
the light emitted from the second mobile device is a headlight of the second mobile device.

### Claim 17 (depends on 14)

The first mobile device according to claim 14, wherein
at least one of the first mobile device or the second mobile device is an automobile.

### Claim 18

A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a first mobile device, cause the first mobile device to perform a method, the method comprising:
acquiring imaging data;
analyzing the imaging data to specify an orientation of a second mobile device observed in the imaging data;
analyzing the imaging data to specify the position of the second mobile device observed in the imaging data;
predicting, based on the orientation and the position, the orientation or the position of the second mobile device at a given future time point; and
controlling, based on a result of predicting the orientation or the position of the second mobile device, movement of the first mobile device.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium according to claim 18, wherein the method further comprising:
specifying the orientation of the second mobile device based on light emitted from the second mobile device, the light being observed in the imaging data.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium according to claim 19, wherein
the light emitted from the second mobile device is a headlight of the second mobile device.
