# US 2024/0257359 A1 · Device, computer program and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0257359 A1
- Document ID: US20240257359A1
- Status: Published application
- Kind: A1
- Application number: 18/633,213
- Filed: 2024-04-11
- Published: 2024-08-01
- Earliest claimed: 2020-03-06
- Assignees: Sony Group Corporation; Sony Europe Limited
- Applicants: Sony Group Corporation; Sony Europe B.V.
- Inventors: Antonio AVITABILE; Salvatore FINATTI
- Web page: https://patentgrep.com/patent/US20240257359A1

## CPC

- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- H04N 19/115: Selection of the code volume for a coding unit prior to coding
- G06N 3/02: Neural networks
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00)
- H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference
- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/124: Quantisation
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method comprising the steps of: receiving an image from a dynamic vision sensor and an image from an image sensor, wherein the dynamic vision sensor and the image sensor capture at least an overlapping field of view; determining, from the image received from the dynamic vision sensor, an area of movement in the field of view of the image from the image sensor; applying a higher level of image compression to the areas of no movement compared with the area of movement in the image from the image sensor to produce a processed image; and outputting the processed image to a neural network.

## Claims

### Claim 1

A method comprising:
receiving a first image data from a dynamic vision sensor and a second image data from an image sensor, wherein the dynamic vision sensor and the image sensor capture images having an overlapping field of view;
determining, from at least the first image data received from the dynamic vision sensor and at least the second image data from the image sensor, an area of movement;
applying image compression based on areas of movement to produce a processed image; and
outputting the processed image to a neural network.

### Claim 2 (depends on 1)

The method according to claim 1, comprising:
receiving a plurality of positions defining the area of movement from the dynamic vision sensor.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the plurality of positions are pixel positions.

### Claim 4 (depends on 2)

The method according to claim 2, wherein the plurality of positions define the corners of the area of movement.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the amount of image compression is determined in accordance with the amount of movement within the area.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the dynamic vision sensor and the image sensor are integrated into a single sensor.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the applying image compression based on areas of movement to produce the processed image includes applying a higher level of compression to areas of non-movement.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the determining the area of movement includes determining a relative movement between the dynamic vision sensor and the image sensor and a detected object.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising applying the image compression based on the determined relative movement.

### Claim 10 (depends on 1)

A non-transitory computer-readable storage medium storing computer readable instructions which, when loaded onto a computer, configure the computer to perform the method according to claim 1.

### Claim 11

An apparatus comprising:
circuitry configured to:
receive a first image data from a dynamic vision sensor and a second image data from an image sensor, wherein the dynamic vision sensor and the image sensor capture images having an overlapping field of view,
determine, from at least the first image data received from the dynamic vision sensor and at least the second image data from the image sensor, an area of movement,
apply image compression based on areas of movement to produce a processed image, and
output the processed image to a neural network.

### Claim 12 (depends on 11)

The apparatus according to claim 11, wherein the circuitry is further configured to:
receive a plurality of positions defining the area of movement from the dynamic vision sensor.

### Claim 13 (depends on 12)

The apparatus according to claim 12, wherein the plurality of positions are pixel positions.

### Claim 14 (depends on 12)

The apparatus according to claim 12, wherein the plurality of positions define the corners of the area of movement.

### Claim 15 (depends on 11)

The apparatus according to claim 11, wherein the amount of image compression is determined in accordance with the amount of movement within the area.

### Claim 16 (depends on 11)

The apparatus according to claim 11, wherein the dynamic vision sensor and the image sensor are integrated into a single sensor.

### Claim 17 (depends on 11)

The apparatus according to claim 11, wherein the circuitry is further configured to apply image compression based on areas of movement to produce the processed image by being further configured to apply a higher level of compression to areas of non-movement.

### Claim 18 (depends on 11)

The apparatus according to claim 11, wherein the circuitry is further configured to determine the area of movement by being further configured to determine a relative movement between the dynamic vision sensor and the image sensor and a detected object.

### Claim 19 (depends on 18)

The apparatus according to claim 18, wherein the circuitry is further configured to apply the image compression based on the determined relative movement.

### Claim 20

An apparatus comprising:
a dynamic vision sensor;
an image sensor; and
circuitry configured to:
receive a first image data from the dynamic vision sensor and a second image data from the image sensor, wherein the dynamic vision sensor and the image sensor capture images having an overlapping field of view,
determine, from at least the first image data received from the dynamic vision sensor and at least the second image data from the image sensor, an area of movement,
apply image compression based on areas of movement to produce a processed image, and
output the processed image to a neural network.
