# US 2017/0256071 A1 · Dynamic scene analysis method, and associated analysis module and computer programme

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0256071 A1
- Document ID: US20170256071A1
- Status: Published application
- Kind: A1
- Application number: 15/316,779
- Filed: 2015-06-02
- Published: 2017-09-07
- Earliest claimed: 2014-06-06
- Assignees: Inria Institut National De Recherche En Informatiqe Et En Automatique; Centre National De La Recherche Scintifique (Cnrs); Commissariat A L'energie Atomique Et Aux Energies Alternatives
- Applicants: Inria Institut National De Recherche En Informatiqe Et En Automatique; Centre National De La Recherche Scintifique (Cnrs); Commissariat A L'energie Atomique Et Aux Energies Alternatives
- Inventors: CHRISTIAN LAUGIER; AMAURY NEGRE; MATHIAS PERROLLAZ; LUKAS RUMMELHARD
- Web page: https://patentgrep.com/patent/US20170256071A1

## CPC

- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- B60T 7/22: initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle {, or by means of contactless obstacle detectors mounted on the vehicle}
- B60W 40/04: Traffic conditions
- G01C 21/3807: {characterised by the type of data}
- G01C 21/3833: {characterised by the source of data}
- G01C 21/387: {Organisation of map data, e.g. version management or database structures}
- G01S 17/87: Combinations of systems using electromagnetic waves other than radio waves
- G01S 17/931: of land vehicles
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/277: involving stochastic approaches, e.g. using Kalman filters
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- B60T 2201/022: Collision avoidance systems
- B60W 40/12: related to parameters of the vehicle itself {, e.g. tyre models}
- B60W 2420/403: Image sensing, e.g. optical camera
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20076: Probabilistic image processing
- G08G 1/04: using optical or ultrasonic detectors
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B60T: VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL (electrodynamic brake systems for vehicle, in general B60L; brakes per se, i.e. devices where braking effect occurs, including ultimate brake actuators, F16D); ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A method for analysing a dynamic scene partitioned into cells which involves determining a probability of occupancy of a cell and a probability or probabilities of movement of the cell by solving the equation P(OV|ZC)=ΣAO−1 V-1P(CAOO−1VV−1Z)/ΣAOO−1VV−1P(CAOO−VV−1Z) comprising the determination of the speeds and positions of dummy particles in the grid depending on those determined at the (k−1)th iteration and the probability P(VFV-1); the determination of the particles located in each cell depending on the determined positions, and the solving of the equation, for a cell, is split into the solving of a static part corresponding to P(O=empty, V=O|ZC) and P(O=occupied, V=O|ZC) and the solving of a dynamic part corresponding to P(O=occ, V=vkl,|ZC), i=32 1 to nk, in which nk is the number of particles determined in cell C for the kth iteration.

## Claims

### Claim 1

A method for analyzing a dynamic scene observed with the aid of one or more sensors, the method comprising:
defining a grid that is partitioned into cells and corresponding to the observed scene;
collecting at least one new observation of the one or more sensors at the kth iterations;
determining, as a function of the new collected observation, a first probability of occupancy of each cell of the grid modeling the operation of the one or more sensors ;
determining, for each cell, at the kth iteration, a second probability of occupancy of the cell and a set of probabilities of motion of the content of the cell as a function of the first probability of occupancy of the cell determined at the kth iteration, wherein the second probability of occupancy of the cell and of the set of probabilities of motion of the content of the cell as a function of the first probability of occupancy of the cell determined at the kth iteration is determined based on:
P

(
OV
|
ZC
)
=
Σ
AO
-
1

V
-
1

P

(
CAOO
-
1

VV
-
1

Z
)
Σ
AOO
-
1

VV
-
1

P

(
CAOO
-
1

VV
-
1

Z
)
an identifier of the cell considered;
an identifier of the cell which contained, at the (k−1)th iteration, what is contained in the cell considered at the kth iteration;
an occupancy state of the cell considered, from among the “empty” and states;
an occupancy state of the cell at the (k−1)th iteration;
a velocity of the cell considered;
a velocity of the cell at the (k−1)th iteration; and
observations of the sensors from the first iteration up to the kth iteration;
wherein respective velocity and the respective position of a set of dummy particles in the grid are determined at the kth iteration as a function of the velocities, positions of the particles are determined at the (k−1)th iteration and of a probability P(V|V−1);
determining the particles located in each cell as a function of the positions determined and in that the solving of the equation, for a cell, is split into the solving of a static part corresponding to P(O=empty, V=0|ZC) and P(O=occupied, V=0|ZC) and into the solving of a dynamic part corresponding to the P(O=occ, V=vik,|ZC), i=1 to nk, where nk is the number of particles determined in the cell C for the kth iteration; and
wherein the static part of the cell C at the kth iteration being determined as a function of the static part of the cell C determined at the (k−1)th iteration and of P(O|O−1); the probability of P(O=occupied, V=vik,|ZC) of the dynamic part of the cell C being determined at the kth iteration as a function of the probability P(O=occupied, V=vik−1|ZA) calculated at the (k−1)th iteration for the dynamic part of the cell A and of P(O|O−1), where the particle pi determined in the cell C at the kth iteration with a velocity vik was situated in the cell A at the (k−1)th iteration with a velocity

### Claim 2 (depends on 1)

The method of claim 1, wherein on completion of the kth iteration, one or more pairs (pi, (vik, xik)), where xik is the position of the particle pi at the kth iteration, are duplicated within a cell or deleted so that the number of pairs per cell is dependent on the dynamic part determined for the cell.

### Claim 3 (depends on 2)

The method of claim 2, further comprising selecting the pair to be duplicated or deleted, wherein the selection of said pair being carried out as a function of the probability P(O=occupied, V=vik|ZC) is determined at the kth iteration, and where vik is a velocity component of the pair.

### Claim 4 (depends on 2)

The method of claim 2, wherein the total number of particles in the grid is constant during the iterations.

### Claim 5 (depends on 1)

The method of claim 1, wherein P(O=empty, V=0|ZC) is determined at the kth iteration is denoted as Pk(O=empty, V=0|ZC), and is determined as a function of the product of a first term dependent on the first probability of occupancy and of a second term, said second term being dependent on:
Pk−1(O=occ, V=0|ZC).(1-ε)+Pk−1(O=empty, V=0|ZC).ε, and/or
coeff (kvi).pk−1)O=occupied, V=vik−1, |ZA).(1-ε), where coeff (vik) is a decreasing function of ∥vik∥, and/or
a probability of appearance pa of a new object in the observed scene.

### Claim 6

(canceled)

### Claim 7

A device effective to analyze a dynamic scene observed with the aid of one or more sensors, the device is configured to:
define a grid partitioned into cells and corresponding to the observed scene;
collect at least one new observation of the one or more sensors at a kth iteration and determining, as a function of the new collected observation, a first probability of occupancy of each cell of the grid and that models the operation of the one or more sensors;
determine, for each cell, at the kth iteration, a second probability of occupancy of the cell and a set of probabilities of motion of the content of the cell as a function of the first probability of occupancy of the cell determined at the kth iteration, wherein the device is adapted to determine said second probability of occupancy of the cell and the set of probabilities of motion of the content of the cell as a function of the first probability of occupancy of the cell determined at the kth iteration based on:
P

(
OV
|
ZC
)
=
Σ
AO
-
1

V
-
1

P

(
CAOO

1

VV

1

Z
)
Σ
AOO
-
1

VV
-
1

P

(
CAOO
-
1

VV
-
1

Z
)
an identifier of the cell considered;
an identifier of the cell which contained, at the (k−1)th iteration, what is contained in the cell considered at the kth iteration;
an occupancy state of the cell considered, from among the empty and occupied states;
an occupancy state of the cell at the (k−1)th iteration;
a velocity of the cell considered;
a velocity of the cell at the (k−1)th iteration;
an observations of the sensors from the first iteration up to the kth iteration;
wherein the device is further configured to;
determine, at the kth iteration, the respective velocity and the respective position of a set of dummy particles in the grid as a function of the velocities, of the positions of the particles determined at the (k−1)th iteration and of the probability P(V|V−1);
determine particles located in each cell as a function of the positions determined and to split the solving of the equation, for a cell, into the solving of a static part corresponding to P(O=empty, V=1|ZC) and P(O=occupied, V=0|ZC) and into the solving of a dynamic part corresponding to the P(O=occ, V=vik,|ZC), i=1 to nk, where nk is the number of particles determined in the cell C for the kth iteration;
determine the static part of the cell C at the kth iteration as a function of the static part of the cell C determined at the (k−1)th iteration and of P(O|O−1); and
determine the probability of P(O=occupied, V=vik,|ZC) of the dynamic part of the cell C at the kth iteration as a function of the probability P(O=occupied, V=vik−1,|ZA) calculated at the (k−1)th iteration for the dynamic part of the cell A and of P(O|O−1), where the particle pi determined in the cell C at the kth iteration with a velocity vik was situated in the cell A at the (k−1)th iteration with a velocity vik−1.

### Claim 8 (depends on 7)

The device of claim 7, adapted to, on completion of the kth iteration, duplicate or delete one or more pairs (pi, (kvi, kxi)), where xik is the position of the particle pi at the kth iteration, within a cell so that the number of pairs per cell is dependent on the dynamic part determined for the cell.

### Claim 9 (depends on 8)

The device of claim 8, adapted to select said pair to be duplicated/deleted, as a function of the probability P(O=occupied, V=vik|ZC) is determined at the kth iteration, and where vik is a velocity component of the pair.

### Claim 10 (depends on 8)

The device of claim 8, wherein the total number of particles in the grid is constant during the iterations.

### Claim 11 (depends on 7)

The device of claim 7, adapted to determine P(O=empty, V=0|ZC) at the kth iteration, denoted Pk(O=empty, V=0|ZC), as a function of the product of a first term dependent on the first probability of occupancy and of a second term, said second term being dependent on:
Pk−1(O=occ,V=0|ZC).(1-ε)+Pk−1(O=empty, V=0|ZC).ε; and/or
coeff (kvi).Pk−1(O=occupied, V=vik−1,|ZA).(1-ε), where coeff (vik) is a decreasing function of ∥vik∥, and/or
a probability of appearance pa of a new object in the observed scene.

### Claim 12 (depends on 1)

A non-transitory computer accessible medium that includes computer-executable instructions stored thereon that are executable by a computing device to perform the method of claim 1.
