# US 12,602,916 B2 · Object modeling with adversarial learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,916 B2
- Document ID: US12602916B2
- Status: Granted
- Kind: B2
- Application number: 17/493,770
- Filed: 2021-10-04
- Published: 2022-01-27
- Granted: 2026-04-14
- Earliest claimed: 2017-11-20
- Assignees: AT&T Intellectual Property I, L.P.
- Applicants: AT&T Intellectual Property I, L.P.
- Inventors: Eric Zavesky; Raghuraman Gopalan; Behzad Shahraray; David Crawford Gibbon; Bernard S. Renger; Paul Triantafyllou
- Web page: https://patentgrep.com/patent/US12602916B2

## CPC

- G06V 10/82: using neural networks
- B60W 30/0953: {the prediction being responsive to vehicle dynamic parameters}
- G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target}
- G06F 18/2413: based on distances to training or reference patterns
- G06N 3/045: Combinations of networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/091: Active learning
- G06N 3/094: Adversarial learning
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/64: Three-dimensional [3D] objects
- B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering
- B64U 2101/30: for imaging, photography or videography
- B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- G05D 1/0246: {using a video camera in combination with image processing means}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06V 20/13: Satellite images
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

Methods, computer-readable media, and devices are disclosed for improving an object model based upon measurements of physical properties of an object via an unmanned vehicle using adversarial examples. For example, a method may include a processing system capturing measurements of physical properties of an object via at least one unmanned vehicle, updating an object model for the object to include the measurements of the physical properties of the object, where the object model is associated with a feature space, and generating an example from the feature space, where the example comprises an adversarial example. The processing system may further apply the object model to the example to generate a prediction, capture additional measurements of the physical properties of the object via the at least one unmanned vehicle when the prediction fails to identify that the example is an adversarial example, and update the object model to include the additional measurements.

## Claims

### Claim 1

A device comprising:
a processing system including at least one processor; and
a computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:
obtaining measurements of a plurality of physical properties of an object in accordance with one or more capture conditions via at least one sensor;
training an object model for the object with the measurements of the plurality of physical properties of the object as training data to distinguish between the object and generated examples of the object, wherein the object model comprises a discriminator subroutine of a generative adversarial network implemented by the processing system, and wherein the object model is associated with a feature space;
generating an example from the feature space, wherein the example is representative of the object, wherein the generating of the example from the feature space uses a generator subroutine of the generative adversarial network implemented by the processing system;
applying the object model comprising the discriminator subroutine to the example to output a prediction of whether the example is a generated example;
obtaining, via the at least one sensor, additional measurements of the plurality of physical properties of the object in response to the prediction failing to identify that the example is the generated example, wherein the obtaining of the additional measurements includes changing at least one of the one or more capture conditions via the at least one sensor;
retraining the object model comprising the discriminator subroutine with the additional measurements of the plurality of physical properties of the object;
comparing the object model to a previous object model for the object to determine at least one difference between the object model and the previous object model; and
generating a report of differences, the report comprising the at least one difference.

### Claim 2 (depends on 1)

The device of claim 1, wherein the object model comprises a three-dimensional model for the object.

### Claim 3 (depends on 1)

The device of claim 1, wherein the additional measures are obtained via the at least one sensor deployed on at least one unmanned vehicle comprising at least one of:
an unmanned aerial vehicle;
an unmanned underwater vehicle;
an unmanned ground vehicle; or
an unmanned surface vehicle.

### Claim 4 (depends on 3)

The device of claim 3, wherein the at least one sensor comprises a camera.

### Claim 5 (depends on 4)

The device of claim 4, wherein the at least one sensor further comprises at least one of:
a microphone;
a light source; or
a control arm.

### Claim 6 (depends on 1)

The device of claim 1, wherein the obtaining the additional measurements of the plurality of physical properties of the object comprises at least one of:
repositioning at least one unmanned vehicle comprising the at least one sensor to capture the additional measurements;
reorienting a camera of at least one unmanned vehicle comprising the at least one sensor to capture the additional measurements; or
changing a lighting condition of the object via a light of at least one unmanned vehicle comprising the at least one sensor to capture the additional measurements.

### Claim 7 (depends on 1)

The device of claim 1, wherein the obtaining the measurements of the plurality of physical properties of the object comprises obtaining the measurements for a first plurality of voxels in a three-dimensional physical space, and wherein the obtaining the additional measurements of the plurality of physical properties of the object comprises obtaining the additional measurements for a second plurality of voxels in the three-dimensional physical space.

### Claim 8 (depends on 1)

The device of claim 1, wherein the generating the example from the feature space is constrained to a predetermined deviation from a measured value of a physical property of the plurality of physical properties of the object.

### Claim 9 (depends on 1)

The device of claim 1, wherein the generating the example from the feature space is constrained to a predetermined deviation from a previous object model for the object.

### Claim 10 (depends on 9)

The device of claim 9, wherein the generating the example from the feature space comprises:
including a known deviation pattern to the previous object model for the object.

### Claim 11 (depends on 1)

The device of claim 1, wherein the additional measurements are obtained over a plurality of cycles of examples generated from the feature space and the object model is retrained until a number or a percentage of the examples are identified as adversarial examples.

### Claim 12 (depends on 1)

The device of claim 1, wherein the operations further comprise:
determining a quality metric for the object model.

### Claim 13 (depends on 12)

The device of claim 12, wherein the quality metric comprises:
a guarantee that a feature of the object model is accurate within a defined range.

### Claim 14 (depends on 12)

The device of claim 12, wherein the operations further comprise:
providing the object model and the quality metric to a subscribing system.

### Claim 15 (depends on 1)

The device of claim 1, wherein the operations further comprise:
providing initial instructions to at least one unmanned vehicle comprising the at least one sensor to obtain the measurements of the plurality of physical properties of the object, wherein the initial instructions are based upon information of the object.

### Claim 16 (depends on 1)

The device of claim 1, wherein the object comprises at least a portion of:
a wireless communications tower;
a power line tower,
a building;
a bridge;
a ship;
a vehicle;
a roadway;
a pipeline; or
a rail line.

### Claim 17

A method comprising:
obtaining, by a processing system including at least one processor, measurements of a plurality of physical properties of an object in accordance with one or more capture conditions via at least one sensor;
training, by the processing system, an object model for the object with the measurements of the plurality of physical properties of the object as training data to distinguish between the object and generated examples of the object, wherein the object model comprises a discriminator subroutine of a generative adversarial network implemented by the processing system, and wherein the object model is associated with a feature space;
generating, by the processing system, an example from the feature space, wherein the example is representative of the object, wherein the generating of the example from the feature space uses a generator subroutine of the generative adversarial network implemented by the processing system;
applying, by the processing system, the object model comprising the discriminator subroutine to the example to output a prediction of whether the example is a generated example;
obtaining, by the processing system via the at least one sensor, additional measurements of the plurality of physical properties of the object in response to the prediction failing to identify that the example is the generated example, wherein the obtaining of the additional measurements includes changing at least one of the one or more capture conditions via the at least one sensor;
retraining, by the processing system, the object model comprising the discriminator subroutine with the additional measurements of the plurality of physical properties of the object;
comparing, by the processing system, the object model to a previous object model for the object to determine at least one difference between the object model and the previous object model; and
generating, by the processing system, a report of differences, the report comprising the at least one difference.

### Claim 18

A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:
obtaining measurements of a plurality of physical properties of an object in accordance with one or more capture conditions via at least one sensor;
training an object model for the object the measurements of the plurality of physical properties of the object as training data to distinguish between the object and generated examples of the object, wherein the object model comprises a discriminator subroutine of a generative adversarial network implemented by the processing system, and wherein the object model is associated with a feature space;
generating an example from the feature space, wherein the example is representative of the object, wherein the generating of the example from the feature space uses a generator subroutine of the generative adversarial network implemented by the processing system;
applying the object model comprising the discriminator subroutine to the example to output a prediction of whether the example is a generated example;
obtaining, via the at least one sensor, additional measurements of the plurality of physical properties of the object in response to the prediction failing to identify that the example is the generated example, wherein the obtaining of the additional measurements includes changing at least one of the one or more capture conditions via the at least one sensor;
retraining the object model comprising the discriminator subroutine with the additional measurements of the plurality of physical properties of the object;
comparing the object model to a previous object model for the object to determine at least one difference between the object model and the previous object model; and
generating a report of differences, the report comprising the at least one difference.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the object model comprises a three-dimensional model for the object.

### Claim 20 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the additional measures are obtained via the at least one sensor deployed on at least one unmanned vehicle comprising at least one of:
an unmanned aerial vehicle;
an unmanned underwater vehicle;
an unmanned ground vehicle; or
an unmanned surface vehicle.