# US 12,759,582 B2 · Context-aware perception > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,759,582 B2 - Document ID: US12759582B2 - Status: Granted - Kind: B2 - Application number: 18/328,007 - Filed: 2023-06-02 - Published: 2024-12-05 - Granted: 2026-10-13 - Earliest claimed: 2023-06-02 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: Sever Ioan Topan; Yuxiao Chen; Edward Fu Schmerling; Karen Yan Ming Leung; Hans Jonas Nilsson; Michael Cox; Marco Pavone - Web page: https://patentgrep.com/patent/US12759582B2 ## CPC - B60W 60/0015: {specially adapted for safety} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2554/4049: Relationship among other objects, e.g. converging dynamic objects - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT ## Abstract In various examples, systems and methods are disclosed relating to refinement of safety zones and improving evaluation metrics for the perception modules of autonomous and semi-autonomous systems. Example implementations can exclude areas in the state space that are not safety critical, while retaining the areas that are safety-critical. This can be accomplished by leveraging ego maneuver information and conditioning safety zone computations on ego maneuvers. A maneuver-based decomposition of perception safety zones may leverage a temporal convolution operation with the capability to account for collision at any intermediate time along the way to maneuver completion. This provides a significant reduction in zone volume while maintaining completeness, thus optimizing or otherwise enhancing obstacle perception performance requirements by filtering out regions of state space not relevant to a system's route of travel. Computation of safety-zones conditioned on the ego maneuver greatly reduces excessive conservatism. ## Claims ### Claim 1 At least one processor comprising: one or more circuits to: obtain sensor data corresponding to one or more objects in a first volume of space perceivable by an autonomous system; determine one or more values of one or more parameters corresponding to one or more predicted trajectories associated with an intended maneuver of the autonomous system; generate, using the sensor data and based at least on conditioning the generation on the one or more values of the one or more parameters, an obstacle perception zone corresponding to a second volume of space, smaller than and within the first volume of space, containing potential threats to the autonomous system and excluding one or more regions of the first volume of space that are not relevant to the one or more predicted trajectories; and based at least on potential threats detected within the obstacle perception zone: performing one or more maneuvers by transmitting control signals to at least one of a propulsion system, a steering system, or a brake system of the autonomous system; and/or transmitting, to one or more control circuits, data corresponding to the one or more maneuvers for performance of the one or more maneuvers by the one or more control circuits via control signals to at least one of the propulsion system, the steering system, or the brake system. ### Claim 2 (depends on 1) The at least one processor of claim 1, wherein the obstacle perception zone is generated, at least in part, by performing temporal convolution that models temporal dependencies between state goals. ### Claim 3 (depends on 2) The at least one processor of claim 2, wherein the temporal dependencies correspond to potential collisions during the intended maneuver. ### Claim 4 (depends on 1) The at least one processor of claim 1, wherein the obstacle perception zone is generated so as to account for a collision at any intermediate time along the way to maneuver completion. ### Claim 5 (depends on 1) The at least one processor of claim 1, wherein the obstacle perception zone is generated, at least in part, using a Hamilton-Jacobi (HJ) reachability formulation constrained by the intended maneuver. ### Claim 6 (depends on 1) The at least one processor of claim 1, wherein the obstacle perception zone is generated, at least in part, by executing a Hamilton-Jacobi-Bellman (HJB) equation. ### Claim 7 (depends on 1) The at least one processor of claim 1, wherein the obstacle perception zone is generated, at least in part, by filtering out regions of space not relevant to a route of the autonomous system. ### Claim 8 (depends on 1) The at least one processor of claim 1, wherein the obstacle perception zone corresponds to regions of space in which it is possible for the autonomous system to collide with one or more other actors. ### Claim 9 (depends on 1) The at least one processor of claim 1, wherein the intended maneuver comprises a change in position of the autonomous system. ### Claim 10 (depends on 1) The at least one processor of claim 1, wherein the intended maneuver comprises a change in direction of the autonomous system. ### Claim 11 (depends on 1) The at least one processor of claim 1, wherein the at least one processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more large language models (LLMs); a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. ### Claim 12 A system comprising: one or more processing units to perform operations comprising: obtaining sensor data corresponding to a sensory perception zone of a machine; determining one or more values of one or more parameters corresponding to one or more predicted trajectories associated with an intended maneuver of the machine; generating, using the sensor data and based at least on conditioning the generation on the one or more values of the one or more parameters, an obstacle perception zone identified within the sensory perception zone based at least on potential threats to the machine, and excluding one or more regions of the sensory perception zone that are not relevant to the one or more predicted trajectories; performing one or more perception operations, using a subset of the sensor data corresponding to the obstacle perception zone, to generate one or more perception outputs; and based at least on the one or more perception outputs: performing, using the machine, one or more control operations by transmitting control signals to at least one of a propulsion system, a steering system, or a brake system of the machine; and/or transmitting, to one or more control circuits, data corresponding to the one or more operations for performance of the one or more control operations by the one or more control circuits via control signals to at least one of the propulsion system, the steering system, or the brake system of the machine. ### Claim 13 (depends on 12) The system of claim 12, wherein the generating the obstacle perception zone comprises performing temporal convolution that models temporal dependencies between state goals, wherein the temporal dependencies correspond to potential collisions during the intended maneuver. ### Claim 14 (depends on 12) The system of claim 12, wherein the obstacle perception zone is determined so as to account for a collision at any intermediate time along the way to maneuver completion. ### Claim 15 (depends on 12) The system of claim 12, wherein the generating the obstacle perception zone comprises a Hamilton-Jacobi (HJ) reachability formulation constrained by the intended maneuver. ### Claim 16 (depends on 12) The system of claim 12, wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for generating or presenting at least one of augmented reality content, virtual reality content, or mixed reality content; a system for hosting one or more real-time streaming applications; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more large language models (LLMs); a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. ### Claim 17 A method comprising: processing, using one or more processing units of a machine, sensor inputs corresponding to a sensory field of view captured by one or more sensors of the machine; identifying, using the one or more processing units and based at least on an input from a path planning module or a user, one or more intended maneuvers of the machine; determining one or more values of one or more parameters corresponding to one or more predicted trajectories associated with the one or more intended maneuvers of the machine; reducing, using the one or more processing units and the sensor inputs, based at least on conditioning the reduction on the one or more values of the one or more parameters, the sensory field of view of the machine to an obstacle perception zone containing potential threats to the machine, thereby filtering out regions of space that are not relevant to the one or more predicted trajectories to reduce requirements for computational resources; and based at least on potential threats detected in the obstacle perception zone using the one or more processing units, performing, by the machine, an intended maneuver of the one or more intended maneuvers by transmitting control signals to at least one of a propulsion system, a steering system, or a brake system of the machine, and/or transmitting, to one or more control circuits, data indicative of the intended maneuver to enable the one or more control circuits to perform the intended maneuver by sending control signals to at least one of the propulsion system, the steering system, or the brake system. ### Claim 18 (depends on 17) The method of claim 17, wherein the reducing the sensory field of the machine to the obstacle perception zone comprises performing temporal convolution that models temporal dependencies between state goals. ### Claim 19 (depends on 18) The method of claim 18, wherein the temporal dependencies correspond to potential collisions during the intended maneuver. ### Claim 20 (depends on 1) The at least one processor of claim 1, wherein the obstacle perception zone is generated based on an interaction dynamics model that accounts for a plurality of dynamically-feasible trajectories of the one or more objects relative to the one or more predicted trajectories of the intended maneuver.