# US 2009/0254236 A1 · System and method for image mapping and visual attention

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2009/0254236 A1
- Document ID: US20090254236A1
- Status: Published application
- Kind: A1
- Application number: 11/546,867
- Filed: 2006-10-11
- Published: 2009-10-08
- Earliest claimed: 2006-10-11
- Assignees: Vanderbilt University
- Inventors: Richard A. Peters, II
- Web page: https://patentgrep.com/patent/US20090254236A1

## CPC

- G05D 1/2465: {using a 3D model of the environment}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G06T 1/0014: {Image feed-back for automatic industrial control, e.g. robot with camera (robots B25J19/023)}
- G05D 1/0227: {using mechanical sensing means, e.g. for sensing treated area}
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0255: {using acoustic signals, e.g. ultra-sonic singals (sonar systems designed for anti-collision purposes G01S15/93)}
- G05D 1/0257: {using a radar (radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931)}
- G05D 1/0259: {using magnetic or electromagnetic means}
- G05D 1/027: {comprising intertial navigation means, e.g. azimuth detector (inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165)}
- G05D 1/0278: {using satellite positioning signals, e.g. GPS}
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 20/35: {Categorising the entire scene, e.g. birthday party or wedding scene}
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method is described for mapping dense sensory data to a Sensory Ego Sphere (SES). Methods are also described for finding and ranking areas of interest in the images that form a complete visual scene on an SES. Further, attentional processing of image data is best done by performing attentional processing on individual full-size images from the image sequence, mapping each attentional location to the nearest node, and then summing all attentional locations at each node. More information is available through this method since attentional processing is repeatedly done on each image in the sequence. An attentional point that has persisted in several adjacent images will have a higher activation value and, therefore, will be deemed more salient than an attentional point found in only one image. Therefore, the confidence that a location deemed salient by this method is an actual salient feature is greater than with the alternative processing methods in which attentional processing is performed only once on the image reconstructed from the foveal windows posted on the SES.

## Claims

### Claim 1

An adaptive autonomous robot comprising:
sensors;
actuators;
a camera for receiving image data from an external source;
a database for associating received image data with points on a portion of a spherical region centered on the robot;
means for performing attentional processing on individual images to identify attentional locations;
means for mapping each attentional location to a nearest point on the portion of the spherical region; and
a summer for summing all the attentional locations mapped to a point.

### Claim 2 (depends on 1)

The robot of claim 1 wherein the sensors comprise:
means for sensing an internal state of the robot; and
means for sensing one or more characteristics of the robot's environment.

### Claim 3 (depends on 2)

The robot of claim 2 wherein the database comprises a sensory ego sphere for representing an object based on the sensed internal state and the sensed environment.

### Claim 4

In an adaptive autonomous robot comprising:
sensors;
actuators;
a camera for receiving image data from an external source;
a database for associating received image data with points on a portion of a spherical region centered on the robot;
a method of finding and ranking areas of interest in images from the external source comprising:
performing attentional processing on individual images to identify attentional locations;
mapping each attentional location to a nearest point on the portion of the spherical region; and
summing all the attentional locations mapped to a point.

### Claim 5

A method of mapping dense sensory data to a database of a robot comprising:
compiling a list of nodes in a portion of a spherical region centered on the robot;
generating an image of a region external to the robot at each node;
extracting a foveal image from a center of each image; and
associating each foveal image with said node in the database.

### Claim 6 (depends on 5)

The method of claim 5 wherein the portion of the spherical region is a sensory ego sphere.
