US 5,067,164 AGranted

Hierarchical constrained automatic learning neural network for character recognition

AssigneeAT&T Bell Laboratories

InventorsJohn S. Denker, Richard E. Howard, Lawrence D. Jackel, Yann LeCun

Life of this patent

Estimating the term…

Loading patent…

Abstract

Highly accurate, reliable optical character recognition is afforded by a layered network having several layers of constrained feature detection wherein each layer of constrained feature detection includes a plurality of constrained feature maps and a corresponding plurality of feature reduction maps. Each feature reduction map is connected to only one constrained feature map in the same layer for undersampling that constrained feature map. Units in each constrained feature map of the first constrained feature detection layer respond as a function of a corresponding kernel and of different portions of the pixel image of the character captured in a receptive field associated with the unit. Units in each feature map of the second constrained feature detection layer respond as a function of a corresponding kernel and of different portions of an individual feature reduction map or a combination of several feature reduction maps in the first constrained feature detection layer as captured in a receptive field of the unit. The feature reduction maps of the second constrained feature detection layer are fully connected to each unit in the final character classification layer. Kernels are automatically learned by constrained back propagation during network initialization or training.

A massively parallel computation network for character recognition of an image map having a plurality of image units, said network comprising

first and second feature detection layer means wherein each of said feature detection layer means includes a plurality of constrained feature maps and a corresponding plurality of feature reduction maps, each constrained feature map and each feature reduction map comprising a plurality of units and a corresponding plurality of computation elements for generating values for said units in said map, each of said feature reduction maps having fewer units than each of said constrained feature maps, said computation element having a weighting kernel associated therewith and being responsive to a plurality of substantially neighboring units from at least a predetermined other map for mapping a dot product of said associated weighting kernel with said predetermined plurality of substantially neighboring units into an output value in accordance with a selected nonlinear criterion, said computation element responsive to a different plurality of substantially neighboring units than each other computation element associated with the same map, said second feature detection layer means having fewer units than said first feature detection layer means,
said constrained feature maps of said first feature detection layer means responsive to image units, each feature reduction map of said first feature detection layer means responsive to units from its corresponding constrained feature map, said constrained feature maps of said second feature detection layer means responsive to units from at least one feature reduction map in said first feature detection layer means, each feature reduction map of said second feature detection layer means responsive to units from its corresponding constrained feature map,
said network further including a character classification layer fully connected to all feature reduction maps of said second feature detection layer means for generating an indication representative of the character recognized by the network.

Abstract

Claims5