# US 2018/0322332 A1 · Method and apparatus for identifying pupil in image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322332 A1
- Document ID: US20180322332A1
- Status: Published application
- Kind: A1
- Application number: 15/773,319
- Filed: 2016-11-04
- Published: 2018-11-08
- Earliest claimed: 2015-11-11
- Assignees: China Unionpay Co., Ltd.
- Applicants: China Unionpay Co., Ltd.
- Inventors: Liang Feng; Zihao Cai; Yawei Yin
- Web page: https://patentgrep.com/patent/US20180322332A1

## CPC

- G06V 40/162: {using pixel segmentation or colour matching}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 5/70: Denoising; Smoothing
- G06T 7/13: Edge detection
- G06T 7/73: using feature-based methods
- G06T 11/26: Drawing of charts or graphs
- G06V 40/193: {Preprocessing; Feature extraction}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method and device for identifying a pupil in an image is provided. The method comprises four steps including: A. image preprocessing, B. edge detecting, C. building connected graphs, and D. screening connected graph pairs.

## Claims

### Claim 1

A method of identifying a pupil in an image, comprising:
A. image preprocessing;
B. edge detecting;
C. building connected graphs, comprising:
marking off connected regions based on edge lines of edge graphs,
dividing connected regions into two categories having a grey-scale value between 100 and 200 and a grey-scale value over 200, respectively,
for connected graphs of these two categories, selecting connected graph pairs, in which edge lines of a connected graph b having a grey-scale value over 200 are included in edge lines of a connected graph a having a grey-scale value between 100 and 200;
D. screening connected graph pairs, comprising:
for each of connected graph pairs, calculating a centroid pixel location b_Location of the connected graph b, a pixel length b_Major of a major axis and a pixel length b_Minor of a minor axis of an ellipse that has the same standardized second-order central moment as its connected region,
connecting the connected graphs a and b to obtain a new image c, calculating a centroid pixel location c_Location of the image c, a pixel length c_Major of a major axis and a pixel length c_Minor of a minor axis of an ellipse that has the same standardized second-order central moment as its connected region,
for each of connected graph pairs, calculating a pixel distance L between the centroid pixel location b_Location and c_Location, a ratio R of b_Major to c_Major, and a ratio P of b_Minor to c_Minor,
for each of connected graph pairs, calculating W=L−m*R−n*P, wherein m and n are weight factors,
selecting the connected graph b in the connected graph pair that has the maximum value of W as the identified pupil.

### Claim 2 (depends on 1)

The method of claim 1, wherein,
the image preprocessing comprises:
building a grey-scale image of an original image,
de-noising the grey-scale image.

### Claim 3 (depends on 2)

The method of claim 2, wherein,
the original image is obtained by:
acquiring an input video, segmenting the video into a plurality of pictures according to time intervals, and taking the first picture as the original image for identifying the pupil.

### Claim 4 (depends on 3)

The method of claim 3, wherein,
the grey-scale image of the original image is built by:
acquiring R, G and B values of each of pixels of the original image, and for each of pixels, selecting the maximum value among its R, G and B values as the grey-scale value of the pixel.

### Claim 5 (depends on 1)

The method of claim 1, wherein,
the edge detecting comprises:
calculating a gradient image of the grey-scale image,
for the gradient image, taking pixels that have a value larger than a specified threshold value as edge pixels to build an edge graph.

### Claim 6

A device for identifying a pupil in an image, comprising:
A. an image preprocessing unit;
B. an edge detecting unit;
C. a connected graph building unit configured to perform steps of:
marking off connected regions based on edge lines of edge graphs,
dividing connected regions into two categories having a grey-scale value between 100 and 200 and a grey-scale value over 200, respectively,
for connected graphs of these two categories, selecting connected graph pairs, in which edge lines of a connected graph b having a grey-scale value over 200 are included in edge lines of a connected graph a having a grey-scale value between 100 and 200;
D. a connected graph pair screening unit configured to perform steps of:
for each of connected graph pairs, calculating a centroid pixel location b_Location of the connected graph b, a pixel length b_Major of a major axis and a pixel length b_Minor of a minor axis of an ellipse that has the same standardized second-order central moment as its connected region,
connecting the connected graphs a and b to obtain a new image c, calculating a centroid pixel location c_Location of the image c, a pixel length c_Major of a major axis and a pixel length c_Minor of a minor axis of an ellipse that has the same standardized second-order central moment as its connected region,
for each of connected graph pairs, calculating a pixel distance L between the centroid pixel location b_Location and c_Location, the ratio R of b_Major to c_Major, and the ratio P of b_Minor to c_Minor,
for each of connected graph pairs, calculating W=L-m*R-n*P, wherein m and n are weight factors,
selecting the connected graph b in the connected graph pair that has the maximum value of W as the identified pupil.

### Claim 7 (depends on 6)

The device of claim 6, wherein,
the image preprocessing unit is configured to perform steps of:
building a grey-scale image of an original image,
de-noising the grey-scale image.

### Claim 8 (depends on 7)

The device of claim 7, wherein,
the original image is obtained by:
acquiring an input video, segmenting the video into a plurality of pictures according to time intervals, and taking the first picture as the original image for identifying the pupil.

### Claim 9 (depends on 8)

The device of claim 8, wherein,
the grey-scale image of the original image is built by:
acquiring R, G and B values of each of pixels of the original image, and for each of pixels, selecting the maximum value among its R, G and B values as the grey-scale value of the pixel.

### Claim 10 (depends on 6)

The device of claim 6, wherein,
the edge detecting unit is configured to perform steps of:
calculating a gradient image of the grey-scale image,
for the gradient image, taking pixels that have a value larger than a specified threshold value as the edge pixels to build an edge graph.
