# US 2010/0142778 A1 · Motion compensated image averaging

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0142778 A1
- Document ID: US20100142778A1
- Status: Published application
- Kind: A1
- Application number: 12/598,592
- Filed: 2008-05-02
- Published: 2010-06-10
- Earliest claimed: 2007-05-02
- Assignees: Agency for Science, Technology and Research
- Inventors: Lang Zhuo; Saravana Kumar; Gideon Ho
- Web page: https://patentgrep.com/patent/US20100142778A1

## CPC

- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/20: using local operators
- G06T 5/70: Denoising; Smoothing
- G06T 5/73: Deblurring; Sharpening
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10064: Fluorescence image
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20036: Morphological image processing
- G06T 2207/30004: Biomedical image processing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method is provided for averaging a sequence of image frames. A noise-reducing filter is applied to the image frames to generate filtered frames. A deconvolution filter is applied to the filtered frames to generate corresponding deconvolved frames. The filtered frames are transformed by an affine transformation to align them, generating aligned frames. The aligned frames are motion corrected by non-linear transformation based on intensity rank matching, generating a sequence of motion-corrected frames. The motion-corrected frames are averaged to generate a resultant frame.

## Claims

### Claim 1

A method of averaging a sequence of image frames taken from a living object with a fluorescent imaging technique, said method comprising:
applying a noise-reducing filter to said image frames to generate a sequence of filtered frames;
applying a deconvolution filter to said sequence of filtered frames to generate a corresponding sequence of deconvolved frames;
subjecting said deconvolved frames to a motion-correction transformation to generate a sequence of corrected deconvolved frames, wherein the ith corrected deconvolved frame is represented by Jiref(x,y), Jiref being the intensity at image point (x,y) in the ith corrected deconvolved frame;
transforming said filtered frames by an affine transformation to align said filtered frames, thus generating a sequence of aligned frames, wherein an ith aligned frame is represented by Iia(x,y), Iia being the intensity at image point (x,y) in the ith aligned frame;
relocating image points in each aligned frame to generate a motion-corrected frame, wherein an ith motion-corrected frame is represented by Iin(x,y), being the intensity at image point (x,y) in the ith motion-corrected frame, Iin(x,y)=Iia(xp,yp), and wherein the location (xp,yp) is selected such that
(i) said location is within a neighborhood centered around (x,y), and
(ii) an intensity rank of Iia(xp,yp) in said neighborhood and an intensity rank of Ji-1ref(x,y) in said neighborhood are the same,
thus generating a sequence of motion-corrected frames; and
averaging said motion-corrected frames to generate a resultant frame.

### Claim 2 (depends on 1)

The method of claim 1, wherein the total number of motion corrected frames is N, and wherein said averaging comprises:
generating a sequence of reference frames, wherein an ith one of said reference frames is represented by Iiref and is calculated as
I
i
ref
=
(
i
-
1
)
×
I
i
-
1
ref
+
I
i
n
i
;
and
generating said resultant frame according to IA=INref, IA representing said resultant frame.

### Claim 3 (depends on 1)

The method of claim 1, wherein said motion-correction transformation of said deconvolved frames comprises
transforming said deconvolved frames by an affine transformation to align said deconvolved frames, thus generating a sequence of aligned deconvolved frames, wherein an ith aligned deconvolved frame is represented by Jia(x,y), Jia being the intensity at image point (x,y) in the ith aligned deconvolved frame;
relocating image points in each aligned deconvolved frame to generate a motion-corrected deconvolved frame, wherein an ith motion-corrected deconvolved frame is represented by Jin(x,y), Jin being the intensity at image point (x,y) in the ith motion-corrected deconvolved frame, Jin(x,y)=Jia(x0,y0), and wherein the location (x0,y0) is selected such that
(i) said location (x0,y0) is within a neighborhood centered around (x,y), and
(ii) an intensity rank of Jia(x0,y0) in said neighborhood and an intensity rank of Ji-1ref(x,y) in said neighborhood are the same.

### Claim 4 (depends on 3)

The method of claim 3, wherein said ith corrected deconvolved frame is generated according to
J
i
ref
=
(
i
-
1
)
×
J
i
-
1
ref
+
J
i
n
i
.

### Claim 5 (depends on 1)

The method of claim 1, comprising:
establishing point-to-point correlation between corresponding landmark points in each adjacent pair of said deconvolved frames;
establishing point-to-point correlation between corresponding landmark points in each adjacent pair of said filtered frames, guided by said point-to-point correlation between said corresponding landmark points in said deconvolved frames; and
wherein said transforming said filtered frames by said affine transformation comprises aligning corresponding landmark points in each adjacent pair of said aligned frames.

### Claim 6 (depends on 5)

The method of claim 5, wherein said establishing said point-to-point correlation between said corresponding landmark points in said each adjacent pair of said deconvolved frames comprises:
calculating a normalized cross-correlation (NCC) measure for each pair of said corresponding landmark points;
determining that said pair of landmark points are correlated when said NCC measure satisfies a pre-selected condition.

### Claim 7

A method of averaging a sequence of image frames taken from a living object with a fluorescent imaging technique, said method comprising:
applying a noise-reducing filter to said image frames to generate a sequence of filtered frames;
applying a deconvolution filter to said sequence of filtered frames to generate a corresponding sequence of deconvolved frames;
determining landmark points in said deconvolved frames, by identifying, in each deconvolved frame, image points that have a peak intensity;
establishing point-to-point correlation between corresponding landmark points in each adjacent pair of said deconvolved frames;
establishing point-to-point correlation between corresponding landmark points in each adjacent pair of said filtered frames, guided by said point-to-point correlation between said corresponding landmark points in said deconvolved frames;
transforming said filtered frames by an affine transformation to sequentially align said filtered frames, thus generating a sequence of aligned frames;
transforming said aligned frames by a non-linear transformation for motion correction, thus generating a sequence of motion-corrected frames; and
averaging said motion-corrected frames to generate a resultant frame.

### Claim 8 (depends on 7)

The method of claim 7, wherein said establishing said point-to-point correlation between said corresponding landmark points in said each adjacent pair of said deconvolved frames comprises:
calculating a normalized cross-correlation (NCC) measure for each pair of said corresponding landmark points;
determining that said pair of landmark points are correlated when said NCC measure satisfies a pre-selected condition.

### Claim 9

A method of averaging a sequence of image frames taken from a living object with a fluorescent imaging technique, said method comprising:
applying a multi-scale Gaussian filter to said image frames for noise reduction, thus generating a sequence of filtered frames;
applying a deconvolution filter to said filtered frames, thus generating a sequence of deconvolved frames;
extracting intensity maxima as landmark points from said deconvolved frames;
tracking landmark points in said deconvolved frames using normalized cross-correlation;
aligning said filtered frames by an affine transformation with reference to said tracked landmark points, thus generating aligned frames;
transforming said aligned frames by a non-linear transformation for motion correction, thus generating motion corrected frames; and
averaging said motion corrected frames to generate a resultant frame.

### Claim 10 (depends on 9)

The method of claim 9, wherein said tracking said landmarks in said deconvolved frames comprises establishing point-to-point correlation between corresponding landmark points in each adjacent pair of said deconvolved frames; said method further comprising
establishing point-to-point correlation between corresponding landmark points in each adjacent pair of said filtered frames, guided by said point-to-point correlation between said corresponding landmark points in said deconvolved frames;
wherein said aligning said filtered frames comprises aligning corresponding landmark points in each adjacent pair of said aligned frames.

### Claim 11 (depends on 7)

The method of claim 7, wherein an ith one of said motion corrected frames is represented by Ini, where i=1 to N, N being the total number of motion corrected frames, and wherein said averaging comprises:
generating a sequence of reference frames, an ith one of said reference frames is represented by Iiref and is calculated as
I
i
ref
=
(
i
-
1
)
×
I
i
-
1
ref
+
I
i
n
i
;
and
generating said resultant frame according to IA=INref, IA representing said resultant frame.

### Claim 12 (depends on 1)

The method of claim 1, wherein said sequence of filtered frames are generated by:
applying a plurality of Gaussian filters to each one of said image frames to generate a plurality of Gaussian maps associated with said each image frame, each one of said Gaussian filters is characterized by a distinct filter width and a standard deviation dependent on said filter width;
generating a filtered frame corresponding to said each image frame, by assigning each image point in said filtered frame an intensity equaling a maximum intensity at said each image point in said plurality of Gaussian maps associated with said each image frame, thus generating said first sequence of said filtered frames.

### Claim 13 (depends on 12)

The method of claim 12, wherein the intensity, Mi,j(x,y), at an image point (x,y) in the jth Gaussian map associated with the ith image frame in said sequence of image frames is calculated by
M
i
,
j

(
x
,
y
)
=
∑
x
j


∑
y
j


h
j

(
x
-
x
j
)

h
j
′

(
y
-
y
j
)

I
i

(
x
j
,
y
j
)
,
where the summation is over possible xj and yj that satisfy (x−wj)≦xj≦(x+wj) and (y−wj)≦yj≦(y−wj), and
wherein Ii(x,y) is the intensity at said image point (x,y) in said ith image frame,
hj(u)=exp(−u2/2σj2) is a Gaussian transfer function for the jth Gaussian filter, u representing a one dimensional variable,
h′j(u) is the transpose of hj(u),
σj is a standard deviation, and
wj is a filter width for said jth Gaussian filter.

### Claim 14 (depends on 13)

The method of claim 13, wherein wj=j.

### Claim 15 (depends on 13)

The method of claim 13, wherein σj is selected so that it has the largest value that satisfies
exp(−wj2/2σj2)≦10−3.

### Claim 16 (depends on 13)

The method of claim 13, wherein said plurality of Gaussian filters consists of 3 to 10 Gaussian filters.

### Claim 17 (depends on 1)

The method of claim 1, wherein said living object is a mouse retina.

### Claim 18 (depends on 1)

The method of claim 1, wherein said fluorescent imaging technique is a glial fibrillary acidic protein-green fluorescence protein (GFAP-GFP) imaging technique.

### Claim 19 (depends on 1)

A computer comprising a processor and a computer readable medium storing thereon computer-executable instructions which, when executed by said processor, adapts said computer to perform the method of claim 1.

### Claim 20 (depends on 1)

A computer readable medium storing thereon computer-executable instructions which, when executed by a processor, adapts said processor to perform the method of claim 1.
