# US 2014/0192042 A1 · System and method for optical three-dimensional particle localization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0192042 A1
- Document ID: US20140192042A1
- Status: Published application
- Kind: A1
- Application number: 13/734,438
- Filed: 2013-01-04
- Published: 2014-07-10
- Earliest claimed: 2013-01-04
- Assignees: Roper Scientific, Inc.
- Inventors: Jason McClure
- Web page: https://patentgrep.com/patent/US20140192042A1

## CPC

- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G02B 21/16: adapted for ultraviolet illumination {; Fluorescence microscopes (G02B21/0076 takes precedence)}
- G02B 21/367: {providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison}
- G06F 18/2136: based on sparsity criteria, e.g. with an overcomplete basis
- G06V 10/143: Sensing or illuminating at different wavelengths
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06V 20/693: {Acquisition}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments include methods that may be used to optically obtain the precise three-dimensional location of multiple objects from one or more two dimensional images. An optical point spread function having a transverse shape which varies with axial distance may be implemented to obtain depth information. The transverse variation in the PSF with depth may be produced using a cylindrical lens. The objects may be imaged by a focal plane array detector. One or more 2D images may be used to find the 3D location of the objects using sparse signal reconstruction methods.

## Claims

### Claim 1

A method for localizing an object containing an ensemble of light-switchable elements in three-dimensional space comprising the steps of:
causing a first sparse subset of the elements to illuminate;
acquiring a first image b having dimensions m(1/2)×m(1/2) of said activated first sparse subset through optics having a point spread function (PSF) that varies predictably with axial object distance;
causing said first sparse subset of the elements to extinguish;
repeating the previous three steps for different sparse subsets of elements;
defining a sensing array Φ=m×n×l which represents a discrete set of possible PSF's that any of said elements can generate within a given volume having dimensions n(1/2)×n(1/2)×l; and
creating a three dimensional image of said elements by solving the following linear programming problem:
min Σi∥xln∥1 subject to ∥Σl(Φ1mnlx1n)−bm∥2≦ε(Σmbmm)1/2 and xin≧0,
where: x=l-many n×1 image vectors comprising said n(1/2)×n(1/2)×l three dimensional volume and c=user set tolerance on convergence.

### Claim 2 (depends on 1)

The method of claim 1, further comprising processing simultaneously a plurality of said sensing arrays.

### Claim 3 (depends on 1)

The method of claim 1, wherein each of said steps is repeated to create a plurality of results, and further comprising summing said results.

### Claim 4

A method for localizing an object containing an ensemble of light-switchable elements in three-dimensional space comprising the steps of:
causing a first sparse subset of the elements to illuminate;
acquiring a first image b having dimensions m(1/2)×m(1/2) of said activated first sparse subset through optics having a point spread function (PSF) that varies predictably with axial object distance;
causing said first sparse subset of the elements to extinguish;
repeating the previous three steps for different sparse subsets of elements;
defining a sensing array Φ=m×n×l which represents a discrete set of possible PSF's that any of said elements can generate within a given volume having dimensions n(1/2)×n(1/2)×l; and
creating a three dimensional image of said elements by solving the following linear programming problem:

min

∑
i

c
i
nT

x
i
n

subject


to


∑
i

(
Φ
1
mnl

x
1
n
)
-
b
m

?
≤
ɛ
(
∑
m

b
m
m
)
1
/
2



and



x
l
n
≥
0
,

?

indicates text missing or illegible when filed
where: x=l-many n×1 image vectors comprising said n(1/2)×n(1/2)×l three dimensional volume,
ε=user set tolerance on convergence and
weighting factors are defined for n-many transverse locations each of said elements can occupy for each axial plane l, and wherein said weighing factors are represented by the equation
c
l
n
=
1
max

(
[
φ
11

l
+
φ
21

l
+
…


φ
m


1

l
⋮
φ
1

n


l
+
φ
2

nl
+
…


φ
mnl
]
)

[
φ
11

l
+
φ
21

l
+
…


φ
m


1
!
⋮
φ
1

nl
+
φ
2

nl
+
…


φ
mn
!
]

### Claim 5 (depends on 4)

The method of claim 4, wherein each of said steps is repeated to create a plurality of results, and further comprising summing said results.

### Claim 6

A system for creating an image of an object comprising an ensemble of light-switchable elements, the system comprising:
an image sensor;
optics having a point spread function (PSF) that varies predictably with axial object distance;
an energy source; and
a computer; wherein
said computer: controls said energy source to illuminate a first portion of said light-switchable elements; controls said image sensor to acquire a first image of said light-switchable elements; controls said energy source to illuminate a second portion of said light-switchable elements; controls said image sensor to acquire a second image of said light-switchable elements and
defines a sensing array Φ=m×n×l which represents a discrete set of possible PSF's that any of said elements can generate within a given volume having dimensions n(1/2)×n(1/2)×l; and
creates a three dimensional image of said elements by solving the following linear programming problem:
min Σl∥xin∥1 subject to ∥Σl(Φ1mnlx1n)−bm∥2≦ε(Σmbmm)1/2 and xin≧0,
where: x=l-many n×1 image vectors comprising said n(1/2)×n(1/2)×l three dimensional volume and ε=user set tolerance on convergence.

### Claim 6 (depends on 5)

The system of claim 5, wherein said defining of said sensing array, said creating of said three dimensional image and said solving are performed on a field programmable gate array.

### Claim 7 (depends on 5)

The system of claim 5, wherein said optics include a cylindrical lens.