Time-of-flight camera having improved dynamic range and method of generating a depth map

A computer-implemented method of generating a depth map using a time-of-flight image sensor having a plurality of pixels, the method comprising:

• (a) at each pixel of the plurality of pixels, determining a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) for said each pixel, determining by a computer processor an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generating the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels,
• wherein step (a) comprises weighting each phase-correlation value of the plurality of phase-correlation values according to a weighting function associated with said at least one exposure duration.

A computer-implemented method of generating a depth map using a time-of-flight image sensor having a plurality of pixels, the method comprising:

• (a) at each pixel of the plurality of pixels, determining a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset, and determining, in response to the plurality of phase-correlation values, an invalid-pixel mask associated with the plurality of pixels;
• (b) for said each pixel, determining by a computer processor an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generating the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels,
• wherein step (c) comprises generating the depth map in response to the plurality of accumulated correlations and the invalid-pixel mask.

A computer-implemented method of generating a depth map using a time-of-flight image sensor having a plurality of pixels, the method comprising:

• (a) at each pixel of the plurality of pixels, determining a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) for said each pixel, determining by a computer processor an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• (c) generating the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels;
• and
• (d) determining, in response to the plurality of phase-correlation values and said at least one exposure duration, a normalization mask associated with the plurality of pixels.

A computer-implemented method of generating a depth map using a time-of-flight image sensor having a plurality of pixels, the method comprising:

• (a) at each pixel of the plurality of pixels, determining a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) for said each pixel, determining by a computer processor an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generating the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels,
• wherein step (b) comprises determining, for said each pixel, a plural set of accumulated correlations of the plurality of accumulated correlations in response to a plurality of sets of the plurality of phase-correlation values such that each said accumulated correlation is associated with one unique phase offset of the at least one phase offset in response to each set of the plurality of sets being associated with one exposure duration of said at least one exposure duration, said plural set being of a plurality of plural sets associated with the plurality of pixels, wherein step (c) comprises generating the depth map in response to the plurality of plural sets, and wherein step (b) comprises incrementing said each accumulated correlation by a corresponding phase-correlation value of the plurality of phase-correlation values if said corresponding phase-correlation value is a member of a valid set of the plurality of sets.

A computer-implemented method of generating a depth map using a time-of-flight image sensor having a plurality of pixels, the method comprising:

• (a) at each pixel of the plurality of pixels, determining a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) for said each pixel, determining by a computer processor an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generating the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels,
• wherein step (b) comprises determining, for said each pixel, a plural set of accumulated correlations of the plurality of accumulated correlations in response to a plurality of sets of the plurality of phase-correlation values such that each said accumulated correlation is associated with one unique phase offset of the at least one phase offset in response to each set of the plurality of sets being associated with one exposure duration of said at least one exposure duration, said plural set being of a plurality of plural sets associated with the plurality of pixels, wherein step (c) comprises generating the depth map in response to the plurality of plural sets, and wherein step (a) comprises weighting each phase-correlation value of the plurality of phase-correlation values according to a weighting function associated with said at least one exposure duration;
• and wherein step (b) comprises determining said plural set such that said each set is associated with one unique exposure duration of said at least one exposure duration.

A computer-implemented method of generating a depth map using a time-of-flight image sensor having a plurality of pixels, the method comprising:

• (a) at each pixel of the plurality of pixels, determining a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) for said each pixel, determining by a computer processor an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generating the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels,
• wherein step (b) comprises determining, for said each pixel, a plural set of accumulated correlations of the plurality of accumulated correlations in response to a plurality of sets of the plurality of phase-correlation values such that each said accumulated correlation is associated with one unique phase offset of the at least one phase offset in response to each set of the plurality of sets being associated with one exposure duration of said at least one exposure duration, said plural set being of a plurality of plural sets associated with the plurality of pixels, wherein step (c) comprises generating the depth map in response to the plurality of plural sets, and wherein the method comprises determining, in response to the plurality of sets, an invalid-pixel mask indicating as invalid any pixel of the plurality of pixels for which each and every set of the plurality of sets associated with said any pixel contains one or more invalid phase-correlation values of the plurality of phase-correlation values.

A computer-implemented method of generating a depth map using a time-of-flight image sensor having a plurality of pixels, the method comprising:

• (a) at each pixel of the plurality of pixels, determining a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) for said each pixel, determining by a computer processor an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generating the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels,
• wherein step (b) comprises determining, for said each pixel, a plural set of accumulated correlations of the plurality of accumulated correlations in response to a plurality of sets of the plurality of phase-correlation values such that each said accumulated correlation is associated with one unique phase offset of the at least one phase offset in response to each set of the plurality of sets being associated with one exposure duration of said at least one exposure duration, said plural set being of a plurality of plural sets associated with the plurality of pixels, wherein step (c) comprises generating the depth map in response to the plurality of plural sets, and wherein the method comprises determining a normalization mask indicating for said each pixel a sum associated with all valid exposure durations of said at least one exposure duration such that said all valid exposure durations are associated with a valid set of the plurality of sets for which all phase-correlation values of said valid set are valid.

A computer-implemented method of generating a depth map using a time-of-flight image sensor having a plurality of pixels, the method comprising:

• (a) at each pixel of the plurality of pixels, determining a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) for said each pixel, determining by a computer processor an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generating the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels,
• wherein step (b) comprises determining, for said each pixel, a plural set of accumulated correlations of the plurality of accumulated correlations in response to a plurality of sets of the plurality of phase-correlation values such that each said accumulated correlation is associated with one unique phase offset of the at least one phase offset in response to each set of the plurality of sets being associated with one exposure duration of said at least one exposure duration, said plural set being of a plurality of plural sets associated with the plurality of pixels, wherein step (c) comprises generating the depth map in response to the plurality of plural sets, and wherein step (b) comprises, in response to an invalid set of the plurality of sets, linearly extrapolating said plural set on the basis of one or more valid sets of the plurality of sets.

A time-of-flight camera operable to generate a depth map, the camera comprising a time-of-flight image sensor having a plurality of pixels, the camera comprising a processor, the camera being configured to:

• (a) determine, at each pixel of the plurality of pixels, a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) determine by the processor, for said each pixel, an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generate the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels.

A time-of-flight camera operable to generate a depth map, the camera comprising a time-of-flight image sensor having a plurality of pixels, the camera comprising a processor, the camera being configured to:

• (a) determine, at each pixel of the plurality of pixels, a plurality of phase-correlation values associated with at least one exposure duration and at least one phase offset;
• (b) determine by the processor, for said each pixel, an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations associated with the plurality of pixels;
• and
• (c) generate the depth map in response to the plurality of accumulated correlations associated with the plurality of pixels,
• wherein the processor is configured to determine, for said each pixel, a plural set of accumulated correlations of the plurality of accumulated correlations in response to a plurality of sets of the plurality of phase-correlation values such that each said accumulated correlation is associated with one unique phase offset of said at least one phase offset in response to each set of the plurality of sets being associated with one exposure duration of said at least one exposure duration, said plural set being of a plurality of plural sets associated with the plurality of pixels, wherein the processor is configured to generate the depth map in response to the plurality of plural sets, and wherein the processor is configured to increment each said accumulated correlation by a corresponding phase-correlation value of the plurality of phase-correlation values if said corresponding phase-correlation value is a member of a valid set of the plurality of sets.

A time-of-flight camera comprising:

• (a) means for determining a plurality of phase-correlation values;
• (b) means for determining an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations;
• and
• (c) means for generating a depth map in response to the plurality of accumulated correlations.

A time-of-flight camera comprising:

• (a) means for determining a plurality of phase-correlation values;
• (b) means for determining an accumulated correlation in response to the plurality of phase-correlation values, said accumulated correlation being of a plurality of accumulated correlations;
• and
• (c) means for generating a depth map in response to the plurality of accumulated correlations,
• wherein the means for determining the accumulated correlation in response to the plurality of phase-correlation values comprises means for incrementing the accumulated correlation.