US 11,700,365 B2Granted

Position detection method, position detection device, and display device

AssigneeSeiko Epson Corporation

InventorsJun Yang, Guoyi Fu

Life of this patent

Estimating the term…

Loading patent…

Abstract

Position detection methods and systems are disclosed herein. The position detection method of detecting a position in an operation surface pointed by a pointing element includes obtaining a first taken image with the first infrared camera, obtaining a second taken image with the second infrared camera, removing a noise component from the first and second images converting the first and second taken into converted images without the noise component, forming a difference image between the first converted taken image and the second converted taken image, extracting a candidate area in which a disparity amount between the first converted taken image and the second converted taken image is within a predetermined range, detecting a tip position of the pointing element from the candidate area, and determining a pointing position of the pointing element and whether or not the pointing element had contact with the operation surface based on the detecting.

Claims3

Dependent claims

A position detection method of detecting a position in an operation surface pointed by a pointing element, the method comprising:

irradiating with infrared light toward the operation surface;
obtaining a first taken image by imaging the operation surface with a first camera configured to take an image with the infrared light;
obtaining a second taken image by imaging the operation surface with a second camera different in imaging viewpoint from the first camera and configured to take an image with the infrared light;
removing a noise component from the first taken image and the second taken image based on a degree of coincidence in luminance gradient between a first background image and the first taken image, and a degree of coincidence in luminance gradient between a second background image and the second taken image, the first background image being obtained by imaging the operation surface with the first camera when the pointing element does not point the operation surface, and the second background image being obtained by imaging the operation surface with the second camera when the pointing element does not point the operation surface;
converting the first taken image from which the noise component removed into a first converted taken image calibrated with respect to the operation surface;
converting the second taken image from which the noise component removed into a second converted taken image calibrated with respect to the operation surface;
forming a difference image between the first converted taken image and the second converted taken image;
extracting an area in which a disparity amount between the first converted taken image and the second converted taken image is within a predetermined range out of the difference image as a candidate area in which an image of the pointing element is included, wherein the extracting includes
calculating a variant value of pixel values of pixels constituting the difference image,
separating the difference image into a background area and a foreground area which is an area other than the background area based on the variant value, and
extracting the candidate area based on a boundary between the background area and the foreground area and the area in which the disparity amount is within the predetermined range;
detecting a tip position of the pointing element from the candidate area based on a shape of the pointing element;
and
determining a pointing position of the pointing element in the operation surface and whether or not the pointing element had contact with the operation surface based on a result of the detecting.

11 more elements

A position detection device comprising:

a first processor configured to irradiate with infrared light toward an operation surface on which an operation with a pointing element is performed;
and
a second processor configured to
obtain a first taken image obtained by imaging the operation surface with a first camera configured to take an image with the infrared light,
obtain second taken image obtained by imaging the operation surface with a second camera different in imaging viewpoint from the first camera and configured to take an image with the infrared light,
remove a noise component from the first taken image and the second taken image based on a degree of coincidence in luminance gradient between a first background image and the first taken image, and a degree of coincidence in luminance gradient between a second background image and the second taken image, the first background image being obtained by imaging the operation surface with the first camera when the pointing element does not point the operation surface, and the second background image being obtained by imaging the operation surface with the second camera when the pointing element does not point the operation surface,
convert the first taken image from which the noise component removed into a first converted taken image calibrated with respect to the operation surface,
convert the second taken image from which the noise component removed into a second converted taken image calibrated with respect to the operation surface,
form a difference image between the first converted taken image and the second converted taken image,
extract an area in which a disparity amount between the first converted taken image and the second converted taken image is within a predetermined range out of the difference image as a candidate area in which an image of the pointing element is included;
detect a tip position of the pointing element from the candidate area based on a shape of the pointing element, and
determine a pointing position of the pointing element in the operation surface and whether or not the pointing element had contact with the operation surface based on a result of the detecting, wherein
the determining includes inputting a first area image clipped from the first converted taken image based on the tip position and a second area image clipped from the second converted taken image based on the tip to a learned neural network to determine the pointing position and whether or not the pointing element had contact with the operation surface.

10 more elements

A display device comprising:

a light source configured to irradiate with infrared light toward an operation surface on which an operation with a pointing element is performed;
a first camera configured to take an image with infrared light;
a second camera different in imaging viewpoint from the first camera, and configured to take an image with the infrared light;
a processor configured to
obtain a first taken image by imaging the operation surface with the first camera,
obtain a second taken image by imaging the operation surface with the second camera,
remove a noise component from the first taken image and the second taken image based on a degree of coincidence in luminance gradient between a first background image and the first taken image, and a degree of coincidence in luminance gradient between a second background image and the second taken image, the first background image being obtained by imaging the operation surface with the first camera when the pointing element does not point the operation surface, and the second background image being obtained by imaging the operation surface with the second camera when the pointing element does not point the operation surface, wherein the removing includes
removing the noise component from the first taken image and the second taken image based on
the degree of coincidence in luminance gradient between the first background image and the first taken image,
the degree of coincidence in luminance gradient between the second background image and the second taken image, and
a degree of coincidence in luminance gradient between the first taken image and the second taken image,
convert the first taken image from which the noise component removed into a first converted taken image calibrated with respect to the operation surface;
convert the second taken image from which the noise component removed into a second converted taken image calibrated with respect to the operation surface,
form a difference image between the first converted taken image and the second converted taken image;
extract an area in which a disparity amount between the first converted taken image and the second converted taken image is within a predetermined range out of the difference image as a candidate area in which an image of the pointing element is included,
detect a tip position of the pointing element from the candidate area based on a shape of the pointing element,
determine a pointing position of the pointing element in the operation surface and whether or not the pointing element had contact with the operation surface based on a result of the detecting;
and
a light modulation device configured to display an image on the operation surface based on the pointing position.

16 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/177,303Filed 17 Feb 2021
Priority: JP 2020-02406017 Feb 2020
Published: US 2021/0258551 A119 Aug 2021
Granted: 11 Jul 2023
Term ends: Not estimatedThe source records don’t support an estimate.

Classification21 codes

H04N 17/002
Diagnosis, testing or measuring for television systems or their details: for television cameras
G06F 3/0304
Arrangements for converting the position or the displacement of a member into a coded form: detection arrangements using opto-electronic means
G06F 3/0421
By opto-electronic means: by interrupting or reflecting a light beam, e.g. optical touch-screen
G06T 7/174
Segmentation; Edge detection: involving the use of two or more images