Back to results
US 2017/0249745 A1Published application
Fiducial marker patterns, their automatic detection in images, and applications thereof
AssigneeMillennium Three Technologies, Inc.
InventorMark Fiala
Life of this patent
Estimating the term…
Abstract
Fiducial markers are printed patterns detected by algorithms in imagery from image sensors for applications such as automated processes and augmented reality graphics. The present invention sets forth extensions and improvements to detection technology to achieve improved performance, and discloses applications of fiducial markers including multi-camera systems, remote control devices, augmented reality applications for mobile devices, helmet tracking, and weather stations.
Claims58
1-91. (canceled)
A method for detecting a marker in an image, comprising the steps of:
- detecting a marker in one or more previous frames of the image;
- using an edge detector to detect an edge in a current frame of said image;
- tracking line segment edges of the marker detected in the previous frame to find a new set of line segments;
- grouping the new set of line segments to provide a new set of polygons having salient points;
- calculating homography from polygon salient points;
- generating a list of homographies;
- extracting binary data from input image having homographies;
- verifying if the image is a marker by performing check sum and error correction functions;
- and
- if the image is a marker, identify as a marker and verify binary data;
- wherein the image is a consecutive image sequence;
8 more elements
A method for detecting a marker in an image, comprising the steps of:
- splitting the image into sub-images of smaller pixel size than the image;
- using a marker detection algorithm to detect a marker or portion of a marker in each sub-image;
- wherein each sub-image is a different region of the image from every other sub-image so that over several image frames a marker is likely to be detected.
A method for detecting a marker in an image, comprising the steps of:
- detecting a marker in one or more previous frames of the image;
- using an edge detector to detect an edge in a current frame of said image;
- determining blobs from centers of light or dark salient regions of similar brightness in the current frame of the image;
- tracking the centers of the blobs between frames;
- and
- determining motion of markers between frames by using the blobs.
3 more elements
A system comprising:
- a helmet containing a display visible to a users' eyes;
- several image sensors in a multi-camera array that is within the helmet;
- a collection of several image sensors attached together rigidly in a single frame, with each aimed at a different outward facing direction, for measuring position and orientation of the frame relative to an environment;
- and
- an ad hoc arrangement of fiducial marker patterns mounted in the environment;
- wherein the display shows virtual computer generated imagery either to replace or to augment real imagery;
- and
- wherein the multi-camera array comprises multiple outwards facing cameras that cover some or all sections of a complete spherical view
5 more elements
A system comprising:
- various types of media content such as manuals, pictures of interior contents, maintenance information, notes, audio recording notes, video tutorials, PDF documents, warranty and reordering information;
- markers on the media content;
- wherein the markers are detected in an environment.
A remote control and/or augmented reality system comprising:
- a) a mobile device with an outward facing video or still image capture, a display, a micro-computer, and optionally a network connection;
- b) fiducial marker patterns printed and mounted on objects or locations of interest,
- c) software, firmware, or hardware in said mobile device that can recognize said fiducial markers in imagery captured by a camera of said mobile device using a marker detection algorithm,
- d) services which are computer interfaces to some information or control functionality of interest to a user, such as databases which can be accessed from industrial automation systems,
- e) a server located either on a remote computer or within the same mobile device that provides files for use in creating a graphical interface widget for communication with said services,
- f) optionally a network that provides data communication capability between the server and one or more mobile devices, if the server is not inside the mobile device, for the purpose of communicating the widgets and, for applications such as industrial SCADA systems, optionally the services,
- g) requesting means in the mobile device to request widgets from the server according to a unique identifier of one or more fiducials detected in said camera's imagery, and
- h) drawing means inside the mobile device for drawing graphics of the widget on the display.
5 more elements
A remote control and/or augmented reality system for industry applications that provides industrial SCADA (industrial automation acronym for Supervisory Control And Data Acquisition) interaction comprising:
- a) a mobile device with an outward facing video or still image capture, a display, a micro-computer, and a network connection,
- b) fiducial marker patterns printed and mounted on objects or locations of interest such as machines, sensors, valves, storage tanks, and other objects and locations of relevance in an industrial automation system,
- c) software, firmware, or hardware in said mobile device that can recognize said fiducial markers in imagery captured by a camera of said mobile device,
- d) services which are computer interfaces to the SCADA information and/or control functionality of the industrial automation systems,
- e) a network that provides data communication capability between the webserver and one or more mobile devices for the purpose of communicating said widgets which contain code to SCADA systems through the services,
- f) a webserver that is connected over the network to the SCADA system to provide files to describe the graphical interface widget for communication with said services,
- g) requesting means in the mobile device to request widgets from the webserver according to a unique identifier of one or more fiducials detected in said camera's imagery,
- h) one or more web browsers inside the mobile device for drawing graphics of the widget on the display, and
- i) graphics elements to facilitate:
- easy development, use of existing web design expertise, and ability to preview full widget functionality in a conventional web browser.
7 more elements
An augmented reality system comprised of:
- capture means for capturing at least one video or still image, said capture means having a display screen;
- fiducial marker patterns located on one or more objects within view of the capture means;
- recognition means for recognizing said fiducial markers in the at least one video or still image;
- calculation means for calculating a mathematical transform between the display screen of the capture means and arbitrary world coordinates of the fiducial marker patterns;
- graphic drawing means for placing overlay drawings and graphics on the display screen;
- transmission means for transmitting the at least one video or still image to a remote location;
- and
- receiver means for receiving other overlay drawings and graphics from the remote location.
5 more elements
A system using a marker detection algorithm for processing imagery from one or more cameras aimed at a sphere that is constrained in position but has unknown changing rotation, comprising:
- markers mounted on said sphere;
- means for measuring a rotation position of the sphere without physical contact;
- determination means for determining a rotation, such as in rotation matrix, Euler angle, quaternion form;
- and
- output means for outputting the rotation.
2 more elements
Dependencies are inferred from the claim text.