# US 10,373,321 B2 · Sun position tracking method based on black sun effect of image sensor, computer program stored in medium, and solar sensor using the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,321 B2 - Document ID: US10373321B2 - Status: Granted - Kind: B2 - Application number: 15/583,486 - Filed: 2017-05-01 - Published: 2017-11-09 - Granted: 2019-08-06 - Earliest claimed: 2016-05-03 - Assignees: Research & Business Foundation Sungkyunkwan University - Applicants: Research & Business Foundation Sungkyunkwan University - Inventors: Sukhan Lee; Rashid Saleem; Jaewoong Kim - Web page: https://patentgrep.com/patent/US10373321B2 ## CPC - G06T 7/248: {involving reference images or patches} - G06T 5/20: using local operators - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/70: Denoising; Smoothing - G06T 7/136: involving thresholding - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 7/277: involving stochastic approaches, e.g. using Kalman filters - G06T 7/292: Multi-camera tracking - H02S 20/32: specially adapted for solar tracking - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30181: Earth observation - Y02E 10/50: Photovoltaic [PV] energy - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H02S: GENERATION OF ELECTRIC POWER BY CONVERSION OF INFRARED RADIATION, VISIBLE LIGHT OR ULTRAVIOLET LIGHT, e.g. USING PHOTOVOLTAIC [PV] MODULES (obtaining electrical energy from radioactive sources G21H1/12; light sensitive inorganic semiconductor devices H10F; light sensitive organic semiconductor devices H10K30/00; thermoelectric devices H10N10/00; pyroelectric devices H10N15/00) - Y02E: REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION ## Abstract The present invention provides a sun position tracking method based on a black sun effect of an image sensor, a computer program including commands for causing a computer system to perform the method, and a solar sensor using the method. The sun position tracking method includes processing a plurality of images captured using a plurality of image sensors installed to capture a plurality of images of the sun irrespective of the position of the sun to estimate the center point of the black sun and track the position of the sun. ## Claims ### Claim 1 A sun position tracking method, comprising: capturing at least one image using at least one image sensor; extracting solar vectors based on at least one location of at least one black spot appearing in the captured at least one image; and determining a position of a sun based on a rotation involving solar vectors, among the extracted solar vectors, measured at different time points, wherein the at least one black spot is caused by overexposure of pixels in the at least one image sensor corresponding to the at least one location of the at least one black spot in the at least one captured image. ### Claim 2 (depends on 1) The sun position tracking method according to claim 1, wherein the capturing of the at least one image enables the position of the sun to be tracked. ### Claim 3 (depends on 1) The sun position tracking method according to claim 1, wherein a plurality of sun position estimation results is fused based on the captured at least one image to track the position of the sun. ### Claim 4 (depends on 1) The sun position tracking method according to claim 1, further comprising estimating a center point of each black spot of the at least one black spot in the captured at least one image, wherein the extracting of the solar vectors comprises extracting the solar vectors with respect to the center point of each black spot of the at least one black spot, and wherein the determining of the position of the sun comprises transforming the extracted solar vectors into reference coordinates, and fusing the transformed solar vectors. ### Claim 5 (depends on 4) The sun position tracking method according to claim 4, wherein the estimating of the center point of each black spot of the at least one black spot comprises acquiring an image of the sun from among the captured at least one image, repeatedly segmenting the image of the sun while varying an intensity threshold value, detecting corner points, in response to the image of the sun being segmented, and obtaining the center point from the detected corner points at a final segmenting of the image of the sun. ### Claim 6 (depends on 5) The sun position tracking method according to claim 5, wherein the obtaining of the center point comprises obtaining a centroid of strong corner points corresponding to corner points detected from the final segmented image of the sun, and the estimating of the center point further comprises estimating a strong corner point, among the strong corner points, having a minimum distance from the centroid, as the center point. ### Claim 7 (depends on 4) The sun position tracking method according to claim 4, wherein the transforming of the extracted solar vectors into the reference coordinates comprises obtaining an initial rotation using the solar vectors, among the extracted solar vectors, which are represented in sensor coordinates, and other solar vectors, among the extracted solar vectors, represented in the reference coordinates, at the different time points, and determining an average of a plurality of initial rotations calculated by changing the different time points by a predetermined number of times as a final rotation, to transform the extracted solar vectors into the reference coordinates. ### Claim 8 (depends on 4) The sun position tracking method according to claim 4, wherein the fusing of the solar vectors comprises removing noise of azimuths and altitudes of the solar vectors, and fusing the solar vectors to calculate a final solar vector. ### Claim 9 (depends on 8) The sun position tracking method according to claim 8, wherein the removing of the noise comprises using a Kalman filter to remove the noise. ### Claim 10 (depends on 1) A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1. ### Claim 11 (depends on 1) The sun position tracking method of claim 1, wherein an intensity of light impinging on the pixels in the at least one image sensor corresponding to the at least one location of the at least one black spot in the captured at least one image is maximal compared to remaining pixels in the at least one image sensor. ### Claim 12 A solar sensor, comprising: at least one sensor configured to capture at least one image; and at least one processor configured to extract solar vectors based on at least one location of at least one black spot appearing in the captured at least one image, and determine a position of a sun based on a rotation involving solar vectors, among the extracted solar vectors, measured at different time points, wherein the at least one black spot is caused by overexposure of pixels in the at least one image sensor corresponding to the at least one location of the at least one black spot in the captured at least one image. ### Claim 13 (depends on 12) The solar sensor according to claim 12, wherein the captured at least one image enables the position of the sun to be tracked, and the at least one processor is further configured to fuse a plurality of sun position estimation results based on the captured at least one image to track the position of the sun. ### Claim 14 (depends on 12) The solar sensor according to claim 12, wherein the at least one processor comprises a center point estimation module configured to estimate a center point of each black spot of the at least one black spot in the captured at least one image, a solar vector extraction module configured to extract the solar vectors with respect to the center point, a coordinate transformation module configured to transform the extracted solar vectors into reference coordinates, and a fusion module configured to fuse the transformed solar vectors. ### Claim 15 (depends on 14) The solar sensor according to claim 14, wherein the center point estimation module is further configured to acquire an image of the sun from among the captured at least one image, repeatedly segment the image of the sun by a predefined number of times while varying an intensity threshold value, detect corner points, in response to the image of the sun being segmented, obtain a centroid of strong corner points corresponding to corner points, among the corner points, detected from a final segmented image, and estimate a strong corner point, among the corner points, having a minimum distance from the centroid as the center point. ### Claim 16 (depends on 14) The solar sensor according to claim 14, wherein the coordinate transformation module is further configured to obtain an initial rotation using the solar vectors, among the extracted solar vectors, which are represented in sensor coordinates, and other solar vectors, among the extracted solar vectors, represented in the reference coordinates, at the different time points, and determine an average of a plurality of initial rotations calculated by changing the different time points by a predetermined number of times as a final rotation, to transform the extracted solar vectors into the reference coordinates. ### Claim 17 (depends on 14) The solar sensor according to claim 14, wherein the fusion module is further configured to remove noise of azimuths and altitudes of the solar vectors, and fuse the solar vectors to calculate a final solar vector. ### Claim 18 (depends on 17) The solar sensor according to claim 17, wherein the noise is removed using an extended Kalman filter. ### Claim 19 (depends on 12) The sun position tracking method of claim 12, wherein the at least one image sensor comprises six image sensor cameras installed at six vertexes of a virtual icosahedron to enable three of the six image sensor cameras to capture a black sun with hemispherical coverage irrespective of the position of the sun, and the six vertexes include one reference vertex and five vertexes neighboring the reference vertex. ### Claim 20 A sun position tracking method based on a black sun effect of an image sensor, comprising: capturing one or more images using one or more image sensors; and processing the captured one or more images using the black sun effect, comprising estimating a center point of each black sun in the captured one or more images, extracting solar vectors with respect to the center point of each black sun, obtaining an initial rotation using two solar vectors represented in sensor coordinates and two solar vectors represented in reference coordinates at two different time points, determining an average of a plurality of initial rotations calculated by changing the two different time points by a predetermined number of times as a final rotation, to transform the extracted solar vectors into the reference coordinates, and fusing the transformed solar vectors.