# US 12,118,777 B2 · Method and device for situation awareness > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,777 B2 - Document ID: US12118777B2 - Status: Granted - Kind: B2 - Application number: 18/364,281 - Filed: 2023-08-02 - Published: 2023-11-23 - Granted: 2024-10-15 - Earliest claimed: 2018-09-04 - Assignees: Seadronix Corp. - Applicants: Seadronix Corp. - Inventors: Byeol Teo Park; Han Keun Kim; Dong Hoon Kim - Web page: https://patentgrep.com/patent/US12118777B2 ## CPC - G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60) - B63B 79/15: for monitoring environmental variables, e.g. wave height or weather data - G01C 11/04: Interpretation of pictures - G01C 21/203: {specially adapted for water-borne vessels} - G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation - G06F 18/2431: Multiple classes - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 3/092: Reinforcement learning - G06T 7/136: involving thresholding - G06T 7/194: involving foreground-background segmentation - G06T 7/50: Depth or shape recovery - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G08G 3/00: Traffic control systems for marine craft (marking of navigational route B63B22/16, B63B51/00) - G08G 3/02: Anti-collision systems - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06T 7/11: Region-based segmentation - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B63B: SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING (arrangements of vessel ventilation, heating, cooling, or air-conditioning B63J2/00; floating substructures as supports of dredgers or soil-shifting machines E02F9/06) - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract A method for situation awareness is provided. The method comprises: preparing a neural network trained by a learning set, wherein the learning set includes a plurality of maritime images and maritime information including object type information which includes a first type index for a vessel, a second type index for a water surface and a third type index for a ground surface, and distance level information which includes a first level index indicating that a distance is undefined, a second level index indicating a first distance range and a third level index indicating a second distance range greater than the first distance range; obtaining a target maritime image generated from a camera; and determining a distance of a target vessel based on the distance level index of the maritime information being outputted from the neural network which receives the target maritime image and having the first type index. ## Claims ### Claim 1 A method for generating a segmentation image, the method comprising: obtaining a target maritime image generated from a camera, wherein the camera is installed on a port or a vessel, and the target maritime image represents a target object; and generating a target segmentation image using the target maritime image and a neural network, wherein the neural network is trained to output a segmentation image in response to an input image, wherein, in the segmentation image, pixels in an area corresponding to an object represented in the input image are assigned a value corresponding to the object, wherein the value is selected from identification values including at least a first value, a second value, and a third value, wherein the first value corresponds to a water surface, the second value corresponds to a first object at a first distance range, and the third value corresponds to the first object at a second distance range different from the first distance range, wherein the first distance range is closer than the second distance range, wherein, in the segmentation image, different identifiers are assigned to each of objects corresponding to the second value. ### Claim 2 (depends on 1) The method of claim 1, wherein the target maritime image represents a plurality of target objects, and wherein the method comprises: distinguishing the plurality of target objects based on target identifiers corresponding to each of the plurality of target objects in the obtained target segmentation image. ### Claim 3 (depends on 2) The method of claim 2, wherein, in the segmentation image, the object corresponding to the third value is assigned an identifier different from the identifier assigned to the object corresponding to the second value. ### Claim 4 (depends on 3) The method of claim 3, wherein, in the segmentation image, the identifiers assigned to the objects corresponding to the third value are the same. ### Claim 5 (depends on 1) The method of claim 1, wherein the identification values include at least a fourth value and a fifth value, wherein the fourth value is corresponding to a second object at the first distance range and the fifth value corresponding to the second object at the second distance range, and wherein the second object is of a different type than the first object. ### Claim 6 (depends on 5) The method of claim 5, wherein the method comprises: determining a type of the target object based on the value corresponding to the target object in the obtained segmentation image. ### Claim 7 (depends on 5) The method of claim 5, wherein the first object is a first type vessel, and wherein the second object is a second type vessel. ### Claim 8 (depends on 1) The method of claim 1, wherein the identification values include at least a sixth value, wherein the sixth value is corresponding to the first object at a third distance range, and wherein the third distance range is greater than the first distance range and the second distance range. ### Claim 9 (depends on 1) The method of claim 1, wherein the identification values include at least a seventh value, wherein the seventh value corresponding to a rope, wherein the target maritime image represents a target rope, and wherein the method comprises: determining whether the target object is moored based on a value corresponding to the target rope in the obtained segmentation image. ### Claim 10 (depends on 1) The method of claim 1, wherein the identification values include at least an eighth value, and wherein the eighth value corresponding to a buoy. ### Claim 11 A non-transitory computer-readable storage medium storing instructions thereon, the instructions when executed by a processor cause the processor to: obtain a target maritime image generated from a camera, wherein the camera is installed on a port or a vessel, and the target maritime image represents a target object; and generate a target segmentation image using the target maritime image and a neural network, wherein the neural network is trained to output a segmentation image in response to an input image, wherein, in the segmentation image, pixels in an area corresponding to an object represented in the input image are assigned a value corresponding to the object, wherein the value is selected from identification values including at least a first value, a second value, and a third value, wherein the first value corresponds to a water surface, the second value corresponds to a first object at a first distance range, and the third value corresponds to the first object at a second distance range different from the first distance range, wherein the first distance range is closer than the second distance range, wherein, in the segmentation image, different identifiers are assigned to each of objects corresponding to the second value. ### Claim 12 A device for situation awareness, the device comprising: a memory storing a neural network, wherein the neural network is trained to output a segmentation image in response to an input image, wherein, in the segmentation image, pixels in an area corresponding to an object represented in the input image are assigned a value corresponding to the object, wherein the value is selected from identification values including at least a first value, a second value, and a third value, wherein the first value corresponds to a water surface, the second value corresponds to a first object at a first distance range, and the third value corresponds to the first object at a second distance range different from the first distance range, wherein, in the segmentation image, different identifiers are assigned to each of objects corresponding to the second value, and a controller configured to: obtain a target maritime image generated from a camera, wherein the camera is installed on a port or a vessel, and the target maritime image represents a target object, and generate a target segmentation image using the target maritime image and the neural network. ### Claim 13 (depends on 12) The device of claim 12, wherein the target maritime image represents a plurality of target objects, and wherein the controller is configured to: distinguish the plurality of target objects based on target identifiers corresponding to each of the plurality of target objects in the obtained target segmentation image. ### Claim 14 (depends on 13) The device of claim 13, wherein, in the segmentation image, the object corresponding to the third value is assigned an identifier different from the identifier assigned to the object corresponding to the second value. ### Claim 15 (depends on 14) The device of claim 14, wherein, in the segmentation image, the identifiers assigned to the objects corresponding to the third value are the same. ### Claim 16 (depends on 12) The device of claim 12, wherein the identification values include at least a fourth value and a fifth value, wherein the fourth value is corresponding to a second object at the first distance range and the fifth value corresponding to the second object at the second distance range, and wherein the second object is of a different type than the first object. ### Claim 17 (depends on 16) The device of claim 16, wherein the controller is configured to: determine a type of the target object based on the value corresponding to the target object in the obtained segmentation image. ### Claim 18 (depends on 16) The device of claim 16, wherein the first object is a first type vessel, and wherein the second object is a second type vessel. ### Claim 19 (depends on 12) The device of claim 12, wherein the identification values include at least a sixth value, wherein the sixth value is corresponding to the first object at a third distance range, and wherein the third distance range is greater than the first distance range and the second distance range. ### Claim 20 (depends on 12) The device of claim 12, wherein the identification values include at least a seventh value, wherein the seventh value corresponding to a rope, wherein the target maritime image represents a target rope, and wherein the controller is configured to: determine whether the target object is moored based on a value corresponding to the target rope in the obtained segmentation image.