# US 11,441,874 B2 · Remote weapon control device and method for targeting and shooting multiple objects > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,441,874 B2 - Document ID: US11441874B2 - Status: Granted - Kind: B2 - Application number: 15/904,989 - Filed: 2018-02-26 - Published: 2019-05-16 - Granted: 2022-09-13 - Earliest claimed: 2017-11-10 - Assignees: Hanwha Defense Co., Ltd. - Applicants: Hanwha Defense Co., Ltd. - Inventors: Hee Seo Chae - Web page: https://patentgrep.com/patent/US11441874B2 ## CPC - F41A 19/08: remote actuated; lanyard actuated - F41A 23/34: on wheeled or endless-track vehicles - F41G 3/04: for dispersing fire from a battery {; for controlling spread of shots; for coordinating fire from spaced weapons} - F41G 3/06: with rangefinder (rangefinders per se G01C) - F41G 3/065: {Structural association of sighting devices with laser telemeters} - F41G 3/08: with means for compensating for speed, direction, temperature, pressure, or humidity of the atmosphere (measuring G01) - F41G 3/165: {using a TV-monitor} - F41G 3/22: for vehicle-borne armament, e.g. on aircraft - F41G 5/14: for vehicle-borne guns - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60) - H04N 7/183: {for receiving images from a single remote source} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2207/20092: Interactive image processing based on input by user - G06T 2207/30212: Military - F41A: FUNCTIONAL FEATURES OR DETAILS COMMON TO BOTH SMALLARMS AND ORDNANCE, e.g. CANNONS; MOUNTINGS FOR SMALLARMS OR ORDNANCE - F41G: WEAPON SIGHTS; AIMING (optical aspects thereof G02B) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A remote weapon control device for controlling a weapon having a photographing device is provided. The remote weapon control device includes: a communication interface configured to receive an image captured by the photographing device; an object extractor configured to extract objects from the image; a target extractor configured to extract targets from the objects; a shooting order determinator configured to determine an order of the targets for shooting; and a control signal generator configured to generate a shooting control signal for controlling the weapon to shoot the targets in the determined order. ## Claims ### Claim 1 A remote weapon control device for controlling a weapon having a camera, the remote weapon control device comprising: a display panel; a communication interface configured to receive an image captured by the camera; at least one processor configured to: extract objects from the image; identify targets from the objects; determine a shooting order of the targets for shooting; generate a shooting control signal for controlling the weapon to shoot the targets in the shooting order; control the display panel to display the image in a first portion of the display panel with marks that indicate the targets, the image including the targets and at least one of the objects from the image that is not a target; and calculate reference coordinates representing the targets in the image, calculate relative coordinates of the targets with respect to the reference coordinates, and control the display panel to display, in a second portion of the display panel that is different from the first portion while the image is displayed in the first portion, nodes located at the relative coordinates with respect to the reference coordinates while the at least one of the objects from the image that is not a target is not depicted in the second portion of the display panel; and an input interface configured to receive, from a user, a target selection command for selecting the targets from the extracted objects; wherein the target selection command comprises an area division command for dividing the image into a plurality of areas, the area division command comprising manually drawing a boundary line on the image to divide the image into the plurality of areas. ### Claim 2 (depends on 1) The remote weapon control device of claim 1, wherein the at least one processor is further configured to, in response to the area division command and an area selection command being input through the input interface, extract one or more objects located in a selected area as the targets. ### Claim 3 (depends on 1) The remote weapon control device of claim 1, wherein the target selection command comprises a select all command for selecting all objects in the image, and wherein, in response to the select all command being input to the input interface, the at least one processor is further configured to extract all objects in the image as the targets. ### Claim 4 (depends on 1) The remote weapon control device of claim 1, wherein the target selection command comprises an individual selection command for designating objects selected by the user from the objects included in the image, and wherein, in response to the individual selection command being input to the input interface, the at least one processor is further configured to extract the selected objects as the targets. ### Claim 5 (depends on 1) The remote weapon control device of claim 1, wherein the at least one processor is further configured to: detect an object having a friendly mark among the objects, and extract the objects except the detected object as the targets. ### Claim 6 (depends on 1) The remote weapon control device of claim 1, wherein the weapon comprises a distance measuring sensor configured to measure first distances between a location of the weapon and first locations of the targets, and wherein the at least one processor is further configured to determine the shooting order based on the measured first distances between the location of the weapon and the first locations of the targets. ### Claim 7 (depends on 6) The remote weapon control device of claim 6, wherein the at least one processor is further configured to determine the shooting order by prioritizing the targets in an order of the largest to the smallest distance from the location of the weapon. ### Claim 8 (depends on 6) The remote weapon control device of claim 6, wherein the at least one processor is further configured to determine the shooting order by prioritizing the targets in an order of the smallest to the largest distance from the location of the weapon. ### Claim 9 (depends on 6) The remote weapon control device of claim 6, wherein the at least one processor is further configured to estimate second distances between the location of the weapon and second locations of the targets after a reference time, and determine the shooting order by prioritizing the targets in an order of the smallest to the largest estimated second distances from the location of the weapon. ### Claim 10 (depends on 1) The remote weapon control device of claim 1, wherein sizes of the nodes are changed according to amounts of change in distances between the weapon and the targets. ### Claim 11 (depends on 1) The remote weapon control device of claim 1, wherein the at least one processor is further configured to output a camera control signal for controlling the camera such that at least one of the targets is included in the image. ### Claim 12 (depends on 1) The remote weapon control device of claim 1, wherein the weapon is mounted on a movable body, and wherein the at least one processor is further configured to output a movable body control signal for controlling a movement of the movable body such that at least one of the targets is included in the image. ### Claim 13 (depends on 12) The remote weapon control device of claim 12, wherein the at least one processor is further configured to output at least one of a camera control signal for controlling the camera and the movable body control signal for controlling the movement of the movable body such that a target having a highest shooting priority among the targets is included in the image. ### Claim 14 (depends on 1) The remote weapon control device of claim 1, wherein the communication interface transmits the shooting control signal to the weapon in response to a shooting authorization command being input to the input interface. ### Claim 15 (depends on 1) The remote weapon control device of claim 1, wherein the at least one processor is further configured to: detect an object that has been shot among the objects, and extract the objects except the detected object as the targets. ### Claim 16 A remote weapon control method for controlling a weapon having a camera, the remote weapon control method comprising: receiving an image captured by the camera; extracting objects from the image; receiving, from a user, a target selection command for selecting targets for shooting based on the objects; extracting the targets from the objects according to the target selection command; determining a shooting order of the targets for the shooting; generating a shooting control signal for controlling the weapon to shoot the targets in the shooting order; displaying, in a first portion of a display panel, the image with marks that indicate the targets, the image including the targets and at least one of the objects from the image that is not a target; calculating reference coordinates representing the targets in the image, calculating relative coordinates of the targets with respect to the reference coordinates, and displaying, on a second portion of the display panel that is different from the first portion while the image is displayed in the first portion, nodes located at the relative coordinates with respect to the reference coordinates while the at least one of the objects from the image that is not a target is not depicted in the second portion of the display panel; receiving a shooting authorization input from the user; and transmitting the shooting control signal to the weapon according to the shooting authorization input, wherein the target selection command comprises an area division command for dividing the image into a plurality of areas, the area division command comprising manually drawing a boundary line on the image to divide the image into the plurality of areas. ### Claim 17 (depends on 16) The remote weapon control method of claim 16, wherein the receiving the target selection command comprises receiving the area division command for dividing the image into the plurality of areas and an area selection command for selecting at least one of the plurality of areas, and wherein the extracting the targets comprises extracting one or more objects located in a selected area as the targets. ### Claim 18 (depends on 16) The remote weapon control method of claim 16, wherein, in the determining the shooting order, the shooting order is determined based on distances between the weapon and the targets. ### Claim 19 (depends on 9) The remote weapon control device of claim 9, wherein the at least one processor is further configured to estimate the second distances between the location of the weapon and the second locations of the targets by changing the first distances based on an amount of change in the first distances in the reference time.