# US 10,873,719 B2 · Image processing device, image processing system, image processing method, and computer program product > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,873,719 B2 - Document ID: US10873719B2 - Status: Granted - Kind: B2 - Application number: 15/665,459 - Filed: 2017-08-01 - Published: 2017-12-14 - Granted: 2020-12-22 - Earliest claimed: 2015-02-05 - Assignees: Ricoh Company, Ltd. - Applicants: Keiji Ohmura; Takeshi Hasegawa - Inventors: Keiji Ohmura; Takeshi Hasegawa - Web page: https://patentgrep.com/patent/US10873719B2 ## CPC - H04N 7/181: {for receiving images from a plurality of remote sources} - G06F 18/40: Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor - G06T 7/001: {using an image reference approach} - G06V 10/945: {User interactive design; Environments; Toolboxes} - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T) - G09G 5/14: Display of multiple viewports - H04N 5/44504: {Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits} - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)} - H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window} - H04N 21/47: End-user applications - G09G 5/377: for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract An image processing device includes a display device, a video display controller configured to display video data imaged by an imaging device in a predetermined display region to be displayed by the display device, and a region display controller configured to display a plurality of display elements representing determination regions based on a plurality of types of determination functions with respect to the video data in the display region by superimposing the plurality of display elements on the video data in aspects different from each other. ## Claims ### Claim 1 An image processing device, comprising: a display device; and processing circuitry configured to display real-time video data imaged by an imaging device in a predetermined display region to be displayed by the display device, designate a first determination region and a second determination region according to a user operation, the first determination region being a target of a first determination function for performing a first determination with respect to the video data, the second determination region being a target of a second determination function for performing a second determination with respect to the video data, both the first determination region and the second determination region being fixed regions, the first determination function being different from the second determination function, and display a display element representing the first determination region and a display element representing the second determination region in the display region by superimposing the elements on the video data in aspects different from each other, wherein the processing circuitry displays, as one of the determination regions, a display element of a trigger region, which is a portion of the displayed real-time video data imaged by the imaging device, for generating a trigger on the basis of a comparison result of an image currently displayed within the trigger region with respect to a first standard image, and display elements of a plurality of detection regions as a plurality of the determination regions that are portions of the displayed real-time video data imaged by the image device and that are separate from the trigger region, for performing image determination of whether or not there is an abnormality by an image currently displayed within each respective detection region being compared with each of one or more second standard images before a predetermined time or after the predetermined time on the basis of a timing of the trigger, on each of the display regions, and the processing circuitry is further configured to compare an image in the trigger region with the first standard image, to generate a trigger signal on the basis of the comparison result, and compare features in an image in each of the detection regions with features in a corresponding second standard image of the one or more second standard images at a time point, to determine whether or not there is the abnormality based on a threshold corresponding to a similarity or difference between the compared features being exceeded, the time point being set in advance for each of the one or more detection regions, to be earlier or later than when the trigger signal is generated, by a predetermined time, wherein for at least one of the detection regions, the time point is set to be earlier than when the trigger signal is generated, and for at least another of the detection regions, the time point is set to be later than when the trigger signal is generated. ### Claim 2 (depends on 1) The image processing device according to claim 1, wherein the processing circuitry displays at least any one of the display elements of each of the determination regions as frames surrounding the determination regions of colors different from each other. ### Claim 3 (depends on 1) The image processing device according to claim 1, wherein the processing circuitry changes an aspect of the display element in a case where a predetermined state is detected in the determination region to an aspect of the display element in a case where the predetermined state is not detected in the determination region, with respect to at least any one of the display elements of each of the determination regions, and displays the changed aspect. ### Claim 4 (depends on 1) The image processing device according to claim 1, further comprising: a memory configured to store the video data imaged by the imaging device; and the processing circuitry is configured to cause the memory to store information of a date in which each of the determination regions is designated and information of a date in which the predetermined state is detected in each of the determination regions, wherein the processing circuitry reproduces and displays the video data stored in the storage in the display region, and the processing circuitry displays the display elements representing each of the determination regions in the display region, according to the information of the date stored in the memory. ### Claim 5 (depends on 4) The image processing device according to claim 4, wherein the processing circuitry is configured to display a detection date of at least any one of the predetermined states detected in each of the determination regions in a list, on the basis of the information of the date stored in the memory. ### Claim 6 (depends on 1) The image processing device according to claim 1, wherein the processing circuitry displays the video data imaged by the plurality of imaging devices in the display region. ### Claim 7 (depends on 1) The image processing device according to claim 1, wherein the processing circuitry continues display of each of the display elements representing a determination result of the image determination in each of one or more detection regions until at least all of the image determinations of one or more detection regions associated with the trigger region are ended. ### Claim 8 (depends on 1) The image processing device according to claim 1, wherein the processing circuitry is configured to output sound representing determination end when the image determination in a final detection region of the detection regions associated with the trigger region is ended. ### Claim 9 (depends on 1) The image processing device according to claim 1, wherein the processing circuitry is configured to output sound representing occurrence of an abnormality when the abnormality is detected in the detection region. ### Claim 10 (depends on 1) An image processing system, comprising: the image processing device according to claim 1; and one or more imaging devices. ### Claim 11 An image processing method, comprising: displaying real-time video data imaged by an imaging device in a predetermined display region to be displayed by a display device; designating a first determination region and a second determination region according to a user operation, the first determination region being a target of a first determination function for performing a first determination with respect to the video data, the second determination region being a target of a second determination function for performing a second determination with respect to the video data, both the first determination region and the second determination region being fixed regions, the first determination function being different from the second determination function; and displaying a display element representing the first determination region and a display element representing the second determination region in the display region by superimposing the elements on the video data in aspects different from each other, wherein the method includes displaying, as one of the determination regions, a display element of a trigger region, which is a portion of the displayed real-time video data imaged by the imaging device, for generating a trigger on the basis of a comparison result of an image currently displayed within the trigger region with respect to a first standard image, and displaying elements of a plurality of detection regions as a plurality of the determination regions that are portions of the displayed real-time video data imaged by the image device and that are separate from the trigger region, for performing image determination of whether or not there is an abnormality by an image currently displayed within each respective detection region being compared with each of one or more second standard images before a predetermined time or after the predetermined time on the basis of a timing of the trigger, on each of the display regions, and the method further including comparing an image in the trigger region with the first standard image, to generate a trigger signal on the basis of the comparison result, and comparing features in an image in each of the detection regions with features in a corresponding second standard image of the one or more second standard images at a time point, to determine whether or not there is the abnormality based on a threshold corresponding to a similarity or difference between the compared features being exceeded, the time point being set in advance for each of the one or more detection regions, to be earlier or later than when the trigger signal is generated, by a predetermined time, wherein for at least one of the detection regions, the time point is set to be earlier than when the trigger signal is generated, and for at least another of the detection regions, the time point is set to be later than when the trigger signal is generated. ### Claim 12 A computer program product comprising a non-transitory computer-readable recording medium having stored therein a program that causes a computer to execute: displaying real-time video data imaged by an imaging device in a predetermined display region to be displayed by a display device; designating a first determination region and a second determination region according to a user operation, the first determination region being a target of a first determination function for performing a first determination with respect to the video data, the second determination region being a target of a second determination function for performing a second determination with respect to the video data, both the first determination region and the second determination region being fixed regions, the first determination function being different from the second determination function; and displaying a display element representing the first determination region and a display element representing the second determination region in the display region by superimposing the elements on the video data in aspects different from each other, wherein the method includes displaying, as one of the determination regions, a display element of a trigger region, which is a portion of the displayed real-time video data imaged by the imaging device, for generating a trigger on the basis of a comparison result of an image currently displayed within the trigger region with respect to a first standard image, and displaying elements of a plurality of detection regions as a plurality of the determination regions that are portions of the displayed real-time video data imaged by the image device and that are separate from the trigger region, for performing image determination of whether or not there is an abnormality by an image currently displayed within each respective detection region being compared with each of one or more second standard images before a predetermined time or after the predetermined time on the basis of a timing of the trigger, on each of the display regions, and the method further including comparing an image in the trigger region with the first standard image, to generate a trigger signal on the basis of the comparison result, and comparing features in an image in each of the detection regions with features in a corresponding second standard image of the one or more second standard images at a time point, to determine whether or not there is the abnormality based on a threshold corresponding to a similarity or difference between the compared features being exceeded, the time point being set in advance for each of the one or more detection regions, to be earlier or later than when the trigger signal is generated, by a predetermined time, wherein for at least one of the detection regions, the time point is set to be earlier than when the trigger signal is generated, and for at least another of the detection regions, the time point is set to be later than when the trigger signal is generated. ### Claim 13 (depends on 1) The image processing device according to claim 1, wherein the determination regions are set in advance by a user of the image processing device. ### Claim 14 (depends on 1) The image processing device according to claim 1, wherein the first detection function and the second detection function comprise a full-time detecting function and a flash detecting function, the full-time detecting function being a function of constantly executing image determination of whether or not an image in a full-time detection region as one of the determination regions is abnormal, the flash detecting function is a function of executing image determination of whether or not an image in a detection region of the determination regions is abnormal at a specific timing. ### Claim 15 (depends on 1) The image processing device according to claim 1, wherein the features in the image in each of the one or more detection regions and the features in the corresponding second standard image of the one or more second standard images are pixel values in each respective image. ### Claim 16 (depends on 1) The image processing device according to claim 1, wherein for the detection regions, the abnormality is based on a threshold corresponding to a similarity between the compared features being exceeded, or based on a threshold corresponding to a difference between the compared features being exceeded.