# US 9,438,822 B2 · Image processing device, display device, image processing method, and computer-readable recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,438,822 B2 - Document ID: US9438822B2 - Status: Granted - Kind: B2 - Application number: 14/816,170 - Filed: 2015-08-03 - Published: 2015-11-26 - Granted: 2016-09-06 - Earliest claimed: 2013-10-03 - Assignees: Olympus Corporation - Applicants: Olympus Corporation - Inventors: Osamu Nonaka - Web page: https://patentgrep.com/patent/US9438822B2 ## CPC - G03B 37/02: with scanning movement of lens or cameras - G06T 11/00: Two-dimensional [2D] image generation - G06T 11/60: Creating or editing images; Combining images with text - H04N 5/2624: {for obtaining an image which is composed of whole input images, e.g. splitscreen} - H04N 5/265: Mixing - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image processing device includes: a determination unit that determines, based on a plurality of pieces of image data which is generated by continuously taking images of an area of field of view and input from an imaging unit provided outside the image processing device, whether the area of field of view of the imaging unit has been changed; an image composition unit that superimposes overlapping areas of imaging regions of a plurality of images corresponding to the plurality of pieces of image data along a direction in which the area of field of view of the imaging unit has been changed, to generate composite image data when the area of field of view of the imaging unit has been changed; and a display control unit that causes a display unit provided outside the imaging processing device to display a composite image corresponding to the generated composite image data. ## Claims ### Claim 1 An image processing device comprising: a determination unit, included in a CPU-based control unit, configured to determine, based on a plurality of pieces of image data which is generated by continuously taking images of a specified area of field of view and input from an imaging unit, whether the area of field of view of the imaging unit has been changed; an image composition unit, included in the CPU-based control unit, configured to superimpose imaging regions of a plurality of images corresponding to the plurality of pieces of image data along a direction in which the area of field of view of the imaging unit has been changed, to generate composite image data when the determination unit determines that the area of field of view of the imaging unit has been changed; and a display controller configured to cause a display to display a composite image corresponding to the composite image data generated by the image composition unit, wherein the display controller is configured to eliminate an area of the composite image corresponding to older image data among the plurality of pieces of image data combined by the image composition unit, with elapse of time, and to cause the display to display the composite image. ### Claim 2 (depends on 1) The image processing device according to claim 1, wherein when the determination unit determines that the area of field of view of the imaging unit has not been changed after the composite image data is generated, the image composition unit combines images corresponding to the plurality of pieces of image data generated by the imaging unit immediately after determination by the determination unit, in a latest image area corresponding to a latest plurality of pieces of image data which is combined into the composite image data, to update the composite image data. ### Claim 3 (depends on 1) The image processing device according to claim 1, wherein the display controller is configured to reduce an aspect ratio or a display area of the composite image, and to cause the display to display the composite image. ### Claim 4 (depends on 1) The image processing device according to claim 1, wherein the determination unit is configured to determine whether the area of field of view of the imaging unit has been changed, based on a rate of change in imaging regions of two images corresponding to two pieces of image data. ### Claim 5 (depends on 1) The image processing device according to claim 1, further comprising a sensor-based acceleration detection unit configured to detect acceleration of the image processing device, wherein the determination unit is configured to determine whether the area of field of view of the imaging unit has been changed, based on a change in the acceleration detected by the sensor-based acceleration detection unit. ### Claim 6 An image processing device comprising: a determination unit, included in a CPU-based control unit, configured to determine, based on a plurality of pieces of image data which is generated by continuously taking images of a specified area of field of view and input from an imaging unit, whether the area of field of view of the imaging unit has been changed; an image composition unit, included in the CPU-based control unit, configured to superimpose imaging regions of a plurality of images corresponding to the plurality of pieces of image data along a direction in which the area of field of view of the imaging unit has been changed, to generate composite image data when the determination unit determines that the area of field of view of the imaging unit has been changed; and a display controller configured to cause a display to display a composite image corresponding to the composite image data generated by the image composition unit, wherein the display controller is configured to reduce an aspect ratio or a display area of the composite image, and to cause the display to display the composite image. ### Claim 7 (depends on 6) The image processing device according to claim 6, wherein when the determination unit determines that the area of field of view of the imaging unit has not been changed after the composite image data is generated, the image composition unit combines images corresponding to the plurality of pieces of image data generated by the imaging unit immediately after determination by the determination unit, in a latest image area corresponding to a latest plurality of pieces of image data which is combined into the composite image data, to update the composite image data. ### Claim 8 (depends on 6) The image processing device according to claim 6, wherein the determination unit is configured to determine whether the area of field of view of the imaging unit has been changed, based on a rate of change in imaging regions of two images corresponding to two pieces of image data. ### Claim 9 (depends on 6) The image processing device according to claim 6, further comprising a sensor-based acceleration detection unit configured to detect acceleration of the image processing device, wherein the determination unit is configured to determine whether the area of field of view of the imaging unit has been changed, based on a change in the acceleration detected by the sensor-based acceleration detection unit. ### Claim 10 An image processing device comprising: a determination unit, included in a CPU-based control unit, configured to determine, based on a plurality of pieces of image data which is generated by continuously taking images of a specified area of field of view and input from an imaging unit, whether the area of field of view of the imaging unit has been changed; an image composition unit, included in the CPU-based control unit, configured to superimpose imaging regions of a plurality of images corresponding to the plurality of pieces of image data along a direction in which the area of field of view of the imaging unit has been changed, to generate composite image data when the determination unit determines that the area of field of view of the imaging unit has been changed; and a display controller configured to cause a display to display a composite image corresponding to the composite image data generated by the image composition unit, wherein the determination unit is configured to determine whether the area of field of view of the imaging unit has been changed, based on a rate of change in imaging regions of two images corresponding to two pieces of image data. ### Claim 11 (depends on 10) The image processing device according to claim 10, wherein when the determination unit determines that the area of field of view of the imaging unit has not been changed after the composite image data is generated, the image composition unit combines images corresponding to the plurality of pieces of image data generated by the imaging unit immediately after determination by the determination unit, in a latest image area corresponding to a latest plurality of pieces of image data which is combined into the composite image data, to update the composite image data. ### Claim 12 (depends on 10) The image processing device according to claim 10, further comprising a sensor-based acceleration detection unit configured to detect acceleration of the image processing device, wherein the determination unit is configured to determine whether the area of field of view of the imaging unit has been changed, based on a change in the acceleration detected by the sensor-based acceleration detection unit. ### Claim 13 An image processing device comprising: a determination unit, included in a CPU-based control unit, configured to determine, based on a plurality of pieces of image data which is generated by continuously taking images of a specified area of field of view and input from an imaging unit, whether the area of field of view of the imaging unit has been changed; an image composition unit, included in the CPU-based control unit, configured to superimpose imaging regions of a plurality of images corresponding to the plurality of pieces of image data along a direction in which the area of field of view of the imaging unit has been changed, to generate composite image data when the determination unit determines that the area of field of view of the imaging unit has been changed; a display controller configured to cause a display to display a composite image corresponding to the composite image data generated by the image composition unit; and a sensor-based acceleration detection unit configured to detect acceleration of the image processing device, wherein the determination unit is configured to determine whether the area of field of view of the imaging unit has been changed, based on a change in the acceleration detected by the sensor-based acceleration detection unit. ### Claim 14 (depends on 13) The image processing device according to claim 13, wherein when the determination unit determines that the area of field of view of the imaging unit has not been changed after the composite image data is generated, the image composition unit combines images corresponding to the plurality of pieces of image data generated by the imaging unit immediately after determination by the determination unit, in a latest image area corresponding to a latest plurality of pieces of image data which is combined into the composite image data, to update the composite image data.