# US 2014/0193032 A1 · Image super-resolution for dynamic rearview mirror > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0193032 A1 - Document ID: US20140193032A1 - Status: Published application - Kind: A1 - Application number: 14/041,058 - Filed: 2013-09-30 - Published: 2014-07-10 - Earliest claimed: 2013-01-07 - Assignees: GM Global Technology Operations LLC - Applicants: GM Global Technology Operations LLC - Inventors: WENDE ZHANG; JINSONG WANG; KENT S. LYBECKER; JEFFREY S. PIASECKI; RYAN M. FRAKES; TRAVIS S. HESTER - Web page: https://patentgrep.com/patent/US20140193032A1 ## CPC - G06T 5/73: Deblurring; Sharpening - G06T 3/04: Context-preserving transformations, e.g. by using an importance map (panospheric to cylindrical image transformations G06T3/12) - G06T 2207/10016: Video; Image sequence - G06T 2207/20012: Locally adaptive - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Method for applying super-resolution to images captured by a camera device of a vehicle includes receiving a plurality of image frames captured by the camera device. For each image frame, a region of interest is identified within the image frame requiring resolution related to detail per pixel to be increased. Spatially-implemented super-resolution is applied to the region of interest within each image to enhance image sharpness within the region of interest. ## Claims ### Claim 1 Method for applying super-resolution to images captured by a camera device of a vehicle, comprising: obtaining a plurality of image frames captured by the camera device; for each image frame: identifying a region of interest within the image frame requiring resolution related to detail per pixel to be increased; and applying spatially-implemented super-resolution to the region of interest within the image frame to enhance image sharpness within the region of interest. ### Claim 2 (depends on 1) The method of claim 1, wherein the identified region of interest corresponds to an enlarged center region of a display image frame subsequent to stretching a raw image captured by the camera device having optical aberrations, the enlarged center region having a diminished resolution related to detail per pixel. ### Claim 3 (depends on 1) The method of claim 1, wherein the identified region of interest comprises the entire image frame. ### Claim 4 (depends on 1) The method of claim 1, wherein the applied spatially-implemented super-resolution to the region of interest within the image frame comprises: applying a de-noising filter to reduce noise through smoothing one or more identified planes within the region of interest while preserving one or more identified edges within the identified region of interest. ### Claim 5 (depends on 4) The method of claim 4, wherein applying the de-noising filter comprises: monitoring image gradient within the identified region of interest; identifying the one or more edges where the monitored image gradient is at least a gradient edge threshold; identifying the one or more planes where the monitored image gradient is less than the gradient edge threshold; and replacing a pixel value for each pixel within the identified one or more planes by a weighted average of pixel values from nearby pixels within the one or more identified planes. ### Claim 6 (depends on 1) The method of claim 1, wherein the applied spatially-implemented super-resolution to the region of interest within each image frame comprises: applying a location-dependent sharpening filter to achieve a desired increased sharpening within the region of interest, said desired increased sharpening achieved based on adjusting filter settings of the sharpening filter associated with the region of interest. ### Claim 7 (depends on 6) The method of claim 6 wherein applying the location-dependent sharpening filter further achieves a desired decreased sharpening in regions outside of the region of interest, said desired decreased sharpening achieved based on adjusting filter settings of the sharpening filter associated with respective regions outside the region of interest. ### Claim 8 (depends on 6) The method of claim 6, wherein the adjusted filter settings of the sharpening filter comprises: an adjusted radius of the identified edges within the region of interest, the adjusted radius indirectly proportional to the adjusted magnitude of overshoot; and an adjusted intensity threshold for controlling a minimum intensity change of pixels associated with the identified edges that will be sharpened. ### Claim 9 (depends on 1) The method of claim 1, further comprising: for the plurality of image frames having the spatially-implemented super-resolution applied thereto: applying temporally-implemented super-resolution to the image frames dependent upon a location within each of the image frames, said location comprising the identified region of interest; and generating a resultant image based on the temporally-implemented super-resolution applied to the image frames. ### Claim 10 (depends on 9) The method of claim 9, further comprising: detecting a degree of object motion among the plurality of image frames having the spatially-implemented super-resolution applied thereto; and only applying the temporally-implemented super-resolution to the image frames if the degree of object motion includes one of pitch, no motion, and a magnitude of motion less than a motion threshold. ### Claim 11 (depends on 9) The method of claim 9, wherein applying the temporally-implemented super-resolution to the image frames comprises: for each of the image frames: identifying stationary regions and moving regions; applying image registration for recognizing at least one of features and objects within each of the stationary and moving regions; identifying differences between each of the image frames based on the applied image registration; removing the identified differences from respective ones of the image frames; and subsequent to removing the identified differences, applying the temporally-implemented super-resolution among the input image frames to generate the resulting image frame having enhanced resolution. ### Claim 12 Method for applying super-resolution to one or more images captured by a camera device of a subject vehicle, comprising: for multiple image frames over a period of time of a scene representing a field of view rearward of the subject vehicle: identifying a region of interest within the multiple image frames; detecting a degree of object motion among the multiple image frames; if the degree of object motion includes one of pitch, no motion, and a small magnitude of motion less than a motion threshold, applying image registration for recognizing at least one of objects and features within the input image frames; applying temporally-implemented super-resolution to enhance the resolution within the region of interest of the multiple image frames; and generating a resulting image based on the multiple image frames with the temporally-implemented super-resolution applied thereto. ### Claim 13 (depends on 12) The method of claim 12, further comprising: for a single image frame of the scene representing the field of view rearward of the subject vehicle: identifying the region of interest within the single image frame; applying location-dependent spatially-implemented super-resolution to enhance resolution within the identified region of interest, said spatially-implemented super-resolution including at least one of a de-noising filter and a sharpening filter each dependent upon the location corresponding to the region of interest; ### Claim 14 (depends on 12) The method of claim 12, wherein each of the multiple image frames corresponds to one of: respective ones of single image frames having location-dependent spatially-implemented super-resolution applied thereto prior to detecting the degree of object motion; and respective ones of raw image frames captured by the camera device. ### Claim 15 (depends on 13) The method of claim 13, wherein the single image having the location-dependent spatially-implemented super-resolution applied to the identified region of interest corresponds one of: the generated resulting image based on the multiple image frames; and a raw image frame captured by the camera device. ### Claim 16 (depends on 12) The method of claim 12, wherein the identified region of interest corresponds to an enlarged center region of a display image generated by stretching a raw image captured by the camera device, the enlarged center region having a diminished resolution related to detail per pixel. ### Claim 17 (depends on 13) The method of claim 13, wherein the sharpening filter comprises an unsharp masking filter and the de-noising filter comprises a bilateral filter. ### Claim 18 (depends on 12) The method of claim 12, further comprising: applying image stabilization prior to applying image registration if the degree of object motion detects pitch. ### Claim 19 Apparatus for applying super-resolution to one or more images captured by a vision-based imaging system of a subject vehicle, comprising: a rear-view camera device configured to capture a plurality of image frames of a scene representing a field of view rearward of the subject vehicle; and a processing device configured to: receive the plurality of image frames captured by the rear-view camera device, identify, within each image frame, an enlarged center region having diminished resolution related to detail per pixel, apply at least one of a de-noising filter and a sharpening filter to the enlarged center region within each image frame to enhance the resolution within the enlarged center region, for the plurality of image frames having the at least one of the de-noising filter and the sharpening filter applied thereto: apply temporally-implemented super-resolution to the image frames dependent upon a location within each of the image frames, said location comprising the identified region of interest, generate a resultant image based on the temporally-implemented super-resolution applied to the image frames, and a rearview mirror display device configured to display images captured by the rear-view camera device.