# US 12,120,288 B2 · Fail safe surround view > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,120,288 B2 - Document ID: US12120288B2 - Status: Granted - Kind: B2 - Application number: 17/538,661 - Filed: 2021-11-30 - Published: 2023-06-01 - Granted: 2024-10-15 - Earliest claimed: 2021-11-30 - Assignees: Texas Instruments Incorporated - Applicants: Texas Instruments Incorporated - Inventors: Shashank Dabral; Aishwarya Dubey; Gowtham Abhilash Tammana - Web page: https://patentgrep.com/patent/US12120288B2 ## CPC - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - H04N 17/002: {for television cameras} - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - H04N 7/181: {for receiving images from a plurality of remote sources} - H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image} - H04N 23/80: Camera processing pipelines; Components thereof - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06F 15/7807: {System on chip, i.e. computer system on a single chip; System in package, i.e. computer system on one or more chips in a single package} - G06T 2200/28: involving image processing hardware - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A technique including capturing, by one or more cameras of a set of cameras disposed about a vehicle, one or more images, wherein a surround view system of the vehicle is configured to render a surround view image using a first hardware accelerator based on the one or more images, determining that a first hardware accelerator is unavailable, and rendering the surround view image using a second hardware accelerator based on the captured one or more images. ## Claims ### Claim 1 A method comprising: capturing, by a set of cameras disposed about a vehicle, one or more images; monitoring a shared register for an indication that a first hardware accelerator has output a frame, the first hardware accelerator being accessible using a first operating system; and responsive to determining that the indication has not been received within a threshold period of time: rendering a surround view image using a second hardware accelerator based on the one or more images captured by the set of cameras, the second hardware accelerator being accessible using a second operating system different from the first operating system; and restarting the first operating system. ### Claim 2 A method comprising: capturing, by a set of cameras disposed about a vehicle, one or more images, wherein a surround view system of the vehicle is configured to render a surround view image based on the one or more images; determining that a first hardware accelerator of the surround view system is unavailable to render the surround view image; and responsive to determining that the first hardware accelerator is unavailable to render the surround view image, rendering the surround view image using a second hardware accelerator of the surround view system based on the captured one or more images, wherein determining that the first hardware accelerator is unavailable to render the surround view image includes monitoring output frames of the first hardware accelerator. ### Claim 3 A method comprising: capturing, by a set of cameras disposed about a vehicle, one or more images, wherein a surround view system of the vehicle is configured to render a surround view image based on the one or more images; determining that a first hardware accelerator of the surround view system is unavailable to render the surround view image; rendering the surround view image using a second hardware accelerator of the surround view system based on the captured one or more images; determining a first camera, of the set of cameras, is unavailable; and based on the determination that the first camera is unavailable, rendering a compensated surround view image by adjusting blend lines of one or more cameras, of the set of cameras, adjacent to the first camera. ### Claim 4 (depends on 3) The method of claim 3, wherein the determination that the first camera is unavailable is based on at least one of control data or image data from the first camera. ### Claim 5 An electronic device, comprising: a first surround view sub-system including a first processor and a first hardware accelerator; and a second surround view sub-system including a second processor and a second hardware accelerator, wherein the first processor is configured to use a first operating system to access the first hardware accelerator, and the second processor is configured to use a second operating system different from the first operating system to access the second hardware accelerator, and wherein the second processor is configured to: monitor a shared register for an indication that the first hardware accelerator has output a frame; determine a failure associated with the first surround view sub-system based on that the indication has not been received within a threshold period of time; and responsive to a determination of the failure, cause the second hardware accelerator to render a surround view image based on the one or more images captured by one or more cameras. ### Claim 6 (depends on 5) The electronic device of claim 5, wherein the first hardware accelerator is implemented using a graphics processing unit, and wherein the second hardware accelerator is implemented using an image signal processor. ### Claim 7 (depends on 5) The electronic device of claim 5, wherein to determine the failure associated with the first surround view sub-system, the second processor is configured to: read frame information in a register shared by the first and second hardware accelerators; and determine that the frame information has not been updated by the first hardware accelerator within a threshold period of time. ### Claim 8 (depends on 6) The electronic device of claim 6, wherein the second processor is further configured to restart the first operating system based on the determination of the failure. ### Claim 9 (depends on 6) The electronic device of claim 6, wherein the first operating system is a high-level operating system, and the second operating system is a real-time operating system. ### Claim 10 An electronic device comprising: an image signal processor configured to receive one or more images captured by a set of cameras, disposed about a vehicle; a first processor configured to execute instructions to cause the first processor to render a surround view image around the vehicle using a first hardware accelerator, when available, based on the received one or more images; and a second processor configured to execute instructions to cause the second processor to: determine that the first hardware accelerator is unavailable; and responsive to a determination that the first hardware accelerator is unavailable, cause the surround view image to be rendered using a second hardware accelerator based on the one or more images captured, wherein the instructions to determine that the first hardware accelerator is unavailable comprises instructions to cause the second processor to monitor output frames of the first hardware accelerator. ### Claim 11 An electronic device comprising: an image signal processor configured to receive one or more images captured by a set of cameras, disposed about a vehicle; a first processor configured to execute instructions to cause the first processor to render a surround view image around the vehicle using a first hardware accelerator, when available, based on the received one or more images; and a second processor configured to execute instructions to cause the second processor to: determine that the first hardware accelerator is unavailable; and cause the surround view image to be rendered using a second hardware accelerator based on the one or more images captured; wherein the instructions cause the first processor or the second processor to: determine a first camera, of the set of cameras, is unavailable; and based on the determination that the first camera is unavailable, render a compensated surround view image by adjusting blend lines of one or more cameras, of the set of cameras, adjacent to the first camera. ### Claim 12 (depends on 11) The electronic device of claim 11, wherein the instructions cause the first processor or the second processor to determine that the first camera is unavailable based on at least one of control data or image data from the first camera. ### Claim 13 A non-transitory program storage device comprising instructions stored thereon when executed to cause: a first processor to render, when available, a surround view image around a vehicle using a first hardware accelerator based on one or more images captured by one or more cameras disposed about the vehicle, the first hardware accelerator accessible using a first operating system; and a second processor to: monitor a shared register for an indication that the first hardware accelerator has output a frame; and responsive to a determination that the indication has not been received within a threshold period of time, cause the surround view image to be rendered using a second hardware accelerator based on the one or more captured images, the second hardware accelerator accessible using a second operating system different from the first operating system. ### Claim 14 (depends on 13) The non-transitory program storage device of claim 13, wherein the first hardware accelerator is implemented using a graphics processing unit, and wherein the second hardware accelerator is implemented using an image signal processor. ### Claim 15 (depends on 13) The non-transitory program storage device of claim 13, wherein the instructions to cause the second processor to determine that the first hardware accelerator is unavailable comprise instructions to cause the second processor to: read frame information in a register shared by the first and second hardware accelerators; and determine that the frame information has not been updated by the first hardware accelerator within a threshold period of time. ### Claim 16 (depends on 14) The non-transitory program storage device of claim 14, wherein the instructions cause the second processor to restart the first operating system based on the determination that the first hardware accelerator is unavailable. ### Claim 17 (depends on 14) The non-transitory program storage device of claim 14, wherein the first operating system is a high-level operating system, and the second operating system is a real-time operating system.