# US 10,482,594 B2 · Multi-perspective imaging system and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,594 B2 - Document ID: US10482594B2 - Status: Granted - Kind: B2 - Application number: 15/712,600 - Filed: 2017-09-22 - Published: 2018-03-29 - Granted: 2019-11-19 - Earliest claimed: 2016-09-23 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Daniel E. Potter; Arthur Y. Zhang - Web page: https://patentgrep.com/patent/US10482594B2 ## CPC - G06T 7/001: {using an image reference approach} - G06T 1/0007: {Image acquisition} - G06T 7/74: {involving reference images or patches} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - H04N 19/102: characterised by the element, parameter or selection affected or controlled by the adaptive coding - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/17: the unit being an image region, e.g. an object - H04N 19/597: specially adapted for multi-view video sequence encoding - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A system includes a processor to capture a first image of a scene by a first imaging device of the array of imaging devices, capture a second image of the scene by a second imaging device of the array of imaging devices, perform image analysis of the first image and the second image and determining that an object is present in the first image and the object is present in the second image, the first image representing a first perspective of the object and the second image representing a second perspective of the object different from the first perspective of the object, classify the object with a classification based on a list of known objects and weight an object portion of the first image and an object portion of the second image based on the classification, compress the first image and the second image based on the weighting, encode the first image and the second image as multi-perspective image data, and store the multi-perspective image data in the non-transitory computer-readable medium based on the classification. ## Claims ### Claim 1 A system, comprising: an array of imaging devices; a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, cause the processor to perform operations comprising: capturing a first image of a scene by a first imaging device of the array of imaging devices; capturing a second image of the scene by a second imaging device of the array of imaging devices; performing image analysis of the first image and the second image and determining that an object is present in the first image and the object is present in the second image, the first image representing a first perspective of the object and the second image representing a second perspective of the object different from the first perspective of the object; classifying the object with a classification based on a list of known objects and weighting an object portion of the first image and an object portion of the second image based on the classification; compressing the first image and the second image based on the weighting, the object portion of the first image and the object portion of the second image being compressed at a particular compression level based on at least one of the classification or the weighting; encoding the first image and the second image as multi-perspective image data; and storing the multi-perspective image data in the non-transitory computer-readable medium. ### Claim 2 (depends on 1) The system of claim 1, the operations further comprising: decoding the multi-perspective image data and displaying a representation of the scene using the first image and the second image. ### Claim 3 (depends on 1) The system of claim 1, the operations further comprising: compressing the first image and the second image at a compression level based on one or more of the classification and the weighting. ### Claim 4 (depends on 1) The system of claim 1, the operations further comprising: compressing the object portion of the first image and the object portion of the second image at a first particular compression level based on one or more of the classification and the weighting, and compressing a non-object portion of the first image and a non-object portion of the second image at a second particular compression level, the first particular compression level comprising a lossless compression level and the second particular compression level comprising a lossy compression level. ### Claim 5 (depends on 1) The system of claim 1, the operations further comprising: discarding the first image and the second image when the weighting does not meet a particular threshold. ### Claim 6 (depends on 1) The system of claim 1, the system further comprising a sensor device, the operations further comprising: determining a distance to the object from the first image device and the second imaging device using the sensor device to determine time-of-flight; weighting the object portion of the first image and the object portion of the second image based on the distance; and compressing the first image and the second image based on the weighting, and encoding the first image and the second image as the multi-perspective image data. ### Claim 7 (depends on 1) The system of claim 1, the system further comprising a Global Positioning System (GPS) device, the operations further comprising: determining a location of the object using the GPS device; weighting the object portion of the first image and the object portion of the second image based on the location of the object; and compressing the first image and the second image based on the weighting, and encoding the first image and the second image as the multi-perspective image data. ### Claim 8 (depends on 1) The system of claim 1, the operations further comprising: determining bidirectional reflectance distribution function (BRDF) information for the object based on the first image and the second image; weighting the object portion of the first image and the object portion of the second image based on the BRDF information; and compressing the first image and the second image based on the weighting, and encoding the first image and the second image as the multi-perspective image data. ### Claim 9 (depends on 1) The system of claim 1, the operations further comprising: activating the array of imaging devices to begin capturing images in response to user input, the user input comprising input to one or more of a touch screen device and a microphone. ### Claim 10 (depends on 1) The system of claim 1, the operations further comprising: activating the array of imaging devices to begin capturing images in response to user input, the user input comprising one or more of a gesture made on a touch screen and a verbal command. ### Claim 11 (depends on 1) The system of claim 1, the operations further comprising: activating the array of imaging devices to begin capturing images when the array of imaging devices determine that the object is present in one or more of the first image and the second image. ### Claim 12 (depends on 1) The system of claim 1, the system further comprising a Global Positioning System (GPS) device, the operations further comprising: determining a location using the GPS device; activating the array of imaging devices to begin capturing images based on the location. ### Claim 13 (depends on 1) The system of claim 1, the system further comprising a vehicle, the array of imaging devices mounted on an exterior of the vehicle. ### Claim 14 (depends on 1) The system of claim 1, the system further comprising a vehicle, the array of imaging devices mounted on an interior of the vehicle. ### Claim 15 (depends on 1) The system of claim 1, the operations further comprising: generating a panoramic image of the object using the multi-perspective image data. ### Claim 16 (depends on 1) The system of claim 1, the operations further comprising: displaying a representation of the scene including displaying the object from a perspective including a front of a vehicle, a first side of the vehicle, a second side of the vehicle, and a rear of the vehicle. ### Claim 17 (depends on 1) The system of claim 1, the operations further comprising: displaying a representation of the scene including displaying the object from the first perspective of the object and transitioning to displaying the object from the second perspective of the object. ### Claim 18 (depends on 1) The system of claim 1, the image analysis further comprising: determining the object in the object portion of the first image is a same object as the object in the object portion of the second image by comparing the object portion of the first image and the object portion of the second image to features and patterns in images of objects in the list of known objects. ### Claim 19 (depends on 18) The system of claim 18, the image analysis further comprising determining that the object in the object portion of the first image is the same object as the object in the object portion of the second image based on depth information, BRDF information, map data, and location information. ### Claim 20 (depends on 18) The system of claim 18, the image analysis further comprising determining the object portion of the first image as a subset of the first image representing the object and determining the object portion of the second image as a subset of the second image representing the object. ### Claim 21 A method comprising: obtaining a first image of a scene captured by a first imaging device of an array of imaging devices; obtaining a second image of the scene captured by a second imaging device of the array of imaging devices; performing, by at least one processor, image analysis of the first image and the second image and determining that an object is present in the first image and the object is present in the second image, the first image representing a first perspective of the object and the second image representing a second perspective of the object different from the first perspective of the object; classifying, by the at least one processor, the object with a classification based on a list of known objects and weighting an object portion of the first image and an object portion of the second image based on the classification; compressing the first image and the second image based on the weighting, the object portion of the first image and the object portion of the second image being compressed at a particular compression level based on at least one of the classification or the weighting; encoding the first image and the second image as a multi-perspective image data; and storing the multi-perspective image data in a non-transitory computer-readable medium. ### Claim 22 A non-transitory computer-readable medium comprising instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising: obtaining a first image of a scene captured by a first imaging device of an array of imaging devices; obtaining a second image of the scene captured by a second imaging device of the array of imaging devices; performing image analysis of the first image and the second image and determining that an object is present in the first image and the object is present in the second image, the first image representing a first perspective of the object and the second image representing a second perspective of the object different from the first perspective of the object; classifying the object with a classification based on a list of known objects and weighting an object portion of the first image and an object portion of the second image based on the classification; compressing the first image and the second image based on the weighting, the object portion of the first image and the object portion of the second image being compressed at a particular compression level based on at least one of the classification or the weighting; and encoding the first image and the second image as multi-perspective image data, the multi-perspective image data being stored according to at least one of the classification or the weighting.