# US 2018/0322342 A1 · Method and apparatus for detecting and classifying objects associated with vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322342 A1 - Document ID: US20180322342A1 - Status: Published application - Kind: A1 - Application number: 15/585,764 - Filed: 2017-05-03 - Published: 2018-11-08 - Earliest claimed: 2017-05-03 - Assignees: GM Global Technology Operations LLC - Applicants: GM Global Technology Operations LLC - Inventors: David Clifford; Orhan Demirovic; Orhan Bulan; Eric Hatfield; David Hiemenga; Mohannad Murad - Web page: https://patentgrep.com/patent/US20180322342A1 ## CPC - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 7/11: Region-based segmentation - G06T 7/254: involving subtraction of images - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions - H04N 7/183: {for receiving images from a single remote source} - B60R 2300/30: characterised by the type of image processing - B60R 2300/8006: for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo - G06F 2218/12: Classification; Matching - G06T 2207/10016: Video; Image sequence - G06T 2207/30268: Vehicle interior - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR ## Abstract A method and apparatus for detecting and classifying objects associated with a vehicle are provided. The method includes capturing a first image and a second image of a cabin of a vehicle, dividing the first image and the second image into a plurality of regions, comparing a first region of the first image to a second region of the second image, in response to detecting a difference between the second region and the first region, classifying an object present in the difference between the second region and the first region and labeling the classified object, and providing a notification of the classified object to at least one from among an occupant of the vehicle and an operator of the vehicle. ## Claims ### Claim 1 A method for detecting and classifying objects associated with a vehicle, the method comprising: capturing a first image and a second image of a cabin of a vehicle; dividing the first image and the second image into a plurality of regions; comparing a first region of the first image to a second region of the second image, the second region of the second image corresponding to the first region in the first image; in response to detecting a difference between the second region and the first region, classifying an object present in the difference between the second region and the first region and labeling the classified object; and providing a notification of the classified object to at least one from among an occupant of the vehicle and an operator of the vehicle. ### Claim 2 (depends on 1) The method of claim 1, wherein the capturing the first image of the cabin is performed prior to an occupant entering the vehicle, and wherein the capturing the second image of the cabin is performed after to the occupant exits the vehicle. ### Claim 3 (depends on 1) The method of claim 1, further comprising: receiving an input to re-label the classified object from the operator of the vehicle; and re-classifying the object present in the difference between the second region and the first region based on the received input. ### Claim 4 (depends on 1) The method of claim 1, wherein the plurality of regions comprise two or more from among a front passenger seat, a front driver side, a rear left seat, a rear right seat, a rear seat area, a front passenger floor, a front driver floor, a rear left floor, a rear right floor, a rear floor, a dashboard, a cup holder, a center console, a trunk area and a surface adjacent to rear window. ### Claim 5 (depends on 1) The method of claim 1, wherein the providing the notification of the detected difference comprises transmitting information on the second image or the classified object to an operator. ### Claim 6 (depends on 1) The method of claim 1, wherein the capturing the first image and the second image of the cabin of the vehicle is performed by a camera. ### Claim 7 (depends on 1) The method of claim 1, wherein the comparing the first region of the first image to the second region of the second image comprises detecting a change in pixel values in the between the second region and the first region. ### Claim 8 (depends on 1) The method of claim 1, wherein the classifying the object in present in the difference between the second region and the first region comprises comparing an image of the object to images of objects stored in a database and classifying the object according to a classification of an image stored in the database that is closest to the image of the object. ### Claim 9 (depends on 1) The method of claim 1, wherein the classifying the object in present in the difference between the second region and the first region comprises identifying the object by performing at least one from among edge matching, greyscale matching and gradient matching. ### Claim 10 (depends on 1) A non-transitory computer readable medium comprising computer instructions executable by a computer to perform the method of claim 1. ### Claim 11 An apparatus that detects and classifies objects associated with a vehicle, the apparatus comprising: at least one memory comprising computer executable instructions; and at least one processor configured to read and execute the computer executable instructions, the computer executable instructions causing the at least one processor to: capture a first image and a second image of an area of a vehicle; divide the first image and the second image into a plurality of regions; compare a first region of the first image to a second region of the second image, the second region of the second image corresponding to the first region in the first image; in response to detecting a difference between the second region and the first region, classify an object present in the difference between the second region and the first region and labeling the classified object; and provide a notification of the classified object to at least one from among an occupant of the vehicle and an operator of the vehicle. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the computer executable instructions cause the at least one processor to capture the first image prior to an occupant entering the vehicle, and wherein the computer executable instructions cause the at least one processor to capture the second image after to the occupant exits the vehicle. ### Claim 13 (depends on 11) The apparatus of claim 11, wherein the computer executable instructions cause the at least one processor to: receive an input to re-label the classified object from the operator of the vehicle; and reclassify the object present in the difference between the second region and the first region based on the received input. ### Claim 14 (depends on 11) The apparatus of claim 11, wherein the plurality of regions comprise two or more from among a front passenger seat, a front driver side, a rear left seat, a rear right seat, a rear seat area, a front passenger floor, a front driver floor, a rear left floor, a rear right floor, a rear floor, a dashboard, a cup holder, a center console, a trunk area and a surface adjacent to rear window. ### Claim 15 (depends on 11) The apparatus of claim 11, wherein the computer executable instructions further cause the at least one processor to provide the notification of the detected difference by transmitting information on the second image or the classified object to an operator. ### Claim 16 (depends on 11) The apparatus of claim 11, further comprising a camera mounted in the vehicle, wherein the computer executable instructions further cause the at least one processor to control the camera to capture the first image and the second image. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the computer executable instructions further cause the at least one processor to compare the first region of the first image to the second region of the second image by detecting a change in pixel values in the between the second region and the first region. ### Claim 18 (depends on 11) The apparatus of claim 11, wherein the computer executable instructions further cause the at least one processor to classify the object in present in the difference between the second region and the first region by comparing an image of the object to images of objects stored in a database and classifying the object according to a classification of an image stored in the database that is closest to the image of the object. ### Claim 19 (depends on 11) The apparatus of claim 11, wherein the computer executable instructions further cause the at least one processor to classify the object in present in the difference between the second region and the first region by identifying the object by performing at least one from among edge matching, greyscale matching and gradient matching. ### Claim 20 A non-transitory computer readable medium comprising computer readable instructions, the computer readable instructions executable by a computer to perform a method comprising: receiving a first image and a second image of a cabin of a vehicle; dividing the first image and the second image into a plurality of regions; comparing a first region of the first image to a second region of the second image, the second region of the second image corresponding to the first region in the first image; in response to detecting a difference between the second region and the first region, classifying an object present in the difference between the second region and the first region and labeling the classified object; and providing a notification of the classified object to at least one from among an occupant of the vehicle and an operator of the vehicle.