# US 2019/0318456 A1 · Images for perception modules of autonomous vehicles > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318456 A1 - Document ID: US20190318456A1 - Status: Published application - Kind: A1 - Application number: 16/381,707 - Filed: 2019-04-11 - Published: 2019-10-17 - Earliest claimed: 2018-04-12 - Assignees: Tusimple, Inc. - Applicants: TuSimple - Inventors: Pengfei CHEN; Nan YU; Naiyan WANG; Xiaodi HOU - Web page: https://patentgrep.com/patent/US20190318456A1 ## CPC - G06T 5/70: Denoising; Smoothing - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - B60W 40/02: related to ambient conditions - G06T 5/75: Unsharp masking - G06T 5/92: based on global image properties - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - H04N 25/13: characterised by the spectral characteristics of the filter elements - H04N 25/134: {based on three different wavelength filter elements} - H04N 25/445: by skipping some contiguous pixels within the read portion of the array - B60W 2420/403: Image sensing, e.g. optical camera - G06T 2207/10024: Color image - H04N 25/447: by preserving the colour pattern with or without loss of information - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Disclosed are devices, systems and methods for processing an image. In one aspect a method includes receiving an image from a sensor array including an x-y array of pixels, each pixel in the x-y array of pixels having a value selected from one of three primary colors, based on a corresponding x-y value in a mask pattern. The method may further include generating a preprocessed image by performing preprocessing on the image. The method may further include performing perception on the preprocessed image to determine one or more outlines of physical objects. ## Claims ### Claim 1 A method for processing an image, comprising: receiving an image including an x-y array of pixels from a sensor array, each pixel in the x-y array of pixels having a value selected from one of three primary colors, based on a corresponding x-y value in a mask pattern; generating a preprocessed image by performing preprocessing on the image; and performing computerized perception on the preprocessed image to determine one or more outlines of physical objects. ### Claim 2 (depends on 1) The method of claim 1, wherein the mask pattern is a Bayer pattern. ### Claim 3 (depends on 2) The method of claim 2, wherein the Bayer pattern is a red-green-green-blue pattern assigned to a 2-pixel by 2-pixel array repeated across the raw image. ### Claim 4 (depends on 1) The method of claim 1, wherein the pattern includes a greater number of green pixel values than both red and blue. ### Claim 5 (depends on 1) The method of claim 1, wherein the pattern is selected such that a value of every other pixel along a row in the reduced image corresponds to green value of the raw image. ### Claim 6 (depends on 1) The method of claim 1, wherein the pattern is selected such that a value of every other pixel along a column in the reduced image corresponds to green value of the raw image. ### Claim 7 (depends on 1) The method of claim 1, wherein, each x-y value in the pattern is from one of three possible values. ### Claim 8 (depends on 1) The method of claim 1, wherein the image is generated using one or more color-selective filters. ### Claim 9 (depends on 1) The method of claim 1, wherein the preprocessing is performed on the image from the sensor array without human perception image enhancement. ### Claim 10 (depends on 9) The method of claim 9, wherein the human perception image enhancement includes one or more of de-mosaicing, white balancing, and noise reduction. ### Claim 11 (depends on 1) The method of claim 1, wherein the preprocessing does not include scaling one or more pixels' R, G, or B value for white balancing. ### Claim 12 (depends on 1) The method of claim 1, wherein the preprocessing does not include reconstruction a full color image from incomplete color samples output from the sensor array overlaid with a color filter array for de-mosaicing. ### Claim 13 (depends on 1) The method of claim 1, wherein the preprocessing does not include noise reduction, wherein noise reduction includes reduction of salt and pepper noise, wherein a noisy pixel bears little relation to the color of surrounding pixels, or reduction of Gaussian noise. ### Claim 14 A computer apparatus comprising a processor, wherein the processor is configured to receive an image including an x-y array of pixels from a sensor array, each pixel in the x-y array of pixels having a value selected from one of three primary colors, based on a corresponding x-y value in a mask pattern, generate a preprocessed image by performing preprocessing on the image, and perform perception of the preprocessed image to determine one or more outlines of physical objects. ### Claim 15 (depends on 14) The apparatus of claim 14, wherein the mask pattern is a Bayer pattern. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the Bayer pattern is a red-green-green-blue pattern assigned to a 2-pixel by 2-pixel array repeated across the raw image. ### Claim 17 (depends on 14) The apparatus of claim 14, wherein the pattern includes a greater number of green pixel values than both red and blue. ### Claim 18 (depends on 14) The apparatus of claim 14, wherein the processor selects the pattern such that a value of every other pixel along a row in the reduced image corresponds to green value of the raw image. ### Claim 19 A computer readable program medium having code stored thereon, the code, when executed by a processor, causing the processor to implement a method comprising: receiving an image including an x-y array of pixels from a sensor array, each pixel in the x-y array of pixels having a value selected from one of three primary colors, based on a corresponding x-y value in a mask pattern; generating a preprocessed image by performing preprocessing on the image; and performing computerized perception on the preprocessed image to determine one or more outlines of physical objects. ### Claim 20 (depends on 19) The computer readable program medium of claim 19, wherein the preprocessing is performed on the image from the sensor array without human perception image enhancement. ### Claim 21 (depends on 19) The computer readable program medium of claim 19, wherein the preprocessing excludes scaling one or more pixels' R, G, or B value for white balancing.