# US 9,519,737 B2 · Filter modules for aperture-coded, multiplexed imaging systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,737 B2 - Document ID: US9519737B2 - Status: Granted - Kind: B2 - Application number: 14/611,844 - Filed: 2015-02-02 - Published: 2015-06-18 - Granted: 2016-12-13 - Earliest claimed: 2011-03-04 - Assignees: Ricoh Company, Ltd. - Applicants: Ricoh Company, Ltd. - Inventors: Kathrin Berkner; Sapna A. Shroff - Web page: https://patentgrep.com/patent/US9519737B2 ## CPC - G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) - G01J 1/0492: {using at least two different filters} - G01J 3/0208: {using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction} - G01J 3/0229: {using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters} - G01J 3/2823: {Imaging spectrometer} - G01J 3/36: Investigating two or more bands of a spectrum by separate detectors - G01J 4/00: Measuring polarisation of light - G02B 5/201: {in the form of arrays} - G02B 5/3025: {Polarisers, i.e. arrangements capable of producing a definite output polarisation state from an unpolarised input state (G02B5/3008, G02B5/3016 take precedence)} - H04N 25/134: {based on three different wavelength filter elements} - G01J 3/0224: {using polarising or depolarising elements} - G02B 2207/129: Coded aperture imaging - H04N 23/55: Optical parts specially adapted for electronic image sensors; Mounting thereof - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method for designing the spatial partition of a filter module used in an aperture-multiplexed imaging system. The filter module is spatially partitioned into filter cells, and the spatial partition is designed by considering data captured at the sensor in light of an application-specific performance metric. ## Claims ### Claim 1 A computer-implemented method for designing a filter module for use in an aperture-multiplexed imaging system, wherein the aperture-multiplexed imaging system performs a predefined task, and the aperture-multiplexed imaging system comprises a sensor that captures multiplexed images of an object, the filter module positioned approximately in a conjugate plane to the sensor to provide aperture coding of the multiplexed images, the computer-implemented method comprising a computer system performing the steps of: accessing a candidate design for the filter module, the candidate design for the filter module including a candidate spatial partition of filter cells within the filter module; simulating the imaging system with the candidate filter module forming multiplexed images, wherein the multiplexed images are aperture coded by the candidate filter module; simulating refocusing of the multiplexed images; calculating a performance metric that is a function of the simulated multiplexed images and the refocused multiplexed images, wherein the performance metric is indicative of the predefined task and the predefined task is also a function of the simulated multiplexed images and the refocused multiplexed images; and iteratively modifying the candidate spatial partition of filter cells to improve the calculated performance metric; wherein all of the above steps are performed by one or more processors within the computer system. ### Claim 2 (depends on 1) The computer-implemented method of claim 1 wherein the predefined task includes multi-modal detection based on the aperture coded multiplexed images, and the performance metric is indicative of the multi-modal detection. ### Claim 3 (depends on 1) The computer-implemented method of claim 1 wherein the predefined task includes multi-modal detection based on the aperture coded multiplexed images including the refocused multiplexed images, and the performance metric is indicative of the multi-modal detection. ### Claim 4 (depends on 1) The computer-implemented method of claim 1 wherein the performance metric is indicative of a sharpness of the refocused multiplexed images. ### Claim 5 (depends on 1) The computer-implemented method of claim 1 wherein the predefined task includes multi-modal detection, and the performance metric provides a trade-off between at least two of the following: image resolution of the multiplexed images, number of refocus planes used for refocusing, and multi-modal detection performance. ### Claim 6 (depends on 1) The computer-implemented method of claim 1 wherein the predefined task includes multi-modal detection, and the performance metric penalizes unwanted diffraction effects. ### Claim 7 (depends on 1) The computer-implemented method of claim 1 wherein modifying the candidate spatial partition comprises modifying a position of filter cells within the filter module. ### Claim 8 (depends on 1) The computer-implemented method of claim 1 wherein modifying the candidate spatial partition comprises modifying a size of filter cells within the filter module. ### Claim 9 (depends on 1) The computer-implemented method of claim 1 wherein modifying the candidate spatial partition comprises scaling a size of the candidate spatial partition. ### Claim 10 (depends on 1) The computer-implemented method of claim 1 wherein modifying the candidate spatial partition comprises changing a total number of filter cells in the filter module. ### Claim 11 (depends on 1) The computer-implemented method of claim 1 wherein the filter module further comprises guard bands between filter cells, and modifying the candidate spatial partition comprises modifying the size and/or position of the guard bands. ### Claim 12 (depends on 1) The computer-implemented method of claim 1 wherein each filter cell is characterized by an optical property, and the method further comprises: iteratively modifying the optical property of at least one filter cell to improve the calculated performance metric. ### Claim 13 (depends on 12) The computer-implemented method of claim 12 wherein the optical property is a spectral filter response. ### Claim 14 (depends on 12) The computer-implemented method of claim 12 wherein the optical property is a polarization filter response. ### Claim 15 (depends on 1) The computer-implemented method of claim 1 wherein improving the performance metric reduces crosstalk between the aperture coding of adjacent multiplexed images. ### Claim 16 (depends on 15) The computer-implemented method of claim 15 wherein different filter cells are characterized by different spectral filter responses, and the performance metric reduces spectral crosstalk between adjacent multiplexed images. ### Claim 17 (depends on 1) The computer-implemented method of claim 1 further comprising: iteratively modifying a design of the imaging system in conjunction with modifying the candidate design for the filter module, to improve the calculated performance metric. ### Claim 18 (depends on 1) The computer-implemented method of claim 1 wherein the predefined task is a detection of a substance based on its spectral properties. ### Claim 19 (depends on 1) The computer-implemented method of claim 1 wherein the predefined task is a detection of a presence of an object in noise. ### Claim 20 A non-transitory computer readable medium containing software instructions than, when executed on a hardware computer system, cause the hardware computer system to perform a method for designing a filter module for use in an aperture-multiplexed imaging system, wherein the aperture-multiplexed imaging system performs a predefined task, and the aperture-multiplexed imaging system comprises a sensor that captures multiplexed images of an object, the filter module positioned approximately in a conjugate plane to the sensor to provide aperture coding of the multiplexed images, the method comprising: accessing a candidate design for the filter module, the candidate design for the filter module including a candidate spatial partition of filter cells within the filter module; simulating the imaging system with the candidate filter module forming multiplexed images, wherein the multiplexed images are aperture coded by the candidate filter module; simulating refocusing of the multiplexed images; calculating a performance metric that is a function of the simulated multiplexed images and the refocused multiplexed images, wherein the performance metric is indicative of the predefined task and the predefined task is also a function of the simulated multiplexed images and the refocused multiplexed images; and iteratively modifying the candidate spatial partition of filter cells to improve the calculated performance metric.