# US 2019/0378304 A1 · Image processing system, optical sensor, and learning apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0378304 A1 - Document ID: US20190378304A1 - Status: Published application - Kind: A1 - Application number: 16/478,867 - Filed: 2017-03-09 - Published: 2019-12-12 - Earliest claimed: 2017-02-01 - Assignees: OMRON Corporation - Applicants: OMRON Corporation - Inventors: Tanichi ANDO - Web page: https://patentgrep.com/patent/US20190378304A1 ## CPC - G06V 10/147: Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) - G02B 3/0056: {arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses (G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085)} - G03B 15/00: Special procedures for taking photographs; Apparatus therefor - G06T 7/00: Image analysis - G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)} - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/764: using classification, e.g. of video objects - G06V 20/64: Three-dimensional [3D] objects - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N 23/55: Optical parts specially adapted for electronic image sensors; Mounting thereof - H04N 23/60: Control of cameras or camera modules - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image processing system according to an aspect of the invention includes: an optical sensor including: an optical member configured to refract light from a subject, the optical member including a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears; and an information processing apparatus configured to calculate attribute information representing an attribute of the subject by inputting the captured images obtained by the image sensors to a learning device that has learned the attribute of the subject. ## Claims ### Claim 1 An image processing system comprising: an optical sensor comprising: an optical member configured to refract light from a subject, the optical member comprising a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears, the plurality of image sensors being provided in correspondence with the transmissive units; and a processor configured with a program to perform operations comprising operation as an information processing apparatus configured to acquire attribute information representing an attribute of the subject by inputting the captured images obtained by the image sensors to a trained learning device that has been trained to determine the attribute of the subject, wherein the plurality of transmissive units in the optical member comprise a first transmissive unit that is arranged at a center and has a predetermined first converging angle, and second transmissive units that are arranged around the first transmissive unit and have a second converging angle that is greater than the first converging angle of the first transmissive unit. ### Claim 2 (depends on 1) The image processing system according to claim 1, wherein the first converging angle is 2.0 degrees at most, and the second converging angle is at least 20 degrees. ### Claim 3 (canceled) ### Claim 4 An image processing system comprising: an optical sensor comprising: an optical member configured to refract light from a subject, the optical member comprising a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears, the plurality of image sensors being provided in correspondence with the transmissive units; and an information processing apparatus configured to acquire attribute information representing an attribute of the subject by inputting the captured images obtained by the image sensors to a trained learning device that has been trained to determine the attribute of the subject, wherein the optical properties of the light projecting units are irregular. ### Claim 5 (depends on 1) The image processing system according to claim 1, wherein the learning device is constituted by a learning device that is trained using a neural network, a support vector machine, a self-organizing map, or reinforcement learning. ### Claim 6 (depends on 1) The image processing system according to claim 1, wherein the optical member comprises at least one of a lens array comprising a plurality of lenses, a diffraction grating, a scattering lens, and a holographic lens. ### Claim 7 (depends on 1) The image processing system according to claim 1, wherein the optical member comprises a lens array comprising a plurality of lenses as the plurality of transmissive units, and the optical properties of the transmissive units are realized by at least one of a size, a material, a shape, and a color of the lenses. ### Claim 8 (depends on 1) The image processing system according to claim 1, wherein the processor is configured with the program to perform operations such that operation as the information processing apparatus is further configured to output, as the attribute information, one of, or a combination of, a position, a direction, an orientation, a size, a shape, a motion, a type, individual identification information, a color, a brightness, and environmental information of the subject. ### Claim 9 (depends on 1) The image processing system according to claim 1, wherein the processor is configured with the program to perform operations such that operation as the information processing apparatus is further configured to create an image depicting the subject based on output from the learning device, and outputs the created image as the attribute information. ### Claim 10 An optical sensor comprising: an optical member configured to refract light from a subject, the optical member comprising a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears, the plurality of image sensors being provided in correspondence with the transmissive units, wherein the plurality of transmissive units in the optical member comprise a first transmissive unit that is arranged at a center and has a predetermined first converging angle, and second transmissive units that are arranged around the first transmissive unit and have a second converging angle that is greater the first transmissive unit. ### Claim 11 (canceled) ### Claim 12 (depends on 10) The optical sensor according to claim 10, wherein the first converging angle is 2.0 degrees at most, and the second converging angle is at least 20 degrees. ### Claim 13 An optical sensor comprising: an optical member configured to refract light from a subject, the optical member comprising a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears, the plurality of image sensors being provided in correspondence with the transmissive units, wherein the optical properties of the transmissive units are irregular. ### Claim 14 (depends on 10) The optical sensor according to claim 10, wherein the optical member comprises at least one of a lens array comprising a plurality of lenses, a diffraction grating, a scattering lens, and a holographic lens. ### Claim 15 (depends on 10) The optical sensor according to claim 10, wherein the optical member comprises a lens array comprising a plurality of lenses as the plurality of transmissive units, and the optical properties of the transmissive units are realized by at least one of a size, a material, a shape, and a color of the lenses. ### Claim 16 (depends on 10) A learning apparatus comprising: a learning image acquisition unit configured to acquire the captured images captured by the image sensors from the optical sensor according to claim 10; and a learning processing unit configured to train a learning device to output attribute information regarding the subject in response to the acquired captured images being input. ### Claim 17 (depends on 16) The learning apparatus according to claim 16, wherein the learning processing unit trains the learning device to output, as the attribute information, one of, or a combination of, a position, a direction, an orientation, a size, a shape, a motion, a type, individual identification information, a color, a brightness, and environmental information of the subject, and an image depicting the subject.