# US 11,440,466 B2 · Semiconductor device and message image signal output method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,440,466 B2 - Document ID: US11440466B2 - Status: Granted - Kind: B2 - Application number: 16/435,040 - Filed: 2019-06-07 - Published: 2019-12-26 - Granted: 2022-09-13 - Earliest claimed: 2018-06-21 - Assignees: Renesas Electronics Corporation - Applicants: Renesas Electronics Corporation - Inventors: Hirofumi Kawaguchi; Koji Yasuda; Akihide Takahashi - Web page: https://patentgrep.com/patent/US11440466B2 ## CPC - H04N 9/3182: {Colour adjustment, e.g. white balance, shading or gamut (white balance per se H04N9/73; control of amplitude of colour signals H04N9/68; colour control circuits for displays in general G09G5/02, G09G3/2003, G09G3/3607)} - B60Q 1/50: for indicating other intentions or conditions, e.g. request for waiting or overtaking - G01C 21/365: {Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself} - G06T 5/20: using local operators - G08G 1/096861: {where the immediate route instructions are output to the driver, e.g. arrow signs for next turn} - H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)} - H04N 9/3194: {including sensor feedback} - H04N 9/73: Colour balance circuits, e.g. white balance circuits or colour temperature control (camera processing pipelines for colour balance H04N23/88) - H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control - B60Q 2400/50: Projected symbol or information, e.g. onto the road or car body - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract A semiconductor device has a position estimation part calculating an estimated position of a mobile at a scheduled time to project a message image, based on movement information of the mobile, a reference image signal output part deciding a reference area being an area to project a reference image, based on a relative positional relation between the mobile at the estimated position and the projection area, and outputting a reference image signal being a signal of the reference image, a test image signal acquisition part acquiring a test image signal being a signal of an image obtained by imaging the reference area with the reference image projected thereon, and an image adjustment part adjusting the message image signal, based on the reference image signal and the test image signal. ## Claims ### Claim 1 A semiconductor device outputting a message image signal being a signal of a message image to be projected onto a projection area located at a predetermined distance from a mobile, the semiconductor device comprising: a position estimation circuit which calculates an estimation position of the mobile at a scheduled time to project the message image based on movement information of the mobile; a reference image signal output circuit which decides a reference area being an area to project a reference image based on a relative positional relation between the mobile at the estimation position and the projection area, and outputs a reference image signal being a signal of the reference image to be projected onto the reference area; a test image signal acquisition circuit which acquires a test image signal being a signal of a test image obtained by imaging the reference area onto which the reference image is projected; a comparison circuit which compares the reference image signal with the test image signal; an image adjustment circuit which adjusts the message image signal based on comparing the reference image signal with the test image signal; and a message image output circuit which outputs the message image signal adjusted by the image adjustment circuit. ### Claim 2 (depends on 1) The semiconductor device according to claim 1, wherein the reference image signal output circuit decides a time to output the reference image signal, based on the estimation position. ### Claim 3 (depends on 1) The semiconductor device according to claim 1, wherein the image adjustment circuit adjusts a pixel value included in the message image signal. ### Claim 4 (depends on 1) The semiconductor device according to claim 1, wherein the reference image signal output circuit outputs the reference image signal in which brightness values of red, green, and blue are equal to each other. ### Claim 5 (depends on 1) The semiconductor device according to claim 1, wherein the comparison circuit compares a brightness value included in the reference image signal with a brightness value included in the test image signal to calculate a difference between the brightness values. ### Claim 6 (depends on 5) The semiconductor device according to claim 5, wherein the comparison circuit makes an angle of view of the reference image and an angle of view of the test image correspond to each other and compares the reference image signal and the test image signal for each pixel. ### Claim 7 (depends on 6) The semiconductor device according to claim 6, wherein the image adjustment circuit makes an angle of view of the test image and an angle of view of the message image correspond to each other and adjusts the message image signal for each pixel of the message image signal. ### Claim 8 (depends on 1) The semiconductor device according to claim 1, further including an optimum projection area decision circuit, wherein the reference image signal output circuit outputs a reference image signal having a width larger than that of the message image, wherein the comparison circuit extracts a plurality of candidate areas each having the same width as the message image and at different positions in a width direction of the reference image, and calculates a difference in brightness value between the reference image signal and the test image signal for each candidate area, and wherein the optimum projection area decision circuit decides as an optimum projection area, one of the candidate areas, in which the difference in brightness value is minimum. ### Claim 9 (depends on 1) The semiconductor device according to claim 1, further including an adjustment signal acquisition circuit which acquires an adjustment signal corresponding to a reference image projected by the mobile from a target moved around the mobile, and wherein the image adjustment circuit adjusts a message image based on the adjustment signal acquired from the target. ### Claim 10 (depends on 9) The semiconductor device according to claim 9, further including a permission determination circuit which determines whether the target permits projection of the reference image. ### Claim 11 (depends on 9) The semiconductor device according to claim 9, wherein the adjustment signal acquisition circuit acquires a signal for adjusting a pixel value included in the message image signal. ### Claim 12 (depends on 9) The semiconductor device according to claim 9, wherein the position estimation circuit estimates a position of the mobile and a position of the target at the schedule time to project the message image based on the movement information of the mobile and movement information of the target respectively. ### Claim 13 (depends on 9) The semiconductor device according to claim 9, wherein the adjustment signal acquisition circuit acquires a signal for performing trapezoidal correction on the message image signal. ### Claim 14 (depends on 1) The semiconductor device according to claim 1, further including an adjustment signal output circuit, wherein the test image signal acquisition circuit acquires a sample image signal being a signal of an image obtained by capturing a sample image projected by a projecting device moved around the mobile, wherein the comparison circuit compares the reference image signal with the sample image signal, and wherein the adjustment signal output circuit outputs an adjustment signal for adjusting the sample image signal based on a result of comparison between the reference image signal and the sample image signal. ### Claim 15 (depends on 14) The semiconductor device according to claim 14, further including a permission signal output circuit which outputs a signal to permit projection of the sample image to the projecting device when the projecting device projects the sample image. ### Claim 16 (depends on 1) The semiconductor device according to claim 1, further including an optimum route decision circuit, wherein the reference image signal output circuit outputs a reference image signal larger in width than the message image and longer in length than the message image in a depth direction, wherein the comparison circuit extracts a plurality of candidate areas at an angle of view corresponding to the message image and at different positions with respect to the reference image and calculates a difference in brightness value between the reference image signal and the test image signal for each candidate area, and wherein the optimum route decision circuit calculates a plurality of candidate routes corresponding to the candidate areas along an advancing direction of the mobile and decides as an optimum route, the candidate route in which the difference in brightness value is minimum, of the candidate routes. ### Claim 17 (depends on 16) The semiconductor device according to claim 16, wherein the optimum route decision circuit performs trapezoidal correction on the test image signal, divides the test image into a plurality of grids, and calculates a route corresponding to each divided grid. ### Claim 18 (depends on 16) The semiconductor device according to claim 16, further including a steering instruction circuit which gives instruction to steer the mobile, based on the optimum route. ### Claim 19 A message image signal output method of outputting a message image signal being a signal of a message image to be projected onto a projection area located at a predetermined distance from a mobile, the method comprising: calculating an estimation position of the mobile at a scheduled time to project the message image based on movement information of the mobile; deciding a reference area being an area to project a reference image based on a relative positional relation between the mobile at the estimation position calculated by the calculating and the projection area; outputting a reference image signal being a signal of the reference image to be projected onto the reference area; acquiring a test image signal being a signal of a test image obtained by imaging the reference area onto which the reference image is projected; comparing the reference image signal with the test image signal; adjusting the message image signal based on a comparison result between the reference image signal and the test image signal; and outputting the adjusted message image signal.