# US 11,440,407 B2 · Non-contact operating apparatus for vehicle and vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,440,407 B2 - Document ID: US11440407B2 - Status: Granted - Kind: B2 - Application number: 16/751,443 - Filed: 2020-01-24 - Published: 2020-10-01 - Granted: 2022-09-13 - Earliest claimed: 2019-03-27 - Assignees: Subaru Corporation - Applicants: Subaru Corporation - Inventors: Hideyuki Sase; Takemi Oguri; Tomoyuki Kitamura - Web page: https://patentgrep.com/patent/US11440407B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/211: {producing three-dimensional [3D] effects, e.g. stereoscopic images} - B60K 35/25: using haptic output - B60K 35/26: using acoustic output - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60K 35/80: Arrangements for controlling instruments - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - B60K 35/22: Display screens - B60K 35/81: for controlling displays - B60K 2360/11: Instrument graphical user interfaces or menu aspects - B60K 2360/146: Instrument input by gesture - B60K 2360/149: Instrument input by detecting viewing direction not otherwise provided for - B60K 2360/177: Augmented reality - B60K 2360/21: using cameras - B60K 2360/31: Virtual images - G06T 2207/30196: Human being; Person - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A non-contact operating apparatus, for a vehicle, includes a generating unit, a projecting device, an operation detecting device, an operation determining unit, and a motion detecting device. The generating unit is configured to generate and update an image containing an image object. The image object is operable by an occupant present within a vehicle compartment. The projecting device is configured to project the image in a predetermined display region. The operation detecting device is configured to detect an operation site of the occupant. The motion detecting device is configured to detect motion of the vehicle. The operation determining unit is configured to adjust a detection result of the operation detecting device to suppress the motion of the occupant on the basis of a detection result of the motion detecting device, and determine presence or absence of the operation against the image object by the operation site. ## Claims ### Claim 1 A non-contact operating apparatus for a vehicle, the non-contact operating apparatus comprising: a generating unit configured to generate and update an image containing an image object, a projecting device configured to project the image in a predetermined display region within a vehicle compartment of the vehicle, the image object being projected at a predetermined position within the predetermined display region and being visible by an occupant present within the vehicle compartment of the vehicle; an operation detecting device configured to detect an operation site of the occupant positioned in a vicinity of the predetermined display region within the vehicle compartment; an operation determining unit configured to set an inputtable region that is invisible by the occupant within the predetermined display region, and determine, on a basis of a position or motion of the operation site of the occupant with respect to the inputtable region, whether the operation site of the occupant detected by the operation detecting device performs a non-contact operation against the image object, a motion detecting device configured to detect motion of the occupant caused by the motion of the vehicle based on motion at least one of an upper body, a shoulder or a head of the occupant; and a traveling state detecting device configured to detect weight acting on the vehicle or an acceleration of the vehicle, wherein when the weight is equal to or lower than a first predetermined value or the acceleration is equal to or lower than a second predetermined value, the operation determining unit is configured to i) set a position of the inputtable region at an initial position where the inputtable region overlaps with the image object, and ii) determine presence or absence of the operation against the image object by the operation site of the occupant, wherein when the weight is higher than the first predetermined value or the acceleration is higher than the second predetermined value, the operation determining unit is configured to i) move the position of the inputtable region from the initial position based on the detected motion of the occupant caused by the motion of the vehicle in response to receiving motion data from the motion detecting device indicative of the motion of the occupant caused by the motion of the vehicle, to suppress the motion of the occupant caused by the motion of the vehicle, and ii) determine presence or absence of the operation against the image object by the operation site of the occupant after the position of the inputtable region is moved, wherein the generating unit is configured to generate the image so that a position of the image object is fixed at the predetermined position, even though the position of the inputtable region is moved by the operation determining unit. ### Claim 2 (depends on 1) The non-contact operating apparatus for a vehicle according to claim 1, further comprising: a stimulation output device configured to output a tactile stimulation based on an ultrasound field to the operation site of the occupant; and a stimulation response output unit configured to cause the stimulation output device to output, on a basis of a determination result of the operation determining unit, the tactile stimulation as a response to the operation, to the operation site of the occupant performing the non-contact operation against the image object, wherein even in a case where the operation determining unit moves the position of the inputtable region on the basis of the detection result of the motion detecting device and determines the presence or the absence of the operation by the operation site of the occupant, the stimulation response output unit is configured to cause the stimulation output device to output the tactile stimulation to the position of the operation site of the occupant detected by the operation detecting device. ### Claim 3 (depends on 1) The non-contact operating apparatus for a vehicle according to claim 1, wherein the operation determining unit is configured to move, on the basis of the detection result of the motion detecting device, the position of the inputtable region to suppress the motion of the occupant caused by the motion of the vehicle when the weight is higher than the first predetermined value or the acceleration is higher than the second predetermined value. ### Claim 4 (depends on 2) The non-contact operating apparatus for a vehicle according to claim 2, wherein the operation determining unit is configured to move, on the basis of the detection result of the motion detecting device, the position of the inputtable region to suppress the motion of the occupant caused by the motion of the vehicle when the weight is higher than the first predetermined value or the acceleration is higher than the second predetermined value. ### Claim 5 (depends on 1) The non-contact operating apparatus for a vehicle according to claim 1, wherein the operation determining unit is configured to determine the presence or the absence of the operation by the occupant on a basis of motion of the occupant against the image object projected by the projecting device, and adjust, on the basis of the detection result of the motion detecting device, the detection result of the operation detecting device to suppress the motion of the occupant caused by the motion of the vehicle, and determine the presence or the absence of the operation by the operation site of the occupant. ### Claim 6 (depends on 2) The non-contact operating apparatus for a vehicle according to claim 2, wherein the operation determining unit is configured to determine the presence or the absence of the operation by the occupant on a basis of motion of the occupant against the image object projected by the projecting device, and adjust, on the basis of the detection result of the motion detecting device, the detection result of the operation detecting device to suppress the motion of the occupant caused by the motion of the vehicle, and determine the presence or the absence of the operation by the operation site of the occupant. ### Claim 7 (depends on 1) The non-contact operating apparatus for a vehicle according to claim 1, wherein the motion detecting device is configured to detect, on a basis of an image in which the occupant is captured, the motion of the occupant caused by the motion of the vehicle. ### Claim 8 (depends on 2) The non-contact operating apparatus for a vehicle according to claim 2, wherein the motion detecting device is configured to detect, on a basis of an image in which the occupant is captured, the motion of the occupant caused by the motion of the vehicle. ### Claim 9 (depends on 3) The non-contact operating apparatus for a vehicle according to claim 3, wherein the motion detecting device is configured to detect, on a basis of an image in which the occupant is captured, the motion of the occupant caused by the motion of the vehicle. ### Claim 10 (depends on 4) The non-contact operating apparatus for a vehicle according to claim 4, wherein the motion detecting device is configured to detect, on a basis of an image in which the occupant is captured, the motion of the occupant caused by the motion of the vehicle. ### Claim 11 (depends on 1) A vehicle, comprising: the non-contact operating apparatus according to claim 1; and two or more control devices coupled to the non-contact operating apparatus through an internal network, each of the two or more control devices being configured to control an operation of the vehicle, each of the two or more control devices being configured to acquire input information from the non-contact operating apparatus through the internal network, the input information being generated on a basis of the non-contact operation of the occupant against the image object in the image, the image being projected within the vehicle compartment by the non-contact operating apparatus. ### Claim 12 A non-contact operating apparatus for a vehicle, comprising: circuitry configured to control an operation of the non-contact operating apparatus; a projecting device configured to project an image containing an image object in a predetermined display region within a vehicle compartment of the vehicle, the image object being visible by an occupant present within the vehicle compartment of the vehicle; an operation detecting device configured to detect an operation site of the occupant positioned in a vicinity of the predetermined display region within the vehicle compartment; a motion detecting device configured to detect motion of the occupant caused by the motion of the vehicle based on motion at least one of an upper body, a shoulder or a head of the occupant; and a traveling state detecting device configured to detect weight acting on the vehicle or an acceleration of the vehicle, wherein the circuitry being configured to generate and update the image so that a position of the image object is fixed at a predetermined position within the predetermined display region, set an inputtable region that is invisible by the occupant within the predetermined display region, determine, on a basis of a position or motion of the operation site of the occupant with respect to the inputtable region, whether the operation site of the occupant detected by the operation detecting device performs a non-contact operation against the image object, when the weight is equal to or lower than a first predetermined value or the acceleration is equal to or lower than a second predetermined value, i) set a position of the inputtable region at an initial position where the inputtable region overlaps with the image object, and ii) determine presence or absence of the operation against the image object by the operation site of the occupant, and when the weight is higher than the first predetermined value or the acceleration is higher than the second predetermined value, i) move the position of the inputtable region from the initial position based on the detected motion of the occupant caused by the motion of the vehicle in response to receiving motion data from the motion detecting device indicative of the motion of the occupant caused by the motion of the vehicle, to suppress the motion of the occupant caused by the motion of the vehicle, ii) determine presence or absence of the operation against the image object by the operation site of the occupant after the position of the inputtable region is moved. ### Claim 13 (depends on 1) The non-contact operating apparatus for a vehicle according to claim 1, wherein the motion detecting device is configured to detect an acceleration in a front-back direction of the vehicle, wherein the operation determining unit is configured to move the position of the inputtable region along the front-back direction of the vehicle based on the detected acceleration in the front-back direction of the vehicle when the weight is higher than the first predetermined value or the acceleration is higher than the second predetermined value.