# US 11,906,317 B2 · Presenting location related information and implementing a task based on gaze, gesture, and voice detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,906,317 B2 - Document ID: US11906317B2 - Status: Granted - Kind: B2 - Application number: 17/342,504 - Filed: 2021-06-08 - Published: 2022-04-21 - Granted: 2024-02-20 - Earliest claimed: 2020-10-17 - Applicants: Chian Chiu Li - Inventors: Chian Chiu Li - Web page: https://patentgrep.com/patent/US11906317B2 ## CPC - G01C 21/3608: {using speech input, e.g. using speech recognition} - G01C 21/343: {Calculating itineraries (travelling salesman problem G06Q10/04; optimisation of routes G06Q10/047)} - G01C 21/3617: {using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - G06Q 30/0261: {based on user location} - G06Q 30/0601: Electronic shopping [e-shopping] - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - H04W 4/02: Services making use of location information - G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer - G06F 2221/2111: Location-sensitive, e.g. geographical location, GPS - G10L 15/1822: {Parsing for meaning understanding} - G10L 2015/223: {Execution procedure of a spoken command} - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract Systems and methods for presenting information and executing a task. In an aspect, when a user gazes at a display of a standby device, location related information is presented. In another aspect, when a user utters a voice command and gazes or gestures at a device, a task is executed. In another aspect, a voice input, a gesture, and user information are used to determine a destination for a trip or a product for a purchase. In another aspect, a voice input and user information are used to determine a destination when a user hails a vehicle. ## Claims ### Claim 1 A method performed for assisting a vehicle hailing process, comprising: 1) Monitoring a user input from a user using a plurality of sensors including a microphone at an electronic device; 2) Detecting a voice input from the user using the microphone and a voice recognition technique; 3) Detecting destination information from the voice input; 4) When it is detected that the voice input includes an address or a name of a location, determining a destination based on the address or the name of the location, and proceeding with the vehicle hailing process; and 5) when it is detected that the voice input includes one or more elements and that the voice input does not include an address or a place name, displaying a destination based on user information related to a previous ride of the user to a place and that the one or more elements match a product provided at the place, a service provided at the place, or a representation of the place, or partially match an address or a name of the place, and proceeding with the vehicle hailing process, wherein the user information related to the previous ride of the user is obtained after the user is recognized. ### Claim 2 (depends on 1) The method according to claim 1, wherein proceeding with the vehicle hailing process includes sending a vehicle hailing message to a service facility in response to activation of a graphic object configured on a screen of the electronic device. ### Claim 3 (depends on 2) The method according to claim 2 further including presenting a pickup time on the screen of the electronic device after receiving a message from the service facility. ### Claim 4 (depends on 1) The method according to claim 1 further including recognizing the user using a recognition mechanism. ### Claim 5 (depends on 4) The method according to claim 4 further including recognizing the user using a facial recognition method, a password, or a fingerprint method. ### Claim 6 (depends on 1) The method according to claim 1 further including presenting the destination on a screen of the electronic device after the destination is determined. ### Claim 7 (depends on 1) The method according to claim 1 further including when it is detected that the voice input includes the one or more elements and that the voice input does not include an address or a place name, determining a destination based on the one or more elements, the user information, and a current time. ### Claim 8 A method performed for assisting a vehicle hailing process, comprising: 1) Monitoring a user input from a user using a plurality of sensors at an electronic device; 2) Configuring an area on a screen of the electronic device for the user to input or submit destination information; 3) Configuring an interactive graphic object on the screen for the user to submit a vehicle hailing request; 4) When it is detected that an address or a name of a place is entered or submitted in the area on the screen, determining a destination based on the address or the name of the place, and proceeding with the vehicle hailing process; and 5) when it is detected that an address or a name of a place is not received and that the interactive graphic object is activated by the user, determining a destination based on a current time, user information related to a previous ride of the user, and a time of the previous ride, displaying the destination on the screen, and proceeding with the vehicle hailing process, wherein the user information is obtained after the user is recognized. ### Claim 9 (depends on 8) The method according to claim 8 further including detecting a voice input from the user using a voice recognition technique and determining a destination based on the voice input. ### Claim 10 (depends on 9) The method according to claim 9 further including determining a destination based on the voice input and the user information. ### Claim 11 (depends on 8) The method according to claim 8, wherein proceeding with the vehicle hailing process includes sending a vehicle hailing message to a service facility. ### Claim 12 (depends on 11) The method according to claim 11 further including presenting a pickup time on the screen of the electronic device after receiving a message from the service facility. ### Claim 13 (depends on 8) The method according to claim 8 further including recognizing the user using a recognition mechanism. ### Claim 14 (depends on 8) The method according to claim 8, wherein after it is detected that an address or a name of a place is not received and that the interactive graphic object is activated by the user, proceeding with the vehicle hailing process includes sending a vehicle hailing message to a service facility after it is detected that the interactive graphic object is activated by the user again. ### Claim 15 A method performed for assisting a vehicle hailing process, comprising: 1) Monitoring a user input from a user using a plurality of sensors including a microphone at an electronic device; 2) Configuring an interactive graphic object on a screen of the electronic device for the user to submit a vehicle hailing request; 3) In response to detecting a voice input from the user and that the voice input includes an address or a name of a place, determining a destination based on the address or the name of the place, and proceeding with the vehicle hailing process; and 4) in response to that no input with respect to destination information is received from the user and that the interactive graphic object is activated by the user, determining a destination based on a current time, user information related to a previous ride of the user, and a time of the previous ride, displaying the destination on the screen, and proceeding with the vehicle hailing process, wherein the user information is obtained after the user is recognized. ### Claim 16 (depends on 15) The method according to claim 15 further including configuring an area on the screen of the electronic device for the user to input or submit information of destination and determining a destination based on the information of destination. ### Claim 17 (depends on 15) The method according to claim 15 further including determining a destination based on the voice input and the user information. ### Claim 18 (depends on 15) The method according to claim 15, wherein proceeding with the vehicle hailing process includes sending a vehicle hailing message to a service facility. ### Claim 19 (depends on 18) The method according to claim 18 further including presenting a pickup time on the screen of the electronic device after receiving a message from the service facility. ### Claim 20 (depends on 15) The method according to claim 15, wherein after no input with respect to the destination information is received from the user and that the interactive graphic object is activated by the user, proceeding with the vehicle hailing process includes sending a vehicle hailing message to a service facility after it is detected that the interactive graphic object is activated by the user again.