# US 12,010,267 B2 · Systems and methods for providing multi-modal interaction via user equipment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,267 B2 - Document ID: US12010267B2 - Status: Granted - Kind: B2 - Application number: 18/150,972 - Filed: 2023-01-06 - Published: 2023-05-18 - Granted: 2024-06-11 - Earliest claimed: 2020-06-15 - Assignees: Verizon Patent and Licensing Inc. - Applicants: Verizon Patent and Licensing Inc. - Inventors: Goldstein D. Daniel; Priyanka Muthukrishnan; Jaidev Haridoss; Marina Sruthi M - Web page: https://patentgrep.com/patent/US12010267B2 ## CPC - H04M 3/493: Interactive information services, e.g. directory enquiries {; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals} - H04M 7/0012: {Details of application programming interfaces [API] for telephone networks; Arrangements which combine a telephonic communication equipment and a computer, i.e. computer telephony integration [CPI] arrangements} - H04M 3/5166: {in combination with interactive voice response systems or voice portals, e.g. as front-ends} - H04M 3/5183: {Call or contact centers with computer-telephony arrangements} - H04M 3/527: Centralised call answering arrangements not requiring operator intervention - H04M 2203/254: where the visual mode comprises menus - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract A system described herein may provide a technique for a multi-modal interaction experience with a User Equipment (“UE”) via multiple concurrent presentation modes. Such multiple modes may include a voice call with an interactive voice response (“IVR”) system, as well as the simultaneous presentation of information and/or selectable options at a UE associated engaged in the voice call with the IVR system. The information may be presented via a graphical user interface at the UE that is engaged in the call with the IVR system. In this manner, the UE may provide a multi-modal experience to a user, in which the user can simultaneously interact with the IVR system via the call, and can also receive or provide further interactions related to call via the GUI. ## Claims ### Claim 1 A device, comprising: one or more processors configured to: output, to a user equipment (UE) that is engaged in a call, a first instruction to present a set of visual information during the call; determine that confirmation of presentation of the set of visual information has not been received from the UE; and output, to the UE and based on the determination that the confirmation has not been received from the UE, a second instruction to obtain a particular application, wherein the UE obtains the particular application based on the second instruction and presents the set of visual information via the particular application during the call. ### Claim 2 (depends on 1) The device of claim 1, wherein the call is between the UE and another device, wherein the one or more processors are further configured to: receive user input provided via the application from the UE; and provide an indication of the user input to the other device. ### Claim 3 (depends on 2) The device of claim 2, wherein the other device includes an interactive voice-based system. ### Claim 4 (depends on 2) The device of claim 2, wherein the user input includes a selection of a particular menu item presented by the particular application, wherein the menu item corresponds to audible information provided by the other device during the call. ### Claim 5 (depends on 4) The device of claim 4, wherein the other device modifies audible presentation of information, via the call, based on the selection of the particular menu item. ### Claim 6 (depends on 1) The device of claim 1, wherein the call is between the UE and another device, wherein the one or more processors are further configured to: receive, from the other device and during the call, an indication of one or more interactions between the other device and the UE; and provide updated information to the UE based on the one or more interactions between the other device and the UE, wherein providing the updated information to the UE causes the UE to present updated visual information that is based on the provided updated information. ### Claim 7 (depends on 1) The device of claim 1, wherein the set of visual information includes one or more selectable options, wherein when presenting the visual information, the UE presents the one or more selectable options during the call. ### Claim 8 A non-transitory computer-readable medium storing a plurality of processor-executable instructions to: output, to a user equipment (UE) that is engaged in a call, a first instruction to present a set of visual information during the call; determine that confirmation of presentation of the set of visual information has not been received from the UE; and output, to the UE and based on the determination that the confirmation has not been received from the UE, a second instruction to obtain a particular application, wherein the UE obtains the particular application based on the second instruction and presents the set of visual information via the particular application during the call. ### Claim 9 (depends on 8) The non-transitory computer-readable medium of claim 8, wherein the call is between the UE and another device, wherein the plurality of processor-executable instructions further include processor-executable instructions to: receive user input provided via the application from the UE; and provide an indication of the user input to the other device. ### Claim 10 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the other device includes an interactive voice-based system. ### Claim 11 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the user input includes a selection of a particular menu item presented by the particular application, wherein the menu item corresponds to audible information provided by the other device during the call. ### Claim 12 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein the other device modifies audible presentation of information, via the call, based on the selection of the particular menu item. ### Claim 13 (depends on 8) The non-transitory computer-readable medium of claim 8, wherein the call is between the UE and another device, wherein the plurality of processor-executable instructions further include processor-executable instructions to: receive, from the other device and during the call, an indication of one or more interactions between the other device and the UE; and provide updated information to the UE based on the one or more interactions between the other device and the UE, wherein providing the updated information to the UE causes the UE to present updated visual information that is based on the provided updated information. ### Claim 14 (depends on 8) The non-transitory computer-readable medium of claim 8, wherein the set of visual information includes one or more selectable options, wherein when presenting the visual information, the UE presents the one or more selectable options during the call. ### Claim 15 A method, comprising: outputting, to a user equipment (UE) that is engaged in a call, a first instruction to present a set of visual information during the call; determining that confirmation of presentation of the set of visual information has not been received from the UE; and outputting, to the UE and based on the determination that the confirmation has not been received from the UE, a second instruction to obtain a particular application, wherein the UE obtains the particular application based on the second instruction and presents the set of visual information via the particular application during the call. ### Claim 16 (depends on 15) The method of claim 15, wherein the call is between the UE and another device, the method further comprising: receive user input provided via the application from the UE; and provide an indication of the user input to the other device. ### Claim 17 (depends on 16) The method of claim 16, wherein the user input includes a selection of a particular menu item presented by the particular application, wherein the menu item corresponds to audible information provided by the other device during the call. ### Claim 18 (depends on 17) The method of claim 17, wherein the other device modifies audible presentation of information, via the call, based on the selection of the particular menu item. ### Claim 19 (depends on 15) The method of claim 15, wherein the call is between the UE and another device, the method further comprising: receive, from the other device and during the call, an indication of one or more interactions between the other device and the UE; and provide updated information to the UE based on the one or more interactions between the other device and the UE, wherein providing the updated information to the UE causes the UE to present updated visual information that is based on the provided updated information. ### Claim 20 (depends on 15) The method of claim 15, wherein the set of visual information includes one or more selectable options, wherein when presenting the visual information, the UE presents the one or more selectable options during the call.