# US 2022/0277282 A1 · Artificial Intelligence (AI) Order Taking System Enabling Agents to Make Corrections Via Point of Sale (POS) Devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277282 A1
- Document ID: US20220277282A1
- Status: Published application
- Kind: A1
- Application number: 17/746,931
- Filed: 2022-05-17
- Published: 2022-09-01
- Earliest claimed: 2021-02-24
- Assignees: ConverseNowAI
- Applicants: ConverseNowAI
- Inventors: Jon Dorch; Zubair Talib; Ruchi Bafna; Akshaya Labh Kayastha; Yuganeshan AJ; Vinay Kumar Shukla; Rahul Aggarwal
- Web page: https://patentgrep.com/patent/US20220277282A1

## CPC

- G06Q 20/203: {Inventory monitoring}
- G06Q 30/0631: {Recommending goods or services}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/23: Clustering techniques
- G06F 18/24: Classification techniques
- G06N 3/045: Combinations of networks
- G06N 20/00: Machine learning
- G06Q 20/202: {Interconnection or interaction of plural electronic cash registers [ECR] or to host computer, e.g. network details, transfer of information from host to ECR or from ECR to ECR}
- G06Q 30/0601: Electronic shopping [e-shopping]
- G06V 40/172: {Classification, e.g. identification}
- G06N 5/041: {Abduction}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A software agent, comprising a machine learning algorithm trained to engage in a conversation with a customer to take an order, receives an utterance from a customer. The utterance is converted to text and an analysis of the text performed. If the software agent determines, based on the analysis, that the software agent is untrained to respond to the text, the software agent establishes a connection to a point-of-sale device associated with a human agent. The human agent may perform a modification (e.g., an edit to the text, a modification to a cart, or provide input) to a modifiable portion displayed by the point-of-sale device. The software agent, based at least in part on the modification, resumes the conversation with the customer. The human agent does not directly interact with the customer during the conversation between the software agent and the customer.

## Claims

### Claim 1

A method comprising:
receiving, by a software agent executing on a server, an utterance from a customer, the software agent comprising a machine learning algorithm trained to engage in a conversation with the customer to take an order;
converting the utterance to text;
determining that the software agent is untrained to respond to the text;
establishing a connection between:
the server hosting the software agent; and
a point-of-sale device that is associated with a human agent;
receiving, from the human agent, a modification to a modifiable portion displayed by the point-of-sale device; and
resuming, by the software agent and based at least in part on the modification, the conversation with the customer, wherein the human agent does not directly interact with the customer during the conversation between the software agent and the customer.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
providing, to the software agent, the modification to the modifiable portion.

### Claim 3 (depends on 1)

The method of claim 1, wherein resuming, by the software agent and based at least in part on the modification, the conversation with the customer comprises:
verbally providing, by the software agent, a response to the customer based at least in part on the modification.

### Claim 4 (depends on 1)

The method of claim 1, wherein the modification to the modifiable portion displayed by the point-of-sale device comprises:
one or more edits, by the human agent, to the text;
one or more modifications, by the human agent, to a cart associated with the customer;
text input provided by the human agent in an input area for the software agent to provide as a verbal response to the customer; or
any combination thereof.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
receiving, by the software agent, an additional utterance from the customer;
converting the additional utterance to additional text;
providing, to the software agent, the additional text;
modifying, by the software agent, the order based at least in part on the additional text to create a modified order, the modifying comprising:
adding an item to the order;
modifying a previously added item in the order; or
any combination thereof; and
verbally providing, by the software agent, a response to the customer based at least in part on the additional text.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
displaying, by the point-of-sale device, the additional text; and
displaying, by the point-of-sale device, the modified order.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
determining, by the software agent, that the customer has completed the order;
processing order data associated with the order; and
initiating fulfillment of the order.

### Claim 8

A server comprising:
one or more processors; and
one or more non-transitory computer readable media storing instructions executable by the one or more processors to perform operations comprising:
receiving, by a software agent, an utterance from a customer, the software agent comprising a machine learning algorithm trained to engage in a conversation with the customer to take an order;
converting the utterance to text;
determining that the software agent is untrained to respond to the text;
establishing a connection between:
the server hosting the software agent; and
a point-of-sale device that is associated with a human agent;
receiving, from the human agent, a modification to a modifiable portion displayed by the point-of-sale device; and
resuming, by the software agent and based at least in part on the modification, the conversation with the customer, wherein the human agent does not directly interact with the customer during the conversation between the software agent and the customer.

### Claim 9 (depends on 8)

The server of claim 8, further comprising:
providing, to the software agent, the modification to the modifiable portion.

### Claim 10 (depends on 8)

The server of claim 8, wherein resuming, by the software agent and based at least in part on the modification, the conversation with the customer comprises:
verbally providing, by the software agent, a response to the customer based at least in part on the modification.

### Claim 11 (depends on 8)

The server of claim 8, wherein the modification to the modifiable portion displayed by the second POS device comprises:
one or more edits, by the human agent, to the text;
one or more modifications, by the human agent, to a cart associated with the customer;
text input provided by the human agent in an input area for the software agent to provide as a verbal response to the customer; or
any combination thereof.

### Claim 12 (depends on 8)

The server of claim 8, further comprising:
receiving, by the software agent, an additional utterance from the customer;
converting the additional utterance to additional text;
providing, to the software agent, the additional text;
modifying, by the software agent, the order based at least in part on the additional text to create a modified order, the modifying comprising:
adding an item to the order;
modifying a previously added item in the order; or
any combination thereof; and
verbally providing, by the software agent, a response to the customer based at least in part on the additional text.

### Claim 13 (depends on 12)

The server of claim 12, further comprising:
displaying, by the point-of-sale device, the additional text; and
displaying, by the point-of-sale device, the modified order.

### Claim 14 (depends on 8)

The server of claim 8, further comprising:
determining, by the software agent, that the customer has completed the order;
processing order data associated with the order; and
initiating fulfillment of the order.

### Claim 15

A memory device to store instructions executable by one or more processors to perform operations comprising:
receiving, by a software agent, an utterance from a customer, the software agent comprising a machine learning algorithm trained to engage in a conversation with the customer to take an order;
converting the utterance to text;
determining that the software agent is untrained to respond to the text;
establishing a connection between:
the server hosting the software agent; and
a point-of-sale device that is associated with a human agent;
receiving, from the human agent, a modification to a modifiable portion displayed by the point-of-sale device; and
resuming, by the software agent and based at least in part on the modification, the conversation with the customer, wherein the human agent does not directly interact with the customer during the conversation between the software agent and the customer.

### Claim 16 (depends on 15)

The memory device of claim 15, further comprising:
providing, to the software agent, the modification to the modifiable portion.

### Claim 17 (depends on 16)

The memory device of claim 16, wherein resuming, by the software agent and based at least in part on the modification, the conversation with the customer comprises:
verbally providing, by the software agent, a response to the customer based at least in part on the modification.

### Claim 18 (depends on 15)

The memory device of claim 15, wherein the modification to the modifiable portion displayed by the second POS device comprises:
one or more edits, by the human agent, to the text;
one or more modifications, by the human agent, to a cart associated with the customer;
text input provided by the human agent in an input area for the software agent to provide as a verbal response to the customer; or
any combination thereof.

### Claim 19 (depends on 15)

The memory device of claim 15, further comprising:
receiving, by the software agent, an additional utterance from the customer;
converting the additional utterance to additional text;
providing, to the software agent, the additional text;
modifying, by the software agent, the order based at least in part on the additional text to create a modified order, the modifying comprising:
adding an item to the order;
modifying a previously added item in the order; or
any combination thereof; and
verbally providing, by the software agent, a response to the customer based at least in part on the additional text.

### Claim 20 (depends on 15)

The memory device of claim 15, further comprising:
determining, by the software agent, that the customer has completed the order;
processing order data associated with the order; and
initiating fulfillment of the order.
