Methods and system for image-based analysis for intelligent item identification and utilization

A system associated with a payment service, the system comprising:

• one or more processors;
• and
• non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
• receiving, via an image sensor of a device on which a payment application associated with the payment service is installed, image data corresponding to an image that includes text, the text representing a list of items;
• preprocessing the image data, wherein preprocessing of the image data comprises one or more of:
• re-sampling to confirm that an image coordinate system is accurate,
• reducing image sensor noise,
• enhancing contrast, or
• conducting scale space representation of the image data to enhance image structures relative to locale scales;
• responsive at least in part to preprocessing the image data, identifying, via application of a computer vision model to the image data, a first portion of the image data corresponding to the list of items, wherein identifying the first portion comprises parsing the image data into the first portion and a second portion, wherein the second portion does not include any items of the list of items, and wherein the computer vision model is trained to identify at least one of one or more strings of text or one or more textual characteristics;
• determining, using optical text recognition on the first portion of the image data, one or more first characters corresponding to a first item of the list of items and one or more second characters corresponding to a second item of the list of items, wherein the optical text recognition is based at least in part on pixel values;
• receiving, from point-of-sale (POS) devices of a plurality of merchants associated with the payment service, transaction data associated with a plurality of transactions of the plurality of merchants, wherein a first transaction of the plurality of transactions comprises a purchase of the first item and a second transaction of the plurality of transactions comprises a purchase of the second item;
• based at least in part on the transaction data, processing respective payments for the plurality of transactions;
• determining, utilizing the transaction data and a machine-trained model, a first merchant offering corresponding to the first item from a first merchant of the plurality of merchants;
• determining, utilizing the transaction data and the machine-trained model, a second merchant offering corresponding to the second item from a second merchant of the plurality of merchants;
• receiving, via the payment application, an indication of user input to one or more user interfaces of the payment application, the user input comprising a request to purchase the first merchant offering and the second merchant offering;
• facilitating, at a first time and utilizing the payment service, purchase of:
• the first item associated with the first merchant offering from the first merchant;
• and
• the second item associated with the second merchant offering from the second merchant;
• and
• withdrawing, from a stored balance account associated with the device, an amount equal to an installment of an installment plan at a second time after the first time.

A computer-implemented method implemented at least in part by one or more servers of a payment service, comprising:

• receiving, by one or more processors of the one or more servers, an image sensor of a device on which a payment application associated with the payment service is stored, image data corresponding to an image that includes text, the text representing a list of items;
• preprocessing, by the one or more processors, the image data, wherein preprocessing of the image data comprises one or more of:
• re-sampling to confirm that an image coordinate system is spatially correct,
• reducing image sensor noise,
• enhancing contrast, or
• conducting scale space representation of the image data to enhance image structures relative to locale scales;
• responsive at least in part to preprocessing the image data, identifying, by the one or more processors via application of a computer vision model to the image data, a first portion of the image data corresponding to the list of items, wherein identifying the first portion comprises parsing the image data into the first portion and a second portion, wherein the second portion does not include any items of the list of items, and wherein the computer vision model is trained to identify at least one of one or more strings of text or one or more textual characteristics;
• determining, by the one or more processors using optical text recognition on the first portion of the image data, one or more first characters corresponding to a first item of the list of items and one or more second characters corresponding to a second item of the list of items, wherein the optical text recognition is based at least in part on pixel values;
• receiving, by the one or more processors from point-of-sale (POS) devices of a plurality of merchants associated with the payment service, transaction data associated with a plurality of transactions of the plurality of merchants, wherein a first transaction of the plurality of transactions comprises a purchase of the first item and a second transaction of the plurality of transactions comprises a purchase of the second item;
• based at least in part on the transaction data, processing, by the one or more processors, respective payments for the plurality of transactions;
• determining, by the one or more processors and utilizing the transaction data, a first merchant offering corresponding to the first item from a first merchant of the plurality of merchants;
• determining, by the one or more processors and utilizing the transaction data, a second merchant offering corresponding to the second item from a second merchant of the plurality of merchants;
• receiving, by the one or more processors and via the payment application, an indication of user input to one or more user interfaces of the payment application, the user input comprising a request to purchase the first merchant offering and the second merchant offering;
• and
• facilitating, by the one or more processors and utilizing the payment service, purchase of:
• the first item associated with the first merchant offering from the first merchant;
• and
• the second item associated with the second merchant offering from the second merchant.