# US 2024/0256874 A1 · Systems and methods for hybrid optimization training of multinomial logit models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0256874 A1
- Document ID: US20240256874A1
- Status: Published application
- Kind: A1
- Application number: 18/104,021
- Filed: 2023-01-31
- Published: 2024-08-01
- Earliest claimed: 2023-01-31
- Assignees: Walmart Apollo, LLC
- Applicants: Walmart Apollo, LLC
- Inventors: Reza Yousefi Maragheh; Ramin Giahi; Aysenur Inan; Hyun Duk Cho; Kaushiki Nag; Sushant Kumar; Kannan Achan
- Web page: https://patentgrep.com/patent/US20240256874A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for hybrid optimization of training ranking models is disclosed. A training dataset including a plurality of anchor items, a plurality of recommended item sets, and ground truth data is obtained from a database. A base machine learning model including a step function configured to determine a relevance score is iteratively trained to generate a trained ranking model. The plurality of anchor items and the plurality of recommended item sets are provided as an input to the base machine learning model and the ground truth is provided as a target output. The step function is trained using an adaptive step size according to a first order Barzilai-Borwein (BB) method and a line search method. The trained ranking model is stored in non-transitory memory.

## Claims

### Claim 1

A system, comprising:
a non-transitory memory configured to store a training dataset comprising a plurality of anchor items, a plurality of recommended item sets, and ground truth data;
a processor communicatively coupled to the non-transitory memory, wherein the processor is configured to read a set of instructions to:
obtain, from the non-transitory memory, the training dataset;
obtain a base machine learning model including a step function configured to determine a relevance score;
iteratively train the base machine learning model to generate a trained ranking model, wherein the plurality of anchor items and the plurality of recommended item sets are provided as an input to the base machine learning model and the ground truth data is provided as a target output, and wherein the step function is trained using an adaptive step size according to a first order Barzilai-Borwein (BB) process and a line search process; and
store the trained ranking model in the non-transitory memory.

### Claim 2 (depends on 1)

The system of claim 1, wherein the base machine learning model includes a gradient descent model.

### Claim 3 (depends on 2)

The system of claim 2, wherein iteratively training the base machine learning model includes implementing a plurality of parallel gradient descent steps, wherein each of the plurality of parallel gradient descent steps operates on a chunk of the training dataset.

### Claim 4 (depends on 1)

The system of claim 1, wherein the base machine learning model comprises a multinomial logit model, and wherein the trained ranking model comprises a multinomial logit choice model.

### Claim 5 (depends on 1)

The system of claim 1, wherein the adaptive step size is determined based on a difference between optimized parameter points and a gradient information vector at a prior iteration and a current iteration.

### Claim 6 (depends on 1)

The system of claim 1, wherein the relevance score is determined based on a set of features related to a recommended item in one of the plurality of recommended item sets, a set of features related to a corresponding anchor item in the plurality of anchor items, and features related to a seller of the recommended item.

### Claim 7 (depends on 1)

The system of claim 1, wherein the base machine learning model is trained to optimize a log-likelihood of an interaction with a first recommended item in one of the plurality of recommended item sets given a related anchor item in the plurality of anchor items.

### Claim 8 (depends on 1)

The system of claim 1, wherein iteratively training the base machine learning model includes converting the training dataset to a n-dimensional matrix.

### Claim 9 (depends on 1)

The system of claim 1, wherein the BB process and line search process are performed iteratively with different frequencies.

### Claim 10

A computer-implemented method, comprising:
obtaining, from a first database, a training dataset comprising a plurality of anchor items, a plurality of recommended item sets, and ground truth data;
obtaining a base machine learning model including a step function configured to determine a relevance score;
training the base machine learning model to generate a trained ranking model, wherein the plurality of anchor items and the plurality of recommended item sets are provided as an input to the base machine learning model and the ground truth data is provided as a target output, and wherein the step function is trained using an adaptive step size according to a first order Barzilai-Borwein (BB) process and a line search function; and
storing the trained ranking model in a second database.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein iteratively training the base machine learning model includes implementing a plurality of parallel gradient descent steps, wherein each of the plurality of parallel gradient descent steps operates on a chunk of the training dataset.

### Claim 12 (depends on 10)

The computer-implemented method of claim 10, wherein the base machine learning model comprises a multinomial logit model, and wherein the trained ranking model comprises a multinomial logit choice model.

### Claim 13 (depends on 10)

The computer-implemented method of claim 10, wherein the adaptive step size is determined based on a difference between optimized parameter points and a gradient information vector at a prior iteration and a current iteration.

### Claim 14 (depends on 10)

The computer-implemented method of claim 10, wherein the relevance score is determined based on a set of features related to a recommended item in one of the plurality of recommended item sets, a set of features related to a corresponding anchor item in the plurality of anchor items, and features related to a seller of the recommended item.

### Claim 15 (depends on 10)

The computer-implemented method of claim 10, wherein the base machine learning model is trained to optimize a log-likelihood of an interaction with a first recommended item in one of the plurality of recommended item sets given a related anchor item in the plurality of anchor items.

### Claim 16 (depends on 10)

The computer-implemented method of claim 10, wherein iteratively training the base machine learning model includes converting the training dataset to a n-dimensional matrix.

### Claim 17

A non-transitory computer-readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:
obtaining, from a first database, a training dataset comprising a plurality of anchor items, a plurality of recommended item sets, and ground truth data;
obtaining a base machine learning model including a step function configured to determine a relevance score;
training the base machine learning model to generate a trained ranking model, wherein the plurality of anchor items and the plurality of recommended item sets are provided as an input to the base machine learning model and the ground truth data is provided as a target output, and wherein the step function is trained using an adaptive step size according to a first order Barzilai-Borwein (BB) process and a line search function; and
storing the trained ranking model in a second database.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the base machine learning model comprises a multinomial logit model, and wherein the trained ranking model comprises a multinomial logit choice model, and wherein iteratively training the base machine learning model includes implementing a plurality of parallel gradient descent steps, wherein each of the plurality of parallel gradient descent steps operates on a chunk of the training dataset.

### Claim 19 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the adaptive step size is determined based on a difference between optimized parameter points and a gradient information vector at a prior iteration and a current iteration and wherein the relevance score is determined based on a set of features related to a recommended item in one of the plurality of recommended item sets, a set of features related to a corresponding anchor item in the plurality of anchor items, and features related to a seller of the recommended item.

### Claim 20 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein iteratively training the base machine learning model includes converting the training dataset to a n-dimensional matrix.
