# US 2025/0148038 A1 · Pairwise labelling to capture subjective estimates

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148038 A1
- Document ID: US20250148038A1
- Status: Published application
- Kind: A1
- Application number: 18/941,732
- Filed: 2024-11-08
- Published: 2025-05-08
- Earliest claimed: 2023-11-08
- Assignees: Schlumberger Technology Corporation
- Applicants: Schlumberger Technology Corporation
- Inventors: Pontus Loviken
- Web page: https://patentgrep.com/patent/US20250148038A1

## CPC

- G06F 16/958: Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods of the present disclosure are configured to enable human annotators to quickly, and potentially in groups, annotate thousands of data points of content, so that each data point of content gets a value with respect to some attribute, so that data points of content showing more of the attribute have higher values than those showing less of the attribute. For example, a method includes presenting two content items of a plurality of content items via a user interface of a web-based application; receiving, via the user interface of a web-based application, an annotation relating to an indication of which of the two content items are associated with a criterion; and updating a score relating to the criterion for each of the plurality of content items based at last in part on the annotation.

## Claims

### Claim 1

A method, comprising:
presenting two content items of a plurality of content items via a user interface of a web-based application;
receiving, via the user interface of a web-based application, an annotation relating to an indication of which of the two content items are associated with a criterion; and
updating a score relating to the criterion for each of the plurality of content items based at last in part on the annotation.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of content items comprise images.

### Claim 3 (depends on 1)

The method of claim 1, wherein the plurality of content items comprise video.

### Claim 4 (depends on 1)

The method of claim 1, wherein the plurality of content items comprise audio.

### Claim 5 (depends on 1)

The method of claim 1, wherein the criterion relates to a smell of the plurality of content items, a taste of the plurality of content items, or a feel of the plurality of content items.

### Claim 6 (depends on 1)

The method of claim 1, comprising training a first artificial intelligence (AI) model to determine trends with a plurality of annotations of the plurality of content items associated with the criterion.

### Claim 7 (depends on 6)

The method of claim 6, comprising training a second AI model to utilize information learned by the first AI model to automatically, without human intervention, annotate the plurality of content items based on the learned information.

### Claim 8

A system, comprising:
a memory storing processor-executable instructions; and
one or more processors configured to execute the processor-executable instructions, wherein the processor-executable instructions, when executed by the one or more processors, cause the system to:
cause two content items of a plurality of content items to be presented via a user interface of a web-based application;
receive an annotation relating to an indication of which of the two content items are associated with a criterion; and
update a score relating to the criterion for each of the plurality of content items based at last in part on the annotation.

### Claim 9 (depends on 8)

The system of claim 8, wherein the plurality of content items comprise images.

### Claim 10 (depends on 8)

The system of claim 8, wherein the plurality of content items comprise video.

### Claim 11 (depends on 8)

The system of claim 8, wherein the plurality of content items comprise audio.

### Claim 12 (depends on 8)

The system of claim 8, wherein the criterion relates to a smell of the plurality of content items, a taste of the plurality of content items, or a feel of the plurality of content items.

### Claim 13 (depends on 8)

The system of claim 8, wherein the processor-executable instructions, when executed by the one or more processors, cause the system to train a first artificial intelligence (AI) model to determine trends with a plurality of annotations of the plurality of content items associated with the criterion.

### Claim 14 (depends on 13)

The system of claim 13, wherein the processor-executable instructions, when executed by the one or more processors, cause the system to train a second AI model to utilize information learned by the first AI model to automatically, without human intervention, annotate the plurality of content items based on the learned information.

### Claim 15

One or more non-transitory computer-readable memory media, comprising:
processor-executable instructions that, when executed by one or more processors, cause the one or more processors to:
cause two content items of a plurality of content items to be presented via a user interface of a web-based application;
receive an annotation relating to an indication of which of the two content items are associated with a criterion; and
update a score relating to the criterion for each of the plurality of content items based at last in part on the annotation.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable memory media of claim 15, wherein the plurality of content items comprise images.

### Claim 17 (depends on 15)

The one or more non-transitory computer-readable memory media of claim 15, wherein the plurality of content items comprise video.

### Claim 18 (depends on 15)

The one or more non-transitory computer-readable memory media of claim 15, wherein the plurality of content items comprise audio.

### Claim 19 (depends on 15)

The one or more non-transitory computer-readable memory media of claim 15, wherein the criterion relates to a smell of the plurality of content items, a taste of the plurality of content items, or a feel of the plurality of content items.

### Claim 20 (depends on 15)

The one or more non-transitory computer-readable memory media of claim 15, wherein the processor-executable instructions, when executed by the one or more processors, cause the one or more processors to train a first artificial intelligence (AI) model to determine trends with a plurality of annotations of the plurality of content items associated with the criterion.

### Claim 21 (depends on 20)

The one or more non-transitory computer-readable memory media of claim 20, wherein the processor-executable instructions, when executed by the one or more processors, cause the one or more processors to train a second AI model to utilize information learned by the first AI model to automatically, without human intervention, annotate the plurality of content items based on the learned information.
