# US 10,311,611 B2 · Aggregating tags in images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,611 B2
- Document ID: US10311611B2
- Status: Granted
- Kind: B2
- Application number: 15/145,055
- Filed: 2016-05-03
- Published: 2016-08-25
- Granted: 2019-06-04
- Earliest claimed: 2013-05-15
- Assignees: Facebook, Inc.
- Applicants: Facebook, Inc.
- Inventors: Dirk John Stoop; Liu Liu
- Web page: https://patentgrep.com/patent/US10311611B2

## CPC

- G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information}
- G06F 16/248: Presentation of query results
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06Q 10/42: Determination of affinities or common interests between users
- G06Q 10/48: using social graphs
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 11/60: Creating or editing images; Combining images with text
- H04L 43/045: for graphical visualisation of monitoring data
- H04L 51/52: for supporting social networking services
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

In one embodiment, a method includes rendering an image comprising a plurality of tags, wherein each tag is associated with a particular location within the image, and wherein at least two tags of the plurality of tags are aggregated into an aggregated tag, calculating for each tag the distance of the location of the tag from the location of every other tag of the image responsive to the first user changing a zoom level for the image, and updating the aggregated tag based on the calculated distance of the locations of the tags to each other such that if the change in zoom level is zooming in, then the updated aggregated tag is aggregated with at least one fewer tag, and if the change in zoom level is zooming out, then the updated aggregated tag is aggregated with at least one more tag.

## Claims

### Claim 1

A method comprising, by a client system of a first user:
rendering, by the client system, an image comprising a plurality of tags, wherein each tag is associated with a particular location within the image and comprising a name string corresponding to the tag, and wherein at least two tags of the plurality of tags within a threshold distance of each other are aggregated into an aggregated tag, wherein the aggregated tag comprises at least one name string corresponding to one of the plurality of tags and a counter of the name strings corresponding to the other tags aggregated into the aggregated tag;
detecting, by the client system, an action of the first user changing a zoom level for the image;
responsive to the first user changing a zoom level for the image, calculating for each tag the distance of the location of the tag from the location of every other tag of the image; and
if the change in zoom level is zooming in, then updating the aggregated tag based on the calculated distance of the locations of the tags to each other, wherein the updated aggregated tag is aggregated with at least one fewer tag of the plurality of tags, and wherein the counter is correspondingly decremented by at least one; and
if the change in zoom level is zooming out, then updating the aggregated tag based on the calculated distance of the locations of the tags to each other, wherein the updated aggregated tag is aggregated with at least one more tag of the plurality of tags, and wherein the counter is correspondingly incremented by at least one.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of tags correspond to a plurality of users, respectively.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
accessing, from a social-networking system, a social-graph affinity for each user corresponding to a tag aggregated into the updated aggregated tag; and
rendering, by the client system, the image with a tag-label for the updated aggregated tag, wherein the tag-label comprises a name of the user having the strongest social-graph affinity with respect to the first user and a counter indicating a number of users corresponding to the updated aggregated tag.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
accessing a social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between them, the nodes comprising:
a first node corresponding to the first user; and
a plurality of second nodes corresponding to a plurality of objects associated with the social-networking system, respectively.

### Claim 5 (depends on 4)

The method of claim 4, wherein one or more users of the plurality of users corresponds to one or more second nodes, respectively.

### Claim 6 (depends on 5)

The method of claim 5, wherein the social-graph affinity for each user is based on a degree of separation between the second node corresponding to the user and the first node.

### Claim 7 (depends on 1)

The method of claim 1, wherein the plurality of tags correspond to a plurality of objects, respectively.

### Claim 8 (depends on 2)

The method of claim 2, wherein the plurality of users are users of an online social network.

### Claim 9 (depends on 2)

The method of claim 2, wherein the plurality of users are users of a communication network.

### Claim 10 (depends on 1)

The method of claim 1, further comprising rendering the image with a tag-label for each tag of the plurality of tags not associated with the updated aggregated tag.

### Claim 11 (depends on 1)

The method of claim 1, wherein updating the aggregated tag based on the calculated distance of the locations of the tags to each other comprises aggregating a set of tags into the updated aggregated tag if the calculated distance between each tag of the set of tags is within the threshold distance.

### Claim 12 (depends on 1)

The method of claim 1, wherein the calculated distance of the locations of the tags to each other is based on a point within the image for each tag.

### Claim 13 (depends on 1)

The method of claim 1, wherein the threshold distance is a threshold fraction of one dimension of the image.

### Claim 14 (depends on 1)

The method of claim 1, wherein the threshold distance is a threshold fraction of one dimension of a viewport of the client system.

### Claim 15 (depends on 1)

The method of claim 1, wherein the threshold distance is a threshold distance across a viewport of the client system.

### Claim 16 (depends on 1)

The method of claim 1, wherein the calculated distance of the locations of the tags to each other is based on an area within the image for each tag.

### Claim 17 (depends on 1)

The method of claim 1, wherein the threshold distance is a threshold percentage overlap of areas associated with each tag.

### Claim 18 (depends on 1)

The method of claim 1, further comprising:
receiving, at the client system, an input from the first user requesting to view the image; and
sending, by the client system to a social-networking system, a request to view the image.

### Claim 19 (depends on 1)

The method of claim 1, wherein the particular location within the image is a point associated with a pixel of the image.

### Claim 20

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
render, by the client system, an image comprising a plurality of tags, wherein each tag is associated with a particular location within the image and comprising a name string corresponding to the tag, and wherein at least two tags of the plurality of tags within a threshold distance of each other are aggregated into an aggregated tag, wherein the aggregated tag comprises at least one name string corresponding to one of the plurality of tags and a counter of the name strings corresponding to the other tags aggregated into the aggregated tag;
detect, by the client system, an action of the first user changing a zoom level for the image;
responsive to the first user changing a zoom level for the image, calculate for each tag the distance of the location of the tag from the location of every other tag of the image; and
if the change in zoom level is zooming in, update the aggregated tag based on the calculated distance of the locations of the tags to each other, wherein the updated aggregated tag is aggregated with at least one fewer tag of the plurality of tags, and wherein the counter is correspondingly decremented by at least one; and
if the change in zoom level is zooming out, update the aggregated tag based on the calculated distance of the locations of the tags to each other, wherein the updated aggregated tag is aggregated with at least one more tag of the plurality of tags, and wherein the counter is correspondingly incremented by at least one.

### Claim 21

A client system of a first user comprising: one or more processors; and a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
render, by the client system, an image comprising a plurality of tags, wherein each tag is associated with a particular location within the image and comprising a name string corresponding to the tag, and wherein at least two tags of the plurality of tags within a threshold distance of each other are aggregated into an aggregated tag, wherein the aggregated tag comprises at least one name string corresponding to one of the plurality of tags and a counter of the name strings corresponding to the other tags aggregated into the aggregated tag;
detect, by the client system, an action of the first user changing a zoom level for the image;
responsive to the first user changing a zoom level for the image, calculate for each tag the distance of the location of the tag from the location of every other tag of the image; and
if the change in zoom level is zooming in, update the aggregated tag based on the calculated distance of the locations of the tags to each other, wherein the updated aggregated tag is aggregated with at least one fewer tag of the plurality of tags, and wherein the counter is correspondingly decremented by at least one; and
if the change in zoom level is zooming out, update the aggregated tag based on the calculated distance of the locations of the tags to each other, wherein the updated aggregated tag is aggregated with at least one more tag of the plurality of tags, and wherein the counter is correspondingly incremented by at least one.

### Claim 22 (depends on 21)

The system of claim 21, wherein the plurality of tags correspond to a plurality of users, respectively.

### Claim 23 (depends on 22)

The system of claim 22, wherein the processors are further operable when executing the instructions to:
access, from a social-networking system, a social-graph affinity for each user corresponding to a tag aggregated into the updated aggregated tag; and
render, by the client system, the image with a tag-label for the updated aggregated tag, wherein the tag-label comprises a name of the user having the strongest social-graph affinity with respect to the first user and a counter indicating a number of users corresponding to the updated aggregated tag.

### Claim 24 (depends on 23)

The system of claim 23, wherein the processors are further operable when executing the instructions to:
access a social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between them, the nodes comprising:
a first node corresponding to the first user; and
a plurality of second nodes corresponding to a plurality of objects associated with the social-networking system, respectively.

### Claim 25 (depends on 24)

The system of claim 24, wherein one or more users of the plurality of users corresponds to one or more second nodes, respectively.

### Claim 26 (depends on 25)

The system of claim 25, wherein the social-graph affinity for each user is based on a degree of separation between the second node corresponding to the user and the first node.

### Claim 27 (depends on 22)

The system of claim 22, wherein the plurality of users are users of an online social network.

### Claim 28 (depends on 22)

The system of claim 22, wherein the plurality of users are users of a communication network.

### Claim 29 (depends on 21)

The system of claim 21, wherein the plurality of tags correspond to a plurality of objects, respectively.

### Claim 30 (depends on 21)

The system of claim 21, wherein the processors are further operable when executing the instructions to render the image with a tag-label for each tag of the plurality of tags not associated with the updated aggregated tag.

### Claim 31 (depends on 21)

The system of claim 21, wherein updating the aggregated tag based on the calculated distance of the locations of the tags to each other comprises aggregating a set of tags into the updated aggregated tag if the calculated distance between each tag of the set of tags is within the threshold distance.

### Claim 32 (depends on 21)

The system of claim 21, wherein the calculated distance of the locations of the tags to each other is based on a point within the image for each tag.

### Claim 33 (depends on 21)

The system of claim 21, wherein the threshold distance is a threshold fraction of one dimension of the image.

### Claim 34 (depends on 21)

The system of claim 21, wherein the threshold distance is a threshold fraction of one dimension of a viewport of the client system.

### Claim 35 (depends on 21)

The system of claim 21, wherein the threshold distance is a threshold distance across a viewport of the client system.

### Claim 36 (depends on 21)

The system of claim 21, wherein the calculated distance of the locations of the tags to each other is based on an area within the image for each tag.

### Claim 37 (depends on 21)

The system of claim 21, wherein the threshold distance is a threshold percentage overlap of areas associated with each tag.

### Claim 38 (depends on 21)

The system of claim 21, wherein the processors are further operable when executing the instructions to:
receive, at the client system, an input from the first user requesting to view the image; and
send, by the client system to a social-networking system, a request to view the image.

### Claim 39 (depends on 21)

The system of claim 21, wherein the particular location within the image is a point associated with a pixel of the image.