# US 2019/0171867 A1 · Systems and methods for providing augmented reality overlays

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171867 A1
- Document ID: US20190171867A1
- Status: Published application
- Kind: A1
- Application number: 16/265,927
- Filed: 2019-02-01
- Published: 2019-06-06
- Earliest claimed: 2016-12-30
- Assignees: Facebook, Inc.
- Applicants: Facebook, Inc.
- Inventors: John Samuel Barnett; Dantley Davis; Congxi Lu; Jonathan Morton; Peter Vajda; Joshua Charles Harris
- Web page: https://patentgrep.com/patent/US20190171867A1

## CPC

- G06T 11/00: Two-dimensional [2D] image generation
- G06N 20/00: Machine learning
- G06Q 10/48: using social graphs
- G06T 5/70: Denoising; Smoothing
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 20/20: in augmented reality scenes
- G06V 40/172: {Classification, e.g. identification}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, methods, and non-transitory computer-readable media can identify one or more objects depicted in a camera view of a camera application displayed on a display of a user device. An augmented reality overlay is determined based on the one or more objects identified in the camera view. The camera view is modified based on the augmented reality overlay.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving, by a computing system, user location information from a first user device;
selecting, by the computing system, a first object recognition machine learning model from a plurality of object recognition machine learning models based on the user location information; and
providing, by the computing system, the first object recognition machine learning model to the first user device for identifying one or more objects depicted through the first user device.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein a plurality of machine learning models associated with the user location information are downloaded to the first user device, the plurality of machine learning models including the first object recognition machine learning model.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein an augmented reality overlay is identified based on the one or more objects, the augmented reality overlay being used to modify a camera interface on the first user device.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the augmented reality overlay comprises context information associated with an object of the one or more objects depicted through the first user device.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein the context information comprises social networking system information associated with the object.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein the social networking system information comprises one or more content items associated with the object posted by users of a social networking system.

### Claim 7 (depends on 3)

The computer-implemented method of claim 3, wherein the augmented reality overlay comprises an interactive augmented reality object that appears to interact with an object of the one or more objects depicted through the first user device.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the interactive augmented reality object is selected based on an association with the object.

### Claim 9 (depends on 3)

The computer-implemented method of claim 3, wherein the augmented reality overlay is identified based on a user selection selecting the augmented reality overlay from a plurality of augmented reality overlays.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving updated user location information from the first user device;
selecting a second object recognition machine learning model from a plurality of object recognition machine learning models based on the updated user location information; and
providing the second object recognition machine learning model to the first user device for identifying objects depicted through the first user device.

### Claim 11

A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
receiving user location information from a first user device;
selecting a first object recognition machine learning model from a plurality of object recognition machine learning models based on the user location information; and
providing the first object recognition machine learning model to the first user device for identifying one or more objects depicted through the first user device.

### Claim 12 (depends on 11)

The system of claim 11, wherein a plurality of machine learning models associated with the user location information are downloaded to the first user device, the plurality of machine learning models including the first object recognition machine learning model.

### Claim 13 (depends on 11)

The system of claim 11, wherein an augmented reality overlay is identified based on the one or more objects, the augmented reality overlay being used to modify a camera interface on the first user device.

### Claim 14 (depends on 13)

The system of claim 13, wherein the augmented reality overlay comprises context information associated with an object of the one or more objects depicted through the first user device.

### Claim 15 (depends on 14)

The system of claim 14, wherein the context information comprises social networking system information associated with the object.

### Claim 16

A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
receiving user location information from a first user device;
selecting a first object recognition machine learning model from a plurality of object recognition machine learning models based on the user location information; and
providing the first object recognition machine learning model to the first user device for identifying one or more objects depicted through the first user device.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein a plurality of machine learning models associated with the user location information are downloaded to the first user device, the plurality of machine learning models including the first object recognition machine learning model.

### Claim 18 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein an augmented reality overlay is identified based on the one or more objects, the augmented reality overlay being used to modify a camera interface on the first user device.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein the augmented reality overlay comprises context information associated with an object of the one or more objects depicted through the first user device.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the context information comprises social networking system information associated with the object.
