# US 12,008,813 B2 · Photo instructions and confirmation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,813 B2 - Document ID: US12008813B2 - Status: Granted - Kind: B2 - Application number: 17/897,778 - Filed: 2022-08-29 - Published: 2024-02-29 - Granted: 2024-06-11 - Earliest claimed: 2022-08-29 - Assignees: United Parcel Service of America, Inc. - Applicants: United Parcel Service of America, Inc. - Inventors: David Carder; Ryan Fannon; William McCauley - Web page: https://patentgrep.com/patent/US12008813B2 ## CPC - G06V 20/50: Context or environment of the image - G06K 7/1443: {locating of the code in an image} - G06Q 10/0833: Tracking - G06Q 10/0836: Recipient pick-ups - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/74: Image or video pattern matching; Proximity measures in feature spaces - G06V 10/82: using neural networks - G06V 20/20: in augmented reality scenes - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - H04N 7/183: {for receiving images from a single remote source} - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30204: Marker - G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Computer program products, methods, systems, apparatus, and computing entities are provided for optimizing package delivery and confirmation are disclosed. In one embodiment, the system's processors accesses a first image of an assigned delivery location captured by a first computing device. The first image comprises an annotation indicating a specific location at the assigned delivery location for delivery of a parcel. At a display device of a second computing device, a real-time image stream comprising the annotation is captured by a camera of the second computing device is displayed. The annotation is overlaid within the real-time image stream based on comparing the first image received from the first computing device to the real-time image stream. The system further captures, from the real-time image stream, a second image when the parcel is positioned at the specific location based on the annotation overlaid within the real-time image stream. ## Claims ### Claim 1 A system for optimizing package delivery and confirmation, the system comprising: one or more processors; and one or more computer storage devices storing computer-useable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: accessing a first image of an assigned delivery location captured by a first computing device, wherein the first image of the assigned delivery location comprises an annotation indicating a specific location at the assigned delivery location for delivery of a parcel; displaying, at a display device of a second computing device, a real-time image stream captured by a camera of the second computing device, wherein the real-time image stream comprises the annotation, the annotation being overlaid within the real-time image stream based on comparing the first image received from the first computing device to the real-time image stream; and capturing, from the real-time image stream, a second image when the parcel is positioned at the specific location based on the annotation overlaid within the real-time image stream. ### Claim 2 (depends on 1) The system of claim 1, wherein the operations further comprise determining a location of the second computing device while the real-time image stream is being captured, wherein the second image is captured from the real-time data stream based on the location of the second computing device relative to the assigned delivery location. ### Claim 3 (depends on 1) The system of claim 1, wherein the operations further comprise: receiving a first location of the first computing device determined at the time the first image is captured; determining a second location of the second computing device while the real-time image stream is being captured; and providing, at the display device of the second computing device, guidance instructions indicating a direction from the second location of the second computing device to the first location of the first computing device. ### Claim 4 (depends on 1) The system of claim 1, wherein the operations further comprise identifying the parcel is positioned at the specific location, as indicated by the annotation, by identifying the parcel using an object recognition model, wherein the second image is automatically captured from the real-time image stream based on identifying the parcel is positioned at the specific location. ### Claim 5 (depends on 1) The system of claim 1, wherein the operations further comprise identifying, at the specific location, as indicated by the annotation, machine readable indicia affixed to the parcel, the machine readable indicia being usable by the second computing device to determine a delivery location for the parcel, wherein the second image is automatically captured from the real-time image stream based on the determined delivery location for the parcel matching the assigned delivery location associated with the first image. ### Claim 6 (depends on 1) The system of claim 1, wherein the first image received from the first computing device is compared to the real-time image stream using a convolutional neural network. ### Claim 7 (depends on 1) The system of claim 1, wherein the annotation comprises a boundary marker, the boundary marker delineating the specific location at the assigned delivery location, and wherein the second image is captured from the real-time image stream when the parcel is positioned within the boundary marker. ### Claim 8 (depends on 1) The system of claim 1, wherein the annotation comprises human readable text. ### Claim 9 A computer implemented method for optimizing package delivery and confirmation, the method comprising: receiving an annotation indicating a specific location at an assigned delivery location for delivery of a parcel, the annotation being determined from a first image of the assigned delivery location captured by a first computing device, wherein the first image of the assigned delivery location is annotated at the first computing device; displaying, at a display device of a second computing device, the annotation indicating the specific location, the annotation being overlaid on a real-time image stream captured by a camera of the second computing device, wherein the annotation indicates the assigned delivery location; and capturing, from the real-time image stream, a second image from the real-time image stream, the second image comprising the parcel positioned at the specific location based on the annotation overlaid within the real-time image stream. ### Claim 10 (depends on 9) The method of claim 9, further comprising determining a location of the second computing device while the real-time image stream is being captured, wherein the second image is captured from the real-time data stream based on the location of the second computing device relative to the assigned delivery location. ### Claim 11 (depends on 9) The method of claim 9, further comprising: receiving a first location of the first computing device determined at the time the first image is captured; determining a second location of the second computing device while the real-time image stream is being captured; and providing, at the display device of the second computing device, guidance instructions indicating a direction from the second location of the second computing device to the first location of the first computing device. ### Claim 12 (depends on 9) The method of claim 9, further comprising identifying the parcel is positioned at the specific location, as indicated by the annotation, by identifying the parcel using an object recognition model, wherein the second image is automatically captured from the real-time image stream based on identifying the parcel is positioned at the specific location. ### Claim 13 (depends on 12) The method of claim 12, wherein the object recognition model is a convolutional neural network trained to identify parcels. ### Claim 14 (depends on 9) The method of claim 9, further comprising identifying, at the specific location, as indicated by the annotation, machine readable indicia affixed to the parcel, the machine readable indicia being usable by the second computing device to determine a delivery location for the parcel, wherein the second image is automatically captured from the real-time image stream based on the determined delivery location for the parcel matching the assigned delivery location associated with the first image. ### Claim 15 (depends on 9) The method of claim 9, wherein the annotation comprises a boundary marker, the boundary marker delineating the specific location at the assigned delivery location, and wherein the second image is captured from the real-time image stream when the parcel is positioned within the boundary marker. ### Claim 16 (depends on 15) The method of claim 15, further comprising detecting the parcel is positioned within the boundary marker using an object recognition model, wherein the second image is automatically captured in response to detecting the parcel is positioned within the boundary marker. ### Claim 17 (depends on 9) The method of claim 9, wherein the annotation comprises human readable text. ### Claim 18 One or more non-transitory computer storage media storing computer readable instructions thereon that, when executed by a processor, cause the processor to perform operations of a method for optimizing package delivery and confirmation, the operations comprising: receiving a first image of an assigned delivery location captured by a first computing device, wherein the first image of the assigned delivery location comprises an annotation indicating a specific location at the assigned delivery location for delivery of a parcel; communicating, to a second computing device, the annotation indicating the specific location, wherein the annotation is overlaid on a real-time image stream captured by a camera of the second computing device and provided for display at a display device of the second computing device; and receiving, from the second computing device, a second image captured from the real-time image stream by the second computing device, the second image comprising the parcel positioned at the specific location based on the annotation overlaid within the real-time image stream. ### Claim 19 (depends on 18) The non-transitory computer storage media of claim 18, wherein the annotation comprises a boundary marker, the boundary marker delineating the specific location at the assigned delivery location, and wherein the second image is captured from the real-time image stream when the parcel is positioned within the boundary marker. ### Claim 20 (depends on 18) The non-transitory computer storage media of claim 18, wherein the annotation comprises human readable text.