US 11,637,796 B2Granted

Image matching server network implementing a score between a server and an image store

AssigneeZebrafish Labs, Inc.

InventorsChristopher Zacharias, Jeremy Larkin, David Birdsong

Life of this patent

Estimating the term…

Loading patent…

Abstract

Technology is disclosed for feeding source images from image stores of client systems into an image processing system using image feeding servers (“the technology”). The technology includes multiple image feeding servers that can feed a source image to the image processing system, using which the image processing system can generate processed images to be served to end users. The image feeding servers can be installed at various locations, e.g., geographically spread, and more proximate to where the source images are stored. An image feeding server obtains the source image from the image store in original size, downsizes the source image, and transmits the downsized source image to the image processing system. The technology selects an image feeding server based on a score of the image feeding server. The score can be determined as a function of one or more image feeding server parameters, e.g., latency, workload, or computing resource.

Claims28

Dependent claims

A method comprising:

receiving, at an edge server of an image processing system, a first request from a client system for serving a first image to a user device, wherein the image processing system is structured as a multi-tier architecture, the multi-tier architecture including:
an edge server tier including a plurality of edge servers that includes the edge server;
a shield server tier including a shield server configured to receive a plurality of requests from the plurality of edge servers of the edge server tier;
a load balancing server tier including a load balancing server;
an image processing tier including an image processing server;
and
a source image tier including a source image server configured to obtain images from a plurality of image feeding servers;
determining the first image is not stored at the edge server tier;
in response to determining the first image is not stored at the edge server tier, transmitting the first request from the edge server tier to the shield server tier;
determining the first image is not stored at the shield server tier;
in response to determining the first image is not stored at the shield server tier, transmitting the first request to the load balancing server tier;
adding, by the load balancing server, the first request to a queue for processing, wherein processing the first request includes producing, by the image processing server of the image processing tier, the first image by processing a source image;
determining the source image is not stored at the image processing tier;
in response to determining that the source image is not stored at the image processing tier, transmitting a request from the image processing tier to the source image tier to obtain the source image;
determining the source image is not stored at the source image tier;
and
in response to determining the source image is not stored at the source image tier:
receiving, at the source image server, an intermediate image from a first image feeding server of the plurality of image feeding servers, wherein the receiving the intermediate image includes:
transmitting, by the source image server, a second request for response to each of the plurality of image feeding servers,
receiving, by the source image server, a response from each server of a subset of the plurality of image feeding servers for performing a downsizing task on the source image, the response including a corresponding score indicative of a match of a corresponding server of the plurality of image feeding servers to perform the downsizing task on the source image, and
sending, by the source image server, a message to the first image feeding server to perform the downsizing task, the downsizing task producing the intermediate image from the source image;
processing, by the image processing server, the intermediate image to generate the first image;
and
transmitting, by the edge server, the first image to the client system for further transmission to the user device.

21 more elements

A non-transitory computer-readable storage medium storing computer-readable instructions, comprising:

instructions for receiving, at an edge server of an image processing system, a first request from a client system for serving a first image to a user device, wherein the image processing system is structured as a multi-tier architecture, the multi-tier architecture including:
an edge server tier including a plurality of edge servers that includes the edge server;
a shield server tier including a shield server configured to receive a plurality of requests from the plurality of edge servers of the edge server tier;
a load balancing server tier including a load balancing server;
an image processing tier including an image processing server;
and
a source image tier including a source image server configured to obtain images from a plurality of image feeding servers;
instructions for determining the first image is not stored at the edge server tier;
instructions for transmitting, in response to determining the first image is not stored at the edge server tier, the first request from the edge server tier to the shield server tier;
instructions for determining the first image is not stored at the shield server tier;
instructions for transmitting, in response to determining the first image is not stored at the shield server tier, the first request to the load balancing server tier;
instructions for adding, by the load balancing server, the first request to a queue for processing, wherein processing the first request includes producing, by the image processing server of the image processing tier, the first image by processing a source image;
instructions for determining the source image is not stored at the image processing tier;
instructions for transmitting, in response to determining that the source image is not stored at the image processing tier, a request from the image processing tier to the source image tier to obtain the source image;
instructions for determining the source image is not stored at the source image tier;
and
instructions for performing, in response to determining the source image is not stored at the source image tier:
receiving, at the source image server, an intermediate image from a first image feeding server of the plurality of image feeding servers, wherein the receiving the intermediate image includes:
transmitting, by the source image server, a second request for response to each of the plurality of image feeding servers, and
receiving, by the source image server, a response from each server of a subset of the plurality of image feeding servers for performing a downsizing task on a source image, the response including a corresponding score indicative of a match of a corresponding server of the subset of the plurality of image feeding servers to perform the downsizing task on the source image;
sending, by the source image server, a message to the first image feeding server to perform the downsizing task, the downsizing task producing the intermediate image from the source image;
processing, by the image processing server, the intermediate image to generate the first image;
and
transmitting, by the edge server, the first image to the client system for further transmission to the user device.

21 more elements

An apparatus comprising:

a processor;
and
a memory storing instructions, execution of which by the processor causes the apparatus to:
receive, at an edge server of image processing system, a first request from a client system for serving a first image to a user device, wherein the image processing system is structured as a multi-tier architecture, the multi-tier architecture including:
an edge server tier including a plurality of edge servers that includes the edge server;
a shield server tier including a shield server configured to receive a plurality of requests from the plurality of edge servers of the edge server tier;
a load balancing server tier including a load balancing server;
an image processing tier including an image processing server;
and
a source image tier including a source image server configured to obtain images from a plurality of image feeding servers;
determining the first image is not stored at the edge server tier;
in response to determining the first image is not stored at the edge server tier, transmit the first request from the edge server tier to the shield server tier;
determining the first image is not stored at the shield server tier;
in response to determining the first image is not stored at the shield server tier, transmit the first request to the load balancing server tier;
add, by the load balancing server, the first request to a queue for processing, wherein processing the first request includes producing, by the image processing server of the image processing tier, the first image by processing a source image;
determine the source image is not stored at the image processing tier;
in response to determining that the source image is not stored at the image processing tier, transmit a request from the image processing tier to the source image tier to obtain the source image;
determine the source image is not stored at the source image tier;
and
in response to determining the source image is not stored at the source image tier:
transmit, by the source image server, a second request for response to each of the plurality of image feeding servers;
receive, by the source image server, a response from each of the plurality of image feeding servers for performing a downsizing task, the response from each server of the plurality of image feeding servers including a corresponding score indicative of a match of a corresponding server of the plurality of image feeding servers to perform the downsizing task on a source image;
send, by the source image server, a message to a first image feeding server of the plurality of image feeding servers to perform the downsizing task, the downsizing task producing an intermediate image from the source image;
receive, by the source image server, an intermediate image from the first image feeding server;
process, by the image processing server, the intermediate image to generate the first image;
and
transmit, by the edge server, the first image to the client system for further transmission to the user device.

24 more elements

A method comprising:

receiving, at an edge server of an image processing system, a first request from a client system for serving a first image to a user device, wherein the image processing system is structured as a multi-tier architecture, the multi-tier architecture including:
an edge server tier including a plurality of edge servers that includes the edge server;
a shield server tier including a shield server configured to receive a plurality of requests from the plurality of edge servers of the edge server tier;
a load balancing server tier including a load balancing server;
an image processing tier including an image processing server;
and
a source image tier including a source image server configured to obtain images from a plurality of image feeding servers;
determining the first image is not stored at the edge server tier;
in response to determining the first image is not stored at the edge server tier, transmitting the first request from the edge server tier to the shield server tier;
determining the first image is not stored at the shield server tier;
in response to determining the first image is not stored at the shield server tier, transmitting the first request to the load balancing server tier;
adding, by the load balancing server, the first request to a queue for processing, wherein processing the first request includes producing, by the image processing server of the image processing tier, the first image by processing a source image;
determining the source image is not stored at the image processing tier;
in response to determining that the source image is not stored at the image processing tier, transmitting a request from the image processing tier to the source image tier to obtain the source image;
determining the source image is not stored at the source image tier;
and
in response to determining the source image is not stored at the source image tier:
selecting, by the source image server, a first image feeding server among the plurality of image feeding servers to perform a downsizing task;
sending, by the source image server, a message to the first image feeding server to perform the downsizing task on a source image;
receiving, at the source image server, an intermediate image from the first image feeding server, the intermediate image resulting from performing the downsizing task on the source image;
processing, by the image processing server, the intermediate image to generate the first image;
and
transmitting, by the edge server, the first image to the client system for further transmission to the user device.

20 more elements

A non-transitory computer-readable storage medium storing computer-readable instructions, comprising:

instructions for receiving, at an edge server of an image processing system, a first request from a client system for serving a first image to a user device, wherein the image processing system is structured as a multi-tier architecture, the multi-tier architecture including:
an edge server tier including a plurality of edge servers that includes the edge server;
a shield server tier including a shield server configured to receive a plurality of requests from the plurality of edge servers of the edge server tier;
a load balancing server tier including a load balancing server;
an image processing tier including an image processing server;
and
a source image tier including a source image server configured to obtain images from a plurality of image feeding servers;
instructions for determining the first image is not stored at the edge server tier;
instructions for transmitting, in response to determining the first image is not stored at the edge server tier, the first request from the edge server tier to the shield server tier;
instructions for determining the first image is not stored at the shield server tier;
instructions for transmitting, in response to determining the first image is not stored at the shield server tier, the first request to the load balancing server tier;
instructions for adding, by the load balancing server, the first request to a queue for processing, wherein processing the first request includes producing, by the image processing server of the image processing tier, the first image by processing a source image;
instructions for determining the source image is not stored at the image processing tier;
instructions for transmitting, in response to determining that the source image is not stored at the image processing tier, a request from the image processing tier to the source image tier to obtain the source image;
instructions for determining the source image is not stored at the source image tier;
and
instructions for performing, in response to determining the source image is not stored at the source image tier:
selecting, at the source image server, a first image feeding server of the plurality of image feeding servers to perform a task on a source image to generate an intermediate image,
said selecting based on a corresponding score indicative of a match of each server of the plurality of image feeding servers to perform the task on the source image;
sending, by the source image server, a message to the first image feeding server to perform the task;
receiving, at the source image server, the intermediate image from the first image feeding server;
processing, by the image processing server, the intermediate image to generate the first image;
and
transmitting, by the edge server, the first image to the client system for further transmission to the user device.

21 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/935,448Filed 22 Jul 2020 · continuation
Priority: 15 Jul 2014
Published: US 2020/0351229 A15 Nov 2020
Granted: 25 Apr 2023
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

H04L 51/10
Characterised by the inclusion of specific contents: multimedia information
H04L 67/101
Server selection for load balancing: based on network conditions
H04L 67/1023
Server selection for load balancing: based on a hash applied to IP addresses or costs
H04L 67/61
Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and…: taking…

Family

Application14/332,063Parent of this continuation · filed 15 Jul 2014
Application15/861,053Parent of this continuation · filed 3 Jan 2018
US 9,894,023 B2Parent of this continuation · filed 13 Feb 2018
US 2020/0351229 A1Pre-grant publication · published 5 Nov 2020
US 10,728,195 B2Parent of this continuation
US 2016/0021041 A1Published application
US 2018/0123990 A1Published application
US 11,637,796 B2This patent

From the US filing history. Foreign family members aren’t listed.