Method and server for indexing web page in index

A method of indexing a web page in an index, the index being hosted in a datacenter system communicatively coupled with a triage server, the index for providing indications of possible search results to a search engine, the method executable by the triage server, the method comprising:

• identifying, by the triage server executing a crawler application, recent data associated with the web page to be indexed;
• generating, by the triage server executing a machine learning algorithm (MLA), an importance score for the web page based on the recent data associated with the web page, the importance score being indicative of usefulness of the web page as a search result, the MLA having been trained based on a training set comprising:
• (i) a training vector indicative of data associated with a training web page at a first moment in time after creation of content on the training web page;
• and
• (ii) a label indicative of usefulness of the training web page as a search result and based on data associated with the training web page at a second moment in time, the second moment in time being later in time than the first moment in time;
• selectively adding, by the triage server, the web page to one of (i) a real-time indexing queue and (ii) a postponed indexing queue based on a comparison between the importance score of the web page and a triage threshold such that:
• if the importance score is below the triage threshold, the web page is added to the postponed indexing queue for postponing the indexing of the web page;
• and
• if the importance score is above the triage threshold, the web page is added to the real-time indexing queue for indexing of the web page in real-time.