Detection of boilerplate content

A computer-implemented method comprising:

• receiving a resource from a server;
• selecting one or more nodes of a Document Object Model (DOM) tree for the resource,
• determining that the selected nodes exhibit one or more predefined traits that are characteristic of boilerplate content, wherein boilerplate content comprises content that is repeated in multiple resources of a particular web site,
• adjusting a boilerplate content score associated with the selected nodes responsive to determining that the selected nodes exhibit the predefined traits that are characteristic of boilerplate content, and
• providing information to a query recommendation engine, the information including textual content associated with the selected nodes, and the information identifying the adjusted boilerplate content score associated with the selected nodes.

A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by a client device cause the client device to perform operations comprising:

• receiving a resource from a server;
• selecting one or more nodes of a Document Object Model (DOM) tree for the resource;
• determining that the selected nodes exhibit one or more predefined traits that are characteristic of boilerplate content, wherein boilerplate content comprises content that is repeated in multiple resources of a particular web site;
• adjusting a boilerplate content score associated with the selected nodes responsive to determining that the selected nodes exhibit the predefined traits that are characteristic of boilerplate content;
• and
• providing information to a query recommendation engine, the information including textual content associated with the selected nodes, and the information identifying the adjusted boilerplate content score associated with the selected nodes.