# US 10,503,715 B1 · Applications search using programmatic application state indexing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,503,715 B1 - Document ID: US10503715B1 - Status: Granted - Kind: B1 - Application number: 15/184,640 - Filed: 2016-06-16 - Granted: 2019-12-10 - Earliest claimed: 2016-06-16 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Daniel Massaguer; Derek Steven Gebhard - Web page: https://patentgrep.com/patent/US10503715B1 ## CPC - G06F 16/2228: {Indexing structures} - G06F 16/313: {Selection or weighting of terms for indexing} - G06F 16/248: Presentation of query results - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Technologies are disclosed for enabling application search through the use of programmatic application state indexing. The disclosed technologies enable applications to be systematically explored using a program to identify the various states of the application. For example, UI displays provided by an application can be explored and text or other content can be captured from each UI display. The text or other content can then be indexed and utilized to search applications and to navigate directly to specific application states. ## Claims ### Claim 1 An apparatus, comprising: a processor; and a non-transitory computer-readable storage medium having instructions stored thereupon which, when executed by the processor, cause the apparatus to: programmatically simulate user interaction with user interface (UI) elements displayed by an application to identify a plurality of application states, the plurality of application states corresponding to unique screen displays generated by the application; identify content associated with the plurality of application states; determine, based in part on the content, at least one of a popularity or usage associated with individual ones of the plurality of application states; generate an application search index that provides a mapping between the content and the plurality of application states; merge the application search index into a local application store search index containing application search indexes for one or more other applications; receive a search query comprising one or more search terms; responsive to receiving the search query, search the local application store search index for the search terms to identify application states of the plurality of application states mapped to the search terms in the search query, wherein identifying the application states is based in part on at least one of the popularity or the usage associated with individual ones of the application states; and provide search results responsive to the search query that identify applications that provide the application states mapped to the search terms in the search query. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the non-transitory computer-readable storage medium has further instructions stored thereupon to programmatically identify the UI elements displayed by the application. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the non-transitory computer-readable storage medium has further instructions stored thereupon to provide the application search index to a computing device for searching of the plurality of application states on the computing device. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein programmatically simulating user interaction with UI elements displayed by the application to identify the plurality of application states comprises: systematically selecting UI elements on the unique display screens generated by the application to identify the unique display screens generated by the application and links between the unique display screens; and generating a UI map identifying the unique display screens and the links between the unique display screens. ### Claim 5 (depends on 4) The apparatus of claim 4, wherein the non-transitory computer-readable storage medium has further instructions stored thereupon to: generate a state graph from the UI map, the state graph comprising data defining the plurality of application states and links between the plurality of application states; and generate the application search index from the state graph. ### Claim 6 A computer-implemented method, comprising: receiving, at a computing device, an application and an associated application search index mapping content to a plurality of application states provided by the application, wherein the application search index is generated by programmatically simulating user interaction with user interface (UI) elements displayed by the application to identify the plurality of application states; determining, based in part on the content, at least one of a popularity or usage associated with individual ones of the plurality of application states; merging, at the computing device, the application search index with a local search index, the local search index containing application search indexes for one or more other applications installed on the computing device; providing, at the computing device, a UI for receiving a search query comprising one or more search terms; responsive to receiving the search query at the computing device, searching the local search index for the search terms to identify application states of the plurality of application states mapped to the search terms in the search query, wherein identifying the application states is based in part on at least one of the popularity or the usage associated with individual ones of the application states; and presenting, at the computing device, search results responsive to the search query that identify applications that provide the application states mapped to the search terms in the search query. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, further comprising: receiving a selection of a search result; and causing an application associated with the selected search result to be configured on the computing device in an application state associated with the search result. ### Claim 8 (depends on 6) The computer-implemented method of claim 6, wherein the application search index is generated by an application store system responsive to submission of the application for inclusion in an application store provided by the application store system. ### Claim 9 (depends on 6) The computer-implemented method of claim 6, wherein the UI elements displayed by the application are programmatically identified. ### Claim 10 (depends on 6) The computer-implemented method of claim 6, further comprising, responsive to receiving the search query at the computing device, causing a search to be made of an application store search index for the search terms, the application store search index comprising the application search index and application search indexes for a plurality of applications available from an application store. ### Claim 11 (depends on 6) The computer-implemented method of claim 6, wherein programmatically simulating user interaction with UI elements displayed by an application to identify the application states comprises: systematically selecting UI elements on the unique display screens generated by the application to identify the unique display screens generated by the application and links between the unique display screens; and generating a UI map identifying the unique display screens and the links between the unique display screens. ### Claim 12 (depends on 11) The computer-implemented method of claim 11, wherein a state graph is generated from the UI map, the state graph comprising data defining the plurality of application states and links between the plurality of application states, and wherein the application search index is generated from the state graph. ### Claim 13 A non-transitory computer-readable storage medium having instructions stored thereupon which are executable by a processor and which, when executed, cause the processor to: programmatically identify a plurality of application states provided by an application by interacting with a user interface (UI) provided by the application; identify content associated with the plurality of application states; determine, based in part on the content, at least one of a popularity or usage associated with individual ones of application states of the plurality of application states; generate an application search index that provides a mapping between the content and the plurality of application states; and merge the application search index into a local application store search index containing application search indexes for one or more other applications. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein programmatically identifying the plurality of application states provided by the application by interacting with a user interface (UI) provided by the application comprises programmatically simulating user interaction with UI elements displayed by the application to identify the plurality of application states, the plurality of application states corresponding to unique screen displays generated by the application. ### Claim 15 (depends on 14) The non-transitory computer-readable storage medium of claim 14, having further instructions stored thereupon to programmatically identify the UI elements displayed by the application. ### Claim 16 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein programmatically simulating user interaction with UI elements displayed by the application to identify the plurality of application states comprises: systematically selecting UI elements on the unique display screens generated by the application to identify the unique display screens generated by the application and links between the unique display screens; and generating a UI map identifying the unique display screens and the links between the unique display screens. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein a state graph is generated from the UI map, the state graph comprising data defining the plurality of application states and links between the plurality of application states, and wherein the application search index is generated from the state graph. ### Claim 18 (depends on 13) The non-transitory computer-readable storage medium of claim 13, having further instructions stored thereupon to: receive a search query comprising one or more search terms; responsive to receiving the search query, search the local application store search index for the search terms to identify application states of the plurality of application states mapped to the search terms in the search query, wherein identifying the application states is based in part on at least one of the popularity or the usage associated with individual ones of the application states; and provide search results responsive to the search query that identify applications that provide the application states mapped to the search terms in the search query. ### Claim 19 (depends on 13) The non-transitory computer-readable storage medium of claim 13, having further instructions stored thereupon to provide the application search index to a computing device for searching of the plurality of application states on the computing device. ### Claim 20 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the application search index is generated by an application store system responsive to submission of the application for inclusion in an application store provided by the application store system.