# US 10,120,876 B2 · Unguided application crawling architecture > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,876 B2 - Document ID: US10120876B2 - Status: Granted - Kind: B2 - Application number: 14/849,540 - Filed: 2015-09-09 - Published: 2016-11-17 - Granted: 2018-11-06 - Earliest claimed: 2015-05-13 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Kalyan Desineni; Manikandan Sankaranarasimhan; Brahm Singh; Sudhir Mohan - Web page: https://patentgrep.com/patent/US10120876B2 ## CPC - G06F 16/1748: {De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641)} - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06F 16/24556: {Aggregation; Duplicate elimination} - G06F 16/24578: {using ranking} - G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses} - G06F 16/9024: {Graphs; Linked lists (G06F16/9027 takes precedence)} - G06F 16/951: Indexing; Web crawling techniques - G06F 16/9535: Search customisation based on user profiles and personalisation - G06F 16/9538: Presentation of query results - G06F 16/954: Navigation, e.g. using categorised browsing - G06F 16/9558: {Details of hyperlinks; Management of linked annotations} - H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system for automated acquisition of content from an application includes a link tracking module that controls an instance of the application executing within an emulator. For a selected state, the link tracking module controls the executing application instance to navigate to the selected state and identifies a first set of application states reachable by user interface interaction. A state storage module stores records based on the first set. A first state record includes content of a first state of the first set and a unique identifier that uniquely identifies the first state. The unique identifier indicates a path followed within the executing application instance from a default state to the first state, including corresponding user interface interaction. A scraper module, for each of the records in the state storage module, navigates to the state specified by the unique identifier using the indicated path and extracts text from the state. ## Claims ### Claim 1 An apparatus for automated acquisition of content from an application and improved searching of the content in response to a query of a user device, the apparatus comprising: at least one processor, the at least one processor configured to track links, the tracking of links including: controlling an executing instance of the application; and for a selected state of the application; controlling the executing application instance to navigate to the selected state, and identifying a first set of application states reachable from the selected state, each of the first set of application states being reachable via a respective user interface interaction with the selected state, wherein the at least one processor is further configured to store records in a state storage based on the first set of application states, a first state record including: (i) a representation of content of a first state of the first set of application states, and (ii) a unique identifier that uniquely identifies the first state within the records of the state storage, the unique identifier of the first state indicating a path followed within the executing application instance from a default state of the application to the first state, and the path including the user interface interaction corresponding to the first state, wherein the at least one processor is further configured to scrape records, including, for each of the stored records, extract text and metadata from the state, information based on the extracted text and metadata being stored in a data store, and wherein the at least one processor is further configured to provide at least one record in response to the query from the user device. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the link tracking by the at least one processor comprises executing the instance of the application within an emulator. ### Claim 3 (depends on 1) The apparatus of claim 1 wherein the unique identifier of the first state is a uniform resource identifier (URI). ### Claim 4 (depends on 1) The apparatus of claim 1 wherein the unique identifier of the first state is based on a concatenation, in order, of each user interface interaction triggered when navigating from the default state to the first state. ### Claim 5 (depends on 1) The apparatus of claim 1 wherein the default state is a home state of the application and wherein, upon execution, the application presents the home state to a user of the application. ### Claim 6 (depends on 1) The apparatus of claim 1 wherein the representation of the content of the first state requires less storage space than does the content of the first state. ### Claim 7 (depends on 6) The apparatus of claim 6 wherein the representation of the content of the first state is based on a calculated hash of the content of the first state. ### Claim 8 (depends on 1) The apparatus of claim 1 wherein: the first state record includes a second unique identifier that uniquely identifies the first state within the records of the state storage; and the unique identifier of the first state indicates a second path followed within the application from the default state to the first state. ### Claim 9 (depends on 8) The apparatus of claim 8 wherein the at least one processor is further configured to detect duplicate content, the detection comprising: determining whether the path and the second path both arrive at the first state, and in response to determining that the path and the second path both arrive at the first state, adding the second unique identifier to the first state record instead of creating a second state record. ### Claim 10 (depends on 8) The apparatus of claim 8, wherein the at least one processor is further configured to select a path, the selection of a path including selecting a single unique identifier for each of the records for use by the at least one processor when scraping records. ### Claim 11 (depends on 10) The apparatus of claim 10 wherein the at least one processor is further configured to, when scraping records, for each of the records in the state storage, (i) navigate to the state specified by the selected single unique identifier using the path indicated by the selected single unique identifier and (ii) extract text and metadata from the state. ### Claim 12 (depends on 10) The apparatus of claim 10 wherein the selection of a path comprises selecting, for each of the records, the single unique identifier that identifies a fastest path to the respective state. ### Claim 13 (depends on 1) The apparatus of claim 1, wherein the link tracking by the at least one processor comprises: identifying when an application programming interface call is available for navigating to the first state; and storing a second unique identifier in the first state record, wherein the second unique identifier indicates the application programming interface call. ### Claim 14 (depends on 13) The apparatus of claim 13 wherein the link tracking by the at least one processor comprises, for a second state reachable from the first state via a first user interface interaction: storing a second state record in the state storage including a unique identifier based on the application programming interface call and the first user interface interaction; and adding a second unique identifier to the second state record, wherein the second unique identifier indicates the first user interface interaction and user interface interactions followed from the default state to the first state. ### Claim 15 (depends on 1) The apparatus of claim 1, wherein the providing of at least one record comprises: in response to the query from the user device, selecting records from the data store to generate a set of records by forming a consideration set of records; processing the consideration set of records by assigning a score to each record of the consideration set of records; and generating results by responding to the user device with a subset of the consideration set of records, wherein the subset is selected based on the assigned scores, and wherein the subset identifies application states of applications that are relevant to the query. ### Claim 16 A method for automated acquisition of content from an application and provision of content for searching by a user device, the method comprising: executing, using at least one processor of an apparatus for automated content acquisition and provision, an instance of the application; for a selected state of the application; (i) controlling, using the at least one processor, the executing application instance to navigate to the selected state, and (ii) identifying, using the at least one processor, a first set of application states reachable from the selected state, each of the first set of application states being reachable via a respective user interface interaction with the selected state; storing records in a state storage based on the first set of application states, wherein a first state record includes: (i) a representation of content of a first state of the first set of application states, and (ii) a unique identifier that uniquely identifies the first state within the stored records, the unique identifier of the first state indicating a path followed within the executing application instance from a default state of the application to the first state, and the path including the user interface interaction corresponding to the first state; for each of the stored records, extracting, using the at least one processor, text and metadata from the state; and storing information based on the extracted text and metadata in a data store. ### Claim 17 (depends on 16) The method of claim 16 wherein the instance of the application is executed within an emulator. ### Claim 18 (depends on 16) The method of claim 16 wherein the unique identifier of the first state is a uniform resource identifier (URI). ### Claim 19 (depends on 16) The method of claim 16 further comprising forming the unique identifier of the first state by concatenating, in order, each user interface interaction triggered when navigating from the default state to the first state. ### Claim 20 (depends on 16) The method of claim 16 wherein the default state is a home state of the application and wherein, upon execution, the application presents the home state to a user of the application. ### Claim 21 (depends on 16) The method of claim 16 wherein the representation of the content of the first state requires less storage space than does the content of the first state. ### Claim 22 (depends on 21) The method of claim 21 further comprising generating the representation of the content of the first state by calculating a hash of the content of the first state. ### Claim 23 (depends on 16) The method of claim 16 wherein: the first state record includes a second unique identifier that uniquely identifies the first state within the stored records; and the unique identifier of the first state indicates a second path followed within the application from the default state to the first state. ### Claim 24 (depends on 23) The method of claim 23 further comprising: determining whether the path and the second path both arrive at the first state; and in response to determining that the path and the second path both arrive at the first state, adding the second unique identifier to the first state record instead of creating a second state record. ### Claim 25 (depends on 23) The method of claim 23 further comprising selecting a single unique identifier for each of the records for use in the extracting. ### Claim 26 (depends on 25) The method of claim 25 further comprising, for each of the stored records, navigating to the state specified by the selected single unique identifier using the path indicated by the selected single unique identifier before extracting text and metadata from the state. ### Claim 27 (depends on 25) The method of claim 25 wherein the selecting includes, for each of the records, selecting the single unique identifier that identifies a fastest path to the respective state. ### Claim 28 (depends on 16) The method of claim 16 further comprising: identifying when an application programming interface call is available for navigating to the first state; and storing a second unique identifier in the first state record, wherein the second unique identifier indicates the application programming interface call. ### Claim 29 (depends on 28) The method of claim 28 further comprising, for a second state reachable from the first state via a first user interface interaction: storing a second state record including a unique identifier based on the application programming interface call and the first user interface interaction; and adding a second unique identifier to the second state record, wherein the second unique identifier indicates the first user interface interaction and user interface interactions followed from the default state to the first state. ### Claim 30 (depends on 16) The method of claim 16, further comprising: in response to receiving a query from a user device, selecting records from the data store to form a consideration set of records; assigning a score to each record of the consideration set of records; and responding to the user device with a subset of the consideration set of records, wherein the subset is selected based on the assigned scores, and wherein the subset identifies application states of applications that are relevant to the query. ### Claim 31 (depends on 16) A non-transitory computer-readable medium storing processor-executable instructions configured to perform the method of claim 16.