# US 9,436,738 B2 · Mechanical Turk integrated IDE, systems and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,738 B2 - Document ID: US9436738B2 - Status: Granted - Kind: B2 - Application number: 13/866,444 - Filed: 2013-04-19 - Published: 2013-10-24 - Granted: 2016-09-06 - Earliest claimed: 2012-04-19 - Assignees: Nant Holdings IP, LLC - Applicants: Nant Holdings IP, LLC - Inventors: Farzad Ehsani; Silke Maren Witt-Ehsani; Demitrios Leo Master - Web page: https://patentgrep.com/patent/US9436738B2 ## CPC - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06F 8/41: Compilation - G06F 16/168: {Details of user interfaces specifically adapted to file systems, e.g. browsing and visualisation, 2d or 3d GUIs (query results presentation G06F16/156)} - G06F 16/24522: {Translation of natural language queries to structured queries} - G06F 16/2471: {Distributed queries} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A Mechanical Turk-Integrated Development Environment system is disclosed. An integrated development environment (IDE) can include one or more interfaces capable of communicating with a mechanical turk engine. As a developer creates applications within the IDE, the developer can use the IDE to submit one or more requests to the mechanical turk engine. The engine constructs a mechanical turk project based on the requests and provides project tasks to workers. The results of the tasks can then be compiled and integrated back into the developer's application via the IDE. An example use includes constructing large domain specific data sets that can be applied to spoken dialog interfaces. ## Claims ### Claim 1 An application development system comprising: an application developer engine including a computing device having a computer processor configured to execute stored software instructions; and a mechanical turk engine including a computing device having a computer processor configured to execute stored software instructions said mechanical turk engine communicatively coupled with the application developer engine and at least one member interface, wherein the mechanical turk engine is configured to: receive a mechanical turk request; convert the mechanical turk request into a mechanical turk project; receive mechanical turk project results from at least one member interface; validate the mechanical turk project results; convert mechanical turk project results data into a module formatted for compilation by the application developer engine; and return project results to the application developer engine. ### Claim 2 (depends on 1) The system of claim 1, wherein the mechanical turk engine comprises a computing device configured as an Integrated Development Environment (IDE). ### Claim 3 (depends on 1) The system of claim 1, wherein the application developer engine and the mechanical turk engine are communicatively coupled over a network. ### Claim 4 (depends on 3) The system of claim 3, wherein the mechanical turk engine comprises an entity unrelated to the application developer engine. ### Claim 5 (depends on 1) The system of claim 1, wherein the mechanical turk request comprises an automatically generated project definition. ### Claim 6 (depends on 1) The system of claim 1, wherein the project results comprise a serialized data stream. ### Claim 7 (depends on 1) The system of claim 1, wherein the project results comprise raw data. ### Claim 8 (depends on 1) The system of claim 1, wherein the project results comprise the domain module. ### Claim 9 (depends on 1) The system of claim 1, wherein the mechanical turk engine comprises the application module generator. ### Claim 10 (depends on 1) The system of claim 1, wherein the application developer engine comprises the application module generator. ### Claim 11 (depends on 1) The system of claim 1, wherein the application developer engine is further configured to generate an application comprising the project results. ### Claim 12 (depends on 11) The system of claim 11, wherein the application configures a computing device on which the application executes to obtain the project results. ### Claim 13 (depends on 12) The system of claim 12, wherein the computing device modifies its behavior according to the application integrating the project results. ### Claim 14 (depends on 1) The system of claim 1, wherein the project results comprise conceptual mappings. ### Claim 15 (depends on 14) The system of claim 14, wherein the conceptual mappings represent a meaning obtained from at least one modality of communication. ### Claim 16 (depends on 15) The system of claim 15, wherein the at least one modality of communication comprises at least one of the following: human speech, text, visual data, kinesthetic data, auditory data, taste data, ambient data, tactile data, haptic data and location data. ### Claim 17 (depends on 1) The system of claim 1 where the mechanical turk engine is further configured to expand initial concepts based at least in part on a phrase thesaurus. ### Claim 18 (depends on 1) The system of claim 1 where the mechanical turk engine is configured to expand initial concepts based at least in part on external data sources that match a domain of the application. ### Claim 19 (depends on 1) The system of claim 1 further configured to convert the mechanical turk project results into a validation project. ### Claim 20 (depends on 19) The system of claim 19 further configured to receive validation project results from at least one member interface. ### Claim 21 (depends on 20) The system of claim 20 wherein a mechanical turk project results validation is based, at least in part, on validation project results. ### Claim 22 (depends on 20) The system of claim 20 wherein a mechanical turk project results validation is based, at least in part, on two or more validation project results. ### Claim 23 (depends on 1) The system of claim 1 wherein mechanical turk project results are stored with data annotations. ### Claim 24 (depends on 23) The system of claim 23 wherein said data annotations comprise at least one of the following: project ID, timestamp, mechanical turk member identification and mechanical turk member demographics. ### Claim 25 (depends on 23) The system of claim 23 wherein the mechanical turk project results are validated, at least in part is, based on said data annotations.