# US 7,610,578 B1 · Test manager for integrated test environments > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,610,578 B1 - Document ID: US7610578B1 - Status: Granted - Kind: B1 - Application number: 10/925,413 - Filed: 2004-08-24 - Granted: 2009-10-27 - Earliest claimed: 2004-08-24 - Assignees: The Math Works, Inc. - Inventors: Greg Taillefer; David Karl Critz - Web page: https://patentgrep.com/patent/US7610578B1 ## CPC - G06F 8/30: Creation or generation of source code - G06F 11/3698: Environments for analysis, debugging or testing of software - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods are provided to automatically generate, distribute and execute code locally and remotely. The test may therefore be performed using the automatically distributed code. Automated parameter mapping may be provided to couple parameters applicable to a test step with the test step. A test step in the test may therefore automatically include the coupled parameters. Data storage may be performed using a multidimensional parametric space. Each point in space may contain arbitrary nonscalar data which define an individual data workspace. These points can also be plotted against each other and overlayed to aid in visualization of multi-dimensional data. ## Claims ### Claim 1 In a computational device providing a test environment, a method for conducting a test for testing units under test, the method comprising: enabling users to edit the test to include a sequence of one or more test steps for testing the units under test, each of the one or more test steps being of a type selected from a group consisting of a test step for testing a textual program, a test step for testing a graphical program, a test step for testing a function provided in a software tool and a test step for testing a hardware device; enabling the users to edit one or more independent parameters, an independent parameter defining a dimension of a parametric space, the parametric space including a plurality of points; generating code for the test; and automatically executing the code using defined values of the one or more independent parameters to produce output data from a number of executions of the test, elements of the output data corresponding to the plurality of points of the parametric space of the one or more independent parameters, wherein automatically executing comprises: using at least two independent parameters to execute the code, wherein at least one of the at least two used independent parameters has a plurality of defined values, determining the number of executions of the test by multiplying the defined values in each of the at least two used independent parameters; initializing a multidimensional parametric space for storing the output data generated from the number of executions of the test, dimensions of the multidimensional parametric space representing and corresponding to a product of the defined values in each of the at least two used independent parameters, and adding the output data to the multidimensional parametric space. ### Claim 2 (depends on 1) The method of claim 1, wherein the enabling the users to edit the test further comprises: determining test steps installed in the computational device; and enabling the users to select the one or more test steps for the test from the test steps installed in the computational device. ### Claim 3 (depends on 1) The method of claim 1, wherein the enabling the users to edit the one or more independent parameters further comprises: determining variables installed in the computational device; and enabling the users to select one or more variables for the test from the variables installed in the computational device so that the users are able to specify variables and parameters of the test. ### Claim 4 (depends on 1) The method of claim 1, wherein the enabling the users to edit the test further comprises: enabling the users to add new software tools to the test environment. ### Claim 5 (depends on 1) The method of claim 1, wherein the executing further comprises: distributing at least a portion of the code for the test to one or more remote computational devices for a distributed execution of the code. ### Claim 6 (depends on 5) The method of claim 5, wherein the distributing comprises: distributing all portions of the test at a time to one or more remote computational devices for a distributed execution of the test; and enabling the all portions of the test to be run in parallel in the one or more remote computational devices. ### Claim 7 (depends on 5) The method of claim 5, wherein the distributing comprises: (a) distributing a portion of the test at a time to one or more remote computational devices; (b) enabling the distributed portion of the test to be run in parallel; and (c) repeating (a) and (b) steps until all portions of the test are distributed and run in parallel. ### Claim 8 (depends on 1) The method of claim 1, wherein the executing comprises: obtaining parameters for a current execution of the test. ### Claim 9 (depends on 8) The method of claim 8, wherein the executing comprises: determining whether there is a remaining execution of the test; and obtaining parameters for a next execution of the test. ### Claim 10 (depends on 1) The method of claim 1, further comprising: storing the output data of the number of executions of the test in the multidimensional parametric space. ### Claim 11 (depends on 10) The method of claim 10, wherein the multidimensional parametric space stores metadata of the output data of the number of executions of the test. ### Claim 12 (depends on 10) The method of claim 10, wherein the output data of the number of executions of the test in the multidimensional parametric space is stored in memory in a binary or textual hierarchical data structure. ### Claim 13 (depends on 10) The method of claim 10, wherein each point in the multidimensional parametric space comprises a reference to one or more information storage and retrieval systems that contain data, the one or more information storage and retrieval systems being loaded one at a time as the data in the one or more information storage and retrieval systems is requested. ### Claim 14 (depends on 13) The method of claim 13, wherein the one or more information storage and retrieval systems comprises a multilevel hierarchical storage. ### Claim 15 (depends on 10) The method of claim 10, wherein the multidimensional parametric space is integrated in the testing environment. ### Claim 16 (depends on 10) The method of claim 10, further comprising: providing a graphical display of the output data stored in the multidimensional parametric space. ### Claim 17 (depends on 16) The method of claim 16, wherein the graphical display is automatically generated based on data generated by the testing environment. ### Claim 18 (depends on 16) The method of claim 16, wherein the graphical display displays multi-dimensional data. ### Claim 19 (depends on 1) The method of claim 1, wherein the units under test comprise at least one of a graphical program, a textual program, a function provided in a software tool, and a hardware device. ### Claim 20 A computer-implemented system for conducting a test for testing units under test, the system comprising: a display; and a processor configured to provide a user interface on the display for users to edit the test to include a plurality of test steps for testing the units under test, the test including a sequence of at least a first test step and a second test step, the second step being of a different type than the first test step, the first test step and the second test step being of types selected from a group consisting of a test step for testing a textual program, a test step for testing a graphical program, a test step for testing a function provided in a software tool and a test step for testing a hardware device; enable the user to edit one or more independent parameters, an independent parameter defining a dimension of a parametric space, the parametric space including a plurality of points; generate code for the test; and execute the code using defined values of the one or more independent parameters to generate output data from a number of executions of test, elements of the output data corresponding to the plurality of points of the parametric space of the one or more independent parameters, wherein execute the code comprising: use at least two independent parameters to execute the code, wherein at least one of the at least two used independent parameters has a plurality of defined values, determine the number of executions of the test by multiplying the defined values in each of the at least two used independent parameters; initialize a multidimensional parametric space for storing the output data generated from the number of executions of the test, dimensions of the multidimensional parametric space representing and corresponding to a product of the defined values in each of the at least two used independent parameters, and add the output data to the multidimensional parametric space. ### Claim 21 (depends on 20) The system of claim 20, wherein the user interface enables the users to select the plurality of test steps for the test so that the users are able to edit the sequence of the test steps in the test. ### Claim 22 (depends on 20) The system of claim 20, wherein the user interface enables the users to select one or more variables for the test so that the users are able to specify variables and parameters of the test. ### Claim 23 (depends on 20) The system of claim 20, wherein the processor is further configured to enable the users to add new software tools to the test environment. ### Claim 24 (depends on 20) The system of claim 20, wherein the code for the test is executed locally with the user interface. ### Claim 25 (depends on 20) The system of claim 20, wherein the code for the test is executed remotely from the user interface. ### Claim 26 (depends on 20) The system of claim 20, wherein the processor is further configured to store the output data of the test as a multidimensional dataset in the test environment. ### Claim 27 (depends on 26) The system of claim 26, wherein the multidimensional dataset comprises metadata of the output data of the test. ### Claim 28 (depends on 26) The system of claim 26, wherein the multidimensional dataset is stored in memory in a binary or textual hierarchical data structure. ### Claim 29 (depends on 26) The system of claim 26, wherein each point in the multidimensional dataset comprises a reference to one or more information storage and retrieval systems that contain data, the one or more information storage and retrieval systems being loaded one at a time as the data in the one or more information storage and retrieval systems is requested. ### Claim 30 (depends on 29) The system of claim 29, wherein the one or more information storage and retrieval systems comprises a multilevel hierarchical storage. ### Claim 31 (depends on 26) The system of claim 26, wherein a dimension of the multidimensional dataset is defined by a number of elements of an independent parameter contained by the test. ### Claim 32 (depends on 26) The system of claim 26, further comprising: a viewer for displaying the output data stored in the multidimensional dataset. ### Claim 33 (depends on 20) The system of claim 20, wherein the units under test comprise at least one of a graphical program, a textual program, a function call provided in a software tool, and a hardware device. ### Claim 34 A system for providing a test environment in a computational device, the system comprising: storage; and a processor configured to: provide a first user interface for users to edit a test to include a sequence of one or more test steps for testing units under test, each of the one or more test steps being of a type selected from a group consisting of a test step for testing a textual program, a test step for testing a graphical program, a test step for testing a function provided in a software tool and a test step for testing a hardware device; provide a second user interface for the users to edit one or more independent parameters, an independent parameter defining a dimension of a parametric space, the parametric space including a plurality of points; generate code for the test; automatically execute the code using defined values of the one or more independent parameters to produce output data from a number of executions of the test, elements of the output data corresponding to the plurality of points of the parametric space of the one or more independent parameters, wherein automatically execute comprises: use at least two independent parameters to execute the code, wherein at least one of the at least two used independent parameters has a plurality of defined values, determine the number of executions of the test by multiplying the defined values in each of the at least two used independent parameters; initialize a multidimensional parametric space for storing the output data generated from the number of executions of the test, dimensions of the multidimensional parametric space representing and corresponding to a product of the defined values in each of the at least two used independent parameters, add the output data to the multidimensional parametric space, and store the output data as a multidimensional dataset in the storage, elements of the multidimensional dataset corresponding to the plurality of points of the parametric space in the at least two used independent parameters. ### Claim 35 (depends on 34) The system of claim 34, wherein the multidimensional dataset comprises metadata of the output data of the test. ### Claim 36 (depends on 34) The system of claim 34, wherein the multidimensional dataset is stored in memory in a binary or textual hierarchical data structure. ### Claim 37 (depends on 34) The system of claim 34, wherein each point in the multidimensional dataset comprises a reference to one or more information storage and retrieval systems that contain data, the one or more information storage and retrieval systems being loaded one at a time when the data in the one or more information storage and retrieval systems is requested. ### Claim 38 (depends on 37) The system of claim 37, wherein the one or more information storage and retrieval systems comprises multilevel hierarchical storage. ### Claim 39 (depends on 34) The system of claim 34, wherein the first user interface enables the users to select the plurality of test steps for the test from test steps installed in the computational device so that the users are able to edit a sequence of the test steps in the test. ### Claim 40 (depends on 34) The system of claim 34, wherein the second user interface enables the users to select one or more variables for the test from variables installed in the computational device so that the users are able to specify variables and parameters of the test. ### Claim 41 (depends on 34) The system of claim 34, further comprising: an application program interface for enabling the users to add new software tools to the test environment. ### Claim 42 (depends on 34) The system of claim 34, further comprising: a viewer for displaying the output data stored in the multidimensional dataset. ### Claim 43 (depends on 34) The system of claim 34, wherein the units under test comprise at least one of a graphical program, a textual program, a function call provided in a software tool, and a hardware device. ### Claim 44 A computer-readable storage medium holding computational device executable instructions for conducting a test for testing units under test, the medium comprising: instructions for enabling users to edit the test to include a sequence of one or more test steps for testing the units under test, each of the one or more test steps being of a type selected from a group consisting of a test step for testing a textual program, a test step for testing a graphical program, a test step for testing a function provided in a software tool and a test step for testing a hardware device; instructions for enabling the users to edit one or more independent parameters, an independent parameter defining a dimension of a parametric space, the parametric space including a plurality of points; instructions for generating code for the test; and instructions for automatically executing the code using defined values of the one or more independent parameters to generate output data from a number of executions of the test, elements of the output data corresponding to the plurality of points of the parametric space of the one or more independent parameters, wherein the instructions for automatically executing comprises: instructions for using at least two independent parameters to execute the code, wherein at least one of the at least two used independent parameters has a plurality of defined values, instructions for determining the number of executions of the test by multiplying the defined values in each of the at least two used independent parameters; instructions for initializing a multidimensional parametric space for storing the output data generated from the number of executions of the test, dimensions of the multidimensional parametric space representing and corresponding to a product of the defined values in each of the at least two used independents, and instruction for adding the output data to the multidimensional parametric space. ### Claim 45 (depends on 44) The medium of claim 44, wherein the instructions for enabling the users to edit the test further comprises: instructions for determining test steps installed in the computational device; and instructions for enabling the users to select the one or more test steps for the test from the test steps installed in the computational device. ### Claim 46 (depends on 44) The medium of claim 44, wherein the instructions for enabling the users to edit the one or more independent parameters further comprises: instructions for determining variables installed in the computational device; and instructions for enabling the users to select one or more variables for the test from the variables installed in the computational device so that the users are able to specify variables and parameters of the test. ### Claim 47 (depends on 44) The medium of claim 44, wherein the instructions for enabling the users to edit the test further comprises: instructions for determining software tools installed in the computational device; and instructions for enabling the users to add new software tools to the test environment. ### Claim 48 (depends on 44) The medium of claim 44, wherein the instructions for executing further comprises: instructions for distributing at least a portion of the code for the test to one or more remote computational devices for a distributed execution of the code. ### Claim 49 (depends on 48) The medium of claim 48, wherein the instructions for distributing comprises: instructions for distributing all portions of the test at a time to one or more remote computational devices for a distributed execution of the test; and instructions for enabling the all portions of the test to be run in parallel in the one or more remote computational devices. ### Claim 50 (depends on 48) The medium of claim 48, wherein the instructions for distributing comprises: instructions for (a1) distributing a portion of the test at a time to one or more remote computational devices; instructions for (b1) enabling the distributed portion of the test to be run in parallel; and instructions for repeating (a1) and (b1) until all portions of the test are distributed and run in parallel. ### Claim 51 (depends on 44) The medium of claim 44, wherein the instructions for executing comprises: instructions for obtaining parameters for a current execution of the test. ### Claim 52 (depends on 51) The medium of claim 51, wherein the instructions for executing comprises: instructions for determining whether there is a remaining execution of the test; and instructions for obtaining parameters for a next execution of the test. ### Claim 53 (depends on 44) The medium of claim 44, further comprising: instructions for storing the output data of the number of executions of the test in the multidimensional parametric space. ### Claim 54 (depends on 53) The medium of claim 53, wherein the multidimensional parametric space stores metadata of the output data of the number of executions of the test. ### Claim 55 (depends on 53) The medium of claim 53, wherein the multidimensional parametric space is stored in memory in a binary or textual hierarchical data structure. ### Claim 56 (depends on 53) The medium of claim 53, wherein each point in the multidimensional parametric space comprises a reference to one or more information storage and retrieval systems that contain data, the one or more information storage and retrieval systems being loaded one at a time as the data in the one or more information storage and retrieval systems is requested. ### Claim 57 (depends on 56) The medium of claim 56, wherein the one or more information storage and retrieval systems comprises a multilevel hierarchical storage. ### Claim 58 (depends on 53) The medium of claim 53, wherein the multidimensional parametric space is integrated with the testing environment. ### Claim 59 (depends on 53) The medium of claim 53, further comprising: instructions for providing a graphical display of the output data stored in the multidimensional parametric space. ### Claim 60 (depends on 59) The medium of claim 59, wherein the graphical display is automatically generated based on data generated by the testing environment. ### Claim 61 (depends on 59) The medium of claim 59, wherein the graphical display displays multi-dimensional data. ### Claim 62 (depends on 44) The medium of claim 44, wherein the units under test comprise at least one of a graphical program, a textual program, a function provided in a software tool, and a hardware device.