# US 11,030,086 B2 · Machine learning model full life cycle management framework > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,086 B2 - Document ID: US11030086B2 - Status: Granted - Kind: B2 - Application number: 16/277,057 - Filed: 2019-02-15 - Published: 2020-08-20 - Granted: 2021-06-08 - Earliest claimed: 2019-02-15 - Assignees: Tencent America LLC - Applicants: Tencent America LLC - Inventors: Min Tu; Shangqing Zhang; Tao Yang; Yaliang Li; Nan Du; Yusheng Xie; Wei Fan - Web page: https://patentgrep.com/patent/US11030086B2 ## CPC - G06F 11/3688: {for test execution, e.g. scheduling of test suites} - G06F 11/3676: {for coverage analysis} - G06F 11/3692: {for test results analysis} - G06F 11/3698: Environments for analysis, debugging or testing of software - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 20/00: Machine learning - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/08: Learning methods - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method and apparatus are provided that includes, in a same framework, storing an artificial intelligence (AI) model, loading the AI model into a serving platform, loading and testing a test unit against the AI model loaded into the serving platform, and collecting reports from results of storing the AI model, loading the AI model into the serving platform and testing the test unit. ## Claims ### Claim 1 An apparatus comprising at least one memory configured to store computer program code; at least one hardware processor configured to access said computer program code and operate as instructed by said computer program code, said computer program code including: model storage code configured to cause the at least one hardware processor to store an artificial intelligence (AI) model; model serving code configured to cause the at least one hardware processor to load the AI model into a serving platform from a plurality of serving platforms; model testing code configured to cause the at least one hardware processor to load and test a test unit against the AI model loaded into the serving platform; and monitoring and reporting code configured to cause the at least one hardware processor to collect reports from results of storing the AI model, loading the AI model into the serving platform and testing the test unit against the AI model loaded into the server platform. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein the model storage code is further configured to cause the at least one hardware processor to remove the AI model from a storage. ### Claim 3 (depends on 1) The apparatus according to claim 1, wherein the model serving code is further configured to cause the at least one hardware processor to periodically fetch a plurality of AI models. ### Claim 4 (depends on 3) The apparatus according to claim 3, wherein the model serving code is further configured to cause the at least one hardware processor to periodically load the plurality of AI models into respective ones of a plurality of serving platforms. ### Claim 5 (depends on 1) The apparatus according to claim 1, wherein the monitoring code is further configured to cause the at least one hardware processor to store the reports in respective ones of a plurality of databases. ### Claim 6 (depends on 1) The apparatus according to claim 1, wherein the model storage code is configured to cause the at least one hardware processor to continuously check whether to store ones of a plurality of models. ### Claim 7 (depends on 1) The apparatus according to claim 1, wherein the model serving code is configured to cause the at least one hardware processor to continuously check whether to load ones of a plurality of models into the serving platform. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein the model serving code is configured to cause the at least one hardware processor to continuously check whether to load and test the test unit against ones of a plurality of models loaded into the serving platform. ### Claim 9 (depends on 1) The apparatus according to claim 1, wherein the model monitoring and reporting code is configured to cause the at least one hardware processor to continuously check whether to provide ones of a plurality of reports. ### Claim 10 (depends on 1) The apparatus according to claim 1, wherein the model monitoring and reporting code is configured to cause the at least one hardware processor to check whether to upgrade or to remove the AI model based on a result of testing the test unit against the AI model loaded into the serving platform. ### Claim 11 A method comprising implementing in a framework: storing an artificial intelligence (AI) model; loading the AI model into a serving platform from a plurality of serving platforms; loading and testing a test unit against the AI model loaded into the serving platform; and collecting reports from results of storing the AI model, loading the AI model into the serving platform and testing the test unit against the AI model loaded into the server platform. ### Claim 12 (depends on 11) The method according to claim 11, further comprising: removing the AI model from a storage. ### Claim 13 (depends on 11) The method according to claim 11, further comprising: periodically fetching a plurality of AI models. ### Claim 14 (depends on 13) The method according to claim 13, further comprising: periodically loading the plurality of AI models into respective ones of a plurality of serving platforms. ### Claim 15 (depends on 11) The method according to claim 11, further comprising: storing the reports in respective ones of a plurality of databases. ### Claim 16 (depends on 11) The method according to claim 11, further comprising: continuously checking whether to store ones of a plurality of models. ### Claim 17 (depends on 11) The method according to claim 11, further comprising: continuously checking whether to load ones of a plurality of models into the serving platform. ### Claim 18 (depends on 11) The method according to claim 11, further comprising: continuously checking whether to load and test the test unit against ones of a plurality of models loaded into the serving platform. ### Claim 19 (depends on 11) The method according to claim 11, further comprising: continuously checking whether to provide ones of a plurality of reports. ### Claim 20 A non-transitory computer readable medium storing a program causing a computer to execute a process, the process comprising: storing an artificial intelligence (AI) model; loading the AI model into a serving platform from a plurality of serving platforms; loading and testing a test unit against the AI model loaded into the serving platform; and collecting reports from results of storing the AI model, loading the AI model into the serving platform and testing the test unit against the AI model loaded into the server platform.