# US 2018/0025274 A1 · Dynamic threshold filtering for watched questions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0025274 A1 - Document ID: US20180025274A1 - Status: Published application - Kind: A1 - Application number: 15/205,077 - Filed: 2016-07-08 - Published: 2018-01-25 - Earliest claimed: 2016-07-08 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Charles E. Beller; Paul J. Chase, JR.; Richard L. Darden; Michael Drzewucki; Edward G. Katz; Christopher Phipps - Web page: https://patentgrep.com/patent/US20180025274A1 ## CPC - G06N 5/02: Knowledge representation; Symbolic representation - G06F 16/90335: {Query processing} - G06N 5/04: Inference or reasoning models - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Embodiments are directed to a watched questions threshold filtering system that functions to determine and deliver to a user relevant and significant data changes with respect to a user's goals, as defined by a notification threshold value provided by the user. The user is provided with an option to flag one or more queries for automatic re-querying. Confidence scores are processed on new data (i.e., data ingested after the original question was asked) by utilizing a confidence threshold for indicating if the new data warrants alerting a user. ## Claims ### Claim 1 A computer implemented method for identifying new informative responses to answer a question based on new data and user input in a system capable of answering questions, the system comprising a processor and a memory comprising instructions executed by the processor, the method comprising: receiving as inputs: a watched question for monitoring; a notification threshold from a user; a first set of responses on a first date with a first confidence level resulting from the watched question; and a time stamp indicating the first date when the first set of responses were returned; querying ingested data sources after the first date for answers to the watched question to determine a second set of responses and a second confidence level; comparing the second confidence level to a dynamic threshold to determine responses from the second set of responses that meet the notification threshold; and responsive to determining the responses from the second set of responses that meet the notification threshold, providing a notification to the user. ### Claim 2 (depends on 1) The method of claim 1, wherein the second set of responses comprises a set of passages and a set of answers, each associated with a respective confidence level; and wherein the comparing step determines at least one of one or more passages and one or more answers that meet the notification threshold. ### Claim 3 (depends on 2) The method of claim 2, wherein the notification includes an indication of the at least one of one or more passages and one or more answers that meet the notification threshold. ### Claim 4 (depends on 1) The method of claim 1, further comprising: providing a user interface allowing the user to input the watched question and the notification threshold. ### Claim 5 (depends on 1) The method of claim 1, wherein the dynamic threshold is a function of the second confidence level and the notification threshold. ### Claim 6 (depends on 1) The method of claim 1, wherein the notification threshold is adjustable by the user of the system to adjust a quality of the responses from the second set of responses that meet the notification threshold. ### Claim 7 (depends on 1) The method of claim 1, wherein one or more of the responses from the second set of responses that meet the notification threshold have a lower confidence level than one or more of original responses from the first set of responses. ### Claim 8 A computer program product for identifying new informative responses to answer a question based on new data and user input in a system capable of answering questions, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to: receive as inputs: a watched question for monitoring; a notification threshold from a user; a first set of responses on a first date with a first confidence level resulting from the watched question; and a time stamp indicating the first date when the first set of responses were returned; query ingested data sources after the first date for answers to the watched question to determine a second set of responses and a second confidence level; compare the second confidence level to a dynamic threshold to determine responses from the second set of responses that meet the notification threshold; and responsive to determining the responses from the second set of responses that meet the notification threshold, provide a notification to the user. ### Claim 9 (depends on 8) The computer program product of claim 8, wherein the second set of responses comprises a set of passages and a set of answers, each associated with a respective confidence level; and wherein the compare step determines at least one of one or more passages and one or more answers that meet the notification threshold. ### Claim 10 (depends on 9) The computer program product of claim 9, wherein the notification includes an indication of the at least one of one or more passages and one or more answers that meet the notification threshold. ### Claim 11 (depends on 8) The computer program product of claim 8, wherein the program instructions executable by the processor further cause the processor to: provide a user interface allowing the user to input the watched question and the notification threshold. ### Claim 12 (depends on 8) The computer program product of claim 8, wherein the dynamic threshold is a function of the second confidence level and the notification threshold. ### Claim 13 (depends on 8) The computer program product of claim 8, wherein the notification threshold is adjustable by the user of the system to adjust a quality of the responses from the second set of responses that meet the notification threshold. ### Claim 14 (depends on 8) The computer program product of claim 8, wherein one or more of the responses from the second set of responses that meet the notification threshold have a lower confidence level than one or more of original responses from the first set of responses. ### Claim 15 A system for identifying new informative responses to answer a question based on new data and user input, the system comprising: a processor configured to: receive as inputs: a watched question for monitoring; a notification threshold from a user; a first set of responses on a first date with a first confidence level resulting from the watched question; and a time stamp indicating the first date when the first set of responses were returned; query ingested data sources after the first date for answers to the watched question to determine a second set of responses and a second confidence level; compare the second confidence level to a dynamic threshold to determine responses from the second set of responses that meet the notification threshold; and responsive to determining the responses from the second set of responses that meet the notification threshold, provide a notification to the user. ### Claim 16 (depends on 15) The system of claim 15, wherein the second set of responses comprises a set of passages and a set of answers, each associated with a respective confidence level; and wherein the compare step determines at least one of one or more passages and one or more answers that meet the notification threshold. ### Claim 17 (depends on 16) The system of claim 16, wherein the notification includes an indication of the at least one of one or more passages and one or more answers that meet the notification threshold. ### Claim 18 (depends on 15) The system of claim 15, wherein the processor is further configured to: provide a user interface allowing the user to input the watched question and the notification threshold. ### Claim 19 (depends on 15) The system of claim 15, wherein the dynamic threshold is a function of the second confidence level and the notification threshold. ### Claim 20 (depends on 15) The system of claim 15, wherein the notification threshold is adjustable by the user of the system to adjust a quality of the responses from the second set of responses that meet the notification threshold.