# US 2014/0195195 A1 · Object investigation and classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0195195 A1
- Document ID: US20140195195A1
- Status: Published application
- Kind: A1
- Application number: 14/151,625
- Filed: 2014-01-09
- Published: 2014-07-10
- Earliest claimed: 2013-01-09
- Assignees: Syntouch, LLC
- Applicants: Syntouch, LLC
- Inventors: Jeremy A. Fishel; Gerald E. Loeb
- Web page: https://patentgrep.com/patent/US20140195195A1

## CPC

- G01B 21/30: for measuring roughness or irregularity of surfaces
- A61B 5/0048: {Detecting, measuring or recording by applying mechanical forces or stimuli (A61B5/021, A61B5/4884 and A61B9/00 take precedence)}
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/01: Measuring temperature of body parts {; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue} (clinical contact thermometers G01K13/20)
- A61B 5/6897: {Computer input devices, e.g. mice or keyboards}
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06V 10/764: using classification, e.g. of video objects
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

An object investigation and classification system may include an object test system, a data storage system, and a data processing system. The object test system may receive a command to perform at least one action with a test object, perform the at least one action with the test object, and return test information indicative of at least one percept resulting from the at least one action. The data storage system may contain an experience database containing data indicative of multiple classifications and, for each classification, at least one action that was performed with at least one previously-observed reference object having this classification, and at least one percept value that is based in whole or in part on the test information resulting from the at least one action. The data processing system may: a) for each of multiple different classifications, compute or receive an initial prior probability that a test object falls within the classification; b) determine at least one action that should be performed with the test object to obtain at least one percept about the test object that is likely to enable the classification of the test object to be more accurately determined based on the initial prior probabilities and the data within the experience database; c) cause the object test system to perform the at least one action with the test object; d) receive test information from the object test system indicative of at least one percept resulting from the at least one action with the test object; e) compute at least one percept value; f) for each of multiple different classifications, determine a posterior probability that the test object falls within the classification based on the initial prior probability, the at least one percept value, and data within the experience database; g) determine whether any of the posterior probabilities meets or exceeds a threshold; h) if none of the posterior probabilities meet or exceed the threshold, repeat b) through i), substituting the posterior probabilities determined in f) for the initial prior probabilities in b); and/or i) when one or more of the posterior probabilities meets or exceeds the threshold, output information indicative of one or more of the classifications that correspond to the one or more posterior probabilities that meets or exceeds the threshold.

## Claims

### Claim 1

An object investigation and classification system comprising:
an object test system that receives a command to perform at least one action with a test object, performs the at least one action with the test object, and returns test information indicative of at least one percept resulting from the at least one action;
a data storage system that contains an experience database containing data indicative of multiple classifications and, for each classification, at least one action that was performed with at least one previously-observed reference object having this classification, and at least one percept value that is based in whole or in part on the test information resulting from the at least one action; and
a data processing system that:
a) for each of multiple different classifications, computes or receives an initial prior probability that a test object falls within the classification;
b) determines at least one action that should be performed with the test object to obtain at least one percept about the test object that is likely to enable the classification of the test object to be more accurately determined based on the initial prior probabilities and the data within the experience database;
c) causes the object test system to perform the at least one action with the test object;
d) receives test information from the object test system indicative of at least one percept resulting from the at least one action with the test object;
e) computes at least one percept value;
f) for each of multiple different classifications, determines a posterior probability that the test object falls within the classification based on the initial prior probability, the at least one percept value, and data within the experience database;
g) determines whether any of the posterior probabilities meets or exceeds a threshold;
h) if none of the posterior probabilities meet or exceed the threshold, repeats b) through i), substituting the posterior probabilities determined in f) for the initial prior probabilities in b); and
i) when one or more of the posterior probabilities meets or exceeds the threshold, outputs information indicative of one or more of the classifications that correspond to the one or more posterior probabilities that meets or exceeds the threshold.

### Claim 2 (depends on 1)

The object investigation and classification system of claim 1 wherein the data in the experience database includes data indicative of a distribution of percept values for at least one of the percepts resulting from an action that has been performed multiple times in association with a given classification or a given previously observed reference object.

### Claim 3 (depends on 1)

The object investigation and classification system of claim 1 wherein the threshold is not the same during all of the repetitions of g).

### Claim 4 (depends on 1)

The object investigation and classification system of claim 1 wherein the data processing system adds data about at least one of the percepts indicated by the received test information to the experience database.

### Claim 5 (depends on 4)

The object investigation and classification system of claim 4 wherein the data processing system determines which of the percepts indicated by the received test information should have data about them added to the experience database based on the degree to which the at least one action that led to each percept caused a change in the probability that the test object has one or more of the classifications.

### Claim 6 (depends on 1)

The object investigation and classification system of claim 1 wherein the data processing system determines if the percept values of at least one percept resulting from the at least one action performed with the rest object is substantially unlike the percept values of the same at least one percept resulting from the same at least one action associated with any of the classifications described in the experience database.

### Claim 7 (depends on 6)

The object investigation and classification system of claim 6 wherein the data processing system adds the percept values of at least one percept resulting from the at least one action with the test object to the experience database in association with a new classification that was not in the experience database when the data processing system determines that the percept values of at least one percept resulting from at least one action performed with the test object is substantially unlike the percept values of the same at least one percept resulting from the same at least one action associated with any of the classifications described in the experience database.

### Claim 8 (depends on 7)

The object investigation and classification system of claim 7 wherein the data processing system determines at least one additional action to perform with the test object to obtain at least one percept associated with the at least one additional action and adds the percept value of the at least one percept resulting from the at least one additional action to the experience database.

### Claim 9 (depends on 1)

The object investigation and classification system of claim 1 wherein the initial prior probabilities are the same.

### Claim 10 (depends on 1)

The object investigation and classification system of claim 1 wherein the initial prior probabilities are weighted based on the number of times each classification has been associated with a previously-observed reference object in the experience database.

### Claim 11 (depends on 1)

The object investigation and classification system of claim 1 wherein:
the experience database also contains data indicative of a time when each percept was obtained; and
the initial prior probabilities are weighted based on the time each percept was obtained.

### Claim 12 (depends on 1)

The object investigation and classification system of claim 1 wherein:
the experience database also contains data indicative of a location where each percept was obtained; and
the initial prior probabilities are weighted based on the location each percept was obtained.

### Claim 13 (depends on 1)

The object investigation and classification system of claim 1 wherein the determines at least one action that should be performed includes:
a) for each classification, computing a probability density function that describes a distribution of percept values expected for a percept resulting from an action that has been performed multiple times in association with the classification;
b) computing a degree to which two different probability density functions for two different classifications result in similar distributions of the percept values of the same percept when performing the same action;
c) multiplying the degree computed in 14b) by the prior probability that the test object has each classification used to compute the degree in 14b);
d) repeating 14b and 14c for all other possible pairs of classifications;
e) summing the results of all of the multiplications in 14c);
f) repeating 14a-14e for each of the other combinations of actions and percepts; and
g) selecting the action that yields the lowest summing value in 14e) for any percept as the action to be performed.

### Claim 14 (depends on 13)

The object investigation and classification system of claim 13 wherein previously performed actions with the test object are given less preference in being re-selected as the action to be performed if the previously performed action was unsuccessful in producing percept values that help discriminate between the most likely classifications.

### Claim 15 (depends on 1)

The object investigation and classification system of claim 1 wherein the object test system includes:
at least one controllable actuator that performs the at least one action with the test object; and
at least one tactile sensor that interacts with the test object and returns the test information indicative of at least one percept resulting from the at least one action.

### Claim 16 (depends on 15)

The object investigation and classification system of claim 15 wherein the at least one action with the test object includes:
sliding across a surface of the test object; or
contacting the test object with a varying force.

### Claim 17 (depends on 15)

The object investigation and classification system of claim 15 wherein the data processing system processes the test information indicative of the at least one percept is processed to indicate:
a type of surface texture on the test object;
a degree of roughness or smoothness of the test object;
a degree of coarseness or fineness of the test object;
a degree of hardness or softness of the test object;
a degree to which the test object has a springiness or dampens; or
a thermal property of the test object.

### Claim 18

A non-transitory, tangible, computer-readable storage medium containing a program of instructions that cause a computer system running the program of instructions to perform the following functions in connection with an object test system that receives a command to perform at least one action with a test object, performs the at least one action with the test object, and returns test information indicative of at least one percept resulting from the at least one action; and a data storage system that contains an experience database containing data indicative of multiple classifications and, for each classification, at least one action that was performed with at least one previously-observed reference object having this classification, and at least one percept value that is based in whole or in part on the test information resulting from the at least one action:
a) for each of multiple different classifications, computes or receives an initial prior probability that a test object falls within the classification;
b) determines at least one action that should be performed with the test object to obtain at least one percept about the test object that is likely to enable the classification of the test object to be more accurately determined based on the initial prior probabilities and the data within the experience database;
c) causes the object test system to perform the at least one action with the test object;
d) receives test information from the object test system indicative of at least one percept resulting from the at least one action with the test object;
e) computes at least one percept value;
f) for each of multiple different classifications, determines a posterior probability that the test object falls within the classification based on the initial prior probability, the at least one percept value, and data within the experience database;
g) determines whether any of the posterior probabilities meets or exceeds a threshold;
h) if none of the posterior probabilities meet or exceed the threshold, repeats b) through i), substituting the posterior probabilities determined in f) for the initial prior probabilities in b); and
i) when one or more of the posterior probabilities meets or exceeds the threshold, outputs information indicative of one or more of the classifications that correspond to the one or more posterior probabilities that meets or exceeds the threshold.

### Claim 19 (depends on 18)

The storage medium of claim 18 wherein the data in the experience database includes data indicative of a distribution of percept values for at least one of the percepts resulting from an action that has been performed multiple times in association with a given classification or a given previously observed reference object.

### Claim 20 (depends on 18)

The storage medium of claim 18 wherein the threshold is not the same during all of the repetitions of g).

### Claim 21 (depends on 18)

The storage medium of claim 18 wherein the program of instructions causes the computer system running the program of instructions to add data about at least one of the percepts indicated by the received test information to the experience database.

### Claim 22 (depends on 21)

The storage medium of claim 21 wherein the program of instructions causes the computer system running the program of instructions to determine which of the percepts indicated by the received test information should have data about them added to the experience database based on the degree to which the at least one action that led to each percept caused a change in the probability that the test object has one or more of the classifications.

### Claim 23 (depends on 18)

The storage medium of claim 18 wherein the program of instructions causes the computer system running the program of instructions to determine if the percept values of at least one percept resulting from the at least one action performed with the test object is substantially unlike the percept values of the same at least one percept resulting from the same at least one action associated with any of the classifications described in the experience database.

### Claim 24 (depends on 23)

The storage medium of claim 23 wherein the program of instructions causes the computer system running the program of instructions to add the percept values of at least one percept resulting from the at least one action with the test object to the experience database in association with a new classification that was not in the experience database when the data processing system determines that the percept values of at least one percept resulting from at least one action performed with the test object is substantially unlike the percept values of the same at least one percept resulting from the same at least one action associated with any of the classifications described in the experience database.

### Claim 25 (depends on 24)

The storage medium of claim 24 wherein the program of instructions causes the computer system running the program of instructions to determine at least one additional action to perform with the test object to obtain at least one percept associated with the at least one additional action and adds the percept value of the at least one percept resulting from the at least one additional action to the experience database.

### Claim 26 (depends on 18)

The storage medium of claim 18 wherein the initial prior probabilities are the same.

### Claim 27 (depends on 18)

The storage medium of claim 18 wherein the initial prior probabilities are weighted based on the number of times each classification has been associated with a previously-observed reference object in the experience database.

### Claim 29 (depends on 18)

The storage medium of claim 18 wherein the experience database also contains data indicative of a time when each percept was obtained and the initial prior probabilities are weighted based on the time each percept was obtained.

### Claim 30 (depends on 18)

The storage medium of claim 18 wherein the experience database also contains data indicative of a location where each percept was obtained and the initial prior probabilities are weighted based on the location each percept was obtained.

### Claim 31 (depends on 18)

The storage medium of claim 18 wherein the determines at least one action that should be performed includes:
a) for each classification, computing a probability density function that describes a distribution of percept values expected for a percept resulting from an action that has been performed multiple times in association with the classification;
b) computing a degree to which two different probability density functions for two different classifications result in similar distributions of the percept values of the same percept when performing the same action;
c) multiplying the degree computed in 31b) by the prior probability that the test object has each classification used to compute the degree in 31b);
d) repeating 31b and 31c for all other possible pairs of classifications;
e) summing the results of all of the multiplications in 31c);
f) repeating 31a-31e for each of the other combinations of actions and percepts; and
g) selecting the action that yields the lowest summing value in 31e) for any percept as the action to be performed.

### Claim 32 (depends on 31)

The storage medium of claim 31 wherein previously performed actions with the test object are given less preference in being re-selected as the action to be performed if the previously performed action was unsuccessful in producing percept values that help discriminate between the most likely classifications.

### Claim 33 (depends on 18)

The storage medium of claim 18 wherein the at least one action with the test object includes:
sliding across a surface of the test object; or
contacting the test object with a varying force.

### Claim 34 (depends on 18)

The storage medium of claim 18 wherein the data processing system processes the test information indicative of the at least one percept is processed to indicate:
a type of surface texture on the test object;
a degree of roughness or smoothness of the test object;
a degree of coarseness or fineness of the test object;
a degree of hardness or softness of the rest object;
a degree to which the test object has a springiness or dampens; or
a thermal property of the test object.