# US 2017/0256052 A1 · Leveraging on local and global textures of brain tissues for robust automatic brain tumor detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0256052 A1
- Document ID: US20170256052A1
- Status: Published application
- Kind: A1
- Application number: 15/061,465
- Filed: 2016-03-04
- Published: 2017-09-07
- Earliest claimed: 2016-03-04
- Assignees: Siemens Healthcare GmbH
- Applicants: Siemens Healthcare GmbH
- Inventors: Subhabrata Bhattacharya; Shanhui Sun; Terrence Chen
- Web page: https://patentgrep.com/patent/US20170256052A1

## CPC

- G06V 20/698: {Matching; Classification}
- A61B 1/00045: {Display arrangement}
- A61B 1/043: {for fluorescence imaging}
- A61B 5/4064: {Evaluating the brain (for intracranial pressure A61B5/031; for cerebral blood gases A61B5/14553; using EEG A61B5/369)}
- A61B 5/7282: {Event detection, e.g. detecting unique waveforms indicative of a medical condition (cough events A61B5/0823; seizures A61B5/4094; sleep apnoea A61B5/4818)}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/23: Clustering techniques
- G06F 18/24: Classification techniques
- G06F 18/254: {of classification results, e.g. of results related to same input data}
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/41: based on statistical description of texture
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data}
- H04N 19/124: Quantisation
- H04N 19/136: Incoming video signal characteristics or properties
- G06F 18/2411: based on the proximity to a decision surface, e.g. support vector machines
- G06F 18/24323: {Tree-organised classifiers}
- G06T 2207/30016: Brain
- G06V 10/467: {Encoded features or binary features, e.g. local binary patterns [LBP]}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for performing cellular classification includes generating a plurality of local dense Scale Invariant Feature Transform (SIFT) features based on a set of input images and converting the plurality of local dense SIFT features into a multi-dimensional code using a feature coding process. A first classification component is used to generate first output confidence values based on the multi-dimensional code and a plurality of global Local Binary Pattern Histogram (LBP-H) features are generated based on the set of input images. A second classification component is used to generate second output confidence values based on the plurality of LBP-H features and the first output confidence values and the second output confidence values are merged. Each of the set of input images may then be classified as one of a plurality of cell types using the merged output confidence values.

## Claims

### Claim 1

A method for performing cellular classification, the method comprising:
generating a plurality of local dense Scale Invariant Feature Transform (SIFT) features based on a set of input images;
converting the plurality of local dense SIFT features into a multi-dimensional code using a feature coding process;
using a first classification component to generate first output confidence values based on the multi-dimensional code;
generating a plurality of global Local Binary Pattern Histogram (LBP-H) features based on the set of input images;
using a second classification component to generate second output confidence values based on the plurality of LBP-H features;
merging the first output confidence values and the second output confidence values to yield merged output confidence values; and
classifying each of the set of input images as one of a plurality of cell types using the merged output confidence values.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
acquiring a plurality of input images;
calculating an entropy value for each of the plurality of input images, each entropy value representative of an amount of texture information in a respective image;
identifying one or more low-entropy images in the set of input images, wherein the one or more low-entropy images are each associated with a respective entropy value below a threshold value; and
generating the set of input images based on the plurality of input images, wherein the set of input images excludes the one or more low-entropy images.

### Claim 3 (depends on 2)

The method of claim 2, wherein the plurality of input images are acquired using an endomicroscopy device during a medical procedure.

### Claim 4 (depends on 2)

The method of claim 2, wherein the plurality of input images are acquired using a digital holographic microscopy device during a medical procedure.

### Claim 5 (depends on 1)

The method of claim 1, wherein the feature coding process performs a bag of words coding process.

### Claim 6 (depends on 5)

The method of claim 5, wherein the bag of words coding process comprises:
randomly selecting a predetermined percentage of descriptors from a training split;
performing k-means clustering to construct a plurality of different vocabularies;
using the plurality of different vocabularies to quantize the SIFT features in each input image included in the set of input images to yield a quantized representation of the SIFT features,
wherein the multi-dimensional code is the quantized representation.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first classification component is a Support Vector Machine (SVM) classifier.

### Claim 8 (depends on 7)

The method of claim 7, wherein the multi-dimensional code is used to train the SVM classifier with a radial basis function (RBF) kernel.

### Claim 9 (depends on 7)

The method of claim 7, wherein the multi-dimensional code is used to train the SVM classifier with a linear kernel.

### Claim 10 (depends on 1)

The method of claim 1, wherein the second classification component is a Random Forest classifier with a plurality of trees and a maximum depth of a plurality of levels for each of the plurality of trees.

### Claim 11 (depends on 1)

The method of claim 1, a multiplicative fusion algorithm is used to merge the first output confidence values and the second output confidence values to yield the merged output confidence values.

### Claim 12

A method for performing cellular classification during a medical procedure, the method comprising:
prior to the medical procedure, performing a training process comprising:
training a first classification component based on a plurality of local dense SIFT features associated with a plurality of training images, and
training a second classification component based on a plurality of global LBP-H features associated with the plurality of training images; and
during the medical procedure, performing a cell classification process comprising:
acquiring an input image using an endomicroscopy device,
using the first classification component to generate first output confidence values based on SIFT features associated with the input image,
using the second classification component to generate second output confidence values based on LBP-H features associated with the input image,
merging the first output confidence values and the second output confidence values to yield merged output confidence values; and
identifying a class label corresponding to the input image based on the merged output confidence values, and
presenting the class label on a display operably coupled to the endomicroscopy device.

### Claim 13 (depends on 12)

The method of claim 12, wherein the class label provides an indication of whether biological material in the input image is healthy, malignant, or benign.

### Claim 14 (depends on 12)

The method of claim 12, wherein the first classification component is an SVM classifier.

### Claim 15 (depends on 14)

The method of claim 14, wherein the second classification component is a Random Forrest classifier.

### Claim 16

A system performing cellular classification, the system comprising:
an endomicroscopy device configured to acquire a set of input images during a medical procedure;
an imaging computer configured to perform a cellular classification process during the medical procedure, the cellular classification process comprising:
using a first classification component to generate first output confidence values based on SIFT features associated with the set of input images,
using a second classification component to generate second output confidence values based on LBP-H features associated with the set of input images,
merging the first output confidence values and the second output confidence values to yield merged output confidence values; and
identifying one or more cellular class labels corresponding to the set of input images based on the merged output confidence values, and
a display configured to present the one or more cellular class labels during the medical procedure.

### Claim 17 (depends on 16)

The system of claim 16, wherein the endomicroscopy device is a Confocal Laser Endo-microscopy device.

### Claim 18 (depends on 16)

The system of claim 16, wherein the endomicroscopy device is a Digital Holographic Microscopy device.

### Claim 19 (depends on 16)

The system of claim 16, wherein the first classification component is a SVM classifier.

### Claim 20 (depends on 19)

The system of claim 19, wherein the second classification component is a Random Forrest classifier.
