# US 10,945,657 B2 · Automated surface area assessment for dermatologic lesions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,945,657 B2 - Document ID: US10945657B2 - Status: Granted - Kind: B2 - Application number: 15/999,123 - Filed: 2018-08-17 - Published: 2019-02-21 - Granted: 2021-03-16 - Earliest claimed: 2017-08-18 - Assignees: Massachusetts Institute of Technology - Applicants: Massachusetts Institute of Technology; The Brigham and Women's Hospital, Inc. - Inventors: Anantha P. Chandrakasan; Priyanka Raina; Jiarui Huang; Victor Huang - Web page: https://patentgrep.com/patent/US10945657B2 ## CPC - A61B 5/444: {Evaluating skin marks, e.g. mole, nevi, tumour, scar} - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - G06T 7/0012: {Biomedical image inspection} - G06T 7/11: Region-based segmentation - G06T 7/62: of area, perimeter, diameter or volume - G06T 5/40: using histogram techniques - G06T 2207/10004: Still image; Photographic image - G06T 2207/10024: Color image - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20161: Level set - G06T 2207/30088: Skin; Dermal - G06T 2207/30096: Tumor; Lesion - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method for assessing a three-dimensional (3D) surface area having one or more lesions is disclosed. The method includes steps of: capturing a two-dimensional (2D) color image and a depth image of the 3D surface area; enhancing contrast of the 2D color image; segmenting the one or more lesions of the 2D color image into one or more segmented lesions; and calculating 3D area of the one or more segmented lesions using information from 2D color image and the depth image. ## Claims ### Claim 1 A method for assessing a three-dimensional (3D) surface area having one or more lesions, the method comprising: capturing a two-dimensional (2D) color image and a depth image of the 3D surface area, the depth image being an image in which each pixel has associated therewith a distance to a corresponding point on the surface being imaged; enhancing contrast of the 2D color image; segmenting the one or more lesions of the 2D color image into one or more segmented lesions; and calculating a 3D surface area of the one or more segmented lesions using enhanced contrast information from the 2D color image and distance information from the depth image. ### Claim 2 (depends on 1) The method of claim 1 further comprising: calibrating at least one parameter for processing the depth image. ### Claim 3 (depends on 1) The method of claim 1 further comprising: mapping pixels of the 2D color image to depth pixel of the depth image. ### Claim 4 (depends on 1) The method of claim 1, the enhancing contrast of the 2D color image further comprising: applying gamma correction to each of color channels of the 2D color image. ### Claim 5 (depends on 1) The method of claim 1, the enhancing contrast of the 2D color image further comprising: applying histogram equalization to the 2D color image. ### Claim 6 (depends on 1) The method of claim 1, the enhancing contrast of the 2D color image further comprising: dividing the 2D color image into a plurality of tiled images; applying the histogram equalization method to each of the tiled images; and combining the plurality of images into a combined two-image. ### Claim 7 (depends on 6) The method of claim 6, the enhancing contrast of the 2D color image further comprising: softening borders of each of the tiled images by applying the bilinear interpolation method. ### Claim 8 (depends on 1) The method of claim 1, further comprising: converting a first color space of the 2D color image into a second color space; adjusting a lightness channel of the 2D color image; and converting the second color space of the 2D color image to the first color space. ### Claim 9 (depends on 1) The method of claim 1, the segmenting the surface area further comprising: segmenting the one or more lesions of the 2D color image into one or more segmented lesions using the watershed transformation. ### Claim 10 (depends on 1) The method of claim 1, the calculating 3D area of the one or more lesions further comprising: calculating depth values for each pixel of the depth image; and calculating the 3D area of one or more lesions considering the depth values. ### Claim 11 (depends on 10) The method of claim 10, the calculating 3D area of the one or more lesions further comprising: calculating the 3D area of one or more lesions considering the gradient of the depth values. ### Claim 12 (depends on 11) The method of claim 11, further comprising: approximating the gradient of the depth values using a central difference of horizontally aligned pixels and vertically aligned pixels in the depth image. ### Claim 13 (depends on 1) The method of claim 1, wherein calculating the 3D surface area of the one or more segmented lesions further comprises: calculating the 3D surface area of the one or more segmented lesions using the following formulas: Lesion ⁢ ⁢ area ≅ ∑ ( i , j ) ∈ Lesion ⁢ D ⁡ ( i , j ) 2 ⁢ D x ⁡ ( i , j ) 2 + D y ⁡ ( i , j ) 2 + 1 ⁢ θ x ⁢ θ y D x ⁡ ( i , j ) = D ⁡ ( i + 1 , j ) - D ⁡ ( i - 1 , j ) 2 ⁢ D ⁡ ( i , j ) ⁢ θ x D y ⁡ ( i , j ) = D ⁡ ( i , j + 1 ) - D ⁡ ( i , j - 1 ) 2 ⁢ D ⁡ ( i , j ) ⁢ θ y where (i, j) are pixel coordinates, D(i, j) is the depth of pixel (i, j), and the camera has a field of view Fx×Fy degrees with a resolution of Rx×Ry pixels and each image pixel corresponds to a field of view of θx×θy degrees where θx=Fx/Rx and θy=Fy/Ry. ### Claim 14 (depends on 1) The method of claim 1 further comprising: measuring distance to each of pixels of the depth image by measuring the time of flight between the each of pixels and a camera capturing the depth image. ### Claim 15 (depends on 1) The method of claim 1, further comprising: converting an RGB color space of the 2D color image into a Lab color space; adjusting a lightness channel of the 2D color image; and converting the Lab color space of the 2D color image to the RGB color space. ### Claim 16 An apparatus for assessing a three-dimensional (3D) surface area having one or more lesions, the apparatus comprising: a camera capturing a two-dimensional (2D) color image and a depth image of the 3D surface area, the depth image being an image in which each pixel has associated therewith a distance to a corresponding point on the surface being imaged; and a processor and a memory, wherein the processor is configured to: enhance contrast of the 2D color image; segment the one or more lesions of the 2D color image into one or more segmented lesions; and calculate a 3D surface area of the one or more segmented lesions using enhanced contrast information from the 2D color image and distance information from the depth image. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the camera measures distance to each of pixels of the depth image by measuring the time of flight between the each of pixels and the camera. ### Claim 18 (depends on 16) The apparatus of claim 16, further comprising a user interface with which a user enhances the contrast of the 2D color image. ### Claim 19 (depends on 18) The apparatus of claim 18, wherein the user selects a type of histogram equalization to enhance the contrast of the 2D color image via the user interface.