# US 10,872,408 B2 · Method and system for imaging and analysis of anatomical features > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,408 B2 - Document ID: US10872408B2 - Status: Granted - Kind: B2 - Application number: 14/702,570 - Filed: 2015-05-01 - Published: 2018-06-28 - Granted: 2020-12-22 - Earliest claimed: 2014-05-02 - Assignees: Etreat Medical Diagnostics Inc. - Applicants: Etreat Medical Diagnostics Inc. - Inventors: Nicholas MacKinnon; Fartash Vasefi; Manuka Shanil Gunasekara - Web page: https://patentgrep.com/patent/US10872408B2 ## CPC - G06T 7/0012: {Biomedical image inspection} - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - A61B 5/1071: {measuring angles, e.g. using goniometers (A61B5/1123 takes precedence)} - A61B 5/1079: {using optical or photographic means} - A61B 5/4528: {Joints (A61B5/4533, A61B5/4538 take precedence)} - A61B 5/6898: {Portable consumer electronic devices, e.g. music players, telephones, tablet computers} - G06F 18/23213: with fixed number of clusters, e.g. K-means clustering - G06T 5/80: Geometric correction - G06T 7/194: involving foreground-background segmentation - G06T 7/62: of area, perimeter, diameter or volume - G06V 10/757: {Matching configurations of points or features} - G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions} - G06V 40/107: {Static hand or arm} - G06V 40/11: {Hand-related biometrics; Hand pose recognition} - G06T 2207/10024: Color image - G06T 2207/30008: Bone - G06T 2207/30088: Skin; Dermal - G16H 30/40: for processing medical images, e.g. editing - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A method and system are provided for characterizing a portion of biological tissue. This invention comprises a smartphone and tablet deployable mobile medical application that uses device sensors, internet connectivity and cloud-based image processing to document and analyze physiological characteristics of hand arthritis. The application facilitates image capture and performs image processing that identifies hand fiduciary features and measures hand anatomical features to report and quantify the progress of arthritic disease. ## Claims ### Claim 1 A system for assessment and monitoring of joint abnormalities in a subject caused by disease or injury, wherein said system captures and analyzes an image of an affected anatomical region of said subject where one or more joints are located, said system comprising: (a) a mobile device comprising a camera for capturing a digital image of said affected anatomical region and a reference object of known dimensions positionable in a background relative to said affected anatomical region; (b) a mobile application executable by said mobile device and configured to collect sensor data relating to said digital image; and (c) processing software executable by a processor and configured to analyze said sensor data to determine anatomical measurements of said affected anatomical region relevant to said assessment and monitoring of joint abnormalities, said measurements comprising the dimensions of said one or more joints and the amount of angular deviation of anatomical structures at said one or more joints; wherein said amount of angular deviation of anatomical structures at said one or more joints comprises the amount of deviation between measurements of one or more of said anatomical structures in said subject having said joint abnormalities and expected values for said anatomical structures in an individual having normal anatomy. ### Claim 2 (depends on 1) The system as defined in claim 1, wherein said subject is a human being, said affected anatomical region is a hand or foot of said subject, and said joint abnormalities are caused by arthritis. ### Claim 3 (depends on 1) The system as defined in claim 1, wherein said mobile device is a smartphone or tablet computer. ### Claim 4 (depends on 3) The system as defined in claim 3, wherein said mobile device has Internet connectivity and wherein said processor is a cloud processing server. ### Claim 5 (depends on 1) The system as defined in claim 1, wherein said sensor data comprises at least one of image data, camera data, ambient light sensor data, orientation sensor data, geographic location data, and time and date data. ### Claim 6 (depends on 1) The system as defined in claim 1, wherein at least one of said mobile application and said processing software is configured to determine whether image capture conditions of said digital image are acceptable for further processing of said digital image. ### Claim 7 (depends on 1) The system as defined in claim 1, wherein said mobile application is configured to provide an interface enabling a user of said mobile device to input biographical and environmental data relevant to said subject, wherein said biographical and environmental data comprises at least one of biographical information, experiential measures of pain and movement, medication use, weather and regional demographics. ### Claim 8 (depends on 7) The system as defined in claim 7, comprising electronic health record software executable on a cloud database server for storing and retrieving said anatomical measurements and said biographical and environmental data specific to said subject. ### Claim 9 (depends on 1) The system as defined in claim 1, wherein said processing software is configured to produce a report indicating the condition of said one or more joints based on said anatomical measurements. ### Claim 10 (depends on 4) The system as defined in claim 4, comprising a cloud database server operable to store and retrieve reference measurements, wherein said processing software is configured to compare said anatomical measurements to said reference measurements. ### Claim 11 (depends on 4) The system as defined in claim 4 wherein said processing software executable on said cloud processing server is configured to perform operations to determine said anatomical measurements, said operations comprising: (a) analyzing said sensor data to make adjustments to said digital image, wherein said adjustments comprise correcting a distortion of said image selected from the group consisting of a spatial distortion, a color balance distortion, and a perspective distortion; (b) segmenting a calibrated image of said affected anatomical region from a background of said digital image to provide a segmented image; (c) determining a boundary of said affected anatomical region and anatomical fiduciary points in said segmented image; and (d) calculating said anatomical measurements based on said boundary and said fiduciary points. ### Claim 12 A method for assessment and monitoring of joint abnormalities in a subject caused by disease or injury, comprising: (a) capturing a digital image of an affected anatomical region of said subject where one or more joints are located and a reference object of known dimensions positionable in a background relative to said affected anatomical region, wherein said image is captured using a mobile device comprising a camera; (b) collecting sensor data from said mobile device relating to said digital image; and (c) analyzing said sensor data to determine anatomical measurements of said affected anatomical region relevant to said assessment and monitoring of joint abnormalities, said measurements comprising the dimensions of said one or more joints and the amount of angular deviation of anatomical structures at said one or more joints; wherein said amount of angular deviation of anatomical structures at said one or more joints comprises the amount of deviation between measurements of one or more of said anatomical structures in said subject having said joint abnormalities and expected values for said anatomical structures in an individual having normal anatomy. ### Claim 13 (depends on 12) The method of claim 12, wherein said subject is a human being, said affected anatomical region is a hand or foot of said subject, and wherein said joint abnormalities are caused by arthritis. ### Claim 14 (depends on 12) The method of claim 12, wherein said mobile device is a smartphone or tablet computer and wherein said method comprises transmitting said sensor data from said mobile device to a cloud processing server. ### Claim 15 (depends on 12) The method as defined in claim 12, wherein said sensor data comprises at least one of image data, camera data, ambient light sensor data, orientation sensor data, geographic location data, and time and date data. ### Claim 16 (depends on 12) The method as defined in claim 12, comprising determining, prior to analyzing said sensor data to determine said anatomical measurements, whether image capture conditions of said digital image are acceptable for further processing, and, if not, capturing a replacement digital image of said affected anatomical region and said reference object. ### Claim 17 (depends on 12) The method as defined in claim 12, comprising collecting biographical and environmental data relevant to said subject using said mobile device, wherein said biographical and environmental data comprises at least one of biographical information, experiential measures of pain and movement, medication use, weather and regional demographics. ### Claim 18 (depends on 17) The method as defined in claim 17, comprising providing a cloud database server for storing and retrieving an electronic health record specific to said subject comprising said anatomical measurements and said biographical and environmental data. ### Claim 19 (depends on 12) The method as defined in claim 12, comprising providing a report indicating the condition of said one or more joints based on said anatomical measurements. ### Claim 20 (depends on 12) The method as defined in claim 12, comprising comparing said anatomical measurements to reference measurements. ### Claim 21 (depends on 20) The method as defined in claim 20, wherein said reference measurements are prior measurements of said one or more joints of said subject, wherein said comparing enables monitoring of any changes to said one or more joints over time. ### Claim 22 (depends on 17) The method as defined in claim 17, comprising comparing said anatomical measurements and said biographical and environmental data to historical reference data, wherein said historical reference data comprises prior anatomical measurements and biographical and environmental data specific to said subject, wherein said comparing enables monitoring of any changes to said one or more joints relative to any changes to said biographical and environmental data over time. ### Claim 23 (depends on 22) The method as defined in claim 22, wherein said monitoring provides a measure of any correlations between said changes to said one or more joints and biographical or environmental factors, wherein said factors are selected from the group consisting of treatment received by said subject, medication received by said subject, physical activity of said subject, geographic location of said subject and weather conditions at said geographic location of said subject. ### Claim 24 (depends on 12) The method as defined in claim 12, wherein said analyzing comprises: (a) analyzing said sensor data to make adjustments to said digital image, wherein said adjustments comprise correcting a distortion of said image selected from the group consisting of a spatial distortion, a color balance distortion, and a perspective distortion; (b) segmenting a calibrated image of said affected anatomical region from a background of said digital image to provide a segmented image; (c) determining in said segmented image a boundary of said affected anatomical region and anatomical fiduciary points within said boundary; and (d) calculating said anatomical measurements based on said boundary and said fiduciary points. ### Claim 25 (depends on 24) The method as defined in claim 24, wherein said perspective distortion is corrected by determining the angle of said camera relative to said affected anatomical region and said reference object when said image is captured and adjusting said image to correct said perspective distortion. ### Claim 26 (depends on 24) The method as defined in claim 24, wherein said spatial distortion is corrected by measuring the size and/or shape of said reference object in said image to determine spatial measurements of said object, comparing said spatial measurements to known values for said object, and adjusting said image to correct said spatial distortion. ### Claim 27 (depends on 24) The method as defined in claim 24, wherein said color balance distortion is corrected by measuring the color of said reference object to determine color measurements, comparing said color measurements to known values for said object, and adjusting said image to correct said color distortion. ### Claim 28 (depends on 24) The method as defined in claim 24, wherein said segmenting comprises using K-means clustering of chrominance information for said image to segment pixels corresponding to said affected anatomical region from background image data. ### Claim 29 (depends on 12) The method of claim 12, wherein said reference object is a paper sheet of known dimensions and color. ### Claim 30 (depends on 16) The method of claim 16, wherein said determining whether said image capture conditions of said digital image are acceptable comprises analyzing said orientation sensor data to determine whether the orientation of said camera relative to said affected anatomical region is at a desirable angle. ### Claim 31 (depends on 16) The method of claim 16, wherein said determining whether image capture conditions of said digital image are acceptable comprises analyzing said ambient light sensor data to determine whether illumination of said affected anatomical region is suitable. ### Claim 32 (depends on 16) The method of claim 16, wherein said determining whether image capture conditions of said digital image are acceptable comprises determining if said reference object is within the field of view. ### Claim 33 (depends on 24) The method of claim 24, wherein said affected anatomical region is a hand and wherein said calculating said anatomical measurements comprises determining the length and location of segments of a finger extending between said one or more joints and determining the relative angle of said segments, wherein determining said relative angle provides a measure of joint breakdown causing deviation of said segments away from an anatomically normal orientation. ### Claim 34 A system for assessment and monitoring of joint abnormalities in a subject caused by disease or injury, wherein said system captures and analyzes an image of an affected anatomical region of said subject where one or more joints are located, said system comprising: (a) a mobile device comprising a camera for capturing a digital image of said affected anatomical region and a reference object of known dimensions positioned in a background independently of said affected anatomical region; (b) a mobile application executable by said mobile device and configured to collect sensor data relating to said digital image; and (c) processing software executable by a processor and configured to analyze said sensor data to determine anatomical measurements of said affected anatomical region relevant to said assessment and monitoring of joint abnormalities, said measurements comprising the dimensions of said one or more joints and the amount of angular deviation of anatomical structures at said one or more joints; wherein said amount of angular deviation of anatomical structures at said one or more joints comprises the amount of deviation between measurements of one or more of said anatomical structures in said subject having said joint abnormalities and expected values for said anatomical structures in an individual having normal anatomy.