# US 10,869,629 B2 · Deep learning for classification of swallows > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,869,629 B2 - Document ID: US10869629B2 - Status: Granted - Kind: B2 - Application number: 16/323,556 - Filed: 2017-08-15 - Published: 2019-06-06 - Granted: 2020-12-22 - Earliest claimed: 2016-08-17 - Assignees: University of Pittsburgh-Of die Commonwealth System of Higher Education - Applicants: University of Pittsburgh—of the Commonwealth System of Higher Education - Inventors: Ervin Sejdic; Amro El-Jaroudi; James Coyle; Joshua Dudik; Mingui Sun; Zhi-Hong Mao - Web page: https://patentgrep.com/patent/US10869629B2 ## CPC - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/08: Learning methods - A61B 5/4205: {Evaluating swallowing} - G06F 18/24: Classification techniques - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/047: Probabilistic or stochastic networks - G06N 3/0499: Feedforward networks - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/09: Supervised learning - G16H 40/63: for local operation - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A method of classifying a swallow of a subject includes obtaining vibration data that is based on and indicative of a number of vibrations resulting from the swallow, and using a computer implemented deep learning classifier to classify the swallow based on the vibration data. Also, a system for classifying a swallow of a subject includes a computing device implementing a deep learning classifier. The computing device includes a processor apparatus structured and configured to receive vibration data that is based on and indicative of a number of vibrations resulting from the swallow, and use the deep learning classifier to classify the swallow based on the vibration data. The deep learning classifier may comprise a single layer or a multi-layer Deep Belief network. ## Claims ### Claim 1 A method of classifying a swallow of a subject, comprising: obtaining vibration data, the vibration data being based on and indicative of a number of vibrations resulting from the swallow; and using a computer implemented deep learning classifier to classify the swallow based on the vibration data; wherein the vibration data is dual-axis data indicative of vibrations resulting from the swallow in a first axis and a second axis, wherein the vibration data comprises first axis vibration data indicative of the vibrations in the first axis and second axis vibration data indicative of the vibrations in the second axis, and wherein the deep learning classifier comprises a first Deep Belief network, a second Deep Belief network, and a combination network, wherein the first Deep belief network uses the first axis vibration data and the second Deep Belief network uses the second axis vibration data. ### Claim 2 (depends on 1) The method according to claim 1, wherein the first axis is aligned approximately parallel to the subject's cervical spine and the second axis is aligned approximately perpendicular to the subject's coronal plane. ### Claim 3 (depends on 1) The method according to claim 1, wherein the first Deep Belief network is a multi-layer Deep Belief network and the second Deep Belief network is a multi-layer Deep Belief network. ### Claim 4 (depends on 1) The method according to claim 1, wherein the vibration data is generated from data produced by an accelerometer. ### Claim 5 (depends on 4) The method according to claim 4, wherein the vibration data comprises a Fourier transform of processed accelerometer data, and wherein the processed accelerometer data is generated by processing the data produced by the accelerometer. ### Claim 6 (depends on 1) The method according to claim 1, wherein the first and second Deep Belief networks are trained using at least one of supervised learning and unsupervised learning. ### Claim 7 (depends on 6) The method according to claim 6, wherein the first and second Deep Belief networks are trained using supervised learning and unsupervised learning, wherein the unsupervised learning is conducted in a greedy, layerwise fashion by implementing a contrastive divergence algorithm. ### Claim 8 (depends on 7) The method according to claim 7, wherein the contrastive divergence algorithm is a stepwise contrastive divergence algorithm. ### Claim 9 (depends on 1) A computer program product, comprising a non-transitory computer usable medium having a computer readable program code embodied therein, the computer readable program code being adapted to be executed to implement a method of classifying a swallow of a subject as recited in claim 1. ### Claim 10 A system for classifying a swallow of a subject, comprising: a computing device implementing a deep learning classifier, the computing device having a processor apparatus structured and configured to: receive vibration data, the vibration data being based on and indicative of a number of vibrations resulting from the swallow; and use the deep learning classifier to classify the swallow based on the vibration data, wherein the vibration data is dual-axis data indicative of vibrations resulting from the swallow in a first axis and a second axis, wherein the vibration data comprises first axis vibration data indicative of the vibrations in the first axis and second axis vibration data indicative of the vibrations in the second axis, and wherein the deep learning classifier comprises a first Deep Belief network, a second Deep Belief network, and a combination network, wherein the first Deep belief network uses the first axis vibration data and the second Deep Belief network uses the second axis vibration data. ### Claim 11 (depends on 10) The system according to claim 10, further comprising an accelerometer coupled to the computing device, the accelerometer being structured to generate first axis accelerometer data based on the vibrations resulting from the swallow in the first axis and second axis accelerometer data based on the vibrations resulting from the swallow in the second axis, wherein the first axis vibration data is based on the first axis accelerometer data and the second axis vibration data is based on the second axis accelerometer data. ### Claim 12 (depends on 11) The system according to claim 11, wherein the first axis is aligned approximately parallel to the subject's cervical spine and the second axis is aligned approximately perpendicular to the subject's coronal plane. ### Claim 13 (depends on 10) The system according to claim 10, wherein the first Deep Belief network is a multi-layer Deep Belief network and the second Deep Belief network is a multi-layer Deep Belief network. ### Claim 14 (depends on 10) The system according to claim 10, wherein the vibration data comprises a Fourier transform of processed accelerometer data, and wherein the processed accelerometer data is generated by processing the data produced by the accelerometer. ### Claim 15 (depends on 10) The system according to claim 10, wherein the first and second Deep Belief networks are trained using at least one of supervised learning and unsupervised learning. ### Claim 16 (depends on 15) The system according to claim 15, wherein the first and second Deep Belief networks are trained using supervised learning and unsupervised learning, wherein the unsupervised learning is conducted in a greedy, layerwise fashion by implementing a contrastive divergence algorithm. ### Claim 17 (depends on 16) The system according to claim 16, wherein the contrastive divergence algorithm is a stepwise contrastive divergence algorithm. ### Claim 18 A system for classifying a swallow of a subject, comprising: a data acquisition component structured and configured to obtain vibration data, the vibration data being based on and indicative of a number of vibrations resulting from the swallow; and a classification component structured and configured to implement a deep learning classifier and use the deep learning classifier to classify the swallow based on the vibration data; wherein the vibration data is dual-axis data indicative of vibrations resulting from the swallow in a first axis and a second axis, wherein the vibration data comprises first axis vibration data indicative of the vibrations in the first axis and second axis vibration data indicative of the vibrations in the second axis, and wherein the deep learning classifier comprises a first Deep Belief network, a second Deep Belief network, and a combination network, wherein the first Deep belief network uses the first axis vibration data and the second Deep Belief network uses the second axis vibration data.