# US 2019/0318228 A1 · Apparatus and method for statistical memory network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318228 A1 - Document ID: US20190318228A1 - Status: Published application - Kind: A1 - Application number: 16/260,637 - Filed: 2019-01-29 - Published: 2019-10-17 - Earliest claimed: 2018-04-11 - Assignees: Electronics and Telecommunications Research Institute - Applicants: Electronics and Telecommunications Research Institute - Inventors: Hyun Woo KIM; Ho Young JUNG; Jeon Gue PARK; Yun Keun LEE - Web page: https://patentgrep.com/patent/US20190318228A1 ## CPC - G06N 3/047: Probabilistic or stochastic networks - G06F 3/0604: {Improving or facilitating administration, e.g. storage management} - G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling} - G06F 3/0673: {Single storage device} - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/0499: Feedforward networks - G06N 3/08: Learning methods - H03H 17/0257: {KALMAN filters} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H03H: IMPEDANCE NETWORKS, e.g. RESONANT CIRCUITS; RESONATORS (waveguides, resonators, lines or other devices of the waveguide type H01P) ## Abstract Provided are an apparatus and method for a statistical memory network. The apparatus includes a stochastic memory, an uncertainty estimator configured to estimate uncertainty information of external input signals from the input signals and provide the uncertainty information of the input signals, a writing controller configured to generate parameters for writing in the stochastic memory using the external input signals and the uncertainty information and generate additional statistics by converting statistics of the external input signals, a writing probability calculator configured to calculate a probability of a writing position of the stochastic memory using the parameters for writing, and a statistic updater configured to update stochastic values composed of an average and a variance of signals in the stochastic memory using the probability of a writing position, the parameters for writing, and the additional statistics. ## Claims ### Claim 1 An apparatus for a statistical memory network, the apparatus comprising: a stochastic memory; an uncertainty estimator configured to estimate uncertainty information of external input signals from the input signals and provide the uncertainty information of the input signals; a writing controller configured to generate parameters for writing in the stochastic memory using the external input signals and the uncertainty information and generate additional statistics by converting statistics of the external input signals; a writing probability calculator configured to calculate a probability of a writing position of the stochastic memory using the parameters for writing; and a statistic updater configured to update stochastic values of signals stored at a position of the stochastic memory corresponding to the probability of a writing position with an average and a covariance of the additional statistics. ### Claim 2 (depends on 1) The apparatus of claim 1, further comprising: a reading controller configured to generate parameters for reading the stochastic memory using the external input signals and the uncertainty information provided by the uncertainty estimator; a reading probability calculator configured to calculate a probability of a reading position of the stochastic memory using the parameters for reading provided by the reading controller; and a reading statistic determiner configured to acquire stochastic values composed of an average and a variance of signals stored at a position of the stochastic memory corresponding to the calculated probability of a reading position, wherein the reading controller generates an external output signal using the stochastic values acquired by the reading statistic determiner. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the uncertainty estimator uses a Kalman filter. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the uncertainty information of the external input signals includes an average and a variance of noise. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the writing controller uses a neural network configured by stacking multiple layers of recurrent neural networks having a long short-term memory (LSTM) structure and then causing a linear transformation in a final layer. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the writing probability calculator calculates a ratio of a normal distribution probability at a stochastic memory position as the probability of a writing position using an average and a covariance of the stochastic memory and an average key and a covariance key of the parameters for writing. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the statistic updater acquires an updated average of the stochastic memory by deleting an existing average of the stochastic memory at a ratio of a product of a deletion weight of the parameters for writing and a deletion constant of the parameters for writing and adding a product of the average of the additional statistics and the probability of a writing position, deletes an existing variance of the stochastic memory as much as a ratio of a product of a square of a difference between the deletion weight of the parameters for writing and the existing average of the stochastic memory and the deletion constant of the parameters for writing, and acquires an updated variance of the stochastic memory by adding a product of the probability of a writing position and a sum of a square of a difference between the average of the additional statistics and the existing average of the stochastic memory and the variance of the additional statistics. ### Claim 8 (depends on 2) The apparatus of claim 2, wherein the reading probability calculator calculates a ratio of a normal distribution probability at a stochastic memory position as the probability of a reading position using an average and a covariance of the stochastic memory and an average key and a covariance key of the parameters for reading. ### Claim 9 (depends on 2) The apparatus of claim 2, wherein the reading statistic determiner uses an average of reading stochastic values, which is an expected value obtained by linearly combining an average key of the parameters for reading and an average of the stochastic memory according to a ratio of a reciprocal of a covariance key of the parameters for reading to a reciprocal of a covariance of the stochastic memory, and a covariance of the reading stochastic values, which is an expected value of an harmonic average of the covariance key of the parameters for reading and the covariance of the stochastic memory. ### Claim 10 A method for a statistical memory network, the method comprising: estimating, by an uncertainty estimator, uncertainty information of external input signals from the input signals and providing the uncertainty information of the input signals; generating, by a writing controller, parameters for writing in a stochastic memory using the external input signals and the uncertainty information and generating additional statistics by converting statistics of the external input signals; calculating, by a writing probability calculator, a probability of a writing position of the stochastic memory using the parameters for writing; and updating, by a statistic updater, stochastic values of signals stored at a position of the stochastic memory corresponding to the probability of a writing position with an average and a covariance of the additional statistics. ### Claim 11 (depends on 10) The method of claim 10, further comprising: generating, by a reading controller, parameters for reading the stochastic memory using the external input signals and the uncertainty information provided by the uncertainty estimator; calculating, by a reading probability calculator, a probability of a reading position of the stochastic memory using the parameters for reading provided by the reading controller; acquiring, by a reading statistic determiner, stochastic values composed of an average and a variance of signals stored at a position of the stochastic memory corresponding to the calculated probability of a reading position; and generating, by the reading controller, an external output signal using the stochastic values acquired by the reading statistic determiner. ### Claim 12 (depends on 10) The method of claim 10, wherein the estimating of the uncertainty information comprises using a Kalman filter. ### Claim 13 (depends on 10) The method of claim 10, wherein the uncertainty information of the external input signals includes an average and a variance of noise. ### Claim 14 (depends on 11) The method of claim 11, wherein the generating of the external output signal comprises using a neural network configured by stacking multiple layers of recurrent neural networks having a long short-term memory (LSTM) structure and then causing a linear transformation in a final layer. ### Claim 15 (depends on 10) The method of claim 10, wherein the calculating of the probability of a writing position of the stochastic memory comprises calculating a ratio of a normal distribution probability at a stochastic memory position as the probability of a writing position using an average and a covariance of the stochastic memory and an average key and a covariance key of the parameters for writing. ### Claim 16 (depends on 10) The method of claim 10, wherein the updating of the stochastic values comprises: acquiring an updated average of the stochastic memory by deleting an existing average of the stochastic memory at a ratio of a product of a deletion weight of the parameters for writing and a deletion constant of the parameters for writing and adding a product of the average of the additional statistics and the probability of a writing position; deleting an existing variance of the stochastic memory as much as a ratio of a product of a square of a difference between the deletion weight of the parameters for writing and the existing average of the stochastic memory and the deletion constant of the parameters for writing; and acquiring an updated variance of the stochastic memory by adding a product of the probability of a writing position and a sum of a square of a difference between the average of the additional statistics and the existing average of the stochastic memory and a variance of the additional statistics. ### Claim 17 (depends on 11) The method of claim 11, wherein the calculating of the probability of the reading position of the stochastic memory comprises calculating a ratio of a normal distribution probability at a stochastic memory position as the probability of a reading position using an average and a covariance of the stochastic memory and an average key and a covariance key of the parameters for reading. ### Claim 18 (depends on 11) The method of claim 11, wherein the generating of the external output signal comprises using an average of reading stochastic values, which is an expected value obtained by linearly combining an average key of the parameters for reading and an average of the stochastic memory according to a ratio of a reciprocal of a covariance key of the parameters for reading to a reciprocal of a covariance of the stochastic memory, and a covariance of the reading stochastic values, which is an expected value of an harmonic average of the covariance key of the parameters for reading and the covariance of the stochastic memory.