# US 10,504,031 B2 · Method and apparatus for determining probabilistic context awareness of a mobile device user using a single sensor and/or multi-sensor data fusion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,504,031 B2 - Document ID: US10504031B2 - Status: Granted - Kind: B2 - Application number: 15/833,457 - Filed: 2017-12-06 - Published: 2018-04-05 - Granted: 2019-12-10 - Earliest claimed: 2015-02-26 - Assignees: STMicroelectronics International N.V.; STMicroelectronics S.r.l. - Applicants: STMicroelectronics International N.V.; STMicroelectronics, Inc. - Inventors: Mahesh Chowdhary; Arun Kumar; Ghanapriya Singh; Kashif R. J. Meer; Indra Narayan Kar; Rajendar Bahl - Web page: https://patentgrep.com/patent/US10504031B2 ## CPC - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06F 16/2455: Query execution - H04M 1/72454: according to context-related or environment-related conditions - H04W 4/02: Services making use of location information - H04W 4/38: for collecting sensor information - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract An electronic device described herein includes a sensing unit having at least one sensor to acquire sensing data. An associated computing device extracts sensor specific features from the sensing data, and generates a motion activity vector, a voice activity vector, and a spatial environment vector as a function of the sensor specific features. The motion activity vector, voice activity vector, and spatial environment vector are processed to determine a base level context of the electronic device relative to its surroundings, with the base level context having aspects each based on the motion activity vector, voice activity vector, and spatial environment vector. Meta level context of the electronic device relative to its surroundings is determined as a function of the base level context, with the meta level context being at least one inference made from at least two aspects of the plurality of aspects of the base level context. ## Claims ### Claim 1 An electronic device, comprising: a sensing unit comprising at least one sensor and being configured to acquire sensing data; and a computing device configured to: extract sensor specific features from the sensing data; generate a motion activity vector, a voice activity vector, and a spatial environment vector as a function of the sensor specific features; process the motion activity vector, voice activity vector, and spatial environment vector so as to determine a base level context of the electronic device relative to its surroundings, the base level context having a plurality of aspects, wherein each aspect is based on at least one of the motion activity vector, voice activity vector, and spatial environment vector; wherein one aspect of the plurality of aspects of the base level context is a mode of locomotion of a user carrying the electronic device, and another aspect of the plurality of aspects of the base level context is a nature of biologically generated sounds within audible distance of the user or a nature of physical space around the user; and determine meta level context of the electronic device relative to its surroundings as a function of the base level context, wherein the meta level context comprises at least one inference made from at least two aspects of the plurality of aspects of the base level context. ### Claim 2 (depends on 1) The electronic device of claim 1, wherein aspects of the base level context based on the motion activity vector are mutually exclusive of one another; wherein aspects of the base level context based on the voice activity vector are mutually exclusive of one another; and wherein aspects of the base level context based on the spatial environment vector are mutually exclusive of one another. ### Claim 3 (depends on 1) The electronic device of claim 1, wherein the mode of locomotion of the user carrying the electronic device is based upon the motion activity vector, the nature of biologically generated sounds within audible distance of the user is based on the voice activity vector, and the nature of physical space around the user is based upon the spatial environment vector. ### Claim 4 (depends on 1) The electronic device of claim 1, wherein the computing device is further configured to facilitate performance of at least one contextual function of the electronic device as a function of the meta level context of the electronic device. ### Claim 5 (depends on 1) The electronic device of claim 1, wherein the mode of locomotion of the user comprises one of the user being stationary, walking, going up stairs, going down stairs, jogging, cycling, climbing, using a wheelchair, and riding in or on a vehicle; wherein the nature of the biologically generated sounds comprises one of a telephone conversation engaged in by the user, a multiple party conversation engaged in by the user, the user speaking, another party speaking, background conversation occurring around the user, and an animal making sounds; and wherein the nature of the physical space around the user comprises an office environment, a home environment, a shopping mall environment, a street environment, a stadium environment, a restaurant environment, a bar environment, a beach environment, a nature environment, a temperature of the physical space, a barometric pressure of, and a humidity of the physical space. ### Claim 6 (depends on 1) The electronic device of claim 1, wherein the computing device is configured to process the motion activity vector, voice activity vector, and spatial environment vector by: generating a motion activity posteriorgram as a function of the motion activity vector, the motion activity posteriorgram representing a probability of each element of the motion activity vector as a function of time; generating a voice activity posteriorgram as a function of the voice activity vector, the voice activity posteriorgram representing a probability of each element of the voice activity vector as a function of time; and generating a spatial environment posteriorgram as a function of the spatial environment vector, the spatial environment posteriorgram representing a probability of each element of the spatial environment vector as a function of time. ### Claim 7 (depends on 6) The electronic device of claim 6, wherein a sum of each probability of the motion activity posteriorgram at any given time equals one; wherein a sum of each probability of the voice activity posteriorgram at any given time equals one; and wherein a sum of each probability of the spatial environment posteriorgram at any given time equals one. ### Claim 8 (depends on 1) The electronic device of claim 1, wherein the sensing unit consists essentially of one sensor. ### Claim 9 (depends on 1) The electronic device of claim 1, wherein the sensing unit comprises a plurality of sensors; and wherein the motion activity vector, voice activity vector, and spatial environment vector are generated as a function of a fusion of the sensor specific features. ### Claim 10 (depends on 9) The electronic device of claim 9, wherein the plurality of sensors comprise at least two sensors selected from the group consisting of an accelerometer, pressure sensor, microphone, gyroscope, magnetometer, GPS unit, and barometer. ### Claim 11 (depends on 1) The electronic device of claim 1, further comprising a printed circuit board (PCB) having at least one conductive trace thereon; further comprising a system on chip (SoC) mounted on the PCB and electrically coupled to the at least one conductive trace; and wherein the computing device comprises a sensor chip mounted on the PCB in a spaced apart relation with the SoC and electrically coupled to the at least one conductive trace such that the sensor chip and SoC are electrically coupled; and wherein the sensor chip comprises an micro-electromechanical system (MEMS) sensing unit, and a control circuit configured to perform the extracting, generating, processing, and determining. ### Claim 12 An electronic device, comprising: a computing device configured to: extract sensor specific features from sensing data; generate a motion activity vector, a voice activity vector, and a spatial environment vector as a function of the sensor specific features; process the motion activity vector, voice activity vector, and spatial environment vector so as to determine a base level context of the electronic device relative to its surroundings, the base level context having a plurality of aspects each based on at least one of the motion activity vector, voice activity vector, and spatial environment vector; wherein at least one aspect of the plurality of aspects of the base level context is one of: a mode of locomotion of a user carrying the electronic device, a nature of biologically generated sounds within audible distance of the user, or a nature of physical space around the user; and determine meta level context of the electronic device relative to its surroundings as a function of the base level context, wherein the meta level context comprises at least one inference made from at least two aspects of the plurality of aspects of the base level context. ### Claim 13 (depends on 12) The electronic device of claim 12, wherein aspects of the base level context based on the motion activity vector are mutually exclusive of one another; wherein aspects of the base level context based on the voice activity vector are mutually exclusive of one another; and wherein aspects of the base level context based on the spatial environment vector are mutually exclusive of one another. ### Claim 14 (depends on 12) The electronic device of claim 12, wherein the computing device is configured to process the motion activity vector, voice activity vector, and spatial environment vector by: generating a motion activity posteriorgram as a function of the motion activity vector, the motion activity posteriorgram representing a probability of each element of the motion activity vector as a function of time; generating a voice activity posteriorgram as a function of the voice activity vector, the voice activity posteriorgram representing a probability of each element of the voice activity vector as a function of time; and generating a spatial environment posteriorgram as a function of the spatial environment vector, the spatial environment posteriorgram representing a probability of each element of the spatial environment vector as a function of time. ### Claim 15 (depends on 14) The electronic device of claim 14, wherein a sum of each probability of the motion activity posteriorgram at any given time equals one; wherein a sum of each probability of the voice activity posteriorgram at any given time equals one; and wherein a sum of each probability of the spatial environment posteriorgram at any given time equals one. ### Claim 16 An electronic device, comprising: a printed circuit board (PCB) having at least one conductive trace thereon; a system on chip (SoC) mounted on the PCB and electrically coupled to the at least one conductive trace; and a sensor chip mounted on the PCB in a spaced apart relation with the SoC and electrically coupled to the at least one conductive trace such that the sensor chip and SoC are electrically coupled, and configured to acquire sensing data; wherein the sensor chip comprises: a micro-electromechanical system (MEMS) sensing unit; an embedded processing node configured to: preprocess the sensing data, extract sensor specific features from the sensing data, generate a motion activity posteriorgram, a voice activity posteriorgram, and a spatial environment posteriorgram as a function of the sensor specific features, process the motion activity posteriorgram, voice activity posteriorgram, and spatial environment posteriorgram so as to determine a base level context of the electronic device relative to its surroundings, the base level context having a plurality of aspects, wherein a first aspect of the plurality of aspects of the base level context is determined based upon the motion activity posteriorgram, a second aspect of the plurality of aspects of the base level context is determined based upon the voice activity posteriorgram, and a third aspect of the plurality of aspects of the base level context is determined based upon the spatial environment posteriorgram, and determine meta level context of the electronic device relative to its surroundings as a function of the base level context and at least one known pattern, wherein the meta level context comprises at least one inference made from at least two aspects of the plurality of aspects of the base level context. ### Claim 17 (depends on 16) The electronic device of claim 16, further comprising at least one additional sensor external to the MEMS sensing unit; wherein the SoC is configured to acquire additional data from the at least one additional sensor; wherein the embedded processing node is further configured to receive the additional data from the SoC and to also extract the sensor specific features from the additional data. ### Claim 18 (depends on 16) The electronic device of claim 16, wherein the embedded processing node is configured to generate the motion activity posteriorgram, voice activity posteriorgram, and spatial environment posteriorgram to represent a probability of each element of a motion activity vector, a voice activity vector, and a spatial environment vector as a function of time, respectively. ### Claim 19 (depends on 16) The electronic device of claim 16, wherein a sum of each probability of the motion activity posteriorgram at any given time equals one; wherein a sum of each probability of the voice activity posteriorgram at any given time equals one; and wherein a sum of each probability of the spatial environment posteriorgram at any given time equals one. ### Claim 20 (depends on 16) The electronic device of claim 16, wherein the sensor chip consists essentially of one MEMS sensing unit. ### Claim 21 (depends on 16) The electronic device of claim 16, wherein the sensor chip comprises a plurality of MEMS sensing units; and wherein the motion activity posteriorgram, voice activity posteriorgram, and spatial environment posteriorgram are generated as a function of a fusion of the sensor specific features. ### Claim 22 A method of operating an electronic device, the method comprising: acquiring sensing data from a sensing unit; extracting sensor specific features from the sensing data, using a computing device; generating a motion activity vector, a voice activity vector, and a spatial environment vector as a function of the sensor specific features, using the computing device; processing the motion activity vector, voice activity vector, and spatial environment vector so as to determine a base level context of the electronic device relative to its surroundings, the base level context having a plurality of aspects based on the motion activity vector, voice activity vector, and spatial environment vector, using the computing device; wherein one aspect of the plurality of aspects of the base level context is a mode of locomotion of a user carrying the electronic device, and another aspect of the plurality of aspects of the base level context is a nature of biologically generated sounds within audible distance of the user or a nature of physical space around the user; and determining meta level context of the electronic device relative to its surroundings as a function of the base level context, wherein the meta level context comprises at least one inference made from at least two aspects of the plurality of aspects of the base level context, using the computing device. ### Claim 23 (depends on 22) The method of claim 22, wherein processing the motion activity vector, voice activity vector, and spatial environment vector comprises: generating a motion activity posteriorgram as a function of the motion activity vector, the motion activity posteriorgram representing a probability of each element of the motion activity vector as a function of time; generating a voice activity posteriorgram as a function of the voice activity vector, the voice activity posteriorgram representing a probability of each element of the voice activity vector as a function of time; and generating a spatial environment posteriorgram as a function of the spatial environment vector, the spatial environment posteriorgram representing a probability of each element of the spatial environment vector as a function of time.