# US 10,771,728 B2 · Electronic device, control method, and recording medium for displaying images based on determined state > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,728 B2 - Document ID: US10771728B2 - Status: Granted - Kind: B2 - Application number: 16/269,535 - Filed: 2019-02-06 - Published: 2019-08-08 - Granted: 2020-09-08 - Earliest claimed: 2018-02-07 - Assignees: KYOCERA Corporation - Applicants: KYOCERA Corporation - Inventors: Shigeki Tanabe; Yasuhiro Ueno; Hideki Morita; Isao Masuike; Koutaro Yamauchi; Manabu Sakuma; Kenji Shimada - Web page: https://patentgrep.com/patent/US10771728B2 ## CPC - G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer} - H04N 5/44504: {Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits} - G06F 1/1684: {Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675} - G06F 1/1686: {the I/O peripheral being an integrated camera} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - H04N 23/63: by using electronic viewfinders - G01K 1/14: Supports; Fastening devices; Arrangements for mounting thermometers in particular locations - G01L 19/0092: {Pressure sensor associated with other sensors, e.g. for measuring acceleration or temperature (G01L9/025, G01L9/045, G01L9/065, G01L9/085, G01L9/105, G01L9/125, G01L19/02, G01L19/04 take precedence; measuring two or more variable G01D21/02; temperature sensors with pressure compensation G01K1/26)} - G01V 7/00: Measuring gravitational fields or waves; Gravimetric prospecting or detecting - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G01K: MEASURING TEMPERATURE; MEASURING QUANTITY OF HEAT; THERMALLY-SENSITIVE ELEMENTS NOT OTHERWISE PROVIDED FOR (radiation pyrometry G01J5/00) - G01L: MEASURING FORCE, STRESS, TORQUE, WORK, MECHANICAL POWER, MECHANICAL EFFICIENCY, OR FLUID PRESSURE (weighing G01G) - G01V: GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS (means for indicating the location of accidentally buried, e.g. snow-buried, persons A63B29/02) ## Abstract An electronic device includes a display (display unit), a camera (imaging unit), sensors, and a controller. The controller determines the state of the electronic device on the basis of a detection result of a first sensor among the sensors. The controller causes the display to display, when the determined state is a first state, a first overlay image in which first sensor information based on a detection result of a second sensor among the sensors is overlaid on an image captured by the camera. The controller causes the display to display, when the determined state is a second state, a second overlay image in which second sensor information based on the detection result of the second sensor among the plurality of sensors is overlaid on the image captured by the camera. The second sensor information differs from the first sensor information. ## Claims ### Claim 1 An electronic device, comprising: a display; a camera; a plurality of sensors; and a controller configured to determine a state of the electronic device on the basis of a detection result of a first sensor among the plurality of sensors, cause the display to display, when the determined state is a first state, a first overlay image in which first sensor information based on a detection result of a second sensor among the plurality of sensors is overlaid on an image captured by the camera, and cause the display to display, when the determined state is a second state, a second overlay image in which second sensor information based on the detection result of the second sensor among the plurality of sensors is overlaid on the image captured by the camera, wherein the second sensor information differs from the first sensor information, the first sensor information and the second sensor information include numerical information and text information based on the detection result of the second sensor, the text information differs between the first sensor information and the second sensor information, and the controller is configured to determine whether the electronic device is underwater on the basis of the detection result of the first sensor, cause the display to display the first overlay image when the electronic device is determined not to be underwater, and cause the display to display the second overlay image when the electronic device is determined to be underwater. ### Claim 2 (depends on 1) The electronic device of claim 1, wherein the first sensor comprises a number of sensors. ### Claim 3 (depends on 1) The electronic device of claim 1, wherein the second sensor comprises a number of sensors. ### Claim 4 (depends on 1) The electronic device of claim 1, wherein the first sensor comprises a sensor identical to a sensor comprised in the second sensor. ### Claim 5 (depends on 1) The electronic device of claim 1, wherein the second sensor comprises a temperature sensor. ### Claim 6 (depends on 1) The electronic device of claim 1, wherein the second sensor comprises a barometric pressure sensor. ### Claim 7 (depends on 1) The electronic device of claim 1, wherein the numerical information in the first sensor information and the second sensor information are calculated on the basis of the detection result of the second sensor, and a calculation method of the numerical information differs between the first sensor information and the second sensor information. ### Claim 8 (depends on 3) The electronic device of claim 3, wherein the second sensor comprises a temperature sensor. ### Claim 9 (depends on 8) The electronic device of claim 8, wherein the second sensor further comprises a barometric pressure sensor. ### Claim 10 A method for controlling an electronic device comprising a display, a camera, a plurality of sensors, and a controller, the method comprising: determining, using the controller, a state of the electronic device on the basis of a detection result of a first sensor among the plurality of sensors; causing the display to display, using the controller, when the determined state is a first state, a first overlay image in which first sensor information based on a detection result of a second sensor among the plurality of sensors is overlaid on an image captured by the camera; and causing the display to display, using the controller, when the determined state is a second state, a second overlay image in which second sensor information based on the detection result of the second sensor among the plurality of sensors is overlaid on the image captured by the camera, wherein the second sensor information differs from the first sensor information, the first sensor information and the second sensor information include numerical information and text information based on the detection result of the second sensor, the text information differs between the first sensor information and the second sensor information, and the method further comprises determining whether the electronic device is underwater on the basis of the detection result of the first sensor, causing the display to display the first overlay image when the electronic device is determined not to be underwater, and causing the display to display the second overlay image when the electronic device is determined to be underwater. ### Claim 11 A non-transitory computer-readable recording medium including computer program instructions to be executed by an electronic device comprising a display, a camera, a plurality of sensors, and a controller, the instructions causing the controller to: determine a state of the electronic device on the basis of a detection result of a first sensor among the plurality of sensors; cause the display to display, when the determined state is a first state, a first overlay image in which first sensor information based on a detection result of a second sensor among the plurality of sensors is overlaid on an image captured by the camera; and cause the display to display, when the determined state is a second state, a second overlay image in which second sensor information based on the detection result of the second sensor among the plurality of sensors is overlaid on the image captured by the camera, wherein the second sensor information differs from the first sensor information, the first sensor information and the second sensor information include numerical information and text information based on the detection result of the second sensor, the text information differs between the first sensor information and the second sensor information, and the instructions, when executed by the controller, cause the controller to further: determine whether the electronic device is underwater on the basis of the detection result of the first sensor, cause the display to display the first overlay image when the electronic device is determined not to be underwater, and cause the display to display the second overlay image when the electronic device is determined to be underwater. ### Claim 12 An electronic device, comprising: a display; a camera; a plurality of sensors; and a controller configured to determine a state of the electronic device on the basis of a detection result of a first sensor among the plurality of sensors, cause the display to display, when the determined state is a first state, a first overlay image in which first sensor information based on a detection result of a second sensor among the plurality of sensors is overlaid on an image captured by the camera, and cause the display to display, when the determined state is a second state, a second overlay image in which second sensor information based on a detection result of a third sensor among the plurality of sensors is overlaid on the image captured by the camera, wherein the controller is configured to determine whether the electronic device is underwater on the basis of the detection result of the first sensor, cause the display to display the first overlay image when the electronic device is determined not to be underwater, and cause the display to display the second overlay image when the electronic device is determined to be underwater. ### Claim 13 (depends on 12) The electronic device of claim 12, wherein the first sensor comprises a number of sensors. ### Claim 14 (depends on 12) The electronic device of claim 12, wherein the second sensor comprises a number of sensors. ### Claim 15 (depends on 12) The electronic device of claim 12, wherein the third sensor comprises a number of sensors. ### Claim 16 (depends on 12) The electronic device of claim 12, wherein at least one of the first sensor, the second sensor, and the third sensor comprises a sensor identical to a sensor comprised in another one of the first sensor, the second sensor, and the third sensor. ### Claim 17 (depends on 12) The electronic device of claim 12, wherein the second sensor comprises an acceleration sensor or a position sensor, and the third sensor comprises a temperature sensor or a barometric pressure sensor. ### Claim 18 A method for controlling an electronic device comprising a display, a camera, a plurality of sensors, and a controller, the method comprising: determining, using the controller, a state of the electronic device on the basis of a detection result of a first sensor among the plurality of sensors; causing the display to display, using the controller, when the determined state is a first state, a first overlay image in which first sensor information based on a detection result of a second sensor among the plurality of sensors is overlaid on an image captured by the camera; and causing the display to display, using the controller, when the determined state is a second state, a second overlay image in which second sensor information based on a detection result of a third sensor among the plurality of sensors is overlaid on the image captured by the camera, wherein the method further comprises determining whether the electronic device is underwater on the basis of the detection result of the first sensor, causing the display to display the first overlay image when the electronic device is determined not to be underwater, and causing the display to display the second overlay image when the electronic device is determined to be underwater. ### Claim 19 A non-transitory computer-readable recording medium including computer program instructions to be executed by an electronic device comprising a display, a camera, a plurality of sensors, and a controller, the instructions causing the controller to: determine a state of the electronic device on the basis of a detection result of a first sensor among the plurality of sensors; cause the display to display, when the determined state is a first state, a first overlay image in which first sensor information based on a detection result of a second sensor among the plurality of sensors is overlaid on an image captured by the camera; and cause the display to display, when the determined state is a second state, a second overlay image in which second sensor information based on a detection result of a third sensor among the plurality of sensors is overlaid on the image captured by the camera, wherein the instructions, when executed by the controller, cause the controller to further: determine whether the electronic device is underwater on the basis of the detection result of the first sensor, cause the display to display the first overlay image when the electronic device is determined not to be underwater, and cause the display to display the second overlay image when the electronic device is determined to be underwater.