# US 10,768,714 B2 · Electronic device and a method for controlling the functions of the electronic device as well as program product for implementing the method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,768,714 B2 - Document ID: US10768714B2 - Status: Granted - Kind: B2 - Application number: 16/167,042 - Filed: 2018-10-22 - Published: 2019-04-25 - Granted: 2020-09-08 - Earliest claimed: 2004-08-17 - Assignees: Conversant Wireless Licensing S.a r.l. - Applicants: Conversant Wireless Licensing S.a.r.l. - Inventors: Mikko Blomqvist; Tommi Laine; Miia Onkalo - Web page: https://patentgrep.com/patent/US10768714B2 ## CPC - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 1/1626: {with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]} - G06F 1/1686: {the I/O peripheral being an integrated camera} - G06F 3/005: {Input arrangements through a video camera} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 2200/1637: Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer - H04M 1/72427: for supporting games or graphical animations - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract The invention relates to an electronic device, which includes a display component, in which at least one controllable element is arranged to be visualized in its entirety, the control of which element is arranged to be based on determining a change (M) relating to the attitude or position of the device and camera means arranged to form image frames (IMAGE1, IMAGE2), and in which, in order to control the functions of the device, such as, for example, to control the element, the change is arranged to be determined, using optical data-flow algorithms, from the image frames formed by the camera means, which image frames are arranged to be compared to each other in a set manner, and on the basis of which comparison the said change is arranged to be determined. The image frames are arranged to be segmented in a selected manner, from which at least one movement vector (SR, SL, SUp, SDown) determining the said change is arranged to be determined from at least some of the segments (SEGMENT1, SEGMENT2). ## Claims ### Claim 1 A method comprising: capturing, via a camera unit of a device, at least two images; dividing a first image of the at least two images into a first plurality of segments, each of the first plurality of segments having a set shape and size; selecting a fixed point from at least one segment of the first plurality of segments of the first image; determining a first location coordinate of the selected fixed point from the at least one segment of the first plurality of segments of the first image; dividing a second image of the at least two images into a second plurality of segments, each of the second plurality of segments having the set shape and size, the second image being the next image following the first image of the at least two images; determining a second location coordinate of the selected fixed point from at least one segment of the second plurality of segments of the second image; calculating a movement vector of the selected fixed point between the determined location coordinate the second determined location coordinate; determining at least one of an attitude change and a position change affecting the device based on at least the movement vector; and controlling functions of the device based on at least one of the attitude change and the position change. ### Claim 2 (depends on 1) The method according to claim 1, further comprising adjusting an image frequency of the camera unit to at least a movement speed of the device. ### Claim 3 (depends on 1) The method according to claim 1, wherein the device is operable in a plurality of operating situations, wherein one of the plurality of operating situations is defined based on a movement speed of the device. ### Claim 4 (depends on 1) The method according to claim 1, wherein at least one of the attitude change and the position change is three-dimensional. ### Claim 5 (depends on 1) The method of claim 1, wherein the camera unit comprises at least two cameras on opposite sides of the device. ### Claim 6 (depends on 1) The method according to claim 1, further comprising defining a mean value for at least the movement vector. ### Claim 7 (depends on 1) The method of claim 1, further comprising adjusting a determination frequency of at least one of the attitude change and the position change according to an operating situation of the device. ### Claim 8 (depends on 1) The method of claim 1, wherein the first and second images are essentially consecutive. ### Claim 9 A device comprising: a display; a camera unit; a memory including program code; a processor; and the memory and the program code being configured with the processor to cause the device to: capture, via the camera unit of a device, at least two images; divide a first image of the at least two images into a first plurality of segments, each of the first plurality of segments having a set shape and size; select a fixed point from at least one segment of the first plurality of segments of the first image; determine a first location coordinate of the selected fixed point from the at least one segment of the first plurality of segments of the first image; divide a second image of the at least two images into a second plurality of segments, each of the second plurality of segments having the set shape and size, the second image being the next image following the first image of the at least two images; determine a second location coordinate of the selected fixed point from at least one segment of the second plurality of segments of the second image; calculate a movement vector of the selected fixed point between the first determined location coordinate and the second determined location coordinate; determine at least one of an attitude change and a position change relative to at least the movement vector; and control a visual controllable element on the display based on at least one of the attitude change and the position change. ### Claim 10 (depends on 9) The device of claim 9, wherein the camera unit includes at least two cameras on opposite sides of the device. ### Claim 11 (depends on 9) The device of claim 9, wherein at least one of the attitude change and the position change is three-dimensional. ### Claim 12 (depends on 9) The device of claim 9, wherein the memory and the program code are further configured with the processor to cause the device to: adjust an image frequency of the camera unit to at least a movement speed of the device. ### Claim 13 (depends on 9) The device of claim 9, wherein the first and second images are consecutive. ### Claim 14 (depends on 9) The device of claim 9, wherein the memory and the program code are further configured with the processor to cause the device to: define a mean value for at least the movement vector. ### Claim 15 (depends on 9) The device of claim 9, wherein the memory and the program code are further configured with the processor to cause the device to: adjust a determination frequency of at least one of the attitude change and the position change according to an operating situation of the device. ### Claim 16 A computer-program product comprising a non-transitory computer-usable medium having computer-readable program code embodied therein, the computer-readable program code adapted to be executed to: capture, via a camera unit of a device, at least two images; divide a first image of the at least two images into a first plurality of segments, each of the first plurality of segments having a set shape and size; select a fixed point at least one segment of the first plurality of segments of from the first image; determine a first location coordinate of the selected fixed point from the at least one segment of the first plurality of segments of the first image; divide a second image of the at least two images into a second plurality of segments, each of the second plurality of segments having the set shape and size, the second image being the next image following the first image of the at least two images; determine a second location coordinate of the selected fixed point from at least one segment of the second plurality of segments of the second image; calculate a movement vector of the selected fixed point between the first determined location coordinate and the determined location coordinate; determine at least one of an attitude change and a position change relative to at least the movement vector; and control a visual controllable element on a display of the device based on the at least one of the attitude change and the position change. ### Claim 17 (depends on 16) The computer-program product of claim 16, wherein at least one of the attitude change and the position change is three-dimensional. ### Claim 18 (depends on 16) The computer-program product of claim 16, wherein the computer-readable program code is further adapted to be executed to: adjust an image frequency of the camera unit to at least a movement speed of the device. ### Claim 19 (depends on 16) The computer-program product of claim 16, wherein the computer-readable program code is further adapted to be executed to: define a mean value for at least the movement vector. ### Claim 20 (depends on 16) The computer-program product of claim 16, wherein the computer-readable program code is further adapted to be executed to: adjust a determination frequency of at least one of the attitude change and the position change according to an operating situation of the device.