# US 11,907,607 B2 · Electronic apparatus and method for outputting image thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,907,607 B2
- Document ID: US11907607B2
- Status: Granted
- Kind: B2
- Application number: 17/514,276
- Filed: 2021-10-29
- Published: 2022-02-17
- Granted: 2024-02-20
- Earliest claimed: 2019-04-29
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Changryong Heo; Jongjin Kim; Jeongmin Park; Jonghwan Shin; Sukjae Lee; Junhee Cho
- Web page: https://patentgrep.com/patent/US11907607B2

## CPC

- G06F 3/147: using display panels
- G06F 1/1616: {with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position}
- G06F 1/1652: {the display being flexible, e.g. mimicking a sheet of paper, or rollable}
- G06F 1/1686: {the I/O peripheral being an integrated camera}
- G06F 3/01: Input arrangements or combined input and output arrangements for interaction between user and computer (G06F3/16 takes precedence)
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06T 15/50: Lighting effects
- G06T 15/506: {Illumination models}
- G06T 15/60: Shadow generation
- G09G 3/035: {for flexible display surfaces}
- G09G 3/20: for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix {no fixed position being assigned to or needed to be assigned to the individual characters or partial characters}
- G09G 3/3406: {Control of illumination source (illumination devices structurally associated with liquid crystal cells G02F1/1336)}
- G06F 2200/1614: Image rotation following screen orientation, e.g. switching from landscape to portrait mode
- G09G 2320/0233: Improving the luminance or brightness uniformity across the screen
- G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen
- G09G 2340/0492: Change of orientation of the displayed image, e.g. upside-down, mirrored
- G09G 2360/144: the light being ambient light
- G09G 2380/02: Flexible displays
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

An electronic device and a method of controlling the same are provided. The electronic device includes a housing foldable about a first axis; a foldable display foldable about a second axis parallel to the first axis; a folding sensor circuit configured to detect a display state of at least one of the housing or the foldable display; a memory storing instructions; and a processor configured to execute the instructions stored in the memory to: determine a position of a virtual light source with respect to the foldable display based on context information associated with the electronic device; render an image based on the display state detected by the folding sensor circuit and the position of the virtual light source; and control the foldable display to display the rendered image.

## Claims

### Claim 1

An electronic device, comprising:
a housing comprising a first housing structure and a second housing structure foldable with respect to the first housing structure about a first axis;
a foldable display foldable about a second axis parallel to the first axis, wherein the foldable display includes a first portion coupled to the first housing structure and a second portion coupled to the second housing structure, and the first portion is foldable with respect to the second portion about the second axis;
a folding sensor circuit configured to detect a display state of at least one of the housing or the foldable display;
an orientation sensing circuit configured to detect an orientation of the electronic device;
a memory storing instructions; and
a processor configured to execute the instructions stored in the memory to:
identify whether a specified condition for determining a position of a virtual light source is satisfied, wherein the specified condition includes at least one of a change of the display state, a predetermined period elapsing, or movement of the electronic device out of a specified range;
in response to satisfaction of the specified condition, determine the position of the virtual light source with respect to the foldable display based on context information associated with the electronic device, wherein the context information includes at least one of an orientation of the electronic device, time information, or illumination around the electronic device;
render an image based on the display state detected by the folding sensor circuit and the position of the virtual light source; and
control the foldable display to display the rendered image,
wherein the processor is further configured to:
determine a shadow region on which a shadowing effect is to be applied in the second portion based on the display state and the position of the virtual light source;
determine a shadowing application mode associated with the shadowing effect based on the orientation of the electronic device;
render the image such that the shadowing effect is applied to the first portion and the shadow region when the shadowing application mode is a landscape mode; and
render the image such that the shadowing effect is displayed on a region adjacent to the second axis in the first portion and a region adjacent to the second axis in the second portion when the shadowing application mode is a portrait mode.

### Claim 2 (depends on 1)

The electronic device of claim 1, further comprising:
wherein the context information comprises the orientation of the electronic device, and
wherein the processor is further configured to:
determine the position of the virtual light source based on the orientation of the electronic device.

### Claim 3 (depends on 2)

The electronic device of claim 2, wherein the time information indicates a time of the position of the virtual light source with respect to the orientation of the electronic device.

### Claim 4 (depends on 1)

The electronic device of claim 1, further comprising:
a plurality of illumination sensors configured to sense the illumination around the electronic device,
wherein the processor is further configured to:
determine the position of the virtual light source based on a value detected by an illumination sensor among the plurality of illumination sensors meeting the specified condition.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein the processor is further configured to:
determine positions of a plurality of virtual light sources based on values detected by the plurality of illumination sensors meeting the specified condition;
determine a plurality of image effects respectively corresponding to the plurality of virtual light sources; and
render the image by applying the plurality of image effects.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to:
render the image such that a number of visual objects included in the image or luminance output in at least a portion of the image is changed according to a change in the display state.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the image includes a visual object and a shadow object corresponding to the visual object, and
wherein the processor is further configured to:
render the image such that the shadow object is displayed in a length determined according to the display state.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to:
divide a basic image to be displayed on the foldable display into a background region and a region of interest;
apply a first image effect to the background region based on the display state; and
apply a second image effect to the region of interest based on at least one of the display state or the position of the virtual light source.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein the first image effect comprises changing luminance of the background region based on the display state.

### Claim 10 (depends on 8)

The electronic device of claim 8, wherein the second image effect comprises displaying a shadow object on the region of interest or a periphery of the region of interest depending on at least one of the display state or the position of the virtual light source.

### Claim 11

A method for outputting an image in an electronic device, the method comprising:
identifying whether a specified condition for determining a position of a virtual light source is satisfied, wherein the specified condition includes at least one of a change of a display state, a predetermined period elapsing, or movement of the electronic device out of a specified range;
in response to satisfaction of the specified condition, obtaining context information associated with the electronic device, wherein the context information includes at least one of an orientation of the electronic device, time information, or illumination around the electronic device;
determining the position of the virtual light source with respect to a foldable display based on the context information;
detecting a display state of at least one of a housing of the electronic device or the foldable display;
rendering an image based on the detected display state and the position of the virtual light source; and
controlling the foldable display to display the rendered image,
wherein the housing comprises a first housing structure and a second housing structure foldable with respect to the first housing structure about a first axis,
wherein the foldable display includes a first portion coupled to the first housing structure and a second portion coupled to the second housing structure, and the first portion is foldable with respect to the second portion about a second axis, and
wherein the method further comprises:
determining a shadow region on which a shadowing effect is to be applied in the second portion based on the display state and the position of the virtual light source; and
determining a shadowing application mode associated with the shadowing effect based on the orientation of the electronic device, and
wherein the rendering of the image comprises:
rendering the image such that the shadowing effect is applied to the first portion and the shadow region when the shadowing application mode is a landscape mode; and
rendering the image such that the shadowing effect is displayed on a region adjacent to the second axis in the first portion and a region adjacent to the second axis in the second portion when the shadowing application mode is a portrait mode.

### Claim 12 (depends on 11)

The method of claim 11, wherein the obtaining the context information comprises:
obtaining the orientation of the electronic device, and
wherein the determining the position of the virtual light source comprises:
determining the position of the virtual light source based on the orientation of the electronic device.

### Claim 13 (depends on 11)

The method of claim 11, wherein the obtaining the context information comprises:
obtaining at least one illumination value around the electronic device; and
determining the position of the virtual light source based on the at least one illumination value meeting the specified condition.

### Claim 14 (depends on 11)

The method of claim 11, wherein the rendering the image comprises:
rendering the image such that a number of visual objects included in the image or luminance output in at least a portion of the image is changed according to a change in the display state.

### Claim 15 (depends on 11)

The method of claim 11, wherein the image includes a visual object and a shadow object corresponding to the visual object, and
wherein the rendering the image further comprises:
rendering the image such that the shadow object is displayed in a length determined according to the display state.

### Claim 16 (depends on 11)

The method of claim 11, further comprising:
obtaining a basic image; and
dividing the basic image into a background region and a region of interest,
wherein the rendering the image further comprises:
applying a first image effect to the background region based on the display state; and
applying a second image effect to the region of interest based on the display state or the position of the virtual light source.

### Claim 17 (depends on 16)

The method of claim 16, wherein the first image effect comprises changing luminance of the background region based on the display state, and
wherein the second image effect comprises displaying a shadow object on the region of interest or a periphery of the region of interest depending on at least one of the display state or the position of the virtual light source.