# US 12,112,008 B2 · Simulating physical materials and light interaction in a user interface of a resource-constrained device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,008 B2 - Document ID: US12112008B2 - Status: Granted - Kind: B2 - Application number: 17/385,770 - Filed: 2021-07-26 - Published: 2022-04-21 - Granted: 2024-10-08 - Earliest claimed: 2012-06-08 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Jesse W. Boettcher; Michael I. Ingrassia, Jr.; Jeri C. Mason; Anton M. Davydov; David J. Rempel; Imran Chaudhri - Web page: https://patentgrep.com/patent/US12112008B2 ## CPC - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods and systems for simulating light interaction and physical materials in a graphical user interface (GUI) of a resource-constrained device are provided. Simulating physical materials, such as glass and metal, in a GUI can allow a user to feel more natural in interacting with the GUI. The user experience can be further enhanced if the simulated physical materials in the GUI can interact with the device's environment in a manner similar to how the actual physical materials would interact. However, continually polling various sensors can be resource-intensive, especially for resource-constrained mobile devices. Accordingly, a mobile device can intelligently determine whether to begin a reduced detail mode, and then render user interface objects in the reduced detail mode to conserve resources. ## Claims ### Claim 1 A method, comprising: at a computer system that is in communication with a display device and one or more input devices: displaying, via the display device, a user interface object corresponding to a first control configured to adjust a first operation of a software application, wherein: the user interface object is displayed as part of an application user interface for the software application that includes a second control configured to adjust a second operation of the software application and a third control configured to adjust a third operation of the software application; the second control and the third control are displayed concurrently with the first control; and the user interface object is made of a simulated physical material and the third control is not made of the simulated physical material; while the computer system is being used in a first environment: while displaying the user interface object, detecting, via the one or more input devices, a first movement; and in response to detecting the first movement, changing an appearance of the user interface object based on the first movement and a first set of one or more simulated light sources selected based on the first environment and maintaining an appearance of the third control; and while the computer system is being used in a second environment different from the first environment: while displaying the user interface object, detecting, via the one or more input devices, a second movement; and in response to detecting the second movement, changing the appearance of the user interface object based on the second movement and a second set of one or more simulated light sources selected based on the second environment that are different from the first set of one or more simulated light sources and maintaining the appearance of the third control. ### Claim 2 (depends on 1) The method of claim 1, wherein the simulated physical material is glass. ### Claim 3 (depends on 1) The method of claim 1, wherein the simulated physical material is metal. ### Claim 4 (depends on 1) The method of claim 1, wherein a color temperature of the first set of one or more simulated light sources is different from a color temperature of the second set of one or more simulated light sources. ### Claim 5 (depends on 1) The method of claim 1, wherein an intensity of light of the first set of one or more simulated light sources is different from an intensity of light of the second set of one or more light sources. ### Claim 6 (depends on 1) The method of claim 1, wherein a sharpness of shadows cast by the first set of one or more simulated light sources is different from a sharpness of shadows cast by the second set of one or more light sources. ### Claim 7 (depends on 1) The method of claim 1, wherein the first environment is outdoors and the second environment is indoors. ### Claim 8 (depends on 1) The method of claim 1, wherein the first environment is at a first time of day and the second environment is at a second time of day, different from the first time of day. ### Claim 9 (depends on 1) The method of claim 1, wherein the first environment is in first weather conditions and the second environment is in second weather conditions, different from the first weather conditions. ### Claim 10 (depends on 1) The method of claim 1, wherein the one or more simulated light sources includes multiple light sources pointed at the user interface object from different simulated directions. ### Claim 11 (depends on 1) The method of claim 1, wherein the user interface object has a color based on content associated with the user interface object. ### Claim 12 (depends on 11) The method of claim 11, further comprising: detecting a change in the content associated with the user interface object; and in response to detecting the change in the content associated with the user interface object, changing the color of the user interface object. ### Claim 13 (depends on 1) The method of claim 1, wherein detecting movement comprises detecting movement of the display device, the method further comprising: in response to detecting the movement of the display device, moving the user interface object to indicate that the movement of the user interface object is independent of movement of the display device. ### Claim 14 (depends on 1) The method of claim 1, wherein the simulated physical material includes a texture. ### Claim 15 (depends on 1) The method of claim 1, further comprising: while the computer system is being used in the first environment: while displaying the second control that is made of a second simulated physical material, detecting, via the one or more input devices, the first movement; and in response to detecting the first movement, changing an appearance of the second control based on the first movement and the first set of one or more simulated light sources selected based on the first environment; and while the computer system is being used in the second environment: while displaying the second control, detecting, via the one or more input devices, the second movement; and in response to detecting the second movement, changing the appearance of the second control based on the second movement and the second set of one or more simulated light sources selected based on the second environment. ### Claim 16 A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with a display device and one or more input devices, the one or more programs including instructions for: displaying, via the display device, a user interface object corresponding to a first control configured to adjust a first operation of a software application, wherein: the user interface object is displayed as part of an application user interface for the software application that includes a second control configured to adjust a second operation of the software application and a third control configured to adjust a third operation of the software application; the second control and the third control are displayed concurrently with the first control; and the user interface object is made of a simulated physical material and the third control is not made of the simulated physical material; while the computer system is being used in a first environment: while displaying the user interface object, detecting, via the one or more input devices, a first movement; and in response to detecting the first movement, changing an appearance of the user interface object based on the first movement and a first set of one or more simulated light sources selected based on the first environment and maintaining an appearance of the third control; and while the computer system is being used in a second environment, different from the first environment: while displaying the user interface object, detecting, via the one or more input devices, a second movement; and in response to detecting the second movement, changing the appearance of the user interface object based on the second movement and a second set of one or more simulated light sources selected based on the second environment that are different from the first set of one or more simulated light sources and maintaining the appearance of the third control. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the simulated physical material is glass. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the simulated physical material is metal. ### Claim 19 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein a color temperature of the first set of one or more simulated light sources is different from a color temperature of the second set of one or more simulated light sources. ### Claim 20 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein an intensity of light of the first set of one or more simulated light sources is different from an intensity of light of the second set of one or more light sources. ### Claim 21 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein a sharpness of shadows cast by the first set of one or more simulated light sources is different from a sharpness of shadows cast by the second set of one or more light sources. ### Claim 22 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the first environment is outdoors and the second environment is indoors. ### Claim 23 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the first environment is at a first time of day and the second environment is at a second time of day, different from the first time of day. ### Claim 24 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the first environment is in first weather conditions and the second environment is in second weather conditions, different from the first weather conditions. ### Claim 25 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the one or more simulated light sources includes multiple light sources pointed at the user interface object from different simulated directions. ### Claim 26 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the user interface object has a color based on content associated with the user interface object. ### Claim 27 (depends on 26) The non-transitory computer-readable storage medium of claim 26, further storing instructions which, when executed by the one or more processors, further causes the one or more processors to: detect a change in the content associated with the user interface object; and in response to detecting the change in the content associated with the user interface object, change the color of the user interface object. ### Claim 28 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein detecting movement comprises detecting movement of the display device, the non-transitory computer-readable storage medium further storing instructions which, when executed by the one or more processors, further causes the one or more processors to: in response to detecting the movement of the display device, move the user interface object to indicate that the movement of the user interface object is independent of movement of the display device. ### Claim 29 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the simulated physical material includes a texture. ### Claim 30 (depends on 16) The non-transitory computer-readable storage medium of claim 16, further storing instructions which, when executed by the one or more processors, further causes the one or more processors to: while the computer system is being used in the first environment: while displaying the second control that is made of a second simulated physical material, detect, via the one or more input devices, the first movement; and in response to detecting the first movement, change an appearance of the second control based on the first movement and the first set of one or more simulated light sources selected based on the first environment; and while the computer system is being used in the second environment: while displaying the second control, detect, via the one or more input devices, the second movement; and in response to detecting the second movement, change the appearance of the second control based on the second movement and the second set of one or more simulated light sources selected based on the second environment. ### Claim 31 A computer system that is in communication with a display device and one or more input devices, the computer system comprising: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: displaying, via the display device, a user interface object corresponding to a first control configured to adjust a first operation of a software application, wherein: the user interface object is displayed as part of an application user interface for the software application that includes a second control configured to adjust a second operation of the software application and a third control configured to adjust a third operation of the software application; the second control and the third control are displayed concurrently with the first control; and the user interface object is made of a simulated physical material and the third control is not made of the simulated physical material; while the computer system is being used in a first environment: while displaying the user interface object, detecting, via the one or more input devices, a first movement; and in response to detecting the first movement, changing an appearance of the user interface object based on the first movement and a first set of one or more simulated light sources selected based on the first environment and maintaining an appearance of the third control; and while the computer system is being used in a second environment different from the first environment: while displaying the user interface object, detecting, via the one or more input devices, a second movement; and in response to detecting the second movement, changing the appearance of the user interface object based on the second movement and a second set of one or more simulated light sources selected based on the second environment that are different from the first set of one or more simulated light sources and maintaining the appearance of the third control. ### Claim 32 (depends on 31) The computer system of claim 31, wherein the simulated physical material is glass. ### Claim 33 (depends on 31) The computer system of claim 31, wherein the simulated physical material is metal. ### Claim 34 (depends on 31) The computer system of claim 31, wherein a color temperature of the first set of one or more simulated light sources is different from a color temperature of the second set of one or more simulated light sources. ### Claim 35 (depends on 31) The computer system of claim 31, wherein an intensity of light of the first set of one or more simulated light sources is different from an intensity of light of the second set of one or more light sources. ### Claim 36 (depends on 31) The computer system of claim 31, wherein a sharpness of shadows cast by the first set of one or more simulated light sources is different from a sharpness of shadows cast by the second set of one or more light sources. ### Claim 37 (depends on 31) The computer system of claim 31, wherein the first environment is outdoors and the second environment is indoors. ### Claim 38 (depends on 31) The computer system of claim 31, wherein the first environment is at a first time of day and the second environment is at a second time of day, different from the first time of day. ### Claim 39 (depends on 31) The computer system of claim 31, wherein the first environment is in first weather conditions and the second environment is in second weather conditions, different from the first weather conditions. ### Claim 40 (depends on 31) The computer system of claim 31, wherein the one or more simulated light sources includes multiple light sources pointed at the user interface object from different simulated directions. ### Claim 41 (depends on 31) The computer system of claim 31, wherein the user interface object has a color based on content associated with the user interface object. ### Claim 42 (depends on 41) The computer system of claim 41, the one or more programs further including instructions for: detecting a change in the content associated with the user interface object; and in response to detecting the change in the content associated with the user interface object, changing the color of the user interface object. ### Claim 43 (depends on 31) The computer system of claim 31, wherein detecting movement comprises detecting movement of the display device, the one or more programs further including instructions for: in response to detecting the movement of the display device, moving the user interface object to indicate that the movement of the user interface object is independent of movement of the display device. ### Claim 44 (depends on 31) The computer system of claim 31, wherein the simulated physical material includes a texture. ### Claim 45 (depends on 31) The computer system of claim 31, the one or more programs further including instructions for: while the computer system is being used in the first environment: while displaying the second control that is made of a second simulated physical material, detecting, via the one or more input devices, the first movement; and in response to detecting the first movement, changing an appearance of the second control based on the first movement and the first set of one or more simulated light sources selected based on the first environment; and while the computer system is being used in the second environment: while displaying the second control, detecting, via the one or more input devices, the second movement; and in response to detecting the second movement, changing the appearance of the second control based on the second movement and the second set of one or more simulated light sources selected based on the second environment.