# US 10,319,127 B2 · Spatial and hierarchical placement of images at runtime > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,319,127 B2 - Document ID: US10319127B2 - Status: Granted - Kind: B2 - Application number: 15/272,723 - Filed: 2016-09-22 - Published: 2018-03-22 - Granted: 2019-06-11 - Earliest claimed: 2016-09-22 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: James R. Kunz; Virginia L. Li; William F. Phillips - Web page: https://patentgrep.com/patent/US10319127B2 ## CPC - G06T 11/60: Creating or editing images; Combining images with text - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00) - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method, computer program product, and system for automatic placement and layering of images at runtime include a processor(s) obtaining images that represent components of an object, including a first image and a second image. The processor(s) annotates the first image with data indicating one or more locations on the first image where the second image can be placed. The processor(s) determines, during runtime, a position or orientation of the one or more locations on the first image, based on the annotated first image. The processor(s) compares each location to data in a system file to determine a first location that is a placement for the second image on the first image. The processor(s) layers the second image over the first image at the first location. The processor(s) renders the two dimensional model of the object, including the layered images, displays it in the graphical user interface. ## Claims ### Claim 1 A computer-implemented method, comprising: obtaining, by one or more processors, images comprising representations of components that comprise an object in three dimensional space, wherein the images comprise a first image and a second image, wherein the first image comprises a two dimensional image of a first portion of the object and the second image comprises a two dimensional image of a second portion of the object; annotating, by the one or more processors, the first image, with data indicating one or more locations on the first image where the second image can be placed, wherein a display in a graphical user interface including the first image and the second image comprises a two dimensional model of the object, wherein one location of the one or more locations comprises an actual location of the first portion of the object relative to the second portion of the object in the three dimensional space; determining, by the one or more processors, during runtime, a position or orientation of the one or more locations on the first image, based on the annotated first image; comparing, by the one or more processors, each location of the one or more locations on the first image to data in a system file to determine one or more designated locations of the one or more locations, wherein each designated location comprises a placement for the second image on the first image; based on the determining, layering, by the one or more processors, the second image over the first image at the one or more designated locations; rendering, by the one or more processors, a runtime generated two dimensional model of the object comprising the second image layered on the first image at the one or more designated locations, wherein the runtime generated two dimensional model of the object is selected from the group consisting of: a model consistent with the two dimensional model of the object, and a model inconsistent with the two dimensional model of the object; and displaying, by the one or more processors, the runtime generated two dimensional model in the graphical user interface. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the images further comprise a third image, the computer-implemented method further comprising: annotating, by the one or more processors, the second image, with data indicating one or more locations on the second image where the third image can be placed, wherein the display in a graphical user interface comprising a two dimensional model of the object further comprises the third image; determining, by the one or more processors, during runtime, a position or orientation of the one or more locations on the second image, based on the annotated second image; comparing, by the one or more processors, each location of the one or more locations on the second image, to other data in the system file, to determine a second location of the one or more locations on the second image that comprises a placement for the third image on the second image; based on the determining, layering, by the one or more processors, the third image over the second image at the second location; updating, by the one or more processors, the runtime generated two dimensional model of the object comprising the second image layered on the first image to include the third image layered on the second image; and displaying, by the one or more processors, the updated runtime generated two dimensional model in the graphical user interface. ### Claim 3 (depends on 1) The computer implemented method of claim 1, wherein the one or more locations on the first image comprise one or more rectangular areas on the first image defined by a user editing the first image in an image editor and the annotating comprises retaining data describing the rectangular areas in the first image file. ### Claim 4 (depends on 1) The computer implemented method of claim 1, further comprising: rotating, by the one or more processors, the second image in the two dimensional model, the rotating comprising: obtaining, by the one or more processors, changes to a selected placement at a location of the one or more designated locations for the second image on the first image, wherein the changes comprise rotating the second image within the location to a new orientation; updated, by the one or more processors, during runtime, the two dimensional model of the object comprising the second image layered on the first image at the new orientation; and displaying, by the one or more processors, the updated runtime generated two dimensional model in the graphical user interface. ### Claim 5 (depends on 4) The computer implemented method of claim 4, wherein the changes to the selected placement for the second image on the first image are defined by a user editing the first image in an image editor. ### Claim 6 (depends on 1) The computer implemented method of claim 1, wherein the object comprises a computing resource comprising the one or more processors. ### Claim 7 (depends on 6) The computer implemented method of claim 6, wherein the computing resource comprises a mainframe computer. ### Claim 8 (depends on 1) The computer implemented method of claim 1, wherein the first image comprises an image of a component selected from the group consisting of: a frame and an I/O drawer. ### Claim 9 (depends on 1) The computer implemented method of claim 1, wherein the first image comprises an image of a frame and the second image comprises an image of an I/O drawer. ### Claim 10 (depends on 1) The computer implemented method of claim 1, wherein the first image comprises an image of an I/O drawer and the second image comprises an image of an I/O card. ### Claim 11 A computer program product comprising: a computer readable storage medium readable by one or more processors and storing instructions for execution by the one or more processors for performing a method comprising: obtaining, by the one or more processors, images comprising representations of components that comprise an object in three dimensional space, wherein the images comprise a first image and a second image, wherein the first image comprises a two dimensional image of a first portion of the object and the second image comprises a two dimensional image of a second portion of the object; annotating, by the one or more processors, the first image, with data indicating one or more locations on the first image where the second image can be placed, wherein a display in a graphical user interface including the first image and the second image comprises a two dimensional model of the object, wherein one location of the one or more locations comprises an actual location of the first portion of the object relative to the second portion of the object in the three dimensional space; determining, by the one or more processors, during runtime, a position or orientation of the one or more locations on the first image, based on the annotated first image; comparing, by the one or more processors, each location of the one or more locations on the first image to data in a system file to determine one or more designated locations of the one or more locations, wherein each designated location comprises a placement for the second image on the first image; based on the determining, layering, by the one or more processors, the second image over the first image at the one or more designated locations; rendering, by the one or more processors, a runtime generated two dimensional model of the object comprising the second image layered on the first image at the one or more designated locations, wherein the runtime generated two dimensional model of the object is selected from the group consisting of: a model consistent with the two dimensional model of the object, and a model inconsistent with the two dimensional model of the object; and displaying, by the one or more processors, the runtime generated two dimensional model in the graphical user interface. ### Claim 12 (depends on 11) The computer program product of claim 11, wherein the images further comprise a third image, the computer-implemented method further comprising: annotating, by the one or more processors, the second image, with data indicating one or more locations on the second image where the third image can be placed, wherein the display in a graphical user interface comprising a two dimensional model of the object further comprises the third image; determining, by the one or more processors, during runtime, a position or orientation of the one or more locations on the second image, based on the annotated second image; comparing, by the one or more processors, each location of the one or more locations on the second image, to other data in the system file, to determine a second location of the one or more locations on the second image that comprises a placement for the third image on the second image; based on the determining, layering, by the one or more processors, the third image over the second image at the second location; updating, by the one or more processors, the runtime generated two dimensional model of the object comprising the second image layered on the first image to include the third image layered on the second image; and displaying, by the one or more processors, the updated runtime generated two dimensional model in the graphical user interface. ### Claim 13 (depends on 11) The computer program product of claim 11, wherein the one or more locations on the first image comprise one or more rectangular areas on the first image defined by a user editing the first image in an image editor and the annotating comprises retaining data describing the rectangular areas in the first image file. ### Claim 14 (depends on 11) The computer program product of claim 11, further comprising: rotating, by the one or more processors, the second image in the two dimensional model, the rotating comprising: rotating, by the one or more processors, the second image in the two dimensional model, the rotating comprising: obtaining, by the one or more processors, changes to a selected placement at a location of the one or more designated locations for the second image on the first image, wherein the changes comprise rotating the second image within the location to a new orientation; updated, by the one or more processors, during runtime, the two dimensional model of the object comprising the second image layered on the first image at the new orientation; and displaying, by the one or more processors, the updated runtime generated two dimensional model in the graphical user interface. ### Claim 15 (depends on 14) The computer program product of claim 14, wherein the changes to the selected placement for the second image on the first image are defined by a user editing the first image in an image editor. ### Claim 16 (depends on 11) The computer program product of claim 11, wherein the object comprises a computing resource comprising the one or more processors. ### Claim 17 (depends on 16) The computer program product of claim 16, wherein the computing resource comprises a mainframe computer. ### Claim 18 (depends on 11) The computer program product of claim 11, wherein the first image comprises an image of a component selected from the group consisting of: a frame and an I/O drawer. ### Claim 19 (depends on 11) The computer program product of claim 11, wherein the first image comprises an image of a frame and the second image comprises an image of an I/O drawer. ### Claim 20 A system comprising: a memory; one or more processors in communication with the memory; and program instructions executable by the one or more processors via the memory to perform a method, the method comprising: obtaining, by the one or more processors, images comprising representations of components that comprise an object in three dimensional space, wherein the images comprise a first image and a second image, wherein the first image comprises a two dimensional image of a first portion of the object and the second image comprises a two dimensional image of a second portion of the object; annotating, by the one or more processors, the first image, with data indicating one or more locations on the first image where the second image can be placed, wherein a display in a graphical user interface including the first image and the second image comprises a two dimensional model of the object, wherein one location of the one or more locations comprises an actual location of the first portion of the object relative to the second portion of the object in the three dimensional space; determining, by the one or more processors, during runtime, a position or orientation of the one or more locations on the first image, based on the annotated first image; comparing, by the one or more processors, each location of the one or more locations on the first image to data in a system file to determine one or more designated locations of the one or more locations, wherein each designated location comprises a placement for the second image on the first image; based on the determining, layering, by the one or more processors, the second image over the first image at the one or more designated locations; rendering, by the one or more processors, a runtime generated two dimensional model of the object comprising the second image layered on the first image at the one or more designated locations, wherein the runtime generated two dimensional model of the object is selected from the group consisting of: a model consistent with the two dimensional model of the object, and a model inconsistent with the two dimensional model of the object; and displaying, by the one or more processors, the runtime generated two dimensional model in the graphical user interface.