# US 2017/0281284 A1 · Generating images for at least two displays in image-guided surgery

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0281284 A1
- Document ID: US20170281284A1
- Status: Published application
- Kind: A1
- Application number: 15/623,299
- Filed: 2017-06-14
- Published: 2017-10-05
- Earliest claimed: 2012-12-21
- Assignees: Brainlab AG
- Applicants: Brainlab AG
- Inventors: Gregor Tuma; Markus Bartenstein; Johannes Manus; Michael Gschwandtner; Fritz Vollmer
- Web page: https://patentgrep.com/patent/US20170281284A1

## CPC

- A61B 90/37: {Surgical systems with images on a monitor during operation}
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B 34/25: {User interfaces for surgical systems}
- G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display}
- G09G 5/003: {Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto (specific for a CRT G09G1/165; for a flat panel G09G3/2092)}
- G09G 5/005: {Adapting incoming signals to the display format of the display terminal}
- G09G 5/32: with means for controlling the display position {(see provisionally G09G5/42)}
- A61B 2017/00216: {with eye tracking or head position tracking control}
- A61B 2034/258: {providing specific settings for specific users}
- A61B 2090/0811: {for the position of a particular part of an instrument with respect to the rest of the instrument, e.g. position of the anvil of a stapling instrument}
- A61B 2090/372: {Details of monitor hardware}
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G09G 2300/04: Structural and physical details of display devices
- G09G 2354/00: Aspects of interface with display user
- G09G 2356/00: Detection of the display position w.r.t. other display screens
- G09G 2380/08: Biomedical applications
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A system, in particular for image-guided surgery, comprising: at least two display devices; a position determinator for determining the relative position of the display devices; and an image generator for generating images, which are to be displayed by the display devices, in accordance with the determined relative position.

## Claims

### Claim 1

A system (1) for use in an associated image-guided procedure, the system (1) comprising:
a first display device (2);
a second display device (3) different than the first display device (2);
a position determinator (8), the position determinator (8) determining a relative position between the first and second display devices (2, 3);
an image generator (9), the image generator being configured to generate a first image to be displayed by the first display device (2) in accordance with the determined relative position between the first and second display devices (2, 3), and the image generator being configured to generate a second image to be displayed by the second display device (3) in accordance with the determined relative position between the first and second display devices (2, 3);
a viewer detector, the viewer detector detecting an associated person viewing at least one of the first display device (2) and/or the second display device (3); and
a viewing tracker, the viewing tracker determining as a viewed display device which of the first and/or second display devices is being viewed by the associated person,
wherein the image generator (9) generates a particular first image for the viewed display device of the at least one of the first and/or second display devices (2, 3) in accordance with the relative position between the first and second display devices (2, 3),
wherein the viewed display device of the at least one of the first and/or second display devices (2, 3) persistently displays at least some of the particular first image.

### Claim 2 (depends on 1)

The system (1) according to claim 1, wherein:
the viewer detector comprises a camera detecting the associated person viewing the at least one of the first and/or second display devices (2, 3);
the viewing tracker comprises an eye tracker determining as the viewed display device which of the first and/or second display devices is being viewed by the associated person by tracking an eye position of the associated person.

### Claim 3 (depends on 2)

The system (1) according to claim 2, further comprising:
a first adjustable mounting (5) supporting the first display device (2) relative to an associated base (4), the first adjustable mounting (5) comprising a plurality of first support elements (5a, 5b) mutually connected via a set of first adjustable joints (5d, 5e);
a second adjustable mounting (6) supporting the second display device (3) relative to the associated base (4), the second adjustable mounting (6) comprising a plurality of second support elements (6a, 6b) mutually connected via a set of second adjustable joints (6d, 6e); and
at least one sensor operably connected with the position determinator (8), the at least one sensor detecting a state of at least one of the first and/or second adjustable mountings (5, 6),
wherein the position determinator (8) determines the relative position between the first and second display devices (2, 3) in accordance with the state of the at least one of the first and/or second adjustable mountings (5, 6) detected by the at least one sensor.

### Claim 4 (depends on 3)

The system (1) according to claim 3, wherein:
the at least one sensor detects the state of the at least one of the first and/or second adjustable mountings (5, 6) by detecting a state of one or more joints of the set of first adjustable joints (5d, 5d) and/or the one or more of the set of second adjustable joints (6d, 6e),
the position determinator (8) determines the relative position between the first and second display devices (2, 3) in accordance with the state of the one or more joints of the set of first adjustable joints (5d, 5d) and/or the one or more of the set of second adjustable joints (6d, 6e) detected by the at least one sensor.

### Claim 5 (depends on 2)

The system (1) according to claim 2, further comprising:
a marker device operatively coupled with at least one of the first and/or second display devices (2, 3),
wherein the position determinator (8) is configured to determine the relative position between the first and second display devices (2, 3) in accordance with a determined position of the marker device.

### Claim 6 (depends on 2)

The system (1) according to claim 2, further comprising:
at least one camera adapted to observe at least one of the first and/or second display devices (2, 3), and to generate an output image of the observed at least one of the first and/or second display devices (2, 3),
wherein the position determinator (8) is configured to determine the relative position between the first and second display devices (2, 3) in accordance with the output image of the observed at least one of the first and/or second display devices (2, 3) generated by the at least one camera.

### Claim 7 (depends on 2)

The system (1) according to claim 2, further comprising:
a camera operatively coupled with at least one of the first and/or second display devices (2, 3), the camera being adapted to generate an output image of an observed field of view of the camera,
wherein the position determinator (8) determines the relative position between the first and second display devices (2, 3) in accordance with the output image of the observed field of view of the camera generated by the camera.

### Claim 8 (depends on 2)

The system (1) according to claim 2, further comprising:
a first adjustable mounting (5) supporting the first display device (2) relative to an associated base (4), the first adjustable mounting (5) comprising a plurality of first support elements (5a, 5b) mutually connected via one or more first adjustable joints (5d, 5e);
a second adjustable mounting (6) supporting the second display device (3) relative to the associated base (4), the second adjustable mounting (6) comprising a plurality of second support elements (6a, 6b) mutually connected via one or more second adjustable joints (6d, 6e); and
at least one actuator, the at least one actuator being operative to adjust a state and/or position of at least one of the first and/or second adjustable joints (5d, 5e, 6d, 6e).

### Claim 9 (depends on 2)

The system (1) according to claim 2, wherein:
the viewer detector identifies the associated person who is viewing the at least one of the first and/or second display devices (2, 3) as an identified viewer; and
the image generator (9) generates the particular first image for the viewed display device of the at least one of the first and/or second display devices (2, 3) in accordance with:
the relative position between the first and second display devices (2, 3); and
the identified viewer.

### Claim 10

A method of generating images developed in an associated image-guided procedure for first and second display devices (2, 3) of an associated system (1), the method comprising:
determining by a position determinator (8) of the associated system (1) a relative position between the first and second display devices (2, 3);
generating by an image generator (9) of the associated system (1) a first image to be displayed by the first display device (2) of the associated system (1) in accordance with the determined relative position between the first and second display devices (2, 3), and a second image to be displayed by the second display device (3) of the associated system (1) in accordance with the determined relative position between the first and second display devices (2, 3);
detecting by a viewer detector an associated person viewing at least one of the first display device (2) and/or the second display device (3);
determining by a viewing tracker as a viewed display device which of the first and/or second display devices is being viewed by the associated person;
generating by the image generator (9) a particular first image for the viewed display device of the at least one of the first and/or second display devices (2, 3) in accordance with the relative position between the first and second display devices (2, 3); and
persistently displaying at least some of the particular first image by the viewed display device of the at least one of the first and/or second display devices (2, 3).

### Claim 11 (depends on 10)

The method according to claim 10, further comprising:
using a camera of the viewer detector to detect the associated person viewing the at least one of the first and/or second display devices (2, 3); and
using an eye tracker of the viewing tracker to determine as the viewed display device which of the first and/or second display devices is being viewed by the associated person by tracking an eye position of the associated person using the eye tracker.

### Claim 12 (depends on 11)

The method according to claim 11, further comprising:
detecting, by at least one sensor operably connected with the position determinator (8), a state of a first adjustable mounting (5) supporting the first display device (2) relative to an associated base (4) and/or a second adjustable mounting (6) supporting the second display device (3) relative to the associated base (4), the first adjustable mounting (5) comprising a plurality of first support elements (5a, 5b) mutually connected via a set of first adjustable joints (5d, 5e), the second adjustable mounting (6) comprising a plurality of second support elements (6a, 6b) mutually connected via a set of second adjustable joints (6d, 6e); and
determining by the position determinator (8) the relative position between the first and second display devices (2, 3) in accordance with the state of the at least one of the first and/or second adjustable mountings (5, 6) detected by the at least one sensor.

### Claim 13 (depends on 12)

The method according to claim 12, further comprising:
detecting, by the at least one sensor, the state of the at least one of the first and/or second adjustable mountings (5, 6) by detecting a state of one or more joints of the set of first adjustable joints (5d, 5d) and/or the one or more of the set of second adjustable joints (6d, 6e); and
determining, by the position determinator (8), the relative position between the first and second display devices (2, 3) in accordance with the state of the one or more joints of the set of first adjustable joints (5d, 5d) and/or the one or more of the set of second adjustable joints (6d, 6e) detected by the at least one sensor.

### Claim 14 (depends on 11)

The method according to claim 11, further comprising:
determining by the position determinator (8) the relative position between the first and second display devices (2, 3) in accordance with a determined position of a marker device operatively coupled with at least one of the first and/or second display devices (2, 3).

### Claim 15 (depends on 11)

The method according to claim 11, further comprising:
observing by at least one associated camera at least one of the first and/or second display devices (2, 3);
generating by the at least one associated camera an output image of the observed at least one of the first and/or second display devices (2, 3); and
determining by the position determinator (8) the relative position between the first and second display devices (2, 3) in accordance with the output image generated by the associated at least one camera.

### Claim 16 (depends on 11)

The method according to claim 11, further comprising:
observing by an associated camera a field of view of the associated camera;
generating by the associated camera an output image of the field of view of the associated camera; and
determining by the position determinator (8) the relative position between the first and second display devices (2, 3) in accordance with the output image generated by the associated camera.

### Claim 17 (depends on 11)

The method according to claim 11, further comprising:
generating a drive signal, the drive signal driving at least one associated actuator for adjusting a state of at least one of first adjustable joints (5d, 5e) mutually connecting a plurality of first support elements (5a, 5b) comprising a first adjustable mounting (5) supporting the first display device (2) relative to an associated base (4) and/or second adjustable joints (6d, 6e) mutually connecting a plurality of second support elements (6a, 6b) comprising a second adjustable mounting (6) supporting the second display device (3) relative to the associated base (4).

### Claim 18 (depends on 11)

The method according to claim 11, further comprising:
identifying by the viewer detector the associated person who is viewing the at least one of the first and/or second display devices (2, 3) as an identified viewer; and
generating by the image generator (9) the particular first image for the viewed display device of the at least one of the first and/or second display devices (2, 3) in accordance with:
the relative position between the first and second display devices (2, 3); and
the identified viewer.

### Claim 19

A non-transitory machine readable storage medium having instructions stored thereon that when executed cause an associated system (1) including first and second display devices (2, 3) to perform in an associated image-guided procedure steps comprising:
determining by a position determinator (8) of the associated system (1) a relative position between the first and second display devices (2, 3);
generating by an image generator (9) of the associated system (1) a first image to be displayed by the first display device (2) of the associated system (1) in accordance with the determined relative position between the first and second display devices (2, 3), and a second image to be displayed by the second display device (3) of the associated system (1) in accordance with the determined relative position between the first and second display devices (2, 3);
detecting by a viewer detector an associated person viewing at least one of the first display device (2) and/or the second display device (3);
determining by a viewing tracker as a viewed display device which of the first and/or second display devices is being viewed by the associated person;
generating by the image generator (9) a particular first image for the viewed display device of the at least one of the first and/or second display devices (2, 3) in accordance with the relative position between the first and second display devices (2, 3); and
persistently displaying at least some of the particular first image by the viewed display device of the at least one of the first and/or second display devices (2, 3).

### Claim 20 (depends on 19)

The storage medium according to claim 19 having the instructions stored thereon that when executed cause the associated system (1) to perform further steps comprising:
using a camera of the viewer detector to detect the associated person viewing the at least one of the first and/or second display devices (2, 3); and
using an eye tracker of the viewing tracker to determine as the viewed display device which of the first and/or second display devices is being viewed by the associated person by tracking an eye position of the associated person using the eye tracker.

### Claim 21 (depends on 20)

The storage medium according to claim 20, having the instructions stored thereon that when executed cause the associated system (1) to perform further steps comprising:
detecting, by at least one sensor operably connected with the position determinator (8), a state of a first adjustable mounting (5) supporting the first display device (2) relative to an associated base (4) and/or a second adjustable mounting (6) supporting the second display device (3) relative to the associated base (4), the first adjustable mounting (5) comprising a plurality of first support elements (5a, 5b) mutually connected via a set of first adjustable joints (5d, 5e), the second adjustable mounting (6) comprising a plurality of second support elements (6a, 6b) mutually connected via a set of second adjustable joints (6d, 6e); and
determining by the position determinator (8) the relative position between the first and second display devices (2, 3) in accordance with the state of the at least one of the first and/or second adjustable mountings (5, 6) detected by the at least one sensor.

### Claim 22 (depends on 21)

The storage medium according to claim 21, having the instructions stored thereon that when executed cause the associated system (1) to perform further steps comprising:
detecting, by the at least one sensor, the state of the at least one of the first and/or second adjustable mountings (5, 6) by detecting a state of one or more joints of the set of first adjustable joints (5d, 5d) and/or the one or more of the set of second adjustable joints (6d, 6e); and
determining, by the position determinator (8), the relative position between the first and second display devices (2, 3) in accordance with the state of the one or more joints of the set of first adjustable joints (5d, 5d) and/or the one or more of the set of second adjustable joints (6d, 6e) detected by the at least one sensor.

### Claim 23 (depends on 20)

The storage medium according to claim 20, having the instructions stored thereon that when executed cause the associated system (1) to perform further steps comprising:
determining by the position determinator (8) the relative position between the first and second display devices (2, 3) in accordance with a determined position of a marker device operatively coupled with at least one of the first and/or second display devices (2, 3).

### Claim 24 (depends on 20)

The storage medium according to claim 20, having the instructions stored thereon that when executed cause the associated system (1) to perform further steps comprising:
observing by at least one associated camera at least one of the first and/or second display devices (2, 3);
generating by the at least one associated camera an output image of the observed at least one of the first and/or second display devices (2, 3); and
determining by the position determinator (8) the relative position between the first and second display devices (2, 3) in accordance with the output image generated by the associated at least one camera.

### Claim 25 (depends on 20)

The storage medium according to claim 20, having the instructions stored thereon that when executed cause the associated system (1) to perform further steps comprising:
observing by an associated camera a field of view of the associated camera;
generating by the associated camera an output image of the field of view of the associated camera; and
determining by the position determinator (8) the relative position between the first and second display devices (2, 3) in accordance with the output image generated by the associated camera.

### Claim 26 (depends on 20)

The storage medium according to claim 20, having the instructions stored thereon that when executed cause the associated system (1) to perform further steps comprising:
generating a drive signal, the drive signal driving at least one associated actuator for adjusting a state of at least one of first adjustable joints (5d, 5e) mutually connecting a plurality of first support elements (5a, 5b) comprising a first adjustable mounting (5) supporting the first display device (2) relative to an associated base (4) and/or second adjustable joints (6d, 6e) mutually connecting a plurality of second support elements (6a, 6b) comprising a second adjustable mounting (6) supporting the second display device (3) relative to the associated base (4).

### Claim 27 (depends on 20)

The storage medium according to claim 20, having the instructions stored thereon that when executed cause the associated system (1) to perform further steps comprising:
identifying by the viewer detector the associated person who is viewing the at least one of the first and/or second display devices (2, 3) as an identified viewer; and
generating by the image generator (9) the particular first image for the viewed display device of the at least one of the first and/or second display devices (2, 3) in accordance with:
the relative position between the first and second display devices (2, 3); and
the identified viewer.