# US 9,547,898 B2 · Automated cytology/histology viewers and related methods > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,547,898 B2 - Document ID: US9547898B2 - Status: Granted - Kind: B2 - Application number: 14/659,236 - Filed: 2015-03-16 - Published: 2015-10-01 - Granted: 2017-01-17 - Earliest claimed: 2014-03-26 - Assignees: Sectra AB - Applicants: Sectra AB - Inventors: Fredrik Häll; Kristian Köpsén; Jesper Molin; Joackim Pennerup; Olle Westman; Tobias Dahlberg; Claes Lundström - Web page: https://patentgrep.com/patent/US9547898B2 ## CPC - G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS - G06T 1/0007: {Image acquisition} - G06T 7/0012: {Biomedical image inspection} - G06T 7/0014: {using an image reference approach} - G06T 11/23: using straight lines or curves - G06V 10/95: {structured as a network, e.g. client-server architectures} - G06V 20/69: Microscopic objects, e.g. biological cells or cellular parts - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods, systems, workstations, viewers, grossing stations and computer include a display and a viewer circuit configured to cause the display to concurrently display: (i) a macroscopic view of a grossing specimen with virtual cut location marks associated with orientation and location of actual physical cut locations used to obtain tissue samples from the grossing specimen and (ii) at least one digital microscopic whole-slide image (WSI) of a tissue section from the specimen. The display can show the at least one WSI image on the display with a relevant cut location mark on the macroscopic view shown visually enhanced from other cut location marks on the macroscopic view to thereby allow a user to visually connect where the tissue section in the WSI image was obtained during a grossing procedure. ## Claims ### Claim 1 A method of obtaining and/or processing digital pathology and cytology images for viewing, comprising: automatically electronically identifying a location on or in a grossing specimen where at least one color is applied to an external surface area of the grossing specimen or an external surface of dissected tissue of the grossing specimen as an external surface marking during a grossing procedure; and automatically electronically generating virtual color location marks on a digital macroscopic image or model of the grossing specimen based on the electronic identification such that the virtual color location marks are elongate and have a correct spatial position on the image or model of the grossing specimen correlated to the location where the at least one color is applied on the grossing specimen. ### Claim 2 (depends on 1) The method of claim 1, wherein the electronic identification comprises electronically automatically obtaining a series of macroscopic images of the grossing specimen during a grossing procedure of the specimen, electronically interrogating the images to detect when a new color is applied to a physical cut location on the grossing specimen, and wherein the electronically generating is carried out to place the virtual color location marks on one or more of the obtained macroscopic images of the grossing specimen for display in a viewer. ### Claim 3 (depends on 1) The method of claim 1, wherein the electronic identification is carried out by electronically tracking movement of a color applicator instrument in a grossing workstation. ### Claim 4 (depends on 1) The method of claim 1, wherein the electronic identification comprises electronically automatically obtaining a series of macroscopic images of the grossing specimen during a grossing procedure of the specimen, and electronically applying image recognition to the obtained images to identify movement and location of the color applicator used to apply the color. ### Claim 5 (depends on 1) The method of claim 1, wherein the electronic identification comprises electronically automatically obtaining a series of macroscopic images of the grossing specimen during a grossing procedure of the specimen, electronically interrogating the images to detect a defined shape of a leading end portion of a color applicator to thereby identify where color is applied to the grossing specimen to generate the virtual color location mark, wherein the leading end portion of the color applicator has a distinct conspicuous visual appearance detectable by image recognition. ### Claim 6 (depends on 1) The method of claim 1, wherein the electronic identification is initiated with hands-free command or input to a color location identification circuit. ### Claim 7 (depends on 1) The method of claim 1, wherein the electronic identification is initiated by the color applicator instrument transmitting a wireless signal at a time of dispensing or otherwise applying color to a cut location on the grossing specimen. ### Claim 8 (depends on 1) The method of claim 1, wherein the automatically electronically generating the virtual color location marks comprises electronically generating respective electronic overlays of a color location object and displaying at least one of the overlays in a view of an acquired macroscopic digital image of the specimen. ### Claim 9 (depends on 1) The method of claim 1, further comprising concurrently displaying (i) a macroscopic image or model of the grossing specimen with at least one of the electronically generated virtual color location marks with (ii) at least one digital microscopic whole-slide image (WSI) of tissue from the grossing specimen, wherein a virtual color location mark associated with the at least one digital WSI is visually connected to a virtual color location mark associated with where a defined color associated with a tissue section from the WSI was applied on the specimen. ### Claim 10 (depends on 9) The method of claim 9, wherein the virtual color location marks are shown as respective object overlays on the macroscopic image or model with the visual connection provided by electronically highlighting a selected color location mark whenever it corresponds to a microscopic WSI image in a viewer. ### Claim 11 A method of obtaining and/or processing digital pathology and cytology images for viewing, comprising: electronically automatically identifying a location on or in a grossing specimen where at least one color is applied during a grossing procedure; electronically automatically generating virtual color location marks on a digital macroscopic image or model of the grossing specimen with a correct spatial position on the image or model of the grossing specimen relative to the grossing specimen based on the electronic identification; concurrently displaying (i) a macroscopic image or model of the grossing specimen with at least one of the electronically generated virtual color location marks with (ii) at least one digital microscopic whole-slide image (WSI) of tissue from the grossing specimen, wherein a virtual color location mark associated with the at least one digital WSI is visually connected to a virtual color location mark associated with where a defined color associated with a tissue section from the WSI was applied on the grossing specimen; electronically automatically obtaining a plurality of macroscopic images of the specimen during the grossing procedure of the specimen, including at least one base macroscopic image obtained prior to any physical cutting to obtain tissue samples, and placing the virtual color location marks on one or more of the obtained macroscopic images of the specimen for a viewer macroscopic image; and electronically adjusting for movement of the grossing specimen that can occur during the grossing procedure using the base image and one or more of the subsequent plurality of images to register respective virtual color locations to the viewer macroscopic image. ### Claim 12 A method of processing digital pathology and cytology images for viewing comprising: concurrently displaying (i) a macroscopic image or model of a grossing specimen with at least one electronically generated virtual color location marks with (ii) at least one digital microscopic whole-slide image (WSI) of tissue from the grossing specimen, wherein the at least one WSI can have different colors associated with different color location marks; electronically identifying a side color of a respective WSI, the side color being a color associated with color applied to an external surface area of the grossing specimen or an external surface area of a dissected tissue sample thereof; and rotating a view of the at least one WSI to automatically display the WSI in an orientation based on the side color in the WSI irrespective of whether the side color is along a long or short side of the WSI before the rotation. ### Claim 13 (depends on 1) The method of claim 1, further comprising: electronically identifying a defined color on a side of the WSI of the color mark associated with one of the at least one color applied to the grossing specimen in a respective microscopic whole-slide image (WSI); and electronically rotating the WSI so that the defined color on the side of the WSI of the color mark has a defined common orientation in a display associated with a viewer irrespective of whether the defined color is along a short or long side of the WSI. ### Claim 14 (depends on 13) The method of claim 13, further comprising defining standardized viewing protocols for automatic rotation of views of the respective microscopic WSI image according to identified applied external surface colors, wherein a single defined color on the side of different WSI images in each view is used for the automatic rotation. ### Claim 15 A histology and/or cytopathology viewer, comprising: a display; and a viewer circuit in communication with the display configured to cause the display to present at least one digital microscopic whole slide image (WSI) of respective sections of tissue samples from a grossing specimen, wherein the viewer circuit is configured to analyze scanned WSI images for determining a defined color and location of an elongate color mark associated with an actual color mark applied to an external surface of the grossing specimen or tissue sample cut therefrom and which can be present on any of four sides of the scanned WSI images and rotate an orientation of the WSI image for the viewer to consistently provide views of the WSI images to the display in a common orientation with respect to a tissue sample location and orientation in or on the grossing specimen based on the determined defined color and location of the elongate color mark. ### Claim 16 (depends on 15) The viewer of claim 15, wherein the viewer electronically identifies the elongate color mark and adjusts the orientation of WSI images on a display so that the WSI images position the elongate color marks of a common color at a common orientation in the WSI image views irrespective of whether the elongate color marks are along a long or short side of a respective original WSI view. ### Claim 17 (depends on 15) The viewer of claim 15, wherein the viewer positions different WSI views with a short side up and others with a long side up based on the adjusted orientation. ### Claim 18 (depends on 15) The viewer of claim 15, further comprising defining standardized viewing protocols for automatic rotation of views of the WSI, wherein a single defined color of the elongate color mark in each view is used for the automatic rotation.