1.INTRODUCTION

Telemedicine, electronic patient records, online medical textbooks, remote education and so on just began to be put to practical use. Then a large amount of digitized images and displays used to observe them are going to prevail among wide-ranging scenes of medical practice, but they have not well proved to be used for reliable medical diagnoses. So far, some standards and guidelines have been established for the usage of grayscale images, but we have done little for color images yet.


2.BACKGROUND

The purpose to use image data in medicine is to make an exact medical diagnosis. So our final target is to ensure that physicians can make the same diagnosis looking at the digital images as they are looking at real patients or at long-established analog images.

The process of medical diagnosis based on usual visual data is broken down into several steps.
(A1) The subject of observation emanates proper light waves of various wavelengths. This step is affected by the characteristics of the surface of the subject and the lighting. There may be penetrated light or invisible electromagnetic waves converted into visible light, etc.
(A2) The light waves pass through an iris, a lens and a vitreous body, and reach to the retina. This step is affected by the color of the iris and the transparency of the path of the light waves.
(A3) The rod cells convert the brightness of the light waves into frequency modulated pulse signals and the three kinds of cone cells convert the color of the light waves into pulse signals, and they are transmitted to optic nerves. There may be many interactions between cells, but most processes are unclear. One important fact is that adaptation of the human retina to surrounding lighting is greatly affected to the signals produced by the sensor cells in the aspects of both color and brightness.
(A4) The signals are transmitted to the brain and converted into image data. This step is mentioned as visual sensation and includes the specialized process of various image abstraction.
(A5) The image is connected to some of the concept accumulated in the memory and given particular meanings. This step is mentioned as visual recognition and the most of its process is unclear.
(A6) The concept is compared to the experiences accumulated in the brain, and deducted by the knowledge established in the brain. This step is mentioned as decision making and its process is in the mystery yet.

On the other hand, the process of medical diagnosis based on digital images is as follow:
(B1) Same as (A1).
(B2) The light waves are converted into digital image data that are composed of color
information of each pixel represented by RGB system and the coordinates of the pixel on the two-dimensional plane. In this step, not only a large part of information that original lights have is lost, but also various biases are mixed which depend on the characteristics of equipment used for A-D conversion.
(B3) The digital image data is stored or transferred to be used at another time or place.
(B4) According to the color information of each pixel of the digital image data, the R, G and B dots of the display equipment corresponding to the pixel emanate red, green and blue light each with appropriate combination of intensities to give the same stimulus to retina as the light wave which has that color. Because there is a large discrepancy between the wavelengths of red, green and blue lights and that of the peaks of the response curve of three types of the cone cells, some kinds of approximation are indispensable in this step.
(B5) The light waves pass through an iris, a lens and a vitreous body, and reach to the retina. This steps is affected by the color of the iris and the transparency of the path of the light waves. Unlike natural lights, these lights include only three bands of the wavelength. Therefore, these influences may cause different results from (A2).
(B6-9) Same as (A3-6)
The steps from (B2) to (B5) are possibly cause to wrong diagnoses.

One of the forefront researches about the diagnostic reliability of digitized color images has promoted by the Morphological Internet Survey Research Project Team. The Ministry of Science, Education and Culture of Japan organized it in 1998, and it consists of nine researchers from seven Universities and 29 co-researchers from various fields of laboratory medicine and clinical pathology in Japan [1] (in detail, consult its home page at http://square.umin.ac.jp/survey/).
The original purpose was proper application of the internet to the control survey of the morphological laboratory tests. But since the preliminary experiment revealed the urgent necessity of standardization of the color information of digitized images, how to calibrate and how to verify the color of display equipment have been its main subjects.


3.RECENT PROGRESS OF THE RESEARCH [2]

1) To investigate the digitized images sent through the internet are properly used for the control survey of the morphological laboratory tests instead of ordinary slidefilms, the pictures of typical specimens from urinary tests, hematology, microbiology, immunology, physiology and pathology were digitized and the diagnostic reliability were evaluated using various display equipment including a extra high resolution (QSXGA, 200 pixel per inch) LCD.
2) The qualities of most of digitized pictures are almost the same as slidefilms, but some of them required to be observed with the QSXGA display to maintain proper qualities.
3) Besides, there are large variations in reproduction of colors among the displays, which may incidentally cause erroneous diagnoses (Fig.1).
4) To overcome this problem, a new calibration system based on a novel way of thinking has been developed.
5) To discuss the same problem possibly arising in other medical fields, the 1st Symposium of the 'Color' of Digital Imaging in Medicine was held on 8th-9th May, 1999 at Tokyo Medical and Dental University, and understanding about this problem and a consensus of the orientation toward its solution were well advanced (in detail, consult a home page at http://square.umin.ac.jp/medicolor/).

These displays have almost the same resolution, and the major difference among them is the performance of color reproduction. The highest grade is 6, and grade less than 3 means unusable. Beyond prior expectation, grades of equipment No. 7 was varied from the highest to unusable. Specimen M-01 etc. got the highest grade with every equipment, but specimen M-06 varied from the highest to unusable according to equipment. Usually users looks at only one display, so they cannot notice this fact, which may incidentally cause erroneous diagnoses according to the combination of medical images and displays.

4.OVERVIEW OF THE RELATED FACTORS [3]

4.1 Precise Reproduction of the Color
At present, the reproduction of the color of CRT displays and some LCD displays can be calibrated with proper commercial equipment. But others such as plasma displays, digital projectors and head mount displays, which are possibly also used to observe the medical images, cannot be yet.
As the range and the variety of colors synthesized by the equipment are very different, it is impossible to equalize the physical specification related to colors among them. The device to adjust the colors of digital images according to the characteristics of each display so as to be sensed as equally by human eyes as possible is called the color management technology. It has been developed initially in the DTP field and must be modified to satisfy the medical requirements. The former should make each printed image equal to the original one, and the latter should make each displayed image equal to the standard image acquired in the brain of experienced physicians.
The human sensation of the color is affected by lighting surrounding displays. To reproduce the precise color under various lighting, the multispectral color data as well as the lighting data are collected simultaneously and used to calculate colors to be displayed equally under another lighting.

4.2 The Color and the Medical Diagnosis
Previous research reveals that the subjects which should be detected in a medical image as well as its physical characteristics greatly affect the required degree of digitizing precision to keep the same diagnosis as the original. Empirically, pictures should be more precise than movies, and grayscale images should be more precise than color ones. In our experiment, a few microscopic photographs of microbiology were not diagnosed properly with some displays that can reproduce less number of colors than others, though they have the same resolution as others. And this situation was greatly improved when QSXGA LCD was used.
The influence of the color on the medical diagnosis has been still unclear, but our experiment has shown that the poor reproduction of color information possibly interferes the proper diagnoses, and that the resolution of displays is also one of the most important factors for the diagnostic reliability.
The process of visual recognition of the human brain has been only fragmentary elucidated, and the study of the relationship between the recognition process and the medical diagnosis is still in a hypothetical stage.


5. A NEXT STEP TO THE STANDARDIZATION

It will take still a more time to establish a comprehensive theory to manage the color in medical imaging. And it will be not practical that every medical terminal should be equipped with expensive displays exclusively for the medical use. So some simple and inexpensive methods to calibrate common displays should be urgently provided.
The research team considered the differences in reproduced colors among displays might be allowed as far as they do not affect the medical diagnosis. And a set of typical medical images with their diagnoses decided by authorities in advance are proposed to be used as a practical calibrator. Namely, medical professionals can evaluate and adjust their displays by comparing the diagnoses of the images observed on their displays with authorized ones.
Some key technologies essential for this concept are on the process of protecting patents, and research collaboration for realizing it as well as an international research organization have just been organized. Now, any specialists or researchers interested in the color of living beings are invited to join it.


ACKNOWLEDGMENTS

A part of this work was supported by a Grant-in-Aid from the Ministry of Science, Education and Culture of Japan (Grant No. 10672172, entitled 'Internet Based Control Surveys for the Morphological Laboratory Tests', 1998-1999). I am grateful to all of the members of the Morphological Internet Survey Research Project Team and the Executive Committee for the Symposium of the 'Color' of Digital Imaging in Medicine for their great support to the research project.


REFERENCES

[1]

Nishibori, M., Itoh, K., Watanabe, K., Kanno, H., Ohba, Y., Use of WWW in a Control Survey of Morphological Laboratory Tests. Proceedings of the Ninth World Congress on Medical Informatics, 803, 1998.

[2]

Nishibori, M.: The Interim Report of the Morphological Internet Survey Research Project Team (Japanese), The Proceedings of the 1st Symposium of the 'Color' of Digital Imaging in Medicine, 55-69, 1999.

[3]

The Proceedings of the 1st Symposium of the 'Color' of Digital Imaging in Medicine (Japanese), 1999.