Contrast and Microscopy 22 ˇ Basics of light Microscopy & iMaging contrast and Microscopy All cats are grey in the dark Why are cats grey in the dark? Here, the term contrast comes into play. Contrast refers to the difference of intensities or colours within an image. Details within an image need intensity or colour differences to be recognised from the adjacent surroundings and overall background. Let us first restrict our view to a greyscale level image, like image fig. 26. While watching the image try to estimate the number of grey scales that you can distinguish ­ and keep that number in mind. Let us now compare this number with another example ­ you enter an office shop in order to buy a selection of card samples in different shades of grey. All of the cards fall down by accident and are scattered over the floor. Now you have to replace them in the correct grey level order ­ how much can you differentiate now? Surprisingly, we are only able to differentiate approximately 50­60 grey levels ­ that means that already the 8 bit image on your monitor with 256 grey scales offers greater possible differentiation than we can discriminate with our own eyes. The card samples in various shades of grey need to a have an approximate difference in contrast level of about 2% in order for us to recognise them as different. However, if we look at the number of image areas (pixels) that represent a discrete intensity (grey level or pixel intensity) within an intensity distribution of the image we can understand and handle contrast and brightness variations more easily. Intensity or grey level distributions are referred to as histograms, see box 8 for further explanation. These histograms allow us to optimise camera and microscope settings so that all the intensity values that are available within the specimen are acquired (fig. 27). If we do not study the available intensity values at the initial stage before image acquisition, they are archived and can not be visualised in additional processing steps. The familiar view ­ brightfield contrast In brightfield transmitted microscopy the contrast of the specimen is mainly produced by the different absorption levels of light, either due to staining or by pigments that are specimen inherent (amplitude objects). With a histological specimen for example, the staining procedure itself can vary the contrast levels that are available for imaging (fig. 27). Nevertheless, the choice of appropriate optical equipment and correct illumination settings is vital for the best contrast. During our explanation about Köhler alignment (see box 2) we described that at the end of this procedure the aperture stop should be closed to approximately 80% of the numerical aperture (NA) of the objective. This setting is to achieve the best contrast setting for our eyes. Further reduction of the aperture stop will introduce an artificial appearance and low resolution to the image. For documentation purposes however, the aperture stop can be set to the same level as the NA of the objective ­ because the camera sensors in use are capable to handle much more contrast levels than our eyes. For specimens lacking natural differences in internal absorption of light, like living cells (phase objects; fig. 28) or Fig. 26: Vitamin C crystals observed in polarised light. Fig. 27: Histological staining. Cells obtained after a transbronchial needle application. Image and histogram show that the overall intensity and contrast have been optimised. contrast and Microscopy Basics of light Microscopy & iMaging ˇ 23 reflected microscopy specimens without significant three dimensional relief structures, the acquired image has flat contrast. To better visualise the existing image features (fig. 28, original image), subsequent digital contrast optimisation procedures may be applied (fig. 28, improved image). But to visualise more details in those specimens optical contrast methods must be used. Like stars in the sky ­ Darkfield Contrast Dust in the air is easily visible when a light beam is travelling through the air in a darkened room. The visibility is only achieved because the dust particles diffract and/or reflect the light and this light is now travelling in all directions. Therefore we can see light originating from the particle in front of a dark background even when the particle itself is too small to be resolved or does not show an appropriate contrast under daylight conditions. This phenomenon is also used in darkfield (or dark ground) microscopy. Light is directed to the specimen in a way that no direct light enters the objective. If there is no light scattering particle the image is dark, if there is something that diffracts or reflects the light, those scattered beams can enter the objective and are visible as bright white structures on a black background (fig. 29). (See also: http://www.olympusmicro.com/ primer/techniques/darkfieldindex.html) Transmitted darkfield For transmitted microscopy including stereo microscopy, this contrast method is especially used to visualise scattering objects like small fresh water micro-organisms or diatoms and fibres (fig. 30). Almost all upright microscopes can be easily equipped for darkfield illumina- tion. The easiest way to achieve simple darkfield is a central light stop insert for the condenser. For better illumination or even high resolving darkfield, special darkfield condensers are required (fig. 29). To ensure that no direct light is entering the objective, the numerical aperture (NA) of the condenser has to be about 15% higher than the NA of the objective. This is in contradiction to all other contrast methods where the objective has a higher or equal NA than the condenser. Remember the NA is a number that describes the angle of the light cone a condenser or objective is using. An objective with high NA like the apochromate is characterised by a high angle of its light cone and therefore it may be possible that some of the direct illumination will also enter the objective. This would destroy the darkfield contrast immediately. For that reason objectives with high NA are Fig. 28: Brightfield image of living mouth epithelial cells on a slide before and after digital contrast optimisation of the archived image and corresponding histograms. See the section "Making it look better", for further explanation. Box 8: Histogram optimisation during acquisition An intensity histogram depicts the intensity distribution of the pixels in an image. The intensity values are plotted along the x axis and range from 0­255 in an 8-bit greyscale image or 24-bit (3x8bit) colour image. The number of pixels per intensity value is displayed along the y axis. The intensity histogram provides the means to monitor general characteristics of a digital image like its overall intensity, its contrast, the dynamic range used, possible saturation, the sample's phases etc. In this respect the histogram gives a more objective criterion to estimate the quality of an image than just viewing it (which is somewhat subjective).When displayed in the live mode during image acquisition, the histogram allows optimising and fine tuning of the acquisition modes and parameters on the microscope as well as the camera. These include microscope alignment, contrast method, microscope settings, or current camera exposure time. So the histogram helps to acquire a better image containing more image information. Left figure is obviously lit correctly but has poor contrast. The corresponding histogram mirrors this: the peak is roughly located in the middle of the intensity range (x-axis), so the camera's exposure time is correctly set. But all pixels are squashed in the middle range between about 75 and 200. Thus, only about half of the dynamic range of the camera system is used. Aligning the microscope better (e.g., light intensity) and adapting the camera exposure time increases the image contrast (right figure). It also spreads the intensity distribution over the whole dynamic range without reducing information by saturation (see the corresponding histogram). Here, more detailed image structures become visible. General rule The overall contrast is usually best when the intensity histogram covers the whole dynamic range of the system; but one should usually avoid creating overflow or saturation at the right side of the histogram, i.e. white pixels. It is also possible to use the histogram to improve the image contrast afterwards. However, you can not increase the image content; you can only improve the image display. Epithel cells viewed with phase contrast: left side with low contrast setting, right sight with contrast optimisation of microscope and camera setting. 24 ˇ Basics of light Microscopy & iMaging contrast and Microscopy Fig. 29: Light path for darkfield compared to brightfield set up in transmitted and reflected illumination. available, designed with an internal iris diaphragm to reduce the NA to the appropriate amount for darkfield observation. Reflected darkfield Within the reflected microscopy applications the darkfield illumination is a very common contrast technique. It allows the visualisation of smallest scratches and changes in height because of the circular oblique illumination (fig. 31). To achieve a reflected darkfield, the amount of special adaptations at the microscope is more sophisticated. The central light stop is located within a cube of the reflected light attachment and the special brightfield/darkfield objective guides the illumination light in an outer ring to the specimen. Only the scattered light from the specimen runs in the normal central part of the objective as image forming light rays (fig. 29). Those objectives can also be used for normal brightfield observation or other techniques like differential interference contrast (DIC) and/or polarisation. Creating destructive interference Phase Contrast Light that is travelling through part of a specimen and is not absorbed by amplitude objects will not produce a clearly visible image. The intensity remains the same, but the phase is changed compared to the light just travelling in the surrounding areas. This phase shift of about a quarter wavelength for a cultured cell is not visible to our eyes. Therefore, additional optical elements are needed to convert this difference into an intensity shift. These optical elements create a contrast where un-deviated and deviated light are wavelength out of phase, which results in destructive interference. This means that details of the cell appear dark against a lighter background in positive phase contrast (see figures in box 8). (See also: www.olym- pusmicro.com/primer/techniques/phase- contrast/phaseindex.html) For phase contrast microscopy two elements are needed. One is a ring slit insert for the condenser, the other is special objectives that contain a phase plate. Objectives for phase contrast are characterised by green lettering and an indication of the size of the phase ring like Ph1, Ph2, Ph3 or PhC, PhL, PhP. Corresponding to these objective phase rings the appropriate inserts for the condenser have to be used. Both elements are located at the so called back focal planes. They are visible Box 10: Alignment of phase contrast 1. A microscope condenser with Köhler illumination has to be in Köhler positioning. 2. Remove one of the eyepieces and look into the empty eyepiece sleeve.When using a centring telescope at the eyepiece sleeve (highly recommended and in some inverted microscopes already build in at the tube e.g. Olympus U-BI90CT ­ binocular tube), bring the bright ring (condenser ring slit) and dark ring (objective phase plate) into focus by turning the eye lens focus. 3. Use the centring screws for the condenser inserts to centre the phase contrast ring, so that the bright ring overlaps the dark ring within the field of view (see figure). 4. Repeat these steps for each phase and contrast ring set. 5. Remove the centring telescope and replace it with the eyepiece. 6. Widen the field iris diaphragm opening until the diaphragm image circumscribes the field of view. Box 9: Alignment of a transmitted darkfield condenser 1. Engage the 10x objective and bring the specimen into focus. 2. While looking through the eyepieces and using the condenser height adjustment knob, carefully adjust the height of the condenser until a dark circular spot becomes visible (Figure A). 3. Turn the condenser centring screws to move the dark spot to the centre of field of view (Figure B).This completes the centration. 4. Engage the desired objective. Using the condenser height adjustment knob, adjust until the darkfield spot is eliminated and a good darkfield image is obtained. contrast and Microscopy Basics of light Microscopy & iMaging ˇ 25 when an eyepiece is removed and can then be aligned under optical control (for better viewing a centring telescope should be used). Due to the optical principles of phase contrast it allows good contrast in transmitted light when the living specimens are unstained and thin. A specimen should not be more than 10 m thick. For these specimens the dark contrast is valid, whereas thicker details and overlaying structures produce a bright halo ring. This halo-effect can be so strong and superimposed that a clear analysis of the underlying morphology becomes critical. Nevertheless, the artificial looking image of thicker structures can be used by expert eyes to determine how many cells within an adherent cell culture are undergoing mitosis or have entered cell death pathways. Fig. 32 shows bright halo rings, visible round the relatively thick astrocyte cell bodies, whereas the fine details show dark phase contrast. When observing cells in a chamber e.g. those of a 24 well plate, the cells which are best for imaging may lie at the border of the well. It is possible to see the cells, but because the light path is changed by the border of the well and the adhesion effect of the medium, the phase contrast is totally misaligned at this position. We can realign the phase rings for this specific specimen position as described below, but will have to realign them again when the stage is moved to other positions. Furthermore, the use of the special Olympus PHC phase contrast inserts instead of the PH1 helps to ensure better contrast in multi well plates where meniscus problems are ap- parent. Making it look better ­ for vision only It is essential to select the optical contrast method appropriate to the sample investigated and depending on the features which are to be made visible. You can only build upon something which is already there. Having done the best possible job here you can try to better visualise these features. It often makes sense to adjust the image contrast digitally. Here you can either elevate the overall image contrast or you can emphasise special structures. Whatever you want to do, you should again utilise the histogram to define exactly the steps for image improvement. Fig. 28 shows a drastic example (left side). The optical contrast is so poor that the cells' relief is hardly visible. The corresponding histogram shows that the Fig. 32: Astrocytes with phase contrast, note: cell bodies are surrounded by halo rings. Fig. 33: Epithelial cells scratched with a spoon from the tongue and transferred to a cover slip. A,B: DIC illumination at different focus plane; C: Oblique contrast; D: Olympus Relief Contrast imaged with lower numerical aperture. Fig. 30: Epithelial cells with transmitted darkfield contrast. Fig. 31: Wafer at reflected darkfield contrast. 26 ˇ Basics of light Microscopy & iMaging contrast and Microscopy Fig. 34: Simple principle of Nomarski DIC microscopy. pixels are grouped in the centre of the histogram indicating that only a small fraction of the camera's dynamic range has been used (intensity values 165­186 out of 256). The image has only 21 different intensity values. The present features can be made visible by stretching the histogram over the whole dynamic range from 0­255. This operation does not change the information content of the image but lets us at least see what is there (right side). This stretching comes to its limits as soon as there is a single black and a single white pixel in the image. In this case a moderate cut of up to 3% on the left side (dark pixels) and on the right side (bright pixels) doesn't cause much information loss but increases the contrast of the image's main features in the central intensity range. To selectively accentuate features of a distinct intensity range (socalled phase), an individual transfer function has to be defined which increases the phase's contrast to the background or other phases. The intensity and contrast image operations are usually performed after image acquisition. Often, the camera control of the image analysis software makes it possible to have the contrast stretched image calculated and displayed in the live mode. Alternatively, the contrast can be set manually on the live histogram while the direct results can be monitored in the live image. Whatever way the digital histogram is used, it is a powerful tool to maintain image fidelity as well as to create a clear picture of physical na- ture. Light alone is not enough ­ Differential Interference Contrast (DIC) If you have ever been skiing on a foggy day, you will know that diffuse light strongly reduces the ability to see differences in the height of the snow ­ and that can cause major problems. An equally distributed light source does not produce clear shadows and this causes reduced visibility of three dimensional structures. Our human vision is triggered to see three dimensions and is well trained to interpret structures if they are illuminated more or less from one point. The resulting dark and bright areas at the surface of a structure allow us to easily recognise and identify them. Using our experience, we get information of height and distance. Therefore, a contrast method that displays differences in a structure as a pattern of bright and dark areas is something that looks very familiar to us and seems to be easy to interpret. Structures within a specimen can be identified and even though they are only two dimensionally displayed they look three dimensional. Real three dimensional images can only be observed at a stereo microscope where two light paths of two combined microscopes are used, sending the image to our eyes at a slightly different angle. But this will be a topic to be described later. Simple from One Side The simplest way to achieve a contrast method that results in a relief-like image is by using oblique illumination: however, the theory of oblique illumination is more complex. Details can be found at: www.olympusmicro.com/primer/ techniques/oblique/obliquehome.html. Special condensers are available to handle this technique on upright microscopes with a lot of comfort. Oblique illumination is also often used on stereo microscopes to enhance the contrast of a 3D surface simply by illuminating the specimen from one side. For reflected light, this can be done with flexible cold light fibres, or with LED ring light systems that offer reproducible illumination of segments that can even be rotated. Box 11: How to set up the transmitted DIC microscope (e.g. Olympus type) J Use proper Köhler positioning of the microscope with a specimen in focus (best with 10x objective). J Insert polariser (condenser side) and analyser (objective side) into the light path. J Remove specimen out of the light path. J Turn the polariser until the image gets to its darkest view (Cross-Nicol position). J Insert the DIC prism at the condenser corresponding to the objective and slider type in use, immediately the image is bright again. J Insert the DIC slider at the objective side. By rotating the fine adjustment knob at this slider the amount of contrast of the DIC can be varied. J The best contrast is achieved when the background shows a grey colour and the specimen is clearly pseudo three dimensional. Another Way to Achieve this Setting Is the Following: J Use proper Köhler positioning of the microscope with a specimen in focus (best with 10x objective). J Insert polariser (condenser side) and analyser (objective side) and the prism slider into the light path. J Remove specimen out of the light path. J Remove one eyepiece and if available look through a centring telescope, rotate the slider adjustment knob until a black interference line is visible (box 11- Figure ). J Rotate the polariser until the black line becomes darkest. J Insert the DIC prism at the condenser and insert the eyepiece for normal observation. J Fine adjustment of DIC-slider can be done for selecting the amount of interference contrast. contrast and Microscopy Basics of light Microscopy & iMaging ˇ 27 In transmitted microscopy the oblique condenser (e.g. the Olympus oblique condenser WI-OBCD) has an adjustable slit that can be rotated. After Köhler alignment of the microscope this slit is inserted in the light path and results in illumination from one side so that specimens that vary in thickness and density are contrasted. Rotating the slit enables us to highlight structures from every side. The contrast itself is produced by the complete thickness of the specimen and the resolution of the image is limited due to the oblique illumination (fig. 33c). To overcome the limitations of oblique contrast, Nomarski Differential Interference Contrast (DIC) is commonly used for high resolving images. The benefit of this method is that the relief like image is only contrasted at the focus area (depth of field). The user can optically section a thicker specimen by changing the focus level. As shown in fig. 33 the contrasted focus layer can be restricted to the layer of the somata (fig. 33a) or the superficial part of these cells (fig. 33b). In addition, using infrared light (mostly used around 700 or 900 nm) instead of white light, this technique allows a very deep look of more than 100 m into thick sections, which is often used in neurobiological research. Nomarski DIC creates an amplified contrast of phase differences which occurs when light passes through material with different refractive indices. Detailed information about the theory and use of the DIC method can be found at www.olympusmicro.com/primer/techniques/dic/dichome.html. Here we will concentrate on the basic introduction and the practical use. To achieve transmitted Nomarski DIC images, four optical elements are needed: a polariser, two prisms and an analyser. For the reflected DIC setup only one DIC prism (the slider) is required. Let us have a closer look at transmitted DIC. The wave vibration direction of light is unified by a polariser located between the light source and condenser Fig. 37: Transmitted polarising microscopy; variation of melted chemicals viewed with crossed polariser. Images courtesy of Norbert Junker, Olympus Europa GmbH, Hamburg, Germany. Fig. 35: Specimens imaged with different shearing values. Left side shows thin NG108 cells imaged with the high contrast prism set, the middle image shows a thick diatom specimen imaged with the high resolution prism set and the right image shows PtK2 cells imaged with the general prism set. Fig. 36: Alignment of Olympus Relief Contrast or Hoffmann Modulation condenser insert, grey and dark areas are located within the objective and areas A and B are located as an insert within the condenser and can be aligned accordingly. 28 ˇ Basics of light Microscopy & iMaging contrast and Microscopy (fig. 34). In the condenser, a special insert ­ a Wollaston prism (matching the magnification of the objective) ­ divides every light ray into two, called the ordinary and the extraordinary, which vibrate at a 90 degree angle to each other. Both light rays then travel a small distance apart, the so called shearing distance. At the specimen, the ray that is passing through e.g. a cell part is delayed compared to the one passing through the surrounding medium. This result in a phase shift of both rays, which are recombined with the help of a second prism located at the objective revolver. Only those combined rays with a phase shift, interfere in a way that they contain vibration planes that pass through the analyser. To create an easily observable pseudo 3D image, a prism can be moved in and out to enhance the phase shift between ordinary and extraordinary ray. At a mid position the background should be dark and the specimens should be visible as if illuminated from the equivalent North-West and South-West simultaneously. When the slider is screwed more to one direction the typical three dimensional view of the specimen will come up, either having the sun in North-West or East-South location. This will only work if no depolarising material (e.g. plastic) is used within the light path. Therefore, a real high resolving DIC method can not be used with plastic Petri dishes or multi well plates. In those cases other methods like Hoffmann Modulation Contrast or the analogous Olympus Relief Contrast are commonly used. The DIC method does not require special objectives, but the prism has to match the objective used. Suitable prisms are available for most fluorite and apochromat objectives. DIC contrast can be offered with shearing values for every optimisation, a high contrast set-up that is best for very thin specimens (higher shearing value, fig. 35 left), a standard prism slider for general use (fig. 35 right) and a high resolution slider for thick specimens (lower shearing value, fig. 35 mid) like C. elegans or zebra fish em- bryos. Different names but the same principle: Hoffman Modulation Contrast ­ Olympus Relief Contrast Whenever high resolution is needed and plastic dishes are used, the Hoffman Modulation or Olympus relief contrast method are common techniques for inverted transmission light microscopes e.g. for in vitro techniques (fig. 33d). In principle it combines the oblique illumination of a specimen where the reBox 12: How to describe image processing operations mathematically From the mathematical point of view, an image processing function is a transfer function which is applied to each pixel of the original image individually and generates a new pixel value (intensity/colour) in the resulting imaging. The transfer functions can be roughly divided into point operations, local operations and global op- erations. J Point Operations: The resulting pixel is only dependent on the intensity or colour of the original pixel, but it is independent from the pixel places (x/y) and the pixel intensities/colours of the neighbouring pixels. Point operations can be monitored, defined and executed via the histogram.All intensity and contrast operations are point operations.To be exact, point operations are not filters in the more specific sense of the word. J Local Operations: These look not only at the pixel itself but also at its neighbouring pixels to calculate the new pixel value (intensity / colour). For example, convolution filters are local operations. Many well known noise reduction, sharpening and edge enhancing filters are convolution filters. Here, the filter's transfer function can be described as a matrix consisting of whole positive or negative numbers, known as weight factors.Any original pixel is situated at the centre of the matrix.The matrix is applied to the original pixel and its neighbours to calculate the resulting pixel intensity. See box 12 ­ Figure 1a as illustration.The 3x3 matrix has nine weight factors W1 ­ W9. Each weight factor is multiplied with the respective intensities I1 ­ I9 of the original image to calculate the intensity of the central pixel I5 after filtering. See a concrete numerical example in box 12 Figure 1 b. In addition to this calculation, the resulting intensity value can be divided by a normalisation factor and added to an offset value in order to ensure that the filter operation does not alter the average image brightness. The matrix weight factors determine the function and the strength of the filter. For example, a 3x3 matrix with all weight factors being one makes a simple average filter used for noise reduction. Sharpen filters consist of matrices having a positive central weight factor surrounded by negative weight factors.The unsymmetrical matrix in fig. 43b creates a pseudo topographical contrast in the resulting image. J Global operations: All pixels of the original image with regard to their intensity / colour and position (x/y) are used to calculate the resulting pixel. Lowpass and bandpass noise reduction filters are examples. In general, all Fourier Transform filters are global operations. b) a) Fig. 38: Brightfield image of a wafer. Left side: The ring shaped background pattern is clearly visible. Right side: The same image with real-time shading correction. contrast and Microscopy Basics of light Microscopy & iMaging ˇ 29 fraction differences at various parts of the specimen shape are used for contrast enhancement. It allows semi-transparent specimens structures to be analysed in a manner difficult to achieve using bright field microscopy. (See also www.olympusmicro.com/primer/techniques/ hoffmanindex.html). For this contrast technique a special condenser and objective are needed. In advanced systems the condenser is equipped with a polariser and a condenser slit plate (according to the objective in use, fig. 36 A and B area). The achromatic or fluorite objective contains a modulator insert at one side of the back focal plane (fig. 36 dark and grey area). The slit plate at the condenser has to be aligned according to the modulator within the objective (fig. 36). This can be done by visualising these elements via a focused centring telescope (remove one eyepiece and insert the telescope). The resulting contrast can be varied by rotating the polariser at the condenser. The interpretation of DIC and relief contrast images is not intuitive. These techniques contrast different refractive indices within a specimen into a pseudothree-dimensional image. This means that specimen details which look like holes or hills on the surface of a structure (see fig. 35 left and right side) may simply be areas of different refraction index but not necessarily different in height. Get more then expected ­ Polarisation DIC equipment on a microscope allows the user to employ a totally different microscopic method, simple polarisation. In this case no prisms are used and only the first settings of aligning polariser and analyser in crossed positioning (see box 11) are needed. This setting will generate a dark image because the vibration direction of light that is travelling through the specimen is exactly the vibration direction that is totally blocked by the analyser. However, if the specimen contains material that is able to turn the light, some light can pass the analyser and is observed as a bright detail on a dark background. Examples of such anisotropic materials are crystalline Vitamin C, emulsions of butter, skeletal muscles, urate crystals (Gout inspection), amyloid, rocks and minerals, as well as metal surfaces or the DIC prism itself. The list is very long and often the combination of simple polarising microscopy and DIC can offer a more detailed analysis of the specimen, simply by using equipment that is already available. Beside all the analytical advantages, polarising microscopy also offers images of high aesthetical value (fig. 37). Revealing structures with imaging filter techniques We are now at the point where the suitable microscope contrast method has been selected and the optimum camera settings have been made. So the field has been tilled and digital filter and image processing techniques come into play. This is explained in more detail below. Along a thin line Image processing can turn the photograph of an ordinary face into an unbelievable beauty or create images that have no counterpart in the real world. Knowing this, digital image processing is often compared with manipulation of results. It might be true for many microscopists that they enjoy the amazing beauty of hidden nature as being mirrored in the images they acquire, but the focus of the daily work lies upon the actual information within the image. Yet this information can be superimposed by artefacts which might be created by specimen preparation, illumination conditions, camera settings or display parameters. At this point, filter techniques come into their own right. Depending on the reason for imaging, a wide variety of processing steps can be applied to reveal new information or to enhance image clarity for the details that are under observation. What kind of artefacts may appear in a digital image? Imperfect digital images may look weak in contrast, unevenly illuminated, wrongly coloured, diffuse, blurred, noisy, dirty etc. These distortions might make the image look unprofessional. But what is more, they might make any automatic measurement routine based on threshold values or edge detection difficult and sometimes even impossible to apply. Fig. 39: Fluorescence image of stem cells. The detail zoom better reveals the noise level. 30 ˇ Basics of light Microscopy & iMaging contrast and Microscopy Many of these artefacts can be reduced by using a suitable digital filtering technique. Often there exist several different filters to improve the image quality. It is not always easy to find the best method and parameter settings to suppress the image defects without disturbing the "real" image information. This is the highest standard which the optimum filter should meet: removing the image artefacts and keeping the specimen structures in the image. This goal cannot always be accomplished. When doing image processing, there is often a thin line between what needs to be done and what would be good to accomplish. In addition, there is a thin line between scientific virtue and creative composition! Now or later ­ both are possible Digital image processing is usually applied after image acquisition and image storage. But the latest image processing systems support "Live Digital Image Fig. 40: The same image as in fig. 39 after application of the Sigma noise reduction filter. The detail zoom makes the improvement better visible. Processing", which means that numerous real-time functions can be executed during image acquisition. These include the online histogram, which is used to monitor the image brightness and contrast. Additionally, the histogram function itself can be used to have the image contrast improved automatically or to set it in the live mode by hand. (How to use the histogram to improve image contrast see box 8.) Another real-time function is the white balance, which corrects colour shifts at the moment of acquisition. Nonuniform specimen illumination can also be immediately corrected in the live image using the online shading correction. This is described below. The operations of contrast enhancement, white balance and shading correction can be applied during or after image acquisition. They are point operations, mathematically speaking (refer to box 12). This makes them suitable for reducing image artefacts without distorting the image content itself. Where there is light, there is also shade One major source of distortion in a light microscopic image is brightness fluctuation, which may arise from out-of-axis illumination conditions or when the microscope is not aligned optimally. Uneven illumination artefacts may also appear under regular observation conditions using low objective magnification and/or in combination with low magnifying camera adapters. Then the shading artefacts are manifested as a ring shaped shade structure which becomes even darker near the edges of the image (fig. 38 left side). The minor shading within the background needs to be corrected. The operation applied here is called background correction, flat-field correction or shading correction. (A simple tool for background correction can be downloaded at www.mic-d.com.) The best method is to record a background image and a dark image in addition to the "raw" specimen image. The background illumination profile is simulated in the background image, whereas the noise level of the camera system is apparent in the dark image. The advantage here is that these images are gathered independently from the specimen image. The background image is usually obtained by removing the specimen and leaving an area of mounting medium and cover slip in place. If this is not possible, the same result can also be achieved by leaving the specimen in place but to defocus the microscope. It is important that the background image does not show any debris. The dark image is acquired by closing the camera shutter. It should use the same exposure time as the specimen image. The shading correction algorithm first subtracts the dark image (d) from the specimen image (s) as well as from the background image (b). Then it divides the corrected background image through the corrected specimen image: (s­d)/(bd). This procedure can also be performed automatically with the live image. Here, the background and the dark images have to be acquired only once for each microscope objective using a standard sample and standard acquisition parameters. This is how the image in fig. 38 (right side) was taken. If the background and dark images are not available, the shading correction can also be derived from the specimen image itself. For example, a very strong smoothing filter averages out all structures and reveals the background. Or a contrast and Microscopy Basics of light Microscopy & iMaging ˇ 31 multi-dimensional surface function can be applied to fit the background illumination profile. Just too much noise Random noise can be an annoying phenomenon which is encountered when the specimen signal is low and/or was enhanced with a high gain factor. An example from fluorescence microscopy is shown in fig. 39. In regular brightfield light microscopy images random noise is usually not visible. But it is often unsheathed when a sharpen filter is applied to the image. See fig. 42, first and second image as an example. There are different smoothing filters which can be used to reduce the noise. Smoothing filters are also applied to suppress artefacts deriving from small structures like dirt, dust, debris or scratches. Yet the challenge is to find a filter which eliminates the noise and the artefacts without smoothing the object edges too much. Fig. 41 shows an artificially created image having only one real structure: a simple rectangle of light shading upon a dark background. This structure is superimposed by two different kinds of distortion: strong statistical noise interference on one hand and so-called hot pixels or shot noise on the other hand. The shot noise is individual bright and dark pixels and can come from a defective camera. These pixels also mirror artefacts from dust particles, dirt, debris or small scratches. The noise reduction filters applied to the image are local or neighbouring operations, i.e., the neighbouring pixels are taken into account to calculate the new pixel value of the resulting image (see also box 12). The Mean and NxN filters just average everything and thus also broaden the structure of the rectangle (fig. 41). The more powerful the smoothing effect, the more noticeable the morphology of the object within the image will be altered. Shot noise will not be removed. Whereas the Mean filter applies fixed settings, the extent to which the NxN filter averages depends on the parameters set. The Sigma or Gaussian blur filter is a special average filter which does not affect any pixels deviating greatly from their surrounding area (fig. 41). So "real" structures and edges like the rectangle are not touched whereas the random noise disappears. The Sigma filter is the method of choice to reduce selectively statistical noise without greatly broadening the specimen structures (see also example in fig. 40). The Median and Rank filters eliminate the shot noise widely (fig. 41). They are especially suited to suppress dust and dirt, in general all small structures which stand out the underground. Comparing the effects the different smoothing filters have on the artificial sample image (fig. 41), the image's two different distortions could be eliminated best by applying two filters successively: first the Sigma filter against the statistical noise, and then the Rank filter against the shot noise. This leads to the somewhat philosophical rule which says that it is best not to strike everything all at once but to cut the problem into pieces and find the appropriate measure to deal with each piece sepa- rately. NxN-, Sigma- and Rank filters have user-definable controls which make it possible to adjust the smoothing effect optimally to the image. There is another option to reduce statistical noise, usually with fixed parameters, which gives quite good results: the lowpass filter. This filter is a global operation (refer to box 12) which filters out high frequency and periodic distortions. Yet here, the edges become somewhat broadened. An even better global operation is the bandpass filter which reduces the noise and preserves the steepness of edges. Revealing the details Another class of filters seen from the application point of view are the sharpen filters. Sharpen filters can be used to enhance fine image details. After processing an image with a sharpen filter, the image appears to look clearer. But sharpen filters have to be applied somewhat carefully because they can create artefacts themselves if overdone. The established sharpen filters are local neighbouring operations like the convolution filters (see box 12). The newer filters use the so-called unsharp mask algorithms. (Please have a look at fig. 42.) Sharpening the first, original image, gives the second image. Unfortunately, this enhances not only the structure but also the noise present in the original image. So the sharpen operation makes the noise visible. Additionally, the filter parameters have been set too aggressively so that the "lamellae" structures of the diatom look artificial. This context has to be kept in mind when applying a sharpen filter: Before the sharpen filter can be applied to an image, a noise reduction filter must often be applied first. Any Fig. 41: Comparison of different smoothing filters. See text for further explanation. 32 ˇ Basics of light Microscopy & iMaging contrast and Microscopy sharpen filter with user controls should be applied conservatively. Here, less is often more. The third image in fig. 42 shows the result after noise removal via a Sigma filter and successive moderate sharpening. The DCE filter is a specially designed sharpen filter and is part of the Olympus image processing software solutions. The letters DCE stand for Differential Contrast Enhancement. The DCE filter enhances weak differences in contrast. It selectively takes the lesser intensity modulations between neighbouring pixels and enhances them, while greater intensity modulations remain as they are. So the DCE filter renders image structures visible which are barely distinguishable in the original image. The filter works the better the more fine structures the image has. Resulting images are more detailed and appear more focused. See fig. 42, last image, and both images in fig. 43 as examples. Putting everything together To integrate what has been said above, there are some image processing operations which are meaningful or even necessary to apply to an image, either in real time or after acquisition. These operations can reduce image distortions, prepare the image for automatic measurements or just make the image look better. The following steps suggest a possible strategy to proceed with digital image operations when acquiring an image (see also: www.olympusmicro.com/primer/digi- talimaging/imageprocessingsteps.html) 1. Shading correction is to equalise uneven background illumination. 2. Contrast enhancement is to optimise brightness and contrast. 3. White balance is to adjust colour shifts. 4. Smoothing is to reduce random noise and suppress shot noise and small artefacts like dust and dirt. 5. Sharpening is to enhance fine edge detail. Here, we have covered a few of the wide range of possible filter operations. There are many more which go beyond scientific applications and transcend the pure picturing of the visible reality towards creative, funny, intelligent and beautiful compositions. Fig. 43: Epithelial cells viewed with phase contrast (left side) and transmitted darkfield contrast (right side). The original images are acquired with optimised microscope settings. These have already been shown in box 8 and fig. 30. The lower right parts of the images are each filtered with the DCE filter. Fig. 42: Brightfield image of a diatom. (a): original image. (b): after application of a strong sharpen filter. This increases the noise. (c): after application of the Sigma noise reduction filter and a moderate sharpen filter. (d): after application of the Sigma noise reduction filter and the DCE sharpen filter. contrast and Microscopy Basics of light Microscopy & iMaging ˇ 33