Martin Masek

Home

Teaching

Research

• Mammograms

HIERARCHICAL SEGMENTATION OF MAMMOGRAMS BASED ON PIXEL INTENSITY*

* this research was undertaken at the Centre for Intelligent Information Systems at The University of Western Australia.

OVERVIEW

My PhD research involved the application of image processing and computer vision to digital mammograms, particularly segmentation and classification.  The implementation details are in my thesis.  Complete result datasets are in the results section further along this page.  Following is a non-technical overview.

Looking at the mammograms below, they don't only show an image of the breast.  Also present are other various artefacts.  Some of these may be useful (labels, etc.), and some are just noise caused by the imaging and scanning process.  Using various image processing algorithms, the images in (a), (b), and (c) were processed and different types of background artefacts segmented using different colours.  This resulted in the segmented images in (d), (e), and (f).

With the background 'noise' identified, the subsequent image processing algorithms can be taught to ignore the noise and focus just on the breast within the image.

Knowing the boundary of the breast is important for applications such as comparison between left and right breast.  The image in (a) below was split up into several sections (b), (c), (d), and (e) and a breast boundary found in each section by fitting a polynomial function to the outline.  This was done accurately and automatically by using an optimisation technique.  The polynomial was fitted only to pixels of certain intensities in order to maximise the polynomial fit and smoothness.

Once a boundary was obtained for each section, they boundaries were integrated into the original image to result in a single breast outline.  This took a bit more effort, since the boundaries from each section often did not match perfectly.  The process is illustrated below using examples from two images.

The triangular shape in the mammogram images is caused by the pectoral muscle.  It's shadow can make lesions harder to spot, although it can also be useful as a reference point when looking for differences between images of the left and right breast.  To extract the pectoral muscle, properties of the image such as intensity gradients and symmetry were first used to determine which orientation the image was in.  Knowing the orientation means that you can be certain about which corner of the image the pectoral muscle lies in.  Further image processing examined this corner and fitted a straight line to the pectoral muscle edge.  Results are shown below, where images in (a) - (d) are re-oriented and the pectoral boundary is found.

Knowing where the breast and pectoral muscle are in the image leads to other useful applications.  Taking the segmented breast from the mammogram and removing the pectoral muscle leaves only the breast tissue.

The actual appearance of the breast tissue is different for each breast.  One reason for the different appearance is the mix of tissue types within the breast.  Fatty tissue appears dark on the mammogram, while 'active' glandular tissue appears bright.  The term density is used to describe the appearance.  The more glandular tissue within the breast, the brighter it will appear and thus the breast is said to be denser than one with less glandular tissue.

Knowing the density of a breast is useful.  In terms of determining the risk of developing an abnormality, a larger amount of glandular tissue increases the risk.  Also, mammograms of dense breasts are harder to read.  Lesions often appear as bright spots on the mammogram.  If the bright white spot is in the middle of bright white glandular tissue it is harder to find.

A new algorithm was devised for interpreting the appearance of the breast tissue.  This algorithm was used to classify mammograms into different density types and also to compare mammograms for simmilarity.  In order to interpret a mammogram, it may be useful to look at mammograms that are similar and examine their diagnosis.  In the example shown in the figure below, image '0' is the reference image.  The other images in the database were ordered in terms of similarity to image '0'.  Some examples are shown with the number indicating their rank.  For example, the second image is labelled '11', so 10 other images in the database were judged as looking more like image '0' than this image.

RESULTS

Included here are segmentation results from the MIAS mammogram database.  All results are from automatic algorithms - and as a result include false positives.  The results are either in the form of overlays - where the boundary of the extracted region is superimposed over the image, or as binary masks showing the extracted region in white.

Before processing, the spatial resolution of the MIAS images was reduced from the original 50mm/pixel to 400mm/pixel using the mean of each non-overlapping 8x8 pixel neighborhood; the grayscale intensity range was kept at 8 bits/pixel (256 grey levels).  This should be noted when using the masks to work with the original images.

The reduction in image size as a result of reduced resolution leads to advantages in that the images can be stored and accessed more easily and operations performed on the images are faster.  Since this work covers the segmentation of relatively large regions of the mammogram, algorithm performance is not adversely affected by the reduced resolution.

The work performed can be divided into the following, published references are provided where available - publication copies can be provided on request, so don't be shy to ask ;)

·        Pre-processing

o       Image orientation [12, Chapter 7] [8].

o       Image Background removal [12, Chapter 7].

o       Label/Scanning Artefact/Scratch/Tape removal – partly covered in [12, Chapter 7], [3].

• Simple bright image background noise location (can extract part of the breast) [Overlays - Red]

• Bright image background noise location (rectangular label + scanning artifact) [Binary Masks]

• Vertical scratch location

• Location of rectangular labels [Binary Masks]

• Edges of horizontal 'tapes' [Binary Masks][Overlays]

·        Segmentation

o       Multi-level global thresholding [12, Chapter 6].

• Near-Skin breast boundary [Binary Masks] –  [12, Chapter 6]

o       Pectoral muscle – straight line approximation [12, Chapter 9], [9].

• Pectoral muscle [Binary Masks]

o       Skin-Air interface.

• Skin-Air interface [Overlays] [Binary Masks] - [12, Chapter 8]

·        Classification

o       Database ordering for non-random presentation to radiologist – [10], [12, Chapter 10].

o       Breast type classification – [11], [12, Chapter 10].

PUBLICATIONS

[12] M. Masek, Hierarchical Segmentation of Mammograms Based on Pixel Intensity, PhD Thesis, The University of Western Australia.

pdf version

[11] M. Masek, S.M. Kwok, C.J.S. deSilva, Y. Attikiouzel, "Classification of Mammographic Density Using Histogram Distance Measures", CD-ROM Proceedings of the World Congress on Medical Physics and Biomedical Engineering, Sydney, Australia, 1pp, August 24-29, 2003.

[10] M. Masek, C.J.S. deSilva, Y. Attikiouzel, "Image Retrieval Based on Histogram Comparison for Digital Mammography Workstations", CD ROM Proceedings of the International Congress on Biological and Medical Engineering, Singapore, 2 pp, 4th-7th December 2002.

  [9] M. Masek, C.J.S. deSilva, Y. Attikiouzel, “Comparison Of Local Median With Modified Cross-Entropy For Pectoral Muscle Segmentation In Mammograms”, Proceedings of the 16th Biennial International Conference Biosignal 2002, Brno, Czech Republic, pp 320-322 ,June 2002.

  [8] M. Masek, C.J.S. deSilva, and Y. Attikiouzel, “Automatic Breast Orientation in Mediolateral Oblique View Mammograms,” in Digital Mammography 2002 (H.O. Peitgen, Ed.). Proc. 6^th Int. Workshop on Digital Mammography, Springer-Verlag, 2003, pp. 207-209.

  [7] M. Masek, R. Chandrasekhar, C.J.S. deSilva, Y. Attikiouzel, "Spatially Based Application of the Minimum Cross-Entropy Thresholding Algorithm to Segment the Pectoral Muscle in Mammograms", Proceedings of the Seventh Australian and New Zealand Intelligent Information Systems Conference, Perth, Western Australia, pp 101-106, 2001.

  [6] M. Masek, "Automatic Breast Orientation in Mediolateral Oblique View Mammograms", Australian Research Centre for Medical Engineering, Technical Report AR200106, 2001.

  [5] M. Masek, Y. Attikiouzel, C. J. S. deSilva, "Combining Data from Different Algorithms to Segment the Skin-Air Interface in Mammograms", CD-ROM Proceedings of the World Congress on Medical Physics and Biomedical Engineering, Chicago, Illinois,  4 pp , July 23-28, 2000.

  [4] M. Masek, Y. Attikiouzel, C. J.S. deSilva, "Skin-Air Interface Extraction from Mammograms Using an Automatic Local Thresholding Algorithm", Proceedings of 15th Biennial International Conference Biosignal 2000, Brno, Czech Republic, pp 204-206 June 2000.

  [3] M. Masek, Y. Attikiouzel, and C. J. S. deSilva. "Automatic Removal of High Intensity Labels and Noise from Mammograms" in Digital Mammography, IWDM 2000. Martin J. Yaffe (ed.), Proceedings of the 5th International Workshop on Digital Mammography, Toronto, Canada, June 2000. Madison, WI: Medical Physics Publishing, pp. 576-581, 2001.

  [2] M. Masek, "Automatic Segmentation of Mammograms", Inter-University Postgraduate Electrical Engineering Symposium, pp. 47-48, Perth, WA, Australia, 5-6 July 1999.

  [1] M. Masek, Computer Support for the Disabled – Predictive Scanning Keyboard.  Honours Thesis, Information Systems Engineering Research Group, Department of Electrical and Electronic Engineering, The University of Western Australia, Perth, Australia, 1997.