Analysis of microscope images_FINAL PRESENTATION

ANALYSIS OF MICROSCOPE IMAGES

George Livanos

Thesis presentationto

The Department of Electronics and Computer Engineeringin partial fulfillment of the requirements

for the degree of the Master Diploma

Technical University of Crete

Supervisory Committee:

Professor Zervakis Michalis (Supervisor) Professor Liavas Athanasios Professor Petrakis Euripides October 2013

Presentation scheme Introduction Thesis contribution Polarimetric imaging

Polarimetry modeling Preliminary results on tissue characterization

Microscope imaging Immunohistochemistry fundamentals Proposed cell segmentation and characterization Image clusteringWatershed transformActive contours membrane boundary estimationHer2 Grade evaluation

Results of tissue evaluation Segmentation performance Comparative estimations on Grade

Conclusion

Introduction Cancer is a disease that arises in molecular level In order to comprehend the cancer generation mechanism, a

deep study of the endocellular functions and extracellular interactions is necessary

Oncogene amplification and/or overexpression has been highly associated with breast tumor evolution

Medical advances have led to the identification of tumor biomarkers (onco-proteins) facilitating the understanding of the molecular basis of tumor progression and treatment response

Prognostic markers aim to objectively estimate the patient’s overall outcome

Predictive markers focus on the objective evaluation of the possible benefits from a specific clinical intervention

Introduction (cont’d) The HER2/neu oncogene is notable both for its role in the

pathogenesis of breast cancer and for its selection as a target of treatment

Overexpression of this receptor in breast cancer is associated with increased disease recurrence, poorer relapse-free survival and worse prognosis

Its predictive implications include resistance to hormonal therapy, resistance to chemotherapy , responsiveness to doxorubicin and, mainly, responsiveness to Trastuzumamb (Herceptin) therapies

Because of its prognostic role as well as its ability to predict response to Trastuzumamb, breast tumors are routinely checked for overexpression of HER2/neu

http://en.wikipedia.org/wiki/Trastuzumab

Introduction (cont’d) With the advent of molecular imaging, biological

and pathological disease processes can be determined in an intact living being by way of temporal and spatial distribution of exogenous probes or endogenous signals

Optical techniques span spatial scales from subcellular to organ level, relying on a disease-specific source of contrast as it modulates a measurable property of light

Backscattered optical polarimetric signatures from a target contain information related to target composition, structure, texture and geometry

Presentation scheme IntroductionThesis contribution

Thesis contribution In this thesis we study the process of tissue

characterization from both a macro and micro point of view

We study polarimetric imaging of tissue at macroscopic level and microscope imaging of stained cells at molecular level

By utilizing polarimetric imaging we extract characteristics of the tissue in a macroscopic level of analysis based on the optical identities of the tissue elements, studying their interaction with polarized light.

Information about the shape, size and physical state (solid or liquid) of such tissue elements can be extracted

By utilizing microscope imaging of tissue slides we are capable of studying tissue at a cellular level, obtaining information about its morphology, shape characteristics of cells and their distinct composite segments

Thesis contribution (cont’d) Diffused polarimetric reflection and backscattering provides

unique, discriminatory material signatures based on the depolarization of the impinging waves from different materials

By combining statistical analysis with polarimetric principles we develop an approach for analyzing the different properties of operational modalities and/or materials depicted in digital images

Qualitative and quantitative Her2 protein evaluation is also performed using immunohistochemistry (IHC) on frozen and archival tissues

Immunohistochemistry is a method for detecting specific antigens in tissues or cells and facilitates the identification of a large number of proteins, enzymes and tissue structures

Thesis contribution (cont’d)

The goal of this thesis is twofold

quantify the contrast effects and parameters of optical polarimetry for material characterization via statistical modeling and metrics and expand the outcome to biomedical applications

process IHC microscope images of breast tissues and automatically determine/define the impact of cancer on the specific organism using image processing techniques

Presentation scheme IntroductionThesis contributionPolarimetric imagingPolarimetry modeling

Polarimetric modeling Polarimetry is a technique that measures the extent to

which a substance interacts with plane polarized light It relies on the properties of polarization of backscattered

light and results in distinct signatures related to surface smoothness, orientation and target composition

Further contrast enhancement of the target can be achieved by modulating the background of the target through doping with polar and high-index-of-refraction molecules

When linearly polarized light is passed through a substance containing optically active molecules (chiral molecules) a rotation of the polarization vector take place. This phenomenon is called optical rotation or optical activity

Glucose molecules and most of the biological molecules such as proteins or enzymes are optically active molecules

Polarimetric modeling

LaserFilter

Polarizer, P1Retarder, R1Beam Expander

Retarder, R2

Polarizer, P2 CCD Camera

Phantom

633 nm laserbeam expander

aqueous optically active molecules

targetobjective lens

P1 R1

CCD

Stokes ParameterAnalysis

• A target phantom is stimulated by a laser beam and the backscattered light is captured through a “detection line”• the optical transceiver geometry consists of a transmitter generator and a receiver analyzer• the imaging system contains two arms; a polarization generating branch and a polarization analyzing branch•the system is utilized to image a specific ROI of the target, achieving optical performance of a microscope

Polarimetric modeling (cont’d)

n=index of refractionn=1.46, similar to hydrated collagen,L-Phenylalanine is an enzymen=1.55, similar to

calcified structures, salt and alcohol are molecular contrast agentsn1=1.54, similar to calcified structures n2=1.49, similar to hydrated collagenn3=1.65, similar to highly calcified mineralized structures

Emulate biological compounds: Micro calcifications depending upon their shape, geometry and composition can be classified as precursors of malignancies in breast mammography. Systematic differences between hydrated collagen in the intensities among the collagen of malignant, benign and normal tissue groups appear to be due to a significantly lower structural order within the malignant tissues

Concept

Polarimetric modeling (cont’d)The regions of different materials in the recorded image can be modeled by means of the intensity or the texture of the corresponding area

Materials that absorb different portions of the incident radiation would be recorded as image regions of quite different intensities

Apply modelling on the smoothed intensity

histogram (mean filtering)

Modalities that favor the dispersion and extensive diffuse of light at grain levels, then texture be preferred as a discriminant factor

Apply modelling on the

Variance of an image

Gaussian distribution as basis

Chi-square distribution as basis


22 2 (x-μ )(x-μ ) (x-μ ) 31 2f(x; A , , )= A exp(- ) + A exp(- ) A exp(- )1 2 3i 2 2 22σ 2σ 2σ1 2 3Target Background Remaining area

i i

2 2/2 1 /2 11 1/2 /22 2 21 2( ; , ) ( ) ( )1 2/2 /21 22 ( / 2) 2 ( / 2)1 2Target Background

n nf e en n

n n

A n

5x5 window convolution on mean and variance of the

original image

Estimation of distribution parameters

Description of image as a sum of basic

functions

Ai ->amplitude factors of distribution iμi ->mean of distribution iσ ι->variance of distribution i

Ai ->amplitude factors of distribution ini ->degrees of freedom of distribution i

No remaining area modeling

due to zero variance


The proximity of the two curves, as well as

the effectiveness of our approximation, is supported by the “goodness of fit

metrics”, the MSE and the SNR quantities, producing a value of order 5x10-3 for the MSE and an SNR db gain value of order

10db

First set of experiments

By decomposing the estimated

mixture models, we can easily

derive the image regions being

modeled by each individual

distributionInformation about the size, shape, texture of the object can be

extracted


The experimental results of this study emphasize the potential of the DOLP-polarimetric modality in biomedical applications, since the experimental setup and calibration of the structures can simulate the function and behavior of human body molecules and substances

Comparison of contrast for different concentrations of Phenylalanine using a 5x5 moving window

72,9268,7562,5058,3356,2549,00

100,0496,8395,6690,4889,3486,63

0,00

20,00

40,00

60,00

80,00

100,00

120,00

0,00% 2,22% 4,44% 6,66% 8,88% 11,10%

Concentration of Phenylalanine in aqueous solution (mg/ML)

Dist

ance

bet

wee

n th

e pe

aks

of

the

two

dist

ribut

ions

Use of mean

Use of variance

Increased contrast is

achieved by increasing the

concentration of optically active

molecules

Proceed to further experimentation on tissue samples

Presentation scheme IntroductionThesis contributionPolarimetric imagingPolarimetry modelingPreliminary results on tissue characterization

Preliminary results on tissue characterization

Cancerous cells exhibit structural, biochemical and metabolic anomaliesPrecancerous lung epithelial tissues exhibit higher reflectance, increased DOP and larger retardance characteristics than healthy tissue, unlike other types of lung cancers of invasive nature. Different lung cancer types and subtypes appear distinct optical differencesLung tissue samples are utilized within the

experimental procedure, directly connecting Stokes parameter imaging to biomedicine

Preliminary results on tissue characterization (cont’d)

Characteristic difference of optical parameters between normal lung tissue and stage I carcinoma Introduce the potential of tissue discrimination through the proposed polarimetric imaging schemeThe key objective and hot prospect of our future study is to determine how lung anatomy and pathology relate to the optical parameters of lung cancer


Microscope imagingImmunohistochemistry fundamentals

Immunohistochemistry fundamentals

GOAL

Detection of Her-2/neu protein in tissues Advanced image analysis techniques are adopted in order to accurately segment the cells within the sample IHC images and precisely extract their membrane contour and degree of staining

Main steps of an automated procedure for the assessment of HER2 status from a

representative Immunohistochemical

input imageThesis contribution

INNOVATION

Automated membrane contour evaluation without prior training of the algorithm on the image content Fusion and parameterization of advanced and fundamental image processing techniques

Immunohistochemistry fundamentals (cont’d) The effect of IHC testing on the tissue sample is to stain the

membranes of tumor cells, partially or completely The percentage of tumor cells that have completely or partially

stained membranes and the intensity of that staining determines the assigned score for HER2 overexpression according to a reference system

Score HER2 Status Staining Pattern

0 Negative: No staining observed, or membrane staining in less than 10% of tumor cells.

+1Negative: A faint/barely perceptible membrane staining detected in morethan 10% of tumor cells. The cells are only stained in part of the membrane.

+2Borderline: A weak to moderate complete membrane staining observed in more than 10% of tumor cells.

+3Positive: A strong complete membrane staining observed in more than10% of the tumor cells.

Immunohistochemistry fundamentals (cont’d)Score 0 Score

+1

Score +2

Score +3

SCORING

EXAMPLES

Immunohistochemistry fundamentals (cont’d)Original image in RGB model

IHC image with DAB staining protocol

Bad tissue preparation – color spread in neighboring areas

IHC image with PAP staining protocol

Bad tissue preparation – wrong cutting of tissue, intersecting nuclei

Tissue preparatio

n procedure affects the quality of the image

Immunohistochemistry fundamentals (cont’d)

Positive reactions with DAB (diaminobenzidine) are identified as a dark brown reaction product on the cell membrane Cell nuclei are identified as blue colored regions

Stained membrane

Cell nucleusThe evaluation procedure is usually performed qualitatively by a pathologist, who carefully observes the IHC samples via microscopy and manually calculates the presence of stained cells in the breast tissue

Intervariability of estimation according to the specialist’s

experience

In order to make immunohistoche

mical studies more objective,

quantitative techniques based

on computer-assisted

microscopy and image analysis

must be developed

IHC PROSPECT


Microscope imagingImmunohistochemistry fundamentalsProposed cell segmentation and characterizati

on

Proposed cell segmentation and characterization

Segmentation scheme combining different image processing techniques

enhance the color difference between the membranes and

nuclei coloring

Improve Image

contrastExtract cell nuclei via mean-shift clustering

Extract cell boundaries

(membranes)via active contours initialized from

clustering results and watershed

transform

Apply mathematical morphology

and extract the final image



onImage clustering

Image clustering

The basic limitation of k-means clustering for a priori knowledge on the number of candidate classes is overcome utilizing the Mean-Shift clustering algorithm

The main idea is to treat the points as an empirical probability density function where dense regions in the feature space correspond to the local maxima or modes of the underlying distribution

Mean-shift clustering


Density Gradient Estimation

Data Density

modes

Image clustering (cont’d)Mean-shift clustering

1. Fix a window h around each data point2. Compute the mean of data within the window3. Shift the window to the mean and repeat till

convergence (no significant alteration in mean value is observed)

Repeat this procedure for each point. The number of different

mean values found represents the number of clusters

STEPS

The only parameter of the algorithm is the bandwidth h The number of classes is internally evaluated The basic drawback of the algorithm is the low speed of convergence The value of the bandwidth parameter is unspecified and application dependent

Remarks


Given a kernel K, n data points {x1, x2,…, xn} and a bandwidth parameter h representing the window size, the Kernel density estimator for a given set of d-dimensional points is given as

2^

1 1

1 1( ) ( )n n

i id d

i i

x x x xf x K k

nh h nh h

Mean shift is assumed to lead to density maximization and can be based on gradient ascent on the density contour. Applying gradient to the kernel density estimator we obtain

2

2 2^ 1

21 1

1

mean shift vector

Assuming ( ) '( )

2 2( ) ( )

ni

in n ii iid d ni i i

i

g x k x

x xx g

hx x x xC Cf x x x g g xnh h nh h x x

gh


21

^ ^2

, 2

2^1,

1

2

G

Setting G(x)=C g( x )2Gradient of f : ( ) ( ) ( )

2KDE of f with kernel G : ( )

Mean Shift Vector : m (x) =

G Gd

ni

G di

ii

Cf x f x m x

nh

x xCf x g

nh h

x xx g

h

1

2

1

^ ^2

G G, 2

2( ) ( ) m (x) m (x) proportional to gradient

n

i

ni

i

gd

xx x

gh

Cf x f x

nh

From x we move by the Mean shift vector towards the density maximum, i.e. cluster center

Mean shift is determined by the gradient

alone

objects in cluster 4

Image clustering (cont’d)

objects in LAB cluster 1 Original image objects in HSV cluster 1



Tissue sample Nucle

i

Intermediate areaMembran

e

Mean-shift clustering clustering

Five classes

produced, bandwidth h set to

0.15

Mean-shift segmentation of breast tissue image sample with Hue color

channel as input

objects in cluster 1objects in cluster 2





onImage clusteringWatershed transform

Watershed transform Segmentation via clustering produces broken boundaries at the points where color variety is met in the neighboring region Thus, a technique extracting closed boundaries is neededWatershed transform is a reference methodology regarding image segmentation The major idea beyond watershed transformation is based on the concept of topographic representation of image intensity Watershed segmentation displays more effectiveness and stability than other segmentation algorithms, producing closed contours and separating intersecting regions

Watershed transform (cont’d) Segmentation via clustering produces broken boundaries at the points where color variety is met in the neighboring region This limitation is overcome via the watershed transformWhen reporting on watershed transform, three fundamental notions arise; minima, catchment basins and watershed lines

Minima

www.mathworks.com

Watershed transform (cont’d)

(1) minima, points belonging to the different minima(2) catchment basins, points at which water would certainly fall

in a single minimum(3) watershed lines, the highest intensity level points at which

water would have equal probability to fall in more than one minimum, i.e. the separating boundaries of segmented basins

Intensity image viewed as a 3D depth/basin structure

The catchment basin CB(mi) of a minimum mi is defined as the set of points which are topographically closer to mi than to any other regional minimum mj:

The watershed of f is the set of points which do not belong to any catchment basin:

i f j f( ) j I\{i}|: f(m )+T ( , ) f(m )+T ( , )i i jCB m x m x m

( ) ( ) ii I

Wshed f D CB m

Watershed transform (cont’d)Approaches for image

segmentation

Start locating the basins and then

find the watershed lines as a set

complement of them

Partition the image into basins

and extract watersheds by

boundary detection

An appropriate labeling of the resulting regions is necessary All points belonging to a given catchment basin have the same unique labelPoints of different catchment basins have different labelsSometimes the watershed transform is not applied directly to the original image, but to its (morphological) gradient producing watersheds at the points of grey value discontinuity

Watershed transform (cont’d)

Key sub-regions or single points within each basin that significantly differ from the remaining basin region

Numerous local minima in the input image, thus regions are divided to smaller and smaller regions

Marked-watershed transform

Watershed transform

Markers

Marker extraction methodsH-minima transform: suppresses all minima in the intensity image whose height is less than a threshold h H-domes transform: finds local maxima using the morphological reconstruction operator

Watershed transform (cont’d)H-domes transform Difference of Blue channel from its

maximum

Input for the watershed transform Watershed lines

Markers for cell nuclei

Input image for the watershed transform

Differece image for the watershed transform

Contours extracted via the wateshed transform


Microscope imagingImmunohistochemistry fundamentalsProposed cell segmentation and characterizatio

nImage clusteringWatershed transformActive contours membrane boundary estimation

Active contours membrane boundary estimationActive contours

modelContour-based techniques are well established in international bibliography, providing accurate and robust results even in noisy environmentHowever they suffer from initialization, local minima and stopping criteria problemsGlobal minimum energy-searching methods have been proved effective in overcoming local minima problems, leading to robust convergence regarding the final contour extractiona curve C is represented via a function φ (the level-set

function) as C={(x, y)|φ(x, y)=0}

Consideration

(x, y) are coordinates in the image plane Curve representation in terms of function φ()

the evolution of the curve is given by the zero level curve at time t of function φ(x, y, t)φ <0 denotes points outside the curve φ >0 represents points belonging to the internal area of the curve

Active contours membrane boundary estimation (cont’d)

At any given time, the level set function simultaneously defines an edge contour (φ=0) and a segment of the image (φ≠0)The curve is evolving according to the partial differential equation below, iteratively converging to a meaningful segmentation of the image.

0, φ(x,y,0)=φ ( , )F x yt

Assuming that the image I consists of two regions of approximately constant distinct intensities I1 and I2 and that the object of interest is represented by the region of value I1 (inside curve C), the “fitting energy functional” is denoted

where F denotes the speed of the curve evolution

2 2

1 2 2 1 1 2 2( , , ) ( ) ( , ) ( , )

( ) ( )insideC outsideC

F c c C F C I x y c dxdy I x y c dxdy

Length C v Area inside C

where C is any variable curve except for the object boundary C0, constants c1 , c2, are the averages of I inside and outside C respectively and μ,ν,λ1,λ2 are non-negative fixed user defined parametersThe contour of the foreground region is the solution of the minimization problem inf C (F(c1, c2, C)) ≈0≈F(c1, c2, C0)

Active contours membrane boundary estimation(cont’d)

Color Pre-processing and Transformation to

Improve ContrastClustering for Region-

Based Segmentatio

n

Edge-Based Segmentation

for contour Refinement

Active contours membrane boundary estimation(cont’d)Active contours membrane boundary estimation

Active contours approaches (snakes) suffer from initialization , especially in the cases of unknown number of objects within the image Clever and targeted initialization by fixed-sized circle regions in the centre of each estimated cell region These circle regions originate from the application of mean-shift clustering and the refinement of the areas using mathematical morphology Too big or too small clustered segments are rejected as they are impossible of representing a cell nuclei area The colour information derived from the A and B colour bands of the equalized immunohistochemical image along with the initialized contour constitutes the input of the active contours segmentation algorithm A and B channels marked with watershed transform in order to secure convergence

Input Image initial contour

200 400 600

100

200

300

400

969 Iterations Global Region-Based Segmentation

Watershed marked equalized image in the

LAB space

Initial contours derived from cell nuclei detected

after clusteringInput for

active contours model

+Input marked image for active contours algorithm

Active contours membrane boundary estimation(cont’d)Active contours

segmentation Having completed the pre-processing steps, apply the active contours segmentation in order to extract the cell areas During the segmentation algorithm, the initial curve evolves until it terminates to closed boundaries forming the membranes of candidate cells Convergence is achieved until no significant change in the size of the evolved curve takes place


200 400 600

100

200

300

400

969 Iterations Global Region-Based Segmentation


200 400 600

100

200

300

400

969 Iterations Global Region-Based SegmentationConverged estimation of φ

superimposed on the original image

Internal area of φ (φ>0)

Active contours membrane boundary estimation(cont’d)Active contours segmentation – application

of morphologyApplication of mathematical morphology based on the characteristics of the extracted regions is performed Irregular shaped and sized regions are rejected, refining the segmentation result and improving its accuracy Regions with no internal blue regions (cells without nuclei) are also rejected

Extracted quantities

Labelled cell regions having nuclei Contours of the segments forming the cell membrane Number of pixels belonging to the cell boundary The intensity value of each pixel of the boundary Area and centroids of each labelled regions


Finally selected components after rejecting non candidate cells due to their shape characteristics

Segmented cell components

Finally extracted components

Finally marked candidate components after rejecting non candidate cells due to their shape characteristics and marking nuclei Initially segmented candidate cells Marked candidate nuclei

Candidate cell segments having nuclei

Finally estimated cell segments


Motivation and final result of the segmentation scheme

Better discriminate cell and nuclei areas based on color difference

Correct image defects

Extract closed boundaries,

adaptive to image content

Filter out irrelevant areas, yet improving

segmentation accuracy

LAB color model

selection

Final output image applying the watershed transform

Finally calculated cell border superimposed on the

original image

Histogram equalization


Intermediate step, used for

active contours

initialization

Active contours

model

Mathematical

morphology

Despite the increased level of region intersection, the

ambiguity of area borders is reduced at the perimeter of

each cell

Watershed transformati

on

Intermediate step, feed snakes with boundary estimation

Presentation scheme Introduction Thesis contribution Polarimetric imagingPolarimetry modelingPreliminary results on tissue characterization

Microscope imaging Immunohistochemistry fundamentalsProposed cell segmentation and characterizationImage clusteringWatershed transformActive contours membrane boundary estimationHer2 Grade evaluation

HER2 Grade Evaluation Membrane staining evaluation and IHC score

assignment


Segmentation result

Extracted quantities

Labelled cell regions having nuclei Contours of the segments forming the cell membrane Number of pixels belonging to the cell boundary The intensity value of each pixel of the boundary Area and centroids of each labelled regions

+ =Number of cells forming the tissue Estimation of staining from intensity thresholding Estimation of staining completeness Number of complete stained cells

Her2 Grade evaluation

Presentation scheme Introduction Thesis contribution Polarimetric imagingPolarimetry modelingPreliminary results on tissue characterization

Microscope imaging Immunohistochemistry fundamentalsProposed cell segmentation and characterizationImage clusteringWatershed transformActive contours membrane boundary estimationHer2 Grade evaluation

Results of tissue evaluation Segmentation performance

Segmentation performance

Sample

1

Sample

3

Sample

5

Sample

7

Sample

9

Sample

11

Sample

13

Sample

15

Sample

17

Sample

19

Sample

21

Sample

23

Sample

25

Sample

27

Sample

29

Sample

31

Sample

33

Sample

35

Sample

37

Sample

390.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Precision of cell segmentation procedure

Sample images (Sample number)

Prec

isio

n (%

)

TP = correctly detected regions as cellsFP = regions that have been wrongly detected as cellsPrecision=TP/(TP+FP)=Correctly detected cells / total regions detected

Possibility of

detecting a cell areaGiven the image content complexity,

this is a very encouraging result!

Average value=54,22%

Segmentation performance (cont’d)

Sample

1

Sample

3

Sample

5

Sample

7

Sample

9

Sample

11

Sample

13

Sample

15

Sample

17

Sample

19

Sample

21

Sample

23

Sample

25

Sample

27

Sample

29

Sample

31

Sample

33

Sample

35

Sample

37

Sample

390.00%

10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%

100.00%

True positive rate (Recall) of cell segmentation pro-cedure


Reca

ll (%

)

TP = correctly detected regions as cellsFN = regions forming cells that have not been detectedRecall =TP/(TP+FP)=Correctly detected cells / true number of cells

Hit rate of cell

detection

The algorithm reveals a quite large fraction of relevant results retrieved ->goal of cell segmentation



Sample

1

Sample

3

Sample

5

Sample

7

Sample

9

Sample

11

Sample

13

Sample

15

Sample

17

Sample

19

Sample

21

Sample

23

Sample

25

Sample

27

Sample

29

Sample

31

Sample

33

Sample

35

Sample

37

Sample

390.00%

10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%

100.00%

F-measure of cell segmentation procedure


F-m

easu

re (

%)

F-measure=2*Precision*Recall/(Precision +Recall) An estimat

e of detectio

n accurac

y

The balanced test statistic confirms the encouraging segmentation results



There is no numerical gold standard in cell segmentation performance estimation The vast majority of the techniques focus on the visual comparison between the extracted and the estimated result A medical expert’s evaluation is considered as a ground truth

Intervariability between specialists’ outcomes, subjective

estimation


However, we compare our methodology to two representative techniques

A specialized one

A classic oneFocus on IHC membrane

evaluation through color deconvolution,

skeletons and particle filtering

Marker-controlled watershed applied on

the gradient of the image marking the foreground and the

backgroundGroun

d truth

The specialist initially marks all candidate cell regions and then selects the true cell areas. Thus, the True Negative notion can now be

adopted.Two images are selected from the dataset: a ”mean” image

corresponding to relatively less complex content and a “difficult” one, where irregular regions mislead our algorithm

Segmentation performance (cont’d)Manually segmented

image

Color deconvolution segmentation

Marker-controlled watershed

Output of our methodology

Output image applying the marked watershed transform

MEAN

IMAGE

Segmentation performance (cont’d)Manually segmented

image

Color deconvolution segmentation

Marked watershed segmentation

Output of our methodology


DIFFICULT

IMAGE

FP regions


Quantitative comparison of our methodology to other representative techniques

Precision Recall Specificity Accuracy FP rate F measure

Mean case

Our methodology74,56% 93,41% 80,54% 85,42% 19,46% 82,93%

Color deconvolution

63,97% 95,60% 67,11% 77,92% 32,89% 76,65%

Marked watershed38,39% 47,25% 53,69% 51,25% 46,31% 42,36%

Difficultcase

Our methodology69,37% 82,80% 53,42% 69,88% 46,58% 75,49%

Color deconvolution

66,39% 84,95% 45,21% 67,47% 54,79% 74,53%

Marked watershed20,31% 14,29% 32,00% 22,29% 69,86% 16,77%

TP = correctly detected regions as cellsFN = regions forming cells that have not been detected TN = regions that the specialist has finally rejected as cellsFP = regions that have been wrongly detected as cells

Our methodology is less affected by misleading color

areas!



Microscope imaging Immunohistochemistry fundamentals Proposed cell segmentation and characterizationImage clusteringWatershed transformActive contours membrane boundary estimationHer2 Grade evaluation


Comparative estimation on Grade

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%0.00%

20.00%40.00%60.00%80.00%

100.00%

54.29%

74.29%82.86%

91.43% 91.43%97.14%100.00%

ROC Curve for IHC classification methodology

1,00%8,00%10,00%12,00%14,00%16,00%20,00%

False Positive Rate (1-Specificity)True

Pos

itiv

e Ra

te (S

enso

tiv-

ity)

Score thresho

ld

Performance metrics of the implemented classification procedure and the reference methodology

Method Precision Recall FP rate Accuracy Specificity F-measure

Our method 96,97% 91,43% 20,00% 90% 80,00% 80,00%

Reference method 87,5% 100% 100% 87,5% 0,00% 93,33%

Promising classifier Overestimates positive cases

Operating point



Microscope imaging Immunohistochemistry fundamentals Proposed cell segmentation and characterization Image clusteringWatershed transformActive contours membrane boundary estimationHer2 Grade evaluation


Conclusion

Conclusion We propose a novel attempt for breast

cancer cell-nuclei segmentation and membrane border extraction

The innovative part of the methodology is on fusing information from region and contour-based techniques on different color spaces

The performance of the contour based approach is increased by the proposed initialization scheme

Preliminary results from the analysis of immunohistochemical images confirm the prospect of the methodology

Conclusion cont’d In algorithmic terms, different

segmentation approaches exploit different information and have different effects

Edge-based approaches (active contours) can resolve boundary uncertainties and derive well defined membrane edges.

Need fusion of these approaches Need good pre-processing in terms of color

and intensity enhancement IHC microscope imaging needs careful and

extensive analysis due to complexity of image

Conclusion Exploitation of several pieces of

information in a modular form enables the progressive derivation and refinement of results

Image structure is obtained from intensity

Cell nucleus and membrane appear in different colors in large distance on the color space.

The cooperation of image processing with pathology specialists can give rise to key findings in cancer diagnosis and treatment

The encouraging results of the proposed scheme give rise for its improvement and incorporation to laboratory assessments

PublicationsConference and journal publications referring to the presented work:Journals

IOP PUBLISHING MEASUREMENT SCIENCE AND TECHNOLOGY Meas. Sci. Technol. 20 (2009)(12pp) Impact factor 1.317 "Stokes parameter imaging of multi-index of refraction biological phantoms utilizing optically active molecular contrast agents“

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 11, 2010 Impact factor 1.106 "Efficient Molecular Imaging Techniques Using Optically Active Molecules"

MEASUREMENT SCIENCE AND TECHNOLOGY Meas. Sci. Technol. 22 (2011) 114018 (12pp) Impact factor 1.494 "Polarimetric phenomenology of photons with lung cancer tissue”

International Society for Analytical Cytology, Cytometry Part A 71A:439–450 (2007) Impact factor 0.939 " Automated Analysis of FISH and Immunohistochemistry Images: A Review "

Image Processing, IET, August 2011, Volume: 5, Issue: 5, page(s): 429 – 439 Impact factor 0.639 " Modeling the characteristics of material distributions in polarimetric images "

PublicationsConference and journal publications referring to the presented work:

Conferences IST 2010, IEEE International Conference, Thessaloniki, 2010 " Backscattered polarimetric detection from biological tissue“

www.springerlink.com : 5th European IFMBE Conference, IFMBE Proceedings 37, pp. 381–384, 2011 "Histogram modeling of polarimetric images for analysis of material properties“

IST 2011, IEEE International Conference, Batu Ferringhi, Penang, Malaysia,2011 "Near Infrared Light Interaction with Lung Cancer Cells"

Thank you!

Analysis of microscope images_FINAL PRESENTATION

Documents

Transcript of Analysis of microscope images_FINAL PRESENTATION