AT-SPACE REPRESENTATION OF IMAGES AND GENERALIZED EDGE DETECTOR

Muhittin Gokmen

Istanbul Technical University Faculty of Electrical and Electronics

Department of Computer Science Maslak, ISTANBUL 80626 - TURKEY

gokmenecs .i t u .edu. t r

Abstract An i m a g e and surface representa t ion based o n regu-

larization theory i s introduced in t h i s paper. T h i s rep- resenta t ion is based o n a hybrid m o d e l derived f r o m t h e physical m e m b r a n e and plate models . T h e represen- ta t ion , called t h e Xr-representa t ion , has t w o d i m e n - sions; one d i m e n s i o n represents smoothness o r scale while t h e other represents the continuity of t h e i m - age or surface. It conta ins images /sur faces sampled both in scale space and t h e weighted Sobolev space of continuous f u n c t i o n s . T h u s , t h i s n e w representa t ion can be viewed as a n ex tens ion of the wel l -known scale space representation. W e have exper imenta l ly s h o w n that t h e proposed hybrid m o d e l results in improved re- sults compared t o t h e t w o ex treme cons t i tuent models, i.e., the m e m b r a n e and t h e plate models . Based o n th i s hybrid model , a generalized edge detector p) which encompasses m o s t of the wel l -known e ge de- tectors u n d e r a c o m m o n f r a m e w o r k i s developed. The ezisting edge detectors c a n be obtained f r o m t h e gen- eralized edge detector by s i m p l y specifying the values of t w o parameters , one of which controls the shape of the fi l ter (r) and the other controls the scale of t h e f i l ter (A). B y sweeping t h e values of these t w o pa- rameters continuously, one c a n generate a n edge rep- resenta t ion in t h e Xr space, w h i c h i s v e r y useful f o r developing a goal-directed edge de tec t ion scheme f o r a specific task . T h e proposed representa t ion and t h e edge detector have been evaluated qualitatively and quanti- tatively o n several dif ferent types of image data s u c h as in tens i ty , range and stereo i m a g e s .

1 Introduction Identifying and locating object boundaries in a

scene from its two-dimensional projection, the image data, is a crucial step in designing a computer vision system. Since the performance of the higher level pro- cesses such as object recognition using these bound- aries relies heavily on the accuracy of the detected edges, the edge detection problem is an important area of research in computer vision. Despite considerable work and progress made on this subject, edge detec- tion is still a challenging research problem due to the

1063-6919/96 $5.00 0 1996 IEEE 764

Ani1 E(. Jain

Michigan State University Department of Computer Science East Lansing, MI 48824 - U.S.A.

j ainQcps.msu.edu

lack of a robust and efficient general purpose algo- rithm.

Most of the efforts in edge detection have been de- voted to the development of an optimum edge detector which can resolve the tradeoff between the detection and localization performances [l, 2, 4, 5 . Further- more, extracting edges a t different scales i 1, 2, 61 and combining these edges [2, 71 have attracted a substan- tial amount of interest. In many vision applications, there is a great demand for a general-purpose edge detector which can produce edge maps with very dif- ferent characteristics in nature, so that one of these edge maps may meet the requirements of the prob- lem under consideration. Our main motivation in this study is to develop an edge detector which has this sort of generalization and abstraction power.

In the course of developing the generalized edge de- tector, we explore the relationship between two seem- ingly different topics: (i) regularization theory, and (ii) image convolution with filters. First, we consider the membrane and plate models used widely under the regularization framework. Then the relationship be- tween these models and filters associated with them is considered; the identified filters are also compared with other existing edge detectors. After partitioning these filters into two groups, we are interested in ob- taining a smooth transition between them by means of a hybrid model. This hybrid model provides a new im- age and edge representation, called the AT-space rep- resentation. The representation can decompose an im- age into a description which is richer than scale space representation. We present the representation of im- age surfaces obtained from different cues such as range scanners and binocular stereo. Finally, a generalized edge detector is developed by finding the filter corre- sponding to this hybrid model. It is shown that most of the high performance edge operators such as Canny edge detector [2], the first- and second-order R-filters 17, 31, Deriche’s detector [4] and Shen and Castan’s edge detector 151 are special instances of the gener- alized edge detector and can be obtained by appro- priately selecting two parameter values. The develop- ment of the generalized edge detector also reveals that

the Gaussian filter is more similar to a filter associated with the hybrid model rather than to the second-order R-filter .

2 Regularization Regularization provides a general framework to

convert ill-posed piroblems in early vision into well- posed problems by restricting the class of admissible solutions using constraints such as smoothness [3, 71 In regularization theory, the smoothness can be im- posed on the solution by minimizing an energy func- tional containing derivatives of the solution. Given noisy data d(a,.y), the regularized solution f ( a , y) can be found by minimizing either

which is the energy functional associated withi mem- brane or

which is the energy functional associated with thin plate. In these functionals, the first term on the right hand side is a measure of the closeness of the solu- tion f ( a , y) to the data d(a , y), and the second term, stabilizer, is a measure of the smoothness. The com- promise between these two terms is controlled by the regularization parameter A . 2.1 The R-filters and Relationship with

Other Edge Operators We now consider the relationship between the regu-

larization and filtering processes. To show this relation we first consider a 1-dimensional case for simplicity, where membrane and plate functionals become string and rod functionals, respectively.

The function which minimizes the string functional can be written as 171 f(a) = Rl(a; A) * d, where the filter

1 ~ 1 ( a ; A) = --e-lXl/’,

2A is called the first-order R-filter. On the other hand, the function which minimizes the energy functional associ- ated with a rod can be found as [3] f(a) = &(z; A)*d, where the filter

(3)

is called the second-order R-filter. A common approach in designing edge operators

is to find a filter, say g(z), which maximizes three edge detection criteria; good detection, good local- ization and only one response to a single edge [2]. Canny [2] mathematically formulated these three cri- teria and showed that the optimum finite length filter in the range [-W, W ] can be approximated by the first derivative of a Gaussian;

g(z) = - - m e - x= 1 2 2

Following a different approach, Torre and Poggio [3] showed that minimizing the energy functional associ- ated with a rod is equivalent to convolving the data d with the cubic spline convolution filter, the second order R-fil ter, which is very similar to a Gaussian. A similar filter was derived by Deriche [4] by extending Canny’s filter to the range [-m, CO]. Thifj modifica- tion leads to the following approximate solution

g(a) = cm-+l , (6) which can be recursively implemented. Shen and Cas- tan [5] deduced an optimum smoothing operator by imposing a high SNR for the first derivative of the filter’s output. The resulting optimum filter is given as

which is similar to the regularization filter Rl(z ; A) obtained from minimizing the string functional.

From the above discussion, we can partition the edge detection algorithms into two broad groups. The first group contains operators associated with the weak membrane functional such as the first-order R- filter and Shen-Castan edge operator. The second group contains operators associated with thin plate functional. The Marr-Hildreth edge operator, Canny’s operator, Deriche’s operator and the second-order R- filter are placed in this group. The operators in the first group possess rather good localization perfor- mance a t the expense of somewhat decreaseid detection performance. The operators in the second group have somewhat balanced detection and localization perfor- mances.

3 The Hybrid Model Our goal is to achieve a smooth transition between

these two groups of filters. For this purpo, <e, we com- bine the membrane and plate stabilizers, and set the problem as follows: Find a function f which minimizes

h(z ) = ce-PIxI with p > 0, (7)

%(f; 5 , Y) = J J,W- dI2 + (8)

-I- f:”,)I:WzdY, (9) - M,” + f,”> + T u : , + where A is the real-valued regularization parameter and T E [0,1] is the real-valued continuity control parameter. Thus, the stabilizer of hybrid functional is a convex combination of membrane and thin plate functionals. The hybrid functional EX^(.^; a, y) can be thought of as a thin plate surface under tension, where A corresponds to cohesion or rigidity and ( 1 - T ) corresponds to surface tens ion . Thus, thin plate ker- nel affects surface curvature, while the membrane ker- nel affects surface area. A similar function was used by Terzopoulos [8] as the continuity control function, where the rigidity and tension of the surface are con- trolled spatially.

Here, this hybrid functional is used to obtain a smooth transition from membrane model t80 the plate model. Note that, for T = 0, the functional1 reduces to the membrane model, for T = 1, it reduces to the plate model and for intermediate values of this parameter

765

we obtain hybrid surfaces. In this hybrid model, as r increases from 0 to 1, so does the order of continuity of the reconstructed surfaces. Increasing the value of r gradually transforms a membrane to a plate. Now, if we consider the relationship with filtering, minimiz- ing this functional is equivalent to convolving the data with the RI - f i l t e r and Rz - f i l t e r for r = 0 and r = 1, respectively. The shape of the correspond- ing filter is gradually transformed for the intermediate values of T . We will consider deriving the analytical expression of this filter. Thus this model enables us to obtain the results of edge detectors in both group 1 and group 2 as well as all the smooth transitions between them. One of the important aspects of this hybrid model is that the problem of selecting the ap- propriate filter is now reduced to the problem of se- lecting the appropriate value of r for a specific task.

Does the hybrid model provide any improved re- sults as compared to the membrane and plate func- tionals? To answer this question we consider a checkerboard image and then analyze the recon- structed images obtained with r = 0 ,0 .5 and 1, which are shown in Fig. 2. It is seen that the membrane solution considerably smoothes out the discontinuities since it tends to minimize the area of the reconstructed surface, whereas the plate solution exhibits overshoots around discontinuities. Since the smoothness of C1 surface conflicts with the sudden changes a t these points, the solution undergoes overshoots around these points. On the other hand, the hybrid solution ob- tained with r = 0.5 does not suffer from oversmoothed discontinuities of the membrane solution or the over- shoots of the plate solution 191.

4 The XT-Space Representation The Xr-space representation of an image contains

regularized solutions minimizing the hybrid energy functional E X ~ ( U ; 2, y) while sweeping the regulariza- tion parameter X > 0 and the continuity-control pa- rameter -r E [0, l]. This representation has two di- mensions specified by X and r and it can be thought of as an extension of the scale space representation which has only the scale parameter A. In the Xr- representation, the X-axis contains images smoothed a t different levels. The regularization parameter X controls the smoothness of these images so that as X increases we obtain coarser and coarser description of the Image. Thus for a fixed r value, we obtain the scale space representation of the image. On the other hand, the r-axis contains solutions which have differ- ent levels of continuity. In fact the images along this axis are the samples of images represented in Sobolev spaces. The real-valued continuity control parameter T E [0, I] determines the degree of continuity of the solution; for r = 0, the membrane solution has a C O continuity, and for r = 1 the plate solution has a C1 continuity. For the intermediate values of r , the hy- brid solutions are obtained. As r increases, the degree of continuity of the solution gradually increases while gradually transforming a membrane surface to a plate surface.

Fig. 4 shows the Xr-representation of a surface re- constructed from a sparse and noisy range da ta in

Fig. 3. As seen from this figure, increasing X intro- duces more smoothness in the image for any r value. One can observe that, for a small value of A, the dif- ference between the solutions with different r values is almost indistinguishable. This is expected because for a small value of A, the first term in the hybrid func- tional becomes dominant, so the smoothing term does not play an important role in the solution. As X in- creases, the smoothing term becomes more and more dominant, and the value of r has a strong influence on the solution. Note that for r = 0, we have smoothed discontinuities determined by the value of A, since the membrane solution tends to minimize the area of the surface. For r = 1, we have somewhat sharper images together with overshoots around discontinuities. At the points of these discontinuities, the plate model can not cope with the sharp intensity changes and creates overshoots. These images look sharper as compared to those obtained with a smaller r value due to the higher contrast caused by overshoots around discon- tinuities. The images obtained with r = 0.5 seem to be a good compromise between the oversmoothed dis- continuities of the membrane model and overshoots of the plate model. However, the natural scale of fea- tures and the amount of noise in an image may lead to different “optimal” values for X and r describing the most suitable image in this representation. It is noteworthy that the Xr-representation can decompose an image into a rich description from which features useful for a specific problem under consideration can be easily extracted.

5 The Generalized Edge Detector ” In order to find a smooth transition between two

filter groups, we now consider the hybrid energy func- tional in 1-D and find the corresponding filter asso- ciated with it. The function that minimizes the one- dimensional hybrid functional

can be found by solving the associated Euler-Lagraggk equation

X ~ f z z z z - X ( 1 - 7 ) f z z + f = d (11) with the appropriate boundary conditions. The solu- tion to this partial differential equation can be given in the form of

f(z) = / T(Z, z’)d(z’)dz,

where r ( z , z’) is the Green function satisfying

a4r(z, a‘) d Z r ( z , d) X r - X ( l - r ) +r(z , z’) = 6(z-z’).

ax4 8x2 For a bi-infinite domain, i .e. , when boundaries arL1ii plus and minus infinity, r ( z , z’) is a function of (z - 2’)

only, that is, ~ ( z , z’) = r(z-z’) . In this case, Eq. (11) becomes a convolution integral, that is,

f(z) = ] R ( z - z ’ )d (d )dz = R ( z ) * d(z) . (14)

766

The filter R(z ) is a function of x as well as A and r and it satisfies

d4R(z; A, r ) a2R(x; A, T ) - B + R(z ; A, .) = 6(z), f l C \

A ax4 8x2 where A and B are defined by A = Xr and #2 A ( 1 - r ) . We assume that the filter R a; A, r ) , as well

vanish a t plus and minus infinity. In Fourier domain, we have

as its derivatives up to and including t h e third degree,

(16) 1

( A w 4 + B w 2 + 1 ) R ( w ; A, r ) = -

where R ( w ; A, r ) is the Fourier transform of R(z ; A, T )

so that

6'

R(z; A, r ) = F-1 - { A w 4 + B w 2 + 1

In the course of calculating this inverse transform, we have identified five different cases determined by the value of A = ( B 2 - 4A). For the sake of brevity, we skip the intermediate steps of this derivation and give only the final expressions of the smoothing filter

and the Generalized Edge Detector (GED which is the first derivative of R(z;A,r

with respect to z: Case I: A > 0

where

Case 11: A < 0

1 eblml cosalzl sinalzl 4Jl;iT a

R(x;A,r) = --(T+- -1

4ALab ~ ( z ; A, r ) = ----e-blml sin a z ,

where a = & cos ( 4 arctan 4% - 1) and

b = Case 111: A = 0

sin (f arctan 4 F ) . 1 1

R(z;A,r ) = -e-alel 2 B (a + 1x1)

-a 2 B - 3563- alzl G ( ~ ; A , T ) =

where a = a.

Case IV: B = 0 , (the plate model)

where a = **

Case V: A = 0, (the membrane model)

As expected, the first- and second-order R-filters are obtained in the cases V and IV, respectively. It is interesting that the filter G(z; A, r ) in case I11 is the same as Deriche's edge detector, which is very similar to the Gaussian filter. In a previous study [3], the filter associated with the plate model has been shown to be very similar to the Gaussian filter. HoTwever, our approach based on the hybrid model reveals that the filter in case I11 is more similar to the Gaussian filter.

Fig. 5 shows the R-filter R(z ; A, r ) and the Gener- alized Edge Detector, i.e., the filter G(z; A, r ) , for the specified values of A and r. The parameter values and the corresponding cases are shown in Table! 1. Fig. 1 shows a sketch of the AT - space, where ihe curves specified by AT/( 1 - T ) ~ = IC are shown for various val- ues of IC. The thick curve with A T / ( 1 - r = 4 derived

membrane model) and r = 1 (the plate model lines,

this space reveals that the shape of the filter does not change as we move along any AT/( 1 - T ) ~ =: constant curve. On the other hand, moving across these curves will considerably change the shape of the correspond- ing filter. One of the important differences between filters in region I and in region I1 is that the R-filter does not have negative values for any X and r in re- gion I, whereas it will have two negative lobes at both sides of the central positive lobe for X and r in re- gion 11. These negative values are the primary reasons for the overshoots around discontinuities ,at the fil- tered images. Thus one can observe that in region I, the first order R-filter is gradually transformed to a Gaussian-like filter as r increases while in region 11, the Gaussian-like filter is transformed to the second order R-filter as r increases.

We have implemented an edge detection algorithm using the generalized edge detector. The algorithm is the same as Canny's edge detection algorithm [2] except that the Gaussian and its first derivative are replaced by the R(z ; A, T ) and G(z; A, r ) filters, respec- tively. The algorithm has the capability of controlling the scale of the filter as well as its shape.

In order to expose the properties of the generalized edge detection algorithm and the advantages of using such a filter for different purposes, we have considered

from A = 0 divides the space, limited i y T = 0 (the

into two regions, say Regions I and 11. The ana 1 ysis of

767

two different set of images. The first set of images shown in Fig. 2 consists of noisy checkerboard image. This synthetic images are useful to determine how well the detector can handle the tradeoff between the de- tection and localization performances and to evaluate the results quantitatively as well. The Xr-space edge representation for checkerboard is shown in Fig. 6. Note that the first-order G-filter gives the best results for this image. The filter corresponding to the hybrid model with T = 0.5 and 1 results in shifted edges. The quantitative evaluation [9] leads to the same conclu- sion. On the other hand, when we obtained the edge representation of the noisy house image [9], the first- order R-filter results in noisy edge fragments and the edges obtained with r = 0.5 are better. These two examples show that different problems require differ- ent edge detectors and the generalized edge detector is capable of producing these different results as samples of the scale-continuity ( A T ) space representation.

0 0.5 1 1 1 v I1 Iv-

X 8 V 111 I V - 16 V 1 I V

6 Summary and Conclusions We have presented a new image representation by

using the hybrid model derived from the membrane and plate models. This representation samples an im- age in the scale space and the continuity space. Com- pared to the scale space representation, this represen- tation can decompose the image/surface into a larger number of descriptions, each of which exhibits differ- ent characteristics of the image or surface.

We have experimentally shown that the hybrid model provides additional information compared to the two extreme constituent models, i .e . , the mem- brane and plate models. We believe that the pro- posed representation will be applicable to a wide vari- ety of problems. For example, one can use the hybrid model to determine the best parameter values for re- constructing a fingerprint image from only ridge and valley contours. This sort of modeling can be used for model-based compression of fingerprint images. Fur- thermore, the hybrid model can be applied to other early vision problems where it is customary to use only a membrane or a plate model.

We have derived a general-purpose edge detector from the hybrid model, which encompasses most of the well-known edge detectors under a unified frame- work. The edge representation generated by this op- erator contains very different edge descriptions, so the desired edge description required for a given problem can be found in this representation. We believe that this generalization and flexibility is very suitable for developing successful vision systems and that the gen- eralized edge detector will find many applications.

Acknowledgment: We thank Kalle Karu and Ay- segiil Gensata for their assistance. This research was partially supported by the NSF grant INT-9500652 and TUBITAK grant EEEAG-154.

References [l] D. Marr and E. Hildreth, “Theory of edge detec-

tion,” in Proc. of Roy. Soc., (Sec B, 207), pp. 187- 217, 1980.

Figure 1: A sketch of the AT-space.

Table 1: Filter types associated with the selected pa- rameter values.

I I / 7 It

[2] J . F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, pp. 679-698, 1986.

[3] V. Torre and T. Poggio, “On edge detection,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, no. 4, pp. 147-163, 1986.

[4] R. Deriche, “Optimal edge detection using re- cursive filtering,” in Proc. IEEE First Int. Conf. Computer Vision, (London), pp. 501-505, 1987.

[5] J . Shen and S. Castan, “An optimal linear oper- ator for edge detection,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, (Mi- ami), pp. 109-114, 1986.

[SI A. P. Witkin, “Scale-space filtering,” in Proc. 8th Int. Joint Conf, on AI., (Karlsruhe, West Ger- many), pp. 1019-1022, 1983.

[7] M. GGkmen and C. Li, “Multiscale edge detec- tion using first order R-fikv,” in Proc. Int. Conf. on Pattern Recognition, The Hague, The Nether- lands, pp. 307-310, 1992.

[8] D. Terzopoulos, “The computation of visual sur- face representations,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-10, no. 4, pp. 417- 438, 1988.

191 M. Gijkmen and A.K.Jain, “AT-Space Represen- tation of Images and Generalized Edge Detec- tor,” Submitted to IEEE Trans. Pattern Anal. Machine Intell., 1995.

768

( d ) T = 1 X = 1 @ -.- (c) T = 0.5 Figure 2: Reconstruction of a noisy checkerboard im- age of S N R = 10d.B.

Figure 5: R- and G-filters with specified values.

Figure 3: A sparse range image of a coffee cup.

Figure 6: An edge representation in the X:r-space for the checkerboard image.

Figure 4: The X-r-representation of a range image.

769