A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm...

32
/32 A Fast Approximation Algorithm for the Gaussian Filter Kentaro Imajo (Yamamoto Laboratory) February 14, Master Thesis Defense 1

Transcript of A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm...

Page 1: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

A Fast Approximation Algorithm for the Gaussian Filter Kentaro Imajo (Yamamoto Laboratory)

February 14, Master Thesis Defense 1

Page 2: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in the area size n (∝ σ2). The naïve method requires O(n) time.

Key Algorithm

February 14, Master Thesis Defense

× Area size is n Target Pixel

2 Introduction

Page 3: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Application: Binary Classifier Extending the key algorithm, •  The proposed method speeds up prediction of SVMs with the Gaussian kernel,

•  The proposed method speeds up training with small loss of accuracy. Training (pre-computation): O(n2) → O(4dnlogdn) Prediction: O(n) → O(4dlogdn) February 14, Master Thesis Defense 3 Introduction

Page 4: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Outline 1. Key Algorithm: A fast approximation algorithm for the Gaussian filter Kentaro Imajo, "Fast Gaussian Filtering Algorithm Using Splines," The 21st International Conference on Pattern Recognition, 2012.

2. Application: Binary classification Kentaro Imajo, Otaki Keisuke, Yamamoto Akihiro, "Binary Classification Using Fast Gaussian Filtering Algorithm,” Algorithms for Large-Scale Information Processing in Knowledge Discovery, 2012.

3. Conclusions

February 14, Master Thesis Defense 4 Introduction

Page 5: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

1. Key Algorithm: Gaussian Filtering 2. Application: Binary Classification 3. Conclusions

February 14, Master Thesis Defense 5 Key Algorithm: Gaussian Filtering

Page 6: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Summary of Key Algorithm It computes one Gaussian-filtered pixel in constant time in the area size n (∝ σ2) within 3% error. The Gaussian filter is convolution with the 2D Gaussian Function:

February 14, Master Thesis Defense 6

1�2��

exp��x2 + y2

2�2

�= where , the parameter σ is variance.

= * convolution

Key Algorithm: Gaussian Filtering

Page 7: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Problem To compute one Gaussian-filtered pixel, •  The Naïve method takes linear time in the area size of the Gaussian function.

•  An FFT method cannot compute pixels apart because FFT depends on all pixels.

•  A down-sampling method has 52% error.

February 14, Master Thesis Defense 7 Key Algorithm: Gaussian Filtering

Page 8: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Solution We approximate the Gaussian function using a spline function: •  A spline function becomes discrete if it is differentiated some times. –  Convolution of an image and a discrete function is computed fast.

A spline function is a piecewise polynomial function that is written as , where (・)+ is max(・, 0). February 14, Master Thesis Defense 8

�̃(x) =�

i

ai (x� bi)n+

Key Algorithm: Gaussian Filtering

Page 9: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

�̃(x) = 3(x + 11)2+ � 11(x + 3)2++11(x� 3)2+ � 3(x� 11)2+

� 2.657� 102 exp�� x2

5.27202

�,

-15 -12.5 -10 -7.5 -5 -2.5 0 2.5 5 7.5 10 12.5 15

80

160

240

Approximation Function

February 14, Master Thesis Defense 9

× ×

× ×

- Approximation - Gaussian func. × Knots

where (・)+ is max(・, 0).

Key Algorithm: Gaussian Filtering

Page 10: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

�̃(x) = 3(x + 11)2+ � 11(x + 3)2++11(x� 3)2+ � 3(x� 11)2+

� 2.657� 102 exp�� x2

5.27202

�,

-15 -12.5 -10 -7.5 -5 -2.5 0 2.5 5 7.5 10 12.5 15

80

160

240

Approximation Function

February 14, Master Thesis Defense 10

× ×

× ×

- Approximation - Gaussian func. × Knots

where (・)+ is max(・, 0).

Key Algorithm: Gaussian Filtering

Page 11: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Speed Up of Convolution The convolution can be reduced by differentiating a spline function until it becomes discrete. Intuition:

February 14, Master Thesis Defense 11

(� � I)(x) =��

d

dx�

��

��I(x)dx

��(x)

Integrated

Differentiated Becomes DISCRETE

�̃

This can be computed fast.

Key Algorithm: Gaussian Filtering

Page 12: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Nature of a Spline Function Differentiating a spline function, we can get a discrete function: The Gaussian-like approximation function can also become a discrete function! February 14, Master Thesis Defense 12

-12 -8 -4 0 4 8 12

-240

-160

-80

80

160

240

-12 -8 -4 0 4 8 12

-25

25

-12 -8 -4 0 4 8 12

-10

-5

5

10

-12 -8 -4 0 4 8 12

-12

-8

-4

4

8

12

Differentiate 3-times

Key Algorithm: Gaussian Filtering

Page 13: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Evaluation Approximation error is about 2%. Higher order improves more: Approximation error of 4th order approximation is about 0.3%.

February 14, Master Thesis Defense 13

�̃(x) = 70(x + 22)4+ � 624(x + 11)4+ + 1331(x + 4)4+�1331(x� 4)4+ + 624(x� 11)4+ � 70(x� 22)4+.

(�

|�̃(x)� �(x)|dx) / (�

|�(x)|dx)

Key Algorithm: Gaussian Filtering

Page 14: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Approximation of 2D

February 14, Master Thesis Defense 14

using a discrete image J , which is written as:

J(x, y) =!

(!x,!y)!Z2+

!xn!ynI(x ! !x, y ! !y).

When a source image I is given, the image J can becomputed in O(n2|I|) time by iterations as well asEquation (6).

As a result, precomputing J at once, for any convo-lution of a source image I and an nth-order 2D splinefunction, a convoluted value (!̃II " I)(x, y) can be com-puted in O(m2) time.

6. Approximated Gaussian Functions

In this section, we present two good sets of param-eters below to approximate the 1D Gaussian function.Absolute values of parameters b of a good set shouldbe small for easy and fast computation, and absolutevalues of the parameters b of the following sets are notlarger than 22, which is small, compared with ordinaryimages.

Second-order approximation:

!̃2(x) = 3(x + 11)2+ ! 11(x + 3)2++11(x ! 3)2+ ! 3(x ! 11)2+, (7)

# 2.657 $ 102 exp"! x2

5.27202

#. (8)

Fourth-order approximation:

!̃4(x) = 70(x + 22)4+ ! 624(x + 11)4++1331(x + 4)4+ ! 1331(x ! 4)4++624(x ! 11)4+ ! 70(x ! 22)4+, (9)

# 7.621 $ 107 exp"! x2

7.767802

#. (10)

6.1. Evaluation of Approximated Functions

Next, we evaluate the approximated functions by cal-culating error of two types EI,!̃ and EII,!̃ , which areerror in 1D and in 2D respectively. Using an approxi-mation function !̃ and its targeted function !, they aredefined as:

EI,!̃ =

$

R

%%%!(x) ! !̃(x)%%% dx

$

R!(x)dx

, (11)

EII,!̃ =

$$

R2

%%%!(x)!(y) ! !̃(x)!̃(y)%%% dxdy

$$

R2|!(x)!(y)| dxdy

. (12)

0 10 20x

0

4 ! 104

8 ! 104

z

· · · Gaussian— Horizontal section- - - Oblique section

Figure 2. Second-order approximation

-15 0 15x

-15

0

15

y

$ $ $ $

$ $ $ $

$ $ $ $

$ $ $ $

· · · Gaussian — Approximation ! Control point

Figure 3. Elevation and control points

Applying the equations, we can calculate values oferror of the two approximations (7) and (9), whose tar-geted functions are (8) and (10) respectively, as follows:

EI,!̃2% 0.02009, EII,!̃2

% 0.03473,EI,!̃4

% 0.00305, EII,!̃4% 0.00583.

Figure 2 compares the 2D Gaussian function and twosections of the second-order approximated function intwo dimension. One of the sections is a horizontal sec-tion z = !̃2(x)!̃2(0), and the other one is an oblique

section z =&!̃2( x"

2)'2

.

Figure 3 shows elevations of every 104 for the Gaus-sian function and the approximation function in two di-mension. As the Gaussian function is rotatable, eleva-tions of the Gaussian function are perfect circles. Onthe other hand, elevations of the approximated Gaus-sian function, which are indicated by red solid lines, arevery similar to perfect circles. Therefore we can use theapproximated Gaussian function as an almost-rotatablefilter function.

�̃(x, y) = (3(x + 11)2+ � 11(x + 3)2+ + 11(x� 3)2+ � 3(x� 11)2+)(3(y + 11)2+ � 11(y + 3)2+ + 11(y � 3)2+ � 3(y � 11)2+)

Contour lines

Key Algorithm: Gaussian Filtering

Page 15: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

The 2D Gaussian function is decomposed: The approximation can be defined as: where

(�̃ � I)(x, y) =m�

i=1

ai

m�

j=1

ajJ(x� bi, y � bj)

2D Convolution

February 14, Master Thesis Defense 15

exp��x2 + y2

2�2

�= exp

�� x2

2�2

�exp

�� y2

2�2

O(m2) linear combination, where m is # of knots for each dimension

and m is equal to 4.

J(x, y) =�

(�x,�y)�Z2+

�xn�ynI(x��x, y ��y)

Key Algorithm: Gaussian Filtering

Page 16: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Experiments

February 14, Master Thesis Defense 16

Method Precomputation Computation for all pixels Computation for one pixelNaı̈ve method No O(NM) O(M)FFT method No O(N log N) N/A

Proposed method O(N) at first O(N) O(1)

Table 1. Comparison of computational orderN and M is the size of a source image and the Gaussian filter respectively.

N/A indicates that the FFT method cannot compute values apart.

100 101 102 103 104

# of application pixels of the filter

10!8

10!7

10!6

10!5

10!4

Com

puta

tiona

ltim

e(s

ec.)

— Proposed method - - - Naı̈ve method

Figure 4. Computational time for one pixelon average

7. Experiments

Comparing with the naı̈ve method and the FastFourier Transform method, we evaluate the proposedmethod with a 1024 ! 1024 gray-scale image and theGaussian filter whose the parameter ! is 47.448 and theapplication area is 99 ! 99 pixels. We implementedthem in C++ and compiled them with GNU C++ Com-piler and execute them on Mac Pro (6-core 2.93GHzXeon). As the result of computation, the naı̈ve methodtook 13.7 seconds, and the FFT method took 0.373 sec-onds, and the proposed method took 0.102 seconds. Inthis case, the proposed method can do it more than 100times as fast as the naı̈ve method does.

Next, we evaluate computational time to figure outa filtered value of one pixel. Table 1 shows computa-tional order of the precomputation and computation ofall pixels and computation of one pixel for each method.Computational time of the naı̈ve method depends on ap-plication area of the filter. Figure 4 shows the proposedmethod is faster than the naı̈ve method if applicationarea of the filter is larger than 8 ! 8.

8. Conclusion

We proposed a fast method to apply the Gaussianfilter to a 2D image. In order to compute one Gaus-sian filtered pixel value, the naı̈ve method spends O(M)time, where M is the size of the Gaussian filter. Whilethe FFT method compute all Gaussian filtered pixel val-ues simultaneously in O(N log N) time, where N is thesize of a source image, it cannot compute them sparsely.The proposed method computes one Gaussian filteredpixel value in constant time on the size of a source im-age and the Gaussian filter.

The proposed method approximates the Gaussian fil-ter function using a spline function whose number of thecontrol points is small. Although precomputation takestime linear in the size of a source image, the precom-putation does not depend on the size of the Gaussianfilter, so we do not need to recalculate the precomputa-tion even if we use a different size of the Gaussian filter.

As a result, the proposed method is more than 100times as fast as the naı̈ve method when application areais 99! 99 pixels. Furthermore, unlike the FFT method,the proposed method can compute a filtered pixel valueat any position with any size of the Gaussian filter inconstant time.

References

[1] David G. Lowe. Object recognition from localscale-invariant features. The Proceedings of theSeventh IEEE International Conference on Com-puter Vision, 2:1150–1157, 1999.

[2] David G. Lowe. Distinctive image features fromscale-invariant keypoints. International Journal ofComputer Vision, 60(2):91–110, 2004.

[3] Herbert Bay, Andreas Ess, Tinne Tuytelaars, andLuc Van Gool. SURF: Speeded up robust fea-tures. European Conference on Computer Vision,110(3):346–359, 2006.

Figure Computational time for one pixel on average

For 70+ pixels our algorithm is faster

Diameter of 9 has 69 pixels

Key Algorithm: Gaussian Filtering

Page 17: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Conclusions of Part 1 (1/2) We approximate the Gaussian function using a 2nd order spline function: •  It has only 2% approximation error, •  It has only 4 knots, •  It has unequal intervals of control points while general approximation functions often have equal intervals. –  It is sufficiently optimized as a spline.

February 14, Master Thesis Defense 17 Key Algorithm: Gaussian Filtering

Page 18: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Conclusions of Part 1(2/2) Using spline approximation and an integrated image of an input image, we can compute every pixel apart in constant time to the area size.

February 14, Master Thesis Defense 18

* =

*=�…�

dx3dy3

Key Algorithm: Gaussian Filtering

Page 19: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32 February 14, Master Thesis Defense 19

1. Key Algorithm: Gaussian Filtering 2. Application: Binary Classification 3. Conclusions

Application: Binary Classification

Page 20: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Summary of Application Extending the key algorithm, •  The proposed method speeds up prediction of SVMs with the Gaussian kernel, –  O(n) → O(4dlogdn)

•  It speeds up training with small loss of accuracy. –  O(n2) → O(4dnlogdn)

February 14, Master Thesis Defense

Judges which is nearer to

20 Application: Binary Classification

Page 21: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Problem An SVM is a good binary classifier, but –  prediction takes O(n) time, –  training takes O(n2)~O(n3) time.

We cannot use SVMs if # of data gets large. Solution •  For prediction, we use the key algorithm. •  For training, we treat data equally with small loss of accuracy.

February 14, Master Thesis Defense 21 Application: Binary Classification

Page 22: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Every pixel has a value

Few pixels have a value

Key Algorithm It assumes that an image is given. Since we cannot integrate an image, we propose to use range trees.

February 14, Master Thesis Defense 22

Image Binary Classification

Application: Binary Classification

Page 23: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Range Tree A range tree [Bentley 1979] is a data structure that can compute the total sum in a box x1-x2 in O(logdn) time with O(nlogdn) space.

February 14, Master Thesis Defense 23 Application: Binary Classification

2k

Balanced tree

We can calculate the total sum with the sum of O(logdn) regions x1

x2

Page 24: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

The Proposed Method The proposed method can decide the class of any point in O(4dlogdn) time, and the pre-computation is O(4dnlogdn) time.

February 14, Master Thesis Defense 24

Range Trees -15 0 15

x

-15

0

15

y

! ! ! !

! ! ! !

! ! ! !

! ! ! !

· · · Gaussian — Approximation ! Control point

Figure 3: Elevation and control points

10

Key Algorithm

Application: Binary Classification

Page 25: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Formulation A range tree can give: Using 9 range trees, J can be computed with:

February 14, Master Thesis Defense 25

Binary Classification Using Fast Gaussian Filtering Algorithm 3

3 FGFA for Sparse Data

FGFA assumes that an input data consists of dense data such as an image. Theprecomputation of FGFA takes linear time on the number of pixels of an inputimage. If we deal with continuous values for coordinates as is, a large numberof pixels are required to approximate. Therefore, this section extends the d-dimensional range query algorithm to calculate integrated values of sparse data,and it applies the extension to FGFA.

3.1 Extension of d-dimensional Range Tree

The d-dimensional range tree algorithm provides a method to calculate sum-mation of a rectangular area of d-dimensional spaces. Using the method, thissection shows a method to calculate J in Equation 6. The d-dimensional rangetree algorithm can compute a value Ki,j(x, y) in O(logd n)-time that is writtenas:

Ki,j(x, y) =!

(!x,!y)!{(0,0),···,(x,y)}

!xi!yjI(!x,!y). (7)

Using the values K, J can be rewritten as:

J(x, y) =

"

#x2

!2x1

$

%

"

#y2

!2y1

$

%T"

#K0,0(x, y) K1,0(x, y) K2,0(x, y)K0,1(x, y) K1,1(x, y) K2,1(x, y)K0,2(x, y) K1,2(x, y) K2,2(x, y)

$

% . (8)

Therefore, J can be computed in O(log2d n) time.

3.2 Extension of FGFA

Since the extension of d-dimensional range tree o!ers a method to calculate J inO(log2d n) time, we reveal that FGFA can be computed in O(4d log2d n) time.

3.3 Proposed Classifier

Using the extension of FGFA, the proposed classifier approximates the followingfunction ytest for binary classification:

ytest = sgn

&n!

i=1

yi exp

&! |xtest ! xi|2

"2

''. (9)

J(x, y) =

�y2

�2y1

�T �

�K0,0(x, y) K1,0(x, y) K2,0(x, y)K0,1(x, y) K1,1(x, y) K2,1(x, y)K0,2(x, y) K1,2(x, y) K2,2(x, y)

�x2

�2x1

� .

Application: Binary Classification

Page 26: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Experiments on Time Time consumption of prediction: The proposed method is faster when # of training data is over 150,000.

February 14, Master Thesis Defense 26

Binary Classification Using Fast Gaussian Filtering Algorithm 5

Table 1. Accuracy of classification of the Iris flower data set(Classification between two species with 5-fold cross validation)

Proposed C-SVMMethod C = 0.1 C = 1 C = 10 C = 106

! = 0.1 93% 58% 68% 69% 69%! = 0.3 93% 83% 94% 93% 93%! = 1 93% 93% 95% 93% 92%! = 3 86% 90% 94% 95% 89%! = 10 84% 85% 90% 93% 92%

Table 2. Computational time to predict a class(Every vector has two real numbers between 0 and 10, ! is 1)

# of training data 1, 000 10, 000 100, 000 1, 000, 000

Proposed Prediction 2.43 ms 2.48 ms 2.55 ms 2.96 msmethod (Precomputation) (0.13 s) (1.48 s) (17.8 s) (203 s)

LIBSVMPrediction 0.02 ms 0.22 ms 2.29 ms 23.6 ms(Training) (0.05 s) (4.47 s) (442 s) (1000+ s)

predicts a class by calculating all the distances between a target and trainingdata. Precomputation of the proposed method takes nearly linear on the numberof training data. Training of LIBSVM takes time that is nearly proportional tothe square of the number of training data.

6 Conclusions

This paper showed that FGFA can be applied to sparse data using the d-dimensional range tree algorithm. This is the same classification as a C-SVMwhose parameter C approaches 0, and it has as good accuracy as C-SVMs. Ad-ditionally, we revealed that this can reduce time of prediction of C-SVMs whenthe number of support vectors is larger than 10,000.

References

1. Kentaro Imajo. Fast Gaussian Filtering Algorithm Using Splines. In The 21stInternational Conference on Pattern Recognition, 2012.

2. Koji Tsuda. Overview of Support Vector Machine. The Journal of the Institute ofElectronics, Information, and Communication Engineers, 83(6):460–466, 2000.

3. Christopher M. Bishop. Pattern Recognition and Machine Learning (InformationScience and Statistics). Springer-Verlag New York, Inc., 2006.

4. Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A Library for Support VectorMachines. ACM Transactions on Intelligent Systems and Technology, 2(3):27, 2011.

5. Jon Louis Bentley. Decomposable Searching Problems. Information ProcessingLetters, 8(5):244–251, 1979.

6. R.A. Fisher. The Use of Multiple Measurements in Taxonomic Problems. Annalsof Human Genetics, 7(2):179–188, 1936.

Table Time of Training and Prediction for 2D Data

Application: Binary Classification

Page 27: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Experiments on Accuracy If we treat input data equally, accuracy of a new classifier and LIBSVM: The loss of accuracy is small. February 14, Master Thesis Defense 27

Binary Classification Using Fast Gaussian Filtering Algorithm 5

Table 1. Accuracy of classification of the Iris flower data set(Classification between two species with 5-fold cross validation)

Proposed C-SVMMethod C = 0.1 C = 1 C = 10 C = 106

! = 0.1 93% 58% 68% 69% 69%! = 0.3 93% 83% 94% 93% 93%! = 1 93% 93% 95% 93% 92%! = 3 86% 90% 94% 95% 89%! = 10 84% 85% 90% 93% 92%

Table 2. Computational time to predict a class(Every vector has two real numbers between 0 and 10, ! is 1)

# of training data 1, 000 10, 000 100, 000 1, 000, 000

Proposed Prediction 2.43 ms 2.48 ms 2.55 ms 2.96 msmethod (Precomputation) (0.13 s) (1.48 s) (17.8 s) (203 s)

LIBSVMPrediction 0.02 ms 0.22 ms 2.29 ms 23.6 ms(Training) (0.05 s) (4.47 s) (442 s) (1000+ s)

predicts a class by calculating all the distances between a target and trainingdata. Precomputation of the proposed method takes nearly linear on the numberof training data. Training of LIBSVM takes time that is nearly proportional tothe square of the number of training data.

6 Conclusions

This paper showed that FGFA can be applied to sparse data using the d-dimensional range tree algorithm. This is the same classification as a C-SVMwhose parameter C approaches 0, and it has as good accuracy as C-SVMs. Ad-ditionally, we revealed that this can reduce time of prediction of C-SVMs whenthe number of support vectors is larger than 10,000.

References

1. Kentaro Imajo. Fast Gaussian Filtering Algorithm Using Splines. In The 21stInternational Conference on Pattern Recognition, 2012.

2. Koji Tsuda. Overview of Support Vector Machine. The Journal of the Institute ofElectronics, Information, and Communication Engineers, 83(6):460–466, 2000.

3. Christopher M. Bishop. Pattern Recognition and Machine Learning (InformationScience and Statistics). Springer-Verlag New York, Inc., 2006.

4. Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A Library for Support VectorMachines. ACM Transactions on Intelligent Systems and Technology, 2(3):27, 2011.

5. Jon Louis Bentley. Decomposable Searching Problems. Information ProcessingLetters, 8(5):244–251, 1979.

6. R.A. Fisher. The Use of Multiple Measurements in Taxonomic Problems. Annalsof Human Genetics, 7(2):179–188, 1936.

Table Accuracy for Iris Flower Data Set

Application: Binary Classification

Page 28: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Conclusions of Part 2 •  It speeds up prediction of SVM: O(n) → O(4dlogdn).

•  With small loss of accuracy, it speeds up training: O(n2) → O(4dlogdn). The proposed method consists of •  The key algorithm: A fast approximation algorithm for the Gaussian filter,

•  Range trees.

February 14, Master Thesis Defense 28 Application: Binary Classification

Page 29: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32 February 14, Master Thesis Defense 29

1. Key Algorithm: Gaussian Filtering 2. Application: Binary Classification 3. Conclusions

Conclusions

Page 30: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Conclusions Key Algorithm: A fast approximation algorithm for the Gaussian filter: •  It uses a spline function for approximation. •  It speeds up Gaussian filtering, which is often used in image processing. Application: Binary classification •  It extends the key algorithm with range trees. •  It speeds up prediction of SVMs. •  It speeds up training with small loss of accuracy.

February 14, Master Thesis Defense 30 Conclusions

Page 31: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Related Achievements International Conference (Reviewed): •  Kentaro Imajo, Otaki Keisuke, Yamamoto Akihiro, "Binary Classification Using Fast Gaussian Filtering Algorithm,” ALSIP 2012.

•  Kentaro Imajo, "Fast Gaussian Filtering Algorithm Using Splines,” ICPR 2012.

February 14, Master Thesis Defense 31 Conclusions

Page 32: A Fast Approximation Algorithm for the Gaussian Filter · I propose a fast approximation algorithm for the Gaussian filter, which computes one Gaussian-filtered pixel in O(1) in

/32

Other Achievements International Conference (Reviewed): •  Takuya Akiba, Kentaro Imajo, Daisuke Okanohara, "Engineering Parallel String Sorting Algorithms,” Algorithms for Large-Scale Information Processing in Knowledge Discovery, 2011.

Journal: •  Kentaro Imajo, "Contest Environment Using Wireless Networks: A Case Study From Japan,” Olympiad in Informatics 5, 26―31, 2011.

•  Mahito Sugiyama, Kentaro Imajo, Keisuke Otaki, Akihiro Yamamoto, "Semi-Supervised Ligand Finding Using Formal Concept Analysis,” IPSJ Transactions on Mathematical Modeling and Its Applications 5(2), 39―48, 2012.

Programming Contests: •  ICFP Programming Contest, the 2nd place, September 2011. •  ACM/ICPC Asia Regional Contest in Fukuoka, Japan, the 5th place, November 2011. •  ACM/ICPC Asia Regional Contest in Hsinchu, Taiwan, the 3rd place, November 2011. •  ACM/ICPC World Finals 2012, an 18th place tie, May 2012.

February 14, Master Thesis Defense 32 Conclusions