2016

:

.: 40311

1

2

................ .................., ............

............., ............40311.............

... ..

, :

(..)

,

.

.

. ..,

, , .

, .

, ,

.. .

..

6 .

18, . 5 .

3

................................................................................................................... 5

..................................................................................................... 6

1. 1: ...................................................................................................... 8

1.1. .................................................................................. 8

1.2. UAV ....................................................................... 9

1.3. ................................................................. 13

2. 2: .............................................................................. 14

2.1. ................................................................................................... 14

2.2. Software ............................................................................................... 15

2.2.1. ROS Robot Operating System ........................................................................ 15

2.2.2. - Open-Source ROS Packages ......................... 18

2.2.3. Gazebo Simulator ................................................ 21

2.3. Hardware ................................................................................................................... 24

2.3.1. Parrot AR Drone 2.0 .......................................................................................... 24

2.3.2. & Sensors & Motors ....................................................... 25

2.3.3. IMU Inertial Measurement Unit ..................................................................... 27

2.3.4. Monocular Camera ............................................. 29

2.4. ............................................................................ 31

2.4.1. PID ..................................................................................................... 31

2.4.2. Bayes Bayes Filter .............................................................................. 36

2.4.3. Kalman Kalman Filter ........................................................................ 40

2.5. ................................................. 46

2.5.1. Simultaneous Localization and Mapping SLAM ........................................... 46

2.5.2. SLAM Monocular SLAM .......... 53

2.5.3. PTAM Parallel Tracking & Mapping ............................................................. 55

3. 3: ................................................................................... 58

4

3.1. ................................................................................................... 58

3.2. & ............................. 59

3.3. UAV Basics

of Wind Turbine Blade Inspection with UAV ..................................................................... 61

3.3.1. Basics and Introduction .......................... 61

3.3.2. & Challenges & Solutions .............................................. 64

3.4. .................................................................. 68

3.4.1. ............................................................................................. 68

3.4.2. ......................................................... 71

3.5. ....................................................................................... 76

4. & ..................................................................................... 79

4.1. .......................................................................................................... 79

4.2. ................................................................................................ 79

....................................................................................................... 80

5

1: (UAV) [63] ............................................ 9

2: [59] ............. 10

3: [60] 11

4: multicopter [61] .. 12

5: (nodes) ROS [62] ................... 15

6: Real-Time Data ................................................................................... 16

7: ........ 17

8: topics

. ............................................................................................................ 20

9: Gazebo. ............... 22

10: drone ........ 23

11: Parrot AR Drone 2.0. [7] ....................................................................................... 24

12: Brushless Motor [58] ........................................................................... 25

13: BLCB AR Drone 2.0 [9] ................................... 26

14: front bottom camera AR.Drone 2.0. .................................................. 29

15: PID ...................................................................................... 31

16: Roll, Pitch, Yaw........................................................... 33

17: PIDs tum_ardrone......... 34

18: & ......................................................... 41

19: HoG [40] ........................ 52

20: Pose Graph [46] ............................................... 54

21: PTAM AR Drone .......................... 57

22: UAV [51] ............................... 58

23: Rope Access Technician [53] ................................................................................ 61

24: [57] .......... 63

25: "" "+" [55] ............................................... 67

26: Gazebo ...... 77

27: Gazebo .............. 78

https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181298https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181299https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181300https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181301https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181302https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181303https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181304https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181305https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181305https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181309https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181313https://d.docs.live.net/26f8174a26873213/Family%20Projects/Panos's%20Thesis/%20.docx#_Toc456181321

6

ROS Robot Operating System

UAV Unmanned Aerial Vehicle

HAL Hardware Abstraction Layer

MAV Micro Air Vehicle

PoI Point of Interest

IRLS Iteratively Reweighted Least Squares

PDF Probability Density Function

SLAM Simultaneous Localization & Mapping

EKF Extended Kalman Filter

SSD Sum of Squared Differences

LSD-SLAM Large-Scale Direct SLAM

7

8

1. 1:

1.1.

,

,

(UAV).

,

,

. ,

. , ,

UAV.

, .

, (software),

(hardware). ,

, PIDs, Bayes

Kalman,

, SLAM, monocular SLAM PTAM.

3,

UAV. ,

,

. ,

drones

drone. , ,

,

. ,

,

,

.

9

1.2. UAV

,

,

.

, 21 .

(2016),

drones ,

.

, ,

,

,

.

1: (UAV) [63]

10

,

Unmanned Aerial Vehicles (UAV). ,

drones

. drones,

,

UAV.

drones

[1] [2].

Fixed-Wing

UAV .

,

drone

. , , drone

,

.

, drones

, . ,

UAV

hover drone

.

2: [59]

11

Rotary-Wing

UAV ,

,

.

,

. ,

,

.

hover

.

3:

[60]

12

- Multi-Rotor

micro UAV.

.

drones .

3 ( tricopter)

4 ( quadcopter)

6 ( hexacopter)

8 ( octocopter)

,

drones 12 16 8 V.

,

drones

.

multi-rotor drones UAV

, . ,

drones

, hover. ,

, .

,

, , ,

. ,

drone, 20

4: multicopter [61]

13

40 UAV. ,

drone

.

,

, multicopter

(quadcopter) .

1.3.

,

.

.

,

drone . ,

,

.

, ,

drone

, .

drone .

,

drone,

. ,

,

, . , drone

,

.

drone,

,

.

14

2. 2:

2.1.

,

. meta-operating

ROS [3] Linux Ubuntu. ,

Python,

C++. ,

(software) ,

drone. ,

(hardware)

.

15

2.2. Software

2.2.1. ROS Robot Operating System

,

ROS [3]. ROS Robot Operating System (

). meta-

operating

, .

,

(HAL), hardware

. ,

/ (Publish/Subscribe),

(nodes)

topics .

, ROS Master,

(nodes)

(.. 127.0.0.32) .

(open-source). ,

,

. ,

5: (nodes) ROS [62]

16

.

,

. , ,

, ROS.

, ' , ROS

UAV

.

ROS,

. ,

.

, , ROS

Gazebo. ,

(real-time data)

, drone,

. ,

drone .

, , ROS

(nodes)

(Dynamic

Reconfigure) ,

.

, ROS

,

driver

drone ,

(script).

6: Real-Time Data

17

nodes, 7

drone .

7 topics ,

.

, ROS

, .

7:

18

2.2.2. - Open-Source ROS Packages

, ROS [3]

.

ROS, (packages).

(scripts), (libraries), ROS nodes,

(configuration files) ..

(plugin).

(repositories) GitHub. repositories,

.

,

, .

, .

,

:

include/_: header files C++.

msg/:

publish ROS nodes.

src/_: (Source files)

Python.

srv/: (services),

, nodes.

scripts/: (scripts).

CMakeLists.txt: .

package.xml: .

19

.

Publish/Subscribe:

nodes. nodes

topics.

topics ,

ROS, publish.

, subscribe,

topic topic.

, publish/subscribe

, .

,

real-time drone.

Topics: topics

publish/subscribe. ,,

. ,

services topics

nodes request. ,

topic

.

subscribe topic,

publish,

. ,

, , topics

8.

, /ardrone/,

drone (

), drone

.

20

8: topics

.

21

Messages: nodes ROS

topics.

, Boolean,

, (arrays).

Services: topics ,

request/reply.

ROS,

Dynamic

Reconfigure .

,

- tum_ardrone [4],

micro UAV.

ardrone_autonomy [5]

driver drone. 3

.

2.2.3. Gazebo Simulator

.

. , ,

. ,

.

, Gazebo [6]. (3D

Dynamic Simulator) Linux ROS.

(Open-Source) , gazebo

.

,

22

,

project . gazebo

,

. , gazebo simulator

, . ,

, open-source ,

, . ,

,

open-source ,

project .

,

gazebo simulator, ,

(,

drone). ,

, drone,

drone. 9 ,

,

.

9: Gazebo.

23

9,

.

. ,

,

(repository).

10

,

.

10: drone

, Gazebo Simulator

, . open-source

,

, (

),

.

24

2.3. Hardware

,

, ,

Parrot AR Drone 2.0 [7].

drone, ,

[8].

2.3.1. Parrot AR Drone 2.0

AR Drone [7]

. Augmented

Reality Drone,

,

, .

11: Parrot AR Drone 2.0. [7]

micro UAV ( MAV)

, .

, ardrone (AR Drone)

.

,

. ,

drone , , open-loop

(feedback)

.

. , ardrone

25

Parrot [7] drone open-

source, ,

,

ardrone.

ardrone,

( ):

4 (brushless motors)

: Parrot P6 ARM9-core (Parrot P6 processor)

: 16bits PIC micro-controller

2 (Monocular Cameras)

(IMU Inertial Measurement Unit)

2 (Ultrasonic Sensors)

2.3.2. & Sensors & Motors

:

, drone .

, brushless motors

brushed motors

. ,

,

. , brushless

motors

, Watt, ,

.

. ,

(electromagnets)

(rotor),

(stator)

.

(brush) 12: Brushless Motor [58]

26

,

.

ardrone 4 brushless motors

(BLCB Brushless Control Board),

.

,

(microcontroller), ATMEGA8L 8bit,

(cutout system)

. 13

ardrone.

13: BLCB AR Drone 2.0 [9]

: , - - 2

, ardrone

, .

6 , ardrone ,

.

ultrasonic .

ultrasonic ,

.

,

. ,

. ardrone Prowave

Ultrasonic Sensors PIC -

.

27

,

UAV

drone. , 40kHz

6 .

, IMU.

2.3.3. IMU Inertial Measurement Unit

IMU,

,

. (Gyroscope)

(Accelerometer) real-time

drone .

, AR Drone IMU

Bosch BMA150

A/D 10 ,

x y, z. IMU ,

200Hz drone .

, (x,y) Invensense

IDG500

PIC 12 A/D .

500 ,

drone . ,

Epson XV3700 ,

.

, IMU drone

,

, . (Specific Force)

drone.

=

/2

drone, g-force.

28

,

.

IMU, dead reckoning.

drone ,

IMU, .

, , ,

,

drone. ,

GPS

(GPS-denied areas). GPS ,

.

, ,

.

dead reckoning ,

. ,

,

, dead reckoning

.

IMU. , IMU

drone. ,

,

UAV .

,

Abbe (Abbe error) [10]. , Abbe error

:

= () (2.1)

:

=

h =

=

29

, IMU

, . ,

, ,

.

2.3.4. Monocular Camera

,

, .

,

(Monocular Camera) ,

. ,

,

.

AR Drone, ,

.

(front camera) ardrone

. ,

93 VGA 640x480 15

frame/second. , (vertical bottom camera)

drone ,

.

64 ,

60 frames/second.

14: front bottom camera AR.Drone 2.0.

30

(Vision Algorithms),

drone ,

,

.

(optical flow)

(multi-resolution)

(B. D. Lucas T. Kanade) [11, pp.

121-130]

(B. G. Schunck B. K. Horn) [12, pp. 81-87].

,

. , ,

.

, ,

,

.

,

(corner tracking), (points of

interest PoI) drone.

(Iteratively reweighted least

squares IRLS), drone

PoI.

(FAST corner trackers), (M.

Trajkovic M. Hedley) [13, pp. 75-87],

, trackers

. ,

IRLS ,

.

,

(contrast), .

, (accuracy)

,

. ,

drone.

31

2.4.

2.4.1. PID

PID

, . 1930,

.

, . ,

(..

Kalman Filter [14]) , ,

.

, PID Proportional (P) Integral (I)

Derivative (D)

. ,

: PI, PD PID.

, ,

(tuning).

PID

Y(s) R(s).

E(s) Error,

,

. , PID

Y(s)

(feedback) .

15: PID

32

15 PID,

. ,

( )

drone. R(s)

(ultrasonic), , ,

, . ,

PID (Ziegler Nichols, Cohen Coon)

drone.

(2.2)

(2.3) PID Controller:

u(t) = KP e(t) + KI e()d + KD de(t)/dt (2.2)

C(s) = KP + KI /s + KD s (2.3)

:

KP (Proportional Gain)

KI (Integral Gain)

KD (Differential Gain)

(2.2)

, PID

.

PID,

:

KP (Proportional Gain)

.

KI (Integral Gain) .

KD (Differential Gain) .

, PID KI ,

. ,

33

. , KD

PD.

PID, 1

PID.

(Rise Time)

(Overshoot)

(Settling Time)

(Error)

P

I

D

1: PID

Parrot AR.Drone 2.0. [7],

PID .

:

Roll (x)

Pitch (y)

Yaw (yaw)

Altitude (z)

, , drone

PID ,

.

, (),

drone,

, . ,

[15] :

= (2.4)

= + (2.5)

16:

Roll, Pitch, Yaw

34

:

() = +

+ (2.6)

(, , ) real-time ,

(Dynamic Reconfigure). tum_ardrone

[4] , real-time

drone

. 17

rqt_reconfigure, PID

nodes tum_ardrone [16].

17: PIDs tum_ardrone.

17,

, yaw, gaz rp.

yaw ,

_, _, _ .

:

() = . (2.7)

, ,

drone .

35

gaz

, drone .

:

() = , +,

+ , (2.8)

PIDs (_, _, _),

, max_gaz_rise

drone

.

rp roll, pitch

UAV.

(_, _, _),

aggressiveness ()

. ,

:

() = , + , (2.9)

() = , + , (2.10)

PIDs

100Hz drone. ,

Extended Kalman Filter (EKF) [17],

,

. ,

3: ,

.

36

2.4.2. Bayes Bayes Filter

Bayes Filter, Recursive Bayesian Estimation

( Bayes), ,

(Probability Density Function PDF)

. , PDF

. ,

(state estimation) (beliefs),

drone

.

State Estimation

Bayes,

, UAV.

, ,

state, xt.

(static) ,

, (dynamic).

(pose) , (velocity)

(acceleration) . ,

(noisy data)

states , state

estimation, . state estimation

belief

.

states (control actions),

(measurements data)

[18]. state estimation , control measurement

data .

37

(measurement data) :

: = , + , + , , (2.11)

: : measurement data t1

t2, t1 t2.

, (control data):

: = , + , + , , (2.12)

: : control data t1

t2, t1 t2.

, measurement data

, control

data .

, drone control action 2

m/sec , 2 ( 4

) 3.9 4.1 , . ,

state estimation

, states

(probabilistic laws).

Markov (Markov Assumption) :

x ,

. ,

state ,

.

:

(Smoothing):

(Filtering):

(Prediction):

38

Markov Assumption :

(|: , : , :) = (| , ) (2.13)

: (| , ) (state transition

probability) (environmental

state) drone control data Ut.

,

belief, prediction belief .

:

() = (|: , :) (2.14)

, t

:

() = (|: , :) (2.15)

,

.

39

Bayes Bayes Filter

beliefs ,

(initial belief) bel t-1,

measurement data Zt control data Ut. , Bayes

Filter :

a) Prediction Step

b) Update Step

(prediction step) (belief)

(motion model 2.17):

(|:) (|:) (2.16)

() = (| , ) () (2.17)

, state estimation , update step.

(sensor model 2.19),

(update) belief

.

(|:) (|:) (2.18)

() = (|) () (2.19)

: belief .

,

, belief (correct state

estimation) navigation .

:

.

40

2.4.3. Kalman Kalman Filter

, Kalman Filter [14]. , Kalman Filter

(state) ,

(2.20)

(2.21):

= + + (2.20)

= ( + ) , ~ (, ) ~ (, ) (2.21)

, Kalman Filter

Bayes .

, ,

. , Kalman Filter

drone ,

.

, Kalman Filter

, ,

. ,

Bayes Filter (

Bayes)

(Smoothing) (Prediction),

.

, (Linear System)

Gauss (Gaussian noise) .

Gauss , PDF

( Gauss).

SLAM,

Kalman (Bayes Filter, Particle

Filter, EKF). SLAM (2.5),

drone

,

SLAM, .

41

(belief),

:

Single-Hypothesis Belief

Multi-Hypothesis Belief

Kalman Single-Hypothesis belief,

(single world state)

(variance)

. Single-Hypothesis

, . Multi-

Hypothesis (states),

.

18: &

, Kalman Filter Single-hypothesis belief

real time

.

42

Kalman Filter

Kalman ,

Bayes. (Prediction)

(Update). (belief) t

t t, , (inputs)

t-1, t-1, Ut Zt.

xt Bayes (2.17) (2.19),

t t.

, (prediction step) :

: = + (2.22)

: = + (2.23)

:

(Motion model)

C (Sensor model)

(matrix operations)

(update step) :

Kalman: = (

+ ) (2.24)

(updated): = + ( ) (2.25)

(updated): = ( ) (2.26)

:

Kalman

Q Gauss (Gaussian noise)

Zt (measurement data)

43

, Kalman bel(xt),

t t.

Kalman

,

, bel(xt-1)

, belief bel(xt)

. , ,

update

step ( , ) (, ).

,

( Kalman), Zt

, Zt .

Bayes Filter Kalman Filter

Kalman Bayes,

Kalman Bayes.

o (2.17):

() = (| , ) ()

:

(| , ) = (; + , ) (2.27)

() = (; , ) (2.28)

(2.17) (2.27) (2.28) :

() = (; + , + )

() = (; , ) (2.29)

44

o (2.19):

() = (|) ()

:

(|) = (; , ) (2.30)

() = (; , ) (2.29)

(2.19) (2.29) (2.30) :

() = (; + ( ), ( ) )

() = (; , ) (2.31)

:

= (

+ ) Kalman

Extended Kalman Filter

Kalman, ,

(linear systems) -

(non-linear system), UAV.

, Extended Kalman Filter EKF [17],

- motion sensor models .

, g h

, C Kalman:

= (, ) (2.32)

= () + (2.33)

Kalman filter, EKF prediction update

.

45

, prediction step

EKF :

: = (, ) (2.34)

: = + (2.35)

update step Kalman (2.34)

(2.35) :

Kalman: = (

+ ) (2.36)

(updated): = + ( ( )) (2.37)

(updated): = ( ) (2.38)

Gt Ht / At Ct .

, , Extended Kalman Filter -

Kalman Filter. ,

drone ,

Kalman.

46

2.5.

2.5.1. Simultaneous Localization and Mapping SLAM

,

,

(Simultaneous Localization and Mapping SLAM). ,

SLAM

. ,

. mapping

,

. , localization

,

.

,

SLAM , .

, SLAM (J. J. Leonard H. F. Durrant-

Whyte) [19] .

mapping

localization . ,

(sonar-based navigation)

.

(localization algorithm) extended Kalman Filter (EKF) [17],

sonar. , SLAM

(indoors) (outdoors)

, .. .

47

SLAM SLAM Techniques:

SLAM ,

UAV, : ,

2.4.2, 2.4.3,

(Scan Matching),

.

, SLAM,

(linearization) (prediction)

(update) , Bayes (

) Kalman Filter, EKF ( ).

,

,

2 [20].

1

(Feature to Feature) F2F

2

(Point to Feature) P2F

3

(Point to Point) P2P

4 Com

2: Scan Matching

, features ,

(PoI), . , F2F,

,

, ,

. ,

, features, points

. , P2P

points ,

.

48

P2P

(Iterative Closest Point ICP)

(Y. Chen G. Medioni) [21]. ,

,

P2P

. P2P

.

Feature Extraction

,

(Feature Extraction

Algorithms). , Feature Extraction

,

localization . (features points)

. , feature extraction

, ,

.

, (edges)

(corners).

o (edges) ,

3:

1 Canny (edge detector) John F. Canny [22]

2 Deriche (edge detector)

Canny Rachid Deriche [23]

3 Differential (edge detection)

4 Sobel filter Irwin Sobel & Gary Feldman [24]

5 Prewitt operator Judith M. S. Prewitt [25]

3:

https://en.wikipedia.org/wiki/Irwin_Sobel

49

3 (Canny edge

detector) [22], . ,

Gauss,

,

. ,

(double threshold) , ,

.

o (corners) ,

, 4.

1 Moravec corner detection Hans Moravec [26]

2 The Harris & Stephens corner detection Chris Harris & Mike Stephens [27]

3 ShiTomasi corner detection J. Shi & C. Tomasi [28]

4 Level Curve Curvature Approach L. Kitchen & A. Rosenfeld [29]

5 SUSAN S. M. Smith & J. M. Brady [30]

6 FAST E. Rosten & T. Drummond [31]

4:

Harris & Stephens [27],

Moravec [26].

Moravec, pixel pixel

pixel.

pixel, .

(similarity)

(SSD) pixel

, .

Harris & Stephens [27],

.

pixels , (differential)

pixels, (autocorrelation),

.

50

Feature Description

,

feature extraction .

, 5,

,

.

1 SIFT Scale-invariant Feature Transform David Lowe [32]

2 SURF Speeded Up Robust Features Herbert Bay, Andreas Ess, Tinne

Tuytelaars, & Luc Van Gool [33]

3 GLOH - Gradient Location and Orientation

Histogram

Krystian Mikolajczyk & Cordelia

Schmid [34]

4 HOG Histogram of Oriented Gradients Navneet Dalal & Bill Triggs [35]

5:

Points of Interest PoI

features .

,

[36].

o SIFT Scale-Invariant Feature Transform [32]

, ,

.

features

,

. SIFT ,

, feature descriptor

,

.

.

(keypoints) (database),

51

. ,

, keypoints

features, . features

good matches

, ,

. , good matches

(clusters) ,

Hough (Hough transform) [37], .

o SURF Speeded Up Robust Features [33]

(Image registration).

SIFT,

. ,

(blob detectors)

. DoH Determinant of Hessian

. SURF

PoI , Multi-resolution

Pyramid [38]. , ,

Laplacian Pyramid Pyramidal Gaussian

[39], , . ,

, scale-space, points of interest

features .

o HOG Histogram of Oriented Gradients [35]

,

, .

, ,

(cells).

, (intensity

gradients) . ,

, pixels

52

(histogram) .

(descriptor) .

,

,

block. ,

block .

(illumination) (shadowing). ,

:

,

.

,

(face detection).

19: HoG [40]

,

, laser ,

. ,

, Parrot AR Drone 2.0. [7]

monocular camera drone,

SLAM, monocular SLAM algorithms.

53

2.5.2. SLAM Monocular SLAM

,

,

(scale estimation) .

,

. ,

monocular SLAM [41] [42]

(real-time positioning), .

, Prof. Andrew Davison 2003,

2007 [43].

,

. , monocular SLAM

,

.

monocular SLAM ,

,

,

. , monocular SLAM

: (1) SLAM (feature-based SLAM) (2)

SLAM (direct SLAM). , feature-based SLAM

(SIFT,

SURF, ), direct SLAM feature extraction

description pixel pixel.

LCD-SLAM (Large-Scale Direct Monocular SLAM) [44]

ROS, AR Drone 2.0.

Feature-Based SLAM

, feature-based SLAM ,

. SIFT, SURF,

. ,

,

54

.

(Inverse Depth Parametrization) [45].

, ,

, 3D .

Direct SLAM LSD-SLAM

, LSD-SLAM [44]

-- (image-to-image alignment)

coarse-to-fine, Huber loss.

, coarse-to-fine (triangulation)

.

,

. , Huber

loss, .

,

.

,

(texture tracking). ,

real-time pose graph,

.

20: Pose Graph [46]

55

, semi-dense,

pixels

. :

https://youtu.be/GnuQzP3gty4 [46]

feature-based SLAM Direct SLAM,

,

. , direct SLAM

,

. ,

UAV.

2.5.3. PTAM Parallel Tracking & Mapping

,

(Parallel Tracking and Mapping PTAM) [47],

. ,

real-time , (

) (tracking). , (mapping)

,

feature-based .

, : (1)

tracking, (2) mapping.

,

, threads multi-core

tracking, points-based

.

1) Tracking: ,

SLAM. ,

(triangulation)

feature extraction description ,

(stereo initialization)

56

.

, (relative position)

,

.

(

SURF),

frames real-time

, .

,

,

, . ,

(stereo initialization)

. ,

features

, . ,

,

. ,

MonoSLAM [43], PTAM

,

. ,

, tracking ,

.

2) Mapping: , tracking mapping

. ,

. , ( )

,

.

mapping,

.

keyframes

frame . ,

real-time ,

, frames

57

. , keyframes frames

,

. ,

, thread

keyframes ,

.

keyframes

, 20 frames .

(1)

keyframes, (2) (map

corruption).

21: PTAM AR Drone

PTAM,

,

. ,

PTAM

drone. , PTAM

tum_ardrone [4] [48] [49] [50].

58

3. 3:

3.1.

,

6

,

multi-megawatt . ,

,

. ,

ROS [3] (1) ardrone_autonomy [5] (2) tum_ardrone [4],

,

Parrot

AR Drone [7].

22: UAV [51]

59

3.2. &

,

, . ,

,

.

,

, UAV Parrot AR Drone 2.0. [7]

.

UAV

UAV .

(initial state), ,

. ,

, drone

, .

drone , .

, ,

, drone

. ,

feature-based [52],

.

, ,

. , , drone

. ,

, UAV

,

.

(3.3.1, 3.3.2), ,

,

.

60

, (3.4.2)

, Parrot AR Drone 2.0. [7]

. ,

drone . ,

open-source Gazebo [6]

drone .

,

. , tum_ardrone [4]

ardrone_autonomy [5] ,

.

,

, ,

. ,

,

. ,

, .

topics nodes,

. (3.4)

,

.

61

3.3.

UAV Basics of Wind Turbine Blade Inspection with UAV

3.3.1. Basics and Introduction

,

. ,

, ,

. ,

,

.

, ,

.

(inspection) ,

(rope access technicians).

23: Rope Access Technician [53]

62

, ,

, ,

, . ,

,

. ,

.

, , ,

, .

drones,

. ,

drones,

, .

, ( ),

drone ( ),

. ,

,

, . ,

, UAV

. ,

.

& Aerodynamics & Wind Power

, .

(rotor blade) (hub)

(upwind) ,

,

[54]. ,

drone.

, drone

,

. ,

63

drone.

, ,

(lift) (wind flow)

(incident flow).

, incident flow

,

.

,

,

. ,

, ,

10% 2 m/sec

14 m/sec ( O&M Operation &

Maintenance). ,

.

,

,

. drone,

. ,

,

.

24:

[57]

64

3.3.2. & Challenges & Solutions

,

,

drone .

,

.

UAV,

25 m/sec, drone

, .

, ,

, ,

.

.

, .

UAV , ,

( 1.5 ),

. ,

UAV drone

,

(initial state). , drone GPS

, laser beacons

,

. , ,

drone (fine-tuned control

system), PIDs . ,

PID (inner loop) , drone

. ,

(overcompensation)

. ,

65

,

- .

control system, drone

, .

(outdoor navigation)

(layered control

architecture),

,

. , drone

,

.

, (flight planner)

,

(. 2.4),

.

drone ,

. ,

,

,

. , .

, drone ,

. , drone

, ,

drone

, .

, , ,

.

, SLAM [19]

, , laser GPS

. ,

EKF [17] drone

66

, . ,

ROS [3]

,

tum_ardrone [4] PTAM [47],

, real-time . ,

feature-based (. 2.5.1) drone

(, , , ..) ,

. ,

UAV, (emergency mode)

,

.

UAV

UAVs, . ,

drone .

(software), (hardware).

, ,

, ,

(pitch velocity). ,

, drone pitch speed

. , UAV

. ,

UAV. ,

stall.

drone (stalling)

, PID

. ,

stall,

drone,

. ,

67

, drone

, ( )

PID. stall,

drone . ,

, UAV.

,

. drone

,

. , ,

8+ . , ,

UAV.

, Octo X Octo +

.

25: "" "+" [55]

, Parrot AR Drone 2.0. [7]

X4 (Quadcopter)

420g,

. , , ardrone

.

68

3.4.

3.4.1.

rdrone autonomy

open-source

, ardrone_autonomy [5] [56]

ROS [3] Parrot AR Drone 2.0 [7]. ,

nodes ardrone.

, ,

ROS,

[56].

,

C++

ardrone. ,

. (2.2.2),

ROS ,

.

:

data/camera_info

drone. ,

,

include/ardrone_autonomy header files

,

.

launch launch launch files, ,

/ nodes,

. ,

, ardrone.launch, ardrone.

msg , (2.2.2),

.

32 ,

drone (navigation data navdata).

69

src ,

.

, ardriver.cpp

driver . , ardrone_sdk.cpp

ardrone [7]

drone ROS. ,

teleop_twist.cpp video.cpp

drone.

srv , services

drone (.. ).

, ardrone_autonomy

Parrot AR Drone ROS

driver ,

ardrone .

tum ardrone

tum_ardrone [4] [16], PTAM [47]

ardrone. , PTAM

ardrone ,

. , drone,

.

C++

PTAM,

drone . ,

, src.

, UINode, autopilot stateestimation,

.

UINode UINode User Interface Node,

.

header files implementation files,

70

,

ROS ardrone.

autopilot , ,

nodes

ardrone , UI.

, ControlNode.cpp, DroneController.cpp

KI. ,

ardrone.

ardrone, ,

. ControlNode.cpp

, topics,

KI.

, drone publishers

,

ardrone. , DroneController.cpp

PIDs EKF.

stateestimation ,

PTAM EKF,

(mapping). , State Estimation

, ,

. , PTAM

(header files implementation files)

PTAM . ,

DroneKalmanFilter.cpp EstimationNode.cpp,

drone ,

EKF. , PTAMWrapper.cpp MapView.cpp,

mapping

.

, tum_ardrone

drone,

ardrone. , nodes

, autopilot stateestimation.

71

3.4.2.

,

Python

, . ,

open-source tum_ardrone [4] ardrone_autonomy [5]

. ,

(functions) , .

DroneNavigation,

.

: __init__()

, publisher

subscriber ,

.

def __init__( self ):

print "initialized"

self.land_pub = rospy.Publisher( "ardrone/land", Empty, queue_size=10 )

self.takeoff_pub = rospy.Publisher( "ardrone/takeoff", Empty, queue_size=10 )

self.reset_pub = rospy.Publisher( "ardrone/reset", Empty, queue_size=10 )

self.command_pub = rospy.Publisher( "/tum_ardrone/com", String, queue_size=10 )

self.predictedPose = rospy.Subscriber( "/ardrone/predictedPose", filter_state,

self.position_update )

self.pose = Point( 0, 0, 0 )

self.pose_yaw = 0.0

self.target = Point( 0, 0, 0 )

self.target_yaw = 0.0

self.target_found = 0

self.current_cmd = Twist()

self.speed = 0.3

self.yaw_speed = 0.3

self.main_logic()

publishers subscribers ,

drone. , publishers

(land_pub, takeoff_pub, reset_pub) ,

72

drone, . , publisher command_pub subscriber

predictedPose, tum_ardrone

.

, , , hover

.

def takeoff( self ):

self.takeoff_pub.publish( Empty() )

def land( self ):

self.land_pub.publish( Empty() )

def reset( self ):

self.reset_pub.publish( Empty() )

def hover( self ):

self.command_pub.publish( "c goto 0 0 0 0" )

, publishers

, tum_ardrone.

: relative_coordinates()

,

.

drone, drone.

73

def relative_coordinates( self ):

## Coordinate Z

rel_z = self.target.z - self.pose.z

## Coordinate Yaw

rel_yaw = self.target_yaw - self.pose_yaw

if rel_yaw > 180:

rel_yaw -= 360

if rel_yaw < -180:

rel_yaw += 360

## Coordinate X, Y

rel_x = self.target.x - self.pose.x

rel_y = self.target.y - self.pose.y

## Rotating dest_vector (difference between reference and pose yaw)

theta = np.deg2rad( self.pose_yaw )

rotated_x = ( rel_x * np.cos( theta ) ) - ( rel_y * np.sin(theta ) )

rotated_y = ( -rel_x * np.sin( theta ) ) - ( rel_y * np.cos(theta ) )

return ( rotated_x,rotated_y,rel_z,rel_yaw )

,

, drone ,

drone. ,

. yaw,

, drone

. , theta

,

. ,

, ,

, rotated_x, rotated_y, rel_yaw, rel_z.

: distance_to_target()

, drone

, .

74

def distance_to_target( self ):

relative = Point ( 0, 0, 0 )

relative.x = self.target.x - self.pose.x

relative.y = self.target.y - self.pose.y

relative.z = self.target.z - self.pose.z

dist_array = np.array( ( relative.x, relative.y, relative.z ) )

return np.sqrt( np.sum( dist_array**2 ) )

,

x, y z,

drone.

, Point,

tum_ardrone .

: angle_to_target()

, angle_to_target (yaw)

drone .

def angle_to_target( self ):

angle = np.absolute( self.pose_yaw - self.target_yaw )

if angle > 180:

angle = np.absolute( angle - 360 )

return angle

,

drone angle,

180 ( ).

: position_update()

position_update ,

drone .

75

def position_update( self, data ):

x = float( data.x )

y = float( data.y )

z = float( data.z )

self.pose_yaw = float( data.yaw )

self.pose = Point ( x, y, z )

topic /ardrone/predictedPose,

tum_ardrone,

, Point.

: moveto()

drone , moveto.

publisher, drone

, real-time

.

def moveto( self, x, y, z, yaw, range ):

self.target = Point( x,y,z )

self.target_yaw = yaw

print self.target

self.command_pub.publish( "c clearCommands" )

self.command_pub.publish( "c goto %f %f %f %f"%( self.target.x,self.target.y,self.target.z,

self.target_yaw ) )

while ( self.distance_to_target() > range or self.angle_to_target() > 6 ):

print "\nWithin distance %f"%( self.distance_to_target() )

print " I am at: x:%f , y:%f , z:%f ,yaw:%f"%( self.pose.x, self.pose.y, self.pose.z,

self.pose_yaw )

print "Going to: x:%f , y:%f , z:%f ,yaw:%f\n"%(

self.target.x,self.target.y,self.target.z, self.target_yaw )

rospy.sleep(0.1)

return True

, drone

command_pub c clearCommands c goto

tum_ardrone drone .

, while()

drone (remaining distance),

76

(current position) (destination position) (

debugging).

: arc_coordinates()

drone

.

def arc_coordinates( self, distance, angle ):

arc_length = 2 * np.sin(angle / 2) * distance

phi = (np.pi - angle) / 2

new_x = arc_length * np.sin(phi)

new_y = arc_length * np.cos(phi)

move_coordinates = (new_x,new_y)

return move_coordinates

,

drone ,

drone,

drone.

: main_logic()

, drone

.

, drone

. , drone

, PTAM tum_ardrone.

, drone

. , drone

,

, .

3.5.

,

,

drone ,

77

Gazebo.

Gazebo.

26: Gazebo

26 drone ,

Gazebo [6] tum_ardrone [4]

ardrone_autonomy [5]. , 27 (timeframe)

, drone

, ,

.

78

27: Gazebo

79

4. &

4.1.

,

, . ,

UAVs,

, .

, ,

. ,

,

. ,

,

. ,

,

.

4.2.

,

,

.

,

.

,

,

. ,

,

. ,

UAV,

.

80

[1] The UAV, []. Available: http://www.theuav.com/. [ 2016].

[2] BAA Training, BAA Training, 2015. []. Available:

https://www.baatraining.com/uav-types-how-to-choose-yours/. [ 2016].

[3] M. Quigley, B. Gerkey, K. Conley, J. Faust, T. Foote, J. Leibs, E. Berger, R. Wheeler A. Ng,

ROS: an open-source Robot Operating System, ICRA Workshop on Open Source Software,

CA, 2009.

[4] J. Engel, Computer Vision Group, Informatik IX, Computer Vision Group, [].

Available: http://vision.in.tum.de/data/software/tum_ardrone.

[5] M. Monajjemi , ardrone_autonomy, Autonomy Lab, Simon Fraser University, [].

Available: http://ardrone-autonomy.readthedocs.io/en/latest/index.html#.

[6] N. Koenig A. Howard, Design and Use Paradigms for Gazebo, An Open-Source Multi-Robot

Simulator, Proceedings 01 2004 IEEE!RSJ International Conference on Intelligent Robots

and Systems, Los Angeles, 2004.

[7] Parrot, Parrot SA, []. Available: http://www.parrot.com/uk/products/ardrone-2/.

[8] P.-J. Bristeau, F. Callou, D. Vissire N. Petit, The Navigation and Control technology inside

the AR.Drone micro UAV, Paris, FRANCE, 2011.

[9] RobotShop, RobotShop Inc., []. Available:

http://www.robotshop.com/en/brushless-motor-kit-ardrone.html.

[10] Wikipedia, []. Available: https://en.wikipedia.org/wiki/Abbe_error.

[11] B. D. Lucas T. Kanade, An iterative image registration technique with an application to

stereo vision, Proceedings of Imaging Understanding Workshop, Pennsylvania, 1981, pp.

121-130.

[12] B. G. Schunck B. K. Horn, "Determining optical flow": a retrospective, Artificial

Intelligence, USA, 1993, pp. 81-87.

[13] M. Trajkovic M. Hedley, Fast corner detection, Image and Vision Computing, 1998, p.

16:7587.

[14] R. E. KALMAN, A New Approach to Linear Filtering and Prediction Problems, Transactions

of the ASME Journal of Basic Engineering, . 82, pp. 35-45, 1960.

[15] A. KOSZEWNIK, THE PARROT UAV CONTROLLED BY PID CONTROLLERS,

Bialystok, 2014.

81

[16] GitHub, TUM Computer Vision Group, []. Available: https://github.com/tum-

vision/tum_ardrone.

[17] S. J. JULIER J. K. UHLMANN, Unscented Filtering and Nonlinear Estimation,

PROCEEDINGS OF THE IEEE, . 92, . 3, pp. 401-422, MARCH 2004.

[18] N. Dijkshoorn, Simultaneous localization and mapping with the AR.Drone, 2012.

[19] J. J. Leonard H. F. Durrant-Whyte, Simultaneous map building and localization for an

autonomous mobile robot, Workshop on Intelligent Robots and Systems, vol 3, . 3, pp. 1442-

47, 1991.

[20] C.-C. Wang C. Thorpe, Simultaneous Localization and Mapping with Detection and

Tracking of Moving Objects, Robotics Institute, Carnegie Mellon University, Pittsburgh.

[21] Y. Chen G. Medioni, Object Modeling by Registration of Multiple Range Images, IEEE

Int. Conf. On Robotics and Automation, 1991.

[22] J. Canny, A Computational Approach To Edge Detection, Pattern Analysis and Machine

Intelligence, IEEE Trans, 1986, p. 679698.

[23] R. Deriche, Using Canny's criteria to derive a recursively implemented optimal edge detector,

Computer Vision, Int. J., 1987, p. 167187.

[24] I. Sobel, History and Definition of the Sobel Operator, 2014.

[25] J. Prewitt , Object Enhancement and Extraction" in "Picture processing and Psychopictorics,

Academic Press, 1970.

[26] H. P. Moravec, Obstacle Avoidance and Navigation in the Real World by a Seeing Robot

Rover, Pennsylvani, 1980.

[27] C. Harris M. Stephens, A combined corner and edge detector, Proceedings of the 4th

Alvey Vision Conference, United Kingdom, 1988, p. 147151.

[28] J. Shi C. Tomasi , Good Features to Track, 1994.

[29] L. Kitchen A. Rosenfeld , Gray-level corner detection, Pattern Recognition Letters,

1982, pp. 95-102.

[30] S. M. Smith J. M. Brady , SUSAN a new approach to low level image processing,

International Journal of Computer Vision, 1997, p. 4578.

[31] E. Rosten T. Drummond, Machine learning for high-speed corner detection, 2006.

[32] D. Lowe, Object recognition from local scale-invariant features, Proceedings of the

International Conference on Computer Vision, 1999, p. 11501157.

[33] H. Bay, A. Ess, T. Tuytelaars L. Van Gool, Speeded Up Robust Features (SURF), ETH

Zurich, Belgium.

82

[34] K. Mikolajczyk C. Schmid, A performance evaluation of local descriptors, IEEE

Transactions on Pattern Analysis \& Machine Intelligence, 2005, pp. 1615--1630.

[35] N. Dalal B. Triggs, Histograms of Oriented Gradients for Human Detection, Conference

on Computer Vision and Pattern Recognition, France, 2005.

[36] P. J. Burt, Fast filter transforms for image processing, Computer Graphics and Image

Processing, . 16, Elsevier Inc., 1981, pp. 20-51.

[37] R. O. Duda P. E. Hart, Use of the Hough Transformation to Detect Lines and Curves in

Pictures, Comm. ACM, Vol. 15, p. 1115, January 1972.

[38] E. H. Adelson, C. H. Anderson, J. R. Bergen, P. J. Burt J. M. Ogden, Pyramid Methods in

Image Processing, RCA Engineer, 1984.

[39] K. G. Derpanis, The Gaussian Pyramid, 2005.

[40] R. Sharma A. Savakis, SPIE, 16 October 2015. []. Available:

http://electronicimaging.spiedigitallibrary.org/article.aspx?articleid=2473553.

[41] S. Weiss, D. Scaramuzza R. Siegwart, Monocular-SLAMBased Navigation for

Autonomous Micro Helicopters in GPS-Denied Environments, Journal of Field Robotics, .

28, . 6, pp. 854-874, 2011.

[42] A. J. Davison, Real-Time Simultaneous Localisation and Mapping with a Single Camera.

[43] A. J. Davison, I. D. Reid, N. D. Molton O. Stasse, MonoSLAM: Real-Time Single Camera

SLAM, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,

. VOL. 29, . 6, JUNE 2007.

[44] J. Engel, T. Schps D. Cremers, LSD-SLAM: Large-Scale Direct Monocular SLAM,

Technical University Munich, Munich, 2014.

[45] J. Civera, A. J. Davison J. M. Martinez Montiel, Inverse Depth Parametrization for

Monocular SLAM, IEEE TRANSACTIONS ON ROBOTICS, . 24, . 5, October 2008.

[46] Computer Vision Group, Computer Vision Group, Informatik IX, []. Available:

http://vision.in.tum.de/research/vslam/lsdslam. [ July 2016].

[47] G. Klein D. Murray, Parallel Tracking and Mapping for Small AR Workspaces, 6th

IEEE and ACM International Symposium on. IEEE, 2007.

[48] J. Engel, J. Sturm D. Cremers, Scale-Aware Navigation of a Low-Cost Quadrocopter with a

Monocular Camera, Munich, 2013.

[49] J. Engel, J. Sturm D. Cremers, Camera-Based Navigation of a Low-Cost Quadrocopter,

Munich, 2013.

[50] J. Engel, J. Sturm D. Cremers, Accurate Figure Flying with a Quadrocopter Using Onboard

Visual and Inertial Sensing, Munich, 2013.

83

[51] MIAT, MIAT College of Technology, []. Available:

https://plus.google.com/+MIATCollegeofTechnologyCanton/posts.

[52] P. Smith, I. Reid A. Davison, Real-Time Monocular SLAM with Straight Lines, UK, 2006.

[53] MAIN PUMP, []. Available: http://www.mainpump.ru/news/lifting/5473.htm.

[54] P. Dvorak, Windpower Engineering & Development, 5 May 2015 . []. Available:

http://www.windpowerengineering.com/design/how-turbulent-wind-abuse-wind-turbine-

drivetrains/. [ 2016].

[55] O. Liang , scarLiang.com, 25 October 2013. []. Available:

https://oscarliang.com/types-of-multicopter/. [ July 2016].

[56] GitHub, Autonomy Lab at SFU, []. Available:

https://github.com/AutonomyLab/ardrone_autonomy.

[57] dianliwenmi, []. Available:

http://www.dianliwenmi.com/postimg_4532273_11.html.

[58] The Electric Energy, []. Available: http://theelectricenergy.com/what-is-a-

brushless-dc-electric-motor/.

[59] C. Hill, Equipment World, 22 December 2015. []. Available:

http://www.equipmentworld.com/n-carolina-dot-issues-its-own-drone-guidelines-for-

recreational-operation/. [ 2016].

[60] Alpha Unmanned Systems, Alpha Unmanned Systems, []. Available:

http://www.alphaunmannedsystems.com/gallery.

[61] Arrishobby.com, ARRIS HOBBY, []. Available: http://www.arrishobby.com/dji-

newly-released-flagship-multirotor-controller-a2-p-995ARRIS H.

[62] Clearpath Robotics, Robohub.org, 29 January 2014. []. Available:

http://robohub.org/ros-101-intro-to-the-robot-operating-system/.

[63] P. Bupe, http://diydrones.com/, 14 April 2015. []. Available:

http://diydrones.com/profiles/blogs/autonomous-uav-clustering-network.

1. 1: 1.1. 1.2. UAV 1.3.

2. 2: 2.1. 2.2. Software2.2.1. ROS Robot Operating System2.2.2. - Open-Source ROS Packages2.2.3. Gazebo Simulator

2.3. Hardware2.3.1. Parrot AR Drone 2.02.3.2. & Sensors & Motors2.3.3. IMU Inertial Measurement Unit2.3.4. Monocular Camera

2.4. 2.4.1. PID2.4.2. Bayes Bayes Filter State Estimation Bayes Bayes Filter

2.4.3. Kalman Kalman Filter Kalman Filter Bayes Filter Kalman FilterExtended Kalman Filter

2.5. 2.5.1. Simultaneous Localization and Mapping SLAM SLAM SLAM Techniques: Feature Extraction Feature Description

2.5.2. SLAM Monocular SLAMFeature-Based SLAM Direct SLAM LSD-SLAM

2.5.3. PTAM Parallel Tracking & Mapping

3. 3: 3.3.1. 3.2. & UAV

3.3. UAV Basics of Wind Turbine Blade Inspection with UAV3.3.1. Basics and Introduction & Aerodynamics & Wind Power

3.3.2. & Challenges & Solutions UAV

3.4. 3.4.1. rdrone autonomy tum ardrone

3.4.2. : __init__() : relative_coordinates(): distance_to_target(): angle_to_target(): position_update(): moveto(): arc_coordinates(): main_logic()

3.5.

4. & 4.4.1. 4.2.