Download - Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

Transcript
Page 1: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

Anomaly Detection Studies in the IP Backbone

Tao Ye

Sprint

Burlingame, CA

2007-09-19

Page 2: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

2

24: “Stop that packet at the router!”

• Detected an anomaly

• Specify and activate a new ACL

• At OC-192 routers? @500Kpps? In 26μs?

Anomaly Detection at the IP Backbone

Page 3: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

3

Outline

• Tier-1 backbone: an overview

• TAPS: connectionless port scan detection and tracking on the backbone

• Scaling up: sampling and anomaly detection

Page 4: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

4

Today’s Tier-1 Backbones

• Topology – high speed routers in points-of-presence (POPs) connected by long-haul fiber > numerous small POPs (e.g., UUNet)> relatively few large POP (e.g., Sprint)

• Technologies > IP over SONET (POS)> IP over ATM (phasing out)> MPLS, VPN tunnel

• Common Engineering Practice> failure protection implemented at IP layer> “over-provisioned” core

Page 5: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

5

What we (Research group @ Sprint ) do

• Measurement: Collect a lot of data from the Internet backbone, understand the current state

• Monitoring: Use of measurement to detect events of (operational) interest

• Hardware> CMON Monitoring boxes in the field, @5 POPs> Storage (30T) and analysis platform at the lab> Website for sharing results

• Algorithms and Software tools> Continuous monitoring> Anomaly detection> Active measurement

• Other: > Wireless

• Paging attacks• Fairness implementations• TCP over wireless

Page 6: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

6

Outline

• Measurement and Monitoring at a tier-1 backbone: an overview from the industry perspective

• TAPS: Connectionless port scan detection and tracking on the backbone

• Scaling up: sampling and anomaly detection

Page 7: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

7

Motivation and Challenges

• Our goals> Detect and track> Understand long term behavior of scanners> On the backbone network

• Why Backbone ? > Detection: Existing work most at stub networks, limited

visibility > Tracking: Honeypots can be evaded> More scanning activities visible at core> Peering point unique vantage point

• Challenges> Backbone traffic unidirectional, asymmetric> High speed (OC-48, OC-192) links, needs fast algorithm> Diverse traffic mix, needs efficient data structure

Page 8: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

8

Intuition: Access Patterns

Page 9: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

9

TAPS: Time-based Access Pattern Sequential hypothesis testing

• Based on 5-tuple flow summary on unidirectional link

• Scanner suspects: source IPs accesses IP/port (or port/IP) ratio > k in time-bin

• Sequential Hypothesis Testing

1

1

0

1

1

0

[ 1 | ] IPif

[ 1 | ] Port

[ 0 | ] IPif

[ 0 | ] Port

i

i

P Y Hk

P Y HiP Y H

kP Y H

Page 10: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

10

TAPS

( ) 1Y

1( )Y

0( )Y

Threshold for tagging source as scanner

Increment when IP/port > K

Decrement when IP/port < K

Threshold for tagging source as benignScanner if i 1

Benign if i 0

> <

SrcIP

Page 11: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

11

Performance: TCP

Detection Algo Comparison

90.20%

81.70%

35.10%

9.80%12.10%

45.60%

25.70%

74.30%

64.90%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Success False Positive False Negative

per

cen

tag

es

TAPS

TRWSYN

SNORT

Page 12: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

12

Online Implementation Architecture

• Use CMON to produce flows in NetFlow5

• Flow Daemon distributes flows

• Keep flows in circular buffer

CMON Flow Collector

Flow

Daemon

Core

App Handler

TAPS Other

Disk Writer

Disk Reader

Circular Buffer

Disk

Flow Daemon

Page 13: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

13

Detector and Tracker Architecture

Page 14: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

14

Design choices: Approximation Counters

• Issues: > Need to keep the fan-out count

for each IP> Heap implementation has

prohibitively high memory requirements

• Probabilistic Counters: > Many recently proposed

counters: • Small SRAM Implementation:

Multi-resolution bitmap, trigger bitmap

> Simple Flajolet-Martin counter

• FM counter performance> 8 hash functions accurate

enough for <>k test> 256, 32 and 8 hash functions

Page 15: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

15

Results

• Data set> OC48 Peering link incoming, ~320Mbps, 22 days> OC48 Peering link outgoing, ~560Mbps, 3 days

Page 16: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

16

Scanner Duration

22 days 3 days

Page 17: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

17

Scanner Rate

Page 18: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

18

Number of Scanner Detected (1)

• Time series of Number of scanners detected (3days)

Page 19: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

19

Scanning Ports

• Port accessed

Page 20: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

20

Conclusion

• Online Scan Detection and Tracking> Targets unidirectional backbone link> Detector: Time-based Access Pattern Sequential

Hypothesis (TAPS)• Combines rate limiting with statistical tests on destination IP

and port access patterns> Implementation design: Queue model and FM counter

• Scanner Behavior> 90-10 split of scanning rate, scanning duration behavior> Spike in number of scanners detected

Page 21: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

21

Outline

• Tier-1 backbone: an overview

• TAPS: connectionless port scan detection on the backbone

• Scaling up: sampling and anomaly detection

Page 22: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

22

Motivation

• Sampling to reduce processing overhead in traffic monitoring

• Sampled data used in:> Traffic Engineering -- computing traffic matrices> Inferring flow statistics from sampled data (Duffield03,

Hohn03)

• Anomaly Detection (DDoS attacks, worm scans):

Does sampled data contain sufficient information for effective anomaly detection?

• The brief answer … it depends> On sampling method> On sampling rate

• The impact of sampling> Number of anomalies detected: decreased> False positives: increased

Page 23: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

23

Methodology

Anomaly Detection Module

Traffic

traces

Anomaly Detection Module

Sampling Module

Results

Results

compare

Page 24: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

24

Anomalies and Detection Algorithms

Type of Anomaly Detection Algorithms

Volume Anomaly :

DoS attacks, flash crowds

1. Wavelet-based change detection [Barford02]

Port Scanning:

Worm/virus propergation

2. Threshold Random Walk [Jung04]

3. Access Pattern: TAPS [Sridharan06]

Anomaly Detection Module

Traffic

tracesAnomaly Detection

ModuleSampling

Module

Results

Results

compare

Page 25: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

25

Sampling Methods

• Random packet sampling: each packet sampled with probability r < 1

> Simple implementation (good for busy routers)> Widely deployed (Cisco NetFlow)> Flow statistics hard to recover

• Random flow sampling: classify flows, each flows sampled with probability p < 1

> High resource requirement> Accurate estimation of flow statistics

Anomaly Detection Module

Traffic

tracesAnomaly Detection

ModuleSampling

Module

Results

Results

compare

Page 26: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

26

Sampling (continue)

• Designer flow sampling: for catching heavy-hitters> Smart Sampling [Duffield02] – flow records selected with a

probability

> Sample-and-Hold [Estan02]:

Each byte of a packet sampled with a small probability h. All the following packets in the flow will be sampled once the a packet in the flow gets sampled.

Page 27: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

27

Comparing Sampling Algorithms

• How to compare: normalizing CPU load, or memory consumption

• Our choice – the percentage of flows sampled> Input to the anomaly detection based on flows,> Number of flows translates to memory consumption.

• Example of sampling parameter settings:

Page 28: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

28

Impact of Sampling on Volume Anomaly Detection (1)• Wavelet-base change

detection on flow rate

• Decomposition

• Re-synthesize into three bands

•High ~ 1sec

•Mid ~ 1min

•Low ~ 15min

• Detection on high/mid

•Sliding window

•Deviation score

Page 29: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

29

Impact of Sampling on Volume Anomaly Detection (2)

• Original detection: 21

• False negatives > Random flow sampling introduces more local variance

> Random packet sampling introduces even more variance> Smart sampling and sample-and-hold flatten the time

series

Page 30: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

30

Impact of Sampling on Port Scan Detection

• Performance Metrics Definition> Success Ratio Rs= Num True Scanners Detected / Num True Scanners> False Positive Ratio Rf+= Num False Scanners Detected / Num True Scanners

• Rs => effectiveness, Rf+= errors

• Ground truth: True scanner set examined by hand.

Page 31: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

31

TRWSYN results

Page 32: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

32

TAPS results

• Flow count reduction – false negatives

• Flow shortening – false positives shoot up in random packet sampling.> A multi-packet TCP flow shrunk to a single SYN-packet flow> The result: scanners and benign hosts are statistically

indistinguishable.

Page 33: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

33

Conclusion

• Implications of Our Results:> Random flow sampling is generally robust to both volume

anomaly and port scan detections.> Random packet sampling is oblivious to any underlying

traffic features, and causes information loss and distortion which degrade the performance of anomaly detection algorithms.

• Smart sampling and sample-and-hold target heavy-hitters, thus not quite suitable for anomaly detections.

• Ongoing work: > Design anomaly detection algorithms robust to sampling,> Design new anomaly-detection-friendly sampling methods.

Page 34: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

34

The End!

• Tier-1 backbone: an overview

• TAPS: Connectionless port scan detection on the backbone and scanner profiling

• Sampling data is not NOT sufficient for anomaly detection purposes

http://research.sprintlabs.com

Page 35: Anomaly Detection Studies in the IP Backbone Tao Ye Sprint Burlingame, CA 2007-09-19.

35

A Backbone POP

Peer

Core Router

Other POPs

Edge Router