Post on 25-Mar-2021
Live TrafficAli Ghandi
December 2020
alighandij.github.io
www.linkedin.com/in/alighandi/
2
Balad
2
• Start: December 2016
• Release: May 2018
Balad
2
• Team members: 110
• Users: +5 million
• POI’s: +700,000
What is Live?
3
xt
4
What is Live?
1 minutes window
Δx/Δt
GPS location error
5
What is Live?
9:039:049:109:01
Car #1Car #2Car #3Car #4
edgeV
ttt
e v
e
6
Change point
BOCD
7
( )mr n
( ), (0)t tr
Bayesian online change point detection
8
BOCD
1
1/ , 0
( | ) 1 1/ , 1
0 .
t t
m
P r m r n m n
o w
Hazard:
let’s assume that the
‘natural chance’ of a
change point is 1/λ
9
BOCD
1
1
1
1 1:
1 1: 1 1 1: 1
1 1 1: 1 1: 1
( , , )
( , | , ) ( , )
( | ) ( | , ) ( , )
t
t
t
t t tr
t t t t t tr
t t t t t t tr
P r r x
P r x r x P r x
P r r P x r x P r x
1:( , )t tP r x
9
BOCD
1
1
1
1 1:
1 1: 1 1 1: 1
1 1 1: 1 1: 1
( , , )
( , | , ) ( , )
( | ) ( | , ) ( , )
t
t
t
t t tr
t t t t t tr
t t t t t t tr
P r r x
P r x r x P r x
P r r P x r x P r x
1:( , )t tP r x
9
BOCD
1
1
1
1 1:
1 1: 1 1 1: 1
1 1 1: 1 1: 1
( , , )
( , | , ) ( , )
( | ) ( | , ) ( , )
t
t
t
t t tr
t t t t t tr
t t t t t t tr
P r r x
P r x r x P r x
P r r P x r x P r x
1:( , )t tP r x
9
BOCD
1
1
1
1 1:
1 1: 1 1 1: 1
1 1 1: 1 1: 1
( , , )
( , | , ) ( , )
( | ) ( | , ) ( , )
t
t
t
t t tr
t t t t t tr
t t t t t t tr
P r r x
P r x r x P r x
P r r P x r x P r x
1:( , )t tP r x
10
BOCD
1:1:
1:
( , )( ) ( | )
( )
t tt t t
t
P r xr n P r x
p x
Bayesian online change point detection
Stateful Stream Processing
11
Online algorithms based on state• Can use Flink for production
• Last 10 minutes of events are state
• Parameters of change point detection are state
1 minutes window 1 minutes window 1 minutes window
states states states
Parameters update
base on online data.
System recover itself
Future Work
12
• Graph neural networks (GNNs)
• Road ranking
Future Work
13
NLP-Topic modeling
Each road can be a document and
Events can be it’s words.
▪ Problems and Challenges (Cont.)Questions
14