Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April...

18
Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014

Transcript of Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April...

Page 1: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Naiad

Timely Dataflow System

Presented by Leeor Peled

Advanced Topics in Computer Architecture

April 2014

Page 2: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Greece, sometime B.C.

Naiads (Ναϊάδες) were a type of nymph (female spirit) who presided over fountains, wells, springs, streams, brooks and other bodies of fresh water.

Dryads (Δρυάδες) are tree nymphs, or female tree spirits

There’s one hiding over here!

Page 3: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Dryad (EuroSys’07)

Workflow modeled as a DAG

Vertices represent computational tasks Edges represent communication channels

Files, TCP pipe, shared memory FIFOs Subsets of edges marked as input/output

Parallelism is achieved by scheduling vertices to run on multiple nodes

Model allows graphs to be extended, merged, … C++ based

Page 4: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Distributed dataflow frameworks

Batch processing MapReduce, Dryad, Spark

Deterministic, stateless, synchronous Stream processing

Storm, Millwheel, Timestream

Asynchronous Graph processing

Pregel, Graphlab, Giraph

Page 5: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Graph descriptive language

Page 6: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Timely Dataflow – from 10k ft ”Naiad” (SOSP`13)

Framework no longer requires a DAG Nodes are stateful, and extended with epoch-based

timestamps Allows simple loop contexts. Can preserve datasets

No global coordination on critical path (some lazy “GC”) Choice between immediate responsiveness and

aggregated syncs Better suited to implement graph algorithms over big-

data streaming input …and it’s open sourced!

Page 7: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Processing example

Page 8: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

For comparison.. Map/Reduce model

Page 9: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Timeliness in depth

Introducing ingress/egress and feedback vertices

Each timestamp includes:

Supporting up to k levels of loop nesting per any given vertice

Page 10: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Timeliness (2)

Update rules

For given and , we say iff both and , where the vectors are ordered lexicographically

Page 11: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Timeliness (3) Message passing API:

Callbacks: OnRecv(Edge, Message, Timestamp) OnNotify(Timestamp)

Calls: SendBy(Edge, Message, Timestamp) NotifyAt(Timestamp)

Restrictions / guarantees: Messages are queued and may reorder OnNotify guaranteed to occur after all OnRecv calls (to the

same vertex) with prior times. Works like a barrier. Calls must advance time monotonously

Page 12: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Example codeclass DistinctCount<S,T> : Vertex<T> {

Dictionary<T, Dictionary<S,int>> counts;

void OnRecv(Edge e, S msg, T time) {if (!counts.ContainsKey(time)) {

counts[time] = new Dictionary<S,int>();this.NotifyAt(time);

}if (!counts[time].ContainsKey(msg)) {

counts[time][msg] = 0;this.SendBy(output1, msg, time);

}counts[time][msg]++;

}

void OnNotify(T time) {foreach (var pair in counts[time])

this.SendBy(output2, pair, time);counts.Remove(time);

}{

Low latency processing (sends

upon receive)

Synchronization point

Page 13: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Framework implementation Single thread:

Scheduler keeps track of events Pointstamp = {Location, Timestamp}. Initialized per epoch and per input vertice

Per each active (pending) Pointstamp, maintain: Occurrence count (number of events associated with it) Precursor count (number of “could-result-in“ events) “Frontier”: set of pointstamps with precursor=0. Ok to Notify.

Distributed updates protocol the “agree” on global state

epoch

Page 14: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Logical graph deployment with parallel physical nodes,

graph is cloned

Each node passes messages locally by default

Framework supports partitioning functions for routing

Allows hash/key partitioning, map/reduce logic or group-by operations

"Could-result-in” relations are calculated over the logical graph (less effective, but simplifies scaling)

Page 15: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Optimizations

Checkpointing (per vertice) for fault tolerance Disable windows TCP delays (Nagle’s algorithm)

Reduce backoff times in concurrency conflicts Reduce GC freq

Page 16: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Results: micro benchmarks

Page 17: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Results: Real-world apps

Page 18: Naiad Timely Dataflow System Presented by Leeor Peled Advanced Topics in Computer Architecture April 2014.

Discussion…

Focus on in-memory workloads Lock contentions and producer/consumer issues Performance comparison done against external

results – fishy… Auto mapping from logical to physical