Let us first understand…

3

HY NODEJS?

Little’s Law for Scalability and Fault Tolerance

L = λW

λ - Average request arrival rate W -Average time each request is processed by the system L - Number of concurrent requests handled by the system

In our system, 1000 requests, on average, arrive each second Each takes 0.5 seconds to process

How many requests does our system need to handle concurrently? 1000*0.5 = 500

1. If we know and we can figure out the rate of requests we can support 2. To handle more requests, we need to increase , our capacity, or

decrease , our processing time, or latency

4

HY NODEJS?

Tim

e

WEB CLIENT

SERVER

FOO

BAR

MICROSERVICES

HTTP SERVICE

• FOO and BAR, each take 500ms on average to return a response,

• Processing latency, is 1 second. That’s

our • We’ve allowed the

web server to spawn up to 2000 threads (that’s

now our )

How many requests can our system handle per second ? λ = L/W = 2000/1 = 2000

5

HY NODEJS?

L is a feature of the environment (hardware, OS, etc.) and its limiting factors. It is the minimum of all limits constraining the number of concurrent requests. What Dominates the Capacity (L) ?

A server can support several tens-of-thousands requests

A server can support 100K to over a million concurrent requests

Assuming a request size of 1 MB this can be over several hundreds-of-thousands requests

Again, this can be over several hundreds-of-thousands requests So, L is somewhere between 100K and 1 million requests?

Oh no no… Wait a minute…

It usually has somewhere between 2K and 15K threads. With thread-per-connection model a server can serve < 20K requests

L = MIN(1,2,3,4,5 ) L is completely dominated by the number of threads the OS can support without adding latency

6

HY NODEJS?

Tim

e

WEB CLIENT

SERVER

FOO BAR

MICROSERVICES

HTTP SERVICE

• FOO and BAR, each take 500ms on average to return a response,

• Processing latency, is 1 second. That’s

our • We’ve allowed the

web server to spawn up to 2000 threads (that’s

now our )

If we call FOO and BAR parallely or asynchronously How many requests can our system handle per second ? λ = 2000/500 = 4000 per second

7

HY NODEJS?

• Even with multiple threads Context switching is not free • Execution stacks(threads) take up memory • Cannot use an OS thread for each connection

Problem 1

Problem 2

We do I/O which is or synchronous Typical blocking things:- • Calls out to web services • Reads/Writes on the Database

8

HAT IS NODEJS?

• Server side Javascript runtime

• It has Google Chrome’s V8 under the hood

• Nodejs.org: “Node.js is a platform built on Chrome’s JavaScript runtime for easily building fast network applications. Node.js uses an ,

model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.”

Little’s Law...huh

9

NodeJS under the hood…

Callbacks + Polling

Epoll, POSIX AIO,Kqueue …

Socket,http etc…

Open source JavaScript engine, platform

Optimizer, JIT, inline caching, GC

Cross-platform asychronous I/O.

Event Loop,Worker Thread Pool

10

hy Google Chrome ?

ultithreaded Server | Architecture Summary

Synchronous, blocking I/O = simpler way of performing I/O.

More threads because of the direct association to

connections.

This causes more memory and higher CPU usage due

to more context switching among threads

12

ultithreaded server

inglethreaded Server | Architecture Summary

Event loop (main thread) at the front and asynchronous I/O at the kernel level.

By not directly associating connections and threads,

needs only a main event loop thread and (kernel) threads to perform I/O.

Fewer threads , consequently yield

, it uses less memory and also less CPU.

14

inglethreaded Server

And the war begins…

Case Study

17

THE PAYPAL STORY

• On November 22, 2013 PayPal's engineering team posted on the PayPal engineering blog a post about PayPal's move from Java back end to Node.js back end for its web applications.

• The first web application to get the node treatment was the

• Two teams were building the same application with the exact same functionality one in Java and other in Javascript.

• The application contained three routes and each route made a between 2-5 API requests, orchestrated the data and rendered the page.

18

The PAYPAL STORY

• Double the requests per second vs. the Java application. • 35% decrease in the average response time for the same page. This

resulted in the pages being served 200ms faster— something users will definitely notice.

19

THE BENEFITS

Using JavaScript on both the front-end and the back-end removed an artificial boundary between the browser and server, allowing engineers to code both.

2. Built almost 3. Written in 4. Constructed with

vs. the Java application. time for the

same page

Wohoooooo………

Should I use NodeJS

for everything now?

No…

SCALABILITY

22

NODE.JS SCALES ON A PER PROCESSOR BASIS AS WELL AS ACROSS SERVERS.

Independently scale the subsystems

A JavaEE 3-tier app is actually not written

with Horizontal clustering in mind

23

Case Study - The Playfield

Reference: http://www.techferry.com/eBooks/NodeJS-vs-J2EE-Stack.html Two small applications simulating following use cases were written. One using Spring/Hibernate running on Tomcat with MySQL database at backend and another application using NodeJS with MySQL database at backend. Jmeter has been used to fire similar test load with varying load patterns to both the applications. 1.Use Case - Write a record : A module to insert a record into one single DB table and the backend returns a success/decline json response. 2.Use Case - Read a record : A module to read the same record by passing a query parameter and the backend returns the record as json string.

24

BENCHMARKING – CPU & MEMORY Average CPU and memory usage were very low for NodeJS for serving same load pattern

WRITE A RECORD - User/Loops (100/1000)

READ A RECORD - User/Loops (100/1000)

25

BENCHMARKING – HITS PER SEC(Scales may differ) NodeJs was able to server better hits per second smoothly over time

WRITE A RECORD - User/Loops (100/1000)

26

BENCHMARKING – Response Times vs Threads (Scales may differ)

NodeJs was able to server better hits per second smoothly over time.(Scales below are different)

READ A RECORD - User/Loops (100/1000)

27

Scenario NODE JS JAVA/J2EE

System Resource usage NodeJS consistently uses comparatively very low CPU and Memory because of fewer threads used for processing

J2EE model scales with threads and hence it is CPU and memory hungry. CPU and Memory usage was comparatively very high

Simple URL based read/writes and increasing User Load

As the load and concurrency grow , NodeJS's scalability potential for lightweight operations improves

Since each request executes as thread, thread pool size can not be set infinite so with a given thread pool size requests will have to wait for their turn which work well on moderate load but performance degrades on extremely high Twitter kind of loads.

NodeJS vs JEE - Comparison Summary

28

Scenario NODE JS JAVA/J2EE CPU Intensive operations

Should be very carefully chosen in this scenario.

Should be preferred due to multi threading capabilities that can execute such operations in parallel without blocking other requests.

Independent and Blocking Asynchronous I/O operations. Like mix of DB calls, Web Service calls, File Read/Write etc.

Such operations can be executed in parallel while NodeJS instead of waiting on blocking I/O operations can process more requests

Performance degrades on higher loads. All sub operations in each request will be performed sequentially even though they are independent. So initially it may scale up to some parallel threads or requests after which waiting time will keep most of the threads waiting (blocking resources)

NodeJS vs JEE - Comparison Summary

ECOSYSTEM

30

ECOSYSTEM 1-1

NODEJS (Upcoming) JAVA/J2EE(Mature)

Language Javascript Java

Runtime Node JS JVM

Server/Middleware Express.JS(Async) J2EE compliant Web Server(Non Async)

Module Repository Node Packaged Modules

Mostly Maven

Paas Support Multiple… • Cloudbees • GoGrid • Cloudcat

Full Stack Javascript all across

• No Java only stack

Libraries MODULES Package.json

JARS MANIFEST.MF

31

NODEJS (Upcoming) JAVA/J2EE(Mature)

Unit Testing Frameworks

NodeUnit Mocha(TDD) + Chai(assertion library) Nock(Mock Http Services)

Junit EasyMock

Build Tools • Grunt • node-ant (Apache Ant

adapter )

Ant, Maven, Gradle

Configuration config-js

Mostly XML based

Standardization of Code Structure

KrakenJS Maven/ pattern based

Auto Restart • forever • Supervisor

• Needs to be explicitly scripted

ECOSYSTEM 1-1

32

NODEJS (Upcoming) JAVA/J2EE(Mature)

Web MVC Angular.js Ember.js Backbone.js

• Struts • Spring MVC • JSF

IOC Framework Event loop and handlers yield an IOC

Spring

ORM frameworks Support exists all across, but still nascent

• JPA • Hibernate • EJB Entity Beans

IDE Support • Sublime Text • Eclipse

• Eclipse • IntelliJ

Debugging • Cluster2 – Live Production Debugging

• node-inspector • Eclipse v8 Plugin

• JDT

Profiling • V8-profiler • node -profiler

• JProfiler • JMeter

ECOSYSTEM 1-1

33

ECOSYSTEM 1-1

NODEJS (Upcoming) JAVA/J2EE(Mature)

Logging frameworks Winston Bunyan Log4Js

log4j logback slf4j

Monitoring node-monitor APPDynamics

Dynatrace, JMX console

API Support(Persistance, Cache, MQ/File System)

HTTP File System Socket.io Redis MongoDB MySQL PostgreSQL Memcached Cassandra RabbitMQ Riak Oracle etc.. and counting…

Most backend systems

34

MODULE COUNT

SO WHEN SHOULD I USE NODE?

36

NODE IS GOOD FOR…

CHAT

JSON API on top of an object based DB (such as MongoDB).

QUEUED INPUTS

DATA STREAMING

PROXY

APPLICATION MONITORING DASHBOARD

BROKERAGE - STOCK TRADER’S DASHBOARD

SYSTEM MONITORING DASHBOARD

BUT I HAVE QUESTIONS…

? What about the Callback Challenges?

39

ASYNC and PROMISES

40

Can Node be used for Compute Intensive

tasks?

41

COMPUTE INTENSIVE WITH NODE

*Courtesy: http://neilk.net/blog/2013/04/30/why-you-should-use-nodejs-for-CPU-bound-tasks/

Yohooo!!…Worker pools, Cluster, Web Workers to the rescue..

42

What About Transactions?

43

NODE.JS TRANSACTION/CONNECTION POOLING SUPPORT

Knex.js is a "batteries included" SQL query builder for Postgres, MySQL, MariaDB and SQLite3, designed to be flexible, portable, and fun to use. It features both traditional node style callbacks as well as a promise interface for cleaner async flow control, a stream interface, full featured query and schema builders, transaction support, connection pooling and standardized responses between different query clients and dialects.

Still unconquered

to the rescue…

44

How about Java NIO?

45

WHY JAVA NIO FAILS

Java still has downsides compared to Node.js in that many

core Java libraries are not non-blocking friendly (such as

JDBC).

Async in Node.js is completely different. Everything is

non-blocking, from file reads to database access

46

What about Design Patterns in Javascript?

47

DESIGN PATTERNS EXIST!!!

48

What are WebSockets?

49

WEB SOCKETS

TCP

AND THE GROWTH CONTINUES…

51

BUSINESS BENEFITS

Motivation • Fast Prototyping • Continuous Delivery

Productivity • An Enterprise Scale Web Server In 5 Lines • >80K Modules On NPM (Growing Ecosystem)

Developer Joy More Happy, Works Better, More Developers Join

Cost Savings • Fewer Developers • Smaller Iterations • Less Hardware footprint

52

SUCCESS STORIES

53

INDUSTRY SUPPORT

54

NODE IS GROWING FASTER THAN EVER

55

WRAPPING UP NODE.JS

56

References

• http://blog.paralleluniverse.co/2014/02/04/littles-

law/ • http://www.google.com/googlebooks/chrome/smal

l_16.html • http://blog.cloudfoundry.org/2012/06/27/future-

proofing-your-apps-cloud-foundry-and-node-js/ • http://strongloop.com/node-js/infographic/

Thank You