Brown bag eventdrivenmicroservices-cqrs

download Brown bag  eventdrivenmicroservices-cqrs

If you can't read please download the document

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Brown bag eventdrivenmicroservices-cqrs

Frontline Solutions Roadmap

Event driven Microservices (s) and CQRSVikash Kodati8/10/2016

Encourage interactive sessionInformal discussionEat lunchMy Goal is to keep us all on the same page at a conceptual level. Please stop me and ask questions1

AGENDA4/6/2016T-Mobile Confidential2

Why Event driven Microservice (s)Overview of event sourcingImplementing Queries in an event sourced application

Event sourcing and Command Query Responsibility Segregations Great ways to build microservices


Lets imagine you are building a large, complex application, e.g., an online store6/13/2016T-Mobile Confidential3

Monolithic Architecture6/13/2016T-Mobile Confidential4

Store front UI ModuleWARCatalog ModuleReviews ModuleOrders ModuleBrowserClient AppHTMLREST/JSONSQL Database

Imagine our Apps: Monolith

We have backend servicesDeploy: pkg it in tomcat = war fileBackend : RDMS ACIDSimple way of building apps (dev,test, deploy at scale)Any changes applied atomicallyScale out by running multiple copied behind load balancerAdoption new technologies


Limitations6/13/2016T-Mobile Confidential5

Monoliths = Trouble

Very intimidating to change things

State of art = deploy many times in a day = difficult with Monolith

The large the app = slower the IDE = slower container

Obstacle to development and ties to a technology choices at the start of the project.

RDBMS issues around Scale, Schema updates, OR mismatch Dont deal well with unstructured data

X Y Z scalaing

Z = sharding at db level

Y: functional decomp

X; Traditional scale out 5

Microservice Architecture 6/13/2016T-Mobile Confidential6

BrowserMobile DeviceStore Front UIAPI GatewayCatalog ServiceReview ServiceOrder Service.ServiceCatalog DatabaseReview DatabaseOrder Database.DatabaseHTMLRestRestRest

Distinct functional area is now a standalone service with its own databaseHelps with scalabilityWe can use no-sql db


NoSQL db options6/13/2016T-Mobile Confidential7

Avoids the limitations of RDBMS

Text Search Solr/CloudSearch

Social (graph) data Neo4J

Highly distributed and available database Cassandra.

We end up with a polyglot persistence architecture.


Drawbacks6/13/2016T-Mobile Confidential8

Complexity of developing a distributed system Implementing inter-process Communication Handling partial failureComplexity of implementing business transactions that span multiple databases (without 2pc)Complexity of testing a distributed systemComplexity of deploying and operating a distributed systemManaging the development and deployment of features that span multiple services

How do we bridge the gap between the current realities and our new MISSION? Make them granular


Issues to address6/13/2016T-Mobile Confidential9

How to deploy the services?How do the services communicate?How do client of application communicate with the services?How to partition the system into services?How to deal with distributed data management problems?

Distributed data management is the issue to solve. 9

#1: SQL + Text search engine6/13/2016T-Mobile Confidential10

Elastic search in our appUpdate rdbms and how to keep the search in sync?


#2: Update 2 entities in NoSQL db6/13/2016T-Mobile Confidential11

Update two document how do I do that w/o real transactions


#3: Indexing in Cassandra6/13/2016T-Mobile Confidential12

How to keep the index tables of Cassandra in sync with the main table. Without a transactionally consistent manner.

How to keep it consistent w/o 2PC12

NOSQL Landscape6/13/2016T-Mobile Confidential13

Talk about eventual consistencyThis concept is also known as BASE (as opposed to ACID) Basically Available, Soft State, Eventually ConsistentACID (Atomicity,Consistency,Isolation,Durability)13

6/13/2016T-Mobile Confidential14

How to deal with distributed data management problems?

Distributed data management is the issue to solve. 14

Solution: Event-Based Architecture4/6/2016T-Mobile Confidential15

Components (e.g., services) publish events when state changesComponents subscribe to eventsMaintain eventual consistency across multiple aggregatesSynchronize replicated data

Subscribers = Other interested parties = they update their data to reflect the change.Moving from 2 PC to event driven approach.15

#1: Shared Databases (todays world)6/13/2016T-Mobile Confidential16

Order ServiceCustomer Service.ServiceTight Coupling

Customer table

Credit LimitOrder table

Order total..The DatabaseSimple and ACID

Atomicity, Consistency, Isolation, Durability

BASE: Basically available soft state eventually consistent 16

#2: Database per services6/13/2016T-Mobile Confidential17

Order Service

Order table

Order total

Customer table

Credit limitCustomer serviceOrder DatabaseCustomer Database

When looking to split a large application into parts, often management focuses on the technology layer, leading to UI teams, server-side logic teams, and database teams.

We can relate this to:Applications having dev and Op responsibilities under different leadershipsDeloitte team doing the dev all by themselves


To maintain consistency a service must atomically publish an event whenever a domain object changes.

2PC (aka. Distributed transactions) is not viable choice for most modern applications. NOSQL DBs do not support it! 6/13/2016T-Mobile Confidential18

How do we atomically publish and update a datastore?Message broker does not support 2PC18

How to maintain data consistency without 2PC6/13/2016T-Mobile Confidential19

Order service

Place order()

Customer service


Invariant:Sum(open xEvent store might only support lookup of events by entity idMust use command Query Responsibility Segeration(CQRS) to handle queries application must handle eventually consistent data


Anatomy of a microservice6/13/2016T-Mobile Confidential32

MicroserviceEvent StoreMessages AdapterHTTP AdapterHTTP RequestMessages RequestsAggregateAggregateEvent AdapterEventsEventsCmdCmd


Implementing queries in an event sourced application

4/6/2016T-Mobile Confidential33

Again a set of statements.33

Find recent, Valuable Customers6/13/2016T-Mobile Confidential34

SELECT * FROM CUSTOMER c, ORDER o = o.ID AND o.ORDER_TOTAL > 100000AND o.STATE = 'SHIPPED AND c.CREATION_DATE > ?Customer ServiceOrder ServiceWhat if sourcing is used?

Divide into two modules: Commands (updates) and Queries ()34

Command query Responsibility Segregation(CQRS)6/13/2016T-Mobile Confidential35

Application Logic


Has a huge impact on architecture.Monolith has this huge single database containing all the data for all business functions. MS should be responsible for its own data and its peristence. Removes integrated at db levelUse data store that makes sense for the problemYou may never talk to another services datastore.You may only talk to another service via its APIChoice of db tech will be upto the service groupIncluding languages.

Event oriented architecture. What if Service-A needs the data of Service-BHow a service landscape is governed and also in terms of data management


Command query Responsibility Segregation(CQRS)6/13/2016T-Mobile Confidential36

AggregateCommand Side

MaterializedViewQuery Side Event Store


View Store = MaterializedView

MongoNeo4JCassandra or MysqlText Queries = Cloud Search36

Command query Responsibility Segregation(CQRS)6/13/2016T-Mobile Confidential37

View Store = MaterializedView

MongoNeo4JCassandra or MysqlText Queries = Cloud Search37

Benefits and Drawbacks of CQRS6/13/2016T-Mobile Confidential38

BenefitsDrawbacksNecessary in an event sourced architectureSeparation of concerns = simpler command and query modelsSupport multiple denormalized viewsImprove Scalability and performanceComplexityPotential code duplicationReplication lag/eventually consistent view

How do we bridge the gap between the current realities and our new MISSION? Make them granular


Summary6/13/2016T-Mobile Confidential39

Use microservices to accelerate developmentUse an event-driven architecture to maintain data consistencyImplement an event-driven architecture using event sourcingUse CQRS to implement materialized views for queries

How do we bridge the gap between the current realities and our new MISSION? Make them granular