Download - Brown bag eventdrivenmicroservices-cqrs

Transcript
Page 1: Brown bag  eventdrivenmicroservices-cqrs

Event driven Microservices (μs) and CQRSVikash Kodati

8/10/2016

Page 2: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential2

AGENDA

4/6/2016

• Why Event driven Microservice (μs)• Overview of event sourcing• Implementing Queries in an event sourced application

Page 3: Brown bag  eventdrivenmicroservices-cqrs

3

• Let’s imagine you are building a large, complex application, e.g., an online store

6/13/2016 T-Mobile Confidential

Page 4: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential4

Monolithic Architecture

6/13/2016

Store front UI Module

WAR

Catalog Module

Reviews Module

Orders Module

Browser

Client App

HTML

REST/JSON

SQL Database

Page 5: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential5

Limitations

6/13/2016

• Monoliths = Trouble

• Very intimidating to change things

• State of art = deploy many times in a day = difficult with Monolith

• The large the app = slower the IDE = slower container

• Obstacle to development and ties to a technology choices at the start of the project.

• RDBMS issues around Scale, Schema updates, OR mismatch • Don’t deal well with unstructured data

Page 6: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential6

Microservice Architecture

6/13/2016

Browser

Mobile Device

Store Front UI

API Gateway

Catalog Service

Review Service

Order Service

….Service

Catalog Database

Review Database

Order Database

….Database

HTML Rest

Rest

Rest

Page 7: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential7

NoSQL db options

6/13/2016

• Avoids the limitations of RDBMS

• Text Search Solr/CloudSearch

• Social (graph) data Neo4J

• Highly distributed and available database Cassandra.

Page 8: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential8

Drawbacks

6/13/2016

• Complexity of developing a distributed system• Implementing inter-process Communication• Handling partial failure

• Complexity of implementing business transactions that span multiple databases (without 2pc)

• Complexity of testing a distributed system• Complexity of deploying and operating a distributed system• Managing the development and deployment of features that span

multiple services

Page 9: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential9

Issues to address

6/13/2016

• How to deploy the services?• How do the services communicate?• How do client of application communicate with the services?• How to partition the system into services?• How to deal with distributed data management problems?

Page 10: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential10

#1: SQL + Text search engine

6/13/2016

Page 11: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential11

#2: Update 2 entities in NoSQL db

6/13/2016

Page 12: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential12

#3: Indexing in Cassandra

6/13/2016

Page 13: Brown bag  eventdrivenmicroservices-cqrs

NOSQL Landscape

6/13/2016 T-Mobile Confidential13

Page 14: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential14 6/13/2016

How to deal with distributed data management problems?

Page 15: Brown bag  eventdrivenmicroservices-cqrs

Solution: Event-Based Architecture

4/6/2016 T-Mobile Confidential15

• Components (e.g., services) publish events when state changes• Components subscribe to events• Maintain eventual consistency across multiple aggregates• Synchronize replicated data

Page 16: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential16

#1: Shared Databases (today’s world)

6/13/2016

Order Service Customer Service ….Service

Tight Coupling

Customer table

Credit Limit

Order table

Order total…..

The DatabaseSimple and

ACID

Page 17: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential17

#2: Database per services

6/13/2016

Order Service

Order table

Order total

Customer table

Credit limit

Customer service

Order Database Customer Database

Page 18: Brown bag  eventdrivenmicroservices-cqrs

18

To maintain consistency a service must atomically publish an event whenever a domain object changes.

2PC (aka. Distributed transactions) is not viable choice for most modern applications. NOSQL DBs do not support it!

6/13/2016 T-Mobile Confidential

Page 19: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential19

How to maintain data consistency without 2PC

6/13/2016

Order service

Place order()

Customer service

updateCreditlimit()

Invariant:Sum(open order.total)<=Customer.creditLimit

Order

Total

CustomerCredit limit

Order management Customer management

Belongs to Has orders

Page 20: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential20

Use event-driven, eventually consistent order processing

6/13/2016

OrderService

Customer Service

Place Order Order Created

Credit Reserved

ORCredit Check Failed

Page 21: Brown bag  eventdrivenmicroservices-cqrs

21

BEFORE: Update State + Publish EventsTwo actions must be atomic

NOW: Persist (and publish) EventsSingle action that is atomic

6/13/2016 T-Mobile Confidential

Page 22: Brown bag  eventdrivenmicroservices-cqrs

Designing Domain Events

6/13/2016 T-Mobile Confidential22

• For each domain object (i.e.,DDD aggregate)• Identify(State changing) domain event• Define Event Classes• Event Enrichment• ID: TimeUUID

• For example,• Shopping Cart:ItemAddedEvent, ItemRemovedEvent,

OrderPlacedEvent• Order:OrderCreated,OrderCancelled,OrderApproved,OrderReje

cted, OrderShipped

Page 23: Brown bag  eventdrivenmicroservices-cqrs

Persists events NOT Current state

6/13/2016 T-Mobile Confidential23

Order

Status

101 Accepted

Order table

Page 24: Brown bag  eventdrivenmicroservices-cqrs

Persists events NOT Current state

6/13/2016 T-Mobile Confidential24

Entity Id Entity Type Event ID Entity Type Event data

101 Order 901 Order Created …..

101 Order 901 Order Approved …..

101 Order 901 Order shipped ….

Page 25: Brown bag  eventdrivenmicroservices-cqrs

Replay events to recreate state

6/13/2016 T-Mobile Confidential25

Order

State

Events

OrderCreated(..)orderAccepted(…)OrderShipped(….)

Periodically snapshot to avoid loading all events…

Page 26: Brown bag  eventdrivenmicroservices-cqrs

Aggregates : Command Events

6/13/2016 T-Mobile Confidential26

Command AggregateEvent

Page 27: Brown bag  eventdrivenmicroservices-cqrs

Request handling in an event sourced application

6/13/2016 T-Mobile Confidential27

HTTPHandler Order

Event Store

PastEvents = findEvents(entityID)

New()

applyEvents(PastEvents)

NewEvenets=processCmd(someCmd)

applyEvents(NewEvents)

saveEvents(NewEvents) (optimistic locking)

Order Service

Page 28: Brown bag  eventdrivenmicroservices-cqrs

Event store publishes events consumed by other services

6/13/2016 T-Mobile Confidential28

Event Store

EventSubscriber

Customer

Subscribe(EventTypes)

Publish(event)

Publish(event)

Update()

Customer Service

Page 29: Brown bag  eventdrivenmicroservices-cqrs

Event store = database + message broker

6/13/2016 T-Mobile Confidential29

Event Store

SaveAggregate Events

GetAggregate Events

Subscribe to Events

• Hybrid database and message broker• Implementations:

• Home Grown/DIY• getEventStore.com by greg Young• http://eventuate.io by Chris Richardson

Page 30: Brown bag  eventdrivenmicroservices-cqrs

Benefits of event Sourcing

6/13/2016 T-Mobile Confidential30

• Solves data consistency issues in a microservice/NoSQL based architecture

• Reliable event publishes events needed by predictive analytics etc, user notifications

• Eliminates O/R mapping problem(mostly)• Reifies state changes:

• Built in, reliable audit log• Temporal queries

• Preserved history More easily implement future requirements

Page 31: Brown bag  eventdrivenmicroservices-cqrs

Drawbacks of event Sourcing

6/13/2016 T-Mobile Confidential31

• Requires application rewrite• Weird and unfamiliar style of programming• Events = a historical record of your bad design decisions• Must detect and ignore duplicate events:

• Idempotent event handlers• Track most recent event and ignore old ones

• Querying the event store can be challenging• Some queries might be complex/inefficient e.g. accounts with balance> x• Event store might only support lookup of events by entity id• Must use command Query Responsibility Segeration(CQRS) to handle queries application

must handle eventually consistent data

Page 32: Brown bag  eventdrivenmicroservices-cqrs

T-Mobile Confidential32

Anatomy of a microservice

6/13/2016

Microservice

Event Store

Messages AdapterHTTP Adapter

HTTP Request Messages Requests

AggregateAggregate

Event AdapterEvents

Events

Cmd

Cmd

Page 33: Brown bag  eventdrivenmicroservices-cqrs

Implementing queries in an event sourced application

4/6/2016 T-Mobile Confidential33

Page 34: Brown bag  eventdrivenmicroservices-cqrs

Find recent, Valuable Customers

6/13/2016 T-Mobile Confidential34

SELECT * FROM CUSTOMER c, ORDER o WHEREc.id = o.ID AND o.ORDER_TOTAL > 100000AND o.STATE = 'SHIPPED‘ AND c.CREATION_DATE > ?

Customer Service Order Service

What if sourcing is used?

Page 35: Brown bag  eventdrivenmicroservices-cqrs

Command query Responsibility Segregation(CQRS)

6/13/2016 T-Mobile Confidential35

Application Logic

Commands Queries

Page 36: Brown bag  eventdrivenmicroservices-cqrs

Command query Responsibility Segregation(CQRS)

6/13/2016 T-Mobile Confidential36

Aggregate

Command Side

MaterializedView

Query Side

Event Store

Commands Queries

Events Events

Page 37: Brown bag  eventdrivenmicroservices-cqrs

Command query Responsibility Segregation(CQRS)

6/13/2016 T-Mobile Confidential37

Page 38: Brown bag  eventdrivenmicroservices-cqrs

Benefits and Drawbacks of CQRS

6/13/2016 T-Mobile Confidential38

Benefits Drawbacks

• Necessary in an event sourced architecture

• Separation of concerns = simpler command and query models

• Support multiple denormalized views

• Improve Scalability and performance

• Complexity• Potential code duplication• Replication lag/eventually

consistent view

Page 39: Brown bag  eventdrivenmicroservices-cqrs

Summary

6/13/2016 T-Mobile Confidential39

• Use microservices to accelerate development• Use an event-driven architecture to maintain data consistency• Implement an event-driven architecture using event sourcing• Use CQRS to implement materialized views for queries