Vibrant Ecosystems: Adoption in Private or Partner Developer ProgramsKay Lummitsch, API Technology Evangelist - Swisscom !Steve Rowland, SVP WW Sales & Operations - Apigee !

Building predictive apps with �λ & μ-services architecture

Alan Ho !

@karlunhoAlan Ho

Context is King Apps

4

Technology Choices Today

5

Datawarehouse RDBMS J2EE AppServer

What is Context ?

6

Historical !Events!

Recent!Events! Now!

Lambda Architecture

7

Batch Storage

Near RT Storage

Batch Processing

(MR)

Near RealTime

Processing

Query DB

Batch Layer! Serving Layer!

Speed Layer!

APIs

Direct Mail

Email

Web

Mobile

Outreach

Req / Res Processing

Historical Events

Recent Events

Netflix Example

8

•  Offline = Batch Layer

•  Nearline = Speed Layer

•  Online = Serving Layer

Batch Layer (Offline)

•  Singular Value Decomposition

•  Batch like operations

•  High algorithm complexity

•  Results Flow into Cassandra

Speed Layer (Near Line)

11

•  Medium request/response!

•  Medium data!

•  Medium algorithm complexity!

•  Intermediate caching, incremental learning algos!

Serving layer (Online)•  Fast request/response,

performance is key – high availability/SLA

•  Fallback mechanism to Nearline/Offline operations

•  Low algorithm complexity

Final Thoughts:!!•  Very complete system!

•  Very custom and high performance!

•  Lots of specialized systems for specialized tasks!

•  Not practical for most enterprises!

Data Scientists & Developers

14

Direct Mail

Email

Web

Mobile

Outreach

How can they work independently ?

App Explosion

15

How can I embed predictive in all apps ?

Micro-services

16

Micro-services Best Practice

17

Isolation REST API

HTTP://<Resource>/

Isolation at all tiers

18

Batch Storage

Near RT Storage

Batch Processing

(MR)

Near RealTime

Processing

Query DB

Serving Layer!

APIs

Direct Mail

Email

Web

Mobile

Outreach

Req / Res Processing

Historical Events

Recent Events

Batch Processing

(MR)

Near RealTime

Processing

Req / Res Processing

Batch Processing

(MR)

Near RealTime

Processing

Req / Res Processing

Batch Processing

(MR)

Near RealTime

Processing

Req / Res Processing

Batch Storage

Near RT Storage

Query DB

Batch Storage

Near RT Storage

Query DB

API at all tiers

19

Batch Layer! Serving Layer!

Speed Layer!

Direct Mail

Email

Web

Mobile

Outreach

/Scores!

/Counts!

/Recommendations!

Historical Events

Recent Events

Insights Deep Dive

Recommendations Example

21

Recommendation Architecture

22

Hadoop

In-Memory

Insights (GRASP)

Node.js

Node.js API BaaS

(APIs)

Direct Mail

Email

Web

Mobile

Outreach

(Batch Scores)

(Counters)

(Query)

Historical Events

Realtime Events

(Context)

Backend

(Mashup)

3 4

5

1

2 6

Node.js

Batch Layer

•  Uses R for building predictive models

•  Algorithms based on Event Sequences

•  Lots and lots of tooling

23

HadoopInsights(GRASP)

API BaaS

(Batch Scores)1

Apigee UI

24

R Code!

25

Scored Export to Serving Layer!

Speed Layer

•  Easy to program via node.js

•  Low algorithm complexity – Storm is overkill for most Apigee customers

•  Simple aggregates via C* counters good enough

•  Good for capturing locations, activity stream, etc.

26

Node.js

API BaaS

Apigee UI

BaaS Node.js Counter Example

var event = new Usergrid.Counter(event_options, function (error, result) { … });

// Then we POST the event to increment the countervar counter_options = { name: productName + ’.add_to_cart', value: 1 }event.increment (counter_options, function (error, result)

27

curl -X GET https://api.usergrid.com/my-org/my-app/counters?counter=appleIpad3.add_to_cart

Input code:

Output code:!

Microservices Layer

•  API Layer (Swagger)

•  Context Injection

•  Service Orchestration

•  Fallback logic

28

Node.js Node.js API BaaS

Direct Mail

Email

Web

Mobile

Outreach

(Context)

Backend

Swagger Based API Design

29

Node.js Code

30

APIs are Everywhere

31

Hadoop

In-Memory

Insights

Node.js Node.jsAPI BaaS

Direct Mail

Email

Web

Mobile

Outreach

HistoricalEvents

RealtimeEvents

Backend

Hive

RHadoop

Node.js

Storm

Spark

Thank you

32

LUNCH��

General Hall / Main Concourse�(Sessions resume at 14:00)

Model-Driven APIs with 127Greg Brail !

Overview•  A bit about API design •  Some ways to turn APIs into code •  A different approach •  Introducing Apigee 127

35

API Design

36

REST

37

API Design is Important•  We have been talking API

design for a long time•  We have an approach that we

like and often recommend•  We believe that it is important to

design APIs carefully, up front

My Perspective on API Design•  URIs are important•  Verbs should be used in a consistent way•  Follow common usage•  Links can be helpful

39

40

Turning Designs into APIs

41

An API Design Story•  I designed a lovely, consistent API

–  In a Google Doc •  The team implemented it

–  In Java •  I verified that the implementation matched the doc

–  Using my eyeballs •  We wrote docs

–  Using information from both places

There must be a better way…

43

Modern Web / API Frameworks•  Write code •  Get an API up quickly •  API design and code are the same

•  Examples: –  Express for Node.js –  Ruby on Rails –  Lots of others

44

Another approach: Code First

45

@Path("/my-resource") @Api(value="/my-resource”, description="Rest api for do operations on admin", produces=MediaType.APPLICATION_JSON) @Produces({ MediaType.APPLICATION_JSON }) class MyResource{ @ApiOperation(value = "Get specific element", httpMethod = "GET", notes = "Fetch the selement of the collection", response = Response.class) @ApiResponses(value = { @ApiResponse(code = 200, message = ”Element found"), @ApiResponse(code = 404, message = “Element not found"),

@ApiResponse(code = 500, message = “Server error due to encoding"), @ApiResponse(code = 400, message = "Bad request: decoding error"), @ApiResponse(code = 412, message = ”Prereq: Required data not found")

}) public Response get(

@ApiParam(value = "UUID of the element", required = true) @PathParam("uuid") String uuid) {

Generate

A New Approach

46

Why Can’t We Do This?1.  Design the API2.  Annotate it with policies3.  Wire it to some back end4.  Run it

Swagger 2.0•  Swagger 2.0•  Describes the shape of the API

–  URIs–  Verbs–  Query parameters–  Headers–  Request and response formats

48

Building a Model in Swagger 2.0•  Specify API design using YAML

–  URI Paths–  Verbs–  Parameters–  Request format (optional)–  Response format (optional)–  Error responses (optional)

49

Plugging in Code•  Point each API operation to some code

–  In our project it is a Node.js module

50

Annotating the Model•  The API definition can have “policies”

–  OAuth–  Quota–  Spike arresting–  Caching

•  Each operation can have parameters–  Quota–  OAuth scope–  Etc.

51

Introducing Apigee 127

52

Introduction to 127

$ npm install -g apigee-127

http://github.com/apigee-127

Enterprise APIs, FastWe Handle These:•  Swagger Tools

–  Message validation & routing–  Authorization (OAuth 2.0)

•  Volos.js–  Caching (Apigee Swagger 2.0 Extension)–  Quota (Apigee Swagger 2.0 Extension)–  Analytics (Apigee Swagger 2.0 Extension)

Developers Handle This:•  Business logic

54

ControllerController.js

Design-First Approach

55

Add Annotations

Implement Controllers

Add Annotations•  Business Logic•  Policies:

•  Authorization (OAuth)•  Caching•  Quota•  Spike Arrest•  Analytics

Implement Controllers•  Controllers in JavaScript via Node.js•  Future ideas:

•  Declarative pipelines•  Other languages

How it Works

56

•  No code generation•  Model is built at start-up time•  A127 middleware applied as defined in Swagger:

–  Caching, Quota, OAuth, Spike Arrest, Analytics, etc.

•  Paths and parameters•  Model Definitions•  Vendor Extensions

(middleware)

Interprets Swagger

Execution Options

57

Anywhere / PaaS

127 On-Premises 127 On Edge

127

Anywhere / PaaS

127 On-Premises + Edge

127 127

Thank you

APIs and Internet of ThingsEd Anuff

VP Product StrategyApigee

John Calagaz,

Chief Technology OfficerCentralite

IoT is more than sensors, beacons, Arduinos, and

Raspberry Pi’s…

60

App to Device Connection - Smartphone & Tire Sensor

Device To Device Connection - Tire Sensor & Air Compressor

62

AIR$1

The “I” in IoT is for Internet

63

AIR$1

Why does this matter?

64

IoT Seeks Network Value

Metcalfe’s LawValue  =  O(n2)  

Why is this hard?

66

Building The Network Via The Ecosystem

67

The number of connected devices is set to dramatically increase

68

Too Many Devices, Too Few Developers•  Connected Devices •  Software Developers

• 2013: 6 Billion• 2018: 20 Billion• 333% Increase

Source: Business Insider Intelligence - The Internet Of Everything: 2014

• 2013: 18 Million• 2019: 26 Million• 45% Increase

Source: Evans Data Corporation - Global Developer Population and Demographic Study 2013

MISSING:�34 Million Software Developers

70

This is a scale problem

71

IoT systems are a complex combination of web, app and embedded systems.�

73

Things Language

Internet Languages

What happens when development is so

fragmented?

74

The Promise: Everything is Connected

Experiences become fragmented…

76

The Reality: Unnerving

Today:

78

• Multiple Experiences• Multiple Apps• Multiple Developers• Multiple Ecosystems & Platforms

• Multiple Devices

This is a scale problem

79

Things Go from Bad

app app app app

To Worse

app app app appapp app app

727272

app

cloud

How are we going to fix this?

82

A common platform?

83

The usual suspects…(And countless others, I just got lazy building this slide…)

NO!

85

The platforms can’t save us…

(Again, but your own favorite IoT, OS, or mobile platform here, still wont save you…)

If only there were something like the web, but for apps

and devices…

87

…so that developers could work together to make

apps, devices, and systems interoperate…

88

Partner Integration

Web site

Wearable Electronics

Gaming Consoles

Smart Appliances

Smart Sensors

Smart phones

Connected Cars

Tablets

KiosksAPI

API

Your Digital Assets

API

API

APIs to the rescue!

Opportunity: Internet of Things ecosystems

90

app app app app app app app

Powered by APIs

Digital Experiences

Business Ecosystems

A Day in the Life�of an Embedded IoT Architect

Centralite: Who are we?

92

The Problem

5-24 months

5-24 months

As fast as I can!

Q & A

Thank you

Afternoon Break��

General Hall / Main Concourse�(Sessions resume at 16:00)

Management API’sMichel BurgerApigee

Herve PouponOrange

Peter DunmoreTMF

1.  From Best Effort to Operation Excellence

109 2015 Apigee Confidential

2.  How to quickly build or easily participate to a Digital Service Ecosystem

So you having fun developing solutions using many API’s

110 2015 Apigee Confidential

And this happens….

111 2015 Apigee Confidential

Or this….

112 2015 Apigee Confidential

Or this….

113 2015 Apigee Confidential

And your solution starts to behave badly…

114 2015 Apigee Confidential

Will it be nice to be informed that something is going wrong your the solution before using it?

115 2015 Apigee Confidential

No need to reinvent the wheel

116 2015 Apigee Confidential

•  Fault•  Configuration•  Accounting•  Performance•  Security

•  Simple Management•  Service Level Agreement•  Performance Management

Going from �Best Effort �

to �Operational Excellence

117 2015 Apigee Confidential

Many service providers are building digital service ecosystems leveraging their core assets

118 2015 Apigee Confidential

Identity

Powered by

Your journey in an Digital Service Ecosystem

119 2015 Apigee Confidential

Product Definition

Product Offering

Product Ordering/Activation

Product Usage and Mediation

Billing Invoicing

•  Trouble ticketing•  Catalog Management •  Product Ordering•  DSRA

No need to reinvent the wheel

120 2015 Apigee Confidential

•  Common Service API’s

•  Based on Open Standards

Unifie

d Pr

ofile

Mgm

t

Unifie

d Id

entity

Mgm

t

Cons

isten

t Ope

ratio

n M

gmt

Even

tuall

y Co

nsist

ent

Ope

ratio

n M

gmt

Analy

tics

Com

mer

ce

API Services

Creating Digital Service Ecosystem �by quickly on-boarding�independent services

121 2015 Apigee Confidential

Management infrastructure needs to be simple and easy to use

122 2015 Apigee Confidential

Focusing on API

Management API’s: are they ..

123 2015 Apigee Confidential

Pain killers

Vitamins

vs

Q & A

Thank you

All about that BaaSTo build enterprise apps !!

@karlunhoAlan Ho

Evolving Digital Architecture

128

ERP Product CatalogsCRM Data LakeData

Warehouse

Systems of Engagement

Partner �Apps

Employee �Apps

Consumer �Apps Insights

Systems of Record

Trend 1 : Application Explosion

129

Data De-normalization per interaction

130

ERP Product CatalogsCRM Data LakeData

Warehouse

Trend 2 : Micro-services & Decentralization

131

Micro-services Types

132

Business Logic Services

Data Services

Microservices Best Practice

133

Isolation REST API

HTTP://<Resource>/

Building Business Logic Services

134

Virtualization Light-weight Scripting

+

Building Data Services Approach

135

Virtualization Light-weight Scripting

+ +

NoSQL

Scaling becomes very hard

136

X X

BaaS : Rethinking of Multi-tenancy

137

Sh

are

d

Sch

em

a

Shared Keyspace S

ep

ara

te

Sch

em

a

Separate Keyspace

+ Isolation

- Scalability

- Scalability

+ Isolation

- Management

+ Isolation

+ Management

+ Scalability

+ Isolation

- Management

+ Scalability

+ Management

API BaaS Walkthrough

138

139

http://usergrid.incubator.apache.org

Powered by

Datastore

Performance Mgmt

Push Notifications

Connections /Social

Location

Users

Scalable Persistence

141

SQL JOINS

“These are not the indexes you are looking for…”-Obi-Wan K.

Full-text Indexing

142

Graph Data Persistence: Entities & Connections

143

User A

Product A

User B

User C

Likes>>

<<Liking

API-Driven : REST Semantics

GET PUT POST DELETE

144

Filters / Limits / Cursors

GET http://api.usergrid.com/MyOrg/MyApp/users?ql=select * where name = ’John*'&limit=50&cursor=LTIxNDg0NDUxNDpnR2tBQVFFQWdITUFDWFJ2YlM1emJXbDBhQUNBZFFBUUQyMVZneExfRWVLRlV3TG9Hc1doZXdDQWRRQVFIYVdjb0JwREVlS1VCd0xvR3NWT0JRQQ ’

145

Datastore

Performance Mgmt

Push Notifications

Connections /Social

Location

Users

API BaaS – Users, Roles & Security

147

Datastore

Performance Mgmt

Push Notifications

Connections /Social

Location

Users

Location Queries

GET /stores?ql=location within 500 of 40.042016, -86.900749

Datastore

Performance Mgmt

Push Notifications

Connections /Social

Location

Users

Push Notifications

Datastore

Performance Mgmt

Push Notifications

Connections /Social

Location

Users

Integrated Application Performance Management

153

Errors & Crashes

API Performance

Remote Application Configuration

154

Monitoring Config

App Config

A/B Testing

Demo

155

Thank you

156

Closing Keynote�Great Hall�