I Love APIs Europe 2015: Developer Sessions
-
Upload
apigee -
Category
Technology
-
view
971 -
download
0
Transcript of I Love APIs Europe 2015: Developer Sessions
Vibrant Ecosystems: Adoption in Private or Partner Developer ProgramsKay Lummitsch, API Technology Evangelist - Swisscom !Steve Rowland, SVP WW Sales & Operations - Apigee !
Building predictive apps with �λ & μ-services architecture
Alan Ho !
@karlunhoAlan Ho
Context is King Apps
4
Technology Choices Today
5
Datawarehouse RDBMS J2EE AppServer
What is Context ?
6
Historical !Events!
Recent!Events! Now!
Lambda Architecture
7
Batch Storage
Near RT Storage
Batch Processing
(MR)
Near RealTime
Processing
Query DB
Batch Layer! Serving Layer!
Speed Layer!
APIs
Direct Mail
Web
Mobile
Outreach
Req / Res Processing
Historical Events
Recent Events
Netflix Example
8
• Offline = Batch Layer
• Nearline = Speed Layer
• Online = Serving Layer
Batch Layer (Offline)
• Singular Value Decomposition
• Batch like operations
• High algorithm complexity
• Results Flow into Cassandra
Speed Layer (Near Line)
11
• Medium request/response!
• Medium data!
• Medium algorithm complexity!
• Intermediate caching, incremental learning algos!
Serving layer (Online)• Fast request/response,
performance is key – high availability/SLA
• Fallback mechanism to Nearline/Offline operations
• Low algorithm complexity
Final Thoughts:!!• Very complete system!
• Very custom and high performance!
• Lots of specialized systems for specialized tasks!
• Not practical for most enterprises!
Data Scientists & Developers
14
Direct Mail
Web
Mobile
Outreach
How can they work independently ?
App Explosion
15
How can I embed predictive in all apps ?
Micro-services
16
Micro-services Best Practice
17
Isolation REST API
HTTP://<Resource>/
Isolation at all tiers
18
Batch Storage
Near RT Storage
Batch Processing
(MR)
Near RealTime
Processing
Query DB
Serving Layer!
APIs
Direct Mail
Web
Mobile
Outreach
Req / Res Processing
Historical Events
Recent Events
Batch Processing
(MR)
Near RealTime
Processing
Req / Res Processing
Batch Processing
(MR)
Near RealTime
Processing
Req / Res Processing
Batch Processing
(MR)
Near RealTime
Processing
Req / Res Processing
Batch Storage
Near RT Storage
Query DB
Batch Storage
Near RT Storage
Query DB
API at all tiers
19
Batch Layer! Serving Layer!
Speed Layer!
Direct Mail
Web
Mobile
Outreach
/Scores!
/Counts!
/Recommendations!
Historical Events
Recent Events
Insights Deep Dive
Recommendations Example
21
Recommendation Architecture
22
Hadoop
In-Memory
Insights (GRASP)
Node.js
Node.js API BaaS
(APIs)
Direct Mail
Web
Mobile
Outreach
(Batch Scores)
(Counters)
(Query)
Historical Events
Realtime Events
(Context)
Backend
(Mashup)
3 4
5
1
2 6
Node.js
Batch Layer
• Uses R for building predictive models
• Algorithms based on Event Sequences
• Lots and lots of tooling
23
HadoopInsights(GRASP)
API BaaS
(Batch Scores)1
Apigee UI
24
R Code!
25
Scored Export to Serving Layer!
Speed Layer
• Easy to program via node.js
• Low algorithm complexity – Storm is overkill for most Apigee customers
• Simple aggregates via C* counters good enough
• Good for capturing locations, activity stream, etc.
26
Node.js
API BaaS
Apigee UI
BaaS Node.js Counter Example
var event = new Usergrid.Counter(event_options, function (error, result) { … });
// Then we POST the event to increment the countervar counter_options = { name: productName + ’.add_to_cart', value: 1 }event.increment (counter_options, function (error, result)
27
curl -X GET https://api.usergrid.com/my-org/my-app/counters?counter=appleIpad3.add_to_cart
Input code:
Output code:!
Microservices Layer
• API Layer (Swagger)
• Context Injection
• Service Orchestration
• Fallback logic
28
Node.js Node.js API BaaS
Direct Mail
Web
Mobile
Outreach
(Context)
Backend
Swagger Based API Design
29
Node.js Code
30
APIs are Everywhere
31
Hadoop
In-Memory
Insights
Node.js Node.jsAPI BaaS
Direct Mail
Web
Mobile
Outreach
HistoricalEvents
RealtimeEvents
Backend
Hive
RHadoop
Node.js
Storm
Spark
Thank you
32
LUNCH��
General Hall / Main Concourse�(Sessions resume at 14:00)
Model-Driven APIs with 127Greg Brail !
Overview• A bit about API design • Some ways to turn APIs into code • A different approach • Introducing Apigee 127
35
API Design
36
REST
37
API Design is Important• We have been talking API
design for a long time• We have an approach that we
like and often recommend• We believe that it is important to
design APIs carefully, up front
My Perspective on API Design• URIs are important• Verbs should be used in a consistent way• Follow common usage• Links can be helpful
39
40
Turning Designs into APIs
41
An API Design Story• I designed a lovely, consistent API
– In a Google Doc • The team implemented it
– In Java • I verified that the implementation matched the doc
– Using my eyeballs • We wrote docs
– Using information from both places
There must be a better way…
43
Modern Web / API Frameworks• Write code • Get an API up quickly • API design and code are the same
• Examples: – Express for Node.js – Ruby on Rails – Lots of others
44
Another approach: Code First
45
@Path("/my-resource") @Api(value="/my-resource”, description="Rest api for do operations on admin", produces=MediaType.APPLICATION_JSON) @Produces({ MediaType.APPLICATION_JSON }) class MyResource{ @ApiOperation(value = "Get specific element", httpMethod = "GET", notes = "Fetch the selement of the collection", response = Response.class) @ApiResponses(value = { @ApiResponse(code = 200, message = ”Element found"), @ApiResponse(code = 404, message = “Element not found"),
@ApiResponse(code = 500, message = “Server error due to encoding"), @ApiResponse(code = 400, message = "Bad request: decoding error"), @ApiResponse(code = 412, message = ”Prereq: Required data not found")
}) public Response get(
@ApiParam(value = "UUID of the element", required = true) @PathParam("uuid") String uuid) {
Generate
A New Approach
46
Why Can’t We Do This?1. Design the API2. Annotate it with policies3. Wire it to some back end4. Run it
Swagger 2.0• Swagger 2.0• Describes the shape of the API
– URIs– Verbs– Query parameters– Headers– Request and response formats
48
Building a Model in Swagger 2.0• Specify API design using YAML
– URI Paths– Verbs– Parameters– Request format (optional)– Response format (optional)– Error responses (optional)
49
Plugging in Code• Point each API operation to some code
– In our project it is a Node.js module
50
Annotating the Model• The API definition can have “policies”
– OAuth– Quota– Spike arresting– Caching
• Each operation can have parameters– Quota– OAuth scope– Etc.
51
Introducing Apigee 127
52
Introduction to 127
$ npm install -g apigee-127
http://github.com/apigee-127
Enterprise APIs, FastWe Handle These:• Swagger Tools
– Message validation & routing– Authorization (OAuth 2.0)
• Volos.js– Caching (Apigee Swagger 2.0 Extension)– Quota (Apigee Swagger 2.0 Extension)– Analytics (Apigee Swagger 2.0 Extension)
Developers Handle This:• Business logic
54
ControllerController.js
Design-First Approach
55
Add Annotations
Implement Controllers
Add Annotations• Business Logic• Policies:
• Authorization (OAuth)• Caching• Quota• Spike Arrest• Analytics
Implement Controllers• Controllers in JavaScript via Node.js• Future ideas:
• Declarative pipelines• Other languages
How it Works
56
• No code generation• Model is built at start-up time• A127 middleware applied as defined in Swagger:
– Caching, Quota, OAuth, Spike Arrest, Analytics, etc.
• Paths and parameters• Model Definitions• Vendor Extensions
(middleware)
Interprets Swagger
Execution Options
57
Anywhere / PaaS
127 On-Premises 127 On Edge
127
Anywhere / PaaS
127 On-Premises + Edge
127 127
Thank you
APIs and Internet of ThingsEd Anuff
VP Product StrategyApigee
John Calagaz,
Chief Technology OfficerCentralite
IoT is more than sensors, beacons, Arduinos, and
Raspberry Pi’s…
60
App to Device Connection - Smartphone & Tire Sensor
Device To Device Connection - Tire Sensor & Air Compressor
62
AIR$1
The “I” in IoT is for Internet
63
AIR$1
Why does this matter?
64
IoT Seeks Network Value
Metcalfe’s LawValue = O(n2)
Why is this hard?
66
Building The Network Via The Ecosystem
67
The number of connected devices is set to dramatically increase
68
Too Many Devices, Too Few Developers• Connected Devices • Software Developers
• 2013: 6 Billion• 2018: 20 Billion• 333% Increase
Source: Business Insider Intelligence - The Internet Of Everything: 2014
• 2013: 18 Million• 2019: 26 Million• 45% Increase
Source: Evans Data Corporation - Global Developer Population and Demographic Study 2013
MISSING:�34 Million Software Developers
70
This is a scale problem
71
IoT systems are a complex combination of web, app and embedded systems.�
73
Things Language
Internet Languages
What happens when development is so
fragmented?
74
The Promise: Everything is Connected
Experiences become fragmented…
76
The Reality: Unnerving
Today:
78
• Multiple Experiences• Multiple Apps• Multiple Developers• Multiple Ecosystems & Platforms
• Multiple Devices
This is a scale problem
79
Things Go from Bad
app app app app
To Worse
app app app appapp app app
727272
app
cloud
How are we going to fix this?
82
A common platform?
83
The usual suspects…(And countless others, I just got lazy building this slide…)
NO!
85
The platforms can’t save us…
(Again, but your own favorite IoT, OS, or mobile platform here, still wont save you…)
If only there were something like the web, but for apps
and devices…
87
…so that developers could work together to make
apps, devices, and systems interoperate…
88
Partner Integration
Web site
Wearable Electronics
Gaming Consoles
Smart Appliances
Smart Sensors
Smart phones
Connected Cars
Tablets
KiosksAPI
API
Your Digital Assets
API
API
APIs to the rescue!
Opportunity: Internet of Things ecosystems
90
app app app app app app app
Powered by APIs
Digital Experiences
Business Ecosystems
A Day in the Life�of an Embedded IoT Architect
Centralite: Who are we?
92
The Problem
5-24 months
5-24 months
As fast as I can!
Q & A
Thank you
Afternoon Break��
General Hall / Main Concourse�(Sessions resume at 16:00)
Management API’sMichel BurgerApigee
Herve PouponOrange
Peter DunmoreTMF
1. From Best Effort to Operation Excellence
109 2015 Apigee Confidential
2. How to quickly build or easily participate to a Digital Service Ecosystem
So you having fun developing solutions using many API’s
110 2015 Apigee Confidential
And this happens….
111 2015 Apigee Confidential
Or this….
112 2015 Apigee Confidential
Or this….
113 2015 Apigee Confidential
And your solution starts to behave badly…
114 2015 Apigee Confidential
Will it be nice to be informed that something is going wrong your the solution before using it?
115 2015 Apigee Confidential
No need to reinvent the wheel
116 2015 Apigee Confidential
• Fault• Configuration• Accounting• Performance• Security
• Simple Management• Service Level Agreement• Performance Management
Going from �Best Effort �
to �Operational Excellence
117 2015 Apigee Confidential
Many service providers are building digital service ecosystems leveraging their core assets
118 2015 Apigee Confidential
Identity
Powered by
Your journey in an Digital Service Ecosystem
119 2015 Apigee Confidential
Product Definition
Product Offering
Product Ordering/Activation
Product Usage and Mediation
Billing Invoicing
• Trouble ticketing• Catalog Management • Product Ordering• DSRA
No need to reinvent the wheel
120 2015 Apigee Confidential
• Common Service API’s
• Based on Open Standards
Unifie
d Pr
ofile
Mgm
t
Unifie
d Id
entity
Mgm
t
Cons
isten
t Ope
ratio
n M
gmt
Even
tuall
y Co
nsist
ent
Ope
ratio
n M
gmt
Analy
tics
Com
mer
ce
API Services
Creating Digital Service Ecosystem �by quickly on-boarding�independent services
121 2015 Apigee Confidential
Management infrastructure needs to be simple and easy to use
122 2015 Apigee Confidential
Focusing on API
Management API’s: are they ..
123 2015 Apigee Confidential
Pain killers
Vitamins
vs
Q & A
Thank you
All about that BaaSTo build enterprise apps !!
@karlunhoAlan Ho
Evolving Digital Architecture
128
ERP Product CatalogsCRM Data LakeData
Warehouse
Systems of Engagement
Partner �Apps
Employee �Apps
Consumer �Apps Insights
Systems of Record
Trend 1 : Application Explosion
129
Data De-normalization per interaction
130
ERP Product CatalogsCRM Data LakeData
Warehouse
Trend 2 : Micro-services & Decentralization
131
Micro-services Types
132
Business Logic Services
Data Services
Microservices Best Practice
133
Isolation REST API
HTTP://<Resource>/
Building Business Logic Services
134
Virtualization Light-weight Scripting
+
Building Data Services Approach
135
Virtualization Light-weight Scripting
+ +
NoSQL
Scaling becomes very hard
136
X X
BaaS : Rethinking of Multi-tenancy
137
Sh
are
d
Sch
em
a
Shared Keyspace S
ep
ara
te
Sch
em
a
Separate Keyspace
+ Isolation
- Scalability
- Scalability
+ Isolation
- Management
+ Isolation
+ Management
+ Scalability
+ Isolation
- Management
+ Scalability
+ Management
API BaaS Walkthrough
138
139
http://usergrid.incubator.apache.org
Powered by
Datastore
Performance Mgmt
Push Notifications
Connections /Social
Location
Users
Scalable Persistence
141
SQL JOINS
“These are not the indexes you are looking for…”-Obi-Wan K.
Full-text Indexing
142
Graph Data Persistence: Entities & Connections
143
User A
Product A
User B
User C
Likes>>
<<Liking
API-Driven : REST Semantics
GET PUT POST DELETE
144
Filters / Limits / Cursors
GET http://api.usergrid.com/MyOrg/MyApp/users?ql=select * where name = ’John*'&limit=50&cursor=LTIxNDg0NDUxNDpnR2tBQVFFQWdITUFDWFJ2YlM1emJXbDBhQUNBZFFBUUQyMVZneExfRWVLRlV3TG9Hc1doZXdDQWRRQVFIYVdjb0JwREVlS1VCd0xvR3NWT0JRQQ ’
145
Datastore
Performance Mgmt
Push Notifications
Connections /Social
Location
Users
API BaaS – Users, Roles & Security
147
Datastore
Performance Mgmt
Push Notifications
Connections /Social
Location
Users
Location Queries
GET /stores?ql=location within 500 of 40.042016, -86.900749
Datastore
Performance Mgmt
Push Notifications
Connections /Social
Location
Users
Push Notifications
Datastore
Performance Mgmt
Push Notifications
Connections /Social
Location
Users
Integrated Application Performance Management
153
Errors & Crashes
API Performance
Remote Application Configuration
154
Monitoring Config
App Config
A/B Testing
Demo
155
Thank you
156
Closing Keynote�Great Hall�