Kubernetes IntroductionAdvanced Technology Group (ATG) for Open Source & Cloud

August 2016

2

What is Kubernetes?

Κυβερνήτης — Greek: A nautical term meaning “helmsman” or “pilot”

“K8s”

3

Kubernetes“Open Source Container Cluster Manager”

• Google — Architect and creator.

• Borg — Google’s internal cluster management software. Kubernetes – complete rewrite, (in Go).

• Google partnered with Linux Foundation to form: Cloud Native Computing Foundation (CNCF) offered Kubernetes as a seed technology

4

Kubernetes History

2013 2014 2015 2016

Apr 2015Tectonic formed (commercial support)

Apr 2015The Borg Paper is published

Sep 2014Kubernetes announced in Wired magazine

Jun 2014Kubernetes 1st GitHub commit

Mar 2013Docker initial release

Aug 2014CoreOS introduces Flannel networking

Oct 2013CoreOS initial release

2008 …2006

2006Google starts work on “Process Containers”(renamed “cgroups”)

Jan 2008cgroups merged into Linux (2.6.24)

2007

July 2015CNCF Formed, K8s v1.0 released, donated to CNCF

Borg development inside Google

5

Kubernetes Tech Specs

Features

• μService Architecture

• Automatic Workload Placement (efficient)

• Auto Remediating (self healing)

• Horizontal Scaling

• Load Balanced

• Declarative Deployment

• Service Discovery included

• A/B & Canary Deployments (testing)

Surrounding Ecosystem

Docker – the container “engine” on each host. etcd (from CoreOS) – distributed K/V store.

CoreOS – the platform. Flannel – overlay networking.

Hosted Service: Google Container Platform GKE is the abbreviation.

6

Network

Client

μService Programming Model — Cloud Native

proxy

μS

…μS

μS

proxy

μS

…μS

μS

proxy

μS

…μS

μS

proxy

μS

…μS

μS

proxy

μS

…μS

μS

proxy

μS

…μS

μS

(HTTP

) Route / P

roxy

Optional

(nginx)

Pod(container)

Service

“Load Balancer”

7

Kubernetes – Programming Model

• Filesystem – that the program uses.• Persistent – how state is saved beyond run-time.• Persistent Volumes are attached and live outside of the

K8s cluster.

Volumes & Persistent Volumes

Pod• One (or more) containers “grouped”• Network (IP address): shared• Volumes: shared

Service• Common API (behavior) replicated across the cluster.• Well Known Endpoint – a consistent IP address,

regardless of changes in specific Pods underneath.

Service

proxy

Host (“node” in K8s)

Pod – different μS

Pod

Container(s)

proxy

Host (“node” in K8s)

Pod

Container(s)

Volume,external

to K8s

Abstract

(Common IP)

8

Kubernetes – Framework Architecture

Client

ControlPlane

Workload

*https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/architecture.md

9

Kubernetes – Framework Architecture

• K8s is extensible• Storage Plugin(s)

- NFS / iSCSI- AWS-EC2 / Google GCE- Ceph (RBD/CephFS) / Gluster- Cinder (OpenStack)

• Other Extension Points- Logging- Access & Auth- Scheduler

Control Plane Worker Node(s) Client

Extension Points

kubelet: local, control plane agent. Pod management using docker-engine.

kube-proxy: internal service routing (i.e. TCP/UDP stream forwarding)

docker-engine: container execution

kube-apiserver: Client’s API access point. Routes requests to appropriate, internal components.

kube-controller-manager: Embeds the core control loops.

• Replication controller• Endpoints controller (proxies)• Namespace controller

kube-scheduler: Workload (Pod) placement. Sophisticated, configurable, globally aware.

etcd (from CoreOS): Distributed, watchable storage The k8s system state

kubectl: CLI into K8s

HTTP — RESTful protocol.

Kubernetes – Deployment ModelA Declarative Model

10

Manifest File(s)

Labels

PodSpec clause – within most descriptors

Replication Controller descriptor

• Optional only in trivial cases. • (trivial = CLI only possible)

• YAML (or JSON) format.

• Key/Value “tags” – placed on any deployable object.

• Selectable – by actions and other declarations.• Configuration Flexibility

• Labeled• allows versioning • other constraint application

• Container(s)• very Dockerfile / docker-compose like.• Image location, (including image version)• Volume requirements• Ports exposed

• “template/spec” clause declares PodSpec configuration.• “replica” clause declares sizing of the service.• Rolling-updates & canary deploys are a supported

pattern.

Descriptor Types (partial list)

• Replication Controller• Deployment

• Pod• Job

• Service

11

Running a Kubernetes Cluster

“There’s more than one way to do it”

– Larry Wall

12

Kubernetes in Public Cloud

Hosted Solution — Google Cloud Platform

Google Container Engine (GKE)

• Kubernetes Getting Started Guide “101”• Hello World Walkthrough

https://cloud.google.com/container-engine/

http://kubernetes.io/docs/hellonode/

Turn-key Solutions

Amazon Web Services (AWS) EC2 http://kubernetes.io/docs/getting-started-guides/aws/

Azure http://kubernetes.io/docs/getting-started-guides/azure/

Free Trial —60 days

$300 credit

13

Kubernetes Run Locally

On a Laptop / Desktop

Minikube• K8s recommended method for single node deploy http://kubernetes.io/docs/getting-started-guides/minikube/

Vagrant — superseded by Minikube, still usable. http://kubernetes.io/docs/getting-started-guides/vagrant/

kube-up.sh — another previous “#1” method by k8s http://containertutorials.com/get_started_kubernetes/index.html

Easy Kubernetes Cluster for macOS• Recently discovered and recommended by our team (ATG). https://github.com/TheNewNormal/kube-cluster-osx

Multi-host / LabCoreOS w/ Fleet • https://github.com/CaptTofu/kubernetes-cluster-fleet

• https://github.com/coreos/coreos-vagrant• https://github.com/mhamrah/kubernetes-coreos-units

14

A Kubernetes Application

15

Kubernetes Application – minimalist application –

1. Construct • Create a standard Docker application, a μService.• Package it as a Docker Image.

2. Deploy • Deploy the Docker Image to a Docker Repository.

3. Run • kubectl run … --image=<Image-Repository-Path>

16

K8s App — Constructapp.py*

from flask import Flaskapp = Flask(__name__)

@app.route('/')def hello_world(): return '-- Hello Flask Dockerized --\n'

if __name__ == '__main__': app.run(debug=True, host='0.0.0.0')

Dockerfile*

FROM ubuntu:latestRUN apt-get update -yRUN apt-get install -y python-pip python-dev build-essentialCOPY . /aptWORKDIR /aptRUN pip install -r requirements.txtENTRYPOINT ["python"]CMD ["app.py"]

*https://github.com/egustafson/ex-py-docker-flask

Build

Run

Verify (in a separate console)

# docker build –t ex-py-docker-flask . ... ...<many lines of output> ...Successfully built 0fb21b16f3dd#

# docker run –p 5000:5000 ex-py-docker-flask * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger pin code: 236-035-556

# curl http://localhost:5000-- Hello Flask Dockerized –-#

run outside localhost(default port: 5000)

17

K8s App — Deploy

Hosted K8s – Google Container EngineLocal “laptop” – Minikube ... (from the construct stage … mostly) ...

# docker build –t gcr.io/<my-proj-id>/ex-py-flask:v1 . ...# gcloud docker push gcr.io/<my-proj-id>/ex-py-flask:v1

# minikube startStarting local Kubernetes cluster...Kubernetes is available at https://192.168.99.100:8443.Kubectl is now configured to use the cluster.# eval $(minikube docker-env)# docker build –t library/ex-py-docker-flask .

Caveat: the method used above is a bit of a “hack”. Using the ‘docker-env’ combined with ‘docker build’ works because Minikube only deploys into a single host. As a consequence the Docker image will be available in the local Docker repository. If Minikube ran across two or more hosts then the node Kubernetes choses to run the Pod (container) on may not match where it was built.

*http://kubernetes.io/docs/hellonode/

GCR Convention

(alternate)

18

K8s App — Run

Hosted K8s – Google Container Engine Local “laptop” – Minikube

# kubectl run flask-node \ -–image=gcr.io/<my-proj-id>/ex-py-flask:v1 \ --port=5000Deployment “flask-node” created# kubectl get podsNAME READY STATUS RESTARTS AGEflask-node-714049816-ztzrb 1/1 Running 0 6m# kubectl expose deployment flask-node -–type=“LoadBalancer”# kubectl get services flask-nodeNAME CLUSTER_IP EXTERNAL_IP PORT(S) AGEhello-node 10.3.246.12 23.251.159.72 5000/TCP 2m

Run

Verify

Run

Verify

# curl http://23.251.159.72:5000-- Hello Flask Dockerized –#

1.

2.

3.

4.

# kubectl run flask-node \ -–image=library/ex-py-docker-flask \ --port=5000Deployment “flask-node” created# kubectl get podsNAME READY STATUS RESTARTS AGEflask-node-714049816-ztzrb 1/1 Running 0 6m# kubectl expose deployment flask-node -–type=“NodePort”

1.

2.

3.

# minikube service flask-node –-urlhttp://192.168.99.100:31992# curl $(minikube service flask-node –-url)-- Hello Flask Dockerized –#

19

Getting Involved

Community http://kubernetes.io/community/

GitHub http://github.com/kubernetes

Project Page & Documents http://kubernetes.io

Slack (chat) (sign-up: http://slack.k8s.io/) https://kubernetes.slack.com

Special Interest Groups (SIGs) (+20 topics)

Community Page SIGs (https://github.com/kubernetes/community/blob/master/README.md#special-interest-groups-sig)

20

Demo

https://github.com/egustafson/ex-gke-webdrop https://github.com/egustafson/webdrop-py

21

Thank youAdvanced Technology Group for Open Source and Cloud

Eric Gustafson gustafson@hpe.comPatrick Galbraith patg@hpe.comClare Springer clarissa.springer@hpe.com

22

Backup Slides(Kubernetes Introduction)

Advanced Technology Group for Open Source & CloudHPE's Advanced Technology Group for Open Source & Cloud embraces a vision that is two steps ahead of today's solutions. We use this vision to drive product adoption and incubate technologies to advance HPE. Through open source initiatives we foster collaboration across HPE and beyond.

23

Patrick Galbraithpatg@hpe.comhttp://patg.net/

Interests: Kubernetes, Ansible, MySQL projects

New Hampshire, USA

Eric Gustafsongustafson@hpe.comhttp://egustafson.github.io/

Interests: Monitoring, Networking, Embedded/IoT

Colorado, USA

Brian Aker, Fellow

Yazz Atlas, Principle Engineer

Hillary Cirimele, Executive Assistant

Matt Farina, Principle Engineer

Patrick Galbraith, Principle Engineer

Eric Gustafson, Principle Engineer

Clare Springer, Program Manager

24

References – Kubernetes Introduction

• “Large-scale cluster management at Google with Borg”• https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43438.pdf

• “Omega: flexible, scalable schedulers for large compute clusters”• https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41684.pdf

• “Borg, Omega, and Kubernetes”• https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44843.pdf

• “Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network”• http://conferences.sigcomm.org/sigcomm/2015/pdf/papers/p183.pdf