2 Years of Real World FP at REA

2 Years of Real World FP at REA@KenScambler

Scala Developer at

λ

Me14 years

5 years

5 years

when possible

when bored

when forced

@

Jul 13 Jan 14 Jul 14 Jan 15

- 3 teams- 17 codebases- 43K LOC

Why Functional Programming?

Compelling, tangible software engineering benefits

Modularity

Abstraction

Composability

Modular, abstract, composable programs

are simple programs

Modularity

Can you reason about something in isolation?

Or do you need to fit everything in your head at once?

A

B C K

Is the cost of replacing Bwith K just writing K?

A

B CK

Is the cost of replacing Bwith K just writing K?

AK

B CK

K

…or rewriting half the program?

glue

glue

A pure function is just input output; no side effects

You can tell what it does without Looking at surrounding context.

Consider: Let’s parse a string like “Richmond, VIC 3121”

def parseLocation(str: String): Location = {val parts = str.split(“,”)val secondStr = parts(1)val parts2 = secondStr.split(“ “)Location(

parts(0), parts2(0), parts(1).toInt)}

Could be null

Possible errors: 1





Might not have enough elements

Possible errors: 2




Possible errors: 3




Possible errors: 4




Possible errors: 5



Might not be an int

Possible errors: 6

def doSomethingElse(): Unit = {// ...Do other stuffparseLocation(“Melbourne, VIC 3000”)

}

Possible errors: 6 + 3 = 9

63

def anotherThing(): Unit = {// ...Do even more stuffdoSomethingElse()

}

Possible errors: 9 + 4 = 13

49

Code inherits all the errors and side-effects of code it calls.

Local reasoning becomes impossible; modularity is lost.

def parseLocation(str: String): Option[Location] = {

val parts = str.split(”,")for {

locality <- parts.optGet(0)theRestStr <- parts.optGet(1)theRest = theRestStr.split(" ")subdivision <- theRest.optGet(0)postcodeStr <- theRest.optGet(1)postcode <- postcodeStr.optToInt

} yieldLocation(locality, subdivision, postcode)

}

Possible errors: 0

All possibilities have been elevated into the type system

Local reasoning is possible!

Local reasoning is possible about things that call it, etc…

Abstraction

Know as little as you need

Know as little as you need

Produce as little as you can

def sumInts(list: List[Int]): Int = {

var result = 0for (x <- list) {

result = result + x}return result

}

def flatten[A](list: List[List[A]]): List[A] = {

var result = List()for (x <- list) {

result = result ++ x}return result

}

def all[A](list: List[Boolean]): Boolean = {

var result = truefor (x <- list) {

result = result && x}return result

}

def sumInts(list: List[Int]): Int = {

var result = 0for (x <- list) {

result = result + x}return result

}

def flatten[A](list: List[List[A]]): List[A] = {

var result = List()for (x <- list) {

result = result ++ x}return result

}

def all[A](list: List[Boolean]): Boolean = {

var result = truefor (x <- list) {

result = result && x}return result

}

trait Monoid[M] {def zero: Mdef append(m1: M, m2: M): M

}

Extract the essence!

def fold[M](list: List[M])(implicit m: Monoid[M]): M

= {

var result = m.zerofor (x <- list) {

result = m.append(result,x)}result

}

def fold[M](list: List[M])(implicit m: Monoid[M]): M

= list.foldLeft(m.zero)(m.append)

fold(List(1, 2, 3, 4)) 10

fold(List(List("a", "b"), List("c"))) List("a", "b", "c")

fold(List(true, true, false, true)) false

Abstraction is always a good thing!

Less repetition More reuse

Less decay, because code can’tgrow tumours around unnecessary detail

Composability

Functions compose.

A => B B => C

C => D D => E

A => E

Functions compose.

Sideways too.

A => E

X => Y

(A,X) => (E,Y)

Sideways too.

This works in the large, as well as the small!

Entire systems can be composable like functions…. without side effects

Truly composablesystems can accrue more and more stuff without getting more complex!

Simplicity

• Modularity – Reason locally

• Abstraction – say only what you need, hide

everything you don’t

• Composability – scale without accruing complexity

The human process

Technical excellence isn’t enough!

Software development is a human process

Coffee Tea

Quiet time

GSD

PoniesReconsider!

Xi’an offshore team

Team

Team

Team

Team

Xi’an offshore team

• Many teams are partly based in Xi’an.

• They’re very good, but…

• Communication is hard!

• It works, but requires great investment of time and

money

Bottom-up tech decisions

GET /foo/bar

PUT

{"mode":

"banana"}

POST {"partyTime": "5:00"}GET /buzz/5/

ArchitectMountain

ArchitectMountain

Just needs more Agile

Don’t forget your

velocity

More meetings, but littler

NONo no no no no no no

no no no no no no no



no no no no not like

this. Wake up it’s a

school day

Bottom-up tech decisions

You have to win

Software Paleontology

Everyone’s got a history…

Scriptozoic era1995 – 2010Mostly Perl

Monolithocene epoch2010 – 2012Ruby, Java


AWS

Monolithocene epoch2010 – 2012Ruby, Java


Microservices2012 –Ruby, Scala, JS

Adoption

June, 2013

λ

λ

λ

λ

λ

λ

λ

Language choiceObject Oriented

Powerful static types

Functional

JVM

Object Oriented

Powerful static types

Functional

JVM

Functional

JVM

Whatever works for you!

The journey

Jul 13 Jan 14

1

LOC

Web

JSON

DB

Dep Inj

Play2

Play2

6K

Squeryl / Play2

Constructors

Type API#1

LOC

Web

JSON

DB

Dep Inj

Play2

Play2

6K

Squeryl / Play2

Constructors

Type API

• Mentor• Code reviews

Learning Investment

#1

LOC

Web

JSON

DB

Dep Inj

Play2

Play2

6K

Squeryl / Play2

Constructors

Type API

• Mentor• Code reviews

Learning Investment Report cardLearning curve

Technical result

Productivity

Sentiment

Steep but ok

OK; slight dip

Great; but FWs too heavy

#1

Jul 13 Jan 14

1 2

Some infrastructure

LOC

Web

JSON

DB

Dep Inj

Play2

Argonaut

3K

Squeryl / Play2

Constructors

Type API#2

LOC

Web

JSON

DB

Dep Inj

Play2

Argonaut

3K

Squeryl / Play2

Constructors

Type API

• Almost noneLearning Investment

#2

LOC

Web

JSON

DB

Dep Inj

Play2

Argonaut

3K

Squeryl / Play2

Constructors

Type API

• Almost noneLearning Investment Report card

Learning curve

Technical result

Productivity

Sentiment

Learning?

Meh

Needed rework; OK in the end

#2

!!!

Lesson #1

New tech, mindset requires investment in learning

Invest in your people!

Another team!

λ

“We’ll have some of that!”

λ

Jul 13 Jan 14

1 2

46

5

New team; new ideas!

3

LOC

Web

JSON

DB

Dep Inj

Unfinagled

Argonaut

2K, 3K, 4K, 1K

Slick

Constructors

Type Web app, libs, API x 2

• 2 x MentorLearning Investment Report card

Learning curve

Technical result

Productivity

Sentiment

Not bad

Great

Great

#3,4,5,6

Jul 13 Jan 14

1 2

46

5

3

7

Theft & innovation

Jul 14

Lesson #2

Having multiple teams is great, because you can steal from each other

LOC

Web

JSON

DB

Dep Inj

Unfinagled

Argonaut

4K

Slick

Monad Transformers

Type API#7

LOC

Web

JSON

DB

Dep Inj

Unfinagled

Argonaut

4K

Slick

Monad Transformers

Type API

• 2 x MentorLearning Investment

#7

LOC

Web

JSON

DB

Dep Inj

Unfinagled

Argonaut

4K

Slick

Monad Transformers

Type API

• 2 x MentorLearning Investment Report card

Learning curve

Technical result

Productivity

Sentiment

Vertical

Great

Great

#7

Monad Transformers

Good technical benefits, but…

Only 2 people could understand the code

Experienced engineers felt totally helpless

Learning curve way too steep

Devops

All-rounders

Gurus

All-rounders

Gurus

JS / CSS / HTML

AWS

All-rounders

Gurus

Scala (originally!)

Gurus

All-rounders

Scala (now)

Gurus

• Smooth learning curve is utterly essential

• We need more all-rounders

• We can’t leave people behind

Lesson #3Familiar, but technically unsound concepts have limited value.

However… if we can’t make a concept learnable, then we can’t use it.

λ

A Ruby team considers its options…

Jul 13 Jan 14

1 2

46

5

3

7

A 3rd team dips its toe in the water

8

Jul 14

LOC

Web

JSON

DB

Dep Inj

Play2

Play2

2K

Anorm

Constructors

Type Web app#8

LOC

Web

JSON

DB

Dep Inj

Play2

Play2

2K

Anorm

Constructors

Type Web app

• Trial and error• Code Katas

Learning Investment

#8

LOC

Web

JSON

DB

Dep Inj

Play2

Play2

2K

Anorm

Constructors

Type Web app

• Trial and error• Code Katas


Technical result

Productivity

Sentiment

Steep

Meh

OK

#8

Lesson #4

It’s really hard learning from scratch

Be prepared for pain up front

Jul 13 Jan 14

1 2

46

5

3

7

Latest iteration

8

Jul 14 Jan 15

17

Design trends

Inheritance/mixins Static functions

Design trends


Partial functions Total functions

Design trends



Exceptions Sum types

Design trends



Strings/primitives Wrapper types

Exceptions Sum types

FP Guild

Every Thursday, in work hours

7 – 12 people each week

Reading, exercises, talks, live coding

LOC

Web

JSON

DB

Dep Inj

Unfiltered

Argonaut

3K

Slick

Free Monads

Type API#17

LOC

Web

JSON

DB

Dep Inj

Unfiltered

Argonaut

3K

Slick

Free Monads

Type API

• 2 x Mentors• Pull Requests• Code reviews• Pairing• FP Guild

Learning Investment

#17

LOC

Web

JSON

DB

Dep Inj

Unfiltered

Argonaut

3K

Slick

Free Monads

Type API

• 2 x Mentors• Pull Requests• Code reviews• Pairing• FP Guild


Technical result

Productivity

Sentiment

Smooth

Great

Brilliant

#17

Pure coreRoutes Controllers Logic Script

Interpreter

Actual DB

“Authenticate”“Use config”“Get from DB”“Update DB”“Log result”

Web Framework

Server

App runtime

Pure coreRoutes Controllers Logic Script

Pure Interpreter

Pure tests

Input

Output

Assert

“Authenticate”“Use config”“Get from DB”“Update DB”“Log result”

Pure core

Interpreter

Actual DB

Web Framework

Server

App runtimeWafer thinE2E tests

Input

Output

Assert

object SearchController {

def getList(uid: UserId): Script[Response]= {

for {searches <- getSearches(uid)cfg <- getConfigresults <- addLinks(searches, cfg)

} yield Ok ~> Response(result.toJson)}

}

Learning curve:

Learning curve:

Smooooth

Flaky pretend servers

just to check

a response code

Flaky tests

that take

30 min to run

Side effects

everywhere

Exceptions

Intrusive

DI frameworks

You shouldn’t be dealing with all that complexity and crap!

Most business software isn’t rocket science.

There is a better way

FP is a tall tree

But there is so much low hanging fruit!

Thank you!

2 Years of Real World FP at REA

Software

Transcript of 2 Years of Real World FP at REA