Post on 16-Jul-2015
2 Years of Real World FP at REA@KenScambler
Scala Developer at
λ
Me14 years
5 years
5 years
when possible
when bored
when forced
@
Jul 13 Jan 14 Jul 14 Jan 15
- 3 teams- 17 codebases- 43K LOC
Why Functional Programming?
Compelling, tangible software engineering benefits
Modularity
Abstraction
Composability
Modular, abstract, composable programs
are simple programs
Modularity
Can you reason about something in isolation?
Or do you need to fit everything in your head at once?
A
B C K
Is the cost of replacing Bwith K just writing K?
A
B CK
Is the cost of replacing Bwith K just writing K?
AK
B CK
K
…or rewriting half the program?
glue
glue
A pure function is just input output; no side effects
You can tell what it does without Looking at surrounding context.
Consider: Let’s parse a string like “Richmond, VIC 3121”
def parseLocation(str: String): Location = {val parts = str.split(“,”)val secondStr = parts(1)val parts2 = secondStr.split(“ “)Location(
parts(0), parts2(0), parts(1).toInt)}
Could be null
Possible errors: 1
def parseLocation(str: String): Location = {val parts = str.split(“,”)val secondStr = parts(1)val parts2 = secondStr.split(“ “)Location(
parts(0), parts2(0), parts(1).toInt)}
def parseLocation(str: String): Location = {val parts = str.split(“,”)val secondStr = parts(1)val parts2 = secondStr.split(“ “)Location(
parts(0), parts2(0), parts(1).toInt)}
Might not have enough elements
Possible errors: 2
def parseLocation(str: String): Location = {val parts = str.split(“,”)val secondStr = parts(1)val parts2 = secondStr.split(“ “)Location(
parts(0), parts2(0), parts(1).toInt)}
Might not have enough elements
Possible errors: 3
def parseLocation(str: String): Location = {val parts = str.split(“,”)val secondStr = parts(1)val parts2 = secondStr.split(“ “)Location(
parts(0), parts2(0), parts(1).toInt)}
Might not have enough elements
Possible errors: 4
def parseLocation(str: String): Location = {val parts = str.split(“,”)val secondStr = parts(1)val parts2 = secondStr.split(“ “)Location(
parts(0), parts2(0), parts(1).toInt)}
Might not have enough elements
Possible errors: 5
def parseLocation(str: String): Location = {val parts = str.split(“,”)val secondStr = parts(1)val parts2 = secondStr.split(“ “)Location(
parts(0), parts2(0), parts(1).toInt)}
Might not be an int
Possible errors: 6
def doSomethingElse(): Unit = {// ...Do other stuffparseLocation(“Melbourne, VIC 3000”)
}
Possible errors: 6 + 3 = 9
63
def anotherThing(): Unit = {// ...Do even more stuffdoSomethingElse()
}
Possible errors: 9 + 4 = 13
49
Code inherits all the errors and side-effects of code it calls.
Local reasoning becomes impossible; modularity is lost.
def parseLocation(str: String): Option[Location] = {
val parts = str.split(”,")for {
locality <- parts.optGet(0)theRestStr <- parts.optGet(1)theRest = theRestStr.split(" ")subdivision <- theRest.optGet(0)postcodeStr <- theRest.optGet(1)postcode <- postcodeStr.optToInt
} yieldLocation(locality, subdivision, postcode)
}
Possible errors: 0
All possibilities have been elevated into the type system
Local reasoning is possible!
Local reasoning is possible about things that call it, etc…
Abstraction
Know as little as you need
Know as little as you need
Produce as little as you can
def sumInts(list: List[Int]): Int = {
var result = 0for (x <- list) {
result = result + x}return result
}
def flatten[A](list: List[List[A]]): List[A] = {
var result = List()for (x <- list) {
result = result ++ x}return result
}
def all[A](list: List[Boolean]): Boolean = {
var result = truefor (x <- list) {
result = result && x}return result
}
def sumInts(list: List[Int]): Int = {
var result = 0for (x <- list) {
result = result + x}return result
}
def flatten[A](list: List[List[A]]): List[A] = {
var result = List()for (x <- list) {
result = result ++ x}return result
}
def all[A](list: List[Boolean]): Boolean = {
var result = truefor (x <- list) {
result = result && x}return result
}
trait Monoid[M] {def zero: Mdef append(m1: M, m2: M): M
}
Extract the essence!
def fold[M](list: List[M])(implicit m: Monoid[M]): M
= {
var result = m.zerofor (x <- list) {
result = m.append(result,x)}result
}
def fold[M](list: List[M])(implicit m: Monoid[M]): M
= list.foldLeft(m.zero)(m.append)
fold(List(1, 2, 3, 4)) 10
fold(List(List("a", "b"), List("c"))) List("a", "b", "c")
fold(List(true, true, false, true)) false
Abstraction is always a good thing!
Less repetition More reuse
Less decay, because code can’tgrow tumours around unnecessary detail
Composability
Functions compose.
A => B B => C
C => D D => E
A => E
Functions compose.
Sideways too.
A => E
X => Y
(A,X) => (E,Y)
Sideways too.
This works in the large, as well as the small!
Entire systems can be composable like functions…. without side effects
Truly composablesystems can accrue more and more stuff without getting more complex!
Simplicity
• Modularity – Reason locally
• Abstraction – say only what you need, hide
everything you don’t
• Composability – scale without accruing complexity
The human process
Technical excellence isn’t enough!
Software development is a human process
Coffee Tea
Quiet time
GSD
PoniesReconsider!
Xi’an offshore team
Team
Team
Team
Team
Xi’an offshore team
• Many teams are partly based in Xi’an.
• They’re very good, but…
• Communication is hard!
• It works, but requires great investment of time and
money
Bottom-up tech decisions
GET /foo/bar
PUT
{"mode":
"banana"}
POST {"partyTime": "5:00"}GET /buzz/5/
ArchitectMountain
ArchitectMountain
ArchitectMountain
ArchitectMountain
Just needs more Agile
Don’t forget your
velocity
More meetings, but littler
NONo no no no no no no
no no no no no no no
no no no no no no no
no no no no no no no
no no no no not like
this. Wake up it’s a
school day
Bottom-up tech decisions
You have to win
Bottom-up tech decisions
You have to win
Bottom-up tech decisions
You have to win
Software Paleontology
Everyone’s got a history…
Scriptozoic era1995 – 2010Mostly Perl
Monolithocene epoch2010 – 2012Ruby, Java
Scriptozoic era1995 – 2010Mostly Perl
AWS
Monolithocene epoch2010 – 2012Ruby, Java
Scriptozoic era1995 – 2010Mostly Perl
Microservices2012 –Ruby, Scala, JS
Adoption
June, 2013
λ
λ
λ
λ
λ
λ
λ
λ
λ
λ
λ
λ
λ
λ
Language choiceObject Oriented
Powerful static types
Functional
JVM
Object Oriented
Powerful static types
Functional
JVM
Functional
JVM
Functional
JVM
Functional
JVM
Functional
JVM
Whatever works for you!
The journey
Jul 13 Jan 14
1
LOC
Web
JSON
DB
Dep Inj
Play2
Play2
6K
Squeryl / Play2
Constructors
Type API#1
LOC
Web
JSON
DB
Dep Inj
Play2
Play2
6K
Squeryl / Play2
Constructors
Type API
• Mentor• Code reviews
Learning Investment
#1
LOC
Web
JSON
DB
Dep Inj
Play2
Play2
6K
Squeryl / Play2
Constructors
Type API
• Mentor• Code reviews
Learning Investment Report cardLearning curve
Technical result
Productivity
Sentiment
Steep but ok
OK; slight dip
Great; but FWs too heavy
#1
Jul 13 Jan 14
1 2
Some infrastructure
LOC
Web
JSON
DB
Dep Inj
Play2
Argonaut
3K
Squeryl / Play2
Constructors
Type API#2
LOC
Web
JSON
DB
Dep Inj
Play2
Argonaut
3K
Squeryl / Play2
Constructors
Type API
• Almost noneLearning Investment
#2
LOC
Web
JSON
DB
Dep Inj
Play2
Argonaut
3K
Squeryl / Play2
Constructors
Type API
• Almost noneLearning Investment Report card
Learning curve
Technical result
Productivity
Sentiment
Learning?
Meh
Needed rework; OK in the end
#2
!!!
Lesson #1
New tech, mindset requires investment in learning
Invest in your people!
λ
Another team!
λ
“We’ll have some of that!”
λ
Jul 13 Jan 14
1 2
46
5
New team; new ideas!
3
LOC
Web
JSON
DB
Dep Inj
Unfinagled
Argonaut
2K, 3K, 4K, 1K
Slick
Constructors
Type Web app, libs, API x 2
• 2 x MentorLearning Investment Report card
Learning curve
Technical result
Productivity
Sentiment
Not bad
Great
Great
#3,4,5,6
Jul 13 Jan 14
1 2
46
5
3
7
Theft & innovation
Jul 14
Lesson #2
Having multiple teams is great, because you can steal from each other
LOC
Web
JSON
DB
Dep Inj
Unfinagled
Argonaut
4K
Slick
Monad Transformers
Type API#7
LOC
Web
JSON
DB
Dep Inj
Unfinagled
Argonaut
4K
Slick
Monad Transformers
Type API
• 2 x MentorLearning Investment
#7
LOC
Web
JSON
DB
Dep Inj
Unfinagled
Argonaut
4K
Slick
Monad Transformers
Type API
• 2 x MentorLearning Investment Report card
Learning curve
Technical result
Productivity
Sentiment
Vertical
Great
Great
#7
Monad Transformers
Good technical benefits, but…
Only 2 people could understand the code
Experienced engineers felt totally helpless
Learning curve way too steep
Devops
All-rounders
Gurus
All-rounders
Gurus
JS / CSS / HTML
AWS
All-rounders
Gurus
Scala (originally!)
Gurus
All-rounders
Scala (now)
Gurus
• Smooth learning curve is utterly essential
• We need more all-rounders
• We can’t leave people behind
Lesson #3Familiar, but technically unsound concepts have limited value.
However… if we can’t make a concept learnable, then we can’t use it.
λ
A Ruby team considers its options…
λ
λ
Jul 13 Jan 14
1 2
46
5
3
7
A 3rd team dips its toe in the water
8
Jul 14
LOC
Web
JSON
DB
Dep Inj
Play2
Play2
2K
Anorm
Constructors
Type Web app#8
LOC
Web
JSON
DB
Dep Inj
Play2
Play2
2K
Anorm
Constructors
Type Web app
• Trial and error• Code Katas
Learning Investment
#8
LOC
Web
JSON
DB
Dep Inj
Play2
Play2
2K
Anorm
Constructors
Type Web app
• Trial and error• Code Katas
Learning Investment Report cardLearning curve
Technical result
Productivity
Sentiment
Steep
Meh
OK
#8
Lesson #4
It’s really hard learning from scratch
Be prepared for pain up front
λ
Jul 13 Jan 14
1 2
46
5
3
7
Latest iteration
8
Jul 14 Jan 15
17
Design trends
Inheritance/mixins Static functions
Design trends
Inheritance/mixins Static functions
Partial functions Total functions
Design trends
Inheritance/mixins Static functions
Partial functions Total functions
Exceptions Sum types
Design trends
Inheritance/mixins Static functions
Partial functions Total functions
Strings/primitives Wrapper types
Exceptions Sum types
FP Guild
Every Thursday, in work hours
7 – 12 people each week
Reading, exercises, talks, live coding
LOC
Web
JSON
DB
Dep Inj
Unfiltered
Argonaut
3K
Slick
Free Monads
Type API#17
LOC
Web
JSON
DB
Dep Inj
Unfiltered
Argonaut
3K
Slick
Free Monads
Type API
• 2 x Mentors• Pull Requests• Code reviews• Pairing• FP Guild
Learning Investment
#17
LOC
Web
JSON
DB
Dep Inj
Unfiltered
Argonaut
3K
Slick
Free Monads
Type API
• 2 x Mentors• Pull Requests• Code reviews• Pairing• FP Guild
Learning Investment Report cardLearning curve
Technical result
Productivity
Sentiment
Smooth
Great
Brilliant
#17
Pure coreRoutes Controllers Logic Script
Interpreter
Actual DB
“Authenticate”“Use config”“Get from DB”“Update DB”“Log result”
Web Framework
Server
App runtime
Pure coreRoutes Controllers Logic Script
Pure Interpreter
Pure tests
Input
Output
Assert
“Authenticate”“Use config”“Get from DB”“Update DB”“Log result”
Pure core
Interpreter
Actual DB
Web Framework
Server
App runtimeWafer thinE2E tests
Input
Output
Assert
object SearchController {
def getList(uid: UserId): Script[Response]= {
for {searches <- getSearches(uid)cfg <- getConfigresults <- addLinks(searches, cfg)
} yield Ok ~> Response(result.toJson)}
}
Learning curve:
Learning curve:
Smooooth
Mocks
Stubs
Flaky pretend servers
just to check
a response code
Flaky tests
that take
30 min to run
Side effects
everywhere
Exceptions
Intrusive
DI frameworks
You shouldn’t be dealing with all that complexity and crap!
Most business software isn’t rocket science.
There is a better way
FP is a tall tree
But there is so much low hanging fruit!
Thank you!