Do Extraterrestrials Use Functional...

158
Manuel M T Chakravarty University of New South Wales Do Extraterrestrials Use Functional Programming? mchakravarty α TacticalGrace TacticalGrace Thursday, 16 May 13 » Straight to next slide [15min Question (λ); 20min Methodology; 15min Application]

Transcript of Do Extraterrestrials Use Functional...

Manuel M T ChakravartyUniversity of New South Wales

Do Extraterrestrials Use Functional Programming?

mchakravarty

α TacticalGraceTacticalGrace

Thursday, 16 May 13

» Straight to next slide

[15min Question (λ); 20min Methodology; 15min Application]

Part 1The Question

Thursday, 16 May 13

This talk will be in three parts.(1) Discussing essence of functional programming. What makes FP tick?(2) How do FP principles influence software dev? Will propose a dev methodology for FP.(3) Look at concrete dev project, where we applied this methodology. »»Let's start with The Question…

“Do Extraterrestrials Use Functional Programming?”

Thursday, 16 May 13

» <Read question>* To visit us, they need to be on an advanced technological level with a deep understanding of science.* They won't speak one of humanity's languages, though. So, how do we establish a common basis?

Thursday, 16 May 13

* How to communicate?* Common idea: universal principles may help establish a basis — universal constants or universal laws.

Thursday, 16 May 13

* How to communicate?* Common idea: universal principles may help establish a basis — universal constants or universal laws.

π?

Thursday, 16 May 13

* How to communicate?* Common idea: universal principles may help establish a basis — universal constants or universal laws.

π?E = mc2

Thursday, 16 May 13

* How to communicate?* Common idea: universal principles may help establish a basis — universal constants or universal laws.

Thursday, 16 May 13

* Computer languages? Agree on a common language of computation?* In 1936, Alonzo Church introduced the lambda calculus: <Explain lambda calculus>* Serve as a common language? Like a computational Esperanto?* Also other calculi/machines. Famous: Turing machines. Which would aliens pick? »» Let's look: how are they related…

Alonzo Church

M, N → x | λx.M | M N

Thursday, 16 May 13

* Computer languages? Agree on a common language of computation?* In 1936, Alonzo Church introduced the lambda calculus: <Explain lambda calculus>* Serve as a common language? Like a computational Esperanto?* Also other calculi/machines. Famous: Turing machines. Which would aliens pick? »» Let's look: how are they related…

Alonzo Church

M, N → x | λx.M | M NM, N → x | λx.M | M N

Thursday, 16 May 13

* Computer languages? Agree on a common language of computation?* In 1936, Alonzo Church introduced the lambda calculus: <Explain lambda calculus>* Serve as a common language? Like a computational Esperanto?* Also other calculi/machines. Famous: Turing machines. Which would aliens pick? »» Let's look: how are they related…

Alonzo Church

M, N → x | λx.M | M NM, N → x | λx.M | M NM, N → x | λx.M | M N

Thursday, 16 May 13

* Computer languages? Agree on a common language of computation?* In 1936, Alonzo Church introduced the lambda calculus: <Explain lambda calculus>* Serve as a common language? Like a computational Esperanto?* Also other calculi/machines. Famous: Turing machines. Which would aliens pick? »» Let's look: how are they related…

Alonzo Church

M, N → x | λx.M | M NM, N → x | λx.M | M NM, N → x | λx.M | M NM, N → x | λx.M | M N

Thursday, 16 May 13

* Computer languages? Agree on a common language of computation?* In 1936, Alonzo Church introduced the lambda calculus: <Explain lambda calculus>* Serve as a common language? Like a computational Esperanto?* Also other calculi/machines. Famous: Turing machines. Which would aliens pick? »» Let's look: how are they related…

Alonzo Church

M, N → x | λx.M | M NM, N → x | λx.M | M NM, N → x | λx.M | M NM, N → x | λx.M | M NM, N → x | λx.M | M N

Thursday, 16 May 13

* Computer languages? Agree on a common language of computation?* In 1936, Alonzo Church introduced the lambda calculus: <Explain lambda calculus>* Serve as a common language? Like a computational Esperanto?* Also other calculi/machines. Famous: Turing machines. Which would aliens pick? »» Let's look: how are they related…

Alonzo Church

M, N → x | λx.M | M N

Alan Turing

Thursday, 16 May 13

* Computer languages? Agree on a common language of computation?* In 1936, Alonzo Church introduced the lambda calculus: <Explain lambda calculus>* Serve as a common language? Like a computational Esperanto?* Also other calculi/machines. Famous: Turing machines. Which would aliens pick? »» Let's look: how are they related…

Alonzo ChurchAlan Turing

M, N → x | λx.M | M N

Thursday, 16 May 13

* The lambda calculus and Turing machines have the same origin.* Beginning 20th century: group of famous mathematicians interested in formalising foundation of mathematics.»» This led to an important question…

M, N → x | λx.M | M N

Lambda Calculus Turing Machine

Thursday, 16 May 13

* The lambda calculus and Turing machines have the same origin.* Beginning 20th century: group of famous mathematicians interested in formalising foundation of mathematics.»» This led to an important question…

M, N → x | λx.M | M N

Lambda Calculus Turing Machine

By-product of a study of the foundation and expressive power

of mathematics.

Thursday, 16 May 13

* The lambda calculus and Turing machines have the same origin.* Beginning 20th century: group of famous mathematicians interested in formalising foundation of mathematics.»» This led to an important question…

David Hilbert

Thursday, 16 May 13

* Challenge posed by David Hilbert, 1928: the Entscheidungsproblem (decision problem)* Church & Turing, 1936, no solution, using lambda calculus & Turing machines.»» So what is the Entscheidungsproblem…

Is there a solution to the Entscheidungsproblem?

David Hilbert

Thursday, 16 May 13

* Challenge posed by David Hilbert, 1928: the Entscheidungsproblem (decision problem)* Church & Turing, 1936, no solution, using lambda calculus & Turing machines.»» So what is the Entscheidungsproblem…

Is there a solution to the Entscheidungsproblem?

David Hilbert

No!

No!

Thursday, 16 May 13

* Challenge posed by David Hilbert, 1928: the Entscheidungsproblem (decision problem)* Church & Turing, 1936, no solution, using lambda calculus & Turing machines.»» So what is the Entscheidungsproblem…

“Is there an algorithm to decide whether a given statement is provable from a set of axioms using the rules of first-order

logic?”

Thursday, 16 May 13

* In other words: Given a world & a set of fixed rules in the world, check whether the world has a particular property.»» In turn, leads to the question…

“How do you prove that an algorithm does not exist?”

Thursday, 16 May 13

* Because we cannot solve the challenge, doesn't mean it is unsolvable?* Need systematic way to rigorously prove that a solution is impossible.»» Church & Turing proceeded as follows…

Thursday, 16 May 13

* 1936, the concept of an algorithm remained to be formally defined

(1) Define a universal language or abstract machine.

(2) Show that the desired algorithm cannot be expressed in the language.

Thursday, 16 May 13

* 1936, the concept of an algorithm remained to be formally defined

Define a universal language or abstract machine.

Thursday, 16 May 13

* Two steps <Explain>* Church & Turing used: (1) lambda term, (2) Turing machine* Hypothesis: universal — ie, any algorithmically computable function can be expressed»» They showed…

Define a universal language or abstract machine.

Lambda Calculus Turing Machine

M, N → x | λx.M | M N

Thursday, 16 May 13

* Two steps <Explain>* Church & Turing used: (1) lambda term, (2) Turing machine* Hypothesis: universal — ie, any algorithmically computable function can be expressed»» They showed…

Lambda Calculus Turing Machine

M, N → x | λx.M | M N

Universal language

Thursday, 16 May 13

* Two steps <Explain>* Church & Turing used: (1) lambda term, (2) Turing machine* Hypothesis: universal — ie, any algorithmically computable function can be expressed»» They showed…

Lambda Calculus Turing Machine

M, N → x | λx.M | M N

Universal language

Church-Turing thesis

Thursday, 16 May 13

* Two steps <Explain>* Church & Turing used: (1) lambda term, (2) Turing machine* Hypothesis: universal — ie, any algorithmically computable function can be expressed»» They showed…

Lambda Calculus Turing Machine

M, N → x | λx.M | M N

Computational Power

=

Thursday, 16 May 13

* Any program expressible in one is expressible in the other.»» However, …

Turing MachineLambda Calculus

M, N → x | λx.M | M N

Generality

Thursday, 16 May 13

* Lambda calculus: embodies concept of (functional) *abstraction** Functional abstraction is only one embodiment of an underlying more general concept.»» This is important, as…

Turing Machine

Lambda Calculus

M, N → x | λx.M | M N

Generality

Thursday, 16 May 13

* Lambda calculus: embodies concept of (functional) *abstraction** Functional abstraction is only one embodiment of an underlying more general concept.»» This is important, as…

“Generality increases if a discovery is independently

made in a variety of contexts.”

Thursday, 16 May 13

» Read the statement.* If a concept transcends one application, its generality increases.»» This is the case for the lambda calculus…

Simply typed lambda calculus

Thursday, 16 May 13

* Firstly, lambda calculus (no polytypes)…»» Mathematicians Haskell Curry & William Howard discovered: it is structurally equivalent to…

Simply typed lambda calculus

Lambda calculus with monotypes

Thursday, 16 May 13

* Firstly, lambda calculus (no polytypes)…»» Mathematicians Haskell Curry & William Howard discovered: it is structurally equivalent to…

Intuitionistic propositional logic

Thursday, 16 May 13

»» Later, Joachim Lambek found: they correspond to…

Intuitionistic propositional logic

Constructive logic

Thursday, 16 May 13

»» Later, Joachim Lambek found: they correspond to…

Simply typed lambda calculus

Intuitionistic propositional logicCartesian closed

categories

Thursday, 16 May 13

* Three independently discovered artefacts share the same structure!* Implies an equivalence between programming & proving.»» The upshot of all this…

Simply typed lambda calculus

Intuitionistic propositional logicCartesian closed

categories

Structure from category theory

Thursday, 16 May 13

* Three independently discovered artefacts share the same structure!* Implies an equivalence between programming & proving.»» The upshot of all this…

Simply typed lambda calculus

Intuitionistic propositional logic

Cartesian closed categories

Curry-Howard-Lambek correspondence

Thursday, 16 May 13

* Three independently discovered artefacts share the same structure!* Implies an equivalence between programming & proving.»» The upshot of all this…

“Alonzo Church didn't inventthe lambda calulus;he discovered it.”

Thursday, 16 May 13

» Read the statement.* Just like Issac Newton didn't invent the Law of Gravity, but discovered it.»» Getting back to our extraterrestrials…

Thursday, 16 May 13

* Lambda calculus: fundamental, inevitable, universal notion of computation.* In all likelihood: extraterrestials know about it, like they will know π.

Thursday, 16 May 13

* Lambda calculus: fundamental, inevitable, universal notion of computation.* In all likelihood: extraterrestials know about it, like they will know π.

M, N → x | λx.M | M N

M, N → x | λx.M | M N

Thursday, 16 May 13

* Lambda calculus: fundamental, inevitable, universal notion of computation.* In all likelihood: extraterrestials know about it, like they will know π.

“So what?”

Thursday, 16 May 13

* Is all this simply a academic curiosity?* Does it impact the practical use of FLs?»» It is crucial for FLs…

Thursday, 16 May 13

* FLs: pragmatic renderings of lambda calculus with syntactic sugar etc for convenience.* Important application: compilation via extended lambda calculi as ILs (eg, GHC)»» Moreover, central language features…

λHaskell

LISP

SchemeClojure

ScalaStandard ML

OCaml

Erlang

F# CleanElm

AgdaRacket

Miranda

FPHope

IdISWIM

SASLSISAL

J

Thursday, 16 May 13

* FLs: pragmatic renderings of lambda calculus with syntactic sugar etc for convenience.* Important application: compilation via extended lambda calculi as ILs (eg, GHC)»» Moreover, central language features…

λHaskell

LISP

SchemeClojure

ScalaStandard ML

OCaml

Erlang

F# CleanElm

AgdaRacket

Miranda

FPHope

IdISWIM

SASLSISAL

J

Thursday, 16 May 13

Central language features of FLs have their origin in the lambda calculus:* HO functions & closures: lambda* Purity & immutable structures: functional semantics* Types & semantics: logic & Curry-Howard

λHaskell

LISP

SchemeClojure

ScalaStandard ML

OCaml

Erlang

F# CleanElm

AgdaRacket

Miranda

FPHope

IdISWIM

SASLSISAL

JPurity

Immutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Thursday, 16 May 13

Central language features of FLs have their origin in the lambda calculus:* HO functions & closures: lambda* Purity & immutable structures: functional semantics* Types & semantics: logic & Curry-Howard

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Language features

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Language features

Practical advantages

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Concurrency&

parallelism

Language features

Practical advantages

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Concurrency&

parallelism

Meta programming

Language features

Practical advantages

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Concurrency&

parallelism

Meta programming Reuse

Language features

Practical advantages

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Concurrency&

parallelism

Meta programming Reuse

Strongisolation

Language features

Practical advantages

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Concurrency&

parallelism

Meta programming Reuse

Strongisolation Safety

Language features

Practical advantages

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

PurityImmutablestructures

Higher-orderfunctions

&closures

Well-definedsemantics

Types

Concurrency&

parallelism

Meta programming Reuse

Strongisolation Safety

Language features

Practical advantagesFormal

reasoning

Thursday, 16 May 13

* Language features lead to practical advantages* Some examples: <explain where they come from>»» Nevertheless, we can gain even more from the foundation of FP than these advantages…

Part 2 From Language to Methodology

Thursday, 16 May 13

* Part 1: FP derives from natural, fundamental concept of computation...* ...which is the root of language conveniences and practical advantages.»» We want to take that concept one step further…

“Functional programming as a development methodology,

not just a language category.”

Thursday, 16 May 13

» We want to use <read the statement>.* Use the principles of the lambda calculus for a software development methodology.[Engineering is based on science. This is the science of programming/software.]»» To do this…

“The key to functional software development is a consistent

focus on properties.”

Thursday, 16 May 13

» We need to realise that <read the statement>* These can be "logical properties" or "mathematical properties".»» More precisely, …

Properties

Thursday, 16 May 13

* Properties are rigorous and precise. (NB: PL is a formal notation.)* We are not talking about specifying the entire behaviour of an applications. (Type signatures are properties.)* In one way or another, they leverage the formal foundation of the lambda calculus.»» Let's look at some examples…

Properties

Rigorous, formal or semi-formal specification

Cover one or more aspects of a program

Leverage the mathematics of the lambda calculus

Thursday, 16 May 13

* Properties are rigorous and precise. (NB: PL is a formal notation.)* We are not talking about specifying the entire behaviour of an applications. (Type signatures are properties.)* In one way or another, they leverage the formal foundation of the lambda calculus.»» Let's look at some examples…

“A pure function is fully specified by a mapping of argument to

result values.”

Thursday, 16 May 13

» Read the statement.* Menas: if you know the arguments, you know the result.* (1) Nothing else influences the result; (2) the function doesn't do anything, but provide the result.* This is semi-formal, but easy to formalise.

“A pure function is fully specified by a mapping of argument to

result values.”

Well known property

Thursday, 16 May 13

» Read the statement.* Menas: if you know the arguments, you know the result.* (1) Nothing else influences the result; (2) the function doesn't do anything, but provide the result.* This is semi-formal, but easy to formalise.

map :: (a -> b) -> [a] -> [b]

eval :: Expr t -> t

n+m≡m+n : ∀ {n m : ℕ} -> m + n ≡ n + m

Thursday, 16 May 13

* map: well known* eval: type-safe evaluator with GADTs* Agda lemma: commutativity of addition»» Types are not just for statically typed languages…

Types are properties

map :: (a -> b) -> [a] -> [b]

eval :: Expr t -> t

n+m≡m+n : ∀ {n m : ℕ} -> m + n ≡ n + m

Thursday, 16 May 13

* map: well known* eval: type-safe evaluator with GADTs* Agda lemma: commutativity of addition»» Types are not just for statically typed languages…

Racket(Scheme dialect)

Thursday, 16 May 13

* HTDP encourages the use of function signatures as part of the design process.* It also uses data definitions (reminiscent of data type definitions)* Racket also supports checked "contracts"

The Process: [..] 2. Write down a signature, [..]

Racket(Scheme dialect)

Thursday, 16 May 13

* HTDP encourages the use of function signatures as part of the design process.* It also uses data definitions (reminiscent of data type definitions)* Racket also supports checked "contracts"

-- QuickCheckprop_Union s1 (s2 :: Set Int) =  (s1 `union` s2) ==? (toList s1 ++ toList s2)

Thursday, 16 May 13

* In formal specifications* But also useful for testing: QuickCheck* Popular specification-based testing framework»» And as the last example of a property…

Logic formulas

-- QuickCheckprop_Union s1 (s2 :: Set Int) =  (s1 `union` s2) ==? (toList s1 ++ toList s2)

Thursday, 16 May 13

* In formal specifications* But also useful for testing: QuickCheck* Popular specification-based testing framework»» And as the last example of a property…

-- return a >>= k == k a -- m >>= return == m -- m >>= (\x -> k x >>= h) -- == (m >>= k) >>= h class Monad m where (>>=) :: m a -> (a -> m b) -> m b return :: a -> m a

Thursday, 16 May 13

* Monads: categorial structures that needs to obey certain laws.* Think of them as API patterns.

Algebraic and categorial structures

-- return a >>= k == k a -- m >>= return == m -- m >>= (\x -> k x >>= h) -- == (m >>= k) >>= h class Monad m where (>>=) :: m a -> (a -> m b) -> m b return :: a -> m a

Thursday, 16 May 13

* Monads: categorial structures that needs to obey certain laws.* Think of them as API patterns.

I/O in Haskell

Thursday, 16 May 13

* Now that we have seen some examples of properties, ...»» ...let's look at an example of guiding a design by properties…

I/O in Haskell

Example of an uncompromising pursuit of properties

Thursday, 16 May 13

* Now that we have seen some examples of properties, ...»» ...let's look at an example of guiding a design by properties…

readName = let firstname = readString () in let surname = readString () in firstname ++ " " ++ surname

(not really Haskell)

Thursday, 16 May 13

* Read two strings from stdin and combine them.* In which order will firstname and surname be read? * Non-strict (or lazy) language: compute when needed»» Problem with I/O, as the following compiler optimisations demonstrate…

readName = let firstname = readString () in let surname = readString () in firstname ++ " " ++ surname

(not really Haskell)

Haskell is a non-strict language

Thursday, 16 May 13

* Read two strings from stdin and combine them.* In which order will firstname and surname be read? * Non-strict (or lazy) language: compute when needed»» Problem with I/O, as the following compiler optimisations demonstrate…

readName = let firstname = readString () in let surname = readString () in firstname ++ " " ++ surname

Thursday, 16 May 13

* Two occurences of the same lambda term must have the same meaning.

readName = let firstname = readString () in let surname = readString () in firstname ++ " " ++ surname

Common subexpression elimination

firstname

Thursday, 16 May 13

* Two occurences of the same lambda term must have the same meaning.

readName = let firstname = readString () in let surname = readString () in firstname ++ " " ++ surname

firstname = readString ()

surname = readString ()

Thursday, 16 May 13

* No data depencency between the two bindings

readName = let firstname = readString () in let surname = readString () in firstname ++ " " ++ surname

firstname = readString ()

surname = readString ()

Reordering

Thursday, 16 May 13

* No data depencency between the two bindings

readName = let firstname = readString () in let surname = readString () in firstname

Thursday, 16 May 13

* If a binding is not used, we should be able to eliminate it.* 1988: Haskell language committee faced the problem of mismatch between non-strictness and I/O»» They saw two options…

readName = let firstname = readString () in let surname = readString () in firstname

Dead code elimination

Thursday, 16 May 13

* If a binding is not used, we should be able to eliminate it.* 1988: Haskell language committee faced the problem of mismatch between non-strictness and I/O»» They saw two options…

Option ❶

Destroy purity

Thursday, 16 May 13

»» To do so, they would need…

Destroy purity

Thursday, 16 May 13

» <Explain>

Destroy purity

Prohibit those code transformations

Enforce strict top to bottom evaluation of let bindings

Thursday, 16 May 13

» <Explain>

Destroy purity

Prohibit those code transformations

Enforce strict top to bottom evaluation of let bindingsNot a good idea!

Thursday, 16 May 13

» <Explain>

WG 2.8, 1992

Thursday, 16 May 13

[This is not the real committee, but a large part.]* Didn't want to give up this property.* Non-strictness kept them honest.»» This left them with the second option…

WG 2.8, 1992

Preserve those code transformations

Thursday, 16 May 13

[This is not the real committee, but a large part.]* Didn't want to give up this property.* Non-strictness kept them honest.»» This left them with the second option…

WG 2.8, 1992

Preserve those code transformations

We want local reasoning

Thursday, 16 May 13

[This is not the real committee, but a large part.]* Didn't want to give up this property.* Non-strictness kept them honest.»» This left them with the second option…

WG 2.8, 1992

Preserve those code transformations

We want local reasoning

Think about concurrency

Thursday, 16 May 13

[This is not the real committee, but a large part.]* Didn't want to give up this property.* Non-strictness kept them honest.»» This left them with the second option…

Keep purity!

WG 2.8, 1992

Preserve those code transformations

We want local reasoning

Think about concurrency

Thursday, 16 May 13

[This is not the real committee, but a large part.]* Didn't want to give up this property.* Non-strictness kept them honest.»» This left them with the second option…

Option ❷

Continuation-based & Stream-based I/O

Thursday, 16 May 13

»» I don't want to explain them in detail, but here is an example…

readName :: [Response] -> ([Request], String)readName ~(Str firstname : ~(Str surname : _)) = ([ReadChan stdin, ReadChan stdin], firstname ++ " " ++ surname)

Thursday, 16 May 13

* Rather inconvenient programming model* Due to lack of a better idea, Haskell 1.0 to 1.2 used continuation-based and stream-based I/O»» Can't we do any better…

readName :: [Response] -> ([Request], String)readName ~(Str firstname : ~(Str surname : _)) = ([ReadChan stdin, ReadChan stdin], firstname ++ " " ++ surname)

readName :: FailCont -> StrCont -> BehaviourreadName abort succ = readChan stdin abort (\firstname -> readChan stdin abort (\surname -> succ (firstname ++ " " ++ surname)))

Thursday, 16 May 13

* Rather inconvenient programming model* Due to lack of a better idea, Haskell 1.0 to 1.2 used continuation-based and stream-based I/O»» Can't we do any better…

“What are the properties of I/O, of general stateful operations?”

Thursday, 16 May 13

* Let's take a step back.» Can we use properties to understand the nature of I/O?»» Let's characterise what stateful (imperative) computing is about…

Arguments Result

State changing function

Thursday, 16 May 13

* In addition to arguments and result...* ...state is threaded through.»» In the case of I/O…

State State'

Arguments Result

State changing function

Thursday, 16 May 13

* In addition to arguments and result...* ...state is threaded through.»» In the case of I/O…

Arguments Result

I/O function

Thursday, 16 May 13

* The state is the whole world»» How can we formalise this…

Arguments Result

I/O function

Thursday, 16 May 13

* Categorial semantics of impure language features: properties of impure features* Lambda calculus with impure features* Characterise the meaning of effects»» How can we use that to write FPs…

Arguments Result

I/O function

Thursday, 16 May 13

* Categorial semantics of impure language features: properties of impure features* Lambda calculus with impure features* Characterise the meaning of effects»» How can we use that to write FPs…

Eugenio Moggi

Arguments Result

I/O functionMonad!

Thursday, 16 May 13

* Categorial semantics of impure language features: properties of impure features* Lambda calculus with impure features* Characterise the meaning of effects»» How can we use that to write FPs…

Eugenio Moggi

Thursday, 16 May 13

* Moggi's semantics is based on the lambda calculus* So, it ought to translate to FLs»» Finally, we can write our example program properly…

Eugenio Moggi

Thursday, 16 May 13

* Moggi's semantics is based on the lambda calculus* So, it ought to translate to FLs»» Finally, we can write our example program properly…

Eugenio Moggi

Philip Wadler -- return a >>= k == k a -- m >>= return == m -- m >>= (\x -> k x >>= h) -- == (m >>= k) >>= h class Monad m where (>>=) :: m a -> (a -> m b) -> m b return :: a -> m a

instance Monad IO where ...

Thursday, 16 May 13

* Moggi's semantics is based on the lambda calculus* So, it ought to translate to FLs»» Finally, we can write our example program properly…

readName :: IO String readName = do firstname <- readString surname <- readString in return (firstname ++ " " ++ surname)

(Real Haskell!)

Thursday, 16 May 13

* Development oriented at properties* Solution has an impact well beyond Haskell I/O»» Functional software development usually doesn't mean to resort to abstract math…

Part 3 Applying the Methodology

Thursday, 16 May 13

* So far, we saw that the genesis of FP resolved around working with and exploiting logical & mathematical properties.»» To get a feel for using such properties, let us look at a concrete development effort, where we used properties in many flavours to attack a difficult problem…

Pure data parallelism

Thursday, 16 May 13

»» Good parallel programming environments are important, because of…

Pure data parallelism

Case study in functional software development

Thursday, 16 May 13

»» Good parallel programming environments are important, because of…

multicore GPU

multicore CPU

Ubiquitous parallelism

Thursday, 16 May 13

* Today, parallelism is everywhere! <Explain>»» We would like a parallel programming environment with meeting the following goals…

Goal ➀

Exploit parallelism of commodity hardware easily:

Thursday, 16 May 13

* We are not aiming at supercomputers* Ordinary applications cannot afford the resources that go into the development of HPC apps.»» To this end…

Goal ➀

Performance is important, but…

…productivity is more important.

Exploit parallelism of commodity hardware easily:

Thursday, 16 May 13

* We are not aiming at supercomputers* Ordinary applications cannot afford the resources that go into the development of HPC apps.»» To this end…

Goal ➁

Semi-automatic parallelism:

Thursday, 16 May 13

* Not fully automatic: computers cannot parallelise algos & seq algos are inefficient on parallel hardware.* Explicit concurrency is hard, non-modular, and error prone.»» How can properties help us to achieve these two goals…

Programmer supplies a parallel algorithm, but

no explicit concurrency (no concurrency control, no races, no deadlocks).

Goal ➁

Semi-automatic parallelism:

Thursday, 16 May 13

* Not fully automatic: computers cannot parallelise algos & seq algos are inefficient on parallel hardware.* Explicit concurrency is hard, non-modular, and error prone.»» How can properties help us to achieve these two goals…

Three property-driven methods

Thursday, 16 May 13

Types: track purity, generate array representations, guide optimisationsState minimisation: localised state transformers, immutable structuresCombinators: parallelisable aggregate array operations, exploit algebraic properties, restricted language for special hardware

Three property-driven methods

Types

Thursday, 16 May 13

Types: track purity, generate array representations, guide optimisationsState minimisation: localised state transformers, immutable structuresCombinators: parallelisable aggregate array operations, exploit algebraic properties, restricted language for special hardware

Three property-driven methods

Types

State minimisation

Thursday, 16 May 13

Types: track purity, generate array representations, guide optimisationsState minimisation: localised state transformers, immutable structuresCombinators: parallelisable aggregate array operations, exploit algebraic properties, restricted language for special hardware

Three property-driven methods

Types

State minimisation

Combinators & embedded languages

Thursday, 16 May 13

Types: track purity, generate array representations, guide optimisationsState minimisation: localised state transformers, immutable structuresCombinators: parallelisable aggregate array operations, exploit algebraic properties, restricted language for special hardware

multicore GPU

multicore CPU

Ubiquitious parallelism

Thursday, 16 May 13

»» What kind of code do we want to write for parallel hardware…

smvm :: SparseMatrix -> Vector -> Vector smvm sm v = [: sumP (dotp sv v) | sv <- sm :]

Thursday, 16 May 13

smvm :: SparseMatrix -> Vector -> Vector smvm sm v = [: sumP (dotp sv v) | sv <- sm :]

Thursday, 16 May 13

smvm :: SparseMatrix -> Vector -> Vector smvm sm v = [: sumP (dotp sv v) | sv <- sm :]

2 1.5

5 36.574

1

sm vThursday, 16 May 13

smvm :: SparseMatrix -> Vector -> Vector smvm sm v = [: sumP (dotp sv v) | sv <- sm :]

2 1.5

5 36.574

1

sm vThursday, 16 May 13

smvm :: SparseMatrix -> Vector -> Vector smvm sm v = [: sumP (dotp sv v) | sv <- sm :]

2 1.5

5 36.574

1

sm vThursday, 16 May 13

smvm :: SparseMatrix -> Vector -> Vector smvm sm v = [: sumP (dotp sv v) | sv <- sm :]

2 1.5

5 36.574

1

ΣΣΣΣΣ

sm vThursday, 16 May 13

smvm :: SparseMatrix -> Vector -> Vector smvm sm v = [: sumP (dotp sv v) | sv <- sm :]

2 1.5

5 36.574

1

ΣΣΣΣΣ

sm vThursday, 16 May 13

“Types ensure purity, purity ensures non-interference.”

Thursday, 16 May 13

* Functions that are not of monadic type are pure.* Pure functions can execute in any order, also in parallel.=> No concurrency control needed [Properties pay off — Types.]»» But we need more than a convenient notation…

“Types ensure purity, purity ensures non-interference.”

Types

Thursday, 16 May 13

* Functions that are not of monadic type are pure.* Pure functions can execute in any order, also in parallel.=> No concurrency control needed [Properties pay off — Types.]»» But we need more than a convenient notation…

High performance

Thursday, 16 May 13

* Performance is not the only goal, but it is a major goal.* Explain fluid flow.»» We can get good performance…

Thursday, 16 May 13

* Repa (blue) is on 7 CPU cores (two quad-core Xenon E5405 CPUs @ 2 GHz, 64-bit)* Accelerate (green) is on a Tesla T10 processor (240 cores @ 1.3 GHz)* Repa talk: Ben Lippmeier @ Thursday before lunch* Accelerate talk: Trevor McDonell @ Friday before lunch

Jos Stam's Fluid Flow Solver

Thursday, 16 May 13

* Repa (blue) is on 7 CPU cores (two quad-core Xenon E5405 CPUs @ 2 GHz, 64-bit)* Accelerate (green) is on a Tesla T10 processor (240 cores @ 1.3 GHz)* Repa talk: Ben Lippmeier @ Thursday before lunch* Accelerate talk: Trevor McDonell @ Friday before lunch

“How do we achieve high performance from purely

functional code?”

Thursday, 16 May 13

»» This presents an inherent tension…

Unboxed,mutable arrays

C-like loops

Thursday, 16 May 13

»» We resolve this tension with local state…

Unboxed,mutable arrays

C-like loops

Performance

Thursday, 16 May 13

»» We resolve this tension with local state…

Unboxed,mutable arrays

C-like loops

Performance

Purefunctions

Thursday, 16 May 13

»» We resolve this tension with local state…

Unboxed,mutable arrays

C-like loops

Performance

Purefunctions

Parallelism &Optimisations

Thursday, 16 May 13

»» We resolve this tension with local state…

Unboxed,mutable arrays

C-like loops

Performance

Purefunctions

Parallelism &Optimisations

Thursday, 16 May 13

»» We resolve this tension with local state…

map :: (Shape sh, Source r a) => (a -> b) -> Array r sh a -> Array D sh b

(Pure)

Thursday, 16 May 13

* We use a library of pure, parallel, aggregate operations* In Repa, types guide array representations»» Despite the pure interface, some combinators are internally impure…

Types

map :: (Shape sh, Source r a) => (a -> b) -> Array r sh a -> Array D sh b

(Pure)

Thursday, 16 May 13

* We use a library of pure, parallel, aggregate operations* In Repa, types guide array representations»» Despite the pure interface, some combinators are internally impure…

Local state

Thursday, 16 May 13

<Explain>* Program transformations and parallelisation on pure level* Then, unfold and optimise imperative program* Type system helps to get this right* Fusion

Local state

Allocate mutable array

Thursday, 16 May 13

<Explain>* Program transformations and parallelisation on pure level* Then, unfold and optimise imperative program* Type system helps to get this right* Fusion

Local state

Allocate mutable array

Initialise destructively

Thursday, 16 May 13

<Explain>* Program transformations and parallelisation on pure level* Then, unfold and optimise imperative program* Type system helps to get this right* Fusion

Local state

Allocate mutable array

Initialise destructively

Freeze!

Thursday, 16 May 13

<Explain>* Program transformations and parallelisation on pure level* Then, unfold and optimise imperative program* Type system helps to get this right* Fusion

Local state

Allocate mutable array

Initialise destructively

Freeze!

State minimisation

Thursday, 16 May 13

<Explain>* Program transformations and parallelisation on pure level* Then, unfold and optimise imperative program* Type system helps to get this right* Fusion

Local state

Allocate mutable array

Initialise destructively

Freeze!

State minimisationCombinators

Thursday, 16 May 13

<Explain>* Program transformations and parallelisation on pure level* Then, unfold and optimise imperative program* Type system helps to get this right* Fusion

Special hardware

Core i7 970CPU

NVIDIA GF100 GPU

12 THREADS24,576 THREADS

Thursday, 16 May 13

* Straight forward code generation is not suitable for all architectures»» GPUs are highly parallel, but also restricted in which operations are efficient…

GPU's don't like

Thursday, 16 May 13

* We won't compile all of Haskell to GPUs anytime soon.

GPU's don't like

SIMD divergence (conditionals)

Thursday, 16 May 13

* We won't compile all of Haskell to GPUs anytime soon.

GPU's don't like

SIMD divergence (conditionals)

Recursion

Thursday, 16 May 13

* We won't compile all of Haskell to GPUs anytime soon.

GPU's don't like

SIMD divergence (conditionals)

Recursion

Function pointers

Thursday, 16 May 13

* We won't compile all of Haskell to GPUs anytime soon.

GPU's don't like

SIMD divergence (conditionals)

Recursion

Function pointers

Automatic garbage collection

Thursday, 16 May 13

* We won't compile all of Haskell to GPUs anytime soon.

dotpAcc :: Vector Float -> Vector Float -> Acc (Scalar Float)

dotpAcc xs ys = let xs' = use xs ys' = use ys in fold (+) 0 (zipWith (*) xs' ys')

Thursday, 16 May 13

* We special purpose compile embedded code.

dotpAcc :: Vector Float -> Vector Float -> Acc (Scalar Float)

dotpAcc xs ys = let xs' = use xs ys' = use ys in fold (+) 0 (zipWith (*) xs' ys')

Acc marks embedded computations

Thursday, 16 May 13

* We special purpose compile embedded code.

dotpAcc :: Vector Float -> Vector Float -> Acc (Scalar Float)

dotpAcc xs ys = let xs' = use xs ys' = use ys in fold (+) 0 (zipWith (*) xs' ys')

Acc marks embedded computations

use embeds values

Thursday, 16 May 13

* We special purpose compile embedded code.

dotpAcc :: Vector Float -> Vector Float -> Acc (Scalar Float)

dotpAcc xs ys = let xs' = use xs ys' = use ys in fold (+) 0 (zipWith (*) xs' ys')

Acc marks embedded computations

use embeds values

Embedded language

Thursday, 16 May 13

* We special purpose compile embedded code.

types >< state languages

Thursday, 16 May 13

Functional software development is

property-driven development

Functional programming is fundamental to computing

types >< state languages

Thursday, 16 May 13

Thank you!

Thursday, 16 May 13

Images from

http://wikipedia.org

http://openclipart.org

http://dx.doi.org/10.1145/1238844.1238856

Thursday, 16 May 13