A Population Approach to Ubicomp Systems Design

74
A Population Approach to Ubicomp System Design Matthew Chalmers 5 60 30 Zippy FanBook 5 2 1 0 open? 3 4 5 6 7 8 dUI dUI BP(update)? getF ix! 9 rtnF ix? BP(sob)? getMatch addP red! noM atch BP(close)? closeApp rtnfocus!

Transcript of A Population Approach to Ubicomp Systems Design

Page 1: A Population Approach to Ubicomp Systems Design

A Population Approach to Ubicomp System Design

Matthew Chalmers

560

30

Zippy

FanBook

5

My predictions interface

210

open?!

3

4

5

6

7

8

dUIdUI

BP (update)?!

getF ix!!

9

rtnF ix?!

BP (sob)?!

getMatchaddPred!!

noMatch

BP (close)?!

closeApp

rtnfocus!!

Fixture server : FS1

2

10

getF ix?!

!

rtnF ix!!

Prediction Server : PS

21

0

addData!!

34

rtnData?!

getPred?!

getData!!addPred?!

rtnPred!!

1

Page 2: A Population Approach to Ubicomp Systems Design

Ubiquitous computing

More than just mobile devices...

Systems that fit with context, interaction and activity

...which are varied and dynamic in ways difficult to predict

The system is part of the user’s context

System interactions, and how users are modelled and presented to others in them, are resources for new user activity

Page 3: A Population Approach to Ubicomp Systems Design

Ubiquitous computing

A holistic view spanning technology, use and users

Mark Weiser: The unit of design should be social people, in their environment, plus your device

Robin Milner: Here we look for synergy between the societal vision on the one hand, and the development of scientific models and engineering principles on the other.

Page 4: A Population Approach to Ubicomp Systems Design

SUMgroup: Social / Ubiquitous / Mobile

Matthew Chalmers, Alistair Morrison, Marek Bell, Scott Sherwood, Don McMillan

Theory, user experience and system design

Use ‘in the wild’ rather than usability in the lab

Page 5: A Population Approach to Ubicomp Systems Design

A Short History: Treasure

Page 6: A Population Approach to Ubicomp Systems Design

A Short History: Feeding Yoshi

Page 7: A Population Approach to Ubicomp Systems Design

A Short History: EyeSpy

Page 8: A Population Approach to Ubicomp Systems Design

A Short History: Replayer

Page 9: A Population Approach to Ubicomp Systems Design

A Short History: Replayer

Page 10: A Population Approach to Ubicomp Systems Design

A Short History: Replayer

Page 11: A Population Approach to Ubicomp Systems Design

A Short History: Replayer

Page 12: A Population Approach to Ubicomp Systems Design

A Short History: shifting to iPhones

Page 13: A Population Approach to Ubicomp Systems Design

A Short History: shifting to iPhones

Page 14: A Population Approach to Ubicomp Systems Design

Mass participation

‘App Store’ model of distribution a huge success

More than 10 billion downloads in first 3 years

Ubicomp field trials generally focus on small numbers of users

Now there’s a potential to reach huge numbers of users relatively easily

What are the challenges:

Methodological? Technical? Ethical?

Page 15: A Population Approach to Ubicomp Systems Design

Hungry Yoshi

Our first study exploring app store recruitment model

Comparison with ‘traditional’ ubicomp field trial

Took 2005 Windows Mobile application, reimplemented it on iOS in 2009

Page 16: A Population Approach to Ubicomp Systems Design

Yoshi (2005)

Scans of WiFi APs

‘Carry’ fruit to Yoshis

Mainly qualitative methods, to study integration of game into everyday life

Original study was “long-term, wide-area”

Played over a week, with16 players in Glasgow, Nottingham and Derby

Page 17: A Population Approach to Ubicomp Systems Design

Yoshi (2009)

Page 18: A Population Approach to Ubicomp Systems Design

Quantitative evaluation

Page 19: A Population Approach to Ubicomp Systems Design

Qualitative evaluation

Qualitative evaluation at a distance

Task mechanism

Many types of ‘task’

Real-time updates

Page 20: A Population Approach to Ubicomp Systems Design

Qualitative evaluation

Over 28,000 responses

Tokens as motivation

Unrewarded: 11% of players

Rewarded: 19% of players

Players vs. participants

Page 21: A Population Approach to Ubicomp Systems Design

Qualitative evaluation

Facebook integration

Use Facebook mail, forums...

Page 22: A Population Approach to Ubicomp Systems Design

Qualitative evaluation

Telephone interviews

Offered payment

A few very keen participants at start, but struggled to get >10

Self-selection biases, language issues

Page 23: A Population Approach to Ubicomp Systems Design

Quantitative evaluation: SGLog

A generic logging framework with phone and server parts

Log user interactions (button taps, screen changes), and device/sensor data (accelerometer, WiFi connections)

Write to locally-stored files on device, but opportunistic uploading to SGLog server over Internet

‘Real-time’ evaluation/monitoring of trial in progress

SGVis: complementary visualisation / analysis desktop tool

Page 24: A Population Approach to Ubicomp Systems Design

Quantitative evaluation: SGVis

Page 25: A Population Approach to Ubicomp Systems Design

Quantitative evaluation

How do we count user numbers for Yoshi?

In Windows Mobile trial : 16 users paid for participation

In iOS trial:

Download?

Register for account?

Play for > x minutes/hours/days?

Page 26: A Population Approach to Ubicomp Systems Design

Quantitative evaluation

How do we count user numbers for Yoshi?

Downloads: 182,714

Unique users launching: 98,556

Users registered: 36,169

Score points: 4,134

Played on 5 or more days: 3,080

Page 27: A Population Approach to Ubicomp Systems Design

Quantitative evaluation: SGVis

Page 28: A Population Approach to Ubicomp Systems Design

Quantitative evaluation: SGVis

Page 29: A Population Approach to Ubicomp Systems Design

SGVis: Categorising users

Page 30: A Population Approach to Ubicomp Systems Design

SGVis: Categorising users

Page 31: A Population Approach to Ubicomp Systems Design

SGVis: Categorising users

Page 32: A Population Approach to Ubicomp Systems Design

SGVis: Categorising users

Calculate centroids of each identified cluster

4 categories of users: top players, commuters, static players, beginners

Page 33: A Population Approach to Ubicomp Systems Design

Categorising users

Targeting evaluation to specific categories

More effective or efficient evaluation?

Users’ reactions to categorisation

User feedback may let us understand whether or how it is meaningful

Users may change their behaviour as a result of being categorised?

Page 34: A Population Approach to Ubicomp Systems Design

Categorising users as part of iterative design

Release initial version of app and then loop:

Use tools to observe usage, analyse patterns and categorise users

Release optional modules to support observed and requested usage of users in specific categories

Sets of modules may be bundled and named as separate apps to support different observed categories of use

Page 35: A Population Approach to Ubicomp Systems Design

The software engineering revolution

Conventional models of the software development process all claim that “at some point, a product is delivered to a user community, at which point, for that revision of the software, the design process is over.”Paul Dourish

Page 36: A Population Approach to Ubicomp Systems Design

“When we look at an interactive system as an evolving artifact in use, it follows that the process of design does not end with the delivery of the system to some community of users. Instead, it continues as they use and adapt the system.”

The software engineering revolution

Page 37: A Population Approach to Ubicomp Systems Design

Concept Demonstrator/Evaluation

Initial evaluation via a simple mobile game called Castles

Game chosen because

Keeps users motivated

In-depth and intense use stresses system

Provides an infrastructure suitable for modular architecture

Page 38: A Population Approach to Ubicomp Systems Design

Game Overview

Players construct kingdoms

Inhabitants

Resources

Buildings

Soldiers

Players may choose to battle when they are in wi-fi range

Page 39: A Population Approach to Ubicomp Systems Design

Game Overview

Players start with a set of common modules and a set unique to them

When players battle:

histories are exchanged

recommendations are made

modules are transferred

Page 40: A Population Approach to Ubicomp Systems Design

Game Overview

After a battle, a player will be informed if there are recommendations

Page 41: A Population Approach to Ubicomp Systems Design

Game Overview

Module recommendations integrated into the building list

Stars show recommendation rank

Gray buildings are newly received modules

Page 42: A Population Approach to Ubicomp Systems Design

Findings

Players progressed more rapidly with recommendations

Nearly all followed recommendations

Modules were spread to and used by others

Page 43: A Population Approach to Ubicomp Systems Design

FlexBook

Mobile social networking app

Dynamic integration of software modules at runtime

Shared via our “module store” server, or ad hoc peer-to-peer

Reviewing and user feedback

Implicit logging of use

Explicit comments and rating

Shown in app and Facebook

Page 44: A Population Approach to Ubicomp Systems Design

My predictions interface

210

open?!

3

4

5

6

7

8

dUIdUI

BP (update)?!

getF ix!!

9

rtnF ix?!

BP (sob)?!

getMatchaddPred!!

noMatch

BP (close)?!

closeApp

rtnfocus!!

Fixture server : FS1

2

10

getF ix?!

!

rtnF ix!!

Prediction Server : PS

21

0

addData!!

34

rtnData?!

getPred?!

getData!!addPred?!

rtnPred!!

1

Page 45: A Population Approach to Ubicomp Systems Design

A population approach to ubicomp system design

Based on mass participation & dynamic software structures

Redefining a fundamental computer science concept: class

A new project, officially starting today

Two universities, four million pounds, five years, nine people

Page 46: A Population Approach to Ubicomp Systems Design

The population approach

Multiple representations of a software class (or type)

Tools to use, create or adapt these representations

Social interactions and practices involving those tools

Page 47: A Population Approach to Ubicomp Systems Design

The traditional representation

A class is a structure, and all instances conform to it

Data structure: variables or state

Code structure: methods, procedures or functions

These form a structural description of potential behaviour and use

Assumption: exact match between the structure of the class definition, and the structure of each instance

Page 48: A Population Approach to Ubicomp Systems Design

The traditional representation

Tests and changes need only be done on one thing at one time: the class structure

What’s true for the class is true for the deployed instances

Poor fit with trends such as plug-ins, add-ins, e.g. what is Firefox?

Phones often user-adapted, via App Store, Cydia, Android Market...

Page 49: A Population Approach to Ubicomp Systems Design

Borrowing from biology: population thinking

A species is a varied and changing population of individuals

A species may be described by unifying and defining characteristics

A species is also continually changing and evolving

Each individual member may differ from others, and variation among individuals is natural and vital to adaptation

The species can also be described by the set of current (or recently recorded) members: the aggregate of each individual’s DNA, size, colour, shape, etc.

Page 50: A Population Approach to Ubicomp Systems Design

A new representation: population

A class is a varied and changing population of instances

A class may be described by unifying and defining characteristics

A class is also continually changing and evolving

Each individual instance may differ from others, and variation among individuals is natural and vital to adaptation

The class can also be described by the set of current (or recently recorded) members: the aggregate of each instance’s structure, context, use, etc.

Page 51: A Population Approach to Ubicomp Systems Design

A new representation: a population

Match between class and each instance may vary

The value of a variable may vary... but also which variables, methods and functions make up internal structure

Variation and dynamism mean complexity of design and analysis

Probabilistic statements and models instead of simpler discrete ones

Page 52: A Population Approach to Ubicomp Systems Design

Populations: variation and dynamism

What’s true for one instance may not be for another

90% of unmodified instances crash, but those with module A added don’t crash. Overall, 75% of instances crash

Analysis results may be different at different times

We advertise A and the percentages; a week later only 50% crash

Results for different contexts may differ

News of module A spreads faster in one country than another, so in Spain 25% crash while in the US 60% crash

Page 53: A Population Approach to Ubicomp Systems Design

New tools and new interactions

Analysis of modules, configurations and contexts

Which configurations and contexts are popular? Which are problematic? How are they used? Are there clusters that should be separately managed and designed for?

How does my setup compare to my friends’? Are there configurations like my own that doesn’t crash so much? How are they used?

Access to modules and usage data

May support free dissemination (P2P) or retain a degree of control

‘Lead users’ keen to try out new or specially instrumented versions

Page 54: A Population Approach to Ubicomp Systems Design

A third representation: ostension

One or more population members are pointed out as examples

Many potential ways to make an ostension

e.g. a category or cluster selected from a population of instances

Page 55: A Population Approach to Ubicomp Systems Design

SGVis: Categorising users

Page 56: A Population Approach to Ubicomp Systems Design

From observed use patterns to software structure

MapTool is usually run along with DGPSLocationStream

UKTrainSchedule is frequently logged as being used in UK locations tagged as train station

FestivalEvents and FestivalVenues modules are frequently used with an unofficial module AlternativeGuideToTheFringe when the user is in Edinburgh Fringe Festival venues

Page 57: A Population Approach to Ubicomp Systems Design

System design and structure

“An ontology is a formal specification of a shared conceptualization” Gruber, via SemanticWeb.org

Traditionally, design is about the perfect hierarchical structure

One structure for all known uses, contexts and people

Adaptation is the problem of changing it

Page 58: A Population Approach to Ubicomp Systems Design

System design and structure

Population approach couples process and structure

Design process in which structure is a resource for use, and in which use creates or adapts structure

Giddens’ duality of structure, Heidegger’s hermeneutic circle

Adaptation of a structure to new contexts and uses is part of the process

A design requirement for ubicomp

Page 59: A Population Approach to Ubicomp Systems Design

Designing for duality of structure

Moving from structure to use is commonplace

That’s what happens when you compile code and users run it

The trick is how to move back again

Making code for a class from a set of instances and usage histories

A process that allows for gradual adaptation of a class

Moving between complementary forms of class definition

Each move is (or should be) a social interaction

Page 60: A Population Approach to Ubicomp Systems Design

Intension

A class, compiled to create an executable shared among users

Extension

the deployed instances, that are adapted by users and which create log data shared with evaluators and developers (and other users)

Ostension

a subset of instances selected by one of these people as particularly interesting or useful, and shared among them for comment and use

analysis to a class signature reflecting ‘family resemblances’ among the configurations and logs of that subset

Page 61: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

FanPhoto: two core components, for text and photo sharing

We give it out to 100 people

and they start using it

We start developing new modules

one for sharing data via Facebook

one for fast compression and sharing of text, photos and video via MANETs

Page 62: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

A few participants are interested in new modules

download them and show off that they are trying them out

Analysts keep track via log data streaming back

start to see a new subcluster forming

973

Page 63: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

In the IDE, developers can see the FanPhoto class

Modules used by a minority of users are shown in grey

▽FanPhoto▷TextSharing▷PhotoSharing▷FacebookSharing▷MANETmodule

Page 64: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

After some time, developers and evaluators meet to play back several weeks’ use

973

Page 65: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

After some time, developers and evaluators meet to play back several weeks’ use

919

Page 66: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

After some time, developers and evaluators meet to play back several weeks’ use

5126

23

Page 67: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

After some time, developers and evaluators meet to play back several weeks’ use

560

30

FanBook

5

Page 68: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

After some time, developers and evaluators meet to play back several weeks’ use

560

30

Zippy

FanBook

5

Page 69: A Population Approach to Ubicomp Systems Design

Scenario: FanPhoto

A developer sets his IDE to do automatic updates

Code defining each new subclass appears

labelled with users’ and analysts’ names

linked to tools to analyse the ostensions that made them

FanPhoto

FanBook Zippy

Page 70: A Population Approach to Ubicomp Systems Design

Ostension revisited

Most forms of ‘analysis’ afford ostensive definition

A way to focus on a subset of structures, contexts, uses and users

e.g. an evaluator’s video of particular users from a system trial, a user’s list of friends on Facebook, a developer choosing a name of a city to customise design for

Log data allow us to link to the software used

Extend previous work that links from video to log data

Page 71: A Population Approach to Ubicomp Systems Design
Page 72: A Population Approach to Ubicomp Systems Design

Iterative design at a large scale

1. Development and refinement of tools, infrastructure, statistical methods, formal analysis methods and theoretical frameworks

2. Initial apps deployed by us to significant numbers of people: social networking and games

3. Larger-scale deployments developed/distributed with partners: Edinburgh Festivals, Rangers FC and other football clubs, Glasgow museums...

4. Go back to step 1 and do it better

Page 73: A Population Approach to Ubicomp Systems Design

A population approach

A combination of theory, design, evaluation and use

Advances in all four needed for success

A circular process designed for duality of structure

Change and human agency as essential elements of the process

Multiple ways of representing a software class

Tools to use, create or adapt these representations

Users’, evaluators’ and developers’ social interactions and practices involving those tools... that feed back into low-level representations

Page 74: A Population Approach to Ubicomp Systems Design

[email protected]/~matthew

560

30

Zippy

FanBook

5

My predictions interface

210

open?!

3

4

5

6

7

8

dUIdUI

BP (update)?!

getF ix!!

9

rtnF ix?!

BP (sob)?!

getMatchaddPred!!

noMatch

BP (close)?!

closeApp

rtnfocus!!

Fixture server : FS1

2

10

getF ix?!

!

rtnF ix!!

Prediction Server : PS

21

0

addData!!

34

rtnData?!

getPred?!

getData!!addPred?!

rtnPred!!

1