The ALMA Software System Joseph Schwarz (ESO). ACS Course, 15 January 2007The ALMA Software System...

The ALMA Software The ALMA Software SystemSystem

Joseph Schwarz (ESO)

ACS Course, 15 January 2007 The ALMA Software System

ALMA is…

• A radio aperture-synthesis telescope covering ν50-1000 GHz (λ ~ 0.3 - 6 mm), and consisting of...

• 54 12m antennas and 12 7m antennas that will make detailed images of astronomical objects.

• They can be positioned as needed with baselines from 0.15 to 14 kilometers so as to give the array a zoom-lens capability, with angular resolution reaching 10 milliarcseconds.

• A leap of over two orders of magnitude in both spatial resolution and sensitivity over existing instruments

• ALMA's great sensitivity and resolution make it ideal for medium scale deep investigations of the structure of the submillimeter sky.

• A joint project of the North American, European and Japanese astronomical communities.

Courtesy of Al Wootten, ALMA/US Project Scientist

Location: the Llano de Chajnantor, Chile, at an altitude of about 5000m.


Complete Frequency Access


Where can such transparent skies be found?

Living Earth

ALMA

...or here

Here...


Performance requirements

• Baseline correlator H/W produces ~ 1 Gbyte/s• Must reduce to average/peak data rates of 6/60

Mbyte/s (baseline)• Raw (uv) data ~ 95%, image data ~ 5% of the total• Implies ~ 180 Tbyte/y to archive• Archive access rates could be ~5 higher (cf. HST)

• Feedback from calibration to operations• ~ 0.5 s from observation to result (pointing, focus, phase

noise)

• Science data processing must keep pace (on average) with data acquisition


Software Scope

• From the cradle… • Proposal Preparation• Proposal Review• Program Preparation• Dynamic Scheduling of Programs• Observation• Calibration & Imaging• Data Delivery & Archiving

• Afterlife: • Archival Research & VO Compliance


And it has to look easy…

• From the Scientific Software Requirements:• “The ALMA software shall offer an easy to use interface to

any user and should not assume detailed knowledge of millimeter astronomy and of the ALMA hardware.

• “The general user shall be offered fully supported, standard observing modes to achieve the project goals, expressed in terms of science parameters rather than technical quantities...”

• The expert must still be able to exercise full control• But what is simple for the user will therefore be

complex for the software developer.• Architecture should relieve developer of unnecessary

complexity• Separation of functional from technical concerns


System data flow

6-60 Mbyte/s

~0.5 s feedback time


Distributed development

ALMAOSF & AOS

DRAO

Penticton

ALMAATF

Arcetri

Observatory

Jodrell Bank

Brera

ObservatoryIRAM

Grenoble

ATC

Edinburgh

NAOJ

ESO

NRAO

Obs de

Paris

Univ.Calgary

c

c

SantiagoSCO

MPI Bonn

DAMIR/IEMMadrid

10


Distributed operations

~30 km

~15

km

This facility must operate 24 hours/day, 365 days/year

~30 km

~15

km

11


12


Distributed runtime

• Devices @ the AOS (5000m)• 66 Antennas

• Every antenna in ALMA has its own computer (ABM) and its own container on that computer.

• 40 devices (LRUs) per antenna• Each hardware device on the antenna is represented as a component in that

antenna’s container.

• Correlator

• 1000 LRUs in the ARTM

• Operations, Archive, Data reduction/analysis• At the OSF (2900m, 30 km from the AOS)

• ACS will have to manage > 3500 components


"What the deuce is it to me?" You say that we go round the sun.If we went round the moon it wouldnot make a pennyworth of difference

to me or to my work."

Separation of concerns

• Functional: Physics, algorithms, hardware…• PIs can concentrate on their research specialties

• Software encapsulates aperture synthesis expertise

• Technical: Communications, databases, etc.• Subsystem teams should concentrate on function

• Technical architecture should provide simple and standard ways to:

• Access remote resources

• Store and retrieve data

• Manage security needs

• Communicate asynchronously between subsystems, components


ALMA Common Software (ACS)

• Main vehicle for handling technical concerns• Framework for distributed object architecture

• Used all the way down to device level in control system• Built on CORBA, but hides its complexity

• Wraps selected CORBA services• Multi-language support: Java, C++, Python• Vendor-independent• High-quality open-source ORBs available (e.g., TAO)

• System evolving to meet developers’ needs• Initial developer resistance• Developers now asking for more

• Dedicated team of systems-oriented developers


Data modeling & code generation

• Define data structure and content abstractly• UML: Project (APDM) & Science (ASDM) data models

• Spreadsheets: Hardware device characteristics

• Open-source code generation framework• Wrapped & supplied by ACS

• Generate automatically:• XML schemas, Java, C++, SQL, docs from UML

• Binding classes from schemas

• Type-safe native language access to data

• Automatic validation possible

• Exchange of XML documents between subsystems• In a language-independent way

• Via the Archive or via the network


Avoiding nasty surprises

• Iterative development• Releases every 6-months (4 major, 3 minor to date)

• 6 major ACS releases to date

• Function-based teams develop & deliver integrated s/w on shorter time scales (typically, 3-4 FBTs per release cycle)

• Periodic “CDR”s: Focus on plans for next FBTs, next release

• Emphasize integrated system • Simulate interferometry while waiting for hardware

• We’ll be ready for first fringes at the ATF in Q2

• Two integrated user tests at the Antenna Test Facility (ATF) so far

• Optical pointing: feedback led to...

• Holography: greatly improved robustness, user satisfaction

20


ATF: Where the action is

VLA site, Plains of San Augustin, New Mexico, USA


ALMA Schedule

• 2007 Q2: • First fringes on prototype antennas at ATF (VLA site)

• First production antenna on site at OSF in Chile

• 2008 Q1: Second production antenna at OSF

• 2008 Q3: Two-antenna interferometer at OSF

• 2008 Q4: “Array software” complete

• 2009 Q1: • Three-antenna interferometer at AOS (@ 5000m)

• Start of commissioning

• 2010 Q4: Early science operations

• 2012 Q3: Hardware & “Observatory s/w” complete• Full science operations (full antenna complement)

The ALMA Software System Joseph Schwarz (ESO). ACS Course, 15 January 2007The ALMA Software System...

Documents

Transcript of The ALMA Software System Joseph Schwarz (ESO). ACS Course, 15 January 2007The ALMA Software System...