Post on 25-Feb-2016
description
1Dirk Wiedner TWEPP2013
The trigger-less TBit/s readout for the
Mu3e experimentDirk Wiedner
On behalf of the Mu3e collaboration
24 Sep 2013
Dirk Wiedner, Mu3e collaboration 2
The Mu3e Signal
9/20/2012
• μ→eee rare in SM• Enhanced in:
o Super-symmetryo Grand unified modelso Left-right symmetric
modelso Extended Higgs sectoro Large extra dimensions
Rare decay (BR<10-12, SINDRUM)• For BR O(10-16)
>1016 muon decaysHigh decay rates
O(109 muon/s)
Dirk Wiedner, Mu3e collaboration 3
The Mu3e Experiment
9/20/2012
• Target double hollow cone• Silicon pixel tracker• Scintillating fiber tracker• Recurl station• Tile hodoscope
• Muon beam O(109/s)• Helium atmosphere• 1 T B-field
Dirk Wiedner, Mu3e collaboration 4
The Mu3e Experiment
9/20/2012
• Target double hollow cone• Silicon pixel tracker• Scintillating fiber tracker• Recurl station• Tile hodoscope
• Muon beam O(109/s)• Helium atmosphere• 1 T B-field
Dirk Wiedner, Mu3e collaboration 5
The Mu3e Experiment
9/20/2012
• Target double hollow cone• Silicon pixel tracker• Scintillating fiber tracker• Recurl station• Tile hodoscope
• Muon beam O(109/s)• Helium atmosphere• 1 T B-field
Dirk Wiedner, Mu3e collaboration 6
The Mu3e Experiment
9/20/2012
• Target double hollow cone• Silicon pixel tracker• Scintillating fiber tracker• Recurl station• Tile hodoscope
• Muon beam O(109/s)• Helium atmosphere• 1 T B-field
Dirk Wiedner, Mu3e collaboration 7
The Mu3e Experiment
9/20/2012
• Target double hollow cone• Silicon pixel tracker• Scintillating fiber tracker• Recurl station• Tile hodoscope
• Muon beam O(109/s)• Helium atmosphere• 1 T B-field
Dirk Wiedner TWEPP2013 8
Readout Requirements
24 Sep 2013
• 2.5 GHz muon decays• 50 ns readout frames (pixel)• O(5000) pixel chips• O(7000) scintillating fibers• O(7000) timing tiles• Online filtering
9
Timing Detectors
24 Sep 2013Dirk Wiedner TWEPP2013
• Scintillating fiber hodoscope
• Timing tiles• On detector zero-
suppression• Poster Session:
o STiC - A Mixed Mode Silicon-Photomultiplier Readout ASIC for Time-of-Flight Applications (Tobias Harion)
O(7000) fibersO(7000) tiles
Dirk Wiedner TWEPP2013 10
Silicon Pixel Detector
24 Sep 2013
• Inner double layer• Outer double layer• Re-curl layers
o Both sides (x2)• Sensor size
o 1x2 cm2 inner layerso 2x2 cm2 outer layers 180 inner sensors
4680 outer sensors
Dirk Wiedner, Mu3e collaboration 11
HV-MAPS
2/5/2013
• High Voltage Monolithic Active Pixel Sensors• HV-CMOS technology• Reversely biased ~60V
o Charge collection via drift Fast O(100 ns)o Thinning to < 50 μm possible
by Ivan PericI. Peric, A novel monolithic pixelated particle detector implemented in high-voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876
Dirk Wiedner, Mu3e collaboration 12
HV-MAPS
2/5/2013
• High Voltage Monolithic Active Pixel Sensors• HV-CMOS technology• Reversely biased ~60V
o Charge collection via drift Fast O(100 ns)o Thinning to < 50 μm possible
by Ivan PericI. Peric, A novel monolithic pixelated particle detector implemented in high-voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876
Dirk Wiedner, Mu3e collaboration 13
HV-MAPS
2/5/2013
• High Voltage Monolithic Active Pixel Sensors• HV-CMOS technology• Reversely biased ~60V
o Charge collection via drift Fast O(100 ns)o Thinning to < 50 μm possible
• Integrated readout electronicso Zero suppressiono 800Mbit/s serial LVDS outputs
by Ivan PericI. Peric, A novel monolithic pixelated particle detector implemented in high-voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876
Dirk Wiedner, Mu3e collaboration 14
HV-MAPS
2/5/2013
• High Voltage Monolithic Active Pixel Sensors• HV-CMOS technology• Reversely biased ~60V
o Charge collection via drift Fast O(100 ns)o Thinning to < 50 μm possible
• Integrated readout electronicso Zero suppressiono 800Mbit/s serial LVDS outputs
by Ivan PericI. Peric, A novel monolithic pixelated particle detector implemented in high-voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876
Dirk Wiedner TWEPP2013 15
Pixel Readout Scheme
24 Sep 2013
Dirk Wiedner TWEPP2013 16
Pixel Readout Scheme
24 Sep 2013
• Pixel logico Pixel address (8 bit)o Frame number (4 bit)o 50 ns frames
• Column logico Pixel datao Column addresso Coarse time
• Frame logico Super Frameo Contains 16 x 50 ns readout
frameso + Sensor header
• Readout buffer• Serializer and fast link(s)
Pixel address
Pixel Logic
Column Logic
Frame logicReadout buffer
Serializer
Fine time
Coarsetime
Columnaddress
Dirk Wiedner TWEPP2013 17
Pixel Readout Scheme
24 Sep 2013
• Pixel logico Pixel address (8 bit)o Frame number (4 bit)o 50 ns frames
• Column logico Pixel datao Column addresso Coarse time
• Frame logico Super Frameo Contains 16 x 50 ns readout
frameso + Sensor header
• Readout buffer• Serializer and fast link(s)
Pixel address
Pixel Logic
Column Logic
Frame logicReadout buffer
Serializer
8 bit
Fine time
4 bit12 bit
Coarsetime
Columnaddress
Dirk Wiedner TWEPP2013 18
Pixel Readout Scheme
24 Sep 2013
• Pixel logico Pixel address (8 bit)o Frame number (4 bit)o 50 ns frames
• Column logico Pixel datao Column addresso Coarse time
• Frame logico Super Frameo Contains 16 x 50 ns readout
frameso + Sensor header
• Readout buffer• Serializer and fast link(s)
Pixel address
Pixel Logic
Column Logic
Frame logicReadout buffer
Serializer
8 bit
Fine time
4 bit12 bit
Coarsetime
Columnaddress
12 bit8 bit
32 bit
Dirk Wiedner TWEPP2013 19
Pixel Readout Scheme
24 Sep 2013
• Pixel logico Pixel address (8 bit)o Frame number (4 bit)o 50 ns frames
• Column logico Pixel datao Column addresso Coarse time
• Frame logico Contains 16 x 50 ns readout
frameso + Sensor header Super Frame
• Readout buffer• Serializer and fast link(s)
Pixel address
Pixel Logic
Column Logic
Frame logicReadout buffer
Serializer
8 bit
Fine time
4 bit12 bit
Coarsetime
Columnaddress
32 bit
4 x serial @ 800 Mb/s
12 bit8 bit
Dirk Wiedner TWEPP2013 20
Data Link SchemeFrom detector slices
to time slices
24 Sep 2013
Dirk Wiedner TWEPP2013 21
Link Overview
24 Sep 2013
• Front end links o Pixel sensor to on-detector
FPGA• 400 – 800 Mbit/s• LVDS
o Timing detector readout• Optical links from detector
o Front end FPGAso … to readout boardso 5 Gbit/s
• Optical links in counting roomo Off-detector read out boardso …to PC Farm
Dirk Wiedner TWEPP2013 22
Link Overview
24 Sep 2013
• Front end links o Pixel sensor to on-detector
FPGA• 400 – 800 Mbit/s• LVDS
o Timing detector readout• Optical links from detector
o Front end FPGAso … to readout boardso 5 Gbit/s
• Optical links in counting roomo Off-detector read out boardso …to PC Farm
Pixel Sensor
Silicon FPGAs
x86
Readout board
x12
PCx48
Dirk Wiedner TWEPP2013 23
Link Overview
24 Sep 2013
• Front end links o Pixel sensor to on-detector
FPGA• 400 – 800 Mbit/s• LVDS
o Timing detector readout• Optical links from detector
o Front end FPGAso … to readout boardso 5 Gbit/s
• Optical links in counting roomo Off-detector read out boardso …to PC Farm
Pixel Sensor Fiber Tile Pixel
Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel
Sensor Fiber Tile
Silicon FPGAs
x86
FiberFPGAs
x48
TileFPGAs
x48
Readout board
x16
Readout board
x8
Readout board
x8
x6336 x7000 x7000
PCx48
O(8Tbit/s)
Dirk Wiedner TWEPP2013 24
Tile
Link Overview
24 Sep 2013
• Front end links o Pixel sensor to on-detector
FPGA• 400 – 800 Mbit/s• LVDS
o Timing detector readout• Optical links from detector
o Front end FPGAso … to readout boardso 5 Gbit/s
• Optical links in counting roomo Off-detector read out boardso …to PC Farm
Pixel Sensor Fiber Pixel
Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel
Sensor Fiber Tile
Silicon FPGAs
x86
FiberFPGAs
x48
TileFPGAs
x48
Readout board
x16
Readout board
x8
Readout board
x8
PCx48
x376
Dirk Wiedner TWEPP2013 25
Link Overview
24 Sep 2013
• Front end links o Pixel sensor to on-detector
FPGA• 400 – 800 Mbit/s• LVDS
o Timing detector readout• Optical links from detector
o Front end FPGAso … to readout boardso 5 Gbit/s
• Optical links in counting roomo Off-detector read out boardso …to PC Farm
Pixel Sensor Fiber Tile Pixel
Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel
Sensor Fiber Tile
Silicon FPGAs
x86
FiberFPGAs
x48
TileFPGAs
x48
Readout board
x16
Readout board
x8
Readout board
x8
PCx48
x376 x192 x192O(4Tbit/s)
Dirk Wiedner TWEPP2013 26
Link Overview
24 Sep 2013
• Front end links o Pixel sensor to on-detector
FPGA• 400 – 800 Mbit/s• LVDS
o Timing detector readout• Optical links from detector
o Front end FPGAso … to readout boardso 5 Gbit/s
• Optical links in counting roomo Off-detector read out boardso …to PC Farm
Pixel Sensor Fiber Tile Pixel
Sensor Fiber TilePixel Sensor Fiber TilePixel
Sensor Fiber Tile
Silicon FPGAs
x86
FiberFPGAs
x48
TileFPGAs
x48
Readout board
x16
Readout board
x8
Readout board
x8
PCx48x192
x96x96
O(4Tbit/s)
Dirk Wiedner TWEPP2013 27
Front End FPGAs
24 Sep 2013
• FPGAs on detectoro 86 (+96) pieces
• Receive sensor datao 108 LVDS inputs
• 5 Gbit/s outputso 8 optical linkso … to counting house
• Switching data between readout boards farms A-D
Front end FPGA
800 Mbit/sLVDS inx 108
5 Gbit/s optical
Readout board
A
Pixel Senso
r
Readout board
B
Readout board
C
Readout board
D
Dirk Wiedner TWEPP2013 28
Front End FPGAs
24 Sep 2013
• FPGAs on detectoro 86 (+96) pieces
• Receive sensor datao 108 LVDS inputs
• 5 Gbit/s outputso 8 optical linkso … to counting house
• Switching data between readout boards farms A-D
Front end FPGA
800 Mbit/sLVDS inx 108
5 Gbit/s optical
Readout board
A
Pixel Senso
r
Readout board
B
Readout board
C
Readout board
D
Dirk Wiedner TWEPP2013 29
Front End FPGAs
24 Sep 2013
• FPGAs on detectoro 86 (+96) pieces
• Receive sensor datao 108 LVDS inputs
• 5 Gbit/s outputso 8 optical linkso … to counting house
• Switching data between readout boards farms A-D
Front end FPGA
800 Mbit/sLVDS inx 108
5 Gbit/s optical
Readout board
A
Pixel Senso
r
Readout board
B
Readout board
C
Readout board
D
Dirk Wiedner TWEPP2013 30
Front End FPGAs
24 Sep 2013
• FPGAs on detectoro 86 (+96) pieces
• Receive sensor datao 108 LVDS inputs
• 5 Gbit/s outputso 8 optical linkso … to counting house
• Switching data between readout boards farms A-D
Front end FPGA
800 Mbit/sLVDS inx 108
5 Gbit/s optical
Readout board
A
Pixel Senso
r
Readout board
B
Readout board
C
Readout board
D
Dirk Wiedner TWEPP2013 31
Front end
FPGA
Readout Board
24 Sep 2013
• FPGA readout boardso 4 per sub-detector
• 5 Gbit/s optical inputso 16-28 inputs
• 10 Gbit/s optical outputo 12 outputs to PCs
• Switching networko A-D sub-farmso One output per PC
Readout board
5 Gbit/s Optical
x28
PC
10 Gbit/s Optical
PC
Sub-farm A
Front end
FPGA
Front end
FPGA
Front end
FPGA
PCx12
Dirk Wiedner TWEPP2013 32
Readout Board
24 Sep 2013
• FPGA readout boardso 4 per sub-detector
• 5 Gbit/s optical inputso 16-28 inputs
• 10 Gbit/s optical outputo 12 outputs to PCs
• Switching networko A-D sub-farmso One output per PC
Front end
FPGA
Readout board
5 Gbit/s Optical
x28
PC
10 Gbit/s Optical
PC
Front end
FPGA
Front end
FPGA
Front end
FPGA
PC
Sub-farm A
x12
Dirk Wiedner TWEPP2013 33
GPU-PC
24 Sep 2013
• PC with GPU• 10 Gbit/s Fiber input
o 8 inputs from sub-detectors
• Data filteringo Timing Filter on FPGAo Track filter on GPUo Data to tape < 100
MB/s
FPGA PCIe board
GPU computer
Optical mezzanine connectors
Dirk Wiedner TWEPP2013 34
GPU-PC
24 Sep 2013
• PC with GPU• 10 Gbit/s Fiber input
o 8 inputs from sub-detectors
• Data filteringo Timing Filter on FPGAo Track filter on GPUo Data to tape < 100
MB/s
GPU computer
Dirk Wiedner TWEPP2013 35
Readout board
Readout board
GPU-PC
24 Sep 2013
• PC with GPU• 10 Gbit/s Fiber input
o 8 inputs from sub-detectors
• Data filteringo Timing Filter on FPGAo Track filter on GPUo Data to tape < 100
MB/sPC
10 Gbit/s Optical
x8
PC
10 Gbit/s Optical
x8
A
Readout board
B
x8 Readout board
Readout boardReadout board
Dirk Wiedner TWEPP2013 36
Timing Filter
24 Sep 2013
• Entire event on PCIe FPGA• Tile and Fiber data
o Easy to matcho Look for three tracks
• Reject data without three hitso … inside time interval
1
3
2
Dirk Wiedner TWEPP2013 37
Timing Filter
24 Sep 2013
• Entire event on PCIe FPGA• Tile and Fiber data
o Easy to matcho Look for three tracks
• Reject data without three hitso … inside time interval
1
3
2
Dirk Wiedner TWEPP2013 38
Vertex Filter
24 Sep 2013
• Entire event on GPU• Large target
o Large spread of muonso Easy vertex separation
• Reject data without three trackso … inside area interval on target
1
3
2
Dirk Wiedner TWEPP2013 39
Vertex Filter
24 Sep 2013
• Entire event on GPU• Large target
o Large spread of muonso Easy vertex separation
• Reject data without three trackso … inside area interval on target
1
3
2
Dirk Wiedner TWEPP2013 40
Summary• Mu3e has 280M pixels @ >109 muons/s• >1 Tbit/s data • 0-suppressed serial data from active pixel sensors• Switched optical network• GPU filter farm with optical inputs
24 Sep 2013
+ +
Dirk Wiedner TWEPP2013 41
Backup Slides
24 Sep 2013
Dirk Wiedner, Mu3e collaboration 42
Physics Motivation
9/20/2012
Standard model:• No lepton flavor violation
Lepton flavor violation?
Dirk Wiedner, Mu3e collaboration 43
Physics Motivation
9/20/2012
Standard model:• No lepton flavor violation, but:
o Neutrino mixingo Branching ratio <10-50 →unobservable
Lepton flavor violation: μ+→e+e-e+
Dirk Wiedner, Mu3e collaboration 44
The Mu3e Signal
9/20/2012
• μ→eee rare in SM• Enhanced in:
o Super-symmetryo Grand unified modelso Left-right symmetric
modelso Extended Higgs sectoro Large extra dimensions
Dirk Wiedner, Mu3e collaboration 45
The Mu3e Background
9/20/2012
• Combinatorial backgroundo μ+→e+νν & μ+→e+νν & e+e-
o many possible combinations
Good time and Good vertex resolution
required
Dirk Wiedner, Mu3e collaboration 46
The Mu3e Background
9/20/2012
• μ+→e+e-e+ννo Missing energy (ν) Good momentum resolution
(R. M. Djilkibaev, R. V. Konoplich,Phys.Rev. D79 (2009) 073004)
Dirk Wiedner TWEPP2013 47
Pixel Sensor Links
24 Sep 2013
• Vertex Sensor chipso 180 chipso 4 LVDS linkso 800 Mbit/s per link
• Central Silicon Trackero 936 chipso 2 LVDS linkso 800 Mbit/s
• Recurl stationso 3744 chipso 1 LVDS linko 400 Mbit/s
Dirk Wiedner TWEPP2013 48
Pixel Sensor Links
24 Sep 2013
• Vertex Sensor chipso 180 chipso 4 LVDS linkso 800 Mbit/s per link
• Central Silicon Trackero 936 chipso 2 LVDS linkso 800 Mbit/s
• Recurl stationso 3744 chipso 1 LVDS linko 400 Mbit/s
Pixel Sensor
800 Mbits/s
Front end FPGA
Dirk Wiedner TWEPP2013 49
Front End FPGAs
24 Sep 2013
• FPGAs on detectoro 86 (+96) pieces
• Receive sensor datao 108 LVDS inputs
• 5 Gbit/s outputso 8 optical linkso … to counting house
• Switching between readout boards A-D
Optical transceiver FE board
Dirk Wiedner TWEPP2013 50
Average Occupancies
24 Sep 2013
• All numbers per frame of 50 ns • Vertex detector
o 2 hits per sensor• Central silicon tracker
o 0.6 hits per sensor• Recurl stations
o 0.13 hit per sensor• Fiber hodoscope
o 0.16 hits per fiber• Timing tiles
o 0.09 hits per tileTile occupancies
Dirk Wiedner TWEPP2013 51
Maximum Occupancies
24 Sep 2013
• All numbers per frame of 50 ns • Vertex detector
o 5 hits per sensor• Central silicon tracker
o 2 hits per sensor• Recurl stations
o 1 hit per sensor• Fiber hodoscope
o 0.24 hits per fiber• Timing tiles
o 0.14 hits per tile
φ
zVertex occupancies