Overview Core technology Available technology Next steps...Beam Synchronous Timing - WR2RF card...
Transcript of Overview Core technology Available technology Next steps...Beam Synchronous Timing - WR2RF card...
CERN, November 2019 - John Gill
Overview
•Overview
•Core technology
•Available technology
•Beam Synchronous Timing
•Next steps
Overview - RFoWR
RFCapture
RF in
Overview - RFoWR
Deterministic Latency - Δt
RFCapture
Phase
Amplitude
Frequency
RF in RF out
Overview - RFoWR
Deterministic Latency - Δt
RFCapture
Phase
Amplitude
Frequency
RF in RF out
Available technology - RFoWR systems
RF over White Rabbit is not new. Two types of system have been developed at CERN:
• The first uses an ADC to digitise an RF signal. Currently unsupported.
https://wikis.cern.ch/display/HT/SPS+RF+distribution+over+White+Rabbit
• The second uses a PLL subsystem to lock and track the incoming RF signal.
https://www.ohwr.org/projects/wr-d3s/wiki
• The third is virtual ! Just transmit phase and FTW setpoints for a future TAI timestamp. Enables arbitrary RF signal or clock distribution.
PLL based-systems are used at the ESRF (production) facility in Grenoble and are being developed at SPring-8.REFURBISHMENT OF THE ESRF ACCELERATOR SYNCHRONISATIONSYSTEM USING WHITE RABBIThttp://inspirehep.net/record/1656131/files/tucpl01.pdf?version=1
Available technology - RFoWR systems
FMC-DAC600M Version 3.0 achieves:
• < 3 ps RMS jitter (100 Hz to 5 MHz) • < 7 ps RMS jitter ( 10 Hz to 5 MHz)• < 12 ps RMS jitter ( 1 Hz to 5 MHz)
for RF signals between 10 - 70 MHz.
https://www.ohwr.org/project/fmc-dac-600m-12b-1cha-dds/wikis/home
A cleaner PLL can also be employed on this receiver to multiply the DAC frequency up to ~500 MHz.
FORM FACTORS: PCIe and VME with SPEC and SVEC carriers
÷ X
Core technology - Direct Digital Synthesis overview
Core technology - Direct Digital Synthesis - phase to amplitude
Time
Time
Addr
ess
of L
UT
FTW
PHASE ACCUMULATOR
PHASE TO AMPLITUDE CONVERSION
LUT
Valu
es
FTW
Output frequency of a DDS can be described as function of the phase accumulator size, frequency tuning word and the clock frequency of the DDS (e.g. WR clock frequency @ 125 MHz).
OUTPUTFREQUENCY
Core technology - Direct Digital Synthesis and White Rabbit
+
ACCUMULATOR
FTW
LUT
DAC
FPGA
If each slave replays FTWs when using free running local oscillators/PLLs, their Numerically Controlled Oscillators (NCO) cannot be guaranteed to be aligned.
NCOs without WR timing
NCOs with WR timing
Beam Synchronous Timing (BST) - SPS + LHC RF distribution system
Beam Synchronous Timing (BST) system.
BST information transmitted via a Timing Trigger and Control (TTC) network.
With SPS - LLRF upgrade we need a new card to generate bunch and turn clocks to drive the BST master.
BST Master
TTC network
BST Receivers (BOBR)
BST-Master BOBR TTC
Beam Synchronous Timing - WR2RF card proposal
Beam Synchronous Timing - WR2RF card proposal
FTW
Beam Synchronous Timing - WR2RF card proposal
BST Master
TTC network
BST Receivers (BOBR)
FTW
WR2RFBunch + Turn Clocks
Beam Synchronous Timing - WR2RF card proposal
BST Master
TTC network
BST Receivers (BOBR)
FTW
WR2RFBunch + Turn Clocks
FTW
WRswitch network
In the future, a BOBR replacement card could
receiveFTWs directly
from the SPS LLRF.
RFoWR receivers
Next steps
Conclusion
The RFoWR user base is increasing: ESRF SHINE SPring-8 and CERN
Ongoing effort with Radio over Ethernet (RoE) IEEE 1914.3 committee to accept our proposals for a structure-aware DDS protocol.
SPS - LLRF system will be a high profile user of RFoWR and a new WR2RF card will be developed to drive the existing BST system.
Beam Synchronous Timing - WR2RF card functions
WR NetworkFTW
FTW
FPGA - Kintex 7
Osc 10 MHz
Osc 1 GHz
RF Signal
SFP
WRCore
ConfigurableNCO + IQ
Clean-upPLL
Clock Multiplier
DACDDS
OCXO
Clk125/250 MHz
FTW
LO
IF
WR2RF card
LLRF
Beam Synchronous Timing - SPS LLRF upgrade
• WR2RF receiver card to be a hybrid version of the eTRM-14/15 cards.
• Plan to use kintex 7 FPGA as on the eTRM-14.
• These cards are already part of the planned SPS LLRF system, so reusing analog circuitry and HDL logic should lower risk.
Beam Synchronous Timing - SPS - LLRF upgrade
• Prototype boards developed for the eRTM-14/15 project have shown:
• White Rabbit clocks are generated with 70 fs RMS jitter (10 Hz and 5 MHz).
• Additional noise from the DDS increases the jitter to 100 fs RMS jitter (10 Hz to 5 MHz).
• We anticipate that the noise for the whole system will be < 1 ps RMS jitter (10 Hz to 5 MHz).
Beam Synchronous Timing - bunch and turn clock generation
• Clocking FPGA logic with the recovered RF signal provides a method to create beam synchronous logic.
• Generating an in-phase bunch clock requires using the regenerated RF signal as a clock.
• Once we have a counter in the RF clock domain, we can coordinate triggers + pulses throughout a WR network.
• RF time.
ACCUMULATOR
FTW
LUT
DAC14b
FPGA
+
RFtime Counter
WR Clock Doman
RF Clock Domain
Div m
Bunch Clock
+
Beam Synchronous Timing - What is the BOBR - main functions
• The BOBR card produces local copies of the bunch and turn clocks.
• Distribute triggers and produces pulses that are beam synchronous.
• Distributes LHC telegram information.
BST-Master BOBR TTC
LHC-
B2
L
HC-
B1
SPS
Beam Synchronous Timing - time of flight compensation
+
ACCUMULATOR
FTW
LUT
DAC+
LUT
Valu
esAd
dres
s of
LUT
• Protons and particularly ions in the SPS accelerator are subject to changes in their Time of Flight (ToF).
• Correcting the RF (and bunch clock) for ToF should be straightforward as it can be expressed as a phase offset to be applied to the output of the NCO.
Beam Synchronous Timing - SPS LLRF is tricky
• SPS RF is dynamic. This makes it difficult to utilise simple NCO driving a DAC approach.
• The BE-RF have developed an approach using a multi-level NCO that compensates for:
• Slip-stacking for ion fills.
• Frequency Shift Keying (FSK).
Beam Synchronous Timing - TTC issues
• TTC network subject to diurnal and seasonal changes in signal delay.
• TTCrx PLL cannot operate with FSK that is utilised in SPS ion fills. PLLs unlock, SPS BST uses average of FSK RF signals.
• Ions experience large accelerations in the SPS and their time of flight is not compensated.
All of the above results in the bunch clock, and the bunch, no longer being synchronous with each other. This makes beam monitoring (position and loss) difficult.
RFtime - As a Coordinate System for Synchrotrons Beam Synchronous Timing - RF Synchronous Events, Pulses and Triggers
Flat Top
Flat Bottom
Ramp
Time
Freq
uenc
y
The input RF is not necessarily constant. The frequency has to change as particles are accelerated.
The RF at injection (flat bottom) into the LHC is ~800 Hz (protons) or ~4600 Hz (ions) less than the frequency reached at flat top.
Bunch clock with varying frequency
Bunch clock with FSK
0 1 2 3 4 5
... 69 70 71 72 73 74 75 76
By expressing events and triggers in RFtime, we remove the need to express events in TAI time and avoid complicated calculations for each receiver.
RFtime
RFtime = #turn x RF harmonic + #bunch
Beam Synchronous Timing - LHC telegram data requirements
Note update frequencies• TURN• CHANGE• 1 Hz - LHC Telegram
GPS Timestamp can be deletedWR provides this.
PLUS BI SPECIFIC DATA
Beam Synchronous Timing - Latency
• Each White Rabbit switch will contribute up to 10 usec of additional latency to the output signal. In some respects the switch appears as a long cable (up to 2 km).
• A reasonable estimate of this latency is assumed to be 100 usec, or approximately 4 turns of the SPS.
• The interaction of a dynamic RF and latency in reproducing that RF will result in a systematic and a random error to the delay to this recovered RF signal.
• Smaller latency is better.
Core technology - Flexible Frequency Synthesis
+
ACCUMULATOR
FTW
LUT
DAC
FPGA
LPF
Core technology - Flexible Frequency Synthesis
+
ACCUMULATOR
FTW
LUT
DAC
FPGA
LPF
BPFPLL