University of Chicago Argonne National Lab
Picosecond Time-of-Flight System
Clock Distribution Subsystem
thAugust 15, 2007
11 1John T. Anderson, Karen Byrum, Gary Drake,2223Henry J. Frisch, Jean-Francois Genat, Harold Sanders, Fukun Tang
1 High Energy Physics Division, Argonne National Laboratory, 9700 S. Cass Ave, Lemont, IL 2 Enrico Fermi Institute, University of Chicago, 5640 S. Ellis Ave, Chicago, IL
3 LPNHE, UNIVERSITE PIERRE ET MARIE CURIE PARIS VI, 4 Place Jussieu 75252 PARIS CEDEX 05
1) Version 0.1, initial draft of document, 5 August 2007.
2) Version 0.2, issued 9 august 2007.
a) Replaced Figure 3 sketch with better, to-scale version showing both DAQ and Analog
b) Modified clock chip discussion to include new information provided by Jean-Francois. c) Added new clock chip information about TI CDCE62005.
3) Version 1.0, issued 16 August, 2007.
a) First “real” version after review and input from rest of group.
b) Fixed title page to correctly show all authors and institutions involved in this effort. c) Revised System Introduction paragraphs per notes from H. Frisch.
d) Updated Figure 1 to clarify and add missing high voltage lines.
e) Corrected “link” terminology, explicitly defined each link in system diagram.
f) Added short textual description of each link, named each link.
g) Added requested table defining each signal in each link, including estimated connector
h) Stripped out technical details with formal language from section 2. Created new section 4
to hold technical detail information until such data is moved from this document into
separate engineering documents specific to each link or module.
i) Added LVDS implementation picture to complement those of LVPECL and CML already
j) Added additional introductory information describing the overall functions of the Modules,
including information about the Time Stretcher chip, in addition to that of the Links.
1. GENERAL INFORMATION
This document describes the Clock Distribution sub-system of the Picosecond Time-of-Flight system. The Clock Distribution sub-system encompasses the receipt of the system clock from a source external to Time-of-Flight system, through the generation of the Acquire Clock that is provided to the Time Stretcher chips.
1.1 System Introduction
The Picosecond Time-of-Flight electronics system consists of a custom ASIC and several commercial chips integrated with a micro-channel plate (MCP) photo-detector that is designed to measure the arrival time of charged particles with a design goal of one picosecond resolution. Effectively, the system acts as a "digital phototube"; all analog signal processing is performed directly by the circuits mounted on the MCP such that the only interfaces that connect to external data collection systems are digital control, digital data output and power supplies. The Picosecond Time-of-Flight system consists of a series of Modules which are connected by Links, where by the term Link we mean all of the interconnecting signals between two Modules. Figure 1 gives the overall system picture.
PMASTER CLOCKSYSTEM CLOCKBIAS VOLTAGEHOCTRL/DATAFPGA SAMPLE CLOCKTBULK DC POWEROANALOG CONTROLANALOG INPUTTHV BIAS POWERTIME DATAUBREGULATED DC & HVE
BOARDMCP LINKINTERBOARDCTL/PWR LINK
Figure 1 - Overall System Picture
1.1.1 Description of Modules in the System
The DAQ board holds interface chips, power supply regulators and an FPGA. Its purpose is to provide interface to the rest of the data acquisition system and provide infrastructure support to the Analog board. The Analog board connects directly to the MCP phototube. The Analog Board takes the Master Clock and uses a commercial clock distribution chip to create a 1GHz Acquire Clock. It contains a set of “Time Stretcher” chips that receive the fast signals from the anodes of the MCP tube. The Time Stretcher chips create Time Data pulses whose width is proportionate (200:1) to the delay between the arrival of the signal from the MCP and the next nd(or 2 next) edge of the Acquire Clock by means of an internal PLL and counter running at 5GHz. This is the “common stop” mode of time digitization. The FPGA on the DAQ board measures the
width of these pulses and provides digital data readout of the pulse width upon demand. 1.1.2 Description of Communications Links in the System
The CTL/PWR Link is the interface between the DAQ board and the rest of the experiment electronics. Bulk DC power is input to the DAQ board, where it is regulated for local
use and re-distributed to the Analog Board. High voltage bias for the MCP is provided external to the DAQ board and is routed through the two boards to the MCP without being used on either board. The System Clock is the main clock that is distributed to all picosecond time-of-flight systems, and is the source of all other clocks within this system. The Ctrl/Data signal(s) provide the method by which the experiment controls the Time-of-Flight system and obtains the data transmitted by the DAQ board.
The Interboard Link is a set of mated connectors between the DAQ board and the Analog board. The DAQ board generates the Master Clock from the System Clock, passes regulated DC and high voltage power to the Analog board, and generates any control signals needed by chips on the Analog board. In return, the Analog board generates an FPGA reference clock from the Master Clock and sends varying-width Time Data pulses to the FPGA on the DAQ board.
The Analog board mounts directly upon the MCP tube. The anodes of the tube provide analog signal input to the Analog board, and the tube‟s high voltage bias is provided by contacts
of the Analog board.
1.1.3 Tabular Description of all signals in each Link
The following tables enumerate the conductor counts and signal levels for all the defined signals in each Link. Specific description of the implementation of each Link (e.g. part numbers, connector types, wire gauges, shielding, etc.) are not listed here but are found in separate documentation specific to each Link.
Signal Name Number of Signal Levels Notes, Special Considerations
Conductors and/or Protocols
System Clock 2 LVDS Frequency must be integer divisor of
1GHz; 62.5MHz assumed for tests.
CTRL/DATA 6 LVDS Pair 1: control data into FPGA
Pair 2: readout enable in
Pair 3: readout data
Control & data are serial.
Bulk DC Power 4 +3.3V and GND All other voltages in the system are
regulated from the bulk +3.3V
HV Bias Power 2 3kV and GND Subminiature connector type may be
required due to board size. HV
GND is routed separately from
return planes of DAQ and Analog
boards, but is tied to LV GND at
Table 1 - Enumeration of signals within the CTL/PWR Link
Signal Name Number of Signal Levels Notes, Special Considerations
Conductors and/or Protocols
The total number of conductors in the Interboard Link is determined by the
number of pins in the connectors between boards. The counts below are
based upon the use of four connectors of 16 pins (total = 64 conductors).
Because of arcing concerns, the HV Bias Power is assumed to be carried on
a unique fifth connector.
Master Clock 2 LVDS Frequency is the same as System
Clock in CTL/PWR link.
FPGA Sample 2 3.3V LVPECL
Analog Control 16 +3.3V and GND Four status/control pins for the
clock chip, four serial bus pins to
control the clock chip, eight spares.
HV Bias Power 2 3kV and GND
Time Data 8 (4 pairs) LVPECL (2.5V? One pair for each of four Time
1.8V?) Stretcher chips on the Analog board.
Regulated DC 36 +3.3V, Two power, four return for each of
+2.5V/+1.8V the four Time Stretcher chips. Four
power + 8 return for the clock chip.
Table 2 - Enumeration of signals within the Interboard Link Signal Name Number of Signal Levels Notes, Special Considerations
Conductors and/or Protocols
Conductor counts are subject to change as the development of the Analog
Board progresses. The MCP Link is implemented using a special
compression connector (elastomer).
Bias Voltage 2 3kV and GND
Analog Input 16 + 4 Tiny analog Conductor count based on 64-
channel MCP tube with anodes
shorted together in groups of four.
Analog input returns (4) are
assumed to be grouped one per
Time Stretcher chip.
Table 3 - Enumeration of signals within the MCP Link
1.2 Distribution of System Clock and derivatives
The System Clock is received by a differential receiver on the DAQ Board. Within the DAQ Board the received clock is connected to an FPGA. Inside the FPGA a delay-locked-loop circuit (DLL) will repeat the System Clock within the FPGA for use as a control/data bus clock. The output of the DLL will also be driven differentially to the Analog Board. This copy of the 62.5MHz clock (the Master Clock) is received by a phase-locked-loop (PLL) on the Analog
Board. The PLL multiplies the Master Clock frequency by a factor of 16 to create a local 1GHz Acquire Clock that is distributed to each of four Time Stretcher chips, as shown in Figure 2.
62.5 MHzMASTER CLOCKTS CHIP+62.5 MHz_PLLFPGASYSTEM
TS CHIPCLOCK250 MHzREFERENCE CLOCK
DAQ BOARDANALOG BOARD
Figure 2 - Detail of clock distribution
Other input frequencies could be used but practically, the input must be an integer divisor of 1GHz. 62.5MHz is used here as an example. The Time Stretcher chips use the 1GHz clock to sample the Analog Input, creating the Time Data signals that are fed back to the FPGA on the
1.2.1 Master Clock Details
The Master Clock is simply a copy of the System Clock, passed through the FPGA on the DAQ board for jitter reduction using internal resources of the FPGA.
1.2.2 Acquire Clock Details
As the purpose of the system is to measure signals with picosecond timing accuracy, minimization of skew between the multiple copies of the Acquire Clock is critical. Similarly, the Acquire Clock must be delivered with minimal jitter. To achieve this, the PLL is located on the Analog Board to be as close to the time stretcher chips as possible with no intervening connectors that can introduce line discontinuities.
1.2.3 FPGA Reference Clock Details
The FPGA must perform the time-to-digital conversion of the output pulse from the Time Stretcher chips. It makes sense to provide another clock from the PLL to the FPGA that is phase- and frequency-locked to the 1GHz Acquire Clock. While it is possible to run small bits of logic within the FPGA at rates up to 450 MHz, bandwidth limitations on the clock input pins limit the external clock to lesser frequencies. Interconnection delay and input buffer delay will result in some phase shift but it should be relatively static. An alternative would be to use a PLL within the FPGA to generate its own 250MHz clock, but this generates the risk that the phase differential between the FPGA‟s PLL output and the Acquire Clock may wander over time or be different each time the system is power cycled.
2. CLOCK SUBSYSTEM THEORY OF OPERATION
The System clock is presumed to come from an external source that provides a large number of delay-matched copies of this signal from a central source. In usual installations many copies of this timing unit are expected to be installed; it is conceivable that the System Clock may be daisy-chained or used in a multi-drop connection, but both of these methods carry considerable risk of increased jitter and noise.
2.1 Power Up & Initialization
The Time-of-Flight system, upon power up or reset, waits for the System Clock to be present before sending the Master Clock to the PLL on the Analog Board. During this time the Acquire Clock will either not be running, or more likely, running at a frequency far lower than the normal speed of 1GHz. The Time Stretcher chips will have to be designed in such a way that lack of clock or one that is far out of range (too low) will not have deleterious effects upon power consumption or recovery to normal operation when the clock is available. As much as possible, the FPGA will try to hold the Analog Board‟s PLL in a known state until the System Clock is
2.1.1 Backup Clock
It may be advisable to consider inclusion of a backup clock source for the FPGA on the DAQ Board to insure that command and control features are still active and functional even in the absence of a System Clock.
2.2 FPGA action upon System Clock
The FPGA will receive the System Clock and use it to drive an internal Delay-locked loop (DLL) or, if available, an internal Phase-locked loop (PLL). The purpose of the DLL (PLL) is to reduce jitter and to provide an internal fan-out of the clock so that it may be used by the FPGA directly. The buffered copy of System Clock, named Master Clock, is transmitted to the PLL on the Analog Board as part of the Interboard Link.
2.3 PLL Action upon Master Clock
The PLL will receive the Master Clock and create from it four identical Acquire Clocks, routed individually to the four time stretcher chips. The Acquire Clocks will be at a frequency 16 times that of the Master Clock (for Master Clock = 62.5MHz, Acquire Clock = 1.0GHz). A fifth output, the Reference Clock, will be created that is at a frequency four times that of the Master Clock (for Master Clock = 62.5 MHz, Reference Clock = 250MHz).
2.3.1 Control of PLL circuit
The PLL circuit is controlled using a two-wire (IC) or three-wire (SPI) interface driven 2
by the FPGA on the DAQ board, plus whatever specific control lines are additionally required. The frequency multiplication ratios for both the Acquire clock and the Reference Clock should be programmable to allow for different System Clock frequencies.
2.4 Commercial clock chip compatibility with Acquire/Reference clock specifications
The Analog Devices AD9516-3 will accept an LVDS input and will provide six LVPECL outputs at 1GHz. Jitter is specified to be in the 400-500fs range. Four of the outputs could be used to drive the Time Stretcher chips and one for the FPGA. This part is relatively large. It is packaged in a 64-pin (9mm X 9mm) package.
Another option is the TI CDCE62005. This chip accepts an LVDS input and provides five LVPECL outputs in a 48-pin quad flat package. This is a new chip still in development, but TI has offered engineering samples. The preliminary documentation states that the outputs will work at 1GHz and that the jitter is less than 1ps RMS.
3. General notes on high speed transmission lines
The clock connections of this system are all high speed transmission lines. Very careful attention to high speed signaling concerns is paramount. This section will provide some basic information to assist designers and users in coping with these concerns.
3.1 LVDS signal levels
LVDS circuit architecture uses current steering to create 1s and 0s by driving a current one way or the other in a pair of conductors, developing voltage across a terminating resistor at the end of the line. The standard has been extended to use in bussed applications, but it was originally designed for, and it still best used in, point-to-point applications.
In normal usage the transmission line is chosen to have an odd-mode characteristic impedance of 100 ohms, and a 100 ohm resistor is used across the conductor pair as the terminator. With typical LVDS currents (5mA), this generates a differential voltage of about 500 millivolts at the input of the receiver. LVDS, unlike other similar high-speed standards, has been codified as an EIA standard (EIA 644). The most common reference to LVDS is the LVDS User‟s Guide published by National Semiconductor (found at
Figure 3 – Typical LVDS implementation
LVPECL is based on PECL, which is based upon yet older ECL. All “xxECL” interfaces use emitter-follower structures on the output that require pulldown resistors to a voltage appropriate to bias the output structure (and, if well designed, also terminate the line in its characteristic impedance). Generation of the termination voltage can present additional infrastructure requirements. PECL, or Positive ECL, is designed to run from a positive voltage rail and be terminated to the return; this is the opposite of ECL that uses negative voltage. LVPECL uses smaller current swings than PECL, but there is no codified standard.