PROJET DAQGEN - indico.in2p3.fr · PROJET DAQGEN DAQGEN NEBULA ... JTAG SI5338 I2C CLK 4 F M C • On board configuration (µC) + • Very low noise synthesizer PLL synthesizer cleaner

1Daniel Charlet Journées IN2P3 Clermont-Ferrand -05-2018

PROJET DAQGEN

DAQGEN

NEBULA

IDROGEN

White Rabbit


ARCHITECTURE

Standard : xTCA for Physics Basée sur du matériel du commerce Contrôleur de châssis : MCH de N.A.T Lecture des données par le Backplane :PCIe 4x Gen3 ou Eth10G.

Transfert des données : Eth10G, PCIe-over cable (industriel) 100GEth (developpement DAQGEN)

Configuration : 1GEth par IP bus.

3

Carte NEBULA


Carte NEBULA

• MTCA 4.0 standard, Double-width, full size AMC.

• FPGA : 5AGTMC7G3F31• Stand alone mode (power)• ADC 2 channels 1GSPS.• White Rabbit compliant.• On board configuration (µC)• Very low noise synthesizer PLL synthesizer cleaner (LM04828)

• Front panel : WR SFP+2x SFP+ 10GbEth

• Backplane connectivity :Gbe IP bus,PCI 4x Gen3, IPMB, CLK & trigger lane.

10Gb

LMK04828

µC ATMEGA

Power CTRLDS1014

IPMI

DataSynch & config

SPI I2C

SPI

FPGA ctrl/cde & conf

SPI

SSRAM512K

Flash x 21Mb

EPLDMAXV

ADC 0810202x 1GSPS Input bandwidth :

2Ghz

Data/config

1Gb

config10Gb

WhiteRabbit

IPMB

WR

IPBus

PCIE x4

Eth

Eth

5

WR componentsPLL Cleaner Carte NEBULA

6

f =490MHz 4℮6 samples




Carte NEBULA, Test ADC

7

Carte NEBULA, systéme de test WR

8

Carte NEBULA, Test PPS

• En collaboration avec le laboratoire SYRTE (Obs Paris INSU).

• 400 Fs apres 1000s et 100 Km fibre• Tests à venir :

• long période d’analyse (Etude des dérives lentes

• Reproductibilité de l’ IP WR enfonction des firmware.

• Modification des fréquences de la DDMTD.

• Modification de la soft PLL

9

Carte IDROGEN


Carte IDROGEN


• FPGA : 10GX027H4F34• Stand alone mode (power 12v)• HighPinCount FMC slot.• White Rabbit compliant.

• Front panel connectivity : WR SFP+QSFP+ 40G, USB

• Backplane connectivity :1Gbe IPbus,PCI 4x Gen3, IPMB, CLK & trigger lane.

● RTM connector : J30.● Low cost

FM

C+

Eth-40G

Eth-1G

PCIe 4x Gen3/ETH 10G

AM

C c

onn

ecto

r

white-Rabbit

Eth-1G

QSFP+

SFP+

Data transfert

4

80 diff

Config. & CTRL

Eth-1G

Serial link 1G

IPBus

Data transfert &Config. & CTRL

8 GX

4

1

1

1

RT

M

28 diff

Clk synthesiserJitter cleaner

ARRIA 10


Carte IDROGEN

ARRIA 10 SX10AX027H4F34

IP bus Ethernet 1G

PCIEx 4x Gen3

IPMB

ATMEGA128EPCQ

SFP+

Ethernet 40G

RTM

WREthernet 1G

White rabbitEthernet 1G

Serial link 40G

spi

Uar

t

Uar

t

PPS / trig.Ext

MAX10

I2C

CypressFX2LP

JTA

G

AS

pro

g

LMK04828LT10104

CLK.Ext

CLK

54

1606

8

I2C SPI

QSFP+

I2C

I2C

USB

IDROGEN board

JTAG

SI5338

I2C

CLK

4 FM

C+• On board configuration (µC)

• Very low noise synthesizer PLL synthesizer cleaner (LM04828) for WR clkand derived clkl.

• Dedicated PLL for serial links● Integrated USBBlaster II.● FPGA configuration : Active serial,

IP bus.● External connectivity : PPS, Trigger,

Ext CLK.

12

VCCP, VCC

MAX8517 PLL 0.7a maxStand-alone mode

LTM4627

LTM4627

1,03v

AMC con. 12v

EN6347 Filter

VCCR_GXB

VCCT_GXB

1,8v

VCCPGMVCCBATVCCIO

VCCH_GXB

3.3v

0,9v Seq 2

Seq 3

WR

Filter

3.3v

Iout 15A

Iout 15A

3.3V Power-management

Seq 1

MAX8527

3.3v

Max 17545Ena

VCCA_FPLL

Filter

Other components

EN6360

Seq 4

Filter

Filter

12V FMCPower con. 12v

Max 17545 3.6v

3.3V FMC

1.8V FMC

12V RTM

Seq 5

Filter

Carte IDROGEN, arbre des alimentations

13

10AX027F35

DACAD5662

DAC 16bAD5662

VCTCXO25MHz

VCXO25MHz

DACAD5662DAC 16bAD5662

DAC SPI

LMK_CLK0

WR DMTDCLK T ou B

RFCLKL 10Gb

LMK04828

FMC_CLK1

GBTCLK

100MHz

FMC_CLK0

100 MHz

MAX 10

CLKB

REFCLKR

CLKT

LMK_CLK1

AMC_clk2

REFCLK&25 MHz

25 MHz

CLK_SMA

CLKT

LMK_CLK2clk_free

ref_in1

ref_in0

XT

CA

co

nne

ctor

AMC_ clk2

FMC clk2

FMC clk2

AMC_clk1

PCIECLK

SI 5338

156.25MHz

50 MHz

125 MHz

RFCLKL 1Gb

RTM

100 MHz

RTM_ clk2

RTM clk2

Carte IDROGEN, arbre d’ horloges

14

Eth1G

FPGA Avallon bus

AM

C c

onn

ecto

r

Eth1G

JTAG

IP_bus

MAX 10Cypress

EPCQ_L

US

B

JTA

G

FMCRTM

Swich JTAG

USB blaster II

JTA

G

AS

pro

g

RemoteUpdate IP

PCIePCIe

SFP+Serial link

MMC

Carte IDROGEN, configuration FPGA

15

FM

C

Eth-40G

Eth-1G

PCIe 4x Gen3/ETH 10G

AM

C c

onn

ecto

r

white-Rabbit

Eth-1G

QSFP+

SFP+

Data transfert

4

80 diff

Config. & CTRL

Eth-1G

Serial link 1G

IP_Bus

Data transfert &Config. & CTRL

8 GX

4

1

1

1

RT

M

28 diff

Carte IDROGEN, bus

16

Eth 1g

VCXO50Mhz

Ref clock generator

PLL 2PLL 1

LMK04828 Clock jitter cleaner

DACVCXO

25MHz

25MHzVCXO

DDMT clock generator

DAC

125MHz

DS18S20Temp &

Serial num

Endpoint

LatticeMicro32

Periph

Phy

Wishbonecrossbar

DDMT PLL

WR PTP core

Flash

FPGA

PLL

125MHz

SysCon

DDMTSoftPLL

PLL

62.5MHz

User core

125MHz

62.5MHz125MHz

https://www.ohwr.org/projects/white-rabbit

Carte IDROGEN, arbre horloge WR

17

Endpoint

Wishbonecrossbar

EB masterwrapper

Mini-NICWishbonecrossbar

RAM

Fabricredirection

LatticeMicro32

SoftPLL

SysCon

PPS

Periph

UART

Bulid ID

WishbonAvalon

Avaloncrossbar

EEPROM

Tunableoscillators

JTAGMaster

Phy

WR PTP Core

Ethernet 1GWR

Wishbonecrossbar

RAMhttps://github.com/GSI-CS-CO/bel_projectshttp://www.ohwr.org/projects/white-rabbit/wiki/WRReferenceDesign

Firmware White Rabbit

18

Phy

WR PTP Core

Data acquisition

Etht 40G

WR Eth 1GEtherbone &Avalon Bridge

WR clkPPS

TBI/serdes

Avalon MM M

WB M

SoSi

WB S

FMCAvalon MM S

SiSo

Extoscillator

Data transfer

Avalon ST So

Avalon ST Si

PCIe

Avalon MM S

Avalon MM S Avalon MM S

IP BusEtht1G

Firmware IDROGEN


Carte IDROGEN


WHITE RABBIT


An extension to Ethernet which provides:– Synchronous mode (Sync-E)– Deterministic routing latency

● Sub-nanosecond synchronization in WR is achieved by using the following three technologies together:– Precision Time Protocol (IEEE1588).– Synchronous Ethernet.– DMTD phase tracking.

Open hardware (CERN )


● Synchronizes local clock with the master clock by measuring and compensating the delay introduced by the link.

● Link delay is measured by exchanging packets with precise hardware transmit/receipt timestamps.

Precision Time Protocol (IEEE1588)


Synchronous Ethernet

–

● All network nodes use the same physical layer clock,generated by the System Timing Master.

● PTP is used only for compensating clock offset.● Having the same clock frequency everywhere enables

phase detector technology as the means of measuring time.


Phase tracking

Measure the phase shift between transmit and receive clock onthe master side, taking the advantage of Synchronous Ethernet.

Monitor phase of bounced-back clock continuously.Phase-locked loop in the slave follows the phase changesmeasured by the master.


Digital Dual Mixer Time Domaine phase detector


Documentation

.

• En principe sur ATRIUM•Mais actuellement problèmes d’accès avec RENATER……….


Software DCOD framwork

DCOD INTEREST

E. Legay - V. LafageALATO - NETWORK, DATA STORAGE AND PROCESSING

Server 1 Server 2

Blue producer

Blue to yellow filter

Yellow consumer

PMHPMH

Blue data

Yellow data

PMHPMH

Yellow data

CTLCTL

TL Yellow

CTLCTL

TL Yellow

External application

POSIX interface

TL Alien

External applicationor electronic device

Alien data

Access to data through PMH:• Buffer by buffer (as

Narval)• Event by event

• DCOD tools :• DCOD lancher : a process to control them all• DCOD monitor (Distributed framework)

• Implemented on further detectors : AGATA, VAMOS,….

• DCOD Framwork • PMH – Posix Memory Handler• CTL Common Transport Layer• Narval acquisition : distributed acquisition system• ENX : slow control et configuration (unified interface)

Standalone Narval capability on personnal computer


Conclusion

Carte en cours de routage, test en fin d’année.

Firmware développé sur la carte d’évaluation (ATILA sce REFLEX).

Le portage de DCOD sur IDROGEN sera testé sur la carte d’évaluation par le

CSNSM.

IDROGEN Disponible début 2019.

Mezzanine FMC basée sur open hardware IDROGEN sera développée pour

projet AGATA.


Carte IDROGEN

ARRIA 10 SX10AX027H4F34

IP bus Ethernet 1G

PCIEx 4x Gen3

IPMB

ATMEGA128EPCQ

SFP+

Ethernet 40G

RTM

WREthernet 1G

White rabbitEthernet 1G

Serial link 40G

spi

Uar

t

Uar

t

PPS / trig.Ext

MAX10

I2C

CypressFX2LP

JTA

G

AS

pro

g

LMK04828LT10104

CLK.Ext

CLK

54

1606

8

I2C SPI

QSFP+

I2C

I2C

USB

IDROGEN board

JTAG

SI5338

I2C

CLK

4 FM

C+


• FPGA : 10GX027H4F34• Stand alone mode (power 12v)• HighPinCount FMC slot.• White Rabbit compliant.• On board configuration (µC)• Very low noise synthesizer PLL synthesizer cleaner (LM04828)

• Front panel connectivity : WR SFP+QSFP+ 40G, USB

• Backplane connectivity :1Gbe IP bus,PCI 4x Gen3, IPMB, CLK & trigger lane.

● RTM connector : J30.● Integrated USBBlaster II.● FPGA configuration : Active serial,

IP bus.● External connectivity : PPS, Trigger,

CLK.● Low cost


1 55 Signaux 100 Ohm

2 35 GND


4 35 PWR 3.3v

5 17 Signaux 50 Ohm

6 70 GND

7 70 PWR 0,9v/12v/1,8v

8 17 Signaux 50 Ohm+PWR

9 35 GND

10 17. Signaux 100 Ohm

11 35 GND


0,460mm

75 1

100 2

100 3

90 4

75 5

64 6

75 7

90 8

100 9

100 10

75 11

954

1,4mm

Empilement

Documents

PROJET DAQGEN - indico.in2p3.fr · PROJET DAQGEN DAQGEN NEBULA ... JTAG SI5338 I2C CLK 4 F M C • On board configuration (µC) + • Very low noise synthesizer PLL synthesizer cleaner