49
CERN-THESIS-2014-265 22/01/2015 Institut Supérieur Industriel de Bruxelles Rue Royale 150 1000 Bruxelles Rue des Goujons 28 1070 Bruxelles www.isib.be Enseignement Supérieur de Type Long et de Niveau Universitaire Haute Ecole Paul-Henri Spaak Catégorie Technique CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA POUR LA QUALIFICATION ET L’AUTOMATISATION DE TESTS DANS LE CADRE DE LA PRODUCTION DE CARTES D’ACQUISITION NUMÉRIQUE AU CERN FPGA and software design for the qualification and automated tests for the production of CERN digital acquisition boards M. Houssem LAROUSSI Travail de fin d’études Effectué au sein du : CERN CH-1211 Geneva 23, Suisse Présenté en vue de l’obtention du grade de Master en Sciences de l’Ingénieur Industriel en Electronique ———————— Année Académique 2013-2014 ———————— Numéro : ISIB-ELIN-TFE-14/06 Classification : TOUT PUBLIC ————————————

CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

CER

N-T

HES

IS-2

014-

265

22/0

1/20

15

Institut Supérieur Industriel de Bruxelles

Rue Royale 150 — 1000 Bruxelles Rue des Goujons 28 — 1070 Bruxelles

www.isib.be

Enseignement Supérieur de Type Long et de Niveau Universitaire

Haute Ecole Paul-Henri Spaak Catégorie Technique

CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA POUR LA QUALIFICATION ET

L’AUTOMATISATION DE TESTS DANS LE CADRE DE LA PRODUCTION DE CARTES D’ACQUISITION

NUMÉRIQUE AU CERN

FPGA and software design for the qualification and automated tests for the production of CERN

digital acquisition boards

M. Houssem LAROUSSI

Travail de fin d’études

Effectué au sein du :

CERN

CH-1211 Geneva 23, Suisse

Présenté en vue de l’obtention du grade

de Master en Sciences de l’Ingénieur Industriel

en Electronique

————————

Année Académique 2013-2014

————————

Numéro : ISIB-ELIN-TFE-14/06

Classification : TOUT PUBLIC

————————————

Page 2: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Abstract

The VFC-HPC is an FPGA-based multipurpose board designed to be the newstandard back-end for the Beam Instrumentation (BI) Group. This contributionis aimed to give a presentation of the test and quali�cation system designed tobe used to verify the compliance of the VFC-HPC to the speci�cations.

We will begin by presenting the CERN, the company, its main purpose andits brief history.

Afterward, we will present the VFC-HPC, the purpose of this board and itsmains characteristics.

We will then focus on the work done to allow the test and quali�cation of theVFC-HPC. This project was focused on the test and quali�cation of the connec-tivity of the board, consisting of the FMC Connector and the SFPs. This workinvolved making several market surveys to choose the proper test componentsto be used (as FMC Mezzanines), setting up a test bench, implementing opticalcommunications through SFPs, writing Verilog �rmwares, writing software testroutines in Python and measuring Bit Error Rates (BER) and eye diagrams.

Finally, we will conclude this thesis by giving a summary of the work done,the most important results and the future prospects.

KEYWORDS: Back-end electronics; Data acquisition concepts; High-speedserial links; Digital electronic circuits

Page 3: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Acknowledgments

I would like to take pro�t of these few lines to thank Laura SAULNIER,Madeleine CATIN and the Human Resources department for taking care ofall administrative procedures to facilitate my arrival in the region.

Afterward, I would like to thank Thibaut LEFEVRE, my section leader, forbeing available and facilitate my integration inside the section, Andrea BOC-CARDI and Manoel BARROS MARIN for being patient (a lot of patience wasneeded!) and sharing their knowledge, experience and books to �ll my gapsin the electronic �eld. They were available and really helpful to me for work-ing on this project. A big thank you also to Drasko DRASKOVIC and SilviaZORZETTI for sharing their tips, experience and feedbacks.

Finally, I would like to thank the whole section and colleagues that I havebeen working with for their warm welcome and cheerfulness. Thanks to you,these 12 months have been wonderful!

Page 4: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Contents

1 Introduction 3

1.1 The CERN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 The BI group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 The VFC-HPC 5

2.1 Structure of the board . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 The FPGA . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.2 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.3 The Memories . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2 Speci�cations and planning . . . . . . . . . . . . . . . . . . . . . 10

3 Test and Quali�cation 13

3.1 General test setup . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.2 The �rmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.3 The FMC connector . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.3.1 Speci�c hardware . . . . . . . . . . . . . . . . . . . . . . . 15

3.3.2 The user de�ned pins test . . . . . . . . . . . . . . . . . . 19

3.3.3 The user de�ned pins quali�cation . . . . . . . . . . . . . 20

3.3.4 The multi-gigabit transceiver data pairs test . . . . . . . . 22

3.3.5 The multi-gigabit transceiver data pairs quali�cation . . . 31

3.3.6 The I2C test . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.3.7 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4 The SFPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.4.1 The SFPs test . . . . . . . . . . . . . . . . . . . . . . . . 35

3.4.2 The SFPs quali�cation . . . . . . . . . . . . . . . . . . . . 37

1

Page 5: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

4 Conclusion 43

2

Page 6: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Chapter 1

Introduction

The main purpose of this thesis is to describe the work done during my technicalstudentship at CERN which was the test and quali�cation of the VME FMCCarrier for High Pin Count (VFC-HPC) mezzanines, a Field-ProgrammableGate Array (FPGA)-based multipurpose board designed to be the new standardback-end for the Beam Instrumentation (BI) Group.

In the following chapters of this thesis are presented the CERN, the BIGroup, the VFC-HPC and the work done to allow the test and quali�cation ofthe board.

1.1 The CERN

The European Organization for Nuclear Research (CERN) is an internationalorganization created with the main purpose of studying the physic of fundamen-tal particles. It consists of a laboratory open to its member states to performexperiments in this �eld. This laboratory is a complex made of several exper-iments and accelerators, among which the Large Hadron Collider (LHC), thebiggest and most powerful accelerator built in the world[1].

Founded in 1954, it was one of the �rst collaborations between Europeancountries and has now 21 member states, including Belgium. It is situatedacross the Franco-Swiss border, near Geneva. The acronym CERN is derivedfrom "`Conseil Europeen pour la Recherche Nucleaire"' which means "`EuropeanCouncil for Nuclear Research"', name coming from the provisional body foundedin 1952 with the aim of founding a fundamental physics research organizationin Europe.

1.2 The BI group

The Beam Instrumentation (BI) Group is bounded to designing, building andmaintaining the instruments in charge of observation and monitoring of the par-

3

Page 7: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

ticle beams for all CERN accelerators[2]. It is part of the Beam (BE) Depart-ment which is in responsible for the beam generation, acceleration, diagnostics,controls and performance optimization for all CERN accelerators.

The BI Group is, commonly, in charge of measuring and monitoring thefollowing parameters:

• Beam position

• Beam loss

• Transfer line to accelerator injection point matching

• Beam intensity

• Beam pro�les (transverse and longitudinal)

The BI Group consists of several sections, whose the QP section which Ijoined for my project.

The accelerators and transfer lines composing the CERN complex are re-markably di�erent, making the use of the same instruments very laborious.Therefore, it has been decided to standardize the BI instrumentation. Theuse of such an approach provides several conveniences: It a�ords the possibil-ity to share spares and already developed �rmwares between di�erent systems,it reduces the amount of bugs brought with every new hardware design andestablishes a more standard interface to the software developers[2].

To allow this standardization, a �rst undertaking has been done in the soft-ware section with the development of the Front End Software Architecture(FESA). The purpose was to give a universal framework to all the Softwaredevelopers and to the end users[2]. A comparable undertaking has been donefor the interfacing between the front end software and the back end electronicsby providing a universal carrier. This carrier, a VME FPGA-based high-speeddigital acquisition board, is the VME FMC Carrier for High Pin Count (VFC-HPC) mezzanines.

4

Page 8: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Chapter 2

The VFC-HPC

As previously mentioned, the VME FMC Carrier for High-Pin Count mezza-nines (VFC-HPC, �gure 2.1) is an electronic board developed in the BI group.

2.1 Structure of the board

The main core of the VFC-HPC is an FPGA around which are arranged andconnected several chips (see �gure 2.2) making the architecture of the boardvery �exible. In terms of connectivity, the VFC-HPC has the following charac-teristics:

• One VME64x interface

• A Rear Transition Module (RTM) on the P2 connector with 40 directconnections with the FPGA

• One FPGA Mezzanine Card (FMC) slot

• General purpose Input/Output (GPIO) on four lemo connectors

• Six small form factor pluggable (SFP) on the front panel. One is foreseenfor beam synchronous timing (BST) distribution, the other one is used forEthernet communication implementation while the four others are userde�ned dedicated.

• Two SRAM memories

2.1.1 The FPGA

A �eld-programmable gate array (FPGA) is an integrated circuit capable ofbeing con�gured after manufacturing[3]. An FPGA is usually con�gured using

5

Page 9: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 2.1: Picture of the VFC-HPC

6

Page 10: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 2.2: Connectivity of the VFC-H: Block diagram

a hardware description language (HDL). For this project, the HDL languagethat has been used is the Verilog[4].

The chosen FPGA for the VFC-HPC is the Altera Arria V GX. The mainfeatures of this FPGA are the following[5]:

• 300K logic elements

• 15462400 Memory bits

• 1517 pins

• 704 I/Os

• 18 6-Gigabits per second (Gbps) transceivers

• 920 DSP blocks

• 12 Fractional PLL

• 1 PCIe Hard Intellectual Property (IP) block

• 2 Memory Controllers

2.1.2 Interfaces

As the VFC-HPC is a multi-purpose board, it must carry several interfaces inorder to comply with the needs of the several users.

7

Page 11: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

The communication with the control and monitoring systems is providedby the VME interface. However, it can also be allowed via the Ethernet SFP,making the board capable of working out of a crate in case of stand aloneapplications or need of dedicated data transmission channels.

2.1.2.1 The VME interface

The VERSA-module Europe (VME) interface implements the ANSI/VITA 1-1994 standard[6]. It de�nes an asynchronous parallel bus protocol with 4-edgehandshaking. It is based on the Master-slave principle with interrupt handlingand implementing the TTL voltage level. It has been developped by Motorolain 1981.

The VME bus is used in many CERN systems because it is an establishedstandard that provides mechanical, electrical and protocol speci�cations. Thisprotocol is simple to achieve and di�erent cards sizes are provided. Finally, thereare a lot of products as crates and I/O modules sustained by the vendors avail-able on the market. Currently, more than 1000 VME systems are implementedat CERN[7].

The BI Group decided to keep VME as protocol for the VFC-HPC alsobecause of the number of crates already installed and it still complies the Groupspeci�cations[2].

2.1.2.2 The FMC slot

The FPGA Mezzanine Card (FMC, �gure 2.3) slot is based on the ANSI/VITA57.1 standard[8]. Its purpose is to exploit the �exibility of the FPGA byproviding the less possible constraints. For example, the standard foresees nobu�er presence between the FPGA and the Mezzanine board in order to exploitthe great �exibility o�ered by modern FPGA I/Os in terms of programmablestandards, voltage levels and directions.

The FMC standard provides 2 connector types: the Low Pin Count (LPC)and the High Pin Count (HPC). Geometrically, they are identical but the HPCimplements more connections than the LPC. The VFC-HPC, as the name sug-gests, implements the HPC connections. This allows the use of LPC Mezzanineson HPC Carriers.

The FMC connector has the following characteristics:

• 80 di�erential/160 single ended user de�ned pins split in three banks (La,Ha and Hb)

• 10 multi-gigabit transceiver data pairs

• I2C interface[9]

• Joint Test Action Group(JTAG) interface[10]

• 5 power supply pins: The 3P3VAUX (3.3V, from Carrier to Mezzanine),the VIO_B_M2C (Adjustable voltage, from Mezzanine to Carrier), the

8

Page 12: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 2.3: Typical example of FMC Mezzanine Module[8]

Figure 2.4: Picture of an SFP

VADJ (Adjustable voltage, from Carrier to Mezzanine), the 12P0V (12V,from Carrier to Mezzanine) and the 3P3V (3.3V, from Carrier to Mezza-nine).

• Some auxiliary pins

2.1.2.3 The SFPs

The small form-factor pluggable (SFP, �gure 2.4) is a hot pluggable small foot-print serial-to-serial data transceiver[11]. It serves as an interface between acarrier device (as the VFC-HPC or a router) and a cable (as a �ber optic).

The SFP is well-known by industries and has been collectively developed byseveral vendors. An SFP slot can accept di�erent types of SFPs, making thechoice of the physical layer �exible.

9

Page 13: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

2.1.3 The Memories

As the VFC-HPC is a data acquistion board, it must possess memory space tostore the results coming from the front-end electronic. In order to allow this,the board is equipped with static random-access memory (SRAM) designed byCypress[12]. Each of these have the following features:

• 72 Mbit memory capacity (36× 2 Mbit cells)

• Can run at up to 167MHz speed

• 3.3V I/O power supply

• Burst capability

• No Bus Latency (NoBL) architecture

2.2 Speci�cations and planning

The aim of my internship project was to test the VFC-HPC. It was focused onthe test and quali�cation of the connectivity:

1. The FMC connector consisting of the following:

• The user de�ned pins

• The multi-gigabit transceiver data pairs

• The I2C interface

• The JTAG interface

• The power supply pins

• The auxiliary pins

2. The SFPs

The �gure 2.5 illustrates the Gantt of my project.

10

Page 14: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 2.5: Gantt of the project

11

Page 15: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

12

Page 16: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Chapter 3

Test and Quali�cation

This chapter will present each interface tested and quali�ed. It will also givean overview of the material selected to perform these tests and quali�cations.Finally, it will provide, for each of them, the procedures developed and theresults.

The de�nition of the test depends on the interface considered: For someinterfaces, it consists of verifying the electrical connectivity between the FPGAor the power sources located on the VFC-HPC and each interface. It will alsoverify that there is no shortcut between the lines. For other interfaces, it consistsof verifying the possibility of implementing a reliable communication. The testpurpose and the parameters that validate the test will be clearly de�ned at thebeginning of each test.

The de�nition of the quali�cation is unique: It consists of evaluating themaximum transfer rate that the interface can endure.

The test has to be distinguished from the quali�cation. Generally speaking,the test aims to determine if a device is working while the quali�cation aims toestablish the quality of the lanes and the available margins by running severalstatistical tests.

As previously mentioned, the test and quali�cation will be focused on theFMC Connector and the SFP slots.

3.1 General test setup

To allow the execution of the di�erent tests and quali�cations, several com-ponents and devices were needed. Therefore, a complete test bench has beensetup. It consists of a VME crate wherein will be plugged the VFC-HPC, afront-end PC and a lab computer.

To control and monitor every test, the lab computer will connect to thefront-end PC through Ethernet by executing Python routines. This front-endPC is a VME board plugged in the crate and in charge of "`translating"' the

13

Page 17: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.1: Picture of the laboratory test bench

Figure 3.2: Diagram of the laboratory test bench

Ethernet requests into VME requests. These requests will be routed by thecrate to the VFC-HPC.

Classical electronic devices as oscilloscopes or multimeters have also beenprocured. A picture of the lab test bench is displayed on the �gure 3.1 whilethe �gure 3.2 is illustrating the previous explanations.

For each speci�c test, additional devices are needed (as FMC Mezzaninesboard to allow the test of the FMC connector or SFPs to allow the test of theSFP slots). Those ones will be presented further.

3.2 The �rmware

To perform the several tests, a �rmware has been written and implemented in-side the FPGA. It is based on a modular architecture: Each module in charge of

14

Page 18: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

one speci�c test can be added or removed from the �rmware without in�uenc-ing the other modules. The communication between them is performed throughthe Wishbone bus. It is a synchronous parallel bus protocol with hand-shakingbased on the Master-Slave principle[13].

The architecture of the �rmware is simple: It is made of one large module,the "`VfcHpc Top"', containing all the test modules and interfacing the FPGApins with these modules. In addition to these modules , there is the "`Vme ToWishbone"', module in charge of the "`translation"' of the VME request comingfrom the software into a Wishbone request.

A block diagram illustrating the previous explanations is shown in �gure 3.3.

Before being synthesized and implemented inside the FPGA, each moduleneed to be veri�ed. Therefore, it is tested with a system simulation includingthe FPGA and the other components of the board, as the bu�ers. The useof such a system allows us to verify the interfacing of the logic with the othercomponents of the board.

3.3 The FMC connector

This section will present the tests and quali�cations performed on each FMCconnector interface. Each presentation will begin by providing the purpose ofthe test. After that, the parameters that will validate the test, the procedureand the result will be presented.

The quali�cation has been performed only for the user de�ned pins andfor the multi-gigabit transceiver data pairs. This is due to the fact that thespeci�c hardware procured (which will be described further) does not allow usto perform the quali�cation on the other lanes. It was also less important toqualify them since they are dedicated to low speed transfers.

3.3.1 Speci�c hardware

To test and qualify the integrality of the FMC connector interfaces, speci�cmezzanines were needed.

These mezzanines should allow us to perform the following:

• Test the user de�ned pins (that will be called I/Os further) by verifyingthe electrical connectivity and qualify them

• Test the multi-gigabit transceiver data pairs (that will be called gigabitlanes further) by implementing a reliable communication and qualify them

• Test the I2C interface by implementing a reliable communication and qual-ify it

• Test the JTAG interface by implementing a reliable communication andqualify it

15

Page 19: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.3: General architecture of the device �rmware

16

Page 20: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

• Test the power supply pins by verifying the electrical connectivity

• Test the auxiliary pins by verifying the electrical connectivity

Based on these requirements, we proceeded to a market survey to choose thecompliant mezzanine boards that will allow us to perform our tests. The fourconsidered boards were the following:

• The AF101 designed by Acquitek[14]

• The XM104 designed by Xilinx[15]

• The XM105 designed by Xilinx[16]

• The FM-S18 designed by Faster Technology[17]

Unfortunately, none of these boards were complying all the requirements. It wastherefore necessary to choose the boards that would comply the majority of theserequirements. Consequently, our choice fell on the AF101 and the XM105. Theydid not allow us to qualify the I2C interface and the JTAG interface neither testsome auxiliary pins. However, they allowed us to test the majority of the FMCconnector pins and qualify the most importants (I/Os and gigabit lanes). The�gure 3.4 illustrates the comparative table of the features of the four consideredFMC boards.

The AF101 (�gure 3.5) is a loopback mezzanine board specially designedto allow the test of FMC connectors[14]. This board will be used to test andqualify the I/Os and the gigabit lanes. It will also allow us to test the I2Cinterface and some power supply pins.

Each half of the I/Os bank is loopbacked to the other half. Concerning thegigabit lanes, each TX pin is loopbacked to the RX pin. For the I2C interface, itis connected to an EEPROM, allowing reading/writing cycles. Finally, we willalso be able to test the 3P3VAUX and the VIO_B_M2C power supply pins asthey are supplying the I2C interface and the Hb I/Os bank respectively[14].

The XM105 (�gure 3.6) is a "`probing"' board[16]. It will be used to testthe JTAG interface (except the TRST_L pin), the remaining power supplypins (VADJ, 12P0V and 3P3V) and the auxiliary pins. Moreover, it will alsobe used if debugging is needed for the I/Os tests. Indeed, it will provide moreprecise I/Os test results than the AF101. The auxiliary pins consists of thePRSNT_M2C_L, PG_C2M, PG_M2C and VREF pins. PRSNT_M2C_Lis the mezzanine present pin. It allows the carrier to determine whether aMezzanine board is plugged or not. PG_C2M and PG_M2C are the powergood pins. They are asserted by the Carrier and the Mezzanine respectivelywhen the power supplies voltage values are within tolerance. The VREF pinsare two reference voltage pins (One for the La and Ha banks and the other forthe Hb bank).

The JTAG interface, the power supply pins and the PG_M2C pin arebreaked out through probe pins. The PG_C2M pin is connected to a LEDwhile the PRSNT_M2C_L pin is connected to the ground (by default, it isasserted on the VFC-HPC by a pull up resistor)[16].

17

Page 21: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.4: Comparative table of the four considered mezzanines boards

Figure 3.5: Schematic of the AF101[14]

18

Page 22: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.6: Picture of the XM105[16]

Unfortunately, the VREF pins are neither connected to the AF101 or theXM105. Therefore, it was impossible to test them.

To summarize, the AF101 will allow us to:

• Test and qualify the I/Os

• Test and qualify the gigabit lanes

• Test the I2C interface

• Test the 3P3VAUX and the VIO_B_M2C power supply pins

The XM105 will allow us to:

• Test the JTAG interface (except the TRST_L pin)

• Test the VADJ, 12P0V and 3P3V power supply pins

• Test the mezzanine present and the power good pins

The remaining pins that cannot be tested:

• The TRST_L pin that is part of the JTAG interface

• The VREF pins

3.3.2 The user de�ned pins test

To test and validate the I/Os of the connector, we want to verify that there isno shortcut between the pins and no broken line between the FMC connectorand the FPGA. In order to do that, a Verilog module has been written. Thismodule will use the I/Os loopback feature provided by the AF101 to test theI/Os and give a "`pass no pass"' result on each tested pin.

The general procedure is the following: All the pins are set to three-stateand connected to pull-up resistors (option o�ered by the FPGA[18]). One pin

19

Page 23: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

is set to low state voltage and the state of the other pins is veri�ed. Therefore,this procedure allows us to evaluate the entirety of the I/Os of the connector.This procedure also prevents against any bus con�ict.

3.3.2.1 Test procedure

The procedure performed by the I/Os test �rmware is the following:

1. The �rst pin of the I/O bus is set at low state before verifying that:

• The pin connected to this pin through the AF101 mezzanine boardalso switched to the low state

• The entirety of the other pins stayed at high state

2. The same procedure is applied for the other pins

This test routine is completely automated and can be controlled and monitoredthrough the Wishbone interface. In case of the automatic test fails, a manualcontrol of each pin is also provided.

3.3.2.2 Test results

Once the �rmware written, the I/Os have been tested and evaluated. The resultsthat we obtained were positive: All the pins of the La and the Ha bank are wellconnected and no short cut has been detected. Regarding to the Hb bank, theHb20P, the Hb20N, the Hb21P and the Hb21N are not connected to the FMCconnector due to an error in the schematics. This issue has been corrected forthe next version of the VFC-HPC. Fortunately, the remaining pins of the Hbbank are well connected

3.3.3 The user de�ned pins quali�cation

To qualify the I/Os, we want to evaluate the maximum transfer rate that theycan endure. In order to do that, we will measure the Bit Error Rate (BER) andthe eye diagram of the communication.

The Bit Error Rate is the number of bit errors received divided by thetotal amount of received bits during a speci�c time interval[19]. Its value isa good quality indicator of a communication. Generally, we consider that acommunication is reasonably well synchronized as long as the BER value is lessthan 10−12[20].

The eye diagram represents the superposition of pseudo-random zero-onesignal transitions[21]. The obtained �gure can give important information. Itcan provide the best point for sampling and the amount of jitter and distortion.The jitter is the deviation of the rising or falling edges from the ideal time[22].

To perform these measures, a Verilog �rmware has been written. It willbe, once again, based on the I/Os loopback feature provided by the AF101.

20

Page 24: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

The main idea consists of sending a series of pseudo random words on a line(TX line) and comparing them with the received words on another line (RXline) connected to the TX through the AF101 loopback. The eye diagramwill be measured by using a di�erential probe on the Mezzanine board and bydisplaying the signal on an oscilloscope. The �gure will be obtained by usingthe "`persistence"' feature provided by the scope.

3.3.3.1 Test procedure

The I/Os quali�cation �rmware consists of the following modules:

• The Pattern Sender module in charge of generating and sending the seriesof words

• The Pattern Checker, module that receives the words and verify them toindicate whether the link is synchronized or not

• The Satistic Checker, module that provides statistics about the line, basedon the results received from the Pattern Checker, to allow the evaluationof the quality of the line. This module is described in details in the sec-tion 3.3.4

• The Phase Controlled Phase-Locked Loop(PLL), module that, as the namesuggests, is based on a PLL with the possibility to change dynamically thephase of the output clock compared to the input clock

• The Data and Phase Aligner, module that performs realignment of theRx Data on the Tx Clock

In this �rmware, the integrality of the I/Os is seen as a parallel bus. There-fore, each bit of the word sent corresponds to one pin of the FPGA.

The Pattern Sender is a Linear Feedback Shift Register (LFSR)[23] whosethe value of the input depends on the previous state. In our case, it is anexclusive-or (XOR). The bit positions that in�uence the value of the input arecalled taps. These taps positions depend on the amount of bits generated bythe LFSR and are given, for each case, by a polynomial. For example, the Labank Pattern Sender uses a LFSR of 17 bits, corresponding to the polynomialx17+x14+1[24]. The input of the shift register is the XOR of the bit 16 and 13.We can prove mathematically that this polynomial provides 217 − 1 = 131071output possibilities and the shift register will provide never the same outputbefore having provided all the possibilities. Therefore, the LFSR acts like arandom generator. It is called a Pseudo Random Number Generator (PRNG).

The Pattern Checker receives the words from the I/Os and verify the se-quence to indicate whether the link is synchronized or not. It is also based ona LFSR that synchronizes itself on the received words sequence by connectingthe input of the shift register to the bit 0 of the received words. Once the LFSRis synchronized, the input of the shift register is connected to the XOR and theword comparison between the LFSR and the received word is performed. Thismodule uses the same state machine written for the gigabit lanes quali�cation,described in the section 3.3.4.

21

Page 25: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

The Phase Controlled PLL is based on a PLL that provides the possibility tochange dynamically the phase of the output clock compared to the input clock.It is a megafunction provided by Altera. The control of the phase of the PLLcan be controlled through the JTAG interface of the FPGA.

The Data and Phase Aligner performs realignment of the Rx Data on theTx Clock. It is simply based on a dual clock FIFO whose the input clock is thedephased clock generated by the phase controlled PLL while the output clockis the Tx clock.

3.3.3.2 Test results

The quali�cation �rmware has been �rst run at 200MHz. Once the communi-cation was locked, we left the �rmware running during four days. As we did notobserve any error during these four days, we can conclude that the BER of thecommunication is better than

1Error

4days× 200Mbps= 1.447× 10−14 (3.1)

After running a BER test, we measured the eye diagram of the communi-cation illustrated on the �gure 3.7. This diagram presents a good quality. Thejitter is about 22ps which is a reasonable small value compared to the signalperiod ( 1

200MHz = 5ns).

Those two observations allow us to conclude that the FMC user de�ned pinsare able to stand a communication at 200MHz.

Unfortunately, we have not been able to run a BER test at 400Mhz due tosome technical issues that we have not been able to solve in time. Nevertheless,measuring an eye diagram can still be useful to, at least, qualify the PCB layoutat that speed.

The appearance of this eye diagram presents also a good quality. With ajitter of about 45.58ps, we are pretty con�dent in the fact that the I/Os shouldbe able to stand a communication at 400MHz.

3.3.4 The multi-gigabit transceiver data pairs test

To test and validate the FMC Multi-Gigabit lines, we want to implement areliable communication. In order to do that, a Verilog �rmware has been de-signed. It will also use the gigabit line loopback feature provided by the AF101.The procedure used here is similar to the one used for the I/Os quali�cation:The �rmware sends continuously a series of words on the transmitter line andcompare it with the received words on the receiver line connected to the TXthrough the AF101 loopback to validate the test.

3.3.4.1 Test procedure

The gigabit lines test �rmware consists of the following modules:

22

Page 26: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.7: Eye diagram at 200MHz

Figure 3.8: Eye diagram at 400MHz

23

Page 27: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

• The transceiver megafunction, provided by Altera and based on the transceiverchannels which are hard blocks implemented inside the FPGA[25]

• The transceiver controller module designed to allow the initialization ofthe communication and monitoring the transceiver channel

• The Comm Checker, module in charge of the generation of the series ofwords, the veri�cation of the received words allowing the assessment ofthe link synchronization

• The Statistic Checker, module that provides statistics about the line,based on the results received from the Comm Checker, to allow the eval-uation of the quality of the line

The �gure 3.9 illustrates the general architecture of the gigabit lines test�rmware.

The transceiver channel (�gure 3.10) consists of the following blocks[18]:

1. A transmitter part made of the following blocks:

• The TX Phase Compensation FIFO: FIFO present for compensatingthe phase shift between the transceiver channel and the FPGA logicclock

• The Byte Serializer: Divides by two the bus width (32 to 16 or 16 to8, depending on the data bus width)

• The 8B/10B Encoder: Generates 10-bit code groups from 8-bit datato achieve DC-balance

• The Serializer: Performs parallel-to-serial data conversion and sendsthe data LSB �rst to the transmitter bu�er

2. A receiver part made of the following blocks:

• The Clock and Data Recovery (CDR): It is the channel PLL (eachchannel transceiver has one dedicated PLL called channel PLL) con-�gured to recover the clock and data from the incoming serial dataand sends the clock and data to the deserializer

• The Deserializer: Performs data restoration by serial-to-parallel dataconversion (opposite of the Serializer)

• The Word Aligner: Block that provides word boundary restorationduring link synchronization. It uses a prede�ned alignment patternword to perform the word boundary restoration

• The 8B/10B Decoder: Restores the 8-bit data from the 10-bit codegroup (opposite of the 8B/10B Encoder)

• The Byte Deserializer: Restores the data bus by multiplying by twothe bus width (opposite of the Byte Serializer)

• The Byte Ordering Block: Restores the byte order that may notmatch the originally transmitted after passing through the the ByteDeserializer

24

Page 28: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.9: General architecture of the gigabit lines test system

25

Page 29: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.10: Architecture of the transceiver channel[18]

• The RX Phase Compensation FIFO: Performs exactly the same ac-tion than the TX Phase Compensation FIFO

The FPGA provides several options to clock the transceivers. There arethree types of clock lines: The x1, the x6 and the xN. Those lines are detailedand charaterized in the Arria V handbook[18].

Those characteristics are summarized in the table 3.11.

Two di�erent types of PLLs can be used to clock the megafunctions[18]:

• The fractional PLL: This type of PLL is distributed all over the FPGAand arround the transceivers. It can run up to 3,125Gbps data speed.

• The Clock Multiplier Unit (CMU) PLL: This PLL can run up to 6,5336Gbps data speed. It implements the channel PLL available in each channeltransceiver. This implies that the channel PLL cannot be used as CDR.So, when the PLL of a channel is used as CMU PLL, the receiver of thechannel cannot be used. In conclusion, the use of one CMU PLL involvesthe sacri�ce of one transceiver channel

The megafunction choosed in this project does not allow us to use a fractionalPLL[25]. The use of a CMU PLL is then inevitable, involving the sacri�ce ofone transceiver channel. The choosed channel to be sacri�ed is the channel 3.

The transceiver controller performs the following procedure:

1. Reset the transceiver at the beginning (when the board is turned on)

2. When the transceiver indicates that it is ready (TX and RX block), it startsending patterns for aligning the Word Aligner and the Byte OrderingBlock

26

Page 30: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.11: Charaterestics of x1, x6 and xN Clock Lines table[18]

3. When the Word Aligner and the Byte Ordering Block indicate that theyare synchronized, the transceiver controller indicates that it is ready tosend words

4. The transceiver controller sends the TX words received from the CommChecker to the transceiver and the RX words received from the transceiverto the Comm Checker

5. If the transceiver TX or RX block is not ready anymore, the transceivercontroller performs a reset

6. If the Word Aligner or the Byte Ordering Block is not synchronized any-more, the transceiver controller repeforms a synchronization by sendingthe patterns

The �gure 3.12 is a �ow diagram illustrating the previous explanations.

The transceiver comm checker performs the following procedure:

1. When the transceiver controller indicates that it is ready, the comm checkerstarts sending the words and checking the received words

2. After reception of a sequence of 8 correct words (programmable value),the comm checker indicates that it is synchronized

3. When the comm checker receives a wrong word, it asserts the error pinfor one clock cycle and indicates which bit(s) is(are) wrong(s)

4. After reception of a sequence of 8 wrong words, the comm checker isconsidered not synchronized anymore. It must receive 8 correct words tosynchronize back

27

Page 31: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.12: Flow diagram of the Transceiver Controller

28

Page 32: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.13: Flow diagram of the Transceiver Comm Checker

The �gure 3.13 is a �ow diagram illustrating the previous explanations the�gure 3.14 illustrates the architecture of the Transceiver Comm Checker

The aim of the statistic checker module is to establish statistics based onthe results received from the Comm Checker. Those statistics are obtained byusing counters and In�nite Impulse Response (IIR) �lters to obtain:

• The average time between errors which is the inverse of the BER

• The average time to synchronize back when the synchronization is lost bythe transceiver controller

• The number of errors obtained since the reset of the module

• The number of multibit errors obtained since the reset of the module

29

Page 33: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.14: Transceiver Comm Checker architecture

30

Page 34: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

The IIR �lter in a low pass �lter giving the average value of the series ofinput values. Its transfer function is

y[n] =(N − 1)y[n− 1] + x[n]

N(3.2)

Where N is the amount of samples and also the bandwidth of the �lter whilex[n] is the nth element of the sequence. The value of N is chosen such that the�lter converges quickly to the expected value. That depends on the amount ofobserved errors.

The statistic checker has been designed to establish the quality of the lines.Therefore, it has not been used for the test part. It will be used in the quali�-cation part.

The �gure 3.14 illustrates the architecture of the Transceiver Statistic Checker

3.3.4.2 Test results

The gigabit lines test �rmware has been �rst run at 3.125Gbps with a 16-bits interface width. During the implementation of this test, a bug has beenhighlighted: The TX and the RX of the lines 0 and 9 were crossed in theschematic. Unfortunately , these two channels are not on the same side of theFPGA. The consequence to that is, because of the structure of this FPGA, theycannot be tested with the clock coming from the mezzanine board. This issuehas been taken into account for the next version of the VFC-HPC. The test ofthe other channels has been performed and validated.

After test and validation at 3.125Gbps, a test at 5Gbps with 32-bits interfacewidth (when the transceiver has to run at a speed higher than 3,7 Gbps, theuse of 32-bits interface width is invevitable) has been undertaken. This test hasalso been a success.

3.3.5 The multi-gigabit transceiver data pairs quali�ca-

tion

To qualify the FMC Multi-Gigabit lanes, we want to evaluate the maximumtransfer rate that they can endure. In order to do that, we will measure theBER of the communication. Unfortunately, it was not possible to obtain an eyediagram because we do not have the adequate material.

In order to measure the BER, the gigabit lanes test system will be the same�rmware used for the test, based on the loopback feature provided by the AF101.

3.3.5.1 Test results

The quali�cation �rmware has been run at 5Gbps. The conditions were similarto the quali�cation of the I/Os: Running the quali�cation during 2 days and,

31

Page 35: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.15: Transceiver Statistic Checker architecture

32

Page 36: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

as no error was observed, we can conclude that the BER of the communicationis better than

1Error

2days× 5Gbps= 1.157× 10−15 (3.3)

This certify that the FMC Multi-Gigabit lanes are able to stand a commu-nication at 5Gbps.

3.3.6 The I2C test

To test and validate the I2C pins, we want to implement a reliable commu-nication. In order to do that, a Verilog module has been used. It is an I2CMaster that can be controlled and monitored through its Wishbone interface.This module will use the AF101 EEPROM connected to the I2C interface of theFMC connector[14]. The test procedure consists of writing a series of words inthe EEPROM before reading back its content. The two series will be compared,allowing the validation of the test.

3.3.6.1 Test procedure

The procedure performed by the I2C test �rmware is the following:

1. The I2C Master sends an 8-bit word at a de�ned address of the EEPROM

2. If the acknowledge is not received from the EEPROM controller, the testfailed

3. An I2C read cycle at the same de�ned address is then performed to com-pare the received word with the transmitted word

4. If the two compared words are the same, the test is considered as a success.Otherwise, the test failed.

3.3.6.2 Test results

On the �rmware written and implemented inside the FPGA, the I2C test hasbeen performed with, as result, a complete success.

3.3.7 Miscellaneous

The remaining pins of the FMC connector to be tested are the JTAG pins, thepower supply pins, the mezzanine present pin, the power good pins and theVREF pins.

33

Page 37: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

3.3.7.1 The JTAG pins

To test and validate the JTAG pins, the ideal solution would be to implementa reliable communication. Unfortunately, the XM105 does not provide thissolution. The interface is just breaked out through pins[16]. Therefore, wedecided to verify the electrical connectivity between the FPGA and the JTAGpins as well as the non presence of shortcut between the lines. In order to dothat, the test procedure used was probing. o that, the test procedure used wasprobing. To allow that, the Mezzanine board used was the XM105. The JTAGpins test has been performed with, as result, a complete success.

3.3.7.2 The power supply pins

To test and validate the power supply pins, we want to verify the electricalconnectivity of these pins with the FPGA or the power sources located on theVFC-HPC and to measure the voltage values.

As previously mentioned, the FMC connector provides �ve power supplypins: The 3P3VAUX, the VIO_B_M2C, the VADJ, the 12P0V and the 3P3V[8].

As mentioned in the AF101 description, the 3P3VAUX pin is used by the I2CEEPROM and the VIO_B_M2C pin is used by the Hb I/Os bank. Therefore,the electrical connectivity of the 3P3VAUX with the power supply situatedon the PCB has been tested and validated with the successful test of the I2Cinterface while the electrical connectivity of the VIO_B_M2C with the FPGAhas been tested and validated with the successful test of the Hb I/Os bank.Although we cannot measure the voltage values provided by these two pins,we can conclude that they are in acceptable ranges since the tests of the I2Cinterface and the Hb I/Os bank have been validated.

As mentioned in the XM105 description, the VADJ, the 12P0V and the3P3V power supply pins are breaked out through pins. Therefore, in order toverify the electrical connectivity of these pins with the power sources and tomeasure the voltage values, we will probe these with a multimeter. These testshave been performed with, as result, a complete success.

3.3.7.3 The mezzanine present pin

To test and validate the mezzanine present pin, we want to verify the electricalconnectivity of this pin with the FPGA.

As previously mentioned in the XM105 description, the PRSNT_M2C_Lpin is by default asserted through a pull-up resistor located on the VFC-HPC.When a mezzanine is plugged on the FMC connector, PRSNT_M2C_L is de-asserted. Therefore, the electrical connectivity of this pin with the FPGA canbe veri�ed by monitoring the voltage state directly from the FPGA when theMezzanine board is plugged or unplugged.

34

Page 38: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

3.3.7.4 The power good pins

To test and validate the power good pins, we want to verify the electrical con-nectivity of these pins with the FPGA.

As mentioned in the XM105 description, the PG_C2M pin is connected toa LED on the Mezzanine board. Therefore, the electrical connectivity of thispin with the FPGA can be tested by controlling the voltage state of the pindirectly from the FPGA and verifying that the LED is turning on/o�. This testhas been performed with, as result, a complete success.

As also mentioned in the XM105 description, the PG_M2C is breaked outthrough a pin. Therefore, the electrical connectivity of this pin with the FPGAcan be tested by controlling its voltage state with a signal generator and veri-fying that the input value e�ectively changed in the FPGA. This test has beenperformed with, as result, a complete success.

3.4 The SFPs

This section will present the tests and quali�cations performed on the SFPslots. The test and the quali�cation will have the same structure: We will beginby providing the purpose of the test/quali�cation. After that, the parametersthat will validate the test/quali�cation, the procedure and the result will bepresented.

3.4.1 The SFPs test

To test and validate the SFP slots, we want to implement a reliable communi-cation. In order to do that, the same �rmware designed for the FMC gigabitlanes test will be used. Here, instead of using electrical loopbacks through theAF101, optical loopbacks through optical �bers will be used. The procedureused here is the same: The �rmware sends continuously a series of words onthe transmitter line and compare it with the received words on the receiver lineconnected to the TX through the optical �ber to validate the test. Please referto the �gure 3.16.

3.4.1.1 Speci�c hardware

To test the integrality of the SFP slots, SFPs were needed. These SFPs wouldallow us to test the SFP slots by running a reliable communication. Based onthese requirements, we proceeded to a market survey to choose compliant SFPsthat will allow us to perform our tests. Therefore, commercial SFPs have beenordered: The FT3814D (�gure 3.17) manufactured by the vendor FTTX[26]. Itpresents the following features:

• Operates at up to 8.5Gbps

• Adapted for Single-Mode optical �bers

35

Page 39: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.16: Optical loopback

36

Page 40: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.17: Picture of the FT3814D

• The wavelength of the signal is 1310nm

• The communication distance can go up to 1.4km

3.4.1.2 Test results

As for the gigabit lines test �rmware, the SFP slots test �rmware has been �rstrun at 3.125Gbps with a 16-bits interface width. This test has been a succes forthe 6 SFP slots.

After test and validation at 3.125Gbps, a test at 5Gbps with 32-bits interfacewidth has been performed. This test was also a complete success.

3.4.2 The SFPs quali�cation

To qualify the SFP slots, we want to evaluate the maximum transfer rate thatthey can endure. In order to do that, we will measure the BER and the eyediagram of the communication. The quali�cation �rmware used here is the sameone used for the SFPs test.

To measure the BER, we will also implement an optical communication withthe FT3814D SFPs and optical �bers. Afterward, we will introduce an opticalattenuation in the communication using a device presented in the section 3.4.2.1.Therefore, we will be able to draw curves giving the BER and the correspondingattenuation. To measure the eye diagram, we will use an oscilloscope and another device also presented in the same section.

3.4.2.1 Speci�c hardware

To perform the signal attenuation needed to measure the BER, an optical atten-uator has been used ( 3.18)[27] . Indeed, instead of performing direct loopback,the optical �bers pass through the attenuator which will perform the signalattenuation.

To perform the measurement of the eye diagram, we will use an optical toelectrical converter receiving the signal from the SFP. Afterward, this converterwill send the generated electrical signal to a 12GHz oscilloscope. The �gure 3.19illustrates the test setup implemented to allow the measurement of the eyediagram.

37

Page 41: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.18: Picture of the Optical Attenuator

Figure 3.19: Eye diagram test setup

38

Page 42: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

3.4.2.2 Test results

The quali�cation �rmware has been run at 3.125Gbps, 5Gbps and 6.25Gbps andthe BER was measured for each of these data speeds. The �gure 3.20 illustratesthe BER curves and the corresponding attenuation for the three mentioned dataspeeds.

As previously mentioned, a communication is reasonably well synchronizedas long as the BER value is less than 10−12. In this case, the following attenu-ations were measured at this BER value:

• For 3.125Gbps: 17.3dB

• For 5Gbps:16.6dB

• For 6.25Gbps: 16.7dB

It is interesting to compare these maximum attenuations that the SFP slotscan endure with the requirements of the BI applications. In order to do this, wewill evaluate the average attenuation that could introduce any BI application.The attenuation sources of any BI application are the following:

• The input loss: The optical �bers are using LC connectors. The typicalattenuation of these connectors is 0.3dB[28]

• The linear attenuation of the optical �ber: The ITU-T G.652 standardspeci�es that a G.652.D optical �ber (used in our lab test) shall present amaximum attenuation of 0.4dB/km for a wavelength of 1310 nm (which isthe wavelength of the signal generated by the FT3814D)[29]. The longestoptical �ber used in the BI Group is about 1.5km. This corresponds toan attenuation of 0.6dB.

• The patch panel loss: The loss due to the patch panel comes from theconnector. So we can consider the same loss than the LC connector whichis 0.3dB. The biggest amount of patch panel for a BI application will be10, which corresponds to a total attenuation of 3dB.

• The output loss: The output loss is the same than the input loss

The previous values give us a total attenuation of about 4.2dB. Therefore,we can conclude that the SFP slots will be able to easily endure the attenuationgenerated by any BI application at up to 6.25Gbps.

It is important to mention that these BER curves have been drawed for oneSFP. The other SFPs have been quali�ed using the same technique used for theFMC gigabit lanes quali�cation. We left the communication running during 2days at 6.25Gbps and, as no error was observed, we can conclude that the BERof the communication is better than

1Error

2days× 6.25Gbps= 9.259× 10−16 (3.4)

39

Page 43: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.20: SFP Bit Error Rate Curve

40

Page 44: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.21: Eye diagram at 5Gbps

After measuring the BER, we proceeded to the measure of the eye diagram at5Gbps, which should be the maximum speed used by any typical BI application.The eye diagram is illustrated on the �gure 3.21.

We can notice that the eye diagram presents a good quality. The jitter wasabout 15.25ps and the eye crossing about 50.9%. Unfortunately, we were notable to measure the eye height because the optical to electrical converter wasnot calibrated. However, this device allowed us to measure the power of theoptic signal which was about 0.5843mW (�gure 3.22).

41

Page 45: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Figure 3.22: Power of the optic signal

42

Page 46: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Chapter 4

Conclusion

The design of the VFC-HPC, a VME FPGA-based high-speed digital acquisitionboard aimed to be the new standard back-end for the BI Group, is part of thestandardization and re-usability group policy. During my technical studentshipat CERN, I have been participating to the development of a test and quali�ca-tion system to verify the compliance of the VFC-HPC to the speci�cations.

Within this project, I learned several standards (as ANSI/VITA57.1 de�n-ing the FMC connector), the Verilog HDL language to write �rmwares, thePython language to perform tests and several engineering techniques (as FPGAtransceivers). These acquired skills allowed me making several market surveysto choose the proper test components to be used (as FMC Mezzanines), settingup a test bench, implementing optical communications through SFPs, writing�rmwares for FPGAs, writing software test routines and measuring BERs andeye diagrams.

These tasks allowed me to test the integrality of the FMC connector exceptthe VREF and the TRST_L pins. The majority of these tests have been suc-cessfully validated. These taks also allowed me to qualify the entirety of themulti-gigabit transceiver data pairs (except the channel 0 and 9 that were notproperly connected) and the user de�ned pins at up to 5Gbps and 200MHz re-spectively. Finally, I have also been able to test and qualify the integrality ofthe SFP slots at up to 6.25Gbps. One of them has been quali�ed by using anoptical attenuator giving, as result, several BER curves and the correspondingattenuations.

The test and quali�cation results and the encountered di�culties o�ers alot of future prospects. To begin, we can reperform a market survey to �ndmaterial that will allow us testing the VREF and the TRST_L pins and testingthe other JTAG pins by implementing a reliable communication. Resetting upthe test bench to allow the quali�cation of the I2C and the JTAG interfaces isalso a possibility. Concerning the quali�cation of the FMC I/Os and the Gigabitlanes, we can solve the technical issues encountered to allow a quali�cation ofthe I/Os at 400MHz while we can qualify the FMC Gigabit lanes at 6.25Gbps.We can also qualify the integrality of the SFP slots with BER curves. Finally,the most interesting future prospect of this project would be automating the

43

Page 47: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

test bench for the production of the VFC-HPC.

44

Page 48: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

Bibliography

[1] CERN. About cern, dec 2014. www.cern.ch.

[2] Andrea Boccardi. Ongoing electronic development in the cern beam instru-mentation group: challenges and solutions for the measurement of particleaccelerator beam parameters. IOPScience, page 11, mar 2013.

[3] Thomas Floyd. Systemes numeriques. Editions Reynald Goulet, 2013.

[4] Design Automation Standards Committee(DASC). IEEE Standard for Sys-temVerilog - Uni�ed Hardware Design, Speci�cation, and Veri�cation Lan-guage. Institute of Electrical and Electronics Engineers(IEEE), 2012.

[5] Altera. Arria V Device Overview. Altera, 2013.

[6] VMEbus International Trade Association(VITA). ANSI/VITA 1-1994:VME64. American National Standards Institute(ANSI), 1995.

[7] Markus Joos. An introduction to vmebus, nov 2014. Powerpoint presenta-tion.

[8] VMEbus International Trade Association(VITA). ANSI/VITA 57.1:FPGA Mezzanine Card(FMC) Standard. American National StandardsInstitute(ANSI), 2010.

[9] NXP Semiconductors. I2C-bus speci�cation and user manual. NXP Semi-conductors, 2012.

[10] IEEE Computer Society. IEEE 1149.1: Standard Test Access Port andBoundary-Scan Architecture. Institute of Electrical and Electronics Engi-neers, 2013.

[11] MSA (Multiple Source Agreement) group. INF-8074i Speci�cation for SFP(Small Formfactor Pluggable) Transceiver. Small Form Factor (SFF) Com-mitte, 2001.

[12] Cypress Semiconductor Corporation. CYC7C1470BV33 datasheet. CypressSemiconductor Corporation, 2014.

[13] OpenCores Community. Wishbone B4. OpenCores Organization, 2010.

[14] ApisSys. AF101 datasheet. ApisSys SAS, 2011.

[15] Xilinx. XM104 Connectivity Card User Guide. Xilinx, 2010.

45

Page 49: CONCEPTION ET DÉVELOPPEMENT LOGICIEL SUR FPGA …from "`Conseil Europeen pour la Recherche Nucleaire"' which means "`European Council for Nuclear Research"', name coming from the

[16] Xilinx. XM105 Debug Card User Guide. Xilinx, 2011.

[17] Faster Technology. FM-S18 datasheet. Faster Technology, 2010.

[18] Altera. Arria V Device Handbook. Altera, 2014.

[19] Eric Bogatin. Signal and Power Integrity. Prentice Hall, 2009.

[20] Rick Pimpinella and Al Brunsting. Correlations between eye diagram andber measurements for multimode �bers. International Wire and Cable Sym-posium (IWCS), page 8, nov 2007.

[21] Onsemi. Understanding data eye diagram methodology for analyzing highspeed digital signals. Onsemi, page 7, jun 2014.

[22] Deepak Behera, Sumit Varshney, Sunaina Srivstava, and Swapnil Tiwari.Eye diagram basics: Reading and applying eye diagrams. Electrical DesignNews (EDN), page 5, dec 2011.

[23] Maria George and Peter Alfke. Linear Feedback Shift Registers in VirtexDevices. Xilinx, 2007.

[24] Peter Alfke. E�cient Shift Registers, LFSR Counters, and Long Pseudo-Random Sequence Generators. Xilinx, 1996.

[25] Altera. Altera Transceiver PHY IP Core User Guide. Altera, 2014.

[26] FTTX. Fttx sfps features, dec 2014. www.fttxtech.com.

[27] EXFO Electro-Optical Engineering Inc. FVA-3100 Variable Attenuatordatasheet. EXFO Electro-Optical Engineering Inc., 2007.

[28] Corning Optical Communications LLC. FuseLite Connector, LC, 900 umTightBu�ered. Corning Optical Communications LLC, 2014.

[29] Telecommunication standardization sector of ITU. ITU-T G.652: Charac-teristics of a single-mode optical �bre and cable. International Telecommu-nication Union (ITU), 2009.

46