5.1 Design Processtqvinh/Lectures/VIP/VIP-Ch5.pdf · 4/6/2012 1 ĐẠ I H ỌC QU ỐC GIA TP.H Ồ...

4/6/2012

1

ĐẠI HỌC QUỐC GIA TP.HỒ CHÍ MINHTRƯỜNG ĐẠI HỌC BÁCH KHOA

KHOA ĐIỆN-ĐIỆN TỬ BỘ MÔN KỸ THUẬT ĐIỆN TỬ

11

VIDEO AND IMAGE PROCESSING USING DSP AND PFGA

Chapter 5: Video and image processing system design on FPGA

5.1 Design Process for video and image processing system

5.2 Hardware architecture for video and image processing

5.3 Video and image processing on FPGA

Bộ môn Kỹ Thuật Điện Tử

5.1 Design Process

• Hardware design image processing requires two skills

– Algorithm development

– Hardware design

2

Problem

Specification

Algorithm

Development

System

Implementation

Architecture

Selection

The four main steps of design process

4/6/2012

2

Bộ môn Kỹ Thuật Điện Tử

Problem Specification• A problem specification should clearly describe

– System functionality

• What the system needs to be able to do

• Specify the desired result of image processing.

– Performance of the system

• how well must it perform the functions

• Specify the maximum allowable latency and the number of images

or frames that must be processed every second

• If the decision is binary, the allowable failure rate should be

specified

– System environment

• Lighting, optics

• Input/output interfacing, user interface

• Other important aspects: encoding, decoding, pre-processing

3

Bộ môn Kỹ Thuật Điện Tử

Problem Specification

• Example: Design a video denoising system

4

No. Specification Description

1 System functionality - Function: remove noise in video source,

receive high quality pictures at output

- Algorithm: LMS filtering

2 Performance of the

system

-Process video in real-time

-Throughput: 30 CIF frames per second

-PSNR > 30dB

3 System environment -Input: capture video from S-video

-Output: display video at LCD through VGA port

-Video format: color video 4:2:2 YCbCr

-Including a deinterlacing unit

-User interface: switch ON/OFF for denoising

4/6/2012

3

Bộ môn Kỹ Thuật Điện Tử

Algorithm Development

• Algorithm development

– formulate the process

– consider preprocessing algorithm

– test the algorithm on different images.

• Algorithm structure

– how the operations work together to give a robust algorithm

– Specify physical or environmental constraints

• Hardware development issues

– convert algorithm-level design to hardware-level design

– consider timing constraints

– Specify system-level architecture

5

Bộ môn Kỹ Thuật Điện Tử

Algorithm Development• Example for pre-processing

6

4/6/2012

4

Bộ môn Kỹ Thuật Điện Tử

Algorithm Development

• Example for architecture using CPU embedded within the

FPGA

7

Bộ môn Kỹ Thuật Điện Tử

Assignments

1. Describe problem specification for the design of

text recognition system

2. Describe problem specification for the design of

color enhancing system

8

4/6/2012

5

Bộ môn Kỹ Thuật Điện Tử

Hardware architecture for video and

image processing

• Hardware architecture

– Serial processing

• CPU-based architecture

• DSP-based architecture

– Parallel processing

• Massively parallel architecture

• Pipelining architecture

• Systolic architecture

– Hybrid processing

• DSP-FPGA based architecture

• System-on-chip (SoC) architecture

9

Bộ môn Kỹ Thuật Điện Tử

Massively parallel architecture

• Characteristics:

– highest throughput

– small delay

– heavy resource

10

Operation 1

Operation 2

Operation 3

+

Input Output

4/6/2012

6

Bộ môn Kỹ Thuật Điện Tử

Pipelining architecture

• Pipelining technique

– splits the logic into smaller blocks,

– spread over multiple clock cycles,

– reducing the propagation delay in any one clock cycle.

– This allows the design to be clocked faster in order to meet timing

constraints.

11

one-stage pipeline two-stage pipeline

Bộ môn Kỹ Thuật Điện Tử

Pipelining architecture

• Some problems in pipelining architecture

– imbalance in the propagation delay

– high logic depth due to multiplication and division

– the input data is requested asynchronously

12

4/6/2012

7

Bộ môn Kỹ Thuật Điện Tử

Systolic Architecture

• Systolic architecture is also called Systolic Arrays

• A network of logic blocks compute and pass data through the

system

• Used as a coprocessor in combination with a host computer

and the behavior is analogous to the flow of blood through the

heart; thus named as systolic.

13

Bộ môn Kỹ Thuật Điện Tử

Systolic Architecture

14

4/6/2012

8

Bộ môn Kỹ Thuật Điện Tử

Systolic Architecture

• Systolic Array Design Methodology

15

Bộ môn Kỹ Thuật Điện Tử

Systolic Architecture• Example: design systolic architecture for a FIR filter

16

Dependence graph

Space-time representation

4/6/2012

9

Bộ môn Kỹ Thuật Điện Tử

Assignments

1. Design a median filter using systolic architecture

2. Design an edge detection filter using pipelining architecture

3. Design a Gaussian filter using pipelining architecture

4. Design a motion estimation unit using systolic architecture

5. How to solve the asynchronous problem in pipelining

architecture?

17

Bộ môn Kỹ Thuật Điện Tử

5.3 Video and Image Processing on FPGA

• FPGA: Field Programmable Gate Array

18

Types of programmable logic array.

4/6/2012

10

Bộ môn Kỹ Thuật Điện Tử

FPGA• FPGA manufactures

– Xilinx

– Altera

– Lattice

– Cypress

19

Altera FPGA board Xilinx FPGA board

Bộ môn Kỹ Thuật Điện Tử

Video and image processing system design

using Xilinx FPGA

20

4/6/2012

11

Bộ môn Kỹ Thuật Điện Tử

Video and image processing system design

using Xilinx FPGA

• Automatically generates

HDL/optimized algorithms

– Shortens learning curve

– HW redesign eliminated

– Optimal implementation

21

Bộ môn Kỹ Thuật Điện Tử

Video and image processing system design

using Xilinx FPGA

• Video system design with IP cores

22

4/6/2012

12

Bộ môn Kỹ Thuật Điện Tử

Video and image processing system design

using Xilinx FPGA

• Video AllianceCOREs

23

Bộ môn Kỹ Thuật Điện Tử

Video and image processing system design

using Altera FPGA

• General design flow

24

4/6/2012

13

Bộ môn Kỹ Thuật Điện Tử

Video and image processing system design

using Altera FPGA

• IP-based design flow

25

Bộ môn Kỹ Thuật Điện Tử

Video and image processing system design

using Altera FPGA

• DSP Builder:

– integrates the algorithm development, simulation, and verification

capabilities of The MathWorks MATLAB and Simulink system-level

design tools with the Altera Quartus II software

26

4/6/2012

14

Example design: Smart Camera

Hardware Architecture

• ALTERA Stratix EP1S60F1020C7

• 4Mpixels LUPA-400 image sensor

• (2) 2d accelerometers

• (3) gyroscopes

• 10Mb SRAM

• 64Mb SDRAM

Smart Camera Hardware Architecture

4/6/2012

15

Bộ môn Kỹ Thuật Điện Tử

Assignments

1. Design median filter in Matlab Simulink and convert the

design to HDL by using Altera DSP Builder

2. Design a color correction filter in Matlab Simulink and

convert the design to HDL by using Altera DSP Builder

3. Build a Gaussian filter IP core and integrate it to Altera SOPC

Builder

4. Build a VGA controller IP core and integrate it to Altera SOPC

Builder

5. Build a SOPC system which performs DCT transform for input

image from ROM.

29