Design of Motion Vision Processing System Using StratixII Series FPGA Devices

With the development of deep sub-micron processes, the capacity and density of FPGAs are increasing. With their powerful parallel multiply-add (MAC) capabilities and flexible dynamic reconfigurability, FPGAs are widely used in many fields such as communications and images. However, in the implementation of complex algorithms, FPGAs are not as convenient as embedded processors, so when designing systems with complex algorithms and control logic, they often need to be used in conjunction with embedded processors. This is SOPC (System on a Programmable chip, which can programming system-on-chip) technology. SoPC is the product of the combination of SoC and FPGA, which is completed by a single programmable reconfigurable chip.

With the development of deep sub-micron processes, the capacity and density of FPGAs are increasing. With their powerful parallel multiply-add (MAC) capabilities and flexible dynamic reconfigurability, FPGAs are widely used in many fields such as communications and images. However, in the implementation of complex algorithms, FPGAs are not as convenient as embedded processors, so when designing systems with complex algorithms and control logic, they often need to be used in conjunction with embedded processors. This is SOPC (System on a Programmable chip, which can programming system-on-chip) technology. SoPC is the product of the combination of SoC and FPGA, and a single programmable reconfigurable chip completes the main functions of the entire system. The design of SoPC is flexible and changeable, and can be designed with schematic diagram, hardware description language or even C/C++ high-level language; at the same time, it also has the characteristics of reconfiguration, cutting, and expansion, and it is easy to upgrade. SoPC combines the advantages of both SoC and FPGA with the following characteristics:

●Abundant IP core resources, including general IP cores and dedicated IP cores

●Programmable in the system, simple and convenient to design, compile, download and debug

●Built-in embedded soft-core processor, memory, peripheral interface controller

●A large number of programmable and reconfigurable logic resources

●Powerful clock management circuit

●Support a variety of I/O interface standards

In addition, due to the reduction of the transmission line distance between the processor and the memory, SoPC has obvious advantages over board-level systems in terms of speed, signal integrity, and electromagnetic compatibility.

1. Functional block diagram

On the basis of the development of the board-level system of digital video vision acquisition and processing, this paper adopts SoPC to realize the design scheme of motion vision processing and control system. The hardware adopts Altera’s StratixII series FPGA, and the software development tools include QuartusII, NiosII5.1 IDE, DSP Builder, MegaCore IP Library5.1 and Matlab7.0. The working principle of the whole SoPC system is as follows: the video image data collected by the camera comes in through the image sensor interface; the digital signal processing block and the digital image and digital video processing IP core complete the processing of the video image, such as motion detection, segmentation, feature extraction, compression, etc.; The Nios II embedded processor mainly completes the control function of the whole system; the I2C bus is used to access the internal registers of the CMOS image sensor; the PC can access the SoPC through the USB interface, and if the distance is long, the data can be transmitted through the Ethernet port ; The video image and its processed results can be stored in CF card, FLASH and other storage media through the external memory interface, and can also be directly displayed on the LCD Display through the LCD display interface. The functional block diagram of the motion vision SoPC is shown in Figure 1:

Design of Motion Vision Processing System Using StratixII Series FPGA Devices

This module is mainly responsible for the acquisition of video images, the setting of camera parameters and the control of the PTZ stepping motor. Among them, the camera internal control register can be set through the I2C bus. In FPGA, there are two ways to realize the I2C bus controller: one is to use software simulation in Nios II; the other is to use IP cores provided by third parties, such as Sciworx, CAST, Digital Core Design and other companies. These IP cores have operating parameters that can be set as required. Taking the I2C bus controller provided by CAST Company as an example, the maximum transmission rate is 100Kbps, and it can work in 4 modes, namely master transmitter mode, master receiver mode, slave transmitter mode and slave receiver mode. The dual-channel CMOS image sensor interface controls the timing, frame synchronization and line synchronization of video image acquisition; the PTZ control signal controls the rotation of the two stepping motors of the PTZ according to the results of video image processing. These two modules need to be developed by themselves to form IP cores with independent intellectual property rights.

2. Video image processing module

The module includes NiosII 32-bit embedded processor, digital signal processing block, digital image and digital video processing IP

MegaCore, and some other logic circuits, which are the focus and core of the motion vision processing and control SoPC design, are introduced in the following sections.

(1) Nios II embedded processor

The Nios II embedded processor is a general-purpose RISC-structured CPU that targets a wide range of embedded applications. In the Nios II IDE integrated development environment, follow the operation prompts to add and set relevant parameters, and a Nios II embedded processor can be generated within a few minutes. The hardware development process is as follows:

① Analyze the functions to be completed and the performance achieved by the system

② Start SOPC Builder and select the specific FPGA model

③ Define CPU, peripheral devices, storage system and other modules

④ Assign base address and interrupt request number (IRQ) to each module

⑤ The Nios system module is generated, the pins are locked, and the software development process of compiling is as follows:

① Start Nios II IDE in SOPC Builder

② Create a C/C++ software project and specify the target hardware

③ Use the engineering model to write the corresponding program

④ After compiling, it can be downloaded to the hardware to run

Nios II IDE can use C/C++ or assembly language to write programs, and the file extensions are .c and .s respectively. A single Nios II/f CPU needs to occupy about 1800 LEs. If some timers, peripheral devices, etc. are added, the occupied logic unit will further increase.

(2) Digital signal processing block

StraTIx II series FPGAs have digital signal processing blocks (DSP Blocks, DSP blocks). The digital signal processing blocks can support multipliers of different data widths (9×9, 18×18, 36×36) and operation modes (multiply, complex multiply, multiply-accumulate, and multiply-accumulate), each DSP block provides DSP data throughput of 2.8 GMACS. The largest StraTIx II device, EP2S180, contains 96 digital signal processing blocks, which can provide a throughput of 284 GMACS and can support 384 18×18 multipliers. In addition, the digital signal processing block adds new rounding and saturation support to facilitate importing DSP firmware code into the FPGA. Some applications, such as speech processing, may use rounding and saturation due to the fixed width of the memory buffer in which the data is stored. Now with digital signal processing blocks that support rounding and saturation, DSP processor-based designs can be easily imported into FPGAs for implementation.

DSP system design on Altera’s programmable devices requires development tools that support both advanced algorithms and hardware description languages. MathWorks’ MATLAB and Simulink system-level design tools provide algorithm development, simulation, and verification capabilities. Altera’s DSP Builder combines these tools with Altera’s development tools to provide a shared DSP development platform for system design, algorithm design, and hardware design.

(3) IP Core for Video Image Processing The third party provides many customizable IP cores for communication, image coding and decoding, and video processing. Reasonable use of these IP cores can greatly shorten the development time while ensuring performance and reliability. The following is the color space conversion IP.

CSC (Color Space Convertorr) is an IP core dedicated to image color space conversion in the MegaCore IP library file provided by Altera Corporation. Compared with software conversion, it has obvious speed advantages and flexibility:

● Complete the conversion of one pixel per clock cycle

● In StraTIx series FPGA, the clock frequency is greater than 200MHz

● Support interchange between RGB, YCbCr, YUV

● Users can customize the correlation coefficient of the transformation matrix

● Supports signed and unsigned numbers

The data width of input and output is 2~32b 4 RAM data buffer area StraTIx II series FPGA contains up to 9Mb of on-chip RAM. These RAMs use the TriMatrix memory structure and consist of three sizes of embedded memory blocks: 512b M512 block, 4Kb M4K block and 512Kb M-RAM block, each of which can be configured to support various features such as single port RAM, dual-port RAM, FIFO, etc., provide solutions for large storage applications.

3. External memory and peripheral interface

Stratix II series FPGAs are optimized for reliable data transfer from external memory, supporting the latest memory interfaces to access off-chip memory. Using Stratix II’s advanced device features and customizable IP cores, developers can quickly and easily integrate a variety of mass storage devices into complex system designs. Stratix II supports a variety of the latest memory interfaces. The on-chip processors and peripherals of Stratix II series FPGAs are connected through the Avalon switched bus. The Avalon switched bus is a dedicated internal wiring technology developed by Altera, which uses the least logic resources to support data bus multiplexing, address decoding, wait cycle generation, peripheral address alignment, interrupt priority specification, etc. . The customizable IP cores for the peripheral interface include controllers such as USB, I2C, Ethernet, and PCI. Most of these IP cores are provided by third parties, which can be tried for free or purchased at a partial cost. The USB2.0 controller and Ethernet interface controller used in this system are provided by Mentor Company.

4. Clock management circuit

Stratix II series FPGAs have up to 48 high-performance, low-skew global clocks that can be used for high-performance functions or global control signals; up to 12 programmable phase-locked loops (PLLs) with complete clock management and frequency Synthesis capabilities including clock switching, PLL reconfiguration, spread spectrum clocking, frequency synthesis, programmable phase offset, programmable delay offset, external feedback, and programmable bandwidth. Stratix II has two general-purpose PLLs: enhanced PLLs and fast PLLs. The enhanced PLL is rich in functions and supports external feedback, spread spectrum clock, programmable bandwidth, etc. The fast PLL is optimized for high-speed differential I/O interfaces and has a dynamic phase adjustment (DPA) function. These high-speed clock networks and rich PLLs combine to provide a strong guarantee for the system to operate with minimal clock skew.

5. Other components of SoPC

The FPGA configuration interface is used for SoPC configuration, compilation and online debugging; the LCD display interface can be connected to an external LCD screen; the alarm signal is the sound or photoelectric signal issued when the moving target is detected and identified, which can be used for security; standard

The I/O port is reserved for future upgrade and expansion.

Innovation point: Before the concept of SoPC was proposed, the integrated design of Electronic systems was mainly based on the board level. With the continuous improvement of the system clock frequency and the increasing complexity of circuit functions, this design method became more and more difficult to achieve, electromagnetic interference and Signal integrity issues are becoming increasingly prominent. Only by optimizing the layout and routing of the PCB can no longer meet the requirements for high-speed signal transmission and processing. With the continuous development of the semiconductor industry, programmable system-on-a-chip will become the mainstream of future electronic product development and design with its high performance, reliability, low power consumption, cost and good portability. The motion vision SoPC can better solve a series of problems of board-level circuits, and can be widely used in many fields such as security monitoring, visual navigation, and intelligent transportation. It will definitely have a good market prospect.

The Links:   FP10R12W1T4 NRL75-8809A-113 PM100RSE120