Department Publications

The optimal selection of approximate computing units towards low-complexity FPGA-based hardware architectures for approximate discrete cosine transform (ADCT) and approximate discrete Fourier transform (ADFT) are explored. The proposed methodology employs an existing library of FPGA based approximate adders to select Pareto-optimal 1 approximate adders for cascaded signal flow architectures found in ADCT and ADFT operations. Benefits are presented with examples from 8-point ADCT based image compression and 16-point ADFT based digital multi-beamforming applications. Examples demonstrate a nearly 25% LUT reduction in FPGA utilization compared to error-free computing, with a corresponding degradation of nearly 1 dB in terms of PSNR in image compression and 0.3 dB loss in SNR improvement in beamforming.

The combined use of algorithms for approximate discrete cosine transform (DCT) with approximate computing is examined toward area-efficient field programmable gate array (FPGA) hardware architectures. The performance of previously reported, multiplierless approximate DCT transforms are explored in the presence of non-exact hardware adders which provide better utilisation of FPGA resources in terms of look-up-tables (LUTs). In the context of 2D image compression, provided examples are demonstrating a 20% LUT reduction in FPGA utilization compared to accurate adders, with a corresponding image quality degradation of 0.6 dB in terms of PSNR. The potential benefit of such area-efficient FPGA hardware designs is explored on 3D (video compression), 4D (static light field compression), and 5D (light field video compression) cases where significant FPGA resource savings are obtained at the cost of …

Recent work done on traffic sign and traffic light detection focus on improving detection accuracy in complex scenarios, yet many fail to deliver real-time performance, specifically with limited computational resources. In this work, we propose a simple deep learning based end-to-end detection framework, which effectively tackles challenges inherent to traffic sign and traffic light detection such as small size, large number of classes and complex road scenarios. We optimize the detection models using TensorRT and integrate with Robot Operating System to deploy on an Nvidia Jetson AGX Xavier as our embedded device. The overall system achieves a high inference speed of 63 frames per second, demonstrating the capability of our system to perform in real-time. Furthermore, we introduce CeyRo, which is the first ever large-scale traffic sign and traffic light detection dataset for the Sri Lankan context. Our dataset …

Deep neural networks (DNNs) have been successfully employed in a multitude of applications with remarkable performance. As such performance is achieved at a significant computational cost, several embedded applications demand fast and efficient hardware accelerators for DNNs. Previously proposed application specific integrated circuit (ASIC) architectures strive to utilize arrays of hundreds of processing elements (PEs) and reduce power-hungry DRAM accesses using multiple dataflows requiring complex PE architectures. These consume significant area and reduce the maximum clock frequency. This paper introduces the Kraken architecture, which optimally processes the convolutional layers, fully-connected layers, and matrix products of any DNN through a hardware-friendly uniform dataflow. This enables maximal data reuse of weights, inputs, and outputs, with a bare-bones PE design and on-the-fly dynamic reconfiguration. Kraken, implemented in 65-nm CMOS technology at 400 MHz, packs 672 PEs in 7.3 mm2, with a peak performance of 537.6 Gops. Kraken processes the convolutional layers of AlexNet, VGG-16, and ResNet-50 at 336.6, 17.5, and 64.2 frames/s, respectively, hence outperforming the state-of-the-art ASIC architectures in terms of overall performance efficiency, DRAM accesses, arithmetic intensity, and throughput, with 5.8x more Gops/mm2 and 1.6x more Gops/W.

Making a single network effectively address diverse contexts---learning the variations within a dataset or multiple datasets---is an intriguing step towards achieving generalized intelligence. Existing approaches of deepening, widening, and assembling networks are not cost effective in general. In view of this, networks which can allocate resources according to the context of the input and regulate flow of information across the network are effective. In this paper, we present Context-Aware Multipath Network (CAMNet), a multi-path neural network with data-dependant routing between parallel tensors. We show that our model performs as a generalized model capturing variations in individual datasets and multiple different datasets, both simultaneously and sequentially. CAMNet surpasses the performance of classification and pixel-labeling tasks in comparison with the equivalent single-path, multi-path, and deeper single-path networks, considering datasets individually, sequentially, and in combination. The data-dependent routing between tensors in CAMNet enables the model to control the flow of information end-to-end, deciding which resources to be common or domain-specific.

SMTPE ST 2110 is a professional video over IP standard which enables uncompressed video to be transported as separate video, audio and ancillary data streams, allowing true flexibility in media workflows. Uncompressed video transmission allows professional video environments to achieve high quality video distribution while meeting strict latency requirements. Although this standard can be implemented on software, hardware solutions can achieve superior performance, lower latency and lower power consumption while efficiently handling jitter and packet bursts. This paper presents an FPGA based hardware implementation of a SMPTE ST 2110 compliant end to end solution supporting uncompressed video resolutions of up to 4K at 30fps. Due to its scalability, the architecture can be easily extended to support higher video resolutions and/or multiple video streams. Additionally, the interface modularity of the …

This paper describes a hardware and software codesign approach for object detection and tracking algorithm which is based on modified ViBe background subtraction method and scale adaptable particle filtering algorithm. The modified ViBe performs with a close accuracy to the original work of ViBe but with less complexity and it outperforms the Mixture of Gaussian(MoG) method. Since the modified algorithm has a low off chip memory access requirement, the real time object detection in embedded level with the help of hardware acceleration techniques could be achieved. The implemented particle filtering algorithm is capable of adapting to the scale variation of the tracking objects. After careful investigation of the time consumption profile of each function in our algorithm, the complete algorithm was implemented in IoT edge devices targeted Avnet Ultra 96 development board which is based on Xilinx Zynq …

Quad High Definition (QHD/4K) Virtual Reality (VR) Headsets with High Dynamic Range (HDR) technology will provide a superior experience to users. Wireless VR headsets are attractive as it offers freedom on mobility and higher user comfort. Cloud and workstations are capable of rendering 4K HDR scenes in real-time with ease. The efficient transmission of these scenes through wireless network requires the use of compression techniques. Decoding 4K HDR frames at VR devices in real-time is a challenge. Given the power and memory efficiency of the hardware decoders, in this paper we propose JPEG-XT Profile C based hardware architecture to decode 4K HDR frames in real-time. We employ novel methods for merged base and residual layer decoding pipelines, design of probability-based hybrid Huffman lookup table architecture and sparsity aware inverse zig-zag processing. Our decoder is implemented …

In recent years, Convolution Neural Network (CNN) gained great success in many applications, especially in computer vision. Now adapting CNN inference on edge devices has become an active research area in embedded vision and hot topic in Edge AI. The primary design hurdles for implementing CNN inference on embedded systems are limited computational resource, memory resource, and power budget. This study presents a novel architecture for SqueezeNet [1] like CNN models, which can be extended to support any CNN model as well. We address two approaches to mitigate resource constraints. First, we use a custom floating point(12 bit for computation and 8bit for storing). Second is slicing the model into a repetitive block called computation blocks. Computation block can be configured dynamically by the host processor to operate in a different mode. We have implemented SqueezeNet v1.1 for …

Convolutional Neural Networks (CNNs) are widely used in deep learning applications, e.g. visual systems, robotics etc. However, existing software solutions are not efficient. Therefore, many hardware accelerators have been proposed optimizing performance, power and resource utilization of the implementation. Amongst existing solutions, Field Programmable Gate Array (FPGA) based architecture provides better cost-energy-performance trade-offs as well as scalability and minimizing development time. In this paper, we present a model-independent reconfigurable co-processing architecture to accelerate CNNs. Our architecture consists of parallel Multiply and Accumulate (MAC) units with caching techniques and interconnection networks to exploit maximum data parallelism. In contrast to existing solutions, we introduce limited precision 32 bit Q-format fixed point quantization for arithmetic representations and …

Screen content coding (SCC) extension to High Efficiency Video Coding (HEVC) offers substantial compression efficiency over the existing HEVC standard for computer generated content. However, this gain in compression efficiency is achieved at the expense of further computational complexity with several resource hungry coding tools. Hence, extension of SCC to HEVC hardware encoders can be challenging. This paper presents resource efficient hardware designs for two key SCC tools, Intra Block Copy and Palette Coding. Moreover, a new hash search approach is proposed for Intra Block Copy, while a hardware friendly palette indices coding scheme is suggested for Palette Coding. These designs are targeted to achieve the throughput necessary for an 1080p 30 frames/s encoder, and incurs coding loss of 11.4% and 5.1% respectively in all intra configurations. The designs are synthesized for a Virtex-7 …

Higher compression efficiency in HEVC encoders comes with increased computational complexity, making real time encoding of high resolution videos a challenging task. This challenge can be addressed by software, yet hardware solutions are more appealing due to their superior performance and low power consumption. This paper presents an FPGA based hardware implementation of an all intra HEVC encoder, which can encode 8 bits per sample, 1920×1080 resolution, 30 frames per second raw video, that is viable in real time even at low operating frequencies. A major obstacle to real time encoding in available architectures is the dependency created by reference generation. Moreover, each coding unit (CU) has to be processed in multiple configurations to determine the most efficient split and prediction mode representation, based on the bit stream generated. We propose a new three stage architecture to …

Increasing demands for faster, more responsive and manageable networks have brought about a different approach to networking - Software Defined Networking (SDN). SDN introduces flow-based networking with a centralized controller. Matching incoming traffic flows with rules sent by the controller is an integral part of traffic forwarding in SDN. How efficiently this is carried out has a huge impact on network performance. In this paper, we present a high-performance, low-cost architecture for flow matching in an SDN switch. Our architecture consists of a dedicated unit for flow matching and a custom processor. Novel design of the flow match unit reduces matching latency significantly, when compared with the generic architecture, while also reducing usage of costly resources like TCAMs. When testing on FPGA platforms, implementing the system required only a minimum amount (around 10% on Virtex-7) of …

With the popularization of a quad high-definition/4K video being dependent on the availability of real-time High Efficiency Video Coding (HEVC) decoders, hardware implementations have become more appealing due to their superior performance and low power consumption. In this paper, a field-programmable gate array (FPGA)-based hardware implementation of a 4K 30 frames/s real-time HEVC decoder is presented. An elastically pipelined decoder architecture is used to absorb variations in processing time and each pipeline stage is optimized to use available FPGA primitives. FPGA-specific challenges in managing critical path delays to achieve a target operating frequency of 150 MHz required many architectural novelties, such as exploitation of the sparsity of transformed coefficient matrix, single-cycle reference pixel processing in intra prediction, and flexible block ordering in deblocking filter/sample …

The inverse transform function of the HEVC Decoder has grown greatly in complexity with the addition of larger transform sizes and recent works have focused on efficient architectures that can achieve the required throughput. In this work we make the observation that a majority of the coefficients in a typical transform operation is zero, and therefore has no impact on the final outcome. We propose an architecture that can efficiently operate on such sparse matrices and introduce a scheduling strategy which completes a 2D IDCT with bare minimum iterations, with the added advantage of being able to integrate seamlessly with the entropy decoder without a coefficient reordering buffer. Experiments show that although the performance of this approach is scalable with the bit rate, a 120 MHz operating frequency is sufficient to handle QHD @ 48 Mbps, which is less than one third of the frequency requirement of prior work.

Complex Event Processing (CEP) is an emerging field in high performance computing paradigm where real time (low latency) computing capability is expected over big data processing (high throughput). Significant number of software architectures have been developed to improve the throughput while reduce the latency but maintaining of the both aspects reaches the limits of the software platforms. This paper proposes a novel custom hardware accelerator architecture framework for CEP in big data domain. The proposed design improves the throughput performance more than 10 times over the software counterpart while keeping the latency value at less than 100 nano seconds. Same Structured Query Language(SQL) type queries used in reference software architecture were used to improve the flexibility. A query compiler based on the same query language grammar was designed to convert the queries in to …

Packet Classification is the enabling function for many Internet functions like QoS and Security. In this paper we propose a Classification engine architecture which exploits parallelism to increase throughput. The architecture also make use of the Temporal locality observed in Internet traffic positively by employing Popular Rule Caching mechanism to increase the classification throughput. We also introduce Rule Splitting mechanism to increase the accuracy of the rule caching mechanism. Simulation results revealed that the architecture is capable of achieving a throughput of more than 200Gbps when lowest amount of temporal locality is present for worst case packet size of 40 bytes.

Motion Blur due to the relative motion between the camera and object can seriously degrade image quality. We present an FPGA based motion blur detection and correction algorithm which is implemented on top of a configurable soft-processor based architecture. The system consists of two main modules. The blur detection module identifies the blur length and angle, and the restoration module uses regularized inverse filtering to remove the motion blur. The Processing algorithms are implemented as separate cores on the FPGA fabric where the soft processor core is only used for managing system configuration. The system can achieve a frame rate of 15fps for a 720p HD video stream.

In order to obtain depth perception in computer vision, it is needed to process pairs of stereo images. This process is computationally challenging to be carried out in real-time, because it requires the search for matches between objects in both images. Such process is significantly simplified if the images are rectified. Stereo image rectification involves a matrix transformation which when done in software will not produce real-time results although it is very demanding. Therefore, the video streaming and matrix transformation are not usually implemented in the same system. Our product is a stereo camera pair which produces a rectified real time image output with a resolution of 320×240 at a frame rate of 15FPS and delivers them via a 100-Ethernet interface. We use a Spartan 3E FPGA for real-time processing within which we implement an image rectification algorithm.

Control the data flow between device interfaces, processing blocks and memories in a vision system is complex in hardware implementation. In the research, high-level synthesis tool is used to design, implement and test the vision system within the context of required control, synchronization, and parameterization on a processor based platform. In addition, both HLS tools and HDL were used for the development of the processing cores, and the performance of the two versions were analyzed and compared. The operational structures of benchmarked vision core consist of custom vision coprocessor with efficient memory and bus interfaces. The performance properties such as accuracy, throughput and efficiency are measured and presented. Xilinx XC5VLX110T FPGA, has been used for prototype the hardware platforms. According to results, without any complex optimizations, pipeline length and resource …

Real-time image processing demands much more processing power than a conventional processor can deliver. As a result hardware acceleration became necessary to augments processors with application-specific coprocessors. Due to the limited resources on FPGA and nature of some sequential algorithms, it is difficult to depend entirely on slice resources. In this research, we implemented a multiprocessor architecture to support real-time image processing on FPGA. Furthermore, we benchmarked and compared our implemented architectures with their counterparts. The operational structure of multiprocessor architecture consists of on-chip processors implemented in a parallel manner with efficient memory and bus architectures. The performance properties such as accuracy, throughput and efficiency are measured and presented. Multiprocessor systems are effective in software level parallelism on FPGA. Our …

The authors report a VLSI design of an advanced systolic array graphics (SAG) engine built from pipelined functional units which can generate realistic images interactively for high-resolution displays. They introduce a structured frame store system as an environment for the advanced SAG engine and present the principles and architecture of the advanced SAG engine. They introduce pipelined functional units into this SAG engine to meet the performance requirements. This is done by a formal approach where the original systolic array is represented at bit level by a finite, vertex-weighted, edge-weighted, directed graph. Two architectures built from pipelined functional units are described. A prototype containing nine processing elements was fabricated in a 1.6- mu m CMOS technology.<>

An array processor design methodology suitable for hard real-time systems is presented. Scheduling and projection of the dependence graph (DG) are solved using integer programming. By exploring the regularity of the DG it is possible to solve the necessary IP problems in an efficient manner. This methodology provides a unified approach for linear and nonlinear projection of regular and partially regular DGs.<>

A Token- Based Arbitration Mechanism for Inter-Systolic-Array Communication - University of Twente Research Information Skip to main navigation Skip to search Skip to main content University of Twente Research Information Home University of Twente Research Information Logo Home Profiles Research units Projects Research output Datasets Activities Prizes Press/Media Search by expertise, name or affiliation A Token- Based Arbitration Mechanism for Inter-Systolic-Array Communication F. El hadidy, JAKS Jayasinghe, OE Herrmann Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review Overview Original language Undefined Title of host publication Proceedings of the International Conference on Microelectronics ICM'91 Place of Publication Cairo, Egypt Pages 271-275 Number of pages 0 Publication status Published - 21 Dec 1991 Keywords …