Department Publications

Predicting the possible future trajectories of the surrounding dynamic agents is an essential requirement in autonomous driving. These trajectories mainly depend on the surrounding static environment, as well as the past movements of those dynamic agents. Furthermore, the multimodal nature of agent intentions makes the trajectory prediction problem more challenging. All of the existing models consider the target agent as well as the surrounding agents similarly, without considering the variation of physical properties. In this paper, we present a novel deep-learning based framework for multimodal trajectory prediction in autonomous driving, which considers the physical properties of the target and surrounding vehicles such as the object class and their physical dimensions through a weighted attention module, that improves the accuracy of the predictions. Our model has achieved the highest results in the nuScenes trajectory prediction benchmark, out of the models which use rasterized maps to input environment information. Furthermore, our model is able to run in real-time, achieving a high inference rate of over 300 FPS.

The Kalman filter is often used with plants without zeros. However, when the plant has zeros, the Kalman filter tends to neglect the effects of zeros, leading to errors in output estimation. Without the effects of zeros, the Kalman filter estimation would lag behind the true output, or it would lead to true output if the zeros are on the RHP. The parallel Kalman filter bank, instead of a single filter is proposed to remedy this problem. Each Kalman filter in the bank operates on a plant submodel that is obtained by decomposing the plant model. The output estimations of all of the Kalman filters in the bank are added together to synthesize the plant output estimation. The method was simulated in a PID-controlled system where better results have been obtained.

The quadrotor is an under-actuated nonlinear system that poses challenges in modeling and controller design. This work uses the translational (Newtonian) and rotational (Eulerian) dynamics of the quadrotor to build a decoupled control system. A linear quadratic Gaussian (LQG) regulator is used as the controller, and two extended Kalman filters are used for state estimation in the two dynamic models. The entire design process from modeling to flight simulation is presented. Model linearization while preserving controllability, state-space modeling under gravity disturbance, feedforward compensation, LQR gain adjustment, actuator dynamics, sensor system for observability, adaptation of extended Kalman filter, and its limitations are presented.

Oceans cover about 70% of Earth's surface and it is difficult to keep them under continuous surveillance with ever increasing criminal activities. Conventional surveillance methods utilize manned vessels to patrol large sea areas, and due to repetitive-monotonous nature, the personnel involved can often make inevitable human errors due to fatigue. As an alternative, solar-charged battery-powered surface vessel that can autonomously navigate in maritime waters is proposed. Autonomous surveillance vessels are required to navigate in oceans for an extended time duration, requiring them to use much power in propulsion. To properly manage propulsion power, the vessel design must minimize the drag acting upon the vessel. In this work, computational fluid dynamics (CFD) techniques are used to identify the most suitable type of vessel shape that can minimize the drag and maximize the lift and stability. According …

This paper addresses the problem of high energy consumption in the standard take-off mode of quad-plane unmanned aerial vehicles (UAV). In the standard mode, the quad-plane takes-off vertically using the quad mode, and when the predefined altitude is reached the plane mode kicks in. However, the" bird" mode, where both quad and plane modes kick in together, the UAV could achieve altitude and airspeed while consuming lower energy. This paper presents the energy consumption and comparison in standard and bird take-off modes with actual experimental results obtained through flight tests of a small 1.35 kg quad-plane. The experiments showed that as much as 22% energy can be saved if the bird take-off mode is used.

Image generation from a single image using generative adversarial networks is quite interesting due to the realism of generated images. However, recent approaches need improvement for such realistic and diverse image generation, when the global context of the image is important such as in face, animal, and architectural image generation. This is mainly due to the use of fewer convolutional layers for mainly capturing the patch statistics and, thereby, not being able to capture global statistics very well. We solve this problem by using attention blocks at selected scales and feeding a random Gaussian blurred image to the discriminator for training. Our results are visually better than the state-of-the-art particularly in generating images that require global context. The diversity of our image generation, measured using the average standard deviation of pixels, is also better.

We propose self-supervised deep algorithms to detect anomalies in heterogeneous autonomous systems using frontal camera video and IMU readings. Given that the video and IMU data are not synchronized, each of them are analyzed separately. The vision-based system, which utilizes a conditional GAN, analyzes immediate-past three frames and attempts to predict the next frame. The frame is classified as either an anomalous case or a normal case based on the degree of difference estimated using the prediction error and a threshold. The IMU-based system utilizes two approaches to classify the timestamps; the first being an LSTM autoencoder which reconstructs three consecutive IMU vectors and the second being an LSTM forecaster which is utilized to predict the next vector using the previous three IMU vectors. Based on the reconstruction error, the prediction error, and a threshold, the timestamp is classified as either an anomalous case or a normal case. The composition of algorithms won runners up at the IEEE Signal Processing Cup anomaly detection challenge 2020. In the competition dataset of camera frames consisting of both normal and anomalous cases, we achieve a test accuracy of 94% and an F1-score of 0.95. Furthermore, we achieve an accuracy of 100% on a test set containing normal IMU data, and an F1-score of 0.98 on the test set of abnormal IMU data.

Extremum seeking control (ESC) is a data-driven optimization technique that can steer a dynamic plant toward an extremum of an unknown but measurable input to steady-state output. In the context of networked control systems, a new implementation method for ESC inspired by the well-known Luus-Jaakola algorithm is proposed. The main motivation is to minimize the communication burden associated with the search phase of ESC. In the proposed method, the controller only requires a notification of a change registered at the sensor, rather than the full information available at the sensor. This event-based approach leads to sporadic packet transmission. In addition, the proposed method is able to directly account for constraints while seeking the desired extremum. The constraints may be of the inequality or equality type. The algorithm's behavior is illustrated on a networked water pump control system.

This paper has addressed the problem of transporting a long object from a given starting location to an end location using leader-follower robot formation in an environment filled with obstacles. Only the leader robot has the authority to make decisions. It possesses the main controller and is capable of navigating through an unknown environment using the data provided by on-board sensors. In the proposed system, a vision-based feedback method is combined with an explicit control method that oversights the virtual trailer between the leader and the follower. The parameters required to maintain a constant distance between the two robots are computed by the leader and transmitted to the follower directly. A fiducial marker is mounted on the front of the follower robot and is captured by a camera attached to the leader robot. Information exchanging methods based only on impedance control can be expensive …

Travelling along routine paths on foot is an exhausting and mundane task. Current personal mobility vehicles (PMVs), which address this issue, are driven manually and are too heavy to be carried by the user when needed. As a solution to this, an autonomous, self-balancing, lightweight, compact and low-cost PMV is developed. The device is capable of simultaneously self-balancing and navigating autonomously in an indoor environment. Moreover, it can be manually controlled using the developed smart mobile phone application. Self-balancing, localization, navigation and obstacle avoidance are achieved in real time using on-board sensors and processing units. Systematic experiments are carried out with the developed PMV in typical indoor environments in order to assess the robustness of the above features. The results show that the robot is able to successfully achieve its navigation tasks both with and …

The widespread use of high definition cameras for surveillance and related tasks has given rise to the concept of edge computing as transmitting and processing video streams in real time have become challenging. However, edge computing at low power and lower cost is difficult with general purpose processor hardware inside cameras. Finding a solution that meets the above requirements and demonstrates flexibility to handle diverse conditions is challenging. In this paper, we propose a coprocessor architecture which is specifically designed to perform machine vision related operations at the edge using very long instruction word (VLIW) architecture. It also supports multiple vision algorithms in the same hardware platform while supporting runtime reconfigurability, architectural flexibility, and extensibility. The system was practically realized on ZedBoard and verified for correct functionality and accuracy at 148.5 …

In this paper, a feasibility study for construction and control of a novel design of a Multi-quadrotor unmanned aerial vehicle (UAV) is presented. This novel concept tests four identical Quadrotors assembled in a bigger cross structure. And, it also tests the ability of using the same controller that is used to control a single quadrotor to fly the novel design of cross assembled four quadrotor UAV. The main objective here is to improve redundancy and robustness and load carrying capacity of quadrotor UAVs. Even though there are four quadrotors, there is only one controller involves. When each individual Quadrotor works on a certain flight maneuver, it is expected that the assembled UAV of four of these in a cross structure to also be able to demonstrate the same maneuver. The prototype of the novel UAV was designed and built, and has been experimentally tested for a proof of concept where it was able to stabilize the …

Personal and medium scale farming has been showing a downward trend throughout the past few decades. Individuals are demotivated to engage in farming activities, due to lack of time, and the higher efficiency in large-scale farming. However, there is a significant health concern arising from the consumption of these products, as they are grown using artificial fertilizers and contain residues of insecticides and pesticides. Automation can motivate personal and medium scale farming, through cutting down the time requirements for farming and by increasing the farming efficiency. Our solution - Scalable Autonomous Agronomical Smartbot (SAASbot) aims at automating farming activities (planting seeds, watering, fertilizing, weed removal) while providing a scalable robotic platform for personal to medium scale farming. In this paper, design and implementation of the SASSBot and testing and validation of the system …

We present a simulation study of a cooperate car following application implemented by the Wireless Access for Vehicular Environ-ments (WAVE) protocol stack, which uses the MAC and Physical lay-ers defined by the IEEE 802.11 p protocol. This is a simple Intelligent

This paper introduces a novel traffic reduction method for Networked Control Systems (NCS) called Deadband Error Modulation (DEM). The deadband modifies the periodic dynamics of Delta Modulation (DM) based techniques such as our previously proposed Event Triggered Adaptive Differential Modulation (ETADM). Like ETADM it can be used for encoding individual scalar elements of a vector (i.e., the state vector or control input). It is also robust to bounded packet drops.

We present a simulation study of a cooperate car following application implemented by the Wireless Access for Vehicular Environments (WAVE) protocol stack, which uses the MAC and Physical layers defined by the IEEE 802.11p protocol. This is a simple Intelligent Transportation Systems (ITS) application which forms the foundation for autonomous driving. As with many safety-critical ITS applications, cooperate car following uses beaconing as the primary means of communication. The beacon frequency and the transmit power of the mobile nodes are considered as the key communication parameters, while the vehicle density is considered as the key traffic parameter. The two main factors affecting the performance, the end-to-end delay and the packet loss, are studied for different scenarios defined by the above parameters. Results indicate how the application performance may be improved by adapting the …

This paper presents on the stability analysis for RAVAN twin boom H-Tail Medium Scale UAV which has been developed by University of Moratuwa. It is required to understand the UAV behavior for the small perturbations in high altitudes while it is flying to be invisible. Indeed, without doing a proper stability analysis in different flying conditions, there is a high risk to fail the UAV for perturbations. Not only that but also it will help to optimize aerodynamic behavior for prolonging the flight endurance and effective deployment. The proper stability analysis will facilitate to increase the control flexibility.XFLR5 computational fluid dynamic software is used to do this stability analysis.

Since recently, UAVs (Unmanned Aerial Vehicles) have increasingly been recognized as a reliable, and effective means of aerial surveillance. Deployment of UAVs in disaster management for aerial surveillance and real-time video feed, when land access is cut off, is one of the most prominent applications. The ability of small, lightweight UAVs in aerial access and gathering data, in areas that are insecure or impossible for humans to reach has significantly improved remote accessibility. In this research, a new low cost small scale UAV with fully-fledged autopilot system on board the aircraft with the capabilities of following a predefined path and returning home autonomously has been designed, built, and tested successfully. This small-scale UAV is capable of navigating through given GPS coordinates at given altitude, while providing real-time flying status back to the ground control station.

Memory Based Event Triggering (MBET) is a pragmatic approach to reduce the effective sampling rate of a control system. In MBET, periodically sampled state variables and control inputs are encoded such that data transmission takes place only when they change by a value that exceeds a predetermined threshold. This results in a significant reduction of network traffic in Networked Control Systems (NCS). Should MBET encoding result in an asymptotic cyclic behavior (limit cycle like behavior), it would result in a lower bound in the required bandwidth. In this paper, the necessary conditions for the formation of such a cycle are investigated. It is demonstrated analytically as well as through empirical simulations that the main requirement for the occurrence of asymptotic dynamic cycles is a diffeomorphism between a scalar control input and a unique equilibrium point to which the system will converge to for a constant …

Frequently, a physical plant of a control system has an optimum operating point such as the spark (or injection) time of an internal combustion engine that results in maximum torque. Extremum Seeking Control (ESC) is a method of adaptive control capable of locating and maintaining a plant at such an optimum operating point in real time. It is capable of doing so with minimal a priori knowledge of the plant and can also track slowly varying changes. Input perturbed ESC schemes that use periodic dither signals have the disadvantage of requiring a high bandwidth for sampling and correlating the plant output with the dither signal. If the feedback path was to be implemented over a packet switched communication network, the high bandwidth requirement could result in increased congestion and consequently packet delays and dropouts. As a solution encoding using sporadic (aperiodic) sampling techniques can be used in the feedback path of the ESC scheme to reduce the required bandwidth. However, in order to ensure convergence of the ESC scheme with encoding, the effect of the signal reconstruction error due to encoding on the critical correlation stage has to be investigated. The contribution of this paper is an investigation of the convergence requirements and bandwidth performance of two encoding schemes; Memory-Based Event Triggering (MBET) and Event Triggered Adaptive Differential Modulation (ETADM). The results show that MBET can fail for objective functions with plateaus. ETADM fails when the number of ETADM steps used for reconstructing the plant output per perturbation cycle is too low to allow correlation. In terms of …

Large debris created after natural disasters restrict access to inner parts of affected regions and slows down disaster relief operations. Power shovels are often used to clear wreckage but the process can take a lot of time. Moreover, it is dangerous to involve human workers in such conditions due to loose debris collapsing and causing severe damage. To speedup access to inner areas, instead of removing obstacles, they can be surmounted with the assistance of a carefully maneuvered power shovel arm. In this paper, an autonomous obstacle surmounting technique for an unmanned power shovel is proposed. Out of different sequences, the one that optimizes the total energy consumption is selected as the best candidate for surmounting a given obstacle by dynamic simulations. The experiments conducted using a miniaturized power shovel show the effectiveness of the proposed method.

Large debris created after natural disasters restrict access to inner parts of affected regions, and slows down disaster relief operations. Power shovels are often used to clear wreckage but the process can take a lot of time. Moreover, it is dangerous to involve human workers operating heavy machinery in such unstable conditions. To speedup access to inner areas, obstacles can be surmounted with the assistance of carefully maneuvered power shovel arm, instead of removing them. In this work, an autonomous obstacle surmounting technique for an unmanned power shovel is proposed. Out of different sequences, the one that optimizes the total energy consumption is chosen as the best candidate for surmounting a given step-like obstacle. Dynamic simulation results show the effectiveness of the proposed method.

Pitch and airspeed responses of a medium scale UAV were identified using the prediction error method. Data collection was conducted with an active closed loop autopilot control system. A naive SISO open loop transfer function was estimated using the parametric direct approach. Since its predictability was not significant, better estimators were developed using MISO parametric direct, and SISO parametric indirect method. The model validations were done to quantify their predictability.

Power shovels are used to clear wreckage after natural disasters but the process can consume a lot of time. Furthermore, operating such heavy machines in those circumstances is very dangerous for the involved human workers. To speed up access to inner parts of disaster stricken areas, obstacles can be surmounted with the assistance of carefully maneuvered arm, instead of removing them. In this paper, an autonomous obstacle surmounting technique for an unmanned power shovel is proposed. Out of different sequences, the one that optimizes the total energy consumption is chosen as the best candidate for surmounting a given step-like obstacle. Simulation and experiment results show the effectiveness of the proposed method.

Delay variation or jitter is an inherent feature of packet switched communication networks due to bottlenecks. In communication systems buffering is commonly used to mitigate in real time multimedia applications at the expense of an additional delay. This method is therefore not suitable for control systems implemented over packet switched communication networks due to the additional delay which can make the system unstable. Hence, jitter modelling and management is necessary. This paper focuses on the development and experimental verification of a suitable jitter model for wireless and wired network bottlenecks. The effects of jitter are then investigated by quantifying the performance of a simulated Networked Control System (NCS). According to the results, the reduction in attenuation due to jitter compared to a jitter free benchmark ranges from approximately 0.01-0.04dB depending on the encoding type.