Department Publications

Vision-Language Navigation (VLN) enables robots to navigate through environments using natural language instructions, making human-robot interaction intuitive. Traditional VLN models often rely on navigation graphs, 360-degree views, and perfect localization which pose significant challenges when adapting these models to real-world settings. This work addresses these limitations by performing a simulation-to-real domain shift of a VLN approach that operates in continuous environments without requiring navigation graphs or panoramic views. The proposed system integrates vision-language models that align visual inputs and linguistic instructions within a shared embedding space, facilitating natural language-driven navigation. We employ a Cross-Modal Attention (CMA) based architecture trained on an existing dataset in a simulated environment and fine-tune it using real-world data collected from a …

A sense of independence is critical for individuals with mobility impairments, with stair negotiation remaining one of the most demanding daily tasks. This article presents a novel track-based stair-climbing mechanism inspired by leech locomotion, designed to enhance adaptability to varying staircase geometries. The mechanism emulates leech-like motion during both ascent and descent, decomposing each maneuver into multiple segments to ensure stable posture and safety. Throughout motion, the mechanism maintains a large contact surface with the stairs, improving stability. Simulation and experimental studies validate the effectiveness of the proposed motion, particularly during transitions at the top and bottom of the staircase. Results demonstrate that the motion profile reduces abrupt movements and improves user comfort during stair-climbing operation.

Solar photovoltaic (PV) systems have become increasingly popular as a sustainable energy source due to the growing need for renewable energy on a global scale. However, the buildup of dust and dirt on solar panels frequently compromises the effectiveness of these systems. Existing cleaning methods often require manual labor, exhibit low efficiency, and are restricted to either a single cleaning method or operational mode. To address these issues, we designed and implemented a semi-autonomous solar panel cleaning robot capable of both dry and wet cleaning. The robot can autonomously navigate on solar panels with a tilt of up to 20 or be manually controlled using a mobile app, and it operates for approximately 75 minutes on a single charge. This study enhances key aspects of the robotic system, integrating advanced edge detection and transition mechanisms using both ultrasonic sensors and image processing techniques. We implemented a robust localization system based on bus-bar detection for precise navigation, upgraded the water supply mechanism and refined the orientation estimation algorithm.

Robust feature tracking is a basic requisite for indoor robot navigation. However, indoor scenes mostly contain poorly localizable features. As a result, indoor feature tracking without artificial markers is challenging and remains an attractive problem. We propose to solve this problem by constraining the locations of a large number of non-distinctive features by several planar homographies which are computed using distinctive features. Use of a large number of non-distinctive features within the constraints imposed by planar homographies allows us to gain robustness against the absence of artificial landmarks. Also, the lesser computational cost in estimating these non-distinctive features helps to maintain the efficiency of the proposed method. Our outcome is a robust in door feature tracker. In this dissertation, we will establish that multi-homographic in door feature tracking is feasible with the combination of …

With the recent advancement of Large Language Models (LLMs), environmental perception methods, and frameworks for simplifying robot functionality implementation, robotic receptionists present an exciting opportunity to minimize humanrelated errors. However, most existing humanoid robots are costly and overqualified for such tasks, as they are primarily designed for research applications. To address this, we introduce a standalone, offline, cost-effective smart mobile robot receptionist built on an edge device that efficiently performs a predefined set of tasks while utilizing minimal hardware and computational resources. Our robot receptionist can engage in smooth conversations, recognize and interact with both new and returning users, perform hand gestures, provide navigation assistance through verbal and physical guidance, and autonomously navigate while avoiding static and dynamic obstacles …

Solar photovoltaic (PV) systems have become a trend as a sustainable energy source on a global scale. However, the dirt accumulated on solar panels often reduces their efficiency, considerably. Existing cleaning methods have limitations such as requiring manual labour, low efficiency, limited to single cleaning method or single operational mode. As a solution, this research study describes the design and implementation of a semi-autonomous solar panel cleaning robot for both dry and wet cleaning. The robot can autonomously navigate on solar panels safely, or user can manually control it using a mobile app. The robot is capable of functioning for around 75 minutes on a single charge on solar panels tilted up to 20°, facilitated by image processing algorithms achieving an accuracy surpassing 96%. The orientation estimation is accomplished through the implementation of three parallel Kalman filters running on …

This study aims to check the behaviour of various controllers that can successfully control a catamaran vessel on the ocean surface for trajectory tracking. Unlike on the ground, the ocean surface has disturbances from water currents, wind forces and ocean waves. Therefore, trajectory tracking is challenging work. A 3D model of a catamaran vessel was simulated to check the vessel’s behaviour with the controllers: Polynomial recurrent neural network-based adaptive PID controller, first and second-order sliding mode controllers, and first-order adaptive controller are among them. The control system is a combination of two controllers. The primary controller controls the vessel’s surge and yaw according to the velocity error. The secondary controller controls the velocity errors required by the primary controller, according to the displacement error of the catamaran vessel from the trajectory and the required forward …

The ability to accurately localize vehicles is essential for driver assistance applications. Existing modern localization systems, while sophisticated, often come with considerable costs and limitations in accessibility. Recognizing the need for a more accessible solution, there is a growing demand for user-friendly and cost-effective alternatives. One such alternative lies in leveraging mobile devices for localization. However, relying solely on Global Positioning System (GPS), Wi-Fi, or cellular networks, the most commonly used technologies in mobile device localization, often suffer from issues with accuracy. Addressing these drawbacks is crucial to ensuring its effectiveness and widespread adoption. Map matching is a commonly pursued and well-researched approach taken to rectify these erroneous location data. This paper presents the implementation of a hybrid of two map-matching implementations on a mobile …

In disaster-stricken environments, it’s vital to assess the damage quickly, analyse the stability of the environment, and allocate resources to the most vulnerable areas where victims might be present. These missions are difficult and dangerous to be conducted directly by humans. Using the complementary capabilities of both the ground and aerial robots, we investigate a collaborative approach of aerial and ground robots to address this problem. With an increased field of view, faster speed, and compact size, the aerial robot explores the area and creates a 3D feature-based map graph of the environment while providing a live video stream to the ground control station. Once the aerial robot finishes the exploration run, the ground control station processes the map and sends it to the ground robot. The ground robot, with its higher operation time, static stability, payload delivery and tele-conference capabilities, can then …

Oceans cover about 70% of Earth's surface and it is difficult to keep them under continuous surveillance with ever increasing criminal activities. Conventional surveillance methods utilize manned vessels to patrol large sea areas, and due to repetitive-monotonous nature, the personnel involved can often make inevitable human errors due to fatigue. As an alternative, solar-charged battery-powered surface vessel that can autonomously navigate in maritime waters is proposed. Autonomous surveillance vessels are required to navigate in oceans for an extended time duration, requiring them to use much power in propulsion. To properly manage propulsion power, the vessel design must minimize the drag acting upon the vessel. In this work, computational fluid dynamics (CFD) techniques are used to identify the most suitable type of vessel shape that can minimize the drag and maximize the lift and stability. According …

This paper introduces a novel locomotion strategy of combining wheeled and legged actuation in a parallel structure. This enables the robot to traverse rough terrains using the articulated limbs and use wheeled locomotion for more efficient traversal over smooth terrains. Authors present a prototype robot architecture to embody the concept and show experimentally the robot traversing a variety of indoor and outdoor terrains. Additionally, a novel butterfly gait is introduced for the robot that enables stable and efficient traversal of challenging terrains.

This research paper presents a novel track based stair-climbing wheelchair design with improved stability and performance. The proposed design has an adaptable arrangement to the staircase especially at the top of the staircase. In the existing wheelchair designs, this particular feature has not been adequately addressed. The proposed design consists of body frame and supporting rubber track arms. Of the two main operations; ascending and descending, the most critical operations is at the top of the staircase, which is smoothly controlled by following the staircase profile. This motion is achieved by properly synchronizing the rubber track arms according to a pre-planned motion profile. As a result, user does not feel sudden motions in position and attitude. Furthermore, the proposed wheelchair is adaptable to any staircase.

We present a novel low cost time-of-flight (ToF) sensor deck for micro UAVs capable of sparse range measurements for localisation and mapping. In disaster scenarios such as building collapses, the larger UAVs equipped with highfidelity sensors such as lidars, structured light cameras and stereo cameras to perceive the environment with high resolution, may have difficulties when navigating through cluttered confined spaces. Micro UAVs are ideal for providing vital data in the aftermath of a disaster scenario but does not have the payload capacity for such sensors. Miniature low cost ToF sensors can provide range measurements albeit with low range and resolution. The proposed solution includes the design and implementation of a novel sensing deck for a commercially available micro UAV with 13 miniature ToF sensors, a low resolution thermal camera and a method to generate reliable 2D maps of GPS denied areas. The sparse observations obtained from range sensors are combined in a Rao-Blackwellized particle filter to create a reliable map. The ToF sensors are arranged to capture a maximum number of points in the horizontal plane at the flying height despite fluctuations in altitude and orientation. Particle weighting and resampling are only performed in a loop closure or when moving through a known area. Importance weight for the particles are computed by an Iterative Closest Point based method by calculating the error between nearest scan points. The proposed method has been tested in indoor environments and verified as a robust method to generate reliable maps of unknown environments.

Indoor positioning by wireless means is of significant interest in a wide range of applications; Wi-Fi and Bluetooth Low Energy (BLE) are popular candidate wireless technologies for this. We present a novel BLE beacon placement algorithm that complements existing Wi-Fi infrastructure in a hybrid Wi-Fi/BLE positioning system. Composite signal strength measurements thus obtained, are used to derive machine learning models for location estimation. Our results show that positioning estimates at table-level granularity (0.9 m × 1.8 m) in a computer laboratory can be achieved with 97% accuracy. This is an improvement of 20% and 30% compared to Wi-Fi and BLE only techniques respectively.

This paper presents a method for localizing an Unmanned Aerial Vehicle (UAV) using geo-referenced aerial images, in case of GPS failure. When flying in urban environments in low altitudes the inaccurate GPS localization has become a major problem. The proposed method fuses measurements from inertial sensors and visual odometer and perform image registration process using the images captured by the onboard down looking camera. A pre-built local orthomosaic map with geotagged aerial images is used for the registration. The problem of unreliable matches caused by the dynamic objects is addressed in our approach. The experiment results show that it is possible to localize the UAV in urban environments with a low margin of error by using our method.

This paper has addressed the problem of transporting a long object from a given starting location to an end location using leader-follower robot formation in an environment filled with obstacles. Only the leader robot has the authority to make decisions. It possesses the main controller and is capable of navigating through an unknown environment using the data provided by on-board sensors. In the proposed system, a vision-based feedback method is combined with an explicit control method that oversights the virtual trailer between the leader and the follower. The parameters required to maintain a constant distance between the two robots are computed by the leader and transmitted to the follower directly. A fiducial marker is mounted on the front of the follower robot and is captured by a camera attached to the leader robot. Information exchanging methods based only on impedance control can be expensive …

Travelling along routine paths on foot is an exhausting and mundane task. Current personal mobility vehicles (PMVs), which address this issue, are driven manually and are too heavy to be carried by the user when needed. As a solution to this, an autonomous, self-balancing, lightweight, compact and low-cost PMV is developed. The device is capable of simultaneously self-balancing and navigating autonomously in an indoor environment. Moreover, it can be manually controlled using the developed smart mobile phone application. Self-balancing, localization, navigation and obstacle avoidance are achieved in real time using on-board sensors and processing units. Systematic experiments are carried out with the developed PMV in typical indoor environments in order to assess the robustness of the above features. The results show that the robot is able to successfully achieve its navigation tasks both with and …

Personal and medium scale farming has been showing a downward trend throughout the past few decades. Individuals are demotivated to engage in farming activities, due to lack of time, and the higher efficiency in large-scale farming. However, there is a significant health concern arising from the consumption of these products, as they are grown using artificial fertilizers and contain residues of insecticides and pesticides. Automation can motivate personal and medium scale farming, through cutting down the time requirements for farming and by increasing the farming efficiency. Our solution - Scalable Autonomous Agronomical Smartbot (SAASbot) aims at automating farming activities (planting seeds, watering, fertilizing, weed removal) while providing a scalable robotic platform for personal to medium scale farming. In this paper, design and implementation of the SASSBot and testing and validation of the system …

Voice Controlled Robot (VCR) is a mobile robot whose motions can be controlled by the user by giving specific voice commands. The speech recognition software running on a PC is capable of identifying the 5 voice commands ‘Run’,‘Stop’,‘Left’,’Right’and ‘Back’issued by a particular user. After processing the speech, the necessary motion instructions are given to the mobile platform via a RF link. Following is the system overview. The speech recognition software is speaker dependant. The special feature of the application is the ability of the software to train itself for the above voice commands for a particular user. The graphical user interface running along with the software provides a very convenient method for the users to train. It also provides many other facilities in operating the robot.

Accurate attitude estimation in high dynamic motion scenarios like unmanned aerial vehicle navigation or human motion and magnetically disturbed indoor environments requires compensation of accelerometer disturbances as well as magnetometer disturbances. This paper proposes a novel algorithm for accelerometer disturbance rejection by adaptive estimation of accelerometer measurement covariance matrix online. Magnetic disturbances are generally lasting long compared to accelerometer disturbances which are generally short term in indoor navigation environments. Novel two step Kalman filter update method is proposed to separate attitude correction from heading correction. The magnetometer is used only for heading update such that the attitude estimation is not affected by errors in magnetometer measurement. Performances of the proposed algorithms are shown by using real world sensor data.

Pose tracking for outdoor rovers is generally a complex task which is further complicated in conditions where a Global Positioning System (GPS) signal is denied such as in planetary exploration, underground mines and covered areas. In these conditions the rover's pose needs to be calculated purely based on the rover's current environment observations. However, conventional wheel odometry is not reliable on rough terrain where wheels are prone to slip and the wheels do not have a common plane of motion due to suspension systems. This paper proposes a Fast State Variable Extension (Fast-SVE) method in which 2D state variables (x, y, yaw) are extended to the full 3D state (x, y, z, roll, pitch, yaw) to achieve effective real time 3D pose tracking of the rover. A particle filter implementation incorporating the Fast-SVE method is used to track the 3D pose of the rover with roll and pitch values used for weighting. An …

We present a sensor platform to be mounted on the white cane used by the visually impaired. This can estimate the direction of user movement and enable the user to detect obstacles lying on the path in advance. The sensor platform contains an ultrasonic sensor and an IMU (Inertia Measurement Unit). We develop a model to estimate distances to obstacles in the path and their width based on sensor measurements. The model is demonstrated to have an overall accuracy of 84%, with accuracy as high as 90% for obstacles within 50cm in front of the user. Knowledge of obstacle locations and their size in advance, would enable us to guide visually impaired persons to deviate from their path and return to it after the obstacle has been passed.