Research projects
Web Applications #2 (2024)
Login Bypasser
Programming a Mozilla Firefox extension to access social media websites without logging in, by removing login popups, bypassing other banners, etc.
Software: JavaScript.
Personal ML / AI Series #2 (2024)
Large Language Models from Scratch
Large language models (LLM) are machine-learning tools for text generation. Their key advantage, beside their performance leap after reaching a certain model size, lies in their attention mechanism, proposed in the paper entitled Attention is All you Need.
This work builds LLMs from scratch. That is, it implements the full neural network, including encoding, embedding, attention heads, and multilayer perceptrons.
Apart from the educational purpose of exploring the underlying components of such a famous AI system, the package allows for building and training an LLM, of any size, on any training text, such as a book, a webpage, etc. It can also fine-tune a GPT2 model by influencing it with any text you provide. Any of those trained models can then be used to generate text.
Software: Python, Pytorch.
Reproduction of Fundamental Computer Systems #1 (2024)
Cellular Automata
The code has 1-D and 2-D examples, such as the Game of Life proposed by Conway and Rule 110 proposed by Wolfram.
Software: Python.
Web Applications #1 (2023)
Youtube AdBlock
This work proposes a Mozilla Firefox extension (aka add-on) accelerating and skipping YouTube ads in less than two seconds.
Software: JavaScript.
Personal ML / AI Series #1 (2022)
Convolutional Neural Network for Digit Recognition
This work proposes an algorithm which detects a digit (from 0 to 9) from an image. Click on the video to test the algorithm in real time.
Image recognition is performed through a convolutional neural network (shown below). The first round reduces the dimension of the input image while keeping its meaningful information (Conv / ReLu / MaxPool / Conv / MaxPool). The second round flattens the 2D tensor and passes it through a classical neural network (multilayer perceptron) to produce 10 values (one associated to each digit). The third round performs a logarithmic softmax on the 10 values to reflect the probability that each digit is rendered on the input image. The last round outputs the digit with the highest of the 10 computed probabilities (maximum likelihood estimator).
It was trained using the MNIST dataset (60k images) across 50 epochs with a batch size of 256, achieving 95% of accuracy on the test dataset.
Software: Python, Pytorch, Torchvision.
Graduate Research at UT Austin (2022): Army Research Laboratory
Multi-Agent Motion Planning with Military-based Maps
This work aims at building motion-planning algorithms in a military environment composed of different maps: obstacles, elevation, valence (favorable areas). The algorithms are based on the Dijkstra algorithm, which finds the least-cost path in a graph with non-uniform cost. The main objective is thus to create a mesh (set of nodes) and associate a cost between the adjacent nodes that are adequate for this application. Multiple cost functions have been developed, for example the 3D distance between the initial and target locations, the energy required based on the agent’s dynamics, the valence representing some favorable zones, or a combination of the three.
Multiple algorithms have also been developed to mitigate the computation time, at the expense of the accuracy. For example, we can have a first loop computing the path over a coarse mesh, and then a second loop computing a precise path for the first section that the agent is taking (path up to the first checkpoint computed by the first rough loop).
Software: Python.
Project in Multi-Agent Systems at UT Austin (Fall 2021)
Covariance Steering Games with Squared Wasserstein Distance Cost
As part of the project in Modelling of Multi-Agent Systems, we designed two algorithms steering a stochastic game: https://sites.google.com/view/ut-ase389-stoch-games.
In this project, we consider discrete-time linear covariance steering dynamic games where the objective of each player is chosen as the sum of control cost and the squared Wasserstein distance between the terminal distribution and their respective desired distribution. To solve this problem, we first turn it into a finite dimensional optimization problem for each player by using disturbance history feedback policy, and used iterative best response by solving the problem associated to each player iteratively by fixing the other player’s policy. In the second approach, we associate the terminal Wasserstein distance with the expectation of a convex quadratic function of the state by linearizing around the terminal mean and covariance from the previous solution. This method allows us to turn the problem into an LQG game whose solution can be found by solving the associated Ricatti equation iteratively. Finally, we evaluate these approaches experimentally in terms of convergence and the final solution.
Software: Python.
Project in Robot Learning at UT Austin (Fall 2021)
Reinforcement Learning for Cooperative Manipulation
This project aims at building a reinforcement learning algorithm using Deep Deterministic Policy Gradient (DDPG) to achieve diverse manipulation tasks. We also added a hindsight experience replay (HER) buffer to the training phase, which results in a faster convergence to a successful policy. We demonstrate our approach in multiple robotic manipulation environments such as OpenAI’s Fetch pick-and-place task and Robosuite’s two arm lifting task.
Software: Python, Pytorch.
Graduate Research at UT Austin (2021-2022): 2nd paper
Vector Field-based Collision Avoidance for Moving Obstacles with Time-Varying Shape
This work presents an algorithm for local motion planning in environments populated by moving elliptical obstacles whose velocity, shape and size may change with time. We base the algorithm on a collision avoidance vector field (CAVF) that aims to steer an agent to a desired final state whose motion is described by a double integrator kinematic model. The proposed algorithm can handle multiple obstacles. In addition, it is applicable in bounded environments for more realistic applications (e.g., motion planning inside a building). We also incorporate a method to deal with agents whose control input is limited so that they safely navigate around the obstacles. To showcase our approach, extensive simulations results are presented in 2D and 3D scenarios.
The paper resulting from this research has been published at the 2022 Modeling, Estimation and Control Conference.
Software: Matlab.
Graduate Research at UT Austin (2020-2021): 1st paper
Greedy Decentralized Auction-based Task Allocation for Multi-Agent Systems
During my master at the University of Texas at Austin, I was doing graduate research in Efstathios Bakolas’ research group which belongs to the Aerospace department. My area focused on the design of algorithms allocating tasks to multi-agent autonomous systems using auctions and machine-learning tools. This task-agent allocation problem requires advanced algorithms that drastically outperform a brute force approach growing exponentially with the number of agents and tasks.
A typical application that will benefit from this research can be illustrated with a fire disaster. The goal is to assign a high number (typically hundreds) of aerial firefighter vehicles (such as drones) to a huge number (typically thousands) of fire points in the area (city, forest,…) in a way that best reduces the amount of human victims and material damage. There exist plenty of other applications such as ride sharing via human-driven cars (Uber, Lyft,…) and naturally self-driving cars in the future (Waymo, Tesla, Cruise,…), warehouse robots reacting in real-time to new tasks (Amazon, Boston Dynamics,…).
We propose a decentralized auction-based algorithm for the solution of dynamic task allocation problems for spatially distributed multi-agent systems. In our approach, each member of the multi-agent team is assigned to at most one task from a set of spatially distributed tasks, while several agents can be allocated to the same task. The task assignment is dynamic since it is updated at discrete time stages (iterations) to account for the current states of the agents as the latter move towards the tasks assigned to them at the previous stage. Our proposed methods can find applications in problems of resource allocation by intelligent machines such as the delivery of packages by a fleet of unmanned or semi-autonomous aerial vehicles. In our approach, the task allocation accounts for both the cost incurred by the agents for the completion of their assigned tasks (e.g., energy or fuel consumption) and the rewards earned for their completion (which may reflect, for instance, the agents’ satisfaction). We propose a Greedy Coalition Auction Algorithm (GCAA) in which the agents possess bid vectors representing their best evaluations of the task utilities. The agents propose bids, deduce an allocation based on their bid vectors and update them after each iteration.
The paper resulting from this research has been published at the 2021 Modeling, Estimation and Control Conference.
Software: Matlab.
I also focused on reading the literature related to motion planning and task allocation for multi-agent autonomous systems using reinforcement learning and deep learning (e.g. deep Q-learning). Such machine-learning tools greatly help reduce the computational cost associated to large state spaces involving multiple robots. Extensive research in this field has been performed for single-agent systems, but some state-of-the-art algorithms need to be extended to large fleets of robots and improve their interaction with the environment (humans or objects).
Project in Machine Learning at UCLouvain (Fall 2019)
Prediction of air-quality in Beijing
This report aims in predicting the concentration of PM2.5 in the air of Beijing using regression models on a dataset composed of meteorological and weather data. This project includes features processing, features selection (e.g., Mutual Information), features extractions (e.g., PCA), error estimation (e.g., Bootstrap 632) and models implementation (Neural Network, KNN, Lasso, Regression trees).
Software: Python, Pytorch, Sklearn, Seaborn.
Space Systems Laboratory at MIT (Summer 2019)
Building of Astrobee
From July to August 2019, an exchange program between UCLouvain and MIT gave me the opportunity to do a research work in the Space Systems Laboratory at the Massachusetts Institute of Technology. We worked in team to give MIT its own copy of Astrobee. Designed by NASA, these are robotic teammates to work alongside astronauts on the International Space Station as they help advance research. They help astronauts reduce time they spend on routine duties, leaving them to focus more on the things that only humans can do.
As a Master’s student in electromechanical engineering, my work consisted to build the avionics of Astrobee: making an avionics diagram, ordering the electronic components, soldering the PCBs and testing the development boards and sensors.
This research work gave me a strong interest for space systems, and more generally the electronics of diverse robots.
Project in Mechatronics (2018 – 2019)
Design of a Wheeled Driving Robot
This project was achieved by 6 students in the first year of the Master in electromechanical engineering (option in mechatronics) at UCLouvain.
It aimed to build a driving robot moving on a predefined map and applying a few actions like grasping and moving tokens. All the rules were given by Eurobot, a French contest gathering the best European robots. The 2019 topic was “Atom factory” and consisted to score points by moving these atoms (hockey pucks) on a weighing scale, an accelerator or the robot starting area.
After passing the robot approval, the robot competed at Robotix’s, the Belgian contest that determines the qualified robots for Eurobot. Our team has been qualified for the European contest with a 2nd Belgian place (and 1st place at UCLouvain).
Regarding the technical implementations, I have analyzed the robot dynamics and control on the map and sized all the parts composing the robot. I also computed and implemented some algorithms in accordance with the rules imposed by the contest. I worked on the low-level control and actuation of the wheels in order to optimize the relation between the voltage applied on the electric motors and the speed of the wheels. The path planning was implemented through the potential field method, for which the robot is repelled by obstacles and attracted by the expected position. The localization was achieved by wheels odometry as well as a LIDAR with high sampling frequency (RPLIDAR A2). Then I implemented a Kalman filter, allowing the LIDAR data to update the localization predicted by odometry. These algorithms were coded inside the Linux kernel of a Raspberry Pi, which was connected to an FPGA (Verilog) development platform (DE0-NANO) allowing the signal transmission to the different sensors and actuators. In the end, I designed an Android application creating a communication channel between a cell phone and the robot through Wi-Fi (for robot control, sensors actuation and data reception).
Regarding the mechanics and electronics, I modeled the robot on SolidWorks (CAD), designed the electropneumatic architecture and machined/printed the mechanical parts. I designed the electrical schematics and synthesized the PCBs (Eagle).
Software: C++, SystemVerilog.
Master’s Thesis on IoT Smart Sensors (2019-2020)
Design of a smart sensor for bird detection
I worked on my Master’s thesis entitled “Design of an ultra-low-power energy-harvesting audio sensor for ecosystem monitoring” under the supervision of Prof. David Bol for my Master in electromechanical engineering at UCLouvain.
The Internet of Things (IoT) is predicted to lead to the deployment of a very large number (possibly trillions) of connected smart sensors for various applications. Such a massive deployment of smart sensors is not environmentally sustainable if the smart sensors are replaced every two years because of the pressure they put on natural resources and the ecotoxicity of the e-waste they generate.
Furthermore, the rising climate change due to ecosystem destruction involves monitoring forests in order to analyze and preserve the ecosystem. Such monitoring is typically achieved manually via a person who samples data less than once a day, which fails to provide strong results and asks for human presence during data acquisition.
To solve these issues, the focus of this Master’s thesis is on the development of an autonomous and efficient audio smart sensor continuously analyzing the forest ecosystem via bird inventory. Such typical sensors can be broken down in 5 modules:
- Inside the energy storage element (battery, supercapacitor, …) is located the most important part of material resources in the sensor. These resources need to be composed of chemical elements which do not wear out. Batteries nowadays suffer from a short (10 years) lifetime and have toxic components such as lead or lithium. Supercapacitors are particularly well suited for low-power applications and high lifetime with few critical resources (e.g. electrostatic double-layer capacitors).
- Power management involves the distribution of the power through the entire circuit, it is an essential element for the minimization of the whole power consumption. A power management unit from e-peas (a Belgian company specialized in low-power chips) assures the transfer of power between the storage element, solar cells and the circuit subsystems. This chip has configurable Linear-Dropout Regulators and Maximum Power Point Tracking to optimize the solar panels efficiency. This work package requires a multivariate optimization of the parameters since it is located at the heart of the sensor design, for which the value of the overall voltage supply needs to be carefully selected for the best behaviour of the different chips.
- Sensing is the step converting sound into an electrical signal, which is then filtered and amplified through an analog front-end (AFE). In this work, after a thorough comparison of state-of-the-art microphones for IoT applications, an electret condenser microphone is used for its small size, low noise and power consumption. The amplification achieved in the AFE highlights a trade-off between the power consumption and the total noise (intrinsic to the microphone, the resistors and the operational amplifier in the AFE). This trade-off leads to a precise op amp selection based on its noise and power capabilities.
- Data processing is done digitally in a low-power microcontroller (STM32). Machine-learning algorithms are implemented to classify the data. They integrate basic models for the sensor non-idealities in order to grow back the inference accuracy decreased by the use of machine-learning algorithms trained in ideal conditions (offline with full resources).
- Wireless communication first allows the transmission of the locally processed input sound towards edge gateways. On the other hand, the sensor gets reconfiguration capabilities to keep up with application, security and communication protocol (LPWAN: LoRa) updates.
This development is the first step for the deployment of a wireless audio sensor network capable to optimize its energy consumption. The power consumption of such sensors leads to a trade-off between the density of the sensor deployment (limited by the frequency of battery replacement by an operator) and the complexity of ecosystem parameters to monitor. Today, ultra-low-power processors for IoT communications, artificial intelligence and data mining open new possibilities. Such processors preserve the collected information and compute feature vectors on each sensor to allow a low-power wireless transmission towards an edge-computing gateway (servers processing data from the sensor and sending the relevant data back through the cloud, reducing bandwidth needs).
To fulfill the energy constraints implied by its total autonomy, this sensor harvests energy from the environment through miniaturized photovoltaic cells sized according to the sun illuminance throughout days and seasons, using an environmentally-friendly and non-toxic supercapacitor to store energy. With a 15+ year lifetime, this fully autonomous device operates at an optimized 2.5 V supply voltage reaching 22.1 mW of average power harvesting/consumption. An electret condenser microphone collects a signal as low as 16 dBSPL (compared to a 14.22 dBSPL input-referred noise), which is then amplified in the full frequency range of bird emission (20 Hz – 20 kHz) by a low-noise and low-power analog front-end. This signal is further processed in an ultra-low-power chip embedding a microcontroller, alternating between run and sleep modes with a one-third duty cycle, and a transceiver optimized for IoT applications with LoRaWAN networks.
The microcontroller detects sounds when birds are active (typically during the day for more than 12 hours) and ensures the radio-frequency communication at night depending on the supercapacitor voltage that is carefully monitored in real time. It sends information about the bird species encountered during the day, as well as their apparition frequency. In case of firmware update, this device receives the associated fragments when its energy is sufficient and it automatically changes the firmware with energy-optimized software requiring only 10.6 J for the whole update.
By computing the weighted average frequency of the received sounds, the smart sensor is able to discriminate between four common birds in Belgium: the pigeon, blackbird, great tit and blue tit. For each species, several songs have been analyzed and used to train a k-nearest neighbors (KNN) classifier working in the real-time embedded system. Its precision, defined as the likelihood to find the correct species, reaches 94% for songs coming from the previously learned database. For newly analyzed sounds, the detection algorithm performs likewise. More complex machine-learning algorithms could finally be further designed to discriminate between more species.
Software: C.
Master’s thesis text, Master’s thesis presentation & GitHub code
