I graduated from The Chinese University of Hong Kong majoring in Financial Technology and Computer Science. While at CUHK, I worked on Decentralized Machine Learning at CUHK's Network Science and Optimization Laboratory with Professor Hoi-To Wai, and Adversarial Robustness with Professor Bei Yu
I am interested in building robust and interpretable machine learning systems that can reliably be used for novel science applications. More recently, my research involves building statistical methods with geospatial data that are robust and resource-efficient enough to operate in low-SNR, hardware-constrained environments.
The Chinese University of Hong Kong Network Science and Optimization Laboratory
Advisor: Prof. Hoi-To Wai
Winter + Fall Research Intern | Topic: Decentralized Optimization In decentralized consensus optimization,
data is partitioned privately among \(\mathbb{N}\) workers, and the goal is to minimize each worker's objective function
\(f_{i}(\theta)\) while ensuring that \(\mathbb{N}\) workers agree about the underlying distribution of \(\theta\).
The catch: \(\mathbb{N}\) workers are distributed over a sparse graph topology, and can only communicate with immidiate neighbours.
Applications include sensor networks and privacy preserving machine learning.
Nov 2020 – May 2021
The Chinese University of Hong Kong Department of Computer Science and Engineering
Advisor: Prof. Bei Yu
Summer Research Intern | Topic: Adversarial RobustnessAdversarial Examples, which can better be visualized as imperceptible ‘distribution’ shifts in data are a natural consequence
of the dimensionality gap between inputs and linear models on which
high-dimensional inputs are trained on. They generalize across different architectures, and can be used
in a ‘black-box’ fashion to threaten real-world deep learning models. The most common strategy to defend against
test-time attacks has been to train models on adversarial data, thus ensuring some ‘robustness‘ against
standard attacks.
May 2020 – Sep 2021
Publications
Qi Sun,
Arjun Ashok Rao,
Xufeng Yao,
Bei Yu,
Shiyan Hu,
“Counteracting Adversarial Attacks in Autonomous Driving”, IEEE/ACM International Conference on Computer-Aided Design (ICCAD), Nov. 2–5, 2020.
[Paper][Slides]
Autonomous system operation requires real-time processing of measurement data which often contain significant
uncertainties and noise. Adversarial attacks have been widely studied to simulate these perturbations in recent years. To counteract
these attacks in autonomous systems, a novel defense method is proposed in this paper.
A stereo-regularizer is proposed to guide the
model to learn the implicit relationship between the left and right
images of the stereo-vision system. Univariate and multivariate
functions are adopted to characterize the relationships between the
two input images and the object detection model. The regularizer is
then relaxed to its upper bound to improve adversarial robustness.
Furthermore, the upper bound is approximated by the remainder
of its Taylor expansion to improve the local smoothness of the loss
surface. The model parameters are trained via adversarial training with the novel regularization term.
Our method exploits basic
knowledge from the physical world, i.e., the mutual constraints of
the two images in the stereo-based system. As such, outliers can be
detected and defended with high accuracy and efficiency. Numerical
experiments demonstrate that the proposed method offers superior
performance when compared with traditional adversarial training
methods in state-of-the-art stereo-based 3D object detection models
for autonomous vehicles.
Arjun Ashok Rao,
Hoi-To Wai,
“An Empirical Study on Compressed Decentralized Stochastic Gradient Algorithms with Overparameterized Models”, Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2021.
[Paper]
This paper considers decentralized optimization with application to machine learning on graphs. The growing size of neural network (NN) models has led prior work surrounding decentralized stochastic gradient algorithms which support communication compression. On the other hand, recent works have demonstrated the favorable convergence and generalization properties of overparameterized NNs. In this work, we present an empirical analysis on the performance of compressed decentralized stochastic gradient (DSG) algorithms with overparameterized NNs. Through simulations on an MPI network environment, we observe that the convergence rates of popular compressed DSG algorithms are robust to the size of NNs. Our findings suggest a gap between theories and practice of the compressed DSG algorithms in the existing literature.
Methane is a highly potent greenhouse gas, and accurately measuring methane emissions is an important step in addressing global environmental change. A large number of human-caused methane emissions originate from small ground structures known as "point-source emitters". These structures typically span a few meters wide and release highly concentrated methane. Current work on detecting these emissions use machine learning models that are trained to identify point-source emissions from aerial imagery. However, these models require large amounts of expensive labelled data, and have shown an inability to adapt to new terrain and environments. To increase the amount of methane emission data available to scientists, we use a collection of synthetic methane measurements. This data is generated with a mathematical model built to realistically replicate the three-dimensional distribution of methane observed in aerial imagery. We train a Convolutional Neural Network on the generated synthetic data, and test our model on realistic datasets containing airborne imagery from diverse terrain and environmental conditions. Training models on a combination of synthetic and real methane data helps reduce false positives on previously unseen scenes.
Despite methane’s important role as a greenhouse gas, the contribution of individual sources to rising methane concentrations in Earth’s atmosphere is poorly understood. This is in part due to the lack of frequent measurements on a global scale, required to accurately quantify fugitive methane sources. Future missions such as Earth Surface Mineral Dust Source Investigation (EMIT), Surface Biology and Geology (SBG), and Carbon Mapper promise to provide global, spatially resolved spectroscopy observations that will allow us to map methane sources. However, the detection and attribution of individual methane sources is challenged by retrieval artifacts and noise in retrieved methane concentrations. Additionally, manual methane plume detection is not scalable to global space-borne observations due to the sheer volume of data generated. A robust automated system to detect methane plumes is needed.
We evaluated the performance and sensitivity of several methane plume detection methods on 30m to 60m hyperspectral imagery, downsampled from airborne campaigns with AVIRIS-NG. To aid the training of the plume detection models, we explored supplementing downsampled airborne imagery with Large Eddy Simulations (LES) of methane plumes. We compared baseline methods such as thresholding and random forest classifiers, as well as state-of-the-art deep learning methods such as convolutional neural networks (CNNs) for classification and conditional adversarial networks (pix2pix) for plume segmentation.
