Arjun Rao

Arjun Rao

I am a final-year undergraduate student at The Chinese University of Hong Kong majoring in Financial Technology and Data Analytics.

My reseach interests broadly encompass optimization for machine learning. I currently work on Decentralized Machine Learning at CUHK's Network Science and Optimization Laboratory with Professor Hoi-To Wai

I previously interned at NASA Jet Propulsion Laboratory's Machine Learning and Instrument Autonomy Group, where I worked on improving robustness of machine learning models on-board large-scale airborne and spaceborne imaging spectrometers advised by Andrew Thorpe and Steffen Mauceri.

                          My CV (08/21)       Google Scholar       
ArjunRao[at]link.cuhk.edu.hk

Research Experience

NASA Jet Propulsion Laboratory
Machine Learning & Instrument Autonomy Group
Summer Research Intern: Caltech SURF@JPL
Jun 2021 – Aug 2021
The Chinese University of Hong Kong
Network Science and Optimization Laboratory
Advisor: Prof. Hoi-To Wai
Winter + Fall Research Intern | Topic: Decentralized Optimization In decentralized consensus optimization, data is partitioned privately among \(\mathbb{N}\) workers, and the goal is to minimize each worker's objective function \(f_{i}(\theta)\) while ensuring that \(\mathbb{N}\) workers agree about the underlying distribution of \(\theta\). The catch: \(\mathbb{N}\) workers are distributed over a sparse graph topology, and can only communicate with immidiate neighbours. Applications include sensor networks and privacy preserving machine learning.
Nov 2020 – May 2021
The Chinese University of Hong Kong
Department of Computer Science and Engineering
Advisor: Prof. Bei Yu
Summer Research Intern | Topic: Adversarial RobustnessAdversarial Examples, which can better be visualized as imperceptible ‘distribution’ shifts in data are a natural consequence of the dimensionality gap between inputs and linear models on which high-dimensional inputs are trained on. They generalize across different architectures, and can be used in a ‘black-box’ fashion to threaten real-world deep learning models. The most common strategy to defend against test-time attacks has been to train models on adversarial data, thus ensuring some ‘robustness‘ against standard attacks.
May 2020 – Sep 2021

Publications


Qi Sun, Arjun Ashok Rao, Xufeng Yao, Bei Yu, Shiyan Hu, “Counteracting Adversarial Attacks in Autonomous Driving”, IEEE/ACM International Conference on Computer-Aided Design (ICCAD), Nov. 2–5, 2020.   [Paper] [Slides]

Autonomous system operation requires real-time processing of measurement data which often contain significant uncertainties and noise. Adversarial attacks have been widely studied to simulate these perturbations in recent years. To counteract these attacks in autonomous systems, a novel defense method is proposed in this paper.

A stereo-regularizer is proposed to guide the model to learn the implicit relationship between the left and right images of the stereo-vision system. Univariate and multivariate functions are adopted to characterize the relationships between the two input images and the object detection model. The regularizer is then relaxed to its upper bound to improve adversarial robustness. Furthermore, the upper bound is approximated by the remainder of its Taylor expansion to improve the local smoothness of the loss surface. The model parameters are trained via adversarial training with the novel regularization term.

Our method exploits basic knowledge from the physical world, i.e., the mutual constraints of the two images in the stereo-based system. As such, outliers can be detected and defended with high accuracy and efficiency. Numerical experiments demonstrate that the proposed method offers superior performance when compared with traditional adversarial training methods in state-of-the-art stereo-based 3D object detection models for autonomous vehicles.

Arjun Ashok Rao, Hoi-To Wai, “An Empirical Study on Compressed Decentralized Stochastic Gradient Algorithms with Overparameterized Models”, Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2021.   [Paper]

This paper considers decentralized optimization with application to machine learning on graphs. The growing size of neural network (NN) models has led prior work surrounding decentralized stochastic gradient algorithms which support communication compression. On the other hand, recent works have demonstrated the favorable convergence and generalization properties of overparameterized NNs. In this work, we present an empirical analysis on the performance of compressed decentralized stochastic gradient (DSG) algorithms with overparameterized NNs. Through simulations on an MPI network environment, we observe that the convergence rates of popular compressed DSG algorithms are robust to the size of NNs. Our findings suggest a gap between theories and practice of the compressed DSG algorithms in the existing literature.

Arjun Ashok Rao, Steffen Mauceri, Andrew K Thorpe, Jake Lee, Brian Bue, Siraput Jongaramrungruang, Riley Duren, “Improving Imaging Spectrometer Methane Plume Detection with Large Eddy Simulations”, American Geophysical Union, Fall Meeting 2021   [Website]

Methane is a highly potent greenhouse gas, and accurately measuring methane emissions is an important step in addressing global environmental change. A large number of human-caused methane emissions originate from small ground structures known as "point-source emitters". These structures typically span a few meters wide and release highly concentrated methane. Current work on detecting these emissions use machine learning models that are trained to identify point-source emissions from aerial imagery. However, these models require large amounts of expensive labelled data, and have shown an inability to adapt to new terrain and environments. To increase the amount of methane emission data available to scientists, we use a collection of synthetic methane measurements. This data is generated with a mathematical model built to realistically replicate the three-dimensional distribution of methane observed in aerial imagery. We train a Convolutional Neural Network on the generated synthetic data, and test our model on realistic datasets containing airborne imagery from diverse terrain and environmental conditions. Training models on a combination of synthetic and real methane data helps reduce false positives on previously unseen scenes.