Arjun Rao

Arjun Rao

I am a final-year undergraduate student at The Chinese University of Hong Kong majoring in Financial Technology and Data Analytics.

My reseach interests broadly encompass optimization for machine learning. I currently work on Decentralized Machine Learning at CUHK's Network Science and Optimization Laboratory with Professor Hoi-To Wai

I previously interned at NASA Jet Propulsion Laboratory's Machine Learning and Instrument Autonomy Group, where I worked on improving robustness of machine learning models on-board large-scale airborne and spaceborne imaging spectrometers advised by Andrew Thorpe and Steffen Mauceri.

                          My CV (08/21)       Google Scholar       
ArjunRao[at]link.cuhk.edu.hk

Research Experience

NASA Jet Propulsion Laboratory
Machine Learning & Instrument Autonomy Group
Summer Research Intern: Caltech SURF@JPL
Jun 2021 – Aug 2021
The Chinese University of Hong Kong
Network Science and Optimization Laboratory
Advisor: Prof. Hoi-To Wai
Winter + Fall Research Intern | Topic: Decentralized Optimization In decentralized consensus optimization, data is partitioned privately among \(\mathbb{N}\) workers, and the goal is to minimize each worker's objective function \(f_{i}(\theta)\) while ensuring that \(\mathbb{N}\) workers agree about the underlying distribution of \(\theta\). The catch: \(\mathbb{N}\) workers are distributed over a sparse graph topology, and can only communicate with immidiate neighbours. Applications include sensor networks and privacy preserving machine learning.
Nov 2020 – May 2021
The Chinese University of Hong Kong
Department of Computer Science and Engineering
Advisor: Prof. Bei Yu
Summer Research Intern | Topic: Adversarial RobustnessAdversarial Examples, which can better be visualized as imperceptible ‘distribution’ shifts in data are a natural consequence of the dimensionality gap between inputs and linear models on which high-dimensional inputs are trained on. They generalize across different architectures, and can be used in a ‘black-box’ fashion to threaten real-world deep learning models. The most common strategy to defend against test-time attacks has been to train models on adversarial data, thus ensuring some ‘robustness‘ against standard attacks.
May 2020 – Sep 2021

Publications


Qi Sun, Arjun Ashok Rao, Xufeng Yao, Bei Yu, Shiyan Hu, “Counteracting Adversarial Attacks in Autonomous Driving”, IEEE/ACM International Conference on Computer-Aided Design (ICCAD), Nov. 2–5, 2020.   [Paper] [Slides]

Autonomous system operation requires real-time processing of measurement data which often contain significant uncertainties and noise. Adversarial attacks have been widely studied to simulate these perturbations in recent years. To counteract these attacks in autonomous systems, a novel defense method is proposed in this paper.

A stereo-regularizer is proposed to guide the model to learn the implicit relationship between the left and right images of the stereo-vision system. Univariate and multivariate functions are adopted to characterize the relationships between the two input images and the object detection model. The regularizer is then relaxed to its upper bound to improve adversarial robustness. Furthermore, the upper bound is approximated by the remainder of its Taylor expansion to improve the local smoothness of the loss surface. The model parameters are trained via adversarial training with the novel regularization term.

Our method exploits basic knowledge from the physical world, i.e., the mutual constraints of the two images in the stereo-based system. As such, outliers can be detected and defended with high accuracy and efficiency. Numerical experiments demonstrate that the proposed method offers superior performance when compared with traditional adversarial training methods in state-of-the-art stereo-based 3D object detection models for autonomous vehicles.

Arjun Ashok Rao, Hoi-To Wai, “An Empirical Study on Compressed Decentralized Stochastic Gradient Algorithms with Overparameterized Models”, Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2021.   [Paper]

This paper considers decentralized optimization with application to machine learning on graphs. The growing size of neural network (NN) models has led prior work surrounding decentralized stochastic gradient algorithms which support communication compression. On the other hand, recent works have demonstrated the favorable convergence and generalization properties of overparameterized NNs. In this work, we present an empirical analysis on the performance of compressed decentralized stochastic gradient (DSG) algorithms with overparameterized NNs. Through simulations on an MPI network environment, we observe that the convergence rates of popular compressed DSG algorithms are robust to the size of NNs. Our findings suggest a gap between theories and practice of the compressed DSG algorithms in the existing literature.