How does your brain know that a stop sign is a stop sign? Computer vision architects attempt to answer this question for many objects, from birds in the wild to mac and cheese dishes. The problem is complex, since a machine must be taught so many aspects of sensory processing that are second nature to humans. We can still recognize a stop sign that has graffiti or stickers on it. How can a computer be taught to do the same?
As technology becomes essential to so many functions of daily life, this question has become more than a matter of utility or convenience. It’s also a critical security issue — one that applies to many forms of data input, from images to audio to text.
Research from Bo Li, an assistant professor of computer science at the University of Illinois Urbana-Champaign, highlights both vulnerabilities and solutions. [Editors’ note: Li joined AWS as a Visiting Academic earlier this year.] In 2017, Li and colleagues showed that even slight alterations to common road signs were usually enough to throw off neural networks tasked with recognizing them — a hurdle for self-driving auto systems. The study proposed a general algorithm designed to uncover such vulnerabilities.
Ongoing work at Li’s Secure Learning Lab aims to “make machine learning algorithms more robust, private, efficient, and interpretable,” with support from a 2020 Amazon Research Award. In 2019, a separate Amazon Research Award for Li laid the foundation for work she is doing today to evaluate the robustness of machine learning algorithms, particularly with respect to privacy.
These types of attacks are very stealthy, a human sitting in front of the computer trying to figure out which image is attacked … cannot do it. You can only train a model to do it.
“These types of attacks are very stealthy,” Li said of the slight alterations to input that can confuse an algorithm. “A human sitting in front of the computer trying to figure out which image is attacked and which one is not cannot do it. You can only train a model to do it.”
The 2020 Amazon Research Award funding so far has produced four publications from Li and colleagues. One, which was accepted by the IEEE Symposium on Security and Privacy being held in May, focuses on graph-structured data. Li and co-authors pinpointed “edge privacy” concerns with graph-structured data, which underlies many services, including social networks.
The paper, “LinkTeller: Recovering Private Edges from Graph Neural Networks via Influence Analysis,” posed a scenario where a service API trained with graph data can be co-opted to access information that should remain private.
The other papers are oriented toward defense and protections. One, which was presented at the Neural Information Processing Systems (NeurIPS) 2021 conference, dealt with the challenge of training a scalable machine learning algorithm that generates usable private data.
“This problem is very important. But so far, there’s no good method that can achieve this for high-dimensional data,” Li said. High-dimensional data has a multitude of features and fewer observations: Common examples include genomics and health records, where large numbers of attributes may be associated with each person.
Li said the NeurIPS paper proposes an algorithm that generates scalable, high-dimensional, differentially private data — meaning there is no way to infer (and thus expose) sensitive information that was used to generate a result. The strategy involves masking private data by hiding it behind a group of “teacher discriminators,” as opposed to relying on one training example for the student algorithm.
The paper “TSS: Transformation-Specific Smoothing for Robustness Certification,” accepted at the 2021 ACM Conference on Computer and Communications Security (CCS), offers a way to certify a machine learning model’s robustness against arbitrary attacks by labeling resolvable disruptions, or transformations, of data. In the stop sign example, the idea is to certify that even if an image of a sign has some unexpected alternation, the algorithm can still identify it with a high level of confidence.
Bo Li’s CVPR 2021Workshop on Autonomous Driving keynote
As an undergraduate in computer science at Shanghai Jiao Tong University, Li focused on pure system security, such as cryptography. But as she embarked on her PhD and postdoc at the University of California Berkeley in 2011, interest in artificial intelligence was growing, and she was drawn to related questions.
Li said she recognized some potential vulnerabilities around AI and private data. She began to explore those by conducting experimental attacks, like the one involving autonomous cars and street signs in 2017, and theoretical analysis to uncover the fundamental principles of AI trustworthiness.
“You can see a lot of news reports about my work on these attacks. Somehow people are more excited about attacks,” she said with a laugh. But she quickly began to do more work on the preventive side as well, working on ways to safeguard and certify systems.
Earlier research produced projects such as Certifying Robust Policies (CROP) for reinforcement learning — also funded by the 2020 Amazon Research Award and — which systematically evaluates different reinforcement learning algorithms based on certification criteria, and Adversarial General Language Understanding Evaluation (GLUE), a benchmarking system that tests and analyzes the vulnerabilities of natural language understanding systems. CROP was recently accepted to the 2022 International Conference on Learning Representations, happening in April.
Li sees these research and open-source efforts as important not just to maintaining security in specific situations, but also to the broader challenge of domain generalization: The idea that an algorithm is flexible and powerful enough to adapt to different settings and uses. For example, will an autonomous car trained to drive in a city know what to do when it gets to a rural area unlike anything it has seen before?
“Domain generalization is an everlasting topic in machine learning,” Li said. “We are trying to tackle this problem from a robustness perspective.”
Beyond the funding and computational resources of the Amazon Research Award, Li also has benefited from talking with Amazon researchers about real-world problems. Her lab’s methodologies can be applied to vision, text, audio, and video. She is aiming for impact, whether it involves integration with AWS tools or inspiration for other researchers.
“We hope researchers will try our methods on different domains,” she says.
