From speech recognition to facial identification, and healthcare innovation to autonomous vehicles, deep neural networks (DNNs) are certainly enjoying the spotlight and helping AI set new records every day. DNNs fall within the scope of machine learning (ML), and provide the critical computational framework needed for AI to solve data-driven problems. Just as diverse regions of the human brain process miscellaneous information, DNNs employ artificial nodes organized in layers to analyze specific features within a given dataset.
While a robust DNN can yield accurate results, the training process is resource-intensive and time-consuming. Presently, billions of dollars are invested in training DNNs to cater to an ever-expanding range of applications. In this respect, determining the difficulty in identifying and extracting all the parameters of such neural networks is like solving an intricate puzzle – time-consuming and often unfeasible, especially for large networks. Cryptographers and mathematicians began working on this problem in the early 1990s, and the latest signpost was an algorithm developed and presented by Dr. Nicholas Carlini, Dr. Matthew Jagielski, and Dr. Ilya Mironov at the International Cryptology (CRYPTO 2020) Conference.
In collaboration with the highly esteemed Prof. Adi Shamir, a distinguished Cryptographer and Inventor at the Weizmann Institute of Science and a valued member of the advisory board of the Cryptography Research Center (CRC) at the Technology Innovation Institute (TII), a dedicated team of TII researchers, including Dr. Isaac A. Canales-Martínez, Senior Cryptographer; Dr. Anna Hambitzer, Senior Data Scientist; Prof. Francisco Rodríguez Henríquez, Technical Director; Dr. Nitin Satpute, Senior Machine Learning Engineer; and Jorge Chavez-Saab, Senior Cryptographer has made significant strides in advancing the field. Their collaborative efforts have led to the development of novel techniques that successfully unravel the complexities of DNN analysis, resulting in substantial reduction in the time required for parameter extraction.
They demonstrated the efficacy of their approach on a practical image-processing network, and the outcome was impressive. In just 30 minutes, they successfully recovered millions of pieces of vital information using a regular computer. This proved to be an absolute game-changer compared to previous methods, which could take days, months, or even years, often making it almost impossible to achieve a verifiable outcome - depending on the size of the targeted DNNs.
This remarkable breakthrough contributes to advancing the field of DNN security while underscoring the importance of continuous research and innovation in addressing emerging challenges. In showcasing an efficient approach for parameter extraction, the CRC team’s outstanding results provide valuable insights into potential vulnerabilities, enabling researchers and practitioners to fortify AI systems against adversarial attacks. This newfound understanding becomes a cornerstone for constructing robust defenses and ensuring the resilience of AI components that are critical in diverse applications.
To learn more about the team’s findings, click here: https://eprint.iacr.org/2023/1526