Enhancing Cybersecurity in Edge AI through Model Distillation and Quantization: A Robust and Efficient Approach
DOI:
https://doi.org/10.56127/ijst.v1i3.1957Abstract
The rapid proliferation of Edge AI has introduced significant cybersecurity challenges, including adversarial attacks, model theft, and data privacy concerns. Traditional deep learning models deployed on edge devices often suffer from high computational complexity and memory requirements, making them vulnerable to exploitation. This paper explores the integration of model distillation and quantization techniques to enhance the security and efficiency of Edge AI systems. Model distillation reduces model complexity by transferring knowledge from a large, cumbersome model (teacher) to a compact, efficient one (student), thereby improving resilience against adversarial manipulations.
Quantization further optimizes the student model by reducing bit precision, minimizing attack surfaces while maintaining performance.
We present a comprehensive analysis of how these techniques mitigate cybersecurity threats such as model inversion, membership inference, and evasion attacks. Additionally, we evaluate trade-offs between model accuracy, latency, and robustness in resource-constrained edge environments. Experimental results on benchmark datasets demonstrate that distilled and quantized models achieve comparable accuracy to their full-precision counterparts while significantly reducing vulnerability to cyber threats. Our findings highlight the potential of distillation and quantization as key enablers for secure, lightweight, and high-performance Edge AI deployments.
References
1. Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016). Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 308–318.
2. Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006). Model compression. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 535–541.
3. Costan, V., & Devadas, S. (2016). Intel SGX explained. IACR Cryptology ePrint Archive, 2016, 86.
4. Fredrikson, M., Jha, S., & Ristenpart, T. (2015). Model inversion attacks that exploit confidence information and basic countermeasures. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 1322–1333.
5. Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. International Conference on Learning Representations (ICLR).
6. Guo, Y., Yao, A., & Chen, Y. (2018). Network decoupling: From regular to depthwise separable convolutions. Proceedings of the British Machine Vision Conference (BMVC), 1–12.
7. Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. NeurIPS Deep Learning and Representation Learning Workshop.
8. Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., ... & Adam, H. (2018). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2704–2713.
9. Kurakin, A., Goodfellow, I., & Bengio, S. (2017). Adversarial examples in the physical world. Artificial Intelligence Safety and Security, 99–112.
10. Lin, J., Gan, C., & Han, S. (2019). Defensive quantization: When efficiency meets robustness. International Conference on Learning Representations (ICLR).
11. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations (ICLR).
12. Papernot, N., McDaniel, P., & Goodfellow, I. (2016). Transferability in machine learning: From phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277.
13. Papernot, N., McDaniel, P., Wu, X., Jha, S., & Swami, A. (2016). Distillation as a defense to adversarial perturbations against deep neural networks. 2016 IEEE Symposium on Security and Privacy (SP), 582–597.
14. Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017). Membership inference attacks against machine learning models. 2017 IEEE Symposium on Security and Privacy (SP), 3–18.
15. Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016). Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5), 637–646.
16. Satyanarayanan, M. (2017). The emergence of edge computing. Computer, 50(1), 30–39.
17. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2014). Intriguing properties of neural networks. International Conference on Learning Representations (ICLR).
18. Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016). Stealing machine learning models via prediction APIs. 25th USENIX Security Symposium (USENIX Security 16), 601–618.
19. Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys (CSUR), 53(3), 1–34.
20. Rakin, A. S., He, Z., & Fan, D. (2019). Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 588–597.
21. Liu, B., Wu, Y., & Yang, Y. (2018). Adversarial examples: Attacks and defenses for deep learning. IEEE Transactions on Neural Networks and Learning Systems, 30(9), 2805–2824.
22. Carlini, N., & Wagner, D. (2017). Towards evaluating the robustness of neural networks. 2017 IEEE Symposium on Security and Privacy (SP), 39–57.
23. Han, S., Mao, H., & Dally, W. J. (2016). Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. International Conference on Learning Representations (ICLR).
24. Iandola, F. N., Moskewicz, M. W., Ashraf, K., Han, S., Dally, W. J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size. arXiv preprint arXiv:1602.07360.
25. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... & Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
26. Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations (ICLR).
27. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 770–778.
28. Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2019). mixup: Beyond empirical risk minimization. International Conference on Learning Representations (ICLR).
29. Xu, W., Evans, D., & Qi, Y. (2018). Feature squeezing: Detecting adversarial examples in deep neural networks. Network and Distributed System Security Symposium (NDSS).
30. Yao, J., Zhang, S., Yao, Y., Wang, F., Ma, J., Zhang, J., ... & Yang, H. (2022). Edge-cloud polarization and collaboration: A comprehensive survey for ai. IEEE Transactions on Knowledge and Data Engineering, 35(7), 6866-6886.
