Toward Interpretable Deep Learning Via Fuzzy Logic

Lixin Fan and Fei-Yue Wang  

Tutorial Abstract

Based on our recent findings [1][2], which disclosed an intriguing connection between fuzzy logic and deep learning, this course will introduce the main concepts of fuzzy logic and its applicability to deep learning and pattern recognition.  A historical review of related fuzzy logic topics such as fuzzy sets, many-valued logics as well as fuzzy neural networks will be given during the course, with the aim to provide necessary background knowledge for understanding our recent findings [1][2].
[1] Revisit Fuzzy Neural Network: Demystifying Batch Normalization and ReLU with Generalized Hamming Network, NIPS 2017. (https://arxiv.org/abs/1710.10328) 
[2] Revisit fuzzy neural network: bridging the gap between fuzzy logic and deep learning, (http://vixra.org/pdf/1711.0265v2.pdf)  

Tutorial Outline

Ø  Introduction (30 mins): 
    · Risks of deep learning
    · The need of transparent and interpretable deep learning
    · What fuzzy logic can offer
    · Fuzzy logic, fuzzy neural network, fuzzy pattern recognition and computer vision
Ø  Historical review of fuzzy logic (45 mins): 
    · Principle of bivalence, Many-valued logic, Fuzz sets, level cuts, operations, fuzzy numbers & intervals, fuzzy relations, approximate reasoning, methods for constructing fuzzy sets, foundations and extension, applications, fuzzy neural networks 
Ø  Bridge the gap: fuzzy logic and deep learning (45 mins): 
    · Fuzzy XOR connectives, generalized hamming distance, connection with binary features, fuzzy logic interpretation of neural computing, Re-interpretation of ReLU and batch normalization, applications,  deep epitomes, visualization of generalized hamming networks,  width vs depth, universal approximation theorem
Ø  More related topics (to be decided, 45 mins). 

Speaker Interlocution

Lixin Fan (Chinese: 范力欣), Principal Scientist, Nokia Technologies, lixin.fan@nokia.com, Dr Lixin Fan is a principal scientist at Nokia Technologies. His research areas of interests include Machine learning & deep learning, Computer vision & pattern recognition, Image and video processing, 3D big data processing, data visualization & rendering, Augmented and virtual reality, Mobile ubiquitous and pervasive computing and Intelligent human-computer interface. Dr Fan is the (co-)author of more than 50 international journal & conference publications. He also (co-)invented dozens of granted and pending patents filed in US, Europe and China. Before joining Nokia in 2004, Dr Fan was affiliated with Xerox Research Center Europe and his research work included the well-recognized Bag of Keypoints method for image categorization. 

Fei-Yue Wang (Chinese: 王飞跃, feiyue.wang@ia.ac.cn ) is a systems engineer in the State Key Laboratory for Management and Control for Complex Systems of the Chinese Academy of Sciences and editor-in-chief of the IEEE Transactions on Computational Social Systems and the IEEE/CAA Journal of Automatica Sinica. Previously he was a Professor of Systems and Industrial Engineering at the University of Arizona, president of the IEEE Intelligent Transportation Systems Society, and editor-in-chief of IEEE Intelligent Systems. Wang was elected as a Fellow of the IEEE in 2004 "for contributions to intelligent control systems and applications to complex systems".  He also became a fellow of the American Association for the Advancement of Science and the ASME in 2007. In 2011 he won the Outstanding Research Award of the IEEE Intelligent Transportation Systems Society, and in 2014 he was given the Norbert Wiener Award of the IEEE Systems, Man, and Cybernetics Society "for fundamental contributions to and innovations in the theory and application of intelligent control and management to complex systems."