【张海樟】Convergence Theory of Deep Neural Networks

Convergence Theory of Deep Neural Networks

In the past few years, deep learning has achieved great successes for a wide range of machine learning problems including face recognition, speech recognition, game intelligence, natural language processing, and autonomous navigation. Compared to the achievements in engineering and applications, research on the mathematical theory of deep neural networks is still at its infancy. By far, most mathematical studies on the nonlinear function representation system of deep neural networks have been focusing on the expressive power of the system. There is little attention paid to the relationship between the parameters (weight matrices and bias vectors) and the convergence or convergence rate of the network.
On the other hand, convergence of a function representation system in terms of its parameters has always been a fundamental problem in pure and applied mathematics. A celebrated example is the Carleson theorem, which states that a Fourier series converges almost everywhere if its coefficients are square-summable. We aims at establishing a convergence theory, which provides characterization of the convergence and convergence rate of a deep neural network in terms of its parameters. In this talk, we present results for deep ReLU networks and deep convolutional neural network. The talk is based on joint work with Prof. Yuesheng Xu and my PhD student Wentao Huang.

2003年本科毕业于北京师范大学数学系，2006年硕士毕业于中科院数学所，2009年博士毕业于美国雪城大学（Syracuse University）数学系，2009年6月-2010年5月 密歇根大学（University of Michigan）博士后。2010年6月起担任中山大学教授、博士生导师。
研究兴趣包括学习理论、应用调和分析和函数逼近. 代表性成果有再生核的Weierstrass逼近定理, 深度神经网络的收敛性理论，以及再生核巴拿赫空间理论. 在Journal of Machine Learning Research、Applied and Computational Harmonic Analysis、Neural Networks, Neural Computation、Neurocomputing、Constructive Approximation、IEEE Transactions系列等发表多篇原创性工作, 单篇最高他引超过360次. 主持包括优秀青年基金在内的多项国家和省部级基金.