

Zico Kolter (CMU)
Time: 2:30 pm on Friday, October 7th, 2022
How can we leverage FFTbased methods in convolutional networks?
Convolutional neural networks are one of the most widely used models in deep learning. However, the actual computation of these networks is typically done by simply "unrolling" a (relatively small) convolutional operator manually, rather than using Fourierbased approaches, and spectral approaches to convolutions have been more or less absent from practical deep learning. In this talk, I'll argue that, in fact, there are several interesting extensions to traditional convolutions networks that can be enabled through FFTbaed approaches: specifically, the ability to _invert_ convolutional operators, offered by the FFTbased approaches, enables us to create a number of interesting new layers types, such as orthogonal convolutions or better solvers for certain classes of fixedpoint layers. I will conclude by discussing some of the larger implications and future outlook of such approaches.


