

Rayan Saab (UCSD)
Time: 1:45 pm on Friday, October 7th, 2022
Quantizing neural networks
Neural networks are highly nonlinear functions often parametrized by a staggering number of weights. Miniaturizing these networks and implementing them in hardware is a direction of research that is fueled by a practical need, and at the same time connects to interesting mathematical problems. For example, by quantizing, or replacing the weights of a neural network with quantized (e.g., binary) counterparts, massive savings in cost, computation time, memory, and power consumption can be attained. Of course, one wishes to attain these savings while preserving the action of the function on domains of interest.
We present datadriven and computationally efficient methods for quantizing the weights of already trained neural networks and we prove that our methods have favorable error guarantees under a variety of assumptions. We also discuss extensions and provide the results of numerical experiments, on large multilayer networks, to illustrate the performance of our methods. Time permitting, we will also discuss open problems and related areas of research.


