Parameterized activation functions to adjust model linearity

Abstract

Certain aspects of the present disclosure provide techniques for parameterized activation functions. Input data is processed with at least one layer of the neural network model comprising a parameterized activation function, and at least one trainable parameter of the parameterized activation function is updated based at least in part on output from the at least one layer of the neural network model. The at least one trainable parameter may adjust at least one of a range over which the parameterized activation function is nonlinear or a shape of the parameterized activation function, and/or may adjust a location of at least one pivot of the parameterized activation function.

CPC Classifications