The penalties are applied on a per-layer basis. The exact API will depend on the layer, but the layers Dense
, Conv1D
, Conv2D
and Conv3D
have a unified API.
These layers expose 2 keyword arguments:
kernel_constraint
for the main weights matrixbias_constraint
for the bias.
Available constraints
MaxNorm weight constraint.
Arguments
- max_value: the maximum norm for the incoming weights.
- axis: integer, axis along which to calculate weight norms. For instance, in a
Dense
layer the weight matrix has shape(input_dim, output_dim)
, setaxis
to0
to constrain each weight vector of length(input_dim,)
. In aConv2D
layer with , the weight tensor has shape(rows, cols, input_depth, output_depth)
, setaxis
to[0, 1, 2]
to constrain the weights of each filter tensor of size(rows, cols, input_depth)
.
References
Constrains the weights to be non-negative.
Constrains the weights incident to each hidden unit to have unit norm.
Arguments
- axis: integer, axis along which to calculate weight norms. For instance, in a
Dense
layer the weight matrix has shape(input_dim, output_dim)
, setaxis
to0
to constrain each weight vector of length(input_dim,)
. In aConv2D
layer withdata_format="channels_last"
, the weight tensor has shape(rows, cols, input_depth, output_depth)
, set to[0, 1, 2]
to constrain the weights of each filter tensor of size(rows, cols, input_depth)
.
MinMaxNorm weight constraint.
Arguments
- min_value: the minimum norm for the incoming weights.
- max_value: the maximum norm for the incoming weights.
- rate: rate for enforcing the constraint: weights will be rescaled to yield
(1 - rate) norm + rate norm.clip(min_value, max_value)
. Effectively, this means that rate=1.0 stands for strict enforcement of the constraint, while rate<1.0 means that weights will be rescaled at each step to slowly move towards a value inside the desired interval.