mxnet.initializer¶

Weight initializer.

Classes

Functions

class mxnet.initializer.Bilinear[source]¶

Bases: mxnet.initializer.Initializer

Initialize weight for upsampling layers.

class mxnet.initializer.Constant(value)[source]¶

Bases: mxnet.initializer.Initializer

Initializes the weights to a given value. The value passed in can be a scalar or a NDarray that matches the shape of the parameter to be set.

Parameters

value (float, NDArray) – Value to set.

Methods

dumps()

Saves the initializer to string

dumps()[source]¶

Saves the initializer to string

Returns

JSON formatted string that describes the initializer.

Return type

str

Examples

>>> # Create initializer and retrieve its parameters
...
>>> init = mx.init.Normal(0.5)
>>> init.dumps()
'["normal", {"sigma": 0.5}]'
>>> init = mx.init.Xavier(factor_type="in", magnitude=2.34)
>>> init.dumps()
'["xavier", {"rnd_type": "uniform", "magnitude": 2.34, "factor_type": "in"}]'

class mxnet.initializer.FusedRNN(init, num_hidden, num_layers, mode, bidirectional=False, forget_bias=1.0)[source]¶

Bases: mxnet.initializer.Initializer

Initialize parameters for fused rnn layers.

Parameters

• init (Initializer) – initializer applied to unpacked weights. Fall back to global initializer if None.

• num_hidden (int) – should be the same with arguments passed to FusedRNNCell.

• num_layers (int) – should be the same with arguments passed to FusedRNNCell.

• mode (str) – should be the same with arguments passed to FusedRNNCell.

• bidirectional (bool) – should be the same with arguments passed to FusedRNNCell.

• forget_bias (float) – should be the same with arguments passed to FusedRNNCell.

class mxnet.initializer.InitDesc[source]¶

Bases: str

Descriptor for the initialization pattern.

Parameters

• name (str) – Name of variable.

• attrs (dict of str to str) – Attributes of this variable taken from Symbol.attr_dict.

• global_init (Initializer) – Global initializer to fallback to.

class mxnet.initializer.Initializer(**kwargs)[source]¶

Bases: object

The base class of an initializer.

Methods

dumps()	Saves the initializer to string
set_verbosity([verbose, print_func])	Switch on/off verbose mode

dumps()[source]¶

Saves the initializer to string

Returns

JSON formatted string that describes the initializer.

Return type

str

Examples

>>> # Create initializer and retrieve its parameters
...
>>> init = mx.init.Normal(0.5)
>>> init.dumps()
'["normal", {"sigma": 0.5}]'
>>> init = mx.init.Xavier(factor_type="in", magnitude=2.34)
>>> init.dumps()
'["xavier", {"rnd_type": "uniform", "magnitude": 2.34, "factor_type": "in"}]'

set_verbosity(verbose=False, print_func=None)[source]¶

Switch on/off verbose mode

Parameters

• verbose (bool) – switch on/off verbose mode

• print_func (function) – A function that computes statistics of initialized arrays. Takes an NDArray and returns an str. Defaults to mean absolute value str((abs(x)/size(x)).asscalar()).

class mxnet.initializer.LSTMBias(forget_bias=1.0)[source]¶

Bases: mxnet.initializer.Initializer

Initialize all biases of an LSTMCell to 0.0 except for the forget gate whose bias is set to custom value.

Parameters

forget_bias (float, default 1.0) – bias for the forget gate. Jozefowicz et al. 2015 recommends setting this to 1.0.

class mxnet.initializer.Load(param, default_init=None, verbose=False)[source]¶

Bases: object

Initializes variables by loading data from file or dict.

Note Load will drop arg: or aux: from name and initialize the variables that match with the prefix dropped.

Parameters

• param (str or dict of str->`NDArray`) – Parameter file or dict mapping name to NDArray.

• default_init (Initializer) – Default initializer when name is not found in param.

• verbose (bool) – Flag for enabling logging of source when initializing.

class mxnet.initializer.MSRAPrelu(factor_type='avg', slope=0.25)[source]¶

Bases: mxnet.initializer.Xavier

Initialize the weight according to a MSRA paper.

This initializer implements Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, available at https://arxiv.org/abs/1502.01852.

This initializer is proposed for initialization related to ReLu activation, it makes some changes on top of Xavier method.

Parameters

• factor_type (str, optional) – Can be 'avg', 'in', or 'out'.

• slope (float, optional) – initial slope of any PReLU (or similar) nonlinearities.

class mxnet.initializer.Mixed(patterns, initializers)[source]¶

Bases: object

Initialize parameters using multiple initializers.

Parameters

• patterns (list of str) – List of regular expressions matching parameter names.

• initializers (list of Initializer) – List of initializers corresponding to patterns.

Example

>>> # Given 'module', an instance of 'mxnet.module.Module', initialize biases to zero
... # and every other parameter to random values with uniform distribution.
...
>>> init = mx.initializer.Mixed(['bias', '.*'], [mx.init.Zero(), mx.init.Uniform(0.1)])
>>> module.init_params(init)
>>>
>>> for dictionary in module.get_params():
...     for key in dictionary:
...         print(key)
...         print(dictionary[key].asnumpy())
...
fullyconnected1_weight
[[ 0.0097627   0.01856892  0.04303787]]
fullyconnected1_bias
[ 0.]

class mxnet.initializer.Normal(sigma=0.01)[source]¶

Bases: mxnet.initializer.Initializer

Initializes weights with random values sampled from a normal distribution with a mean of zero and standard deviation of sigma.

Parameters

sigma (float, optional) – Standard deviation of the normal distribution. Default standard deviation is 0.01.

Example

>>> # Given 'module', an instance of 'mxnet.module.Module', initialize weights
>>> # to random values sampled from a normal distribution.
...
>>> init = mx.init.Normal(0.5)
>>> module.init_params(init)
>>> for dictionary in module.get_params():
...     for key in dictionary:
...         print(key)
...         print(dictionary[key].asnumpy())
...
fullyconnected0_weight
[[-0.3214761  -0.12660924  0.53789419]]

class mxnet.initializer.One[source]¶

Bases: mxnet.initializer.Initializer

Initializes weights to one.

Example

>>> # Given 'module', an instance of 'mxnet.module.Module', initialize weights to one.
...
>>> init = mx.initializer.One()
>>> module.init_params(init)
>>> for dictionary in module.get_params():
...     for key in dictionary:
...         print(key)
...         print(dictionary[key].asnumpy())
...
fullyconnected0_weight
[[ 1.  1.  1.]]

class mxnet.initializer.Orthogonal(scale=1.414, rand_type='uniform')[source]¶

Bases: mxnet.initializer.Initializer

Initialize weight as orthogonal matrix.

This initializer implements Exact solutions to the nonlinear dynamics of learning in deep linear neural networks, available at https://arxiv.org/abs/1312.6120.

Parameters

• scale (float optional) – Scaling factor of weight.

• rand_type (string optional) – Use “uniform” or “normal” random number to initialize weight.

class mxnet.initializer.Uniform(scale=0.07)[source]¶

Bases: mxnet.initializer.Initializer

Initializes weights with random values uniformly sampled from a given range.

Parameters

scale (float, optional) – The bound on the range of the generated random values. Values are generated from the range [-scale, scale]. Default scale is 0.07.

Example

>>> # Given 'module', an instance of 'mxnet.module.Module', initialize weights
>>> # to random values uniformly sampled between -0.1 and 0.1.
...
>>> init = mx.init.Uniform(0.1)
>>> module.init_params(init)
>>> for dictionary in module.get_params():
...     for key in dictionary:
...         print(key)
...         print(dictionary[key].asnumpy())
...
fullyconnected0_weight
[[ 0.01360891 -0.02144304  0.08511933]]

class mxnet.initializer.Xavier(rnd_type='uniform', factor_type='avg', magnitude=3)[source]¶

Bases: mxnet.initializer.Initializer

Returns an initializer performing “Xavier” initialization for weights.

This initializer is designed to keep the scale of gradients roughly the same in all layers.

By default, rnd_type is 'uniform' and factor_type is 'avg', the initializer fills the weights with random numbers in the range of \([-c, c]\), where \(c = \sqrt{\frac{3.}{0.5 * (n_{in} + n_{out})}}\). \(n_{in}\) is the number of neurons feeding into weights, and \(n_{out}\) is the number of neurons the result is fed to.

If rnd_type is 'uniform' and factor_type is 'in', the \(c = \sqrt{\frac{3.}{n_{in}}}\). Similarly when factor_type is 'out', the \(c = \sqrt{\frac{3.}{n_{out}}}\).

If rnd_type is 'gaussian' and factor_type is 'avg', the initializer fills the weights with numbers from normal distribution with a standard deviation of \(\sqrt{\frac{3.}{0.5 * (n_{in} + n_{out})}}\).

Parameters

• rnd_type (str, optional) – Random generator type, can be 'gaussian' or 'uniform'.

• factor_type (str, optional) – Can be 'avg', 'in', or 'out'.

• magnitude (float, optional) – Scale of random number.

class mxnet.initializer.Zero[source]¶

Bases: mxnet.initializer.Initializer

Initializes weights to zero.

Example

>>> # Given 'module', an instance of 'mxnet.module.Module', initialize weights to zero.
...
>>> init = mx.initializer.Zero()
>>> module.init_params(init)
>>> for dictionary in module.get_params():
...     for key in dictionary:
...         print(key)
...         print(dictionary[key].asnumpy())
...
fullyconnected0_weight
[[ 0.  0.  0.]]

mxnet.initializer.register(klass)[source]¶

Registers a custom initializer.

Custom initializers can be created by extending mx.init.Initializer and implementing the required functions like _init_weight and _init_bias. The created initializer must be registered using mx.init.register before it can be called by name.

Parameters

klass (class) – A subclass of mx.init.Initializer that needs to be registered as a custom initializer.

Example

>>> # Create and register a custom initializer that
... # initializes weights to 0.1 and biases to 1.
...
>>> @mx.init.register
... @alias('myinit')
... class CustomInit(mx.init.Initializer):
...   def __init__(self):
...     super(CustomInit, self).__init__()
...   def _init_weight(self, _, arr):
...     arr[:] = 0.1
...   def _init_bias(self, _, arr):
...     arr[:] = 1
...
>>> # Module is an instance of 'mxnet.module.Module'
...
>>> module.init_params("custominit")
>>> # module.init_params("myinit")
>>> # module.init_params(CustomInit())