mxnet.operator¶

numpy interface for operators.

Functions

Classes

class mxnet.operator.CustomOp[source]¶

Bases: object

Base class for operators implemented in python

Methods

assign(dst, req, src)	Helper function for assigning into dst depending on requirements.
backward(req, out_grad, in_data, out_data, …)	Backward interface.
forward(is_train, req, in_data, out_data, aux)	Forward interface.

assign(dst, req, src)[source]¶

Helper function for assigning into dst depending on requirements.

backward(req, out_grad, in_data, out_data, in_grad, aux)[source]¶

Backward interface. Can override when creating new operators.

Parameters

• req (list of str) – how to assign to in_grad. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.

• in_data, out_data, in_grad, aux (out_grad,) – input and output for backward. See document for corresponding arguments of Operator::Backward

forward(is_train, req, in_data, out_data, aux)[source]¶

Forward interface. Can override when creating new operators.

Parameters

• is_train (bool) – whether this is for training

• req (list of str) – how to assign to out_data. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.

• out_data, aux (in_data,) – input, output, and auxiliary states for forward. See document for corresponding arguments of Operator::Forward

class mxnet.operator.CustomOpProp(need_top_grad=True)[source]¶

Bases: object

Base class for operator property class implemented in python.

Parameters

need_top_grad (bool) – The default declare_backward_dependency function. Use this value to determine whether this operator needs gradient input.

Methods

create_operator(ctx, in_shapes, in_dtypes)	Create an operator that carries out the real computation given the context, input shapes, and input data types.
declare_backward_dependency(out_grad, …)	Declare dependencies of this operator for backward pass.
infer_shape(in_shape)	infer_shape interface. Can override when creating new operators.
infer_storage_type(in_stype)	infer_storage_type interface. Used to infer storage type of
infer_storage_type_backward(ograd_stype, …)	infer_storage_type_backward interface. Used to infer storage
infer_type(in_type)	infer_type interface. override to create new operators
list_arguments()	list_arguments interface. Can override when creating new operators.
list_auxiliary_states()	list_auxiliary_states interface. Can override when creating new operators.
list_outputs()	list_outputs interface. Can override when creating new operators.

create_operator(ctx, in_shapes, in_dtypes)[source]¶

Create an operator that carries out the real computation given the context, input shapes, and input data types.

declare_backward_dependency(out_grad, in_data, out_data)[source]¶

Declare dependencies of this operator for backward pass.

Parameters

• out_grad (list of int) – ids of out_grad blobs.

• in_data (list of int) – ids of in_data blobs.

• out_data (list of int) – ids of out_data blobs.

Returns

deps – ids of the needed blobs.

Return type

list of int

infer_shape(in_shape)[source]¶

infer_shape interface. Can override when creating new operators.

Parameters

in_shape (list) – List of argument shapes in the same order as declared in list_arguments.

Returns

• in_shape (list) – List of argument shapes. Can be modified from in_shape.

• out_shape (list) – List of output shapes calculated from in_shape, in the same order as declared in list_outputs.

• aux_shape (Optional, list) – List of aux shapes calculated from in_shape, in the same order as declared in list_auxiliary_states.

infer_storage_type(in_stype)[source]¶

infer_storage_type interface. Used to infer storage type of inputs and outputs in the forward pass. When this interface is not implemented, all stypes will be inferred as default.

Parameters

in_stype (list of stypes, valid stypes are default, row_sparse and) – csr

Returns

• in_stype (list) – list of argument stypes.

• out_stype (list) – list of output types calculated from in_stype, in the same order as declared in list_outputs.

• aux_type (Optional, list) – list of aux types calculated from in_stype, in the same order as declared in list_auxiliary_states.

infer_storage_type_backward(ograd_stype, in_stype, out_stype, igrad_stype, aux_stype)[source]¶

infer_storage_type_backward interface. Used to infer storage type of inputs and outputs in the backward pass.

Will raise an error if undefined storage type is returned. Returned lists have to be the same size as the input lists to infer_storage_type_backward, otherwise an exception will be thrown. When this interface is not implemented, all stypes will be inferred as default.

Parameters

• ograd_stype (list) – list of output gradient storage types

• in_stype (list) – list of input storage types

• out_stype (list) – list of output storage types

• igrad_stype (list) – list of input gradient storage types

• aux_stype (list) – list of auxiliary storage types

Returns

• ograd_stype (list) – list of inferred output gradient storage types

• in_stype (list) – list of inferred input storage types

• out_stype (list) – list of inferred output storage types

• igrad_stype (list) – list of inferred input gradient storage types

• aux_stype (list) – list of inferred storage types for auxiliary states

infer_type(in_type)[source]¶

infer_type interface. override to create new operators

Parameters

in_type (list of np.dtype) – list of argument types in the same order as declared in list_arguments.

Returns

• in_type (list) – list of argument types. Can be modified from in_type.

• out_type (list) – list of output types calculated from in_type, in the same order as declared in list_outputs.

• aux_type (Optional, list) – list of aux types calculated from in_type, in the same order as declared in list_auxiliary_states.

list_arguments()[source]¶

list_arguments interface. Can override when creating new operators.

Returns

arguments – List of argument blob names.

Return type

list

list_auxiliary_states()[source]¶

list_auxiliary_states interface. Can override when creating new operators.

Returns

auxs – list of auxiliary state blob names.

Return type

list

list_outputs()[source]¶

list_outputs interface. Can override when creating new operators.

Returns

outputs – List of output blob names.

Return type

list

class mxnet.operator.NDArrayOp(need_top_grad=True)[source]¶

Bases: mxnet.operator.PythonOp

Base class for numpy operators. numpy operators allow parts of computation in symbolic graph to be writen in numpy. This feature is intended for quickly hacking out a solution for non performance critical parts. Please consider write a c++ implementation if it becomes a bottleneck. Note that if your operator contains internal states (like arrays), it cannot be used for multi-gpu training.

Methods

declare_backward_dependency(out_grad, …)	Declare dependencies of this operator for backward pass.
get_symbol(*args, **kwargs)	Create a symbol from numpy operator.

declare_backward_dependency(out_grad, in_data, out_data)[source]¶

Declare dependencies of this operator for backward pass.

Parameters

• out_grad (list of int) – ids of out_grad blobs.

• in_data (list of int) – ids of in_data blobs.

• out_data (list of int) – ids of out_data blobs.

Returns

deps – ids of the needed blobs.

Return type

list of int

get_symbol(*args, **kwargs)[source]¶

Create a symbol from numpy operator. This should only be called once per instance if the operator contains internal states.

Parameters

args (list) – a list of input arguments (symbols).

Returns

sym

Return type

mxnet.symbol.Symbol

class mxnet.operator.NumpyOp(need_top_grad=True)[source]¶

Bases: mxnet.operator.PythonOp

Base class for numpy operators. numpy operators allow parts of computation in symbolic graph to be writen in numpy. This feature is intended for quickly hacking out a solution for non performance critical parts. Please consider write a c++ implementation if it becomes a bottleneck. Note that if your operator contains internal states (like arrays), it cannot be used for multi-gpu training.

Methods

get_symbol(*args, **kwargs)

Create a symbol from numpy operator.

get_symbol(*args, **kwargs)[source]¶

Create a symbol from numpy operator. This should only be called once per instance if the operator contains internal states.

Parameters

args (list) – a list of input arguments (symbols).

Returns

sym

Return type

mxnet.symbol.Symbol

class mxnet.operator.OperatorArguments(narg, names, types)¶

Bases: tuple

Attributes

names	Alias for field number 1
narg	Alias for field number 0
types	Alias for field number 2

names¶

Alias for field number 1

narg¶

Alias for field number 0

types¶

Alias for field number 2

class mxnet.operator.PythonOp(need_top_grad=True)[source]¶

Bases: object

Base class for operators implemented in Python.

Parameters

need_top_grad (bool) – the default need_top_grad() function returns this value.

Methods

backward(out_grad, in_data, out_data, in_grad)	Backward interface.
forward(in_data, out_data)	Forward interface.
get_symbol(*args, **kwargs)	Create a symbol from numpy operator.
infer_shape(in_shape)	Interface for infer_shape.
list_arguments()	Interface for list_arguments.
list_outputs()	Interface for list_outputs.
need_top_grad()	Whether this operator needs out_grad for backward.

backward(out_grad, in_data, out_data, in_grad)[source]¶

Backward interface. Can override when creating new operators.

Parameters

in_data, out_data, in_grad (out_grad,) – input and output for backward. See document for corresponding arguments of Operator::Backward

forward(in_data, out_data)[source]¶

Forward interface. Override to create new operators.

Parameters

out_data (in_data,) – input and output for forward. See document for corresponding arguments of Operator::Forward

get_symbol(*args, **kwargs)[source]¶

Create a symbol from numpy operator. This should only be called once per instance if the operator contains internal states.

Parameters

args (list) – a list of input arguments (symbols).

Returns

sym

Return type

mxnet.symbol.Symbol

infer_shape(in_shape)[source]¶

Interface for infer_shape. Can override when creating new operators.

Parameters

in_shape (list) – List of argument shapes in the same order as declared in list_arguments.

Returns

• in_shape (list) – List of argument shapes. Can be modified from in_shape.

• out_shape (list) – List of output shapes calculated from in_shape, in the same order as declared in list_arguments.

list_arguments()[source]¶

Interface for list_arguments. Can override when creating new operators.

Returns

in_shape – list of argument shapes in the same order as declared in list_arguments.

Return type

list

list_outputs()[source]¶

Interface for list_outputs. Can override when creating new operators.

Returns

outputs – List of output blob names.

Return type

list

need_top_grad()[source]¶

Whether this operator needs out_grad for backward.

Returns

need_top_grad – Whether this operator needs out_grad for backward. Should be set to False for loss layers.

Return type

bool

mxnet.operator.get_all_registered_operators()[source]¶

Get all registered MXNet operator names.

Returns

operator_names

Return type

list of string

mxnet.operator.get_operator_arguments(op_name)[source]¶

Given operator name, fetch operator arguments - number of arguments, argument names, argument types.

Parameters

op_name (str) – Handle for the operator

Returns

operator_arguments

Return type

OperatorArguments, namedtuple with number of arguments, names and types

mxnet.operator.register(reg_name)[source]¶

Register a subclass of CustomOpProp to the registry with name reg_name.