gluon.utils¶

Parallelization utility optimizer.

Functions

mxnet.gluon.utils.split_data(data, num_slice, batch_axis=0, even_split=True)[source]¶

Splits an NDArray into num_slice slices along batch_axis. Usually used for data parallelism where each slices is sent to one device (i.e. GPU).

Parameters

• data (NDArray) – A batch of data.

• num_slice (int) – Number of desired slices.

• batch_axis (int, default 0) – The axis along which to slice.

• even_split (bool, default True) – Whether to force all slices to have the same number of elements. If True, an error will be raised when num_slice does not evenly divide data.shape[batch_axis].

Returns

Return value is a list even if num_slice is 1.

Return type

list of NDArray

mxnet.gluon.utils.split_and_load(data, ctx_list, batch_axis=0, even_split=True)[source]¶

Splits an NDArray into len(ctx_list) slices along batch_axis and loads each slice to one context in ctx_list.

Parameters

• data (NDArray or ndarray) – A batch of data.

• ctx_list (list of Context) – A list of Contexts.

• batch_axis (int, default 0) – The axis along which to slice.

• even_split (bool, default True) – Whether to force all slices to have the same number of elements.

Returns

Each corresponds to a context in ctx_list.

Return type

list of NDArrays or ndarrays

mxnet.gluon.utils.clip_global_norm(arrays, max_norm, check_isfinite=True)[source]¶

Rescales NDArrays so that the sum of their 2-norm is smaller than max_norm.

Parameters

• arrays (list of NDArray) –

• max_norm (float) –

• check_isfinite (bool, default True) – If True, check that the total_norm is finite (not nan or inf). This requires a blocking .asscalar() call.

Returns

Total norm. Return type is NDArray of shape (1,) if check_isfinite is False. Otherwise a float is returned.

Return type

NDArray or float

mxnet.gluon.utils.check_sha1(filename, sha1_hash)[source]¶

Check whether the sha1 hash of the file content matches the expected hash.

Parameters

• filename (str) – Path to the file.

• sha1_hash (str) – Expected sha1 hash in hexadecimal digits.

Returns

Whether the file content matches the expected hash.

Return type

bool

mxnet.gluon.utils.download(url, path=None, overwrite=False, sha1_hash=None, retries=5, verify_ssl=True)[source]¶

Download a given URL

Parameters

• url (str) – URL to download

• path (str, optional) – Destination path to store downloaded file. By default stores to the current directory with same name as in url.

• overwrite (bool, optional) – Whether to overwrite destination file if already exists.

• sha1_hash (str, optional) – Expected sha1 hash in hexadecimal digits. Will ignore existing file when hash is specified but doesn’t match.

• retries (integer, default 5) – The number of times to attempt the download in case of failure or non 200 return codes

• verify_ssl (bool, default True) – Verify SSL certificates.

Returns

The file path of the downloaded file.

Return type

str

mxnet.gluon.utils.replace_file(src, dst)[source]¶

Implement atomic os.replace with linux and OSX.

Parameters

• src (source file path) –

• dst (destination file path) –