mxnet.recordio¶

Read and write for the RecordIO data format.

Classes

Functions

mxnet.recordio.IRHeader¶

An alias for HEADER. Used to store metadata (e.g. labels) accompanying a record. See mxnet.recordio.pack and mxnet.recordio.pack_img for example uses.

Parameters

• flag (int) – Available for convenience, can be set arbitrarily.

• label (float or an array of float) – Typically used to store label(s) for a record.

• id (int) – Usually a unique id representing record.

• id2 (int) – Higher order bits of the unique id, should be set to 0 (in most cases).

Attributes

flag	Alias for field number 0
id	Alias for field number 2
id2	Alias for field number 3
label	Alias for field number 1

alias of mxnet.recordio.HEADER

class mxnet.recordio.MXIndexedRecordIO(idx_path, uri, flag, key_type=<class 'int'>)[source]¶

Bases: mxnet.recordio.MXRecordIO

Reads/writes RecordIO data format, supporting random access.

Examples

Methods

close()	Closes the record file.
open()	Opens the record file.
read_idx(idx)	Returns the record at given index.
seek(idx)	Sets the current read pointer position.
tell()	Returns the current position of write head.
write_idx(idx, buf)	Inserts input record at given index.

>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'r')
>>> record.read_idx(3)
record_3

Parameters

• idx_path (str) – Path to the index file.

• uri (str) – Path to the record file. Only supports seekable file types.

• flag (str) – ‘w’ for write or ‘r’ for read.

• key_type (type) – Data type for keys.

close()[source]¶

Closes the record file.

open()[source]¶

Opens the record file.

read_idx(idx)[source]¶

Returns the record at given index.

Examples

>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'w')
>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'r')
>>> record.read_idx(3)
record_3

seek(idx)[source]¶

Sets the current read pointer position.

This function is internally called by read_idx(idx) to find the current reader pointer position. It doesn’t return anything.

tell()[source]¶

Returns the current position of write head.

Examples

>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'w')
>>> print(record.tell())
0
>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
...     print(record.tell())
16
32
48
64
80

write_idx(idx, buf)[source]¶

Inserts input record at given index.

Examples

>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()

Parameters

• idx (int) – Index of a file.

• buf – Record to write.

class mxnet.recordio.MXRecordIO(uri, flag)[source]¶

Bases: object

Reads/writes RecordIO data format, supporting sequential read and write.

Examples

Methods

close()	Closes the record file.
open()	Opens the record file.
read()	Returns record as a string.
reset()	Resets the pointer to first item.
write(buf)	Inserts a string buffer as a record.

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'w')
<mxnet.recordio.MXRecordIO object at 0x10ef40ed0>
>>> for i in range(5):
...    record.write('record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(5):
...    item = record.read()
...    print(item)
record_0
record_1
record_2
record_3
record_4
>>> record.close()

Parameters

• uri (string) – Path to the record file.

• flag (string) – ‘w’ for write or ‘r’ for read.

close()[source]¶

Closes the record file.

open()[source]¶

Opens the record file.

read()[source]¶

Returns record as a string.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(5):
...    item = record.read()
...    print(item)
record_0
record_1
record_2
record_3
record_4
>>> record.close()

Returns

buf – Buffer read.

Return type

string

reset()[source]¶

Resets the pointer to first item.

If the record is opened with ‘w’, this function will truncate the file to empty.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(2):
...    item = record.read()
...    print(item)
record_0
record_1
>>> record.reset()  # Pointer is reset.
>>> print(record.read()) # Started reading from start again.
record_0
>>> record.close()

write(buf)[source]¶

Inserts a string buffer as a record.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'w')
>>> for i in range(5):
...    record.write('record_%d'%i)
>>> record.close()

Parameters

buf (string (python2), bytes (python3)) – Buffer to write.

mxnet.recordio.pack(header, s)[source]¶

Pack a string into MXImageRecord.

Parameters

• header (IRHeader) – Header of the image record. header.label can be a number or an array. See more detail in IRHeader.

• s (str) – Raw image string to be packed.

Returns

s – The packed string.

Return type

str

Examples

>>> label = 4 # label can also be a 1-D array, for example: label = [1,2,3]
>>> id = 2574
>>> header = mx.recordio.IRHeader(0, label, id, 0)
>>> with open(path, 'r') as file:
...     s = file.read()
>>> packed_s = mx.recordio.pack(header, s)

mxnet.recordio.pack_img(header, img, quality=95, img_fmt='.jpg')[source]¶

Pack an image into MXImageRecord.

Parameters

• header (IRHeader) – Header of the image record. header.label can be a number or an array. See more detail in IRHeader.

• img (numpy.ndarray) – Image to be packed.

• quality (int) – Quality for JPEG encoding in range 1-100, or compression for PNG encoding in range 1-9.

• img_fmt (str) – Encoding of the image (.jpg for JPEG, .png for PNG).

Returns

s – The packed string.

Return type

str

Examples

>>> label = 4 # label can also be a 1-D array, for example: label = [1,2,3]
>>> id = 2574
>>> header = mx.recordio.IRHeader(0, label, id, 0)
>>> img = cv2.imread('test.jpg')
>>> packed_s = mx.recordio.pack_img(header, img)

mxnet.recordio.unpack(s)[source]¶

Unpack a MXImageRecord to string.

Parameters

s (str) – String buffer from MXRecordIO.read.

Returns

• header (IRHeader) – Header of the image record.

• s (str) – Unpacked string.

Examples

>>> record = mx.recordio.MXRecordIO('test.rec', 'r')
>>> item = record.read()
>>> header, s = mx.recordio.unpack(item)
>>> header
HEADER(flag=0, label=14.0, id=20129312, id2=0)

mxnet.recordio.unpack_img(s, iscolor=-1)[source]¶

Unpack a MXImageRecord to image.

Parameters

• s (str) – String buffer from MXRecordIO.read.

• iscolor (int) – Image format option for cv2.imdecode.

Returns

• header (IRHeader) – Header of the image record.

• img (numpy.ndarray) – Unpacked image.

Examples

>>> record = mx.recordio.MXRecordIO('test.rec', 'r')
>>> item = record.read()
>>> header, img = mx.recordio.unpack_img(item)
>>> header
HEADER(flag=0, label=14.0, id=20129312, id2=0)
>>> img
array([[[ 23,  27,  45],
        [ 28,  32,  50],
        ...,
        [ 36,  40,  59],
        [ 35,  39,  58]],
       ...,
       [[ 91,  92, 113],
        [ 97,  98, 119],
        ...,
        [168, 169, 167],
        [166, 167, 165]]], dtype=uint8)