twodlearn.datasets.base module

Creates a generic dataset from a given data

class twodlearn.datasets.base.Dataset(data_x, data_y=None)

Bases: object

x

samples, [sample_id, sample …]

y

labels for each sample

epochs_completed

index_in_epoch

n_samples

class Iloc(dataset)

Bases: object

property dataset

get_stats()

iloc

property n_samples

next_batch(batch_size, shuffle=True)

Return the next batch_size examples from this data set.

next_batch_x(batch_size, shuffle=True)

Return the next batch_size examples from this data set.

normalizer

Decorator used to specify an optional property inside a model. The decorator works similar to @property, but the specified method correponds to the initialization of the property

sample

Decorator used to specify methods that have a set of local variables.

These methods consist of an initialization method (speficied with the function given to the decorator) and an execute method.

shuffle()

update_data(data_x, data_y=None)

property x

property y

class twodlearn.datasets.base.Datasets(train, valid=None, test=None)

Bases: object

mapper

Decorator used to specify an optional property inside a model. The decorator works similar to @property, but the specified method correponds to the initialization of the property

normalize()

Normalize dataset for having a training dataset with zero mean and standard deviation of one.

normalizer

Decorator used to specify an optional property inside a model. The decorator works similar to @property, but the specified method correponds to the initialization of the property

split_train(p_train, p_valid, p_test=0.0, shuffle=True)

class twodlearn.datasets.base.DfDataset(data_x, data_y=None)

Bases: twodlearn.datasets.base.Dataset

class twodlearn.datasets.base.MaxMinNormalizer(stats)

Bases: twodlearn.datasets.base.Normalizer

denormalize(batch)

normalize(batch)

class twodlearn.datasets.base.Normalizer(stats)

Bases: object

denormalize(batch)

normalize(batch)

class twodlearn.datasets.base.OneHotMap(n_classes)

Bases: object

property n_classes

class twodlearn.datasets.base.ReplaceMap(map)

Bases: object

class twodlearn.datasets.base.StackedMapper(mappers=None)

Bases: object

append(mapper)

property mappers

twodlearn.datasets.base.dense_to_one_hot(labels, num_classes=None)

Convert class labels from scalars to one-hot vectors.

twodlearn.datasets.base.load_gz_as_nparray(work_directory, filename, dtype=<class 'numpy.float32'>)

twodlearn.datasets.base.maybe_download(filename, work_directory, source_url)

Download the data from source url, unless it’s already here. :param filename: string, name of the file in the directory. :param work_directory: string, path to working directory. :param source_url: url to download from if file doesn’t exist.

Returns

Path to resulting file.