twodlearn.optimv2 module

class twodlearn.optimv2.BaseOptimizer(**kargs)

Bases: twodlearn.core.common.TdlModel

Manages and runs an optimization operation over a loss tensor.

The basic usage is as follows

optim = Optimizer(
    loss=loss,
    learning_rate=0.01,
    var_list=tdl.core.get_trainable(model))

BaseOptimizer only runs the optimization operation. The BaseOptimizer can be extended using the wrap() method to construct optimizer classes that include monitoring, status bar, checkpoints, value checks, etc. The class can be extended as follows

MonitoredOptimizer = BaseOptimizer.wrap(CheckNan) \
                                  .wrap(Monitor)  \
                                  .wrap(StatusBar)

Tdl autoinitialization with arguments:

optimizer

(Submodel) Optimizer used to perform the optimization. AdamOptimizer is used by default.

_assert_initialized

(MethodInit) assert variables have been initialized

restart

(MethodInit) calls the initializer for all var_list and optimizer variables.

var_list

(InputArgument)

session

(InputArgument)

loss

(InputArgument)

feed_dict

(Submodel)

step_op

(Submodel) operation that performs the weight update.

learning_rate

(Submodel)

train_step

(LazzyProperty)

cleanup(**kwargs)

feed_dict

get_ops()

learning_rate

loss

optimizer

Optimizer used to perform the optimization. AdamOptimizer is used by default.

restart

calls the initializer for all var_list and optimizer variables.

run(n_steps, feed_dict=None, **kargs)

run_step(**kwargs)

session

step_fn(wrapped_fn=None, **kwargs)

step_op

operation that performs the weight update.

train_step

var_list

warmup(**kwargs)

classmethod wrap(Monitor).wrap(Logger)

class twodlearn.optimv2.CheckNan(**kargs)

Bases: twodlearn.core.common.TdlModel

Tdl autoinitialization with arguments:

check_nan

(SubmodelInit) Autoinit with arguments [‘n_trials’]

check_nan

Autoinit with arguments [‘n_trials’]

step_fn(wrapped_fn=None, **kwargs)

warmup(**kwargs)

class twodlearn.optimv2.CheckProgress(**kargs)

Bases: twodlearn.core.common.TdlModel

CheckProgress resets to last checkpoint if progress deteriorates.

Tdl autoinitialization with arguments:

progress

(SubmodelInit) checks if progress has been made in last step.

progress

checks if progress has been made in last step.

Parameters

• filter_window (int) – size of the buffer used by the moving average filter.

• reset_multiplier (float) – if current loss > reset_multiplier*filtered_loss, we restore the last checkpoint.

step_fn(wrapped_fn=None, **kwargs)

warmup(**kwargs)

class twodlearn.optimv2.Checkpointable(**kargs)

Bases: twodlearn.core.common.TdlModel

Tdl autoinitialization with arguments:

checkpoints

(SubmodelInit) checkpoints save the value of variables.

checkpoints

checkpoints save the value of variables.

Parameters

• buffer_size (int) – number of checkpoints saved in the buffer.

• update_dt (float) – minimum time for the checkpoints to be stored.

step_fn(wrapped_fn=None, **kwargs)

warmup(**kwargs)

class twodlearn.optimv2.CheckpointableProgress(**kargs)

Bases: twodlearn.optimv2.Checkpointable

CheckpointableProgress stores a checkpoint in internal buffer at a given frequency only if progress has been made.

filtered_target

Autoinit with arguments [‘window_size’]

step_fn(wrapped_fn=None, **kwargs)

warmup(**kwargs)

class twodlearn.optimv2.ConstantLr(value)

Bases: object

class twodlearn.optimv2.Monitor(**kargs)

Bases: twodlearn.core.common.TdlModel

Tdl autoinitialization with arguments:

monitor_manager

(SubmodelInit) Autoinit with arguments [‘metrics’, ‘valid_freq’]

feed_valid

(Submodel)

log_folder

(InputArgument)

loss_monitor

(LazzyProperty)

cleanup(**kwargs)

feed_valid

get_ops()

log_folder

loss_monitor

monitor_manager

Autoinit with arguments [‘metrics’, ‘valid_freq’]

step_fn(wrapped_fn=None, **kwargs)

warmup(**kwargs)

class twodlearn.optimv2.Optimizer(**kargs)

Bases: twodlearn.optimv2.Optimizer, twodlearn.optimv2.StatusBar

Tdl autoinitialization with arguments:

optimizer

(Submodel) Optimizer used to perform the optimization. AdamOptimizer is used by default.

_assert_initialized

(MethodInit) assert variables have been initialized

monitor_manager

(SubmodelInit) Autoinit with arguments [‘metrics’, ‘valid_freq’]

feed_valid

(Submodel)

log_folder

(InputArgument)

filtered_target

(SubmodelInit) Autoinit with arguments [‘window_size’]

progress

(SubmodelInit) checks if progress has been made in last step.

loss_monitor

(LazzyProperty)

restart

(MethodInit) calls the initializer for all var_list and optimizer variables.

checkpoints

(SubmodelInit) checkpoints save the value of variables.

check_nan

(SubmodelInit) Autoinit with arguments [‘n_trials’]

var_list

(InputArgument)

session

(InputArgument)

loss

(InputArgument)

feed_dict

(Submodel)

status_bar

(SubmodelInit) Autoinit with arguments [‘update_freq’]

step_op

(Submodel) operation that performs the weight update.

learning_rate

(Submodel)

train_step

(LazzyProperty)

cleanup(**kwargs)

get_ops()

run_step(**kwargs)

warmup(**kwargs)

class twodlearn.optimv2.Saver(**kargs)

Bases: twodlearn.core.common.TdlModel

Tdl autoinitialization with arguments:

saver

(LazzyProperty)

log_folder

(InputArgument)

cleanup(**kwargs)

log_folder

saver

warmup(**kwargs)

class twodlearn.optimv2.StatusBar(**kargs)

Bases: twodlearn.core.common.TdlModel

StatusBar adds a progress bar that also displays some tensor values.

Tdl autoinitialization with arguments:

status_bar

(SubmodelInit) Autoinit with arguments [‘update_freq’]

cleanup(**kwargs)

status_bar

Autoinit with arguments [‘update_freq’]

step_fn(wrapped_fn=None, **kwargs)

warmup(**kwargs)