actorcritic.nn

Contains utilities that concern TensorFlow and neural networks.

Functions

conv2d(input, params, stride, padding)	Creates a 2D convolutional layer with bias (without activation).
conv2d_params(num_input_channels, …)	Creates weights and bias variables for a 2D convolutional layer.
flatten(input)	Flattens inputs but keeps the batch size.
fully_connected(input, params)	Creates a fully connected layer with bias (without activation).
fully_connected_params(input_size, …)	Creates weights and bias variables for a fully connected layer.
linear_decay(start_value, end_value, step, …)	Applies linear decay from start_value to end_value.

Classes

ClipGlobalNormOptimizer(optimizer, clip_norm)

A tf.train.Optimizer that wraps around another optimizer and minimizes the loss by clipping gradients using the global norm (tf.clip_by_global_norm()).

class actorcritic.nn.ClipGlobalNormOptimizer(optimizer, clip_norm, name=None)

Bases: tensorflow.python.training.optimizer.Optimizer

A tf.train.Optimizer that wraps around another optimizer and minimizes the loss by clipping gradients using the global norm (tf.clip_by_global_norm()).

See also

• https://www.tensorflow.org/versions/r1.2/api_docs/python/tf/clip_by_global_norm
• https://stackoverflow.com/questions/36498127/how-to-apply-gradient-clipping-in-tensorflow/43486487#43486487

__init__(optimizer, clip_norm, name=None)

Parameters:

• optimizer (tf.train.Optimizer) – An optimizer whose gradients will be clipped.
• clip_norm (tf.Tensor or float) – Value for the global norm (passed to tf.clip_by_global_norm()).
• name (string, optional) – A name for this optimizer.

apply_gradients(grads_and_vars, global_step=None, name=None)

Apply gradients to variables.

This is the second part of minimize(). It returns an Operation that applies gradients.

Parameters:

• grads_and_vars – List of (gradient, variable) pairs as returned by compute_gradients().
• global_step – Optional Variable to increment by one after the variables have been updated.
• name – Optional name for the returned operation. Default to the name passed to the Optimizer constructor.

Returns:

An Operation that applies the specified gradients. If global_step was not None, that operation also increments global_step.

Raises:

• TypeError – If grads_and_vars is malformed.
• ValueError – If none of the variables have gradients.

actorcritic.nn.conv2d(input, params, stride, padding)

Creates a 2D convolutional layer with bias (without activation).

Parameters:

• input (tf.Tensor) – The input values.
• params (tuple of (tf.Variable, tf.Variable)) – A tuple of (weights, bias). Probably obtained by conv2d_params().
• stride (int) – The stride of the convolution.
• padding (string) – The padding of the convolution. One of ‘VALID’, ‘SAME’.

Returns:

tf.Tensor – The output values.

actorcritic.nn.conv2d_params(num_input_channels, num_filters, filter_extent, dtype, weights_initializer, bias_initializer)

Creates weights and bias variables for a 2D convolutional layer. These can be used in conv2d().

Parameters:

• num_input_channels (int) – The size of the input layer.
• num_filters (int) – The output size. Number of filters to apply.
• filter_extent (int) – The spatial extent of the filters. Determines the size of the weights.
• dtype (tf.DType) – The data type of the variables.
• weights_initializer (tf.keras.initializers.Initializer) – An initializer for the weights.
• bias_initializer (tf.keras.initializers.Initializer) – An initializer for the bias.

Returns:

tuple of (tf.Variable, tf.Variable) – A tuple of (weights, bias).

actorcritic.nn.flatten(input)

Flattens inputs but keeps the batch size.

Parameters:input (tf.Tensor) – Input values of shape [batch_size, d_1, …, d_n]. Returns:tf.Tensor – Flattened input values of shape [batch_size, d1 * … * d_n].

actorcritic.nn.fully_connected(input, params)

Creates a fully connected layer with bias (without activation).

Parameters:

• input (tf.Tensor) – The input values.
• params (tuple of (tf.Variable, tf.Variable)) – A tuple of (weights, bias). Probably obtained by fully_connected_params().

Returns:

tf.Tensor – The output values.

actorcritic.nn.fully_connected_params(input_size, output_size, dtype, weights_initializer, bias_initializer)

Creates weights and bias variables for a fully connected layer. These can be used in fully_connected().

Parameters:

• input_size (int) – The size of the input layer.
• output_size (int) – The output size. Number of units.
• dtype (tf.DType) – The data type of the variables.
• weights_initializer (tf.keras.initializers.Initializer) – An initializer for the weights.
• bias_initializer (tf.keras.initializers.Initializer) – An initializer for the bias.

Returns:

tuple of (tf.Variable, tf.Variable) – A tuple of (weights, bias).

actorcritic.nn.linear_decay(start_value, end_value, step, total_steps, name=None)

Applies linear decay from start_value to end_value. The value at a specific step is computed as:

value = (start_value - end_value) * (1 - step / total_steps) + end_value

Parameters:

• start_value (tf.Tensor or float) – The start value.
• end_value (tf.Tensor or float) – The end value.
• step (tf.Tensor) – The current step (e.g. global_step).
• total_step (int or tf.Tensor) – The total number of steps. Steps to reach end_value.
• name (string, optional) – A name for the operation.

Returns:

tf.Tensor – The linear decayed value.