tf / Graph

TensorFlow 1 version

View source on GitHub

A TensorFlow computation, represented as a dataflow graph.

tf.Graph()

Used in the notebooks

Graphs are used by tf.functions to represent the function's computations. Each graph contains a set of tf.Operation objects, which represent units of computation; and tf.Tensor objects, which represent the units of data that flow between operations.

Using graphs directly (deprecated)

A tf.Graph can be constructed and used directly without a tf.function, as was required in TensorFlow 1, but this is deprecated and it is recommended to use a tf.function instead. If a graph is directly used, other deprecated TensorFlow 1 classes are also required to execute the graph, such as a tf.compat.v1.Session.

A default graph can be registered with the tf.Graph.as_default context manager. Then, operations will be added to the graph instead of being executed eagerly. For example:

g = tf.Graph()
with g.as_default():
  # Define operations and tensors in `g`.
  c = tf.constant(30.0)
  assert c.graph is g

tf.compat.v1.get_default_graph() can be used to obtain the default graph.

Important note: This class is not thread-safe for graph construction. All operations should be created from a single thread, or external synchronization must be provided. Unless otherwise specified, all methods are not thread-safe.

A Graph instance supports an arbitrary number of "collections" that are identified by name. For convenience when building a large graph, collections can store groups of related objects: for example, the tf.Variable uses a collection (named tf.GraphKeys.GLOBAL_VARIABLES) for all variables that are created during the construction of a graph. The caller may define additional collections by specifying a new name.

Methods

add_to_collection

View source

add_to_collection(
    name, value
)

Stores value in the collection with the given name.

Note that collections are not sets, so it is possible to add a value to a collection several times.

add_to_collections

View source

add_to_collections(
    names, value
)

Stores value in the collections given by names.

Note that collections are not sets, so it is possible to add a value to a collection several times. This function makes sure that duplicates in names are ignored, but it will not check for pre-existing membership of value in any of the collections in names.

names can be any iterable, but if names is a string, it is treated as a single collection name.

as_default

View source

as_default()

Returns a context manager that makes this Graph the default graph.

This method should be used if you want to create multiple graphs in the same process. For convenience, a global default graph is provided, and all ops will be added to this graph if you do not create a new graph explicitly.

Use this method with the with keyword to specify that ops created within the scope of a block should be added to this graph. In this case, once the scope of the with is exited, the previous default graph is set again as default. There is a stack, so it's ok to have multiple nested levels of as_default calls.

The default graph is a property of the current thread. If you create a new thread, and wish to use the default graph in that thread, you must explicitly add a with g.as_default(): in that thread's function.

The following code examples are equivalent:

# 1. Using Graph.as_default():
g = tf.Graph()
with g.as_default():
  c = tf.constant(5.0)
  assert c.graph is g

# 2. Constructing and making default:
with tf.Graph().as_default() as g:
  c = tf.constant(5.0)
  assert c.graph is g

If eager execution is enabled ops created under this context manager will be added to the graph instead of executed eagerly.

as_graph_def

View source

as_graph_def(
    from_version=None, add_shapes=False
)

Returns a serialized GraphDef representation of this graph.

The serialized GraphDef can be imported into another Graph (using tf.import_graph_def) or used with the C++ Session API.

This method is thread-safe.

as_graph_element

View source

as_graph_element(
    obj, allow_tensor=True, allow_operation=True
)

Returns the object referred to by obj, as an Operation or Tensor.

This function validates that obj represents an element of this graph, and gives an informative error message if it is not.

This function is the canonical way to get/validate an object of one of the allowed types from an external argument reference in the Session API.

This method may be called concurrently from multiple threads.

clear_collection

View source

clear_collection(
    name
)

Clears all values in a collection.

colocate_with

View source

@tf_contextlib.contextmanager
colocate_with(
    op, ignore_existing=False
)

Returns a context manager that specifies an op to colocate with.

For example:

a = tf.Variable([1.0])
with g.colocate_with(a):
  b = tf.constant(1.0)
  c = tf.add(a, b)

b and c will always be colocated with a, no matter where a is eventually placed.

If op is None then ignore_existing must be True and the new scope resets all colocation and device constraints.

container

View source

@tf_contextlib.contextmanager
container(
    container_name
)

Returns a context manager that specifies the resource container to use.

Stateful operations, such as variables and queues, can maintain their states on devices so that they can be shared by multiple processes. A resource container is a string name under which these stateful operations are tracked. These resources can be released or cleared with tf.Session.reset().

For example:

with g.container('experiment0'):
  # All stateful Operations constructed in this context will be placed
  # in resource container "experiment0".
  v1 = tf.Variable([1.0])
  v2 = tf.Variable([2.0])
  with g.container("experiment1"):
    # All stateful Operations constructed in this context will be
    # placed in resource container "experiment1".
    v3 = tf.Variable([3.0])
    q1 = tf.queue.FIFOQueue(10, tf.float32)
  # All stateful Operations constructed in this context will be
  # be created in the "experiment0".
  v4 = tf.Variable([4.0])
  q1 = tf.queue.FIFOQueue(20, tf.float32)
  with g.container(""):
    # All stateful Operations constructed in this context will be
    # be placed in the default resource container.
    v5 = tf.Variable([5.0])
    q3 = tf.queue.FIFOQueue(30, tf.float32)

# Resets container "experiment0", after which the state of v1, v2, v4, q1
# will become undefined (such as uninitialized).
tf.Session.reset(target, ["experiment0"])

control_dependencies

View source

control_dependencies(
    control_inputs
)

Returns a context manager that specifies control dependencies.

Use with the with keyword to specify that all operations constructed within the context should have control dependencies on control_inputs. For example:

with g.control_dependencies([a, b, c]):
  # `d` and `e` will only run after `a`, `b`, and `c` have executed.
  d = ...
  e = ...

Multiple calls to control_dependencies() can be nested, and in that case a new Operation will have control dependencies on the union of control_inputs from all active contexts.

with g.control_dependencies([a, b]):
  # Ops constructed here run after `a` and `b`.
  with g.control_dependencies([c, d]):
    # Ops constructed here run after `a`, `b`, `c`, and `d`.

You can pass None to clear the control dependencies:

with g.control_dependencies([a, b]):
  # Ops constructed here run after `a` and `b`.
  with g.control_dependencies(None):
    # Ops constructed here run normally, not waiting for either `a` or `b`.
    with g.control_dependencies([c, d]):
      # Ops constructed here run after `c` and `d`, also not waiting
      # for either `a` or `b`.

# WRONG
def my_func(pred, tensor):
  t = tf.matmul(tensor, tensor)
  with tf.control_dependencies([pred]):
    # The matmul op is created outside the context, so no control
    # dependency will be added.
    return t

# RIGHT
def my_func(pred, tensor):
  with tf.control_dependencies([pred]):
    # The matmul op is created in the context, so a control dependency
    # will be added.
    return tf.matmul(tensor, tensor)

Also note that though execution of ops created under this scope will trigger execution of the dependencies, the ops created under this scope might still be pruned from a normal tensorflow graph. For example, in the following snippet of code the dependencies are never executed:

  loss = model.loss()
  with tf.control_dependencies(dependencies):
    loss = loss + tf.constant(1)  # note: dependencies ignored in the
                                  # backward pass
  return tf.gradients(loss, model.variables)

This is because evaluating the gradient graph does not require evaluating the constant(1) op created in the forward pass.

create_op

View source

create_op(
    op_type, inputs, dtypes=None, input_types=None, name=None, attrs=None,
    op_def=None, compute_shapes=True, compute_device=True
)

Creates an Operation in this graph. (deprecated arguments)

This is a low-level interface for creating an Operation. Most programs will not call this method directly, and instead use the Python op constructors, such as tf.constant(), which add ops to the default graph.

device

View source

@tf_contextlib.contextmanager
device(
    device_name_or_function
)

Returns a context manager that specifies the default device to use.

The device_name_or_function argument may either be a device name string, a device function, or None:

• If it is a device name string, all operations constructed in this context will be assigned to the device with that name, unless overridden by a nested device() context.
• If it is a function, it will be treated as a function from Operation objects to device name strings, and invoked each time a new Operation is created. The Operation will be assigned to the device with the returned name.
• If it is None, all device() invocations from the enclosing context will be ignored.

For information about the valid syntax of device name strings, see the documentation in DeviceNameUtils.

For example:

with g.device('/device:GPU:0'):
  # All operations constructed in this context will be placed
  # on GPU 0.
  with g.device(None):
    # All operations constructed in this context will have no
    # assigned device.

# Defines a function from `Operation` to device string.
def matmul_on_gpu(n):
  if n.type == "MatMul":
    return "/device:GPU:0"
  else:
    return "/cpu:0"

with g.device(matmul_on_gpu):
  # All operations of type "MatMul" constructed in this context
  # will be placed on GPU 0; all other operations will be placed
  # on CPU 0.

finalize

View source

finalize()

Finalizes this graph, making it read-only.

After calling g.finalize(), no new operations can be added to g. This method is used to ensure that no operations are added to a graph when it is shared between multiple threads, for example when using a tf.compat.v1.train.QueueRunner.

get_all_collection_keys

View source

get_all_collection_keys()

Returns a list of collections used in this graph.

get_collection

View source

get_collection(
    name, scope=None
)

Returns a list of values in the collection with the given name.

This is different from get_collection_ref() which always returns the actual collection list if it exists in that it returns a new list each time it is called.

get_collection_ref

View source

get_collection_ref(
    name
)

Returns a list of values in the collection with the given name.

If the collection exists, this returns the list itself, which can be modified in place to change the collection. If the collection does not exist, it is created as an empty list and the list is returned.

This is different from get_collection() which always returns a copy of the collection list if it exists and never creates an empty collection.

get_name_scope

View source

get_name_scope()

Returns the current name scope.

For example:

with tf.name_scope('scope1'):
  with tf.name_scope('scope2'):
    print(tf.compat.v1.get_default_graph().get_name_scope())

would print the string scope1/scope2.

get_operation_by_name

View source

get_operation_by_name(
    name
)

Returns the Operation with the given name.

This method may be called concurrently from multiple threads.

get_operations

View source

get_operations()

Return the list of operations in the graph.

You can modify the operations in place, but modifications to the list such as inserts/delete have no effect on the list of operations known to the graph.

This method may be called concurrently from multiple threads.

get_tensor_by_name

View source

get_tensor_by_name(
    name
)

Returns the Tensor with the given name.

This method may be called concurrently from multiple threads.

gradient_override_map

View source

@tf_contextlib.contextmanager
gradient_override_map(
    op_type_map
)

EXPERIMENTAL: A context manager for overriding gradient functions.

This context manager can be used to override the gradient function that will be used for ops within the scope of the context.

For example:

@tf.RegisterGradient("CustomSquare")
def _custom_square_grad(op, grad):
  # ...

with tf.Graph().as_default() as g:
  c = tf.constant(5.0)
  s_1 = tf.square(c)  # Uses the default gradient for tf.square.
  with g.gradient_override_map({"Square": "CustomSquare"}):
    s_2 = tf.square(s_2)  # Uses _custom_square_grad to compute the
                          # gradient of s_2.

is_feedable

View source

is_feedable(
    tensor
)

Returns True if and only if tensor is feedable.

is_fetchable

View source

is_fetchable(
    tensor_or_op
)

Returns True if and only if tensor_or_op is fetchable.

name_scope

View source

@tf_contextlib.contextmanager
name_scope(
    name
)

Returns a context manager that creates hierarchical names for operations.

A graph maintains a stack of name scopes. A with name_scope(...): statement pushes a new name onto the stack for the lifetime of the context.

The name argument will be interpreted as follows:

• A string (not ending with '/') will create a new name scope, in which name is appended to the prefix of all operations created in the context. If name has been used before, it will be made unique by calling self.unique_name(name).
• A scope previously captured from a with g.name_scope(...) as scope: statement will be treated as an "absolute" name scope, which makes it possible to re-enter existing scopes.
• A value of None or the empty string will reset the current name scope to the top-level (empty) name scope.

For example:

with tf.Graph().as_default() as g:
  c = tf.constant(5.0, name="c")
  assert c.op.name == "c"
  c_1 = tf.constant(6.0, name="c")
  assert c_1.op.name == "c_1"

  # Creates a scope called "nested"
  with g.name_scope("nested") as scope:
    nested_c = tf.constant(10.0, name="c")
    assert nested_c.op.name == "nested/c"

    # Creates a nested scope called "inner".
    with g.name_scope("inner"):
      nested_inner_c = tf.constant(20.0, name="c")
      assert nested_inner_c.op.name == "nested/inner/c"

    # Create a nested scope called "inner_1".
    with g.name_scope("inner"):
      nested_inner_1_c = tf.constant(30.0, name="c")
      assert nested_inner_1_c.op.name == "nested/inner_1/c"

      # Treats `scope` as an absolute name scope, and
      # switches to the "nested/" scope.
      with g.name_scope(scope):
        nested_d = tf.constant(40.0, name="d")
        assert nested_d.op.name == "nested/d"

        with g.name_scope(""):
          e = tf.constant(50.0, name="e")
          assert e.op.name == "e"

The name of the scope itself can be captured by with g.name_scope(...) as scope:, which stores the name of the scope in the variable scope. This value can be used to name an operation that represents the overall result of executing the ops in a scope. For example:

inputs = tf.constant(...)
with g.name_scope('my_layer') as scope:
  weights = tf.Variable(..., name="weights")
  biases = tf.Variable(..., name="biases")
  affine = tf.matmul(inputs, weights) + biases
  output = tf.nn.relu(affine, name=scope)

[A-Za-z0-9.][A-Za-z0-9_.\-/]* (for scopes at the root)
[A-Za-z0-9_.\-/]* (for other scopes)

prevent_feeding

View source

prevent_feeding(
    tensor
)

Marks the given tensor as unfeedable in this graph.

prevent_fetching

View source

prevent_fetching(
    op
)

Marks the given op as unfetchable in this graph.

switch_to_thread_local

View source

switch_to_thread_local()

Make device, colocation and dependencies stacks thread-local.

Device, colocation and dependencies stacks are not thread-local be default. If multiple threads access them, then the state is shared. This means that one thread may affect the behavior of another thread.

After this method is called, the stacks become thread-local. If multiple threads access them, then the state is not shared. Each thread uses its own value; a thread doesn't affect other threads by mutating such a stack.

The initial value for every thread's stack is set to the current value of the stack when switch_to_thread_local() was first called.

unique_name

View source

unique_name(
    name, mark_as_used=True
)

Return a unique operation name for name.

unique_name is used to generate structured names, separated by "/", to help identify operations when debugging a graph. Operation names are displayed in error messages reported by the TensorFlow runtime, and in various visualization tools such as TensorBoard.

If mark_as_used is set to True, which is the default, a new unique name is created and marked as in use. If it's set to False, the unique name is returned without actually being marked as used. This is useful when the caller simply wants to know what the name to be created will be.