Keras backends

At this time, Keras has three backend implementations available: the TensorFlow backend, the Theano backend, and the CNTK backend.

• is an open-source symbolic tensor manipulation framework developed by Google.
• Theano is an open-source symbolic tensor manipulation framework developed by LISA Lab at Université de Montréal.
• is an open-source toolkit for deep learning developed by Microsoft.

In the future, we are likely to add more backend options.

Switching from one backend to another

If you have run Keras at least once, you will find the Keras configuration file at:

If it isn't there, you can create it.

NOTE for Windows Users: Please replace $HOME with %USERPROFILE%.

The default configuration file looks like this:

Simply change the field backend to "theano", "tensorflow", or "cntk", and Keras will use the new configuration next time you run any Keras code.

You can also define the environment variable KERAS_BACKEND and this willoverride what is defined in your config file :

KERAS_BACKEND=tensorflow python -c "from keras import backend"
Using TensorFlow backend.

In Keras it is possible to load more backends than "tensorflow", "theano", and "cntk". Keras can use external backends as well, and this can be performed by changing the keras.json configuration file, and the "backend" setting. Suppose you have a Python module called my_module that you wanted to use as your external backend. The keras.json configuration file would be changed as follows:

{
    "image_data_format": "channels_last",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "my_package.my_module"
}

An external backend must be validated in order to be used, a valid backend must have the following functions: placeholder, variable and function.

If an external backend is not valid due to missing a required entry, an error will be logged notifying which entry/entries are missing.

The keras.json configuration file contains the following settings:

{
    "image_data_format": "channels_last",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "tensorflow"
}

You can change these settings by editing $HOME/.keras/keras.json.

• image_data_format: String, either "channels_last" or "channels_first". It specifies which data format convention Keras will follow. (keras.backend.image_data_format() returns it.)
• For 2D data (e.g. image), "channels_last" assumes (rows, cols, channels) while "channels_first" assumes (channels, rows, cols).
• For 3D data, "channels_last" assumes (conv_dim1, conv_dim2, conv_dim3, channels) while "channels_first" assumes (channels, conv_dim1, conv_dim2, conv_dim3).
• epsilon: Float, a numeric fuzzing constant used to avoid dividing by zero in some operations.
• floatx: String, "float16", "float32", or "float64". Default float precision.
• backend: String, "tensorflow", "theano", or "cntk".

Using the abstract Keras backend to write new code

If you want the Keras modules you write to be compatible with both Theano (th) and TensorFlow (tf), you have to write them via the abstract Keras backend API. Here's an intro.

You can import the backend module via:

from keras import backend as K

The code below instantiates an input placeholder. It's equivalent to tf.placeholder() or th.tensor.matrix(), th.tensor.tensor3(), etc.

inputs = K.placeholder(shape=(2, 4, 5))
# also works:
inputs = K.placeholder(shape=(None, 4, 5))
# also works:
inputs = K.placeholder(ndim=3)

The code below instantiates a variable. It's equivalent to tf.Variable() or th.shared().

import numpy as np
val = np.random.random((3, 4, 5))
var = K.variable(value=val)
# all-zeros variable:
var = K.zeros(shape=(3, 4, 5))
# all-ones:
var = K.ones(shape=(3, 4, 5))

Most tensor operations you will need can be done as you would in TensorFlow or Theano:

# Initializing Tensors with Random Numbers
b = K.random_uniform_variable(shape=(3, 4), low=0, high=1) # Uniform distribution
c = K.random_normal_variable(shape=(3, 4), mean=0, scale=1) # Gaussian distribution
d = K.random_normal_variable(shape=(3, 4), mean=0, scale=1)
# Tensor Arithmetic
a = b + c * K.abs(d)
c = K.dot(a, K.transpose(b))
a = K.sum(b, axis=1)
a = K.softmax(b)
a = K.concatenate([b, c], axis=-1)
# etc...

keras.backend.backend()

Returns the name of the current backend (e.g. "tensorflow").

Returns

String, the name of the backend Keras is currently using.

Example

>>> keras.backend.backend()
'tensorflow'

symbolic

keras.backend.symbolic(func)

Decorator used in TensorFlow 2.0 to enter the Keras graph.

Arguments

• func: Function to decorate.

Returns

Decorated function.

eager

keras.backend.eager(func)

Decorator used in TensorFlow 2.0 to exit the Keras graph.

Arguments

• func: Function to decorate.

Returns

Decorated function.

get_uid

keras.backend.get_uid(prefix='')

Provides a unique UID given a string prefix.

Arguments

• prefix: string.

Returns

An integer.

Example

>>> keras.backend.get_uid('dense')
1
>>> keras.backend.get_uid('dense')
2

manual_variable_initialization

keras.backend.manual_variable_initialization(value)

Sets the manual variable initialization flag.

This boolean flag determines whethervariables should be initializedas they are instantiated (default), or ifthe user should handle the initialization.

Arguments

• value: Python boolean.

epsilon

keras.backend.epsilon()

Returns the value of the fuzz factor used in numeric expressions.

Returns

A float.

Example

>>> keras.backend.epsilon()
1e-07

reset_uids

keras.backend.reset_uids()

Resets graph identifiers.

set_epsilon

keras.backend.set_epsilon(e)

Sets the value of the fuzz factor used in numeric expressions.

Arguments

• e: float. New value of epsilon.

Example

>>> from keras import backend as K
>>> K.epsilon()
1e-07
>>> K.set_epsilon(1e-05)
>>> K.epsilon()
1e-05

floatx

keras.backend.floatx()

Returns the default float type, as a string.(e.g. 'float16', 'float32', 'float64').

Returns

String, the current default float type.

Example

>>> keras.backend.floatx()
'float32'

set_floatx

keras.backend.set_floatx(floatx)

Sets the default float type.

Arguments

• floatx: String, 'float16', 'float32', or 'float64'.

Example

>>> from keras import backend as K
>>> K.floatx()
'float32'
>>> K.set_floatx('float16')
>>> K.floatx()
'float16'

cast_to_floatx

keras.backend.cast_to_floatx(x)

Cast a Numpy array to the default Keras float type.

Arguments

• x: Numpy array.

Returns

The same Numpy array, cast to its new type.

Example

>>> from keras import backend as K
>>> K.floatx()
'float32'
>>> arr = numpy.array([1.0, 2.0], dtype='float64')
>>> arr.dtype
dtype('float64')
>>> new_arr = K.cast_to_floatx(arr)
>>> new_arr
array([ 1.,  2.], dtype=float32)
>>> new_arr.dtype
dtype('float32')

image_data_format

keras.backend.image_data_format()

Returns the default image data format convention.

Returns

A string, either 'channels_first' or 'channels_last'

Example

>>> keras.backend.image_data_format()
'channels_first'

set_image_data_format

keras.backend.set_image_data_format(data_format)

Sets the value of the data format convention.

Arguments

• data_format: string. 'channels_first' or 'channels_last'.

Example

>>> from keras import backend as K
>>> K.image_data_format()
'channels_first'
>>> K.set_image_data_format('channels_last')
>>> K.image_data_format()
'channels_last'

learning_phase

keras.backend.learning_phase()

Returns the learning phase flag.

The learning phase flag is a bool tensor (0 = test, 1 = train)to be passed as input to any Keras functionthat uses a different behavior at train time and test time.

Returns

Learning phase (scalar integer tensor or Python integer).

set_learning_phase

keras.backend.set_learning_phase(value)

Sets the learning phase to a fixed value.

Arguments

• value: Learning phase value, either 0 or 1 (integers).

Raises

• ValueError: if value is neither 0 nor 1.

clear_session

keras.backend.clear_session()

Destroys the current Keras graph and creates a new one.

Useful to avoid clutter from old models / layers.

is_sparse

keras.backend.is_sparse(tensor)

Returns whether a tensor is a sparse tensor.

Arguments

• tensor: A tensor instance.

Returns

A boolean.

Example

>>> from keras import backend as K
>>> a = K.placeholder((2, 2), sparse=False)
>>> print(K.is_sparse(a))
False
>>> b = K.placeholder((2, 2), sparse=True)
>>> print(K.is_sparse(b))
True

to_dense

keras.backend.to_dense(tensor)

Converts a sparse tensor into a dense tensor and returns it.

Arguments

• tensor: A tensor instance (potentially sparse).

Returns

A dense tensor.

Examples

>>> from keras import backend as K
>>> b = K.placeholder((2, 2), sparse=True)
>>> print(K.is_sparse(b))
True
>>> c = K.to_dense(b)
>>> print(K.is_sparse(c))
False

variable

keras.backend.variable(value, dtype=None, name=None, constraint=None)

Instantiates a variable and returns it.

Arguments

• value: Numpy array, initial value of the tensor.
• dtype: Tensor type.
• name: Optional name string for the tensor.
• constraint: Optional projection function to be applied to the variable after an optimizer update.

Returns

A variable instance (with Keras metadata included).

Examples

>>> from keras import backend as K
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val, dtype='float64', name='example_var')
>>> K.dtype(kvar)
'float64'
>>> print(kvar)
example_var
>>> K.eval(kvar)
array([[ 1.,  2.],
       [ 3.,  4.]])

is_variable

keras.backend.is_variable(x)

constant

keras.backend.constant(value, dtype=None, shape=None, name=None)

Creates a constant tensor.

Arguments

• value: A constant value (or list)
• dtype: The type of the elements of the resulting tensor.
• shape: Optional dimensions of resulting tensor.
• name: Optional name for the tensor.

Returns

A Constant Tensor.

is_keras_tensor

keras.backend.is_keras_tensor(x)

Returns whether x is a Keras tensor.

A "Keras tensor" is a tensor that was returned by a Keras layer,(Layer class) or by Input.

Arguments

• x: A candidate tensor.

Returns

A boolean: Whether the argument is a Keras tensor.

Raises

• ValueError: In case x is not a symbolic tensor.

Examples

>>> from keras import backend as K
>>> from keras.layers import Input, Dense
>>> np_var = numpy.array([1, 2])
>>> K.is_keras_tensor(np_var) # A numpy array is not a symbolic tensor.
ValueError
>>> k_var = tf.placeholder('float32', shape=(1,1))
>>> # A variable indirectly created outside of keras is not a Keras tensor.
>>> K.is_keras_tensor(k_var)
False
>>> keras_var = K.variable(np_var)
>>> # A variable created with the keras backend is not a Keras tensor.
>>> K.is_keras_tensor(keras_var)
False
>>> keras_placeholder = K.placeholder(shape=(2, 4, 5))
>>> # A placeholder is not a Keras tensor.
>>> K.is_keras_tensor(keras_placeholder)
False
>>> keras_input = Input([10])
>>> K.is_keras_tensor(keras_input) # An Input is a Keras tensor.
True
>>> keras_layer_output = Dense(10)(keras_input)
>>> # Any Keras layer output is a Keras tensor.
>>> K.is_keras_tensor(keras_layer_output)
True

is_tensor

keras.backend.is_tensor(x)

placeholder

keras.backend.placeholder(shape=None, ndim=None, dtype=None, sparse=False, name=None)

Instantiates a placeholder tensor and returns it.

Arguments

• shape: Shape of the placeholder (integer tuple, may include None entries).
• ndim: Number of axes of the tensor. At least one of {shape, ndim} must be specified. If both are specified, shape is used.
• dtype: Placeholder type.
• sparse: Boolean, whether the placeholder should have a sparse type.
• name: Optional name string for the placeholder.

Returns

Tensor instance (with Keras metadata included).

Examples

>>> from keras import backend as K
>>> input_ph = K.placeholder(shape=(2, 4, 5))
>>> input_ph._keras_shape
(2, 4, 5)
>>> input_ph
<tf.Tensor 'Placeholder_4:0' shape=(2, 4, 5) dtype=float32>

is_placeholder

keras.backend.is_placeholder(x)

Returns whether x is a placeholder.

Arguments

• x: A candidate placeholder.

Returns

Boolean.

shape

keras.backend.shape(x)

Returns the symbolic shape of a tensor or variable.

Arguments

• x: A tensor or variable.

Returns

A symbolic shape (which is itself a tensor).

Examples

# TensorFlow example
>>> from keras import backend as K
>>> tf_session = K.get_session()
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> inputs = keras.backend.placeholder(shape=(2, 4, 5))
>>> K.shape(kvar)
<tf.Tensor 'Shape_8:0' shape=(2,) dtype=int32>
>>> K.shape(inputs)
<tf.Tensor 'Shape_9:0' shape=(3,) dtype=int32>
# To get integer shape (Instead, you can use K.int_shape(x))
>>> K.shape(kvar).eval(session=tf_session)
array([2, 2], dtype=int32)
>>> K.shape(inputs).eval(session=tf_session)
array([2, 4, 5], dtype=int32)

int_shape

keras.backend.int_shape(x)

Returns the shape of tensor or variable as a tuple of int or None entries.

Arguments

• x: Tensor or variable.

Returns

A tuple of integers (or None entries).

Examples

>>> from keras import backend as K
>>> inputs = K.placeholder(shape=(2, 4, 5))
>>> K.int_shape(inputs)
(2, 4, 5)
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> K.int_shape(kvar)
(2, 2)

Numpy implementation

def int_shape(x):
    return x.shape

ndim

keras.backend.ndim(x)

Returns the number of axes in a tensor, as an integer.

Arguments

• x: Tensor or variable.

Returns

Integer (scalar), number of axes.

Examples

>>> from keras import backend as K
>>> inputs = K.placeholder(shape=(2, 4, 5))
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> K.ndim(inputs)
3
>>> K.ndim(kvar)
2

Numpy implementation

def ndim(x):
    return x.ndim

size

keras.backend.size(x, name=None)

Returns the size of a tensor.

Arguments

• x: Tensor or variable.

Returns

Size of the tensor.

Examples

>>> from keras import backend as K
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> K.size(inputs)
<tf.Tensor: id=9, shape=(), dtype=int32, numpy=4>

dtype

keras.backend.dtype(x)

Returns the dtype of a Keras tensor or variable, as a string.

Arguments

• x: Tensor or variable.

Returns

String, dtype of x.

Examples

>>> from keras import backend as K
>>> K.dtype(K.placeholder(shape=(2,4,5)))
'float32'
>>> K.dtype(K.placeholder(shape=(2,4,5), dtype='float32'))
'float32'
>>> K.dtype(K.placeholder(shape=(2,4,5), dtype='float64'))
'float64'
# Keras variable
>>> kvar = K.variable(np.array([[1, 2], [3, 4]]))
>>> K.dtype(kvar)
'float32_ref'
>>> kvar = K.variable(np.array([[1, 2], [3, 4]]), dtype='float32')
>>> K.dtype(kvar)
'float32_ref'

Numpy implementation

def dtype(x):
    return x.dtype.name

eval

keras.backend.eval(x)

Evaluates the value of a tensor.

Arguments

• x: A tensor.

Returns

A Numpy array.

Examples

>>> from keras import backend as K
>>> kvar = K.variable(np.array([[1, 2], [3, 4]]), dtype='float32')
>>> K.eval(kvar)
array([[ 1.,  2.],
       [ 3.,  4.]], dtype=float32)

Numpy implementation

def eval(x):
    return x

zeros

keras.backend.zeros(shape, dtype=None, name=None)

Instantiates an all-zeros variable and returns it.

Arguments

• shape: Tuple of integers, shape of returned Keras variable
• dtype: String, data type of returned Keras variable
• name: String, name of returned Keras variable

Returns

A variable (including Keras metadata), filled with 0.0.Note that if shape was symbolic, we cannot return a variable,and will return a dynamically-shaped tensor instead.

Example

>>> from keras import backend as K
>>> kvar = K.zeros((3,4))
>>> K.eval(kvar)
array([[ 0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.]], dtype=float32)

Numpy implementation

def zeros(shape, dtype=floatx(), name=None):
    return np.zeros(shape, dtype=dtype)

ones

keras.backend.ones(shape, dtype=None, name=None)

Instantiates an all-ones variable and returns it.

Arguments

• shape: Tuple of integers, shape of returned Keras variable.
• dtype: String, data type of returned Keras variable.
• name: String, name of returned Keras variable.

Returns

A Keras variable, filled with 1.0.Note that if shape was symbolic, we cannot return a variable,and will return a dynamically-shaped tensor instead.

Example

>>> from keras import backend as K
>>> kvar = K.ones((3,4))
>>> K.eval(kvar)
array([[ 1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.]], dtype=float32)

Numpy implementation

def ones(shape, dtype=floatx(), name=None):
    return np.ones(shape, dtype=dtype)

eye

keras.backend.eye(size, dtype=None, name=None)

Instantiate an identity matrix and returns it.

Arguments

• size: Tuple, number of rows and columns. If Integer, number of rows.
• dtype: String, data type of returned Keras variable.
• name: String, name of returned Keras variable.

Returns

A Keras variable, an identity matrix.

Example

>>> from keras import backend as K
>>> K.eval(K.eye(3))
array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]], dtype=float32)
>>> K.eval(K.eye((2, 3)))
array([[1., 0., 0.],
       [0., 1., 0.]], dtype=float32)

Numpy implementation

def eye(size, dtype=None, name=None):
    if isinstance(size, (list, tuple)):
        n, m = size
    else:
        n, m = size, size
    return np.eye(n, m, dtype=dtype)

zeros_like

keras.backend.zeros_like(x, dtype=None, name=None)

Instantiates an all-zeros variable of the same shape as another tensor.

Arguments

• x: Keras variable or Keras tensor.
• dtype: String, dtype of returned Keras variable. None uses the dtype of x.
• name: String, name for the variable to create.

Returns

A Keras variable with the shape of x filled with zeros.

Example

>>> from keras import backend as K
>>> kvar = K.variable(np.random.random((2,3)))
>>> kvar_zeros = K.zeros_like(kvar)
>>> K.eval(kvar_zeros)
array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.]], dtype=float32)

Numpy implementation

def zeros_like(x, dtype=floatx(), name=None):
    return np.zeros_like(x, dtype=dtype)

ones_like

keras.backend.ones_like(x, dtype=None, name=None)

Instantiates an all-ones variable of the same shape as another tensor.

Arguments

• x: Keras variable or tensor.
• dtype: String, dtype of returned Keras variable. None uses the dtype of x.
• name: String, name for the variable to create.

Returns

A Keras variable with the shape of x filled with ones.

Example

>>> from keras import backend as K
>>> kvar = K.variable(np.random.random((2,3)))
>>> kvar_ones = K.ones_like(kvar)
>>> K.eval(kvar_ones)
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)

Numpy implementation

def ones_like(x, dtype=floatx(), name=None):
    return np.ones_like(x, dtype=dtype)

identity

keras.backend.identity(x, name=None)

Returns a tensor with the same content as the input tensor.

Arguments

• x: The input tensor.
• name: String, name for the variable to create.

Returns

A tensor of the same shape, type and content.

random_uniform_variable

keras.backend.random_uniform_variable(shape, low, high, dtype=None, name=None, seed=None)

Instantiates a variable with values drawn from a uniform distribution.

Arguments

• shape: Tuple of integers, shape of returned Keras variable.
• low: Float, lower boundary of the output interval.
• high: Float, upper boundary of the output interval.
• dtype: String, dtype of returned Keras variable.
• name: String, name of returned Keras variable.
• seed: Integer, random seed.

Returns

A Keras variable, filled with drawn samples.

Example

>>> kvar = K.random_uniform_variable((2,3), 0, 1)
>>> kvar
<tensorflow.python.ops.variables.Variable object at 0x10ab40b10>
>>> K.eval(kvar)
array([[ 0.10940075,  0.10047495,  0.476143  ],
       [ 0.66137183,  0.00869417,  0.89220798]], dtype=float32)

Numpy implementation

def random_uniform_variable(shape, low, high, dtype=None, name=None, seed=None):
    return (high - low) * np.random.random(shape).astype(dtype) + low

random_normal_variable

keras.backend.random_normal_variable(shape, mean, scale, dtype=None, name=None, seed=None)

Instantiates a variable with values drawn from a normal distribution.

Arguments

• shape: Tuple of integers, shape of returned Keras variable.
• mean: Float, mean of the normal distribution.
• scale: Float, standard deviation of the normal distribution.
• dtype: String, dtype of returned Keras variable.
• name: String, name of returned Keras variable.
• seed: Integer, random seed.

Returns

A Keras variable, filled with drawn samples.

Example

# TensorFlow example
>>> kvar = K.random_normal_variable((2,3), 0, 1)
>>> kvar
<tensorflow.python.ops.variables.Variable object at 0x10ab12dd0>
>>> K.eval(kvar)
array([[ 1.19591331,  0.68685907, -0.63814116],
       [ 0.92629528,  0.28055015,  1.70484698]], dtype=float32)

Numpy implementation

def random_normal_variable(shape, mean, scale, dtype=None, name=None, seed=None):
    return scale * np.random.randn(*shape).astype(dtype) + mean

count_params

Returns the static number of elements in a Keras variable or tensor.

Arguments

• x: Keras variable or tensor.

Returns

Integer, the number of elements in x, i.e., the product of thearray's static dimensions.

Example

>>> kvar = K.zeros((2,3))
>>> K.count_params(kvar)
6
>>> K.eval(kvar)
array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.]], dtype=float32)

Numpy implementation

def count_params(x):
    return x.size

cast

keras.backend.cast(x, dtype)

Casts a tensor to a different dtype and returns it.

You can cast a Keras variable but it still returns a Keras tensor.

Arguments

• x: Keras tensor (or variable).
• dtype: String, either ('float16', 'float32', or 'float64').

Returns

Keras tensor with dtype dtype.

Example

>>> from keras import backend as K
>>> input = K.placeholder((2, 3), dtype='float32')
>>> input
<tf.Tensor 'Placeholder_2:0' shape=(2, 3) dtype=float32>
# It doesn't work in-place as below.
>>> K.cast(input, dtype='float16')
<tf.Tensor 'Cast_1:0' shape=(2, 3) dtype=float16>
>>> input
<tf.Tensor 'Placeholder_2:0' shape=(2, 3) dtype=float32>
# you need to assign it.
>>> input = K.cast(input, dtype='float16')
>>> input
<tf.Tensor 'Cast_2:0' shape=(2, 3) dtype=float16>

update

keras.backend.update(x, new_x)

Update the value of x to new_x.

Arguments

• x: A Variable.
• new_x: A tensor of same shape as x.

Returns

The variable x updated.

update_add

keras.backend.update_add(x, increment)

Update the value of x by adding increment.

Arguments

• x: A Variable.
• increment: A tensor of same shape as x.

Returns

The variable x updated.

update_sub

keras.backend.update_sub(x, decrement)

Update the value of x by subtracting decrement.

Arguments

• x: A Variable.
• decrement: A tensor of same shape as x.

Returns

The variable x updated.

moving_average_update

keras.backend.moving_average_update(x, value, momentum)

Compute the moving average of a variable.

Arguments

• x: A Variable.
• value: A tensor with the same shape as x.
• momentum: The moving average momentum.

Returns

An operation to update the variable.

dot

keras.backend.dot(x, y)

Multiplies 2 tensors (and/or variables) and returns a tensor.

When attempting to multiply a nD tensorwith a nD tensor, it reproduces the Theano behavior.(e.g. (2, 3) * (4, 3, 5) -> (2, 4, 5))

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A tensor, dot product of x and y.

Examples

# dot product between tensors
>>> x = K.placeholder(shape=(2, 3))
>>> y = K.placeholder(shape=(3, 4))
>>> xy = K.dot(x, y)
>>> xy
<tf.Tensor 'MatMul_9:0' shape=(2, 4) dtype=float32>

# dot product between tensors
>>> x = K.placeholder(shape=(32, 28, 3))
>>> y = K.placeholder(shape=(3, 4))
>>> xy = K.dot(x, y)
>>> xy
<tf.Tensor 'MatMul_9:0' shape=(32, 28, 4) dtype=float32>

# Theano-like behavior example
>>> x = K.random_uniform_variable(shape=(2, 3), low=0, high=1)
>>> y = K.ones((4, 3, 5))
>>> xy = K.dot(x, y)
>>> K.int_shape(xy)
(2, 4, 5)

Numpy implementation

def dot(x, y):
    return np.dot(x, y)

batch_dot

keras.backend.batch_dot(x, y, axes=None)

Batchwise dot product.

batch_dot is used to compute dot product of x and y whenx and y are data in batches, i.e. in a shape of(batch_size, :).batch_dot results in a tensor or variable with less dimensionsthan the input. If the number of dimensions is reduced to 1,we use expand_dims to make sure that ndim is at least 2.

Arguments

• x: Keras tensor or variable with ndim >= 2.
• y: Keras tensor or variable with ndim >= 2.
• axes: int or tuple(int, int). Target dimensions to be reduced.

Returns

A tensor with shape equal to the concatenation of x's shape(less the dimension that was summed over) and y's shape(less the batch dimension and the dimension that was summed over).If the final rank is 1, we reshape it to (batch_size, 1).

Examples

Assume x = [[1, 2], [3, 4]] and y = [[5, 6], [7, 8]]batch_dot(x, y, axes=1) = [[17], [53]] which is the main diagonalof x.dot(y.T), although we never have to calculate the off-diagonalelements.

Pseudocode:

inner_products = []
for xi, yi in zip(x, y):
    inner_products.append(xi.dot(yi))
result = stack(inner_products)

Shape inference:Let x's shape be (100, 20) and y's shape be (100, 30, 20).If axes is (1, 2), to find the output shape of resultant tensor,loop through each dimension in x's shape and y's shape:

• x.shape[0] : 100 : append to output shape
• x.shape[1] : 20 : do not append to output shape,dimension 1 of x has been summed over. (dot_axes[0] = 1)
• y.shape[0] : 100 : do not append to output shape,always ignore first dimension of y
• y.shape[1] : 30 : append to output shape
• y.shape[2] : 20 : do not append to output shape,dimension 2 of y has been summed over. (dot_axes[1] = 2)output_shape = (100, 30)

>>> x_batch = K.ones(shape=(32, 20, 1))
>>> y_batch = K.ones(shape=(32, 30, 20))
>>> xy_batch_dot = K.batch_dot(x_batch, y_batch, axes=(1, 2))
>>> K.int_shape(xy_batch_dot)
(32, 1, 30)

Numpy implementation

Show the Numpy implementationwzxhzdk:102

transpose

keras.backend.transpose(x)

Transposes a tensor and returns it.

Arguments

• x: Tensor or variable.

Returns

A tensor.

Examples

>>> var = K.variable([[1, 2, 3], [4, 5, 6]])
>>> K.eval(var)
array([[ 1.,  2.,  3.],
       [ 4.,  5.,  6.]], dtype=float32)
>>> var_transposed = K.transpose(var)
>>> K.eval(var_transposed)
array([[ 1.,  4.],
       [ 2.,  5.],
       [ 3.,  6.]], dtype=float32)

>>> inputs = K.placeholder((2, 3))
>>> inputs
<tf.Tensor 'Placeholder_11:0' shape=(2, 3) dtype=float32>
>>> input_transposed = K.transpose(inputs)
>>> input_transposed
<tf.Tensor 'transpose_4:0' shape=(3, 2) dtype=float32>

Numpy implementation

def transpose(x):
    return np.transpose(x)

gather

keras.backend.gather(reference, indices)

Retrieves the elements of indices indices in the tensor reference.

Arguments

• reference: A tensor.
• indices: An integer tensor of indices.

Returns

A tensor of same type as reference.

def gather(reference, indices):
    return reference[indices]

max

keras.backend.max(x, axis=None, keepdims=False)

Maximum value in a tensor.

Arguments

• x: A tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to find maximum values. If None (default), finds the maximum over all dimensions.
• keepdims: A boolean, whether to keep the dimensions or not. If keepdims is False, the rank of the tensor is reduced by 1. If keepdims is True, the reduced dimension is retained with length 1.

Returns

A tensor with maximum values of x.

Numpy implementation

def max(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.max(x, axis=axis, keepdims=keepdims)

min

keras.backend.min(x, axis=None, keepdims=False)

Minimum value in a tensor.

Arguments

• x: A tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to find minimum values. If None (default), finds the minimum over all dimensions.
• keepdims: A boolean, whether to keep the dimensions or not. If keepdims is False, the rank of the tensor is reduced by 1. If keepdims is True, the reduced dimension is retained with length 1.

Returns

A tensor with miminum values of x.

Numpy implementation

def min(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.min(x, axis=axis, keepdims=keepdims)

keras.backend.sum(x, axis=None, keepdims=False)

Sum of the values in a tensor, alongside the specified axis.

Arguments

• x: A tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to sum over. If None (default), sums over all dimensions.
• keepdims: A boolean, whether to keep the dimensions or not. If keepdims is False, the rank of the tensor is reduced by 1. If keepdims is True, the reduced dimension is retained with length 1.

Returns

A tensor with sum of x.

Numpy implementation

def sum(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.sum(x, axis=axis, keepdims=keepdims)

prod

keras.backend.prod(x, axis=None, keepdims=False)

Multiplies the values in a tensor, alongside the specified axis.

Arguments

• x: A tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to compute the product. If None (default), computes the product over all dimensions.
• keepdims: A boolean, whether to keep the dimensions or not. If keepdims is False, the rank of the tensor is reduced by 1. If keepdims is True, the reduced dimension is retained with length 1.

Returns

A tensor with the product of elements of x.

Numpy implementation

def prod(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.prod(x, axis=axis, keepdims=keepdims)

cumsum

keras.backend.cumsum(x, axis=0)

Cumulative sum of the values in a tensor, alongside the specified axis.

Arguments

• x: A tensor or variable.
• axis: An integer, the axis to compute the sum.

Returns

A tensor of the cumulative sum of values of x along axis.Numpy implementation

def cumsum(x, axis=0):
    return np.cumsum(x, axis=axis)

cumprod

keras.backend.cumprod(x, axis=0)

Cumulative product of the values in a tensor, alongside the specified axis.

Arguments

• x: A tensor or variable.
• axis: An integer, the axis to compute the product.

Returns

A tensor of the cumulative product of values of x along axis.Numpy implementation

def cumprod(x, axis=0):
    return np.cumprod(x, axis=axis)

var

keras.backend.var(x, axis=None, keepdims=False)

Variance of a tensor, alongside the specified axis.

Arguments

• x: A tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to compute the variance. If None (default), computes the variance over all dimensions.
• keepdims: A boolean, whether to keep the dimensions or not. If keepdims is False, the rank of the tensor is reduced by 1. If keepdims is True, the reduced dimension is retained with length 1.

Returns

A tensor with the variance of elements of x.Numpy implementation

def var(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.var(x, axis=axis, keepdims=keepdims)

std

keras.backend.std(x, axis=None, keepdims=False)

Standard deviation of a tensor, alongside the specified axis.

Arguments

• x: A tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to compute the standard deviation. If None (default), computes the standard deviation over all dimensions.
• keepdims: A boolean, whether to keep the dimensions or not. If keepdims is False, the rank of the tensor is reduced by 1. If keepdims is True, the reduced dimension is retained with length 1.

Returns

A tensor with the standard deviation of elements of x.Numpy implementation

def std(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.std(x, axis=axis, keepdims=keepdims)

mean

keras.backend.mean(x, axis=None, keepdims=False)

Mean of a tensor, alongside the specified axis.

Arguments

• x: A tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to compute the mean. If None (default), computes the mean over all dimensions.
• keepdims: A boolean, whether to keep the dimensions or not. If keepdims is False, the rank of the tensor is reduced by 1 for each entry in axis. If keepdims is True, the reduced dimensions are retained with length 1.

Returns

A tensor with the mean of elements of x.Numpy implementation

def mean(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.mean(x, axis=axis, keepdims=keepdims)

any

keras.backend.any(x, axis=None, keepdims=False)

Bitwise reduction (logical OR).

Arguments

• x: Tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to compute the logical or. If None (default), computes the logical or over all dimensions.
• keepdims: whether the drop or broadcast the reduction axes.

Returns

A uint8 tensor (0s and 1s).Numpy implementation

def any(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.any(x, axis=axis, keepdims=keepdims)

all

keras.backend.all(x, axis=None, keepdims=False)

Bitwise reduction (logical AND).

Arguments

• x: Tensor or variable.
• axis: An integer or list of integers in [-rank(x), rank(x)), the axes to compute the logical and. If None (default), computes the logical and over all dimensions.
• keepdims: whether the drop or broadcast the reduction axes.

Returns

A uint8 tensor (0s and 1s).Numpy implementation

def all(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.all(x, axis=axis, keepdims=keepdims)

argmax

keras.backend.argmax(x, axis=-1)

Returns the index of the maximum value along an axis.

Arguments

• x: Tensor or variable.
• axis: axis along which to perform the reduction.

Returns

A tensor.Numpy implementation

def argmax(x, axis=-1):
    return np.argmax(x, axis=axis)

argmin

keras.backend.argmin(x, axis=-1)

Returns the index of the minimum value along an axis.

Arguments

• x: Tensor or variable.
• axis: axis along which to perform the reduction.

Returns

A tensor.Numpy implementation

def argmin(x, axis=-1):
    return np.argmin(x, axis=axis)

square

keras.backend.square(x)

Element-wise square.

Arguments

• x: Tensor or variable.

Returns

A tensor.

abs

keras.backend.abs(x)

Element-wise absolute value.

Arguments

• x: Tensor or variable.

Returns

A tensor.

sqrt

keras.backend.sqrt(x)

Element-wise square root.

Arguments

• x: Tensor or variable.

Returns

A tensor.Numpy implementation

def sqrt(x):
    y = np.sqrt(x)
    y[np.isnan(y)] = 0.
    return y

exp

keras.backend.exp(x)

Element-wise exponential.

Arguments

• x: Tensor or variable.

Returns

A tensor.

log

keras.backend.log(x)

Element-wise log.

Arguments

• x: Tensor or variable.

Returns

A tensor.

logsumexp

keras.backend.logsumexp(x, axis=None, keepdims=False)

Computes log(sum(exp(elements across dimensions of a tensor))).

This function is more numerically stable than log(sum(exp(x))).It avoids overflows caused by taking the exp of large inputs andunderflows caused by taking the log of small inputs.

Arguments

• x: A tensor or variable.
• axis: axis: An integer or list of integers in [-rank(x), rank(x)), the axes to compute the logsumexp. If None (default), computes the logsumexp over all dimensions.
• keepdims: A boolean, whether to keep the dimensions or not. If keepdims is False, the rank of the tensor is reduced by 1. If keepdims is True, the reduced dimension is retained with length 1.

Returns

The reduced tensor.Numpy implementation

def logsumexp(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return sp.special.logsumexp(x, axis=axis, keepdims=keepdims)

round

keras.backend.round(x)

Element-wise rounding to the closest integer.

In case of tie, the rounding mode used is "half to even".

Arguments

• x: Tensor or variable.

Returns

A tensor.

sign

keras.backend.sign(x)

Element-wise sign.

Arguments

• x: Tensor or variable.

Returns

A tensor.

pow

keras.backend.pow(x, a)

Element-wise exponentiation.

Arguments

• x: Tensor or variable.
• a: Python integer.

Returns

A tensor.Numpy implementation

def pow(x, a=1.):
    return np.power(x, a)

clip

keras.backend.clip(x, min_value, max_value)

Element-wise value clipping.

Arguments

• x: Tensor or variable.
• min_value: Python float, integer or tensor.
• max_value: Python float, integer or tensor.

Returns

A tensor.Numpy implementation

def clip(x, min_value, max_value):
    return np.clip(x, min_value, max_value)

equal

keras.backend.equal(x, y)

Element-wise equality between two tensors.

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A bool tensor.

Numpy implementation

def equal(x, y):
    return x == y

not_equal

keras.backend.not_equal(x, y)

Element-wise inequality between two tensors.

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A bool tensor.

Numpy implementation

def not_equal(x, y):
    return x != y

greater

keras.backend.greater(x, y)

Element-wise truth value of (x > y).

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A bool tensor.

Numpy implementation

def greater(x, y):
    return x > y

greater_equal

keras.backend.greater_equal(x, y)

Element-wise truth value of (x >= y).

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A bool tensor.

Numpy implementation

def greater_equal(x, y):
    return x >= y

less

keras.backend.less(x, y)

Element-wise truth value of (x < y).

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A bool tensor.

Numpy implementation

def less(x, y):
    return x < y

less_equal

keras.backend.less_equal(x, y)

Element-wise truth value of (x <= y).

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A bool tensor.

Numpy implementation

def less_equal(x, y):
    return x <= y

maximum

keras.backend.maximum(x, y)

Element-wise maximum of two tensors.

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A tensor.

Numpy implementation

def maximum(x, y):

minimum

keras.backend.minimum(x, y)

Element-wise minimum of two tensors.

Arguments

• x: Tensor or variable.
• y: Tensor or variable.

Returns

A tensor.

Numpy implementation

def minimum(x, y):
    return np.minimum(x, y)

sin

keras.backend.sin(x)

Computes sin of x element-wise.

Arguments

• x: Tensor or variable.

Returns

A tensor.

cos

keras.backend.cos(x)

Computes cos of x element-wise.

Arguments

• x: Tensor or variable.

Returns

A tensor.

normalize_batch_in_training

keras.backend.normalize_batch_in_training(x, gamma, beta, reduction_axes, epsilon=0.001)

Computes mean and std for batch then apply batch_normalization on batch.

Arguments

• x: Input tensor or variable.
• gamma: Tensor by which to scale the input.
• beta: Tensor with which to center the input.
• reduction_axes: iterable of integers, axes over which to normalize.
• epsilon: Fuzz factor.

Returns

A tuple length of 3, (normalized_tensor, mean, variance).

batch_normalization

keras.backend.batch_normalization(x, mean, var, beta, gamma, axis=-1, epsilon=0.001)

Applies batch normalization on x given mean, var, beta and gamma.

I.e. returns:output = (x - mean) / sqrt(var + epsilon) * gamma + beta

Arguments

• x: Input tensor or variable.
• mean: Mean of batch.
• var: Variance of batch.
• beta: Tensor with which to center the input.
• gamma: Tensor by which to scale the input.
• axis: Integer, the axis that should be normalized. (typically the features axis).
• epsilon: Fuzz factor.

Returns

A tensor.

Numpy implementation

def batch_normalization(x, mean, var, beta, gamma, axis=-1, epsilon=0.001):
    return ((x - mean) / sqrt(var + epsilon)) * gamma + beta

concatenate

keras.backend.concatenate(tensors, axis=-1)

Concatenates a list of tensors alongside the specified axis.

Arguments

• tensors: list of tensors to concatenate.
• axis: concatenation axis.

Returns

A tensor.

reshape

Reshapes a tensor to the specified shape.

Arguments

• x: Tensor or variable.
• shape: Target shape tuple.

Returns

A tensor.

permute_dimensions

keras.backend.permute_dimensions(x, pattern)

Permutes axes in a tensor.

Arguments

• x: Tensor or variable.
• pattern: A tuple of dimension indices, e.g. (0, 2, 1).

Returns

A tensor.

resize_images

keras.backend.resize_images(x, height_factor, width_factor, data_format, interpolation='nearest')

Resizes the images contained in a 4D tensor.

Arguments

• x: Tensor or variable to resize.
• height_factor: Positive integer.
• width_factor: Positive integer.
• data_format: string, "channels_last" or "channels_first".
• interpolation: A string, one of nearest or bilinear.

Returns

A tensor.

Raises

• ValueError: if data_format is

neither "channels_last" or "channels_first".

resize_volumes

keras.backend.resize_volumes(x, depth_factor, height_factor, width_factor, data_format)

Resizes the volume contained in a 5D tensor.

Arguments

• x: Tensor or variable to resize.
• depth_factor: Positive integer.
• height_factor: Positive integer.
• width_factor: Positive integer.
• data_format: string, "channels_last" or "channels_first".

Returns

A tensor.

Raises

• ValueError: if data_format is

neither "channels_last" or "channels_first".

repeat_elements

keras.backend.repeat_elements(x, rep, axis)

Repeats the elements of a tensor along an axis, like np.repeat.

If x has shape (s1, s2, s3) and axis is 1, the outputwill have shape (s1, s2 * rep, s3).

Arguments

• x: Tensor or variable.
• rep: Python integer, number of times to repeat.
• axis: Axis along which to repeat.

Returns

A tensor.

repeat

Repeats a 2D tensor.

if x has shape (samples, dim) and n is 2,the output will have shape (samples, 2, dim).

Arguments

• x: Tensor or variable.
• n: Python integer, number of times to repeat.

Returns

A tensor.

arange

keras.backend.arange(start, stop=None, step=1, dtype='int32')

Creates a 1D tensor containing a sequence of integers.

The function arguments use the same convention asTheano's arange: if only one argument is provided,it is in fact the "stop" argument and "start" is 0.

The default type of the returned tensor is 'int32' tomatch TensorFlow's default.

Arguments

• start: Start value.
• stop: Stop value.
• step: Difference between two successive values.
• dtype: Integer dtype to use.

Returns

An integer tensor.

tile

keras.backend.tile(x, n)

Creates a tensor by tiling x by n.

Arguments

• x: A tensor or variable
• n: A list of integer. The length must be the same as the number of dimensions in x.

Returns

A tiled tensor.

Example

>>> from keras import backend as K
>>> kvar = K.variable(np.random.random((2, 3)))
>>> kvar_tile = K.tile(K.eye(2), (2, 3))
>>> K.eval(kvar_tile)
array([[1., 0., 1., 0., 1., 0.],
       [0., 1., 0., 1., 0., 1.],
       [1., 0., 1., 0., 1., 0.],
       [0., 1., 0., 1., 0., 1.]], dtype=float32)

Numpy implementation

def tile(x, n):
    return np.tile(x, n)

flatten

keras.backend.flatten(x)

Flatten a tensor.

Arguments

• x: A tensor or variable.

Returns

A tensor, reshaped into 1-D

batch_flatten

keras.backend.batch_flatten(x)

Turn a nD tensor into a 2D tensor with same 0th dimension.

In other words, it flattens each data samples of a batch.

Arguments

• x: A tensor or variable.

Returns

A tensor.

expand_dims

keras.backend.expand_dims(x, axis=-1)

Adds a 1-sized dimension at index "axis".

Arguments

• x: A tensor or variable.
• axis: Position where to add a new axis.

Returns

A tensor with expanded dimensions.

squeeze

keras.backend.squeeze(x, axis)

Removes a 1-dimension from the tensor at index "axis".

Arguments

• x: A tensor or variable.
• axis: Axis to drop.

Returns

A tensor with the same data as x but reduced dimensions.

temporal_padding

keras.backend.temporal_padding(x, padding=(1, 1))

Pads the middle dimension of a 3D tensor.

Arguments

• x: Tensor or variable.
• padding: Tuple of 2 integers, how many zeros to add at the start and end of dim 1.

Returns

A padded 3D tensor.

spatial_2d_padding

keras.backend.spatial_2d_padding(x, padding=((1, 1), (1, 1)), data_format=None)

Pads the 2nd and 3rd dimensions of a 4D tensor.

Arguments

• x: Tensor or variable.
• padding: Tuple of 2 tuples, padding pattern.
• data_format: string, "channels_last" or "channels_first".

Returns

A padded 4D tensor.

Raises

• ValueError: if data_format is

neither "channels_last" or "channels_first".

spatial_3d_padding

keras.backend.spatial_3d_padding(x, padding=((1, 1), (1, 1), (1, 1)), data_format=None)

Pads 5D tensor with zeros along the depth, height, width dimensions.

Pads these dimensions with respectively"padding[0]", "padding[1]" and "padding[2]" zeros left and right.

For 'channels_last' data_format,the 2nd, 3rd and 4th dimension will be padded.For 'channels_first' data_format,the 3rd, 4th and 5th dimension will be padded.

Arguments

• x: Tensor or variable.
• padding: Tuple of 3 tuples, padding pattern.
• data_format: string, "channels_last" or "channels_first".

Returns

A padded 5D tensor.

Raises

• ValueError: if data_format is