# Tensor¶

Each Tensor instance is a multi-dimensional array allocated on a specific Device instance. Tensor instances store variables and provide linear algebra operations over different types of hardware devices without user awareness. Note that users need to make sure the tensor operands are allocated on the same device except copy functions.

## Tensor implementation¶

SINGA has three different sets of implmentations of Tensor functions, one for each type of Device.

• ‘tensor_math_cpp.h’ implements operations using Cpp (with CBLAS) for CppGPU devices.

• ‘tensor_math_cuda.h’ implements operations using Cuda (with cuBLAS) for CudaGPU devices.

• ‘tensor_math_opencl.h’ implements operations using OpenCL for OpenclGPU devices.

## Python API¶

Example usage:

import numpy as np
from singa import tensor
from singa import device

# create a tensor with shape (2,3), default CppCPU device and float32
x = tensor.Tensor((2, 3))
x.set_value(0.4)

# create a tensor from a numpy array
npy = np.zeros((3, 3), dtype=np.float32)
y = tensor.from_numpy(npy)

y.uniform(-1, 1)  # sample values from the uniform distribution

z = tensor.mult(x, y)  # gemm -> z of shape (2, 3)

x += z  # element-wise addition

dev = device.get_default_device()
x.to_device(dev)  # move the data to a gpu device

s = tensor.to_numpy(x)  # tensor -> numpy array


There are two sets of tensor functions,

Tensor member functions

which would change the internal state of the Tensor instance.

Tensor module functions

which accept Tensor instances as arguments and return Tensor instances.

Every Tesor instance must be initialized before reading data from it.

class singa.tensor.Tensor(shape=(), device=None, dtype=0, data=None, requires_grad=True, stores_grad=False, creator=None, name=None)

Python Tensor, which wraps a swig converted Tensor from CPP Tensor.

Parameters
• shape (tuple<int>) – a tuple of integers for the tensor shape. If shape is not specified, the created tensor is called a dummy tensor.

• device – a swig device. If None, the default host device is used.

• dtype – data type. currently, most operations only accept float32.

• data – a numpy array or swig tensor.

• stores_grad – boolean indicator for storing and returning the gradient. Some intermediate tensors’ gradient can be released during the backward propagation. A tensor may require grad but not store grad; But if a tensor stores grad then it must require grad.

T()

shallow copy.

Returns

a new Tensor which shares the underlying data memory (shallow copy).

add_column(v)

Parameters

v (Tensor) – a Tensor to be added as a column to this tensor.

add_row(v)

Parameters

v (Tensor) – a Tensor to be added as a row to this tensor.

as_type(dtype)

Change the data type.

Parameters

dtype – accepts ‘int’, ‘float’, ‘singa.kFloat32’, ‘singa.kInt’

Returns

new tensor with new type

bernoulli(p, inplace=True)

Sample 0/1 for each element according to the given probability.

Parameters
• p (float) – with probability p, each element is sample to 1.

• inplace – inplace flag

Returns

this tensor

clone()
Returns

a new Tensor which does deep copy of this tensor

copy()

shallow copy calls copy constructor of singa::Tensor

Returns

new tensor copied

copy_data(t)

Copy data from other Tensor instance.

Parameters

t (Tensor) – source Tensor.

copy_from_numpy(np_array, offset=0)

Copy the data from the numpy array.

Parameters
• np_array – source numpy array

• offset (int) – destination offset

deepcopy()

Same as clone().

Returns

a new Tensor

div_column(v)

(DEPRECATED, use broadcast)Divide each column of this tensor by v.

Parameters

v (Tensor) – 1d tensor of the same length the column of self.

div_row(v)

(DEPRECATED, use broadcast)Divide each row of this tensor by v.

Parameters

v (Tensor) – 1d tensor of the same length the row of self.

gaussian(mean, std, inplace=True)

Generate a value for each element following a Gaussian distribution.

Parameters
• mean (float) – mean of the distribution

• std (float) – standard variance of the distribution

• inplace – inplace flag

Returns

this tensor

is_empty()
Returns

True if the tensor is empty according to its shape

is_transpose()
Returns

True if the internal data is transposed; otherwise False.

l1()
Returns

the L1 norm.

l2()
Returns

the L2 norm.

memsize()
Returns

the number of Bytes allocated for this tensor.

mult_column(v)

(DEPRECATED, use broadcast)Multiply each column of this tensor by v element-wisely.

Parameters

v (Tensor) – 1d tensor of the same length the column of self.

mult_row(v)

(DEPRECATED, use broadcast)Multiply each row of this tensor by v element-wisely.

Parameters

v (Tensor) – 1d tensor of the same length the row of self.

ndim()
Returns

the number of dimensions of the tensor.

repeat(repeats, axis)

Repeat data of a tensor

Parameters
• repeats (int or a sequence) – the number that the tensor need to repeat for

• axis (int) – the axis to do repeat If it is None, then the repeated tensor will be flattened.If it isn’t None, the repeats could be sequence, but it’s size should match the axis’s shape

Returns

the tensor which has been repeated

reset_like(t)

Reset the shape, dtype and device as the given tensor.

Parameters

t (Tensor) – a tensor

reshape(shape)
Return a new tensor with the given shape, and the original

tensor is not changed.

Parameters

shape (list<int>) – new shape, which should have the same volumn as the original shape.

Returns

new tensor reshaped

set_value(x, inplace=True)

Set all elements of the tensor to be the give value.

Parameters
• x (float) – a float value to be set to all elements.

• inplace – inplace flag

Returns

this tensor

size()
Returns

the number of elements of the tensor.

to_device(device)

Move the tensor data onto a given device.

Parameters

device – a swig Device converted from CudaGPU or CppCPU or OpenclGPU

to_host()

Move the tensor data onto the default host CppCPU device.

transpose(axes=None)

To transpose the tensor

Parameters

axes – axes to transpose

Returns

new transposed tensor

uniform(low, high, inplace=True)

Generate a value for each element following a uniform distribution.

Parameters
• low (float) – the lower bound

• high (float) – the hight bound

• inplace – inplace flag

Returns

this tensor

singa.tensor.abs(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = abs(x), x is an element of t

singa.tensor.add(lhs, rhs, ret=None)

Parameters
• lhs (Tensor) – lhs tensor

• rhs (Tensor) – rhs tensor

• ret (Tensor, optional) – if not None, the result is stored in it; otherwise, a new Tensor would be created for the result.

Returns

the result Tensor

singa.tensor.add_column(alpha, v, beta, M)

Add v to each column of M.

Denote each column of M as m, m = alpha * v + beta * m

Parameters
• alpha (float) – scalar factor

• v (Tensor) – a tensor

• beta (float) – scalar factor

• M (Tensor) – 2d tensor

Returns

Resulted tensor M

singa.tensor.add_row(alpha, v, beta, M)

Add v to each row of M.

Denote each row of M as m, m = alpha * v + beta * m

Parameters
• alpha (float) – scaling factor

• v (Tensor) – a tensor

• beta (float) – scaling factor

• M (Tensor) – 2d tensor

Returns

Resulted tensor M

singa.tensor.average(t, axis=None)
Parameters
• t (Tensor) – input Tensor

• axis (int, optional) – if None, average all elements; otherwise average along the given dimension. 0 for averaging each column; 1 for averaging each row.

Returns

a float value if axis is None; otherwise, a new Tensor for the result.

singa.tensor.axpy(alpha, x, y)

Element-wise operation for y += alpha * x.

Parameters
• alpha (float) – scaling factor

• x (Tensor) – a tensor

• y (Tensor) – a tensor

Returns

y

singa.tensor.bernoulli(p, t)

Generate a binary value for each element of t.

Parameters
• p (float) – each element is 1 with probability p; and 0 with 1 - p

• t (Tensor) – the results are put into t

Returns

t

singa.tensor.ceil(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = ceil(x), x is an element of t

singa.tensor.concatenate(tensors, axis)

concatenate list of tensors together based on given axis

Parameters
• tensors – list of tensors.

• axis – number of axis to cancatenate on, all the dim should be the same except the axis to be concatenated.

Returns

new tensor concatenated

singa.tensor.copy_data_to_from(dst, src, size, dst_offset=0, src_offset=0)

Copy the data between two Tensor instances which could be on different devices.

Parameters
• dst (Tensor) – destination Tensor

• src (Tensor) – source Tensor

• size (int) – number of elements to copy

• dst_offset (int) – offset in terms of elements to the start of dst

• src_offset (int) – offset in terms of elements to the start of src

singa.tensor.copy_from_numpy(data, np_array)
Copy the data from the numpy array.

used as static method

Parameters
• data – singa ctensor

• np_array – source numpy array

singa.tensor.div(lhs, rhs, ret=None)

Elementi-wise division.

Parameters
• lhs (Tensor) – lhs tensor

• rhs (Tensor) – rhs tensor

• ret (Tensor, optional) – if not None, the result is stored in it; otherwise, a new Tensor would be created for the result.

Returns

the result Tensor

singa.tensor.einsum(ops, *args)

function TODO list to finish the function in cpp(just like numpy function): 1.sum(A,axis = None) 2.repeat(A,repeats) 3.transpose(A,axes = None) Do the matrix to matrix einsum calculation according to the operands Warning : this function could only support two matrix’ einsum calcultion

Parameters
• ops (string) – the string specifies the subscripts for summation such as ‘ki,kj->kij’ Here all the 26 lowercase letter can be used here.

• args (list of array_like) – These are the tensors for the operation, but here only support two tensors.

Returns

Singa.Tensor the output matirx of the einsum calculation

The best way to understand this function is to try the examples below: A_ = [0,1,2,3,4,5,6,7,8,9,10,11] A = A_.reshape(4,3) B = A_.reshape(3,4)

Here this einsum calculation is the same as normal ‘mult’ Res = einsum(‘ij,jk->ik’,A,B)

>>> [[ 20  23  26  29]
[ 56  68  80  92]
[ 92 113 134 155]
[128 158 188 218]]


A_ = [0,1,2,3,4,5,6,7,8,9,10,11] A = A_.reshape(4,3) B = A_.reshape(4,3)

Here the einsum calculation is the same as normol ‘eltwise_mult’ Res = einsum(‘ki,ki->ki’,A,B)

>>> [[  0   1   4]
[  9  16  25]
[ 36  49  64]
[ 81 100 121]]


A = [0,1,2,3,4,5,6,7,8,9,10,11] A = A.reshape(4,3)

Res = einsum(‘ki,kj->kij’,A,A) >>> [[[ 0 0 0]

[ 0 1 2] [ 0 2 4]]

[[ 9 12 15]

[ 12 16 20] [ 15 20 25]]

[[ 36 42 48]

[ 42 49 56] [ 48 56 64]]

[[ 81 90 99]

[ 90 100 110] [ 99 110 121]]]

A_ = [0,1,2,3,4,5,6,7,8,9,10,11] A = A_.reshape(3,2,2)

Res = einsum(‘kia,kja->kij’,A,A) >>> [[[ 1 3]

[ 3 13]]

[[ 41 59]

[ 59 85]]

[[145 179]

[179 221]]]

singa.tensor.eltwise_mult(lhs, rhs, ret=None)

Elementi-wise multiplication.

Parameters
• lhs (Tensor) – lhs tensor

• rhs (Tensor) – rhs tensor

• ret (Tensor, optional) – if not None, the result is stored in it; otherwise, a new Tensor would be created for the result.

Returns

the result Tensor

singa.tensor.exp(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = exp(x), x is an element of t

singa.tensor.from_numpy(np_array)

Create a Tensor instance with the shape, dtype and values from the numpy array.

Parameters

np_array – the numpy array.

Returns

A Tensor instance allocated on the default CppCPU device.

singa.tensor.gaussian(mean, std, t)

Generate values following a Gaussian distribution.

Parameters
• mean (float) – the mean of the Gaussian distribution.

• std (float) – the standard variance of the Gaussian distribution.

• t (Tensor) – the results are put into t

Returns

t

singa.tensor.ge(t, x)

Elementi-wise comparison for t >= x.

Parameters
• t (Tensor) – left hand side operand

• x (Tensor or float) – right hand side operand

Returns

0.0f, or t[i] >= x[i] ? 1.0f:0.0f

Return type

a Tensor with each element being t[i] >= x ? 1.0f

singa.tensor.gt(t, x)

Elementi-wise comparison for t > x.

Parameters
• t (Tensor) – left hand side operand

• x (Tensor or float) – right hand side operand

Returns

0.0f, or t[i] > x[i] ? 1.0f:0.0f

Return type

a Tensor with each element being t[i] > x ? 1.0f

singa.tensor.le(t, x)

Elementi-wise comparison for t <= x.

Parameters
• t (Tensor) – left hand side operand

• x (Tensor or float) – right hand side operand

Returns

0.0f, or t[i] <= x[i] ? 1.0f:0.0f

Return type

a Tensor with each element being t[i] <= x ? 1.0f

singa.tensor.log(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = log(x), x is an element of t

singa.tensor.lt(t, x)

Elementi-wise comparison for t < x

Parameters
• t (Tensor) – left hand side operand

• x (Tensor or float) – right hand side operand

Returns

0.0f, or t[i] < x[i] ? 1.0f:0.0f

Return type

a Tensor with each element being t[i] < x ? 1.0f

singa.tensor.mult(A, B, C=None, alpha=1.0, beta=0.0)

Do matrix-matrix or matrix-vector multiplication. This function returns C = alpha * A * B + beta * C Currently below cases are supported

case 1 - matrix * vector:

A (Tensor): 2d Tensor B (Tensor): 1d Tensor, GEMV would be invoked

case 2 - matrix * matrix:

A (Tensor): 2d Tensor B (Tensor): 2d Tensor, GEMM would be invoked

case 3 - batched matrix * batched matrix:

A (Tensor): 3/4d Tensor B (Tensor): 3/4d Tensor, batched GEMM would be invoked Where first/first and second dimension(s) of A, B should be exactly the same e.g. C{2,3,4,6} = A{2,3,4,5} * B{2,3,5,6}

Parameters
• A – n-d tensor

• B – n-d tensor

• C (Tensor, optional) – for storing the result; If None, a new Tensor would be created.

• alpha (float) – scaling factor

• beta (float) – scaling factor

Returns

the result Tensor

singa.tensor.pow(t, x, out=None)
Parameters
• t (Tensor) – input tensor

• x (float or Tensor) – y[i] = t[i]^x if x is a float value; otherwise, y[i]= t[i]^x[i] if x is a tensor.

• out (None or Tensor) – if None, a new Tensor would be constructed to store the result; otherwise, the result is put into out.

Returns

the result tensor.

singa.tensor.repeat(t, repeats, axis=None)

Return the repeated tensor

Parameters
• t (tensor) – the tensor to be repeated

• repeats (int or a sequence) – the number that the tensor need to repeat for

• axis (int) – the axis to do repeat If it is None, then the repeated tensor will be flattened.If it isn’t None, the repeats could be sequence, but it’s size should match the axis’s shape

Returns

the tensor which has been repeated

singa.tensor.reshape(tensor, shape)

Reshape the input tensor with the given shape and the original tensor is not changed

Parameters
• tensor (Tensor) – the tensor to be changed

• shape (list<int>) – the new shape, which should have the same volumn as the old shape.

Returns

the new Tensor

singa.tensor.sigmoid(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = sigmoid(x); x is an element of t

singa.tensor.sign(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = sign(x)

singa.tensor.sizeof(dtype)

Get size of datatype

Parameters

dtype – singa datatype

Returns

the number of bytes of the given SINGA data type defined in core.proto

singa.tensor.softmax(t, out=None)

Apply SoftMax for each row of the Tensor.

Parameters
• t (Tensor) – the input 1d or 2d tensor

• out (Tensor, optional) – if not None, it is used to store the result

Returns

the result Tensor

singa.tensor.sqrt(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = sqrt(x), x is an element of t

singa.tensor.square(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = x * x, x is an element of t

singa.tensor.sub(lhs, rhs, ret=None)

Elementi-wise subtraction.

Parameters
• lhs (Tensor) – lhs tensor

• rhs (Tensor) – rhs tensor

• ret (Tensor, optional) – if not None, the result is stored in it; otherwise, a new Tensor would be created for the result.

Returns

the result Tensor

singa.tensor.sum(t, axis=None, out=None)

Sum of tensor elements over given axis

Parameters
• t – Singa.tensor The array_like tensor to be sumed

• axis – None or int or tuple of ints, optional Axis or axes along which a sum is performed. The default, axis=None, will sum all of the elements of the input array. If axis is negative it counts from the last to the first axis. If axis is a tuple of ints, a sum is performed on all of the axes specified in the tuple instead of a single axis or all the axes as before.

• out – Singa.tensor optional Alternative output array in which to place the result. It must have the same shape as the expected output, but the type of the output values will be cast if necessary.

Returns

A tensor with the same shape as t, with the specified axis removed. If a is a 0-d array, or if axis is None, a scalar is returned. If an output array is specified, a reference to out is returned

singa.tensor.sum_columns(M)

Sum all columns into a single column.

Parameters

M (Tensor) – the input 2d tensor.

Returns

a new Tensor as the resulted column.

singa.tensor.sum_rows(M)

Sum all rows into a single row.

Parameters

M (Tensor) – the input 2d tensor.

Returns

a new Tensor as the resulted row.

singa.tensor.tanh(t)
Parameters

t (Tensor) – input Tensor

Returns

a new Tensor whose element y = tanh(x), x is an element of t

singa.tensor.tensordot(A, B, axes=2)

Returns the tensor multiplication of two tensors along specified axes.

This is equivalent to compute dot product along the specified axes which are treated as one axis by reshaping.

Parameters
• A – Singa.Tensor

• B – Singa.Tensor

• axes

• If it is an integer, then ‘’axes’’ represent axes at the last of ‘’a’’ and the first of ‘’b’’ are used.

• If it is a pair of sequences of integers, then these two sequences specify the list of axes for ‘’a’’ and ‘’b’’. The corresponding axes are paired for sum-product.

Returns

The tensor product of ‘’A’’ and ‘’B’’ along the axes specified by ‘’axes’’.

Return type

singa.tensor

Thanks to numpy.tensordot. the link is https://github.com/numpy/numpy/blob/v1.14.0/numpy/core/numeric.py#L1123-L1306

singa.tensor.to_host(t)

Copy the data to a host tensor.

Parameters

t (Tensor) – a Tensor

Returns

new Tensor at host

singa.tensor.to_numpy(t)

Copy the tensor into a numpy array.

Parameters

t (Tensor) – a Tensor

Returns

a numpy array

singa.tensor.transpose(t, axes=None)

To transpose the tensor

Parameters
• t – input tensor

• axes – axes to transpose

Returns

the transposed tensor

singa.tensor.uniform`(low, high, t)

Generate values following a Uniform distribution.

Parameters
• low (float) – the lower bound

• high (float) – the higher bound

• t (Tensor) – the results are put into t

Returns

t