5 Most Interesting Functions of PyTorch

PyTorch is defined as an open source machine learning library for Python. It is used for applications such as natural language processing. It is initially developed by Facebook artificial-intelligence research group, and Uber’s Pyro software for probabilistic programming which is built on it. There are some basics functions of PyTorch, like creating Tensors, statcking, Transpose, Taking values from Tensors and gathers values from Tensors and make new ones with working and Non-working examples. So the Functions we are going to learn today are-

Function 1 — torch.arange

Function 2 — torch.hstack

Function 3 — torch.transpose

Function 4 — torch.take

Function 5 — torch.gather

Explanation of these Function are as follows —

Function 1 — torch.arange

torch.arange(start=0, end, step=1, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

Returns a 1-D tensor with values from the interval (start, end) taken with common difference step beginning from start.

• start (Number) — the starting value for the set of points. Default: 0.
• end (Number) — the ending value for the set of points
• step (Number) — the gap between each pair of adjacent points. Default: 1.

In [4]:

# Example 1 - working
torch.arange(5,10)

Out[4]:

tensor([5, 6, 7, 8, 9])

Returns a 1-D array start with 5 and end at 9. by default it takes 1 as step value

In [5]:

# Example 2 - working
torch.arange(10,15,1.5)

Out[5]:

tensor([10.0000, 11.5000, 13.0000, 14.5000])

Returns a 1-D array start with 10 and end at 14. with given 1.5 as step value

In [6]:

# Example 3 - breaking
torch.arange(15,10,1)

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-6-60938c7178c8> in <module>() 1 # Example 3 - breaking ----> 2 torch.arange(15,10,1) RuntimeError: upper bound and larger bound inconsistent with step sign

There is error becouse it is higher to lower value count, for it step size must be taken in minus. e.g -1

This Function can be used while creating array with some step size

Function 2 — torch.hstack

torch.hstack(tensors, *, out=None) → Tensor

Stack tensors in sequence horizontally (column wise).

This is equivalent to concatenation along the first axis for 1-D tensors, and along the second axis for all other tensors.

• tensors (sequence of Tensors) — sequence of tensors to concatenate
• out (Tensor, optional) — the output tensor.

In [10]:

# Example 1 - working
a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])
torch.hstack((a,b))

Out[10]:

tensor([1, 2, 3, 4, 5, 6])

Two tensors a and b are horizontally stacked with one by one.

In [11]:

# Example 2 - working
a = torch.tensor([[1, 2],
                  [2, 5]])
b = torch.tensor([[4, 5],
                  [5, 1]])
torch.hstack((a,b))

Out[11]:

tensor([[1, 2, 4, 5],
        [2, 5, 5, 1]])

2-D array a and b horizontally stacked on same dimenssion

In [12]:

# Example 3 - breaking 
a = torch.tensor([1, 2, 3])
b = torch.tensor([[4, 5],
                  [5, 1]])
torch.hstack((a,b))

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-12-f5f4d322d8c2> in <module>() 3 b = torch.tensor([[4, 5], 4 [5, 1]]) ----> 5 torch.hstack((a,b)) RuntimeError: Tensors must have same number of dimensions: got 1 and 2

Different Dimensions tensor can’t be stacked.

This Fucntion can be use when Two tensors need to be stacked horizontally

Function 3 — torch.transpose

torch.transpose(input, dim0, dim1) → Tensor

Returns a tensor that is a transposed version of input. The given dimensions dim0 and dim1 are swapped.

The resulting out tensor shares it’s underlying storage with the input tensor, so changing the content of one would change the content of the other.

• input (Tensor) — the input tensor.
• dim0 (int) — the first dimension to be transposed
• dim1 (int) — the second dimension to be transposed

In [14]:

# Example 1 - working
# Creating random array 
x = torch.randn(3, 4)
x

Out[14]:

tensor([[ 1.1200,  1.0093, -1.0626, -1.1144],
        [ 0.0670,  1.4120, -0.6033, -0.9302],
        [-0.1642,  0.7182,  0.8911, -0.7708]])

In [15]:

# Transpose array 
torch.transpose(x, 0, 1)

Out[15]:

tensor([[ 1.1200,  0.0670, -0.1642],
        [ 1.0093,  1.4120,  0.7182],
        [-1.0626, -0.6033,  0.8911],
        [-1.1144, -0.9302, -0.7708]])

Created a Tensor of size (3,4) with some random values and then just to take its Transpose dim 0 and dim 1 is given

In [16]:

# Example 2 - working
torch.transpose(x, 1, 1)

Out[16]:

tensor([[ 1.1200,  1.0093, -1.0626, -1.1144],
        [ 0.0670,  1.4120, -0.6033, -0.9302],
        [-0.1642,  0.7182,  0.8911, -0.7708]])

We transposed dimension 1 to 1. which gave the same result as was input tensor

In [17]:

# Example 3 - breaking 
torch.transpose(x, 1, 2)

--------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-17-b2dda30a3f41> in <module>() 1 # Example 3 - breaking ----> 2 torch.transpose(x, 1, 2) IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

This error occured when we try to transpose a tensor from unexpected range value

This Function can be used when Transpose of a 2D- Tensor need to be taken

Function 4 — torch.take

torch.take(input, index) → Tensor

Returns a new tensor with the elements of input at the given indices.

• input (Tensor) — the input tensor.
• indices (LongTensor) — the indices into tensor

In [19]:

# Example 1 - working
x = torch.tensor([[6, 3, 7],
                  [3, 9, 8]])
torch.take(x, torch.tensor([0, 3, 5]))

Out[19]:

tensor([6, 3, 8])

In First Step We have created a Tensor X. and in the second step we took values using index which was 0, 3 and 5.

In [20]:

# Example 2 - working
x = torch.tensor([[6, 3, 7],
                  [3, 9, 8],
                  [4, 8, 2]])
torch.take(x, torch.tensor([0, 3, 5, 8]))

Out[20]:

tensor([6, 3, 8, 2])

In this example we have created a 3-D Tensor. and took values from index 0, 3, 5, and 8

In [21]:

# Example 3 - breaking 
x = torch.tensor([[6, 3, 7],
                  [3, 9, 8],
                  [4, 8, 2]])
torch.take(x, torch.tensor([0, 3, 5, 9]))

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-21-5dccd0c9006a> in <module>() 3 [3, 9, 8], 4 [4, 8, 2]]) ----> 5 torch.take(x, torch.tensor([0, 3, 5, 9])) RuntimeError: invalid argument 2: out of range: 9 out of 9 at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:212

In this example we got the error becouse we tried to get value from index value 9. while indexing start with 0 in the tensors, so for this Tensor index range was 0 to 8. becouse it is 3*3 matrix.

So this function can be used when we want to make new tensor from existing tensor’s values

Function 5 — torch.gather

###torch.gather(input, dim, index, *, sparse_grad=False, out=None) → Tensor Gathers values along an axis specified by dim.

• input (Tensor) — the source tensor
• dim (int) — the axis along which to index
• index (LongTensor) — the indices of elements to gather
• sparse_grad (bool,optional) — If True, gradient w.r.t. input will be a sparse tensor.
• out (Tensor, optional) — the destination tensor

In [23]:

# Example 1 - working
t = torch.tensor([[1,2],[3,4]])
torch.gather(t, 1, torch.tensor([[0,0],[1,0]]))

Out[23]:

tensor([[1, 1],
        [4, 3]])

From a 2-D tensor we have gathered values along axis 1 whith given index values

In [24]:

# Example 2 - working
t = torch.tensor([[1,2],[3,4],[5,6]])
torch.gather(t, 1, torch.tensor([[0,0,1],[1,0,0],[0,1,1]]))

Out[24]:

tensor([[1, 1, 2],
        [4, 3, 3],
        [5, 6, 6]])

Here we gathered values from a 32 tensor and created a 33 tensor

In [25]:

# Example 3 - breaking 
t = torch.tensor([[1,2],[3,4]])
torch.gather(t, 1, torch.tensor([[0,0],[1,2]]))

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-25-f87064beae61> in <module>() 1 # Example 3 - breaking 2 t = torch.tensor([[1,2],[3,4]]) ----> 3 torch.gather(t, 1, torch.tensor([[0,0],[1,2]])) RuntimeError: index 2 is out of bounds for dimension 1 with size 2

we got runtime error becouse it is 2*2 tensor and we tried to get value at index 2 from second list. as list start with index 0. so we just have 0 and 1 in this tensor.

We can use this function when we want to gethers values from a any size Tensor and want to create same or another shape Tensor

Conclusion

We have covered some must known functions in PyTorch. which can be very useful while working on Deep learning or NLP projects. for more explanation and more functions you can read the documentation of Pytorch. link is attached in the refernce.

Reference Links

Provide links to your references and other interesting articles about tensors

• Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html
• https://jovian.ai/vtyagi490/01-tensor-operations