A Beginner’s Guide to PyTorch

Image by Editor | Midjourney & Canva

Deep Learning is widely used in many areas of Artificial Intelligence research and has contributed to technological advancements. For example, text generation, facial recognition, and voice synthesis applications are based on deep learning research.

One of the most used Deep Learning packages is PyTorch. It is an open-source package created by Meta AI in 2016 and has since been used by many.
There are a lot of PyTorch advantages, including:

• Flexible model architecture
• Native Support for CUDA (Can use GPU)
• Python-based
• Providing lower-level controls, which are useful for research and many use cases
• Active development by the developer and community

Let's explore PyTorch with this article to help you get started.

Preparation

You should visit their installation webpage and select the one that suits your environment's requirements. The below code is the installation example.

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

With the PyTorch ready, let's get into the central part.

PyTorch Tensor

Tensor is the building block in PyTorch. It is similar to the NumPy array but has access to a GPU. We can try to create a PyTorch Tensor using the following code:

a = torch.tensor([2, 4, 5])  print(a)

Output>>   tensor([2, 4, 5])

Like the NumPy array Tensor, it allows matrix operations.

e = torch.tensor([[1, 2, 3],                  [4, 5, 6]])  f = torch.tensor([7, 8, 9])    print(e * f)

Output>>  tensor([[ 7, 16, 27],          [28, 40, 54]])

It’s also possible to perform the matrix multiplication.

g = torch.randn(2, 3)  h = torch.randn(3, 2)  print( g @ h)

Output>>   tensor([[-0.8357,  0.0583],          [-2.7121,  2.1980]])

We can access the Tensor information by using the code below.

x = torch.rand(3,4)    print("Shape:", x.shape)  print("Data type:", x.dtype)  print("Device:", x.device)

Output>>  Shape: torch.Size([3, 4])  Data type: torch.float32  Device: cpu

Neural Network Training with PyTorch

By defining the NN using the nn.Module class, we can develop a simple model. Let’s try it out with the code below.

import torch    class SimpleNet(nn.Module):      def __init__(self, input, hidden, output):          super(SimpleNet, self).__init__()          self.fc1 = torch.nn.Linear(input, hidden)          self.fc2 = torch.nn.Linear(hidden, output)            def forward(self, x):          x = torch.nn.functional.relu(self.fc1(x))          x = self.fc2(x)          return x    inp = 10  hid = 10  outp = 2  model = SimpleNet(inp, hid, out)    print(model)

Output>>  SimpleNet(    (fc1): Linear(in_features=10, out_features=10, bias=True)    (fc2): Linear(in_features=10, out_features=2, bias=True)  )

The above code defines a SimpleNet class that inherits from nn.Module, which sets up the layers. We use nn.Linear is for the layers, and relu as the activation function.

We can add more layers or use different layers like Conv2D or CNN. But we would not use that.

Next, we would train the SimpleNet we developed with sample Tensor data.

import torch    inp = torch.randn(100, 10)   tar = torch.randint(0, 2, (100,))   criterion = torch.nn.CrossEntropyLoss()  optimizr = torch.optim.SGD(model.parameters(), lr=0.01)    epochs = 100  batchsize = 10    for epoch in range(numepochs):      model.train()            for i in range(0, inp.size(0), batchsize):          batch_inp = inputs[i:i+batch_size]          batch_tar = targets[i:i+batch_size]                    out = model(batch_inp)          loss = criterion(out, batch_tar)                 optimizer.zero_grad()          loss.backward()          optimizr.step()            if (epoch + 1) % 10 == 0:          print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {round(loss.item(),4})')

During the training above, we use random Tensor data and initiate the loss function called CrossEntropyLoss. Also, we initiate the SGD optimizer to manage the model parameters to minimize the loss.

The training process runs multiple times according to the epoch numbers and then performs the optimization process. This is the usual deep-learning process.

We can add several steps to more complex training to improve training, like early stopping, learning rate, and other techniques.

Lastly, we can evaluate the model we have trained with the unseen data. The following code allows us to do that.

from sklearn.metrics import classification_report    model.eval()  test_inputs = torch.randn(20, 10)  test_targets = torch.randint(0, 2, (20,))    with torch.no_grad():      test_outputs = model(test_inputs)      _, predicted = torch.max(test_outputs, 1)    print(classification_report(test_targets, predicted))

What happened above is that we switched the model into the evaluation mode, which turned off dropout and batch normalization updates. Additionally, we disable the gradient computation process to speed up the process.

You can visit the PyTorch documentation to learn further about what you can do.

Conclusion

In this article, we will go through the basics of PyTorch. From tensor creation to tensor operations and developing a simple NN model. The article is an introductory level that every beginner should be able to follow quickly.

Cornellius Yudha Wijaya is a data science assistant manager and data writer. While working full-time at Allianz Indonesia, he loves to share Python and data tips via social media and writing media. Cornellius writes on a variety of AI and machine learning topics.

More On This Topic

• A Practical Guide to Transfer Learning using PyTorch
• Learn To Reproduce Papers: Beginner’s Guide
• A Beginner's Guide to End to End Machine Learning
• Essential Machine Learning Algorithms: A Beginner's Guide
• A Beginner's Guide to Q Learning
• A Beginner’s Guide to Web Scraping Using Python