reticulate and torch Basics

Deep Learning and Computer Vision in R: A Practical Introduction

SSA ACT workshop - reticulate and torch Basics

Weihao (Patrick) Li

RSFAS, ANU

Content summary

• Overview of computer vision (CV)
• > reticulate and torch basics
• Image classification, neural networks and Keras
• Convolutional neural networks
• Vision transformers
• Object detection
• Image segmentation

R Package reticulate

Almost all modern deep learning frameworks and most computer vision tools are implemented in Python.

reticulate is an R package that lets you call Python code seamlessly from R.

• Converts common R and Python objects automatically for easy exchange
• Supports multiple Python versions and environments

Installation

Please run scripts/setup.R to set up reticulate and Python.

The script will:

1. Install or update the R packages required for this workshop, including reticulate
2. Install Miniconda, a Python environment manager
3. Create a new conda environment (similar to an R project, but with associated R packages)
4. Install the Python libraries needed for the workshop in the conda environment

If you have any installation issues, please let me know.

Use the conda environment

All the required packages and libraries are already installed in the ssa-cv-workshop conda environment.

We can instruct reticulate to use this environment with the following code:

library(reticulate)
library(scrubwren)
use_condaenv("ssa-cv-workshop")

To verify that reticulate is correctly linked to the environment, run:

py_config()

Object conversions (r_to_py)

R integer / double \rightarrow Python int / float

r_to_py(1L)

1

r_to_py(1)

1.0

R character / logical \rightarrow Python str / bool

r_to_py("abc")

'abc'

r_to_py(TRUE)

True

R NULL \rightarrow Python None

r_to_py(NULL)

None

R vector \rightarrow Python list

r_to_py(c(1, 2, 3))

[1.0, 2.0, 3.0]

r_to_py(list(1, 2, 3))

[1.0, 2.0, 3.0]

R array \rightarrow NumPy array

r_to_py(matrix(1:4, ncol = 2))

array([[1, 3],
       [2, 4]], dtype=int32)

R named list \rightarrow Python dict

r_to_py(list(a = 1, b = 2))

{'a': 1.0, 'b': 2.0}

r_to_py(head(cars, 2))

{'speed': [4.0, 4.0], 'dist': [2.0, 10.0]}

Object conversions (py_to_r)

Python list will be simplified to atomic vector when possible.

py_to_r(py_builtins$list(
  list(1, 2, 3)
))

[1] 1 2 3

py_to_r(py_builtins$list(
  list(1, 2, list(3, 4))
))

[[1]]
[1] 1

[[2]]
[1] 2

[[3]]
[1] 3 4

Some Python objects are non-convertible.

py_builtins$set(c(1, 2, 3)) |>
  py_to_r() |>
  print() |>
  class()

{1.0, 2.0, 3.0}

[1] "python.builtin.set"    "python.builtin.object"

torch tensor

Import torch

torch <- import("torch", convert = FALSE)

Create torch tensor

• data type (torch$float32, torch$int32, …)
• storage type/device ("cpu", "cuda", "mps", …)
• torch$accelerator$current_accelerator()

x <- torch$tensor(matrix(1:4, ncol = 2),
                  dtype = torch$float32,
                  device = "cpu")
x

tensor([[1., 3.],
        [2., 4.]])

torch tensor operations

x + x

tensor([[2., 6.],
        [4., 8.]])

x * x * 2

tensor([[ 2., 18.],
        [ 8., 32.]])

x %*% x

tensor([[ 7., 15.],
        [10., 22.]])

x$sum()

tensor(10.)

x$mean()

tensor(2.5000)

Many other operations can be performed on tensors, check out the documentation.

torch tensor indexing

Python uses 0-based indexing, and torch indexing syntax is quite similar to that of R.

x[0]    # First element along the first dimension

x[0, 0] # First element of the first dimension, 
        # first element of the second dimension
        
x[, 0]  # First element along the second dimension

x[, -1] # Last element along the second dimension

x[0:2]  # First two elements along the first dimension

x[0:-1] # All elements along the first dimension except the last

x[0] <- torch$tensor(5:6) # Replace the first element along the first 
                          # dimension with (5, 6)

Convert torch tensor back to R array

Unfortunately, safely converting a torch tensor back to an R array can be quite complicated, especially when the tensor resides on the GPU or is part of a computation graph.

It’s generally best to keep it as a torch tensor throughout your workflow whenever possible.

Best practice:

x$detach()$cpu()$numpy()$copy() |>
  py_to_r()

     [,1] [,2]
[1,]    1    3
[2,]    2    4

Compute gradient for \mathbb{R}^n \to \mathbb{R}

Define y = \sum_{i=1}^n x_i^2 + x_i.

x <- torch$tensor(c(1, 2, 3), requires_grad = TRUE)
y <- torch$sum(x^2 + x)
y

tensor(20., grad_fn=<SumBackward0>)

Compute \left(\frac{\partial y}{\partial x_1}, \dots , \frac{\partial y}{\partial x_n}\right), which is (2x_1 + 1, \dots, 2x_n + 1 )

y$backward()

None

x$grad

tensor([3., 5., 7.])

Exercises

01-torch-basics-ex.R — 10 minutes (5 to walk through, 5 to work)

1. Generate a random tensor x and another random tensor e, each of shape (50), using torch$rand().

2. Create a tensor beta of two elements, both equal to 2. Enable gradient tracking for beta.

3. Compute y as: \boldsymbol{y} = 1 + 3 \boldsymbol{x} + \boldsymbol{e}

4. Compute y_hat as: \hat{\boldsymbol{y}} = \beta_0 + \beta_1 \boldsymbol{x}

5. Compute the loss as: \text{loss} = \frac{1}{50} \sum_{i=1}^{50} (y_i - \hat{y}_i)^2

6. Compute the gradient of loss with respect to beta.