• Intro to TensorFlow

• Deep Learning

• R Interfaces to TensorFlow

• Supporting Tools

• Learning More

• A new general purpose numerical computing library!
  • Hardware independent
  • Distributed execution
  • Large datasets
  • Automatic differentiation

• Very general built-in optimization algorithms (SGD, Adam) that don’t require that all data is in RAM

• Robust foundation for machine learning and deep learning applications

• TensorFlow models can be deployed with a low-latency C++ runtime

• R has a lot to offer as an interface language for TensorFlow

• Tensors

• Data flow

• Runtime execution

• Some example uses

Note that samples is always the first dimension

head(data.matrix(iris), n = 10)

      Sepal.Length Sepal.Width Petal.Length Petal.Width Species
 [1,]          5.1         3.5          1.4         0.2       1
 [2,]          4.9         3.0          1.4         0.2       1
 [3,]          4.7         3.2          1.3         0.2       1
 [4,]          4.6         3.1          1.5         0.2       1
 [5,]          5.0         3.6          1.4         0.2       1
 [6,]          5.4         3.9          1.7         0.4       1
 [7,]          4.6         3.4          1.4         0.3       1
 [8,]          5.0         3.4          1.5         0.2       1
 [9,]          4.4         2.9          1.4         0.2       1
[10,]          4.9         3.1          1.5         0.1       1

I’m confident the R community will also find novel new uses for TensorFlow…

• What is deep learning?

• What is it useful for?

• Why should R users care?

• How does it work?

• Some examples

Special thanks to François Chollet (creator of Keras) for the concepts and figures used to explain deep learning! (all drawn from Chapter 1 of his Deep Learning with R book)

• A layer is a geometric transformation function on the data that goes through it (transformations must be differentiable for stochastic gradient descent)

• Weights determine the data transformation behavior of a layer

The layers of representation in a deep learning model are the feature engineering for the model (i.e. feature transformations are learned rather than hard coded).

A new take on learning representations from data that puts an emphasis on learning successive layers of increasingly meaningful representations.

Other possibly more appropriate names for the field:

• Layered representations learning
• Hierarchical representations learning
• Chained geometric transformation learning

Modern deep learning often involves tens or even hundreds of successive layers of representation

Other approaches to machine learning tend to focus on learning only one or two layers of representations of the data

• New problem domains for R:
  • Computer vision
  • Computer speech recognition
  • Reinforcement learning applications
• Improved techniques for our traditional domains?
  • Analyzing data with complex spatial or sequence dependencies
  • Analyzing data which requires a large amount of (potentially brittle) feature engineering to model effectively

Deep learning is proven to be effective at various complex “perceptual” tasks but not yet proven to be of widespread benefit in other domains.

• Basics of machine learning algorithms
• Machine learning vs. statistical modeling
• MNIST example
  • Model definition in R
  • Layers of representation
• The training loop

A simple mechanism that, once scaled, ends up looking like magic

library(keras)
model <- keras_model_sequential() %>%
  layer_conv_2d(filters = 32, kernel_size = c(3,3), activation = 'relu',
                input_shape = c(28,28,1)) %>% 
  layer_conv_2d(filters = 64, kernel_size = c(3,3), activation = 'relu') %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>% 
  layer_flatten() %>% 
  layer_dense(units = 128, activation = 'relu') %>% 
  layer_dense(units = 10, activation = 'softmax')

Deep-learning models are mathematical machines for uncrumpling complicated manifolds of high-dimensional data.

Deep learning is turning meaning into vectors, into geometric spaces, and then incrementally learning complex geometric transformations that map one space to another.

How can we do this with simple parametric models trained with gradient descent? We just need sufficiently large parametric models trained with gradient descent on sufficiently many examples.

• Computer vision

• Natural language processing

• Time series

• Biomedical

• What next?

Paper submissions could just indicate a passing fad or could be a fundamental transformation, we don’t know yet!

Produced predictions for a wide range of clinical problems and outcomes that outperformed state-of-the-art traditional predictive models.

• Not easily interpreted (black box)
• Can be brittle (see “adversarial examples”)
• Typically require large amounts of data to perform well
• Are often very computationally expensive to train

• Deep learning is experiencing unprecedented hype, and…
• The current generation of tools make deep learning accessible to many with no prior experience in statistics or modeling, however…
• It can be difficult to outperform traditional modeling techniques without a lot of effort, so…
• We will be trolled by bad deep learning models for the foreseeable future

Of course we can help here by promoting a more balanced dialog about the strengths and weaknesses of these methods.

• High-level R interfaces for neural nets and traditional models

• Low-level interface to enable new applications (e.g. Greta)

• Tools to facilitate productive workflow / experiment management

• Straightforward access to GPUs for training models

• Breadth and depth of educational resources

library(keras)
model <- keras_model_sequential() %>%
  layer_conv_2d(filters = 32, kernel_size = c(3,3), activation = 'relu',
                input_shape = input_shape) %>% 
  layer_conv_2d(filters = 64, kernel_size = c(3,3), activation = 'relu') %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>% 
  layer_dropout(rate = 0.25) %>% 
  layer_flatten() %>% 
  layer_dense(units = 128, activation = 'relu') %>% 
  layer_dropout(rate = 0.5) %>% 
  layer_dense(units = 10, activation = 'softmax')

• Step-by-step example
• Keras layers
• Compiling models
• Losses, optimizers, and metrics
• More examples

model <- keras_model_sequential()  %>% 
  layer_dense(units = 256, activation = 'relu', input_shape = c(784)) %>% 
  layer_dropout(rate = 0.4) %>% 
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dropout(rate = 0.3) %>%
  layer_dense(units = 10, activation = 'softmax')

model %>% compile(
  loss = 'categorical_crossentropy',
  optimizer = optimizer_rmsprop(),
  metrics = c('accuracy')
)

# Modify model object in place (note that it is not assigned back to)
model %>% compile(
  optimizer = 'rmsprop',
  loss = 'binary_crossentropy',
  metrics = c('accuracy')
)

• Keras models are directed acyclic graphs of layers whose state is updated during training.

• Keras layers can be shared by multiple parts of a Keras model.

history <- model %>% fit(
  x_train, y_train,
  batch_size = 128,
  epochs = 10,
  validation_split = 0.2
)

• Feed 128 samples at a time to the model (batch_size = 128)

• Traverse the input dataset 10 times (epochs = 10)

• Hold out 20% of the data for validation (validation_split = 0.2)

plot(history)

model %>% evaluate(x_test, y_test)

$loss
[1] 0.1078904

$acc
[1] 0.9815

model %>% predict_classes(x_test[1:100,])

  [1] 7 2 1 0 4 1 4 9 5 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1 3 1 3 4 7
 [36] 2 7 1 2 1 1 7 4 2 3 5 1 2 4 4 6 3 5 5 6 0 4 1 9 5 7 8 9 3 7 4 6 4 3 0
 [71] 7 0 2 9 1 7 3 2 9 7 7 6 2 7 8 4 7 3 6 1 3 6 9 3 1 4 1 7 6 9

layer_dense(units = 128)

output = activation(dot(input, kernel) + bias)

model <- keras_model_sequential() %>%
  layer_embedding(input_dim = 10000, output_dim = 8,
                  input_length = 20) %>%
  layer_flatten() %>%
  layer_dense(units = 1, activation = "sigmoid")

• Learn the embeddings jointly with the main task you care about (e.g. classification); or

• Load pre-trained word embeddings (e.g. Word2vec, GloVe)

Model compilation prepares the model for training by:

1. Converting the layers into a TensorFlow graph
2. Applying the specified loss function and optimizer
3. Arranging for the collection of metrics during training

model %>% compile(
  loss = 'categorical_crossentropy',
  optimizer = optimizer_rmsprop(),
  metrics = c('accuracy')
)

• Image classification on small datasets
• Time series forecasting with recurrent networks
• Deep learning for cancer immunotherapy
• Credit card fraud detection using an autoencoder
• Classifying duplicate questions from Quora
• Deep learning to predict customer churn
• Learning word embeddings for Amazon reviews
• Work on explainability of predictions

• Many fields of inquiry have a deep learning frontier that has not yet been reached or even well approached.

• In these cases traditional methods are invariably cheaper and more accurate.

• Some possible approaches:
  • Wait and see
  • Follow/evaluate the latest research
  • Attempt to approach the frontier in your own work

• To approach the frontier you need understanding of the full range of DL techniques, iteration, patience, and lots of compute power!

• Success requires a large amount of experimentation with:
  • Model architecture (i.e. number and type of layers)
  • Hyperparameters (layer attributes, losses, learning rate, dropout, etc.)
  • Frustration and blind alleys are inevitable!

• Over time you will develop stronger intuitions about the various tools available and how they might be successfully combined to model your data.

• Need lots of compute power!

• Need tools that help you get the most out of the compute you have.

Successful deep learning requires a huge amount of experimentation.

This requires a systematic approach to conducting and tracking the results of experiments.

The training_run() function is like the source() function, but it automatically tracks and records output and metadata for the execution of the script:

library(tfruns)
training_run("mnist_mlp.R")

# define flags and their defaults
FLAGS <- flags(
  flag_integer("dense_units1", 128),
  flag_numeric("dropout1", 0.4),
  flag_integer("dense_units2", 128),
  flag_numeric("dropout2", 0.3)
)

# use flag 
layer_dropout(rate = FLAGS$dropout1)

# train with flag
training_run("mnist_mlp.R", flags = list(dropout1 = 0.3))

Train on default CPU instance:

library(cloudml)
cloudml_train("mnist_mlp.R")

Automatically uploads contents of working directory along with script

Automatically installs all required R packages on CloudML servers

# Train on a GPU instance
cloudml_train("mnist_mlp.R", master_type = "standard_gpu")

# Train on an NVIDIA Tesla P100 GPU
cloudml_train("mnist_mlp.R", master_type = "standard_p100")

job_collect()

• TensorFlow was built from the ground up to enable deployment using a low-latency C++ runtime.

• Deploying TensorFlow models requires no runtime R or Python code.

• Key enabler for this is the TensorFlow SavedModel:

  SavedModel provides a language-neutral format to save machine-learned models that is recoverable and hermetic. It enables higher-level systems and tools to produce, consume and transform TensorFlow models.

• TensorFlow models can be deployed to servers, embedded devices, mobile phones, and even to a web browser!

model <- keras_model_sequential( %>% )
  layer_dense(units = 256, activation = 'relu', input_shape = c(784),
              name = "image") %>%
  layer_dense(units = 128, activation = 'relu') %>%
  layer_dense(units = 10, activation = 'softmax',
              name = "prediction")

Note that we give the input and output layers names (“image” and “prediction”)

# ...compile and fit model

# export model
library(tfdeploy)
export_savedmodel(model, "savedmodel")

library(cloudml)
cloudml_deploy("savedmodel", name = "keras_mnist")

Copying file://savedmodel/variables/variables.data-00000-of-00001 ...
Copying file://savedmodel/saved_model.pb ...
Copying file://savedmodel/variables/variables.index ...
/ [3/3 files][  1.9 MiB/  1.9 MiB] 100% Done                                    
Operation completed over 3 objects/1.9 MiB.

Model created and available in https://console.cloud.google.com/mlengine/models/keras_mnist

• Straightforward to load already trained models into Shiny applications.

• Training code (and hardware!) not required for inference.

model <- load_model_hdf5("model.hdf5")    # Keras 
model %>% predict(input)

predict_savedmodel(input, "savedmodel")   # SavedModel

• Run on embedded devices (e.g. Raspberry Pi)

• Keras models can be converted to iOS CoreML

• Keras models can be deployed to the browser with Keras.js

• Additional projects:
  • TensorFlow Lite
  • deeplearn.js

• TensorFlow is a new general purpose numerical computing library with lots to offer the R community.

• Deep learning has made great progress and will likely increase in importance in various fields in the coming years.

• R now has a great set of APIs and supporting tools for using TensorFlow and doing deep learning.

Slides: https://rstd.io/ml-with-tensorflow-and-r/

Subscribe to the blog to stay up to date!

https://tensorflow.rstudio.com/blog/