Use git to Blog(down) About Your Projects

Creating

All my blog posts live here:

website2/content/blog

I like creating a sub-directory that will hold the submodules for any project:

website2/content/blog/external_data

Then we can create the submodule with the submodule command:

Timothys-MacBook-Pro:external_data Tim$ git submodule add https://github.com/tmastny/git-blogdown-workflow.git
Cloning into '/Users/Tim/website2/content/blog/external_data/git-blogdown-workflow'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 10 (delta 0), reused 7 (delta 0), pack-reused 0
Unpacking objects: 100% (10/10), done.

When we git add/commit/push the changes to the website2 repo you’ll notice that the folder icon has changed. This indicates that website’s repo is now tracking the project’s repo at a certain commit, rather than a copy of those files at a moment in time.

In fact, if you click the folder you will be taken to the git-blogdown-workflow Github repo. This is exactly what we want. Instead of a static copy of the project’s file, we have a version controlled copy that is linked to the project’s Github repo.

Lastly, you should also edit your config.toml file to ignore your external_data folder, or where ever you plan to put your submodules.

ignoreFiles = ["\\.Rmd$", "_files$", "_cache$",
               "blogdown$", "external_data$"]

Since Hugo can’t build .Rmd files, blogdown compiles your .Rmd to .html. These .html files include all the necessary data, so there is no reason for Hugo to copy the submodules to the public/ directory.

Updating and Version Control

On a normal git repo, to receive the latest changes to your local branch you use git pull. But git pull only brings in changes to the top level git repo, which is the website’s repo. This is by design.

Imagine you write a post that includes content from a project submodule. Months later a teammate changes a feature your blog post depends on. If the submodule was updated automatically with pull your blog posts could break unexpectedly.

Therefore, to avoid unexpected conflicts we need to explicitly receive updates from submodules.

The first method is to navigate to the submodule and use git fetch/merge. For example,

Timothys-MacBook-Pro:blog Tim$ cd git-blogdown-workflow/
Timothys-MacBook-Pro:git-blogdown-workflow Tim$ git fetch
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 1), reused 3 (delta 1), pack-reused 0
Unpacking objects: 100% (3/3), done.
From https://github.com/tmastny/git-blogdown-workflow
   3149843..e207121  master     -> origin/master
Timothys-MacBook-Pro:git-blogdown-workflow Tim$ git merge origin/master
Updating 3149843..e207121
Fast-forward
 plot_model.R | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 plot_model.R

Note that git pull, the shortcut for git fetch/merge used on normal git repos does not work for submodules.

Or if you want to update all your submodules, you can use the shortcut git submodule update --remote:

Timothys-MacBook-Pro:blog Tim$ git submodule update --remote
Submodule path 'git-blogdown-workflow': checked out 'ce0d3a2f200eac6cd2a2980110137f7917e30b1d'
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 6 (delta 3), reused 5 (delta 2), pack-reused 0
Unpacking objects: 100% (6/6), done.
From https://github.com/tmastny/periodization-meta-analysis
   b29483b..de596ae  master     -> origin/master
Submodule path 'periodization-meta-analysis': checked out 'de596aed6a5535f7f4cdd270644e7993625b3f78'

But remember, pulling unexpected changes from your submodules can break your blog posts.

website2/content/blog

website2/content/blog/external_data/git-blogdown-workflow

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, warning=FALSE, message=FALSE, 
                      results='show', cache=FALSE, autodep=FALSE)
knitr::opts_knit$set(root.dir = 'external_data/git-blogdown-workflow/')
```

```{r}
knitr::read_chunk(path = 'your_script.R')
```

## ---- chunk_name ----

```{r chunk_name}
```

Example 1: The Basics

We’ll start simple by working with the R script sim.R.

Before we can call the chunks in our file, we need to initialize the .R script with read_chunk.¹

```{r}
knitr::read_chunk(path = 'sim.R')
```

knitr::read_chunk(path = 'sim.R')

This does not execute the code, but allows your blog post’s .Rmd environment to utilize chunks from that .R script.

After we’ve initialized the script with read_chunk, we can now call the code via the code externalization decorators we’ve added to the source code of sim.R. For example,

```{r simulate_function}
```

which produces

library(tidyverse)
library(magrittr)

monty_hall_sim <- function (door, switch) {
  doors <- c('a', 'b', 'c')
  prize <- sample(doors, size = 1)
  open_door <- sample(setdiff(doors, c(prize, door)), size = 1)
  if (switch) {
    door <- setdiff(doors, c(door, open_door))
  }
  return(door == prize)
}

This helps with portability and maintainability. We haven’t copied any code or scripts. The project script sim.R is being pull from the repo, so if the file is modified there, it will be modified here as well.

And another useful feature is that calling simulate_function evaluates the code in the .Rmd environment of the blog post. So we can use the function within the post!

monty_hall_sim('a', TRUE)

## [1] TRUE

This is nice. We can easily add simple, exploratory examples to the blog post that we wouldn’t want cluttering the .R script.

Let’s call the next chunk:

```{r run_simulation}
```

d <- tibble(
  door = sample(c('a', 'b', 'c'), size = 1000, replace = TRUE),
  switch = sample(c(TRUE, FALSE), size = 1000, replace = TRUE)
)
d %<>%
  mutate(prize = mapply(monty_hall_sim, door, switch))

Now we have the tibble d in our blog environment. For example, we can show

head(d)

## # A tibble: 6 x 3
##   door  switch prize
##   <chr> <lgl>  <lgl>
## 1 c     F      F    
## 2 b     T      T    
## 3 c     F      F    
## 4 b     T      T    
## 5 a     F      F    
## 6 c     T      F

You do need to be careful with this data. Any future chunks you call from sim.R evaluate the code in the current environment. So if d were gone, the next chunk would fail:

a <- d
d <- NULL

```{r plot_results}
```

d %>%
  group_by(switch) %>%
  mutate(attempt = row_number()) %>%
  mutate(success_ratio = cumsum(prize)/attempt) %>%
  ungroup() %>%
  ggplot() +
  geom_line(aes(x = attempt, y = success_ratio, color = switch))

## Error in UseMethod("group_by_"): no applicable method for 'group_by_' applied to an object of class "NULL"

So you need to make sure you don’t change the environmental variables in a way future chunks don’t except. A good rule of thumb is to not change or make new variables within the blog environment. Only explore and view objects created from the chunks.

Let’s fix our mistake and call the plot chunk again.

d <- a

```{r plot_results}
```

d %>%
  group_by(switch) %>%
  mutate(attempt = row_number()) %>%
  mutate(success_ratio = cumsum(prize)/attempt) %>%
  ungroup() %>%
  ggplot() +
  geom_line(aes(x = attempt, y = success_ratio, color = switch))

Example 2: External Data

The last example was nice, but a little unrealistic. Usually we use R to work with external data.

Again, we’ll do this by example. This time we want to showcase fit_model.R.

knitr::read_chunk('fit_model.R')

This script fits a linear model to tidied_periodization.csv² and plots the results.

Let’s try the first chunk:

```{r read_data}
```

d <- read_csv(file = 'tidied_periodization.csv')

Even though our blog post’s .Rmd is actually in the directory above tidied_periodization.csv, we can read file as if the .Rmd was in the project’s directory. This is because we changed the root directory in our setup chunk.

The next two chunks involve fitting and saving a linear model.

# copied from fit_model.R for reference
## ---- fit_model ----
m <- lm(post ~ pre, data = d)

## ---- save_fit ----
saveRDS(m, file = 'saved_model.rds')

Often we’d like to show how to fit the model, but we don’t want to do the computation in the blog post. For me, writing a blog post is 90% rewriting and reorganizing. Waiting to fit models is very disruptive, especially when working with machine learning algorithms or Bayesian packages such as brms or rstan that take a very long time to fit.

There is a straightforward way work-around. First we call the fit_model chunk without evaluating it, so we can show how to fit the model.

```{r fit_model, eval=FALSE}
```

m <- lm(post ~ pre, data = d)

Next, we load the model from the .RDS file within the project, with echo=FALSE so it doesn’t disrupt the narrative of the post.

```{r, echo=FALSE}
m <- readRDS('saved_model.rds')
```

And even though we haven’t executed fit_model.R linearly, we can still execute the final chunk because we’ve read in the model:

```{r plot_model}
```

d %>%
  mutate(pred = predict(m, newdata = .)) %>%
  ggplot(aes(x = pre)) +
  geom_point(aes(y = post)) +
  geom_line(aes(y = pred)) + 
  xlim(0, 420) + ylim(0, 500)