Controlling long outputs with knitr hooks • Adventures in data

Introduction

This blog is written in a combination of Markdown and R Markdown (using the R package blogdown).

As such, some R Markdown posts contain code chunks who generate very lengthy and verbose output.
An example is this one, which generates a sequence of strings separated by newlines:

n <- 10
m <- 10
matrix(rnorm(m*n), m, n)
##              [,1]       [,2]       [,3]        [,4]        [,5]
##  [1,] -0.56047565  1.2240818 -1.0678237  0.42646422 -0.69470698
##  [2,] -0.23017749  0.3598138 -0.2179749 -0.29507148 -0.20791728
##  [3,]  1.55870831  0.4007715 -1.0260044  0.89512566 -1.26539635
##  [4,]  0.07050839  0.1106827 -0.7288912  0.87813349  2.16895597
##  [5,]  0.12928774 -0.5558411 -0.6250393  0.82158108  1.20796200
##  [6,]  1.71506499  1.7869131 -1.6866933  0.68864025 -1.12310858
##  [7,]  0.46091621  0.4978505  0.8377870  0.55391765 -0.40288484
##  [8,] -1.26506123 -1.9666172  0.1533731 -0.06191171 -0.46665535
##  [9,] -0.68685285  0.7013559 -1.1381369 -0.30596266  0.77996512
## [10,] -0.44566197 -0.4727914  1.2538149 -0.38047100 -0.08336907
##              [,6]        [,7]       [,8]         [,9]      [,10]
##  [1,]  0.25331851  0.37963948 -0.4910312  0.005764186  0.9935039
##  [2,] -0.02854676 -0.50232345 -2.3091689  0.385280401  0.5483970
##  [3,] -0.04287046 -0.33320738  1.0057385 -0.370660032  0.2387317
##  [4,]  1.36860228 -1.01857538 -0.7092008  0.644376549 -0.6279061
##  [5,] -0.22577099 -1.07179123 -0.6880086 -0.220486562  1.3606524
##  [6,]  1.51647060  0.30352864  1.0255714  0.331781964 -0.6002596
##  [7,] -1.54875280  0.44820978 -0.2847730  1.096839013  2.1873330
##  [8,]  0.58461375  0.05300423 -1.2207177  0.435181491  1.5326106
##  [9,]  0.12385424  0.92226747  0.1813035 -0.325931586 -0.2357004
## [10,]  0.21594157  2.05008469 -0.1388914  1.148807618 -1.0264209

and this one, which generates a single long string:

long_string <- paste(rep(letters, 30), collapse = '')
long_string
## [1] "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"

Enter: knitr hooks

As detailed by the knitr documentation, knitr provides a set of functions to transparently modify a chunk, without changing the inner contents, but specifying chunk options. Such functions are called hooks.

Knitr hooks can be of two types: chunk hooks, output hooks and option hooks.
To our purpose, we are interested in output hooks.

These are functions which, in general, take two parameters: the chunk output, and the chunk options (e.g. fig.width).

From the documentation, an output hook can deal with 8 different types of outputs:

source: the source code
output: ordinary R output (i.e., what would have been printed in an R terminal) except warnings, messages and errors
warning: warnings from warning()
message: messages from message()
error: errors from stop() (applies to errors in both code chunks and inline R code)
plot: graphics output
inline: output of inline R code
chunk: all the output of a chunk (i.e., those produced by the previous hooks)
document: the output of the whole document (is base::identity by default)

Again, we are interested in output hooks for the output, as we want to preserve source code, errors, warnings and messages.

Line truncation

This is directly taken from knitr examples: 052-suppress-output.Rmd

First, e.g. in the setup chunk, we retrieve the default output hook:

# the default output hook
hook_output_default <- knitr::knit_hooks$get('output')

We define a function which truncates the output up to n lines. x is the chunk output, as a character vector. If n is missing, the output passes through, unaffected.

truncate_to_lines <- function(x, n) {
   if (!is.null(n)) {
      x = unlist(stringr::str_split(x, '\n'))
      if (length(x) > n) {
         # truncate the output
         x = c(head(x, n), '...\n')
      }
      x = paste(x, collapse = '\n') # paste first n lines together
   }
   x
}

Finally, we can overwrite the default knitr hook, and truncate if option max.lines is specified:

knitr::knit_hooks$set(output = function(x, options) {
   max.lines <- options$max.lines
   x <- truncate_to_lines(x, max.lines)

   hook_output_default(x, options)
})

Let’s see the first chunk, now truncated to 3 lines. We set max.lines=3 in the chunk options:

```{r, max.lines = 3}
n <- 10
m <- 10
matrix(rnorm(m*n), m, n)
```

and here is the output:

n <- 10
m <- 10
matrix(rnorm(m*n), m, n)
##              [,1]        [,2]        [,3]        [,4]       [,5]
##  [1,] -0.71040656 -0.57534696  0.11764660  1.44455086  0.7017843
##  [2,]  0.25688371  0.60796432 -0.94747461  0.45150405 -0.2621975
...

Character truncation

This time we need to define a function which truncates up to a specified amount of characters, if n_chars is specified:

truncate_to_chars <- function(x, n_chars) {
   if (!is.null(n_chars)) {
      if (min(nchar(x), n_chars) < nchar(x)) {
         x <- substr(x, 1, min(nchar(x), n_chars))
         x <- paste(x, ' ...\n')
      }
   }
   x
}

Then, we can add the truncation function to the output hook:

knitr::knit_hooks$set(output = function(x, options) {
    max.lines <- options$max.lines
    x <- truncate_to_lines(x, max.lines)

    # Here is our new code
    max.chars <- options$max.chars
    x <- truncate_to_chars(x, max.chars)

    hook_output_default(x, options)
})

And here is the second chunk, truncated to 20 chars (setting max.chars = 20 in the chunk options):

long_string <- paste(rep(letters, 30), collapse = '')
long_string
## [1] "abcdefghijkl  ...

Notice that the code is not part of the character count (it is an output hook on the output!), but the prompt and the whitespaces are (up to "...").

And, of course, both truncations can be combined.
E.g., let’s see the first chunk to 3 lines and 160 characters:

n <- 10
m <- 10
matrix(rnorm(m*n), m, n)
##              [,1]       [,2]        [,3]        [,4]       [,5]
##  [1,]  2.19881035  0.1192452 -0.57397348  1.95529397 -0.7886220
##  [2,]  1.31241298  0.24  ...

Thanks for reading!