---
title: "Plyr (Optional)"
author: "Statistical Computing, 36-350"
date: "Monday September 17, 2018"
---

The `plyr` package
===

One of the most downloaded R package of all time: `plyr`. This is true for a good reason! Provides extremely useful family of apply-like functions

- Advantage over the built-in `apply()` family is its consistency
- All `plyr` functions are of the form `**ply()`
- Replace `**` with characters denoting types:
    * First character: input type, one of `a`, `d`, `l` 
    * Second character: output type, one of `a`, `d`, `l`, or `_` (drop)

`a*ply()`: input is an array
===

The signature for all `a*ply()` functions is:

```
a*ply(.data, .margins, .fun, ...)
```

- `.data` : an array
- `.margins` : index (or indices) to split the array by
- `.fun` : the function to be applied to each piece
- `...` : additional arguments to be passed to the function

Note that this resembles:

```
apply(X, MARGIN, FUN, ...)
```

Examples of `a*ply()`
===

```{r}
library(plyr)
head(aaply(state.x77, 1, mean)) # Get back array
head(adply(state.x77, 1, mean)) # Get back data frame
head(alply(state.x77, 1, mean)) # Get back list
```

```{r}
mean.sd = function(x) c("mean"=mean(x), "sd"=sd(x))
head(aaply(state.x77, 1, mean.sd)) # Get back array
head(adply(state.x77, 1, mean.sd)) # Get back data frame
head(alply(state.x77, 1, mean.sd)) # Get back list
```

`l*ply()`: input is a list
===

The signature for all `l*ply()` functions is:

```
l*ply(.data, .fun, ...)
```

- `.data` : a list
- `.fun` : the function to be applied to each element
- `...` : additional arguments to be passed to the function

Note that this resembles:

```
lapply(X, FUN, ...)
```

Examples of `l*ply()`
===

```{r}
my.list = list(nums=rnorm(1000), lets=letters, pops=state.x77[,"Population"])
laply(my.list, range) # Get back array
ldply(my.list, range) # Get back data frame
llply(my.list, range) # Get back list
```

---

```{r, error=TRUE}
laply(my.list, summary) # Doesn't work! Outputs have different types/lengths
ldply(my.list, summary) # Doesn't work! Outputs have different types/lengths
llply(my.list, summary) # Works just fine
```

The fourth option for `*`
===

The fourth option for `*` is `_`: the function `a_ply()` (or `l*ply()`) has no explicit return object, but still runs the given function over the given array (or list), possibly producing side effects

```{r}
par(mfrow=c(3,3), mar=c(4,4,1,1))
a_ply(state.x77, 2, hist, breaks=30, col="pink")
```

`d*ply()` : the input is a data frame
===

The signature for all `d*ply()` functions is:

```
d*ply(.data, .variables, .fun, ...)
```

- `.data` : a data frame
- `.variables` : variable (or variables) to split the data frame by
- `.fun` : the function to be applied to each piece
- `...` : additional arguments to be passed to the function

Note that this resembles:

```
tapply(X, INDEX, FUN, ...)
```

Examples of `d*ply()`
===

```{r}
state.df = data.frame(state.x77, Region=state.region, Division=state.division)
daply(state.df, .(Region), function(df) mean.sd(df$Frost)) # Get back array
ddply(state.df, .(Region), function(df) mean.sd(df$Frost)) # Get back df
dlply(state.df, .(Region), function(df) mean.sd(df$Frost)) # Get back list
```

Splitting on two (or more) variables
===

The function `d*ply()` makes it very easy to split on two (or more) variables: we just specify them, separated by a "," in the `.variables` argument

```{r}
# First create a variable that indicates whether the area is big or not
state.df$AreaBig = state.df$Area > 50000
# Now use (say) ddply() to compute the mean and sd Frost, for each region, but
# separately over big and small areas
ddply(state.df, .(Region, AreaBig), function(df) mean.sd(df$Frost))
```

---

```{r}
# We can also create factor variables on-the-fly with I() 
ddply(state.df, .(Region, I(Area > 50000)), function(df) mean.sd(df$Frost))
```

Parallelization
===

What happens if we have a really large data set and we want to use split-apply-combine?

If the individual tasks are unrelated, then we should be speed up the computation by performing them **in parallel**

The `plyr` functions make this quite easy: let's take a look at the full signature for `daply()`:

```
daply(.data, .variables, .fun = NULL, ..., .progress = "none",
  .inform = FALSE, .drop_i = TRUE, .drop_o = TRUE, .parallel = FALSE,
  .paropts = NULL)
```

The second to last argument `.parallel` (default FALSE) is for parallelization. If set to TRUE, then it performs the individual tasks in parallel, using the `foreach` package

The last argument `.paropts` is for more advanced parallelization, these are additional arguments to be passed to `foreach`