---
title: "Reading in and Reordering Data"
author: "Statistical Computing, 36-350"
date: "Wednesday October 26, 2016"
---

Reading in data from the outside
===

All along, we've already been reading in data from the outside, using:

- `readLines()`: reading in lines of text from a file or webpage; returns vector of strings
- `read.table()`: read in a data table from a file or webpage; returns data frame
- `read.csv()`: like the above, but assumes comma separated data; returns data frame

Today we'll focus on `read.table()`, `read.csv()` and their counterparts `write.table()`, `write.csv()`, respectively

Reading in data from a previous R session
===

Reminder that we've already learned two ways to read/write in data in specialized R formats:

- `readRDS()`, `saveRDS()`: functions for reading/writing single R objects from/to a file
- `load()`, `save()`: functions for reading/writing any number of R objects from/to a file

Advantage: these can be a lot more memory efficient than what we'll cover today. Disadvantage: they're limited in scope in the sense that they can only communicate with R; to read in data from the "outside world", or write data that other programs can deal with, we'll need to use `read.table()`, `write.table()`, etc.

`read.table()`, `read.csv()`
===

Have a table full of data, just not in the R file format? Then `read.table()` is the function for you. It works as in:

- `read.table(file=file.name, sep=" ")`, to read data from a local file on your computer called `file.name`, assuming (say) space separated data
- `read.table(file=webpage.link, sep="\t")`, to read data from a webpage up at `webpage.link`, assuming (say) tab separated data

The function `read.csv()` is just a shortcut for using `read.table()` with `sep=","`. (But note: these two actually differ on some of the default inputs!)

Easy example of reading in a data table
===

```{r}
# This data table is comma separated, so we can use read.csv()
strike.df = read.csv("http://www.stat.cmu.edu/~ryantibs/statcomp-F16/data/strikes.csv")
class(strike.df)
head(strike.df)
```

Helpful input arguments
===

The following inputs apply to either `read.table()` or `read.csv()` (though these two functions actually have different default inputs in general---e.g., `header` defaults to TRUE in `read.table()` but FALSE in `read.csv()`)

- `header`: boolean, TRUE is the first line should be interpreted as column names
- `sep`: string, specifies what separates the entries; empty string "" is interpreted to mean any white space
- `quote`: string, specifies what set of characters signify the beginning and end of quotes; empty string "" disables quotes altogether

Other helpful inputs: `skip`, `row.names`, `col.names`. You can read about them in the help file for `read.table()` 

More examples of reading in data tables
===

```{r}
# This data table is tab separated, and it comes with column names
anss.df = read.table("http://www.stat.cmu.edu/~ryantibs/statcomp-F16/data/anss.dat",
                     sep="\t", header=TRUE)
class(anss.df)
head(anss.df)

# This data table is tab separated, it comes with column names, and also has 
# some apostrophe marks in the City column, so we have to set quote="". Also
# note that we've disabled the row names
sprint.df = read.table("http://www.stat.cmu.edu/~ryantibs/statcomp-F16/data/sprint.dat",
                       sep="\t", header=TRUE, quote="")
class(sprint.df)
head(sprint.df)
unique(grep("'", sprint.df$City, value=TRUE)) # This is the troublemaker
```

`write.table()`, `write.csv()`
===

To write a data frame (or matrix) to a text file, use `write.table()` or `write.csv()`. These are the counterparts to `read.table()` and `read.csv()`, and they work as in:

- `write.table(my.df, file="my.df.txt", sep=" ", quote=FALSE)`, to write `my.dat` to the text file "my.df.txt" (to be created in your working directory), with (say) space separation, and no quotes around the entries
- `write.csv(my.df, file="my.df.csv", quote=FALSE)`, to write `my.dat` to the text file "my.df.csv" (to be created in your working directory), with comma separation, and no quotes

Note that `quote=FALSE`, signifying that no quotes should be put around the printed data entries, seems always preferable (default is `quote=TRUE`). Also, setting `row.names=FALSE` and `col.names=FALSE` will disable the printing of row and column names (defaults are `row.names=TRUE` and `col.names=TRUE`)

Reordering data
===

Sometimes it's convenient to reorder our data, say the rows of our data frame (or matrix). Recall:

- The function `order()` takes in a vector, and returns the vector of indices that put this vector in increasing order
- Set the input `decreasing=TRUE` in `order()` to get decreasing order
- We can apply compute an appropriate vector of indices, and then use this on rows of our data frame to reorder all of the columns simultaneously

Examples of reordering
===

```{r}
# The sprint data has its rows ordered by the fastest 100m time to the slowest.
# Suppose we wanted to reorder, from slowest to fastest
i.slow = order(sprint.df$Time, decreasing=TRUE) # By decreasing sprint time
sprint.df.slow = sprint.df[i.slow,] # Reorder rows by decreasing sprint time
head(sprint.df.slow)

# Suppose we wanted to reorder the rows by sprinter name, alphabetically
i.name = order(sprint.df$Name) # By sprinter name
sprint.df.name = sprint.df[i.name,] # Reorder rows by name
head(sprint.df.name)
```