---
title: "Regex Patterns"
author: "Statistical Computing, 36-350"
date: "Monday September 12, 2016"
---

Why do we need regex patterns?
===

In last week's lectures, we computed word tables by splitting up text and counting unique words, from documents of interest. Snippet:

```
> clinton.wordtab[1:5]
  —   …  “a “do “go 
 37  26   1   1   1 

```

These are not all actual words (they include punctuation marks). We need to learn how to **better split text**, and for this we need regular expressions. This will also help us **better search text**

What are regex patterns?
===

- A **regular expression** or **regex** is a specially structured string that allow us to match certain patterns occurring in text
- (Note: regexes follow a well-defined set of rules, independent of the R language)
- Any string defines a valid regex. To get us started, we'll consider **literals**, which are just strings that we want to match, literally. E.g.,
    - "fly" matches "superfly", "why walk when you can fly"  
    - "fly" does not match "time flies like an arrow", "fruit flies like bananas"
- OR of two regexes is a regex. E.g.,
    - "fly|flies" tries to match "fly" or "flies" 
- Concatenation of regexes is a regex. E.g., 
    - "(time|fruit) (fly|flies)" tries to match "time" or "fruit", then a space, then "fly" or "flies"
- Parentheses define groups; more on this later

Scanning for matches to a regex
====

Scan a vector of strings for matches to a regex, using `grep()`

```{r}
str.vec = c("time flies when you're having fun in 350",
            "time does not fly in 350, because it's not fun",
            "Flyers suck, Penguins rule")
grep("fly", str.vec) 
grep("fly", str.vec, value=TRUE)
grep("fly|flies", str.vec, value=TRUE)
```

More examples
===

```{r}
str.vec.2 = c("time flies when you're having fun in 350",
              "fruit flies when you throw it",
              "a fruit fly is a beautiful creature",
              "how do you spell fruitfly?")
grep("(time|fruit)(fly|flies)", str.vec.2, value=TRUE)
grep("(time|fruit) (fly|flies)", str.vec.2, value=TRUE)
grep("(time|fruit)  (fly|flies)", str.vec.2, value=TRUE)
```

Metacharacters
===

- **Metacharacters** are special characters that have a special meaning, and are not interpreted literally
- Important example: square braces, used to indicate that we want to match anything inside the square braces, for one character position. E.g.,
    - "[abcde]" matches the "a" in "Ryan"
    - "[123]" matches the "3" in "StatComp 350"
    - "[aeiou]" tries to match any lower case vowel
- A dash inside square braces is used to denote a range. E.g.,
    - "[a-e]" is the same as "[abcde]"
    - "[0-9]" is the same as "[0123456789]"
- Rules for combining regexes apply as before. E.g., 
    - "(Baker|Porter) 229[A-J]" tries to match "Baker" or "Porter", then a space, then "229", then any upper case letter between A and J
    
More metacharacters
===

- "[:alnum:]" tries to match any alphanumeric character (same as "[a-zA-Z0-9]")
- "[:punct:]" tries to match any punctuation mark
- "[:space:]" tries to match any white space character (including tabs and line breaks)
- A caret inside square braces negates what follows. E.g., 
    - "[^0-9]" tries to match anything but a number between 0 and 9
    - "[^aeiou]" tries to match anything but a lower case vowel
- A period "." tries to match any character (don't even need square braces)

More examples
===

```{r}
str.vec.3 = c("R2D2","r2d2","RJD2","RT85")
grep("[A-Z][0-9]", str.vec.3, value=TRUE)
grep("[A-Z][0-9][A-Z][0-9]", str.vec.3, value=TRUE)
grep("[A-Za-z][0-9][A-Za-z][0-9]", str.vec.3, value=TRUE)
grep("[A-Z][^0-9][^0-9][0-9]", str.vec.3, value=TRUE)
```

More examples
===

In R, we need to use double brackets for special abbreviated metacharacter classes like "[:punct:]" (to distinguish this from "[:punct]", which has its own interpretation)

```{r}
str.vec.4 = c("im simple i dont like punctuation",
              "I'm, all; about! p.u.n.c.t.u.a.t.i.o.n.")
grep("[:punct:]", str.vec.4, value=TRUE)
grep("[[:punct:]]", str.vec.4, value=TRUE)
```

Escape sequences
===

- In regexes, the characters ".", "$", "^", "{", "[", "(", "|", ")", "]", "}", "*", "+", "?", "\\" are all metacharacters and have special meaning
- An **escape sequence** is a way of turning them into literals: simply place a backslash in front. E.g.,
    - "\\[" tries to match a left square brace
    - "\\?" tries to match a question mark
    - "\\\\" tries to match a backslash
    - "\\\\\\\\" tries to match two backslashes
    
More examples
===

In R, we always have to use double the number of backslashes (because the backslash itself is a special character in an R string)

```{r}
str.vec.5 = c("Stat + Computing = Magic", 
              "Stat - Computing = Boring Theorems",
              "Do you have the time?")
grep("Stat \\+|Stat -", str.vec.5, value=TRUE)
grep("time\\?", str.vec.5, value=TRUE)
```