Jeromy Anglim's Blog: Psychology and Statistics

Example Reproducible Report using R Markdown: Analysis of California Schools Test Data

2012-05-18T21:22:00.000+10:00

This is a quick set of analyses of the California Test Score dataset. The post was produced using R Markdown in RStudio 0.96. The main purpose of this post is to provide a case study of using R Markdown to prepare a quick reproducible report. It provides examples of using plots, output, in-line R code, and markdown. The post is designed to be read along side the R Markdown source code, which is available as a gist on github.

Preliminaries

This post builds on my earlier post which provided a guide for Getting Started with R Markdown, knitr, and RStudio 0.96
The dataset analysed comes from the AER package which is an accompaniment to the book Applied Econometrics with R written by Christian Kleiber and Achim Zeileis.

Load packages and data

# if necessary uncomment and install packages.  install.packages('AER')
# install.packages('psych') install.packages('Hmisc')
# install.packages('ggplot2') install.packages('relaimpo')
library(AER)  # interesting datasets
library(psych)  # describe and psych.panels
library(Hmisc)  # describe
library(ggplot2)  # plots: ggplot and qplot
library(relaimpo)  # relative importance in regression

# load the California Schools Dataset and give the dataset a shorter name
data(CASchools)
cas <- CASchools

# Convert grade to numeric

# table(cas$grades)
cas$gradesN <- cas$grades == "KK-08"

# Get the set of numeric variables
v <- setdiff(names(cas), c("district", "school", "county", "grades"))

Q 1 What does the CASchools dataset involve?

Quoting the help (i.e., ?CASchools), the data is “from all 420 K-6 and K-8 districts in California with data available for 1998 and 1999” and the variables are:

* district: character. District code.
* school: character. School name.
* county: factor indicating county.
* grades: factor indicating grade span of district.
* students: Total enrollment.
* teachers: Number of teachers.
* calworks: Percent qualifying for CalWorks (income assistance).
* lunch: Percent qualifying for reduced-price lunch.
* computer: Number of computers.
* expenditure: Expenditure per student.
* income: District average income (in USD 1,000).
* english: Percent of English learners.
* read: Average reading score.
* math: Average math score.

Let's look at the basic structure of the data frame. i.e., the number of observations and the types of values:

str(cas)

## 'data.frame':    420 obs. of  15 variables:
##  $ district   : chr  "75119" "61499" "61549" "61457" ...
##  $ school     : chr  "Sunol Glen Unified" "Manzanita Elementary" "Thermalito Union Elementary" "Golden Feather Union Elementary" ...
##  $ county     : Factor w/ 45 levels "Alameda","Butte",..: 1 2 2 2 2 6 29 11 6 25 ...
##  $ grades     : Factor w/ 2 levels "KK-06","KK-08": 2 2 2 2 2 2 2 2 2 1 ...
##  $ students   : num  195 240 1550 243 1335 ...
##  $ teachers   : num  10.9 11.1 82.9 14 71.5 ...
##  $ calworks   : num  0.51 15.42 55.03 36.48 33.11 ...
##  $ lunch      : num  2.04 47.92 76.32 77.05 78.43 ...
##  $ computer   : num  67 101 169 85 171 25 28 66 35 0 ...
##  $ expenditure: num  6385 5099 5502 7102 5236 ...
##  $ income     : num  22.69 9.82 8.98 8.98 9.08 ...
##  $ english    : num  0 4.58 30 0 13.86 ...
##  $ read       : num  692 660 636 652 642 ...
##  $ math       : num  690 662 651 644 640 ...
##  $ gradesN    : logi  TRUE TRUE TRUE TRUE TRUE TRUE ...

# Hmisc::describe(cas) # For more extensive summary statistics

Q. 2 To what extent does expenditure per student vary?

qplot(expenditure, data = cas) + xlim(0, 8000) + xlab("Money spent per student ($)") + 
    ylab("Count of schools")


round(t(psych::describe(cas$expenditure)), 1)

##            [,1]
## var         1.0
## n         420.0
## mean     5312.4
## sd        633.9
## median   5214.5
## trimmed  5252.9
## mad       487.2
## min      3926.1
## max      7711.5
## range    3785.4
## skew        1.1
## kurtosis    1.9
## se         30.9

The greatest expenditure per student is around double that of the least expenditure per student.

Q. 3a What predicts expenditure per student?

# Compute and format set of correlations
corExp <- cor(cas["expenditure"], cas[setdiff(v, "expenditure")])
corExp <- round(t(corExp), 2)
corExp[order(corExp[, 1], decreasing = TRUE), , drop = FALSE]

##          expenditure
## income          0.31
## read            0.22
## math            0.15
## calworks        0.07
## lunch          -0.06
## computer       -0.07
## english        -0.07
## teachers       -0.10
## students       -0.11
## gradesN        -0.17

More is spent per student in schools :

where people with greater incomes live
reading scores are higher
that are K-6

Q. 4 what is the relationship between district level maths and reading scores?

ggplot(cas, aes(read, math)) + geom_point() + geom_smooth()

At the district level, the correlation is very strong (r = The correlation is 0.92). From prior experience I'd expect correlations at the individual-level in the .3 to .6 range. Thus, these results are consistent with group-level relationships being much larger than individual-level relationships.

Q. 5 What is the relationship between maths and reading after partialling out other effects?

# command has strange syntax requiring column numbers rather than variable
# names
partial.r(cas[v], c(which(names(cas[v]) == "read"), which(names(cas[v]) == 
    "math")), which(!names(cas[v]) %in% c("read", "math")))

## partial correlations 
##      read math
## read 1.00 0.72
## math 0.72 1.00

The partial correlation is still very strong but is substantially reduced.

Q. 6 What fraction of a computer does each student have?

cas$compstud <- cas$computer/cas$students
describe(cas$compstud)

## cas$compstud 
##       n missing  unique    Mean     .05     .10     .25     .50     .75 
##     420       0     412  0.1359 0.05471 0.06654 0.09377 0.12546 0.16447 
##     .90     .95 
## 0.22494 0.24906 
## 
## lowest : 0.00000 0.01455 0.02266 0.02548 0.04167
## highest: 0.32770 0.34359 0.34979 0.35897 0.42083

qplot(compstud, data = cas)

## stat_bin: binwidth defaulted to range/30. Use 'binwidth = x' to adjust this.

The mean number of computers per student is 0.136.

Q. 7 What is a good model of the combined effect of other variables on academic performance (i.e., math and read)?

# Examine correlations between variables
psych::pairs.panels(cas[v])

pairs.panels shows correlations in the upper triangle, scatterplots in the lower triangle, and variable names and distributions on the main diagonal.
After examining the plot several ideas emerge.

# (a) students is a count and could be log transformed
cas$studentsLog <- log(cas$students)

# (b) teachers is not the variable of interest:
#   it is the number of students per teacher
cas$studteach <- cas$students /cas$teachers
# (c) computers is not the variable of interest:
#  it is the ratio of computers to students
# table(cas$computer==0) 
# Note some schools have no computers so ratio would be problematic.
# Take percentage of a computer instead
cas$compstud <- cas$computer / cas$students 

# (d) math and reading are correlated highly, reduce to one variable
cas$performance <- as.numeric(
        scale(scale(cas$read) + scale(cas$math)))

Normally, I'd add all these transformations to an initial data transformation file that I call in the first block, but for the sake of the narrative, I'll leave them here.

Let's examine correlations between predictors and outcome.

m1cor <- cor(cas$performance, cas[c("studentsLog", "studteach", "calworks", 
    "lunch", "compstud", "income", "expenditure", "gradesN")])
t(round(m1cor, 2))

##              [,1]
## studentsLog -0.12
## studteach   -0.23
## calworks    -0.63
## lunch       -0.87
## compstud     0.27
## income       0.71
## expenditure  0.19
## gradesN     -0.16

Let's examine the multiple regression.

m1 <- lm(performance ~ studentsLog + studteach + calworks + lunch + 
    compstud + income + expenditure + grades, data = cas)
summary(m1)

## 
## Call:
## lm(formula = performance ~ studentsLog + studteach + calworks + 
##     lunch + compstud + income + expenditure + grades, data = cas)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.8107 -0.2963 -0.0118  0.2712  1.5662 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  8.99e-01   4.98e-01    1.80    0.072 .  
## studentsLog -3.83e-02   1.91e-02   -2.01    0.045 *  
## studteach   -1.11e-02   1.59e-02   -0.70    0.487    
## calworks     1.96e-03   2.96e-03    0.66    0.508    
## lunch       -2.65e-02   1.48e-03  -17.97  < 2e-16 ***
## compstud     7.88e-01   3.86e-01    2.04    0.042 *  
## income       2.82e-02   4.89e-03    5.77  1.6e-08 ***
## expenditure  5.87e-05   4.90e-05    1.20    0.232    
## gradesKK-08 -1.21e-01   6.49e-02   -1.87    0.062 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 
## 
## Residual standard error: 0.457 on 411 degrees of freedom
## Multiple R-squared: 0.795,   Adjusted R-squared: 0.791 
## F-statistic:  199 on 8 and 411 DF,  p-value: <2e-16 
##

And some indicators of predictor relative importance.

# calc.relimp from relaimpo package.
(m1relaimpo <- calc.relimp(m1, type = "lmg", rela = TRUE))

## Response variable: performance 
## Total response variance: 1 
## Analysis based on 420 observations 
## 
## 8 Regressors: 
## studentsLog studteach calworks lunch compstud income expenditure grades 
## Proportion of variance explained by model: 79.48%
## Metrics are normalized to sum to 100% (rela=TRUE). 
## 
## Relative importance metrics: 
## 
##                  lmg
## studentsLog 0.009973
## studteach   0.016695
## calworks    0.177666
## lunch       0.492866
## compstud    0.025815
## income      0.251769
## expenditure 0.014785
## grades      0.010432
## 
## Average coefficients for different model sizes: 
## 
##                   1X        2Xs        3Xs        4Xs        5Xs
## studentsLog -0.08771 -0.0650133 -0.0558756 -0.0519312 -4.926e-02
## studteach   -0.11918 -0.0861199 -0.0629499 -0.0462155 -3.372e-02
## calworks    -0.05473 -0.0427576 -0.0324658 -0.0233760 -1.535e-02
## lunch       -0.03199 -0.0310310 -0.0301497 -0.0293300 -2.856e-02
## compstud     4.15870  3.0673338  2.2639604  1.6844348  1.287e+00
## income       0.09860  0.0850555  0.0726892  0.0614726  5.140e-02
## expenditure  0.00030  0.0001986  0.0001374  0.0001013  8.061e-05
## grades      -0.45677 -0.3345683 -0.2529014 -0.1981200 -1.628e-01
##                    6Xs        7Xs        8Xs
## studentsLog -4.626e-02 -4.252e-02 -3.833e-02
## studteach   -2.418e-02 -1.687e-02 -1.109e-02
## calworks    -8.399e-03 -2.612e-03  1.962e-03
## lunch       -2.785e-02 -2.718e-02 -2.654e-02
## compstud     1.034e+00  8.828e-01  7.884e-01
## income       4.250e-02  3.477e-02  2.821e-02
## expenditure  6.882e-05  6.206e-05  5.871e-05
## grades      -1.414e-01 -1.291e-01 -1.215e-01

Thus, we can conclude that:

Income and indicators of income (e.g., low levels of lunch vouchers) are the two main predictors. Thus, schools with greater average income tend to have better student performance.
Schools with more computers per student have better student performance.
Schools with fewer students per teacher have better student performance.

For more information about relative importance and the relaimpo package measures check out Ulrike Grömping's website.
Of course this is all observational data with the usual caveats regarding causal interpretation.

Now, let's look at some weird stuff.

Q. 8.1 What are common words in Californian School names?

# create a vector of the words that occur in school names
lw <- unlist(strsplit(cas$school, split = " "))

# create a table of the frequency of school names
tlw <- table(lw)

# extract cells of table with count greater than 3
tlw2 <- tlw[tlw > 3]

# sorted in decreasing order
tlw2 <- sort(tlw2, decreasing = TRUE)

# values as proporitions
tlw2p <- round(tlw2/nrow(cas), 3)

# show this in a bar graph
tlw2pdf <- data.frame(word = names(tlw2p), prop = as.numeric(tlw2p), 
    stringsAsFactors = FALSE)
ggplot(tlw2pdf, aes(word, prop)) + geom_bar() + coord_flip()

# make it log counts
ggplot(tlw2pdf, aes(word, log(prop * nrow(cas)))) + geom_bar() + 
    coord_flip()

The word “Elementary” appears in almost all school names (98.3%). The word “Union” appears in around half (43.3%).

Other common words pertain to:

Directions (e.g., South, West),
Features of the environment (e.g., Creek, Vista, View, Valley)
Spanish words (e.g., rio for river; san for saint)

Q. 8.2 Is the number of letters in the school's name related to academic performance?

cas$namelen <- nchar(cas$school)
table(cas$namelen)

## 
## 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 37 38 39 
##  1  4  9 26 28 31 33 27 30 45 38 28 36 30 18 10  5  4  6  3  1  2  2  2  1

round(cor(cas$namelen, cas[, c("read", "math")]), 2)

##      read math
## [1,] 0.03    0

The answer appears to be “no”.

Q. 8.3 Is the number of words in the school name related to academic performance?

cas$nameWordCount <- sapply(strsplit(cas$school, " "), length)
table(cas$nameWordCount)

## 
##   2   3   4   5 
## 140 202  72   6

round(cor(cas$nameWordCount, cas[, c("read", "math")]), 2)

##      read math
## [1,] 0.05 0.01

The answer appears to be “no”.

Q. 8.4 Are schools with nice popular nature words in their name doing better academically?

tlw2p  #recall the list of popular names

## lw
## Elementary      Union       City     Valley      Joint       View 
##      0.983      0.433      0.060      0.040      0.031      0.019 
##   Pleasant        San      Creek        Oak      Santa       Lake 
##      0.017      0.017      0.014      0.014      0.014      0.012 
##   Mountain       Park        Rio      Vista      Grove   Lakeside 
##      0.012      0.012      0.012      0.012      0.010      0.010 
##      South    Unified       West 
##      0.010      0.010      0.010

# Create a quick and dirty list of popular nature names
naturenames <- c("Valley", "View", "Creek", "Lake", "Mountain", "Park", 
    "Rio", "Vista", "Grove", "Lakeside")

# work out whether the word is in the school name
schsplit <- strsplit(cas$school, " ")
cas$hasNature <- sapply(schsplit, function(X) length(intersect(X, 
    naturenames)) > 0)
round(cor(cas$hasNature, cas[, c("read", "math")]), 2)

##      read math
## [1,] 0.09 0.08

So we've found a small correlation.

Let's graph the data to see what it means:

ggplot(cas, aes(hasNature, read)) + geom_boxplot() + geom_jitter(position = position_jitter(width = 0.1)) + 
    xlab("Has a nature name") + ylab("Mean student reading score")

So in the sample nature schools have slightly better reading score (and if we were to graph it, maths scores). However, the number of schools having nature names is actually somewhat small (n= 61) despite the overall quite large sample size.

But is it statistically significant?

t.read <- t.test(cas[cas$hasNature, "read"], cas[!cas$hasNature, 
    "read"])
t.math <- t.test(cas[cas$hasNature, "math"], cas[!cas$hasNature, 
    "math"])

So, the p-value is less than .05 for reading (p = 0.046) but not quite for maths (p = 0.083). Bingo! After a little bit of data fishing we have found that reading scores are “significantly” greater for those schools with the listed nature names.

But wait: I've asked three separate exploratory questions or perhaps six if we take maths into account.

$\frac{.05}{3} =$ 0.0167
$\frac{.05}{6} =$ 0.0083

At these Bonferonni corrected p-values, the result is non-significant. Oh well…

Review

Anyway, the aim of this post was not to make profound statements about California schools. Rather the aim was to show how easy it is to produce quick reproducible reports with R Markdown. If you haven't already, you may want to open up the R Markdown file used to produce this post in RStudio, and compile the report yourself.

In particular, I can see R Markdown being my tool of choice for:

Blog posts
Posts to StackExchange sites
Materials for training workshops
Short consulting reports, and
Exploratory analyses as part of a larger project.

The real question is how far I can push Markdown before I start to miss the control of LaTeX. Markdown does permit arbitrary HTML. Anyway, if you have any thoughts about the scope of R Markdown, feel free to add a comment.

Getting Started with R Markdown, knitr, and Rstudio 0.96

2012-05-17T14:31:00.000+10:00

This post examines the features of R Markdown using knitr in Rstudio 0.96. This combination of tools provides an exciting improvement in usability for reproducible analysis. Specifically, this post (1) discusses getting started with R Markdown and knitr in Rstudio 0.96; (2) provides a basic example of producing console output and plots using R Markdown; (3) highlights several code chunk options such as caching and controlling how input and output is displayed; (4) demonstrates use of standard Markdown notation as well as the extended features of formulas and tables; and (5) discusses the implications of R Markdown. This post was produced with R Markdown. The source code is available here as a gist. The post may be most useful if the source code and displayed post are viewed side by side. In some instances, I include a copy of the R Markdown in the displayed HTML, but most of the time I assume you are reading the source and post side by side.

Getting started

To work with R Markdown, if necessary:

Install R
Install the lastest version of RStudio (at time of posting, this is 0.96)
Install the latest version of the knitr package: install.packages("knitr")

To run the basic working example that produced this blog post:

Open R Studio, and go to File - New - R Markdown
If necessary install ggplot2 and lattice packages: install.packages("ggplot2"); install.packages("lattice")
Paste in the contents of the gist (which contains the R Markdown file used to produce this post) and save the file with an .rmd extension
Click Knit HTML

opts_knit$set(upload.fun = imgur_upload)  # upload all images to imgur.com

Prepare for analyses

set.seed(1234)
library(ggplot2)
library(lattice)

Basic console output

To insert an R code chunk, you can type it manually or just press Chunks - Insert chunks or use the shortcut key. This will produce the following code chunk:

```{r}

```

Pressing tab when inside the braces will bring up code chunk options.

The following R code chunk labelled basicconsole is as follows:

```{r basicconsole}
x <- 1:10
y <- round(rnorm(10, x, 1), 2)
df <- data.frame(x, y)
df
```

The code chunk input and output is then displayed as follows:

x <- 1:10
y <- round(rnorm(10, x, 1), 2)
df <- data.frame(x, y)
df

##     x    y
## 1   1 1.31
## 2   2 2.31
## 3   3 3.36
## 4   4 3.27
## 5   5 5.04
## 6   6 6.11
## 7   7 8.43
## 8   8 8.98
## 9   9 8.38
## 10 10 9.27

Plots

Images generated by knitr are saved in a figures folder. However, they also appear to be represented in the HTML output using a data URI scheme. This means that you can paste the HTML into a blog post or discussion forum and you don't have to worry about finding a place to store the images; they're embedded in the HTML.

Simple plot

Here is a basic plot using base graphics:

```{r simpleplot}
plot(x)
```

plot(x)

Note that unlike traditional Sweave, there is no need to write fig=TRUE.

Multiple plots

Also, unlike traditional Sweave, you can include multiple plots in one code chunk:

```{r multipleplots}
boxplot(1:10~rep(1:2,5))
plot(x, y)
```

boxplot(1:10 ~ rep(1:2, 5))

plot(x, y)

`ggplot2` plot

Ggplot2 plots work well:

qplot(x, y, data = df)

`lattice` plot

As do lattice plots:

xyplot(y ~ x)

Note that unlike traditional Sweave, there is no need to print lattice plots directly.

R Code chunk features

Create Markdown code from R

The following code hides the command input (i.e., echo=FALSE), and outputs the content directly as code (i.e., results=asis, which is similar to results=tex in Sweave).

```{r dotpointprint, results='asis', echo=FALSE}
cat("Here are some dot points\n\n")
cat(paste("* The value of y[", 1:3, "] is ", y[1:3], sep="", collapse="\n"))
```

Here are some dot points

The value of y[1] is 1.31
The value of y[2] is 2.31
The value of y[3] is 3.36

Create Markdown table code from R

```{r createtable, results='asis', echo=FALSE}
cat("x | y", "--- | ---", sep="\n")
cat(apply(df, 1, function(X) paste(X, collapse=" | ")), sep = "\n")
```

x	y
1	1.31
2	2.31
3	3.36
4	3.27
5	5.04
6	6.11
7	8.43
8	8.98
9	8.38
10	9.27

Control output display

The folllowing code supresses display of R input commands (i.e., echo=FALSE) and removes any preceding text from console output (comment=""; the default is comment="##").

```{r echo=FALSE, comment="", echo=FALSE}
head(df)
```

Control figure size

The following is an example of a smaller figure using fig.width and fig.height options.

```{r smallplot, fig.width=3, fig.height=3}
plot(x)
```

plot(x)

Cache analysis

Caching analyses is straightforward. Here's example code. On the first run on my computer, this took about 10 seconds. On subsequent runs, this code was not run.

If you want to rerun cached code chunks, just delete the contents of the cache folder

```{r longanalysis, cache=TRUE}
for (i in 1:5000) {
    lm((i+1)~i)
}
```

Basic markdown functionality

For those not familiar with standard Markdown, the following may be useful. See the source code for how to produce such points. However, RStudio does include a Markdown quick reference button that adequatly covers this material.

Dot Points

Simple dot points:

Point 1
Point 2
Point 3

and numeric dot points:

Number 1
Number 2
Number 3

and nested dot points:

A
- A.1
- A.2
B
- B.1
- B.2

Equations

Equations are included by using LaTeX notation and including them either between single dollar signs (inline equations) or double dollar signs (displayed equations). If you hang around the Q&A site CrossValidated you'll be familiar with this idea.

There are inline equations such as $y_i = \alpha + \beta x_i + e_i$.

And displayed formulas:

$$\frac{1}{1+\exp(-x)}$$

knitr provides self-contained HTML code that calls a Mathjax script to display formulas. However, in order to include the script in my blog posts I took the script and incorporated it into my blogger template. If you are viewing this post through syndication or an RSS reader, this may not work. You may need to view this post on my website.

Tables

Tables can be included using the following notation

A	B	C
1	Male	Blue
2	Female	Pink

Hyperlinks

If you like this post, you may wish to subscribe to my RSS feed.

Images

Here's an example image:

Code

Here is Markdown R code chunk displayed as code:

```{r}
x <- 1:10
x
```

And then there's inline code such as x <- 1:10.

Quote

Let's quote some stuff:

To be, or not to be, that is the question: Whether 'tis nobler in the mind to suffer The slings and arrows of outrageous fortune,

Conclusion

R Markdown is awesome.
- The ratio of markup to content is excellent.
- For exploratory analyses, blog posts, and the like R Markdown will be a powerful productivity booster.
- For journal articles, LaTeX will presumably still be required.
The RStudio team have made the whole process very user friendly.
- RStudio provides useful shortcut keys for compiling to HTML, and running code chunks. These shortcut keys are presented in a clear way.
- The incorporated extensions to Markdown, particularly formula and table support, are particularly useful.
- Jump-to-chunk feature facilitates navigation. It helps if your code chunks have informative names.
- Code completion on R code chunk options is really helpful. See also chunk options documentation on the knitr website.
Other recent posts on R markdown include those by :
- Christopher Gandrud
- Markcus Gesmann
- Rstudio on R Markdown
- Yihui Xie: I really want to thank him for developing knitr. He has also posted this example of R Markdown.

Questions

The following are a few questions I encountered along the way that might interest others.

Annoying `<br/>`'s

Question: I asked on the Rstudio discussion site: Why does Markdown to HTML insert <br/> on new lines?

Answer: I just do a find and delete on this text for now. Specifically, I have a sed command that extracts just the content between the body tags and removes br tags. I can then, readily incorporate the result into my blogposts.

sed -i -e '1,/<body>/d' -e'/^<\/body>/,$d' -e 's/<br\/>$//' filename.html

Temporarily disable caching

Question: I asked on StackOverflow about How to set cache=FALSE for a knitr markdown document and override code chunk settings?

Answer: Delete the cache folder. But there are other possible workflows.

Equivalent of Sexpr

Question: I asked on Stack Overvlow about whether there an R Markdown equivalent to Sexpr in Sweave?.

Answer: Include the code between brackets of “backtick r space” and “backtick”. E.g., in the source code I have calculated 2 + 2 = 4 .

Image format

Question: When using the URI scheme images don't appear to display in RSS feeds of my blog. What's a good strategy?

Answer: One strategy is to upload to imgur. The following provides an example of exporting to imgur.

Add the following lines of code near the top of the file:

``` {r optsknit}
opts_knit$set(upload.fun = imgur_upload) # upload all images to imgur.com
```

I found that the function failed when I was at work behind a firewall, but worked at home.

How to plot three categorical variables and one continuous variable using ggplot2

2012-05-03T22:31:00.000+10:00

This post shows how to produce a plot involving three categorical variables and one continuous variable using ggplot2 in R.

The following code is also available as a gist on github.

1. Create Data

First, let's load ggplot2 and create some data to work with:

library(ggplot2)

set.seed(4444)
Data <- expand.grid(group=c("Apples", "Bananas", "Carrots", "Durians", 
            "Eggplants"),
            year=c("2000", "2001", "2002"),
            quality=c("Grade A", "Grade B", "Grade C", "Grade D", 
            "Grade E"))
Group.Weight <- data.frame(
    group=c("Apples", "Bananas", "Carrots", "Durians", "Eggplants"),
    group.weight=c(1,1,-1,0.5, 0))
Quality.Weight <- data.frame(
    quality=c("Grade A", "Grade B", "Grade C", "Grade D", "Grade E"),
    quality.weight = c(1,0.5,0,-0.5,-1))
Data <- merge(Data, Group.Weight)
Data <- merge(Data, Quality.Weight)
Data$score <- Data$group.weight + Data$quality.weight + 
    rnorm(nrow(Data), 0, 0.2)
Data$proportion.tasty <- exp(Data$score)/(1 + exp(Data$score))

2. Produce Plot

And here's the code to produce the plot.

ggplot(data=Data, 
       aes(x=factor(year), y=proportion.tasty, 
           group=group,
           shape=group,
           color=group)) + 
               geom_line() + 
               geom_point() +
               opts(title = 
               "Proportion Tasty by Year, Quality, and Group") +
               scale_x_discrete("Year") +
               scale_y_continuous("Proportion Tasty") + 
        facet_grid(.~quality )

And here's what it looks like:

Getting Started with JAGS, rjags, and Bayesian Modelling

2012-04-11T15:50:00.001+10:00

This post provides links to various resources on getting started with Bayesian modelling using JAGS and R. It discusses: (1) what is JAGS; (2) why you might want to perform Bayesian modelling using JAGS; (3) how to install JAGS; (4) where to find further information on JAGS; (5) where to find examples of JAGS scripts in action; (6) where to ask questions; and (7) some interesting psychological applications of Bayesian modelling.

What is JAGS?

JAGS stands for Just Another Gibbs Sampler. To quote the program author, Martyn Plummer, "It is a program for analysis of Bayesian hierarchical models using Markov Chain Monte Carlo (MCMC) simulation..." It uses a dialect of the BUGS language, similar but a little different to OpenBUGS and WinBUGS.

Why JAGS?

The question of why you might want to use JAGS can be approached in several different ways:

Why Bayesian rather than Null Hypothesis Significance Testing (NHST) approaches?
- To quote John D. Cook quoting Anthony O'Hagan, the benefits of "the bayesian approach are that it is 1. fundamentally sound, 2. very flexible, 3. produces clear and direct inferences, and 4. makes use of all available information." (see John's blog post for elaboration)
- John K. Kruschke made a similar argument in an Open Letter extolling the benefits of the bayesian approach summarised as: "(1) Scientific disciplines from astronomy to zoology are moving to Bayesian data analysis. We should be leaders of the move, not followers. (2) Modern Bayesian methods provide richer information, with greater flexibility and broader applicability than 20th century methods. Bayesian methods are intellectually coherent and intuitive. Bayesian analyses are readily computed with modern software and hardware. (3) Null-hypothesis significance testing (NHST), with its reliance on p values, has many problems. There is little reason to persist with NHST now that Bayesian methods are accessible to everyone."
Why JAGS/BUGS rather than coding in a low-level language?
- It's simpler; for models that BUGS can handle, BUGS can shield you from some of the thorny details related to numeric integration.
- There are simple interfaces with R.
Why JAGS rather than WinBUGS or OpenBUGS?
- I'm using JAGS because it works well on Ubuntu. WinBUGS is broadly Windows specific, although I've read that it may work with the emulation software Wine.
- JAGS interfaces well with R. I'm comfortable writing scripts. Thus, I don't personally see the benefits of using a dedicated GUI like WinBUGS. I can leverage what I know about R.
- However, ultimately converting code between different flavours of BUGS is not that difficult.
- For further discussion of the issue, see this r-help discussion and this discussion on CrossValidated.

More than anything I found that JAGS provided a useful entry point into the world of Bayesian modelling. This in turn appealed to me for several reasons:

Even when I perform analyses using an NHST approach I often intuitively think of empirical research questions in terms of probability densities on a parameter of interest that changes as empirical and theoretical evidence is accumulated. See for example Thompson's (2002) concept of meta-analytic thinking. Bayesian analysis provides tools for formalising this orientation.
More broadly, I appreciate the explicitness that a Bayesian approach requires and encourages. E.g., specifying the distribution of the error term, specifying a prior, specifying the distribution of parameters in a mixed effects model, and so on.
There are several modelling challenges that I'm currently working through where a Bayesian approach offers substantial flexibility and applicability. In particular, I'm interested in modelling individual differences in the effect of practice on strategy use and task performance and then relating these individual differences to factors like intelligence, prior experience, and personality.

JAGS Installation

JAGS runs on Linux, Mac, and Windows. I run JAGS on Ubuntu through an interface with R called rjags.

The following sets out a basic installation process:

If necessary Download and install R and potentially a user interface to R like R Studio (see here for tips on getting started with R).
Download and install JAGS as per operating system requriements.
Install additional R packages: e.g., in R install.packages("rjags") . In particular, I use the packages rjags to interface with JAGS and coda to process MCMC output.

Information on JAGS

The manual for different versions of JAGS is located here. e.g., the pdf of the manual for 3.1.0. Several particularly relevant sections include:
- the list of supported distributions and how they are parameterised. This is often important given that the code looks similar to R but often uses different parameterisation (e.g., precision is used instead of standard deviation for a normal distribution).
- It summarises differences between WinBUGS and JAGS.
- It sets out available functions and operators.
The rjags help pdf for information about how to interface with JAGS from R.
Martin Plummer has a blog called JAGS NEWS
The Bayesian Task View on CRAN lists and briefly describes the many R packages related to Bayesian statistics.
Lunn and colleagues have a 2009 article called The BUGS project: Evolution, critique and future directions. It provides a useful historical perspective on the broader BUGS project, although it does not mention much about JAGS specifically.

Examples JAGS Scripts

I find it easier to pick up a new language by playing with examples. The following provides links to example JAGS code, often with accompanying explanations:

John Myles White
- A course on statistical models that is under development with JAGS scripts on github
- A model of Cannabalt scores using a gamma distribution
- Simple introductory examples of fitting a normal distribution, linear regression, and logistic regression
- A follow-up post demonstrating the use of the coda package with rjags to perform MCMC diagnostics.
John K. Kruschke
- John Krushke wrote a book called Doing Bayesian Data Analysis: A Tutorial with R and BUGS. It's an excellent entry point into the world of Bayesian statistics for the social and behavioural scientist who has reasonable quantiative training, but is not necessarily ready to absorb the kinds of books that are used in graduate-level statistics courses.
- The book has a website that provides all the examples used in the book all the examples used in the book. See this blog post for a link to the zip file containing the JAGS code.
BUGS Project
- BUGS is well known for the large set of examples that accompany the project.
- The PDF providing documentation for Volume 1 and 2 of the examples is available here.
- You can see the JAGS code used to run these examples here.
Patrick J Mineault
- An example from Gelman et al examining the effect of training programs on SAT scores
Miguel Lobo
- A short tutorial
Simon Jackman
- Simon Jackman wrote the book Bayesian Analysis for the Social Sciences that has accompanying JAGS code.
- The book's website has several useful resources including example papers using Bayesian methods.
- An associated course that uses the book as a text book has slides and many examples of using and R and JAGS.
Johannes Karreth
- A course on applied bayesian modelling with examples of data, and code using the R2jags interface.
Myself
- I also plan to post a few examples in upcoming blog posts. I typically will share the code for these on my github account: jeromyanglim. If you are reading this through syndication you may wish to subscribe to the RSS feed of the source blog jeromyanglim.blogspot.com.

More broadly, examples and tutorials designed for WinBUGS can generally be adapted to be useful for JAGS. So for example, you can explore these WinBUGS examples:

Michael Lee and Eric-Jan Wagemakers have a free online book called A Course in Bayesian Graphical Modeling for Cognitive Science: see PDF and website.
The website for the book Markov Chain Monte Carlo has several WinBUGS examples.
There is an extensive list of BUGS resources on the BUGS project website.

Asking questions

There are several places to ask questions about JAGS, R, and Bayesian statistics.

JAGS, BUGS, and bayesian questions on stats.stackexchange.com (aka CrossValidated).
JAGS discussion forum
There's also a BUGS discussion list

In general, I prefer the Stack Exchange model for asking and answering questions on the internet, although the most important issue is typically where the experts are located.

Interesting Psychological Applications of Bayesian Modelling

If you want to see some examples of Bayesian modelling applied to psychological data, I found the following articles quite interesting. PDFs are available online.

Shiffrin, Lee, Kim, and Wagenmakers (2008, PDF) present a tutorial on hierarchical bayesian methods in the context of cognitive science.
Michael Lee (2011, PDF) in Journal of Mathematical Psychology discusses the benefits of hiearchical Bayesian methods to modelling psychological data and provides several example applications.
Lee Averell and Andrew Heathcote (2010, PDF) in Journal of Mathematical Psychology analyse individual differences in the forgetting curve using a hierarchical Bayesian approach.

If you know of any other interesting JAGS resources or have any comments about my choice of software for Bayesian data analysis, feel free to post a comment.

New Psychology and Cognitive Science Question and Answer Site: COGSCI.SE

2012-02-17T21:21:00.000+11:00

There is now a new website for researchers to ask and answer questions on topics related to psychology and cognitive science. The site is cogsci.stackexchange.com. From the success of earlier released sites in the Stack Exchange network such as those on programming, statistics, and latex, the site for psychology and cognitive science has the potential be a great resource for researchers. I'm actively contributing on the site. So, if you are a researcher in psychology, I hope you'll check it out. The rest of this post sets out (a) a little history of Stack Exchange question and answer sites as they relate to psychology and statistics; (b) why I think this new site for psychology and cognitive science has so much potential; and (c) why, if you are a professional or student researcher in psychology, you might want to get involved.

A little history

I first learnt about the Stack Exchange network back in 2009. While I was busy learning R, a number of people in the online data science world such as JD Long, Michael Driscoll, Drew Conway, and many more were promoting a programmer's question and answer site called Stack Overflow as a place to ask and answer R related questions. It was a site pitched at overcoming the many problems of discussion boards, mailing lists, and the like: e.g., off topic threads, spam, extended discussion difficulty finding the correct answers, poor indexing by Google, etc. As of Feb 2011, it now has over 10,000 questions with the R tag.

Shortly afterwards, the Stack Exchange Network developed Area51. The idea was to take the question and answer infrastructure that made Stack Overflow a success in the programming world, and extend it to all sorts of other domains. Instead of going down the model of Quora or, shudder to think, Yahoo Answers, Stack Exchange did not permit the creation of a site until a sufficient community of active users existed to maintain the site at a high standard. Thus, my main interests, statistics and psychology were going to have to wait.

A site for statistics questions was the first to join the network (stats.stackexchange.com). Professor Rob Hyndman proposed the site, and perhaps given the overlapping worlds of programmers and data analysts, the site launched a few months later in July 2010. At the time of posting it has over 7,000 questions. I've been actively involved in the site. I've used it to get advice on my own research. I've also used it extensively in various statistical consulting roles. In particular, I've encouraged others who would otherwise send me an email about statistics, to post the question on stats.se so that any answer can be an ongoing resource for others.

In the case of psychology and cognitive science, I've had to wait a lot longer. The overlap between programming and psychology communities is much smaller, and site proposals were split over separate cognitive science, psychology, and psychiatry proposals. Finally, in December 2011 these three proposals were merged and on January 19th 2012 the site was launched in Beta under the title Cognitive Sciences at the url cogsci.stackexchange.com. Although the initial name is suggestive of a focus on "cognitive" science. The history of the merging of site proposals, the inclusion of the "s", plural "sciences", and the attitude of current participants admits the full range of questions in cognitive science, psychology, and psychiatry.

At the time of posting the site is growing at a healthy rate. Most questions are getting good answers, and the community norms around question quality, references, scope and so on are being clarified on the meta site. However, there is also the challenge of getting the word out about the site to academics, researchers, and graduate students who are not otherwise familiar with the Stack Exchange network of sites. In my opinion, Stack Exchange provides the best currently available infrastructure for building a high quality question and answer site. However, it still relies on a community of expert contributors.

So, if you're a researcher in psychology or cognitive science, why might you want to get involved? And why might you want to talk to fellow researchers about the?

Reasons to participate as an academic

If you are an academic, Lecturer, or Post Doc, there are many reasons why you might want to participate:

Answering questions is a way of facilitating knowledge transfer to the broader community; this can be intrinsically enjoyable especially when you get direct feedback on the number of people who read your answers.
If you use your own name, as many people do, the voting and reputation system, and various other mechanisms provide a means for your contribution to be recognised.
You get immediate feedback on what others think of your answers; Thus, it creates an environment of feedback conducive to learning.
I see sites like Stack Exchange as part of a broader model of open science. As you create and develop knowledge, you encounter challenges. The idea is to record these challenges as questions and then add the resolutions as answers. Thus, when others encounter the same problems, good answers are only a Google search away. I'm not saying that question and answer sites replace journal articles, but they can fill a bridging role linking the language of questions to the answers provided in journal articles.
Furthermore, the content on Stack Exchange is licenced under creative commons, so even if the site disappeared the content would still be available on other sites that reproduce the material. This is much better than the policy of almost all journals which copyright your, typically, state-sponsored research and lock it up behind a pay wall, thereby frustrating the process of knowledge dissemination.
While contributing to Wikipedia is another great way to improve the sum of all knowledge, unlike Wikipedia, your answers generally stay there; in contrast to Wikipedia, where your contributions can and are often completely removed by other editors.

Reasons to participate as a student researcher

If you are doing a thesis in psychology or cognitive science, or possibly even if you are just studying a few subjects, many of the above reasons for participating will also apply.

However, you may also find that the capacity to ask questions will be particularly useful. As a side point, if it is early days in your career, you may or may not want to use your real name.

In particular, I'd encourage students completing a thesis to incorporate asking and answering questions into their scholarly process. You might encounter questions like:

Is there a meta analysis on X?
What are the main theory about Y?
What is the best measure of Z?
What is the empirical support for theory W?

These kinds of questions come up all the time when doing research. Of course, as researchers we have strategies for finding answers ourselves. However, the stack exchange model encourages you to learn from the answers of others and also to "leave crumbs" so that others can follow in your footsteps more easily. The idea is not be shy. Post questions frequently. If you're able to answer your question, contribute a self-answer.

Thus, even if only a handful of people ever read a thesis, by asking and answering many questions along the way, resources will be left that thousands of people will learn from and discover through Google searches in the years to come. Even if you don't have answers, your question can be the trigger for an expert to share knowledge to create a valuable Internet artefact.

Getting Started

If you want to learn more or give the site a go:

Have a read through the FAQ on cogsci.se
Ask a question
See if you can answer one of the currently unanswered questions

I'm floating around on the site, so if you're a researcher in psychology, I hope to see you there.

Tips for Undergraduates Interested in a Career in Organisational Psychology: Australian Perspective

2011-07-24T21:40:00.000+10:00

Undergraduate psychology students often ask me about careers in organisational psychology. This post aims to provide a few links and resources to assist such students to learn about the profession and the career pathways. The post includes (a) a basic description of organisational psychology, (b) links to Australian educational and professional society resources, (c) discussion of PhD and academic options, and (d) additional resources to learn more about the profession.

Overview of Organisational Psychology

What is the profession called?

Before discussing the profession some consideration should be given to what to call it. 'Organisational psychology' goes by various names and abbreviations:

Organisational Psychology (Org Psych)
Industrial/Organisational Psychology (I/O or I/O Psych)
Work Psychology
Occupational Psychology

Different names imply both historical and present differences in focus. However, such terms are also often used interchangeably. See SIOP's 'What's in a Name?' article for an overview of various job titles.

Names vary by region. In the United States, "I/O" is preferred. In Australia, "Organisational Psychology" is arguably the more common term, consistent with the APS college name and many course names. Thus, I'll tend to use this term in this post.

What is organisational psychology?

Here are a few descriptions:

"Organisational Psychology is the science of people at work. Organisational psychologists specialise in analysing organisations and their people, and devising strategies to recruit, motivate, develop, change and inspire." - prize winning elevator pitch (APS COP)
Industrial / Organisational psychologists "Apply principles of psychology to personnel, administration, management, sales, and marketing problems. Activities may include policy planning; employee screening, training and development; and organizational development and analysis. May work with management to reorganize the work setting to improve worker productivity. - Industrial/Organisational Psychologist job description on O*N
"Industrial-organizational (I-O) psychology is the scientific study of the workplace. Rigor and methods of psychology are applied to issues of critical relevance to business, including talent management, coaching, assessment, selection, training, organizational development, performance, and work-life balance." - SIOP: Student Section

The APS College of Organisational Psychology has a page that describes "What is an organisational psychologist" and "Areas of Specialisation".

Learning more about the profession:

A good strategy for learning more about the profession is to browse the various society pages:

SIOP - Division of the American Psychological Association: The United States is huge; and I/O is huge in the United States. The SIOP web page has heaps of useful online resources.
Division of Occupational Psychology: British Psychological Society
Australian Psychological Society: College of Organisational Psychologists

Organisational Psychology in Australia

Registration

"Psychologist" is a regulated term in Australia. It is illegal to call yourself a psychologist, if you are not appropriately registered.
Pathways to registration are set out by the Psychology Board of Australia.
The traditional pathway for registration has involved first completing a four year accredited undergraduate psychology sequence, followed by either two years of supervised practice or the completion of an accredited post-graduate program (e.g., Masters, Doctorate, Masters / PhD). Over recent years, rules for registration have been changing. So, make sure you do your own research.
I should also mention that even if you can't call yourself a "psychologist", completing an undergraduate major in psychology, particularly one with honours in psychology (and perhaps also an undergraduate subject in organisational psychology) can open doors to many roles related to organisational psychology (e.g., HR, selection and recruitment, marketing research, etc.).

Finding organisational psychology university programs in Australia:

The APAC accreditation site lists approved postgraduate psychology programs.
To find organisational psychology courses, last I checked, the following worked
- Click Search for courses - Australia
- Click on the State you want to search for
- Search for "org"

Groups and networking opportunities

The Australian Psychological Society: College of Organisational Psychologists is the main group representing organisational psychologists in Australia.
It is made up of various state branches.
The society sometimes runs sessions suited to students wanting to learn more (e.g., careers fairs).

A few informal online groups are also good places to learn more about the profession in Australia. Both welcome professionals and students:

Facebook group: Organisational Psychology in Australia
LinkedIn group: Organisational Psychology in Australia

Salary Surveys

There are many reasons to find a career in organisational psychology intellectually stimulating and meaningful. There has also often been financial reasons to find it attractive:

A career in academia

PhD on a topic related to organisational psychology or related area

Doing a PhD on a topic related to organisational psychology can create many opportunities. Such a PhD can open up doors to academic positions in a wide range of departments including, psychology, HRM, management, business, and so on. The solid background in statistics and research methods provides a particular advantage for an academic career. Of course, academic positions are competitive and generally require a good publication track record.
Choosing a good PhD supervisor is important. In addition to supervisors in departments that offer organisational psychology programs, it's possible to look at supervisors in departments and universities that don't offer such programs.
The skills learnt can also readily be applied in many social science research related roles in industry.

Examples of eminent organisational psychology academics

For those considering pursuing an academic career related to organisational psychology, the past SIOP award recipients, particularly in the categories Distinguished Scientific Contribution, and Distinguished Early Career Contribution, provide motivating examples of successful I/O psychology researchers.

Example academic websites

The following links point to examples of successful academics in I/O psychology.
I also selected these particular pages because each one provides PDFs for many of the respective academic's publications. This can give a flavour of the kind of work, focus, and specialisation that an academic in I/O might engage in.

Journals to read

Further understanding of the research done in organisational psychology and related disciplines can be gained from reading some of the core journals. A good starting point can be gained by perusing the following ranked list of journals generated by Michael Zickar and Scott Highhouse back in 2001 based on a survey of SIOP members:

Journal of Applied Psychology
Personnel Psychology
Academy of Management Journal
Academy of Management Review
Organizational Behavior and Human Decision Processes
Administrative Science Quarterly
Journal of Management
Journal of Organizational Behavior
Organizational Research Methods
Journal of Vocational Behavior

Additional Resources

Richard Landers has a series of posts providing advice on pursuing a career in I/O psychology from the U.S. perspective:
TIP is the official newsletter of SIOP. Current and back issues are available online and provide a good insight into the profession including the interface between professional practice and scientific research.

Correlation Resources: SPSS, R, Causality, Interpretation, and APA Style Reporting

2011-07-17T22:30:00.000+10:00

This post provides links to a range of resources related to the use and interpretation of correlations. I wanted to provide a page with links to a number of additional resources that would be useful both for those of my students who might be keen to learn more and for anyone else who might be interested. Specifically, this post provides links to: (a) introductory book-style chapters on correlation, (b) resources related to assorted issues in correlation (i.e., discussion of causal inference, correlation with various variable types, range restriction, statistical power, correlation interpretation, and significance testing), (c) tutorials on computing correlations using SPSS and R, and (d) tips for reporting correlations in APA Style.

Introductions to correlation

The following provide general textbook style overviews of correlation:

David Kenny's Chapter 16 Testing Measures of Association provides a textbook overview of correlation designed for psychology undergraduate students. It also includes several practice questions. David Kenny has kindly made his entire textbook 'Statistics for the Social and Behavioral Sciences' available online for free as either an overall pdf or individual chapters.
David Stockburger's Introductory Statistics chapter on Correlation
My own slides and notes on correlation

Assorted Issues

Correlation and Causation

Knowing how to reason about causality in the behavioural and social sciences is a really important skill.

Check out this earlier post on correlation and causation which includes links to PDFs of important journal articles on the topic.
Joy of Stats on Correlation provides a 4 minute video with a few entertaining examples of correlations and their connection with causal inference.

Types of variables

The prototypical correlation example is based on two continuous, normally distributed variables. However, in practice there are many other types of variables that you might wish to correlate. The following provide pages provide links to suggestions for how to analyse some other common scenarios:

Range restriction

HyperStat has a general discussion of range restriction
See this simulation on connexions showing the effect of range restriction

Statistical Power

Statistical power within the context of correlation is the probability of obtaining a statistically significant correlation in a study given that a true correlation exists.

This earlier post provides (a) some simple rules of thumb for power analysis for correlations, (b) how to calculate statistical power using free software called G-Power, and (c) links to additional reading on the important topic of statistical power.

Interpretation

When I first learnt about the correlation coefficient, I found it challenging to truly grok what a particular value meant. Learning the standard interpretation was easy. The challenging part was understanding the practical and theoretical implications for a correlation of a given size.

The following are some of the standard interpretations of a correlation:
- Pearson's correlation is an index of the direction and strength of linear association between two variables.
- The square of the correlation between X and Y is the percentage of variance shared between X and Y (e.g., if r = .50, then the two variables share .50 * .50 = 25% of variance).
- If X and Y were standardised (i.e., made so that the mean of both variables was zero and the standard deviation was one) then, the correlation would be the same as the regression coefficient of X predicting Y or Y predicting X. Thus, for example, if r = .25 you could say that "a value one standard deviation greater on X predicts a .25 standard deviation greater value on Y".
Strategies for building an intuition of what a correlation means:
- Play with the Regression by Eye simulation. The simulation generates a scatterplot, and you are asked to indicate which of a set of correlations corresponds to the scatterplot. It helps to build a mapping between the graphical intuitiveness of a scatterplot and the numeric summary of the linear association in the scatterplot (i.e., the correlation coefficient).
- Memorise some of the rules of thumbs for describing correlation effect sizes (see this discussion by Andy Field), but don't take the rules of thumb too seriously.
- Try to build up a frame of reference for correlations in different contexts by reading results sections. Meta analyses can also be particularly useful in this regard.
- Read the article 'Meyer, G. J., et al (2001). Psychological Testing and Psychological Assessment: A Review of Evidence and Issues. American Psychologist, 56(2), 128-165.' (PDF) which provides large tables of meta-analytic correlations for a wide range of medical and psychological domains sorted by the size of the correlation. Studying these tables can help build an intuition and a context for interpretation of correlations.

Graphical approaches

As with most statistical techniques, there are various ways of representing the data. The correlation coefficient provides a very brief summary of the association between two variables. However, graphical representations of association are much richer.

The following are some general heuristics that I find useful when plotting data that might also be represented as a correlation:

Use scatterplots to explore features of the association (e.g., presence of outliers, linearity, distributional properties, spread of data around any trend line, etc.);
If one of the variables is positively skewed, consider plotting the corresponding axis on a log scale;
If there are a lot of data points (e.g., n > 1000), adopt a different strategy such as using some form of partial transparency (e.g., see use of the alpha property in ggplot2), or sampling the data;
If one of the variables takes on a limited number of discrete categories, consider using a jitter or a sunflower plot;
If there are three or more variables, consider using a scatterplot matrix;
Fitting some form of trend line is often useful;
Adjust the size of the plotting character to the sample size (for bigger n, use a smaller plotting character).

Significance tests on correlations

There are a wide range of possible significance tests that can be performed on correlations. The following links provide some suggestions and links for different scenarios.

General post on comparing significance of two correlations under various conditions.
Significance of correlation using Pearson's table

Statistical Software

Calculating a correlation coefficient and its associated statistical significance is a standard task that almost any statistical package can perform. Many psychology students are taught to use SPSS. It is a proprietary (i.e., you can't run it at home without a paid licence) data analysis system with a strong empahsis on a GUI and making it easy to perform various standardised analyses common in the social sciences.

My preferred tool for performing data analysis is R. It is open source (thus, you can run it at home for free) and is often described as the lingua franca of statistics. It generally requires a more sophisticated understanding of statistics and computing to use effectively. Thus, for the interested psychology student or researcher I have this introduction to R for researchers in psychology.

Below I list resources for performing correlation analysis in SPSS and R.

SPSS

Andy Field has a chapter on correlation which discusses correlation using SPSS.
This video tutorial on running and interpreting a correlation analysis using SPSS goes for about 7 minutes and is elementary.

R

R makes it easy to perform correlations on datasets. Specifically, the following links provide example syntax:

Quick-R on correlations
Quick-R on scatterplots
More generally, William Revelle has some great resources on R for psychology.

Reporting Correlations in APA Style

APA Style Manual: When required to report results using APA style, the authoritative source is the Publication Manual of the APA.
Article Deconstruction: Another general strategy is to find a journal article that (a) reports a similar statistical test as you require, and (b) that is published in an APA journal or at least is in a journal that uses APA style.
- APA journals are listed here
- A quick search on Google Scholar will often be sufficient and quicker, although PsycInfo (a subscription service) is more reliable if you have access to it (many universities do). E.g., a quick search for apa "significant correlation between" psychology revealed several relevant articles and some with immediate PDF access.
- I also have a separate post on this general approach of deconstructing journal articles to discern writing principles.
Correlation Matrices: Many psychological studies, particularly those based on correlational/observational designs, involve the measurement of a range of numeric variables. It is particularly useful, and common, in such cases to report a correlation matrix between sets of variables. I have a post with instructions on formatting a correlation matrix in APA style using a combination of SPSS, Excel, and Word. The post also includes links to examples of correlation matrices being reported.
General overview of reporting statistics including correlations using APA style

Transition to Linux and Ubuntu: The Experience of a Quantitative Social Scientist

2011-05-23T16:14:00.000+10:00

This post discusses my experience migrating to Ubuntu from Windows from the perspective of a quantitatively oriented researcher in psychology. It discusses (a) my general transition to open source tools for doing research; (b) examination of Cygwin; (c) choosing a Linux distribution; (d) preparing for installation of Ubuntu; (e) initial reflections on installing Ubuntu; and (f) a long list of various solutions to specific problems that I encountered whilst getting started with Ubuntu.

Context of transition

Anyone following this blog for a while will know that I've been progressively migrating to using open source software for doing research. My first foray into open source started in 2007 when I decided to replace SPSS with R.

As is the case with many open source tools, they have a habit of mutually reinforcing each other. The path went something like this (with links pointing to associated blog posts):

LaTeX replaced Word as my scientific document format.
Markdown replaced Word as my document format for note taking and informal documents.
Bibtex, Jabref, and other tools replaced Endnote as a bibliographic reference management system.
Vim replaced a host of other IDEs, and text editors; it even replaced MS Word, in that it became the editor for the plain text file formats (e.g., LaTeX, Sweave, Markdown, etc.) that I use the most.
git replaced ad hoc version control systems.
make replaced various manual processes for building final products.

For a general summary of this workflow (albeit, prior to the adoption of Vim), see this video that I gave at Melbourne R Users in late 2010.

However, I still needed to cut ties from Windows, the operating system I'd been using for 15 years. I'd been putting off adopting Linux for several reasons. I was concerned about:

hardware support for my main computer,
the temporary drop in performance that results from adopting a new operating system,
difficulties collaborating with other researchers operating in the Windows world,
lack of support for various key programs (e.g., Outlook; Inquisit), and
the possibility that some tasks would be less efficient under Linux.

Despite these concerns, I decided to finally dive in.

Initial foray into Cygwin

About a month prior to making the switch I'd installed Cygwin on Windows. This gave me access to many Linux command-line programs. However, ultimately, while Cygwin whet my appetite, it failed to satisfy.

By default, Cygwin does not always play nicely with non-Cygwin programs. File paths often get confused. If it can't play nicely with Windows programs, I might as well go all the way.
There are issues of where to store files (Cygwin home or Windows user home).
Some Linux programs are not included.
The speed of the terminal and shell was often poor.
In general, interacting with other programs was a frustrating experience.

While some of these challenges probably could have been overcome, it felt silly investing time in learning the idiosyncrasies of a system that only goes half-way to creating a genuine unix experience.

Choosing a Linux distribution

The first decision involved choosing a Linux distribution. I decided on Ubuntu for several reasons.

It's a popular Linux distribution.
It seems like a good distribution for people new to Linux.
Hardware support looked encouraging.
I'd heard it was well suited as a personal desktop
I'd also heard that ultimately Linux is configurable and thus, the distribution does not provide major constraints; furthermore basic unix command-line programs should work in all distributions.

Preparing to install Ubuntu

I approached the installation of Ubuntu with trepidation.

I read a lot about partitioning and dual-booting on Ubuntu documentation and psychocats.
I backed up my data (of course) before starting installation.
I thought a lot about setting up my partitions. In the end I decided on a dual-boot partition. I have a solid state drive which is really fast, but capacity is not that high (only 120gb). Thus, I had to be careful with assigning space. I gave 40gb to Windows and 80gb to Ubuntu. This required the removal of many unwanted files from the Windows partition.
I created and trialled the Ubuntu Live USB to check for hardware support, and everything seemed to be working surprisingly well on my Dell Latitude laptop.
I then installed Ubuntu, letting the Ubuntu installer handle the hard drive partitioning.

To my pleasant surprise, the installation was smooth, involving minimal choices. The Ubuntu installation successfully partitioned my hard drive, and the core features of my laptop still worked (sleep, hibernate, wifi, volume buttons, graphics card, and so on). Furthermore, I was still able to boot into Windows for when I really needed to use Windows-only software.

Reflection on the installation

Switching operating systems is painful even when the new operating system is better than the old one. Many little tasks that were once automatic need to be automated in a new way. Furthermore, to fully grok the new system, it is often necessary to do more than map the old workflow onto the new system.

For the most part the switch to Ubuntu has been smoother than I was expecting. The first week involved dealing with a wide range of fundamental issues related to software installation and system configuration. However, in general, these issues have been resolved by a few Internet searches and asking questions on Ask Ubuntu (a Stack Exchange Q+A site).

The following were some of the main issues I encountered:

Linking with my university email / calendar Exchange server has been painful. After two weeks, this still remains my biggest pain; The email program Evolution basically works, but is painfully slow at times, often freezes, and often produces errors.
Configuring the Unity desktop (the default behaviour of the 11.04 launcher is awful) took a bit of searching.
Configuring so that I could move between laptop monitor configurations required some fiddling.
Configuring my vimrc took some work. I had a number of file format issues related to text files having inappropriate line feeds (i.e., Windows text files instead of Unix text files).

Despite these issues I'm really enjoying using Ubuntu. Adopting a Vim + command-line workflow is so much easier, and I'm gradually incorporating some of the great Linux tools into my workflow.

Some reading

As part of the transition, I've also been reading a few books:

Harley Hahn's guide to Unix and Linux - Harley Hahn
- This one was quite pedagogical in flavour and a bit wordy at times, but did provide a helpful introduction to Linux.
UNIX power tools - Jerry Peek, Tim O'Reilly, Mike Loukide
- This book is massive.
- It provides heaps of short articles outlining useful tricks for getting more out of Unix / Linux.
Learning the bash Shell - Cameron Newham, Bill Rosenblatt

Looking forward

As I get settled into the new operating system I'm looking forward to really incorporating the tools available into my research workflow.

Some of the things I'm looking forward to (which I may blog about at some point, see here for Blog RSS subscription options) include:

Writing some bash scripts and Vim commands to enable me to link bibtex citations to stored pdfs of journal articles;
Taking advantage of the command-line for file manipulation, running R, creating LaTeX documents, and more; and
Dabbling in sed, awk, Python for better text manipulation.

Now with the first two weeks of initial adjustment out of the way, the subtler phase of tweaking and refinement begins. If you've made the switch to Linux particularly within the context of research and data analysis, I'd be keen to hear your experience.

Assorted Resolved Issues

Overview

The rest of this post documents a wide range of problems that I encountered when getting started with Ubuntu and the solutions that I adopted. The caveat applies that I'm pretty new to Ubuntu, so the solutions may not be best practice. Furthermore, they may not make for interesting reading unless perhaps you've stumbled onto this post through Google searching for the specific problem.

Overhaul file system

Question

What are the implications of Linux for how my file system should be organised?

Discussion

I used the transition to Linux to overhaul my file system. I was keen to use the terminal to access files. Good file names for terminal access are different to that for a point-and-click environment.

Some general principles that guided the new file system

Remove spaces from file names
- It's a pain to have to escape spaces (\)
- I replaced spaces with hyphen "-"
Convert file names to lower case where possible
- Linux is case sensitive
- It's quicker to type lower case
- Consistency reduces confusion
Prefer file and folder names unique in the first few letters
- This facilitates tab completion
Prefer short, descriptive, and memorable file names

I also found it particularly useful to do a general reflection on my file structure. I cleaned up the hierarchy so that all my core activities had a location in the file system.

Accessing Windows Drive from Ubuntu

Question

Are there any issues with accessing the Windows OS partition from Ubuntu?

Discussion

It's easy. It shows up as a mounted drive called "OS". It's path was /media/OS

Monitoring processes and use of system resources

Question

What is the Ubuntu equivalent of Ctrl+Alt+Delete for viewing active processes and memory and processor usage?

Discussion

There is a program called "System Monitor"

Command prompt for terminal colour

Question

It seems to me that it might be useful to make the command prompt a different colour to help make it clear where one command starts and output begins a little clearer.

How can this be done?
What are good colour options?

Discussion

Sites discussing the issue
- cyberciti
- archlinux wiki
The .bashrc file that came with Ubuntu had a setting that could be enabled to have a colour prompt.

Opening common binary files from command-line

Question

How can common binary files (e.g., PDF, Word, Excel, PowerPoint) be opened from the command line?

Discussion

gnome-open
- This command can be followed by filenames, web addresses, directories, and will typically do some appropriate default action.
- gnome-open . opens the current directory in Nautilus
- alias go=’gnome-open’ or equivalent may be useful

Keyboard shortcuts for size window left or right half of screen

Question

I tend to work on a large monitor with space for Windows on the left and right side of the screen. I like to have a keyboard shortcut which sizes a window to the left or right of the screen.

How can this be done on Ubuntu?

Discussion

Ubuntu does support half Desktop sizing by dragging a window with the mouse to the left or right most side of the desktop.
keyboard shortcuts offers some options for moving windows, including maximise vertically or maximise horizontally, but no option for positioning window on left half of screen.
Ctrl+Alt+NumPad Key
- I learnt the following on askubuntu
- By default Ubuntu supports shortcut keys for window manipulation (e.g., Ctrl+Alt+NumPad4 maximises a window vertically on the left half of the screen and Ctrl+Alt+NumPad6 does the same but to the right.
- The main annoying aspect of this is that these keys are not suitable for touch typing.
Solution
- Install Advanced Desktop Effects settings (Compiz with Compiz Config)
- Start CompizConfig
- Go to Window Manager - Grid
- Change the key bindings (I chose the following, which have a certain consistency with Vim movement keys)
  - Super + h (left)
  - Super + L (right)
  - Super + K (maximise)
  - Super + J (restore)

Shortcut key to initiate web search

Question

On windows I had a global shortcut key (Windows+Z) to initiate a Google search.

How can this be done on Ubuntu?

Discussion

googlizer
- This can be used to create command line programs for searching.
Chrome
- Start chrome
- Type search term in address bar
- Making google.com the home page can speed this up
- Evaluation:
  - The responsiveness of this is not adequate (a search box is instant; chrome takes around 1 second to start up)
  - It results in the loss of the default Chrome homepage
AutoKey
- A program modelled on AutoHotKey might provide the desired functionality.
I asked the question on askubuntu
- Google Desktop Search with Desktop search disabled was a good choice.
- I changed the default shortcut key to Ctrl+F12

Using a laptop with multiple external monitors

Question

How can a laptop monitor be disabled and an external monitor enabled?
I use two external monitors, one at home, and one at work. How can Ubuntu be configured to automatically detect which one is connected?

Discussion

When I configure to just use my external monitor and not my laptop monitor and then unplug the external monitor, the laptop monitor is not activated. Thus, I can't use the mouse to reactivate the laptop monitor. It would be good if Ubuntu could automatically detect the removal of the external monitor, or alternatively I could have a shortcut key to activate laptop only mode.
ubuntugeek describes configuring Ubuntu so that Ctrl+Alt+Backspace can force a restart of X
A user asks about automatic switching of monitor configurations on Ubuntu forums
More suggestions
I asked the question on askubuntu
- I added my answer to the question
- Essentially it involves using a program called disper
- disper -s and disper -S activate primary and secondary monitors respectively. I've configured these commands to be triggered by shortcut keys using CompizConfig.

Launching programs with keyboard shortcuts

Question

What is a good strategy for launching programs from keyboard shortcuts?

Discussion

CompizConfig makes it easy to create commands and assign shortcut keys.
The Unity Launcher enables shortcut keys combining the Super key and a number or typing the Super key and typing a key word.

Restart to Windows

Question

My bootloader defaults to Ubuntu.

Is there a way to shutdown Ubuntu and boot to Windows without having to make a selection from the bootloader?

Discussion

I asked the question on Ask Ubuntu
- I received a good answer, and I added further details on AskUbuntu.

Default terminal size

Question

What is a good default Terminal size?

Discussion

I can always press F11 to get a full-screen terminal. However, sometimes it is useful to be able to have a reasonable size terminal come up that does not take over the whole screen.

I've configured the default profile to be 95 columns by 40 rows. I would choose more rows, but I want the setup to work on my smallest monitor (i.e., my laptop).

Prevent Unity menu from showing

Question

At first I thought the Unity menu was okay. However, I don't like:

the screen real estate that launcher consumes
the way that it defaults to placing left aligned windows indented with space to the launcher
The size of the launcher icons (although at least this can be reduced)
The way that it stays in focus sometimes for a period of time

Discussion

I asked the question on askubuntu and received a good suggestion.

In Compix Config set Unity - Hide Launcher to Auto-hide.

Configuring Ubuntu for running basic scripts

Question

What is a basic strategy for running scripts in Ubuntu?
How do permanently add the custom script to the path?
Where should I store throw-away scripts?
What is a good way to run custom scripts?

Discussion

There is a discussion of where to store scripts here.
A common place seems to be ~/bin

I did the following:

mkdir ~/bin
added export PATH="$PATH:~/bin" to .bashrc

Find file by name

Question

How can I find a file by a name or part of its name within a specified directory (or subdirectories)?

Discussion

Cyberciti provides a page explaining several options.
find command
- The following command would find all files with the txt extension in ~/dir
- find ~/dir -name "*.txt"

Accessing Ubuntu drive from Windows on dual-boot machine

Question

I store my data on my Ubuntu drive.

How can I access my Ubuntu drive from Windows?

Discussion

This page has three options, the last of which, looks like what I want.

Ext2 Installable File System for Windows
- This did not work on Windows 7 (perhaps this will change in the future)
DiskInternals Linux Reader
- Evaluation
  - This worked adequately
  - This has both the benefit and the problem that it does not alter the Linux partition
- How did it work?
  - Start Windows 7
  - Install and Start DiskInternals
  - Save selected Linux partition files onto Windows partition (I had a special section)
  - View, edit, and change files
  - Restart Ubuntu and copy files from Windows partition back into appropriate location

Script fur renaming files in folder

Question

I sometimes download files sent to me and want to convert the file names into more appropriate ones.

The main conversions that I'd like to see are:

replace spaces with dashes ("-")
replace uppercase with lower case

Others may arise as time goes by.

The script should take a list of file names.

How can this be done?

Discussion

I wrote the following script and called it cleannames

#!/bin/bash
for file in * ; do mv "$file" `echo "$file" | sed 's/ /-/g'`; done
for file in * ; do mv "$file" `echo "$file" | sed 's/.*/\L&/g'`; done

Minimise current window with keyboard shortcut

Question

What is the keyboard shortcut to minimise a Window?

Discussion

Alt+F9 minimises
Alt+F10 maximise
Launcher - System Settings - Shortcuts allows for custom shortcut keys

Terminator Terminal

Question

Should I adopt Terminator Terminal?

Discussion

At present I'm using Gnome Terminal.
Various reviews suggest that Gnome terminal is a reasonable option in terms of speed and features.

Preventing automatic shading when watching flash videos

Question

I had an ABC video on full screen and after a few minutes, Ubuntu shaded the screen.

How can this be prevented?

Discussion

Power Management has some options
I disabled the screensaver. That seemed to fix the problem.

Batch renaming of files

Question

Unix is case sensitive. It also encourages substantially greater use of the command-line. As such, it is useful to adopt file naming conventions that make it easier to work with files on the command-line.

How can I batch rename files
- replace underscore with dash
- replace upper case letter with corresponding lower case letter

Discussion

Some options
- KRename
- for loop
  - Tutorial on using for loops
- rename

Convert underscore to dash:

`for file in * ; do mv $file `echo $file | sed 's/_//'` ; done`

Convert upper case letter This tutorial shows how to use the convmv command.

Kill a program

Question

Windows has Ctrl+Alt+Delete which facilitates killing a process.

What does Ubuntu involve?

Discussion

The system monitor lists processes and enables killing processes
Ctrl+Alt+Backspace kills X (once this feature is activated)

Better console / terminal colours

Question

The default background for Ubuntu terminal is a type of dark purple. While it's "pretty", I find the contrast to be relatively low.

What are better settings?

Discussion

I applied the following settings:

Edit - Preferences:
- Colors - Built-in schemes= white on black
- Background = solid colour
export TERM="xterm-256color" enables better colour support for Vim

Problems displaying full screen video

Question

I find that full screen video on YouTube seems to freeze. How can this be fixed?

Discussion

There is a tutorial on YouTube on how to fix this.

sudo mkdir /etc/adobe
sudo su
sudo echo "OverrideGPUValidation = 1" >> /etc/adobe/mms.cfg"

Reducing Grub loader times

Question

I think it would be useful to reduce the default grub display time to something like 1 or 2 seconds. I almost always boot to Ubuntu, and if I want to boot to Windows, I have a command from within Ubuntu that changes the default boot for one occasion.

Is this a good idea?
How can I do this?

Discussion

I think 2 seconds would make sense as this is long enough to easily enter manual mode if I am watching carefully.
sudo vim/etc/default/grub
I modified the value of GRUB_TIMEOUT and made it GRUB_TIMEOUT=2
sudo update-grub

Set up git on Ubuntu

Question

How do I configure git on Ubuntu?

Discussion

General information on configuring git and github on Linux is available here

Guide to Getting Started with R: 2011 Update

2011-05-04T16:46:00.001+10:00

In mid-2009, I wrote a post on getting started with R. A lot has happened in the world of R over the last two years. New books, videos, online documentation, blogs and other resources have emerged. New community structures have emerged. As such I've given the guide a major overhaul both to increase its usability and acknowledge the many exciting changes in the world of R. The updated Guide to Getting Started with R is available here..

Case Study in Customising Syntax Highlighting and Folding in Vim for a Niche Scripting Language Called Inquisit

2011-04-25T17:26:00.000+10:00

This post presents my efforts to setup a productive environment for editing Inquisit scripts in Vim. In addition to being relevant to people who write Inquisit scripts, the post is designed as a general case study in customising Vim for a niche programming language. Specifically, the post discusses how to setup in Vim: (a) code folding using a custom expression, (b) custom syntax highlighting, and (c) interactions with the command-line.

Overview

Inquisit is proprietary software typically used for running computerised experiments in the behavioural sciences. The Inquisit Website has further explanation and includes a free limited trial. I have previously discussed the benefits of the software for running online psychological experiments.

Inquisit includes desktop software that runs on Windows. It can be used to write, debug, and run Inquisit scripts that control the flow of the experiment. I have provided a simple example of a script here as a gist on github. It contains a simple RT task, four choice RT task, and a typing test. Further example scripts can be seen in the Task Library on the Inquisit Website.

More recently I have started using Vim to do my text editing (see my previous post on this conversion. The conversion was motivated by an aim to increase the power and consistency of my text editing environment. Thus, instead of using a separate text editor for each programming language that I code in, I wanted, as much as possible to be able to use Vim for as many tasks as possible.

Vim has great built-in support for many languages, however, Inquisit is a niche scripting language. Thus, Vim lacked built-in support for the language. Nonetheless, to true power of Vim is its customisability.

Below I explain the customisations I developed to work more effectively with Inquisit. It is still a work in progress, and imperfect. However, even at this stage, I find writing Inquisit scripts far more enjoyable than using the editor built-in to the Inquisit software.

I present this example as a case study in customising Vim for a new programming language.

If you want to follow any updates, check out:

the git repository with the entire contents of my vimfiles and plugins
inside the repository is my vimrc which includes various customisations for Inquisit file types.
my syntax highlighting file for Inquisit files

Below I describe a few of these customisations. The sections of greatest general interest are likely to be those on syntax highlighting and code folding.

Overview of an Inquisit script

Before describing my customisations, below is a short snippet from an Inquisit script:

# general stimuli
<text cross>
/position=(50%, 50%)
/items=("+")
/fontstyle = ("Arial", 60pt, false)
</text>

<text error.toosoon>
/position=(50%, 50%)
/items=("PRESSED TOO SOON")
/fontstyle = ("Arial", 30pt, false)
/txcolor =(255,0,0)
</text>

The line beginning with a hash is my own internal section marker. All text outside element tags <...>...</...> is not interpreted by Inquisit.
The code is made up of elements which have a keyword name (e.g., text) and an identifier (e.g., cross or error.toosoon).
In the above example, these elements are used to represent objects displayed on a screen such as a plus symbol or some text.
Elements have attributes, which have a name and a value. Attributes are contained between opening (e.g., <text ...>) and closing element tags (e.g., </text>). Attributes begin with a slash, then the name of the attribute (e.g., position, then equals sign, and then the value of the attribute.
Values of attributes have various syntactical forms typically involving either parentheses or square brackets.

My scripts often range between 500 and 2000 lines of code. For a more complete example, see the gist mentioned earlier. You can even download a free trial of the software if you'd like to see how it runs. The remainder of this post describes the customisations that I developed to make writing Inquisit scripts in Vim more productive.

Launching an Inquisit Script from Vim

Question

How can Inquisit Scripts be tested directly from Vim command-line?

Discussion

Inquisit Help lists command line options

I adopted the following strategy. First, I added the Inquisit executable to the path; E.g., I added C:\Program Files\Millisecond Software\Inquisit 3 to my path. This means that entering inquisit on the command-line starts Inquisit.

Then I added the following code to my vimrc file.

function! RunInquisit()
    let cmdstring = "!start inquisit " . '"%:p"'
    exe cmdstring
endfunction

Thus, when I have an Inquisit script open in Vim, I just have to type :call RunInquisit() and this will start Inquisit and send the script in the active buffer to Inquisit for running. Once run, this enables debugging of any errors (Inquisit has a good built-in debugger). If no errors are obtained, the experiment will start, which facilitates the identification of errors in the actual design of the experiment.
function! declares a new function overriding any previous function by the same name.
RunInquisit() is the arbitrary name of the function, although it does need to start with a capital letter. In this case, it takes no arguments.
let cmdstring... assigns a string to the variable cmdstring. The period (.) is the concatenation operator. !start is means of starting a program from the command line asynchronously. inquisit is the name of the Inquisit executable. %:p is a file modifier pattern where % represents the active buffer and :p indicates that the pattern should be replaced with the file name with complete path for the active buffer. See :h filename-mod for more information.
exe executes the string on the Vim command-line.

Code Folding

Question

What is a useful strategy for code folding for Inquisit Scripts?

General discussion of code folding

I'm a huge fan of code folding in general, and in Vim specifically. When done appropriately, code folding provides many benefits. In particular, it makes large files easier to navigate, and it makes it easier to get an overview of the contents of a file.

After some reflection I have a few principles that I use to guide the design of a folding scheme.

Use the language
- Folds should where possible take advantage of natural markers in the programming language.
- Using built-in markers means that time does not need to be allocated to the task of managing the creation of folds.
Use unobtrusive markers
- When markers are used that are not part of the language, they should be simple an unobtrusive.
- Markers should not accidentally occur.
- This typically involves taking advantage of the commenting system in the language.
Two or three fold levels is typically sufficient
- Too many folds creates navigational issues whereby excessive time is spent navigating between folds.
- As a general rule I like it when the lowest level of folding contains at least 5 or more lines and not much more than a page or page and a half of text.
Avoid Fold-1 - text - Fold-2 pattern
- I.e., when a fold is expanded, there should not be a passage of expanded text and then a lower level fold.

Inquisit scripts and code folding

Inquisit scripts are made up of elements that look somewhat like HTML. Good coding practice suggests grouping related elements into sections. This lead me to adopt the following folding scheme.

Level 1 folds
- These are lines that begin with a single hash #.
- As such they are not defined by the language, but the language permits putting almost any text outside elements.
- They are designed to represent logical groups of Inquisit elements.
- I typically have sections for:
  - Experiment: a section that sets out any experiment level information such as global variables, data format, random allocation of subjects, and so forth.
  - Blocks: a section that sets out each block in the experiment .
  - One for each block: a section that contains all the trials, stimuli, and item code for a given block.
Level 2 folds
- These represent each element.
- Thus, they are automatically created by the language.

My code implementing folding for Inquisit scripts

The code below achieves this design. It is placed in my vimrc file.

function! InquisitLevel(elements)
    " elements: 0 or 1
    if getline(v:lnum) =~ '^# .*$'
        return ">1"
    endif
    if a:elements && getline(v:lnum) =~ '^<[a-zA-Z].*$'
        return ">2"
    endif
    return "=" 
endfunction                  
au BufEnter *.exp setlocal foldexpr=InquisitLevel(1)
au BufEnter *.exp setlocal foldmethod=expr

It contains a function that takes an argument called elements. If it is 0, then only section headings are folded and not elements.
getline is a Vim function that returns the content of a line. In this case v:lnum is used by expr fold method to indicate a particular line number.
=~ is a logical operator that sees whether the text on the left is matched by the regular expression on the right.
'^# .*$' means match from the start of the line (^) where the hash character is in the first column, followed by a space, and then any number (*) of characters (.) followed by the end of the line ( $).
If such a match does occur, then return ">1" which means that heading 1 starts on that line. See h fold-expr for more information.
a:elements is a variable that contains the value of the argument elements passed to the function. The a: is a necessary prefix for arguments. In Vim 0 is false and 1 is true. The double ampersand (&&) is the logical AND operator.
The second match matches lines that begin (^) with the less than sign followed by at least one alphabetical character and then any number of additional arbitrary characters. This provides a simple way that adequately matches for my purposes the start of Inquisit elements.
Thus, if elements is non-zero and there is a match, this line is the start of a level 2 heading (">2").
If no matches are obtained, a value of = is returned, which tells expr to use the fold level of the previous line.
The code is set up automatically using the two au commands.
BufEnter *.exp is an event that occurs when the buffer is entered through file opening or buffer switching.
*.exp is a file name filter. Thus, the command only works once the file has a file names.
foldexpr is a Vim setting that takes a custom function that determines fold levels.
foldmethod is a Vim setting that indicates the type of folding to use.

Using code folding

The main keys that I use for code folding are as follows.

zj an zk
- to move the cursor down and up between fold headings.
- This is not needed when text is already folded, but is useful when the cursor is expanded text and I want to get the cursor to the previous or next fold.
- I often use zjzx to move down and show only the next fold.
zc
- to close a fold.
- I often follow this up with j and k to navigate over folds.
zx
- to show only the current fold.
- It is an awesome command that I use all the time. It functions like magnifying glass zooming in on the code of interest while at the same time providing hierarchical context.
zM
- to close all folds to level 1.
- I often use zMzx when folds have been expanded. This then gives the effect of only showing the active fold.
zR
- to expand all folds.
zO
- to expand a single fold and its sub-folds.
- I often use this when a section of text has more folds than is desirable.

Syntax highlighting

Question

What is a basic syntax highlighting system for Inquisit scripts?

Basic installation instructions

If all you want to do is make use of my syntax highlighting for Inquisit files, follow these steps:

Copy my syntax highlighting file for Inquisit files into your syntax directory in your custom vimfiles directory.
Add the following command to your vimrc file au BufEnter *.exp set ft=inquisit
- It creates a file type called inquisit that then lets Vim know that it should apply the Inquisit syntax highlighting.
- Note that it assumes that you are using the standard *.exp file extension.

Screenshots

The following are some screenshots showing what the folding and syntax highlighting look like with a molokai colour scheme.

Image with sections folded

Image of folding with element expanded

Basic image of the syntax highlighting

Overview of writing the syntax highlighting file

Before describing the syntax file in more detail, I should present a few caveats. I am new to syntax files in Vim, and I am far from a master in regular expressions. I hacked out a syntax highlighting scheme that works for me, adequately. It is far from perfect. But I guess that's a lesson in itself when it comes to using Vim. Vim allows for near infinite customisation, but you have to consider the return on investment of any customisation efforts.

In general, when developing the syntax file I had a look through a few example syntax files for what I thought might be similar file types. For every command that I did not understand I did Vim help searches. And in general I just fiddled around with a sample Inquisit file open until I got something that basically worked. For people needing a little more regular expression training, there's a great Vim-oriented tutorial here.

Discussion of the syntax highlighting file

At the top of the file was the following:

syntax case ignore
syn clear

This tells Vim that syntax highlighting should be case insensitive
syn clear clears any existing syntax highlighting applied to the file, although this is often not required.

The syntax file then contains many syn keyword commands, such as the following:

syn keyword inquisitElementKeyword contained survey 
syn keyword inquisitElementKeyword contained surveypage 
syn keyword inquisitElementKeyword contained textbox 
...

Words like survey, surveypage, etc. are all element names and keywords within the language.
keyword is a simple form of matching that matches the keyword exactly without resorting to patterns.
inquisitElementKeyword is a grouping name which I use later to assign colours and show where such keywords are contained.
The contained keyword allows the scope of keywords to be limited to only situations where they are contained within some other syntactical structure.

The syntax file then contains code for matching operators, an example of which is shown below:

syn match inquisitOperator contained "||"
syn match inquisitOperator contained "&&"

This involved using the match approach. This was required because the characters in operators are not alphabetical characters.

I then added code to syntax highlight numbers.

syn match inquisitNumber contained "\<[0-9%][0-9%]*"

The code shows the diverse options that match provides. It matches one or more numbers or the percent sign. The combination of backslash and less-than-sign means that numbers must be at the start of what Vim sees as a word.

Then came some code to represent regions of text

syn region inquisitString start=+"+ end=+"+

The above code defined a region based on quotation marks.
The plus sign is used as an alternate delimiter in order to indicate that the double quotes are the actual text used to match the start and end of the region.

syn region inquisitElementBlock start="<..*>" end="" transparent
The above code represents an Inquisit element.
The similarity with HTML tags may be apparent.
The transparent keyword is used to indicate that this merely represents a logical unit and does not impose any actual syntax highlighting.

syn region inquisitAttributeKeyword contained start="\<" end=">" syn region inquisitAttributeRegion start="\/" end="=" oneline contains=inquisitAttributeKeyword
The above code creates a quick and dirty way of highlighting attribute names in elements.
Such attribute names always appear as a word contained between a slash and an equals sign.
\< and \> match the start and end of a word, but this is contained.
The second line indicates that the first line is contained in it.
The oneline keyword ensures that the match occurs entirely on one line.

After some additional code comes the assignment of colours to groups. The following are a few examples:

hi link inquisitElementKeyword Keyword
hi link inquisitOperator Operator
hi link inquisitDelimiter Delimiter  
hi link inquisitString Character

Each command creates a highlighting link between the custom groups created in the script and a common logical highlighting object used by many colour schemes.
In case you were interested, I use molokai as my default colour scheme. It is a popular dark scheme available from vim.org, and from Thomas Restrepo's blog.

Getting Inquisit Help while in Vim

Question

Inquisit provides clear and well-organised documentation. In Desktop Inquisit, pressing F1 in a script brings up the help for the element under the cursor.

How can I make it easy to get help on Inquisit commands while continuing to work within Vim?

Discussion

The simplest strategy is just to keep a copy of the Inquisit help file open while coding and type in search terms directly. In the end, I have found this to be quite adequate. However, I record some thoughts and ideas I explored for more automated approaches below:

HH "C:\Program Files\Millisecond Software\Inquisit 3\inquisit.chm" will open Inquisit help (at least on my system given my installation path and version).
Alternatively there is a online version of help: < http://www.millisecond.com/support/docs/v1/index.htm >
I asked a question on StackOverflow for more information.
I installed keyhh. The following command basically did the trick: keyhh -#klink "data element" c:\Program Files\Millisecond Software\Inquisit 3\inquisit.chm where "data element" is a hypothetical search term.
I also found this vimscript designed to assist interaction with chm help files.

Future refinements

In the future I may:

Improve the syntax highlighting
Introduce an Inquisit code formatting customisation
Improve integration with the Inquisit language particularly with regard to attributes
Develop code that facilitates writing Inquisit scripts, such as sequences of numbers for trial and block sequences

General lessons Learnt

Tinkering with the syntax highlighting, folding, and command-line interaction in Vim encouraged me to think more deeply about each feature and how the feature could be optimised for my workflow.
The ability to customise Vim as I have done here validates my decision to adopt Vim. Vim encourages the development of amazing text editing shortcut keys; the more they can be applied to the languages that I write in, the better.
Investment in customisation pays off both in terms of (a) making the immediate task of programming in the given language easier, and (b) making it easier to customise other languages in the future.

Additional Resources

Using Vim and VimOutliner as a Brainstorming and Outlining Tool

2011-04-16T00:09:00.001+10:00

This post discuses using Vim as an outlining tool. It focuses particularly on VimOutliner. In particular the post (a) discusses reasons why I like outlining tools, (b) presents some links for getting started and installing VimOutliner, (c) presents my verdict on the personal utility of VimOutliner, and (d) presents an assortment of little issues that I encountered and resolved while learning to use VimOutliner.

Overview:

I've been a huge fan of outlining tools for more than ten years. It all started when I stumbled on Outline View in MS Word. I tweaked the settings a little bit, reducing the tab width to zero, and adding a few additional shortcut keys and macros.

I used Outline View to do much of my project-related brainstorming. For example, if I had a yes/no decision to make, I'd type the name of the problem with subheadings: description, yes, no, and decision. The description would articulate the nature of the decision. I'd then put reasons for yes, and reasons for no under their respective headings. This facilitated taking both perspectives. Once I'd reached the decision, I could fold up the decision and move it from a pending section to a resolved section.

On a typical project there might 20, 30, or more decisions to be made.
Using the hierarchical structure of Outline View allowed me to zoom in and out of problems. And this is but one example. I used outlines for brainstorming, articulating problems, recording a log of events, organising notes, and more.

Of course, I'm not the first to discover the power of outlining. Outlining is part of many people's brainstorming and thinking toolkit (see Free Mind for an open source mind mapping tool), this post on essay writing strategies, etc.).

The beauty of a tool like Outline View is:

the ability to quickly manipulate the outline, promoting and demoting elements, moving elements or whole trees up and down, and so on
the ability to quickly navigate the outline
the ability to selectively show elements of the outline
Outline View also had benefits in that it had all the standard tools of MS Word available.

However, Outline View also had problems:

It was limited to 9 heading levels.
Large files over about 200,000 words were often sluggish, and sometimes crashed.
It was difficult to convert the file into other formats.
It lacked some fold navigation and viewing options, such as collapsing up a fold or just showing the folds required to view the current line.
Many of the keyboard shortcuts for Outline navigation and viewing required movement of the hands from the home key position.

VimOutliner

I previously posted about my transition to using Vim. As part of this transition, I was keen to bring as many tasks as possible into Vim. I saw the appeal of plain text. I wanted to use cross-platform, open source tools. I wanted to take advantage of the power of the text editing keys in Vim. Thus, I wanted to switch from MS Word Outline View to an outliner in Vim.

VimOutliner is a major outlining tool for Vim. Raymon Li wrote instructions for installing VimOutliner on Windows.

Here are some additional links relevant to Vim, VimOutliner, and outlining more broadly.

Steve Lit has a video on VimOutliner
The VimOutliner Plugin is also available on github
Discussion of Vim Outlines, Code, and Software by Steve Litt
Peter Stuifzand wrote about using VimOutliner and GTD
Org-mode: I also hear great things about Emacs Org-Mode.
There is also VimOrganiser which is designed as a partial clone of Emacs org-mode.
But as of April 2011 it is at the relatively early stages of development.

Evaluation of VimOutliner after 3 months

At the time of posting I've been using Vim for around three months. However, for the last month and half I have settled into a workflow that does not involve VimOutliner. Instead, I've switched to using Markdown in Vim with custom folding for my brainstorming.

I'll talk more about my Markdown brainstorming setup at a later date (for an overview of Markdown in Vim, see this video on Vim and Markdown if you're curious). However, in general my preference for using Markdown has largely flowed from the benefits provided by Vim when working with text file formats. Specifically in MS Word, Outline View relative to Normal View had the advantage of being able to quickly hide and show the relevant text in a hierarchical document. However, Vim's folding tools can facilitate hiding and showing of text in a variety of text formats. Thus, with Markdown I can write text in a more structured format. The content can more readily be used in other sources, such as blog posts. Yet using Vim I can still hide and show relevant sections of text and move sections around fluidly.

Assorted problems and solutions

The remainder of this post records a few challenges that I encountered when getting oriented with VimOutliner. I've recorded them here in case they might be useful to others. Bear in mind that they are effectively the working notes of someone new both to Vim and VimOutliner.

1. Promote or demote multiple lines

Question:

How can multiple lines be promoted or demoted?

Discussion:

Visual line selection approach: 1. V to enter visual line mode * select lines to be promoted or demoted * > or < to demote or promote * . to repeat, u to undo

Collapsed fold approach: 1. collapse a fold (e.g., zc) 2. >> or << to demote or promote

2. Tab width for outline files

Question:

What is the best tab width for outlines?
If this is different to the default setting, how can such a change be limited to Vim Outlines?

Discussion:

Tab width of 2 might be better.

I added the following commands to my Vimrc

au BufEnter *.otl setlocal tabstop = 2
au BufEnter *.otl setlocal shiftwidth=2

3. Using virtualedit

Question:

Should virtualedit be turned off when working with outline mode?

Discussion

Enter in insert mode at the end of a heading line starts a new line with proper indentation
o and O in normal mode add new lines below and above the current heading at the same heading level

Thus, vitualedit can be the source of formatting problems. However, with care, this can be prevented, and I like virtualedit enough to not turn it off for this file type.

4. Opening just subheadings

Question:

How can a collapsed heading only be opened to show the first set of headings?

Discussion:

The documentation for VimOutliner does not mention this. General Vim help on folding (h folding) is relevant. zx and zv are the two relevant key combinations.

5. Add heading below fold at same level as fold

Question

I often edit a fold, fold it up and then want to add another level as a fold. For example, take the following:

text1
    text2.1
    text2.2
    text2.3
    text2.4

I might want to fold up text1 to hide text2.1 to text2.4 and then want to add a new heading at the same level as text1 below test2.4. Pressing o adds a line below text2.4 at the same level as text2.4. How do I add a heading below test2.4?

Discussion

Assume in insert mode
Ensure cursor is actually on fold: kj
Move cursor to start of text ^
Copy preceding tabs y0
Collapse headings again zc
Add line o
Exit mode <ESC>
Paste tabs P
Enter insert mode at end of line A

And optionally

Collapse unnecessary folds <ESC>zx
Return to insert mode A

Taken together and with some seemingly necessary tweaking, this makes:

au FileType vo_base  map <leader>o kjzO^y0zco<Esc>0DPA<Esc>zxA

Issues

It does not work when body text is at the bottom
- it adds an extra body text line
It sometimes does not work when the outline is corrupted
- it displays various error messages
It does not work when the heading is not folded
- it leaves a blank line

6. Preventing outline file format corruption

Question

What kinds of corruption of the outline file format can occur?
How can this be prevented?

Discussion:

Spaces where tabs should be is a major source of problems.

Outline format uses tabs for indentation. In general, if spaces precede the first printed character, the file format becomes corrupted.

The following are common causes of this

spaces can be inserted by using space instead of tab
spaces can be inserted when virtual edit is on

Potential Resolutions:

Prevent
- Disable virtualedit (i.e., set virtualedit=""). However, I like virtualedit and disabling it for one file type makes my editing experience less consistent. So I chose not to do this.
- Follow the rule of always starting a new heading using a limited set of keys: Enter, o, O.
- Always change heading level using standard keys:
  - Tab, Ctrl+D, Ctrl+T (insert mode)
  - >> << (normal mode)
- Avoid leaving blank lines
  - If a new heading is created, but no text is entered, delete it before moving the cursor.
  - The above behaviour could be automated with a substitution command.
Diagnose Problem
- set list shows whether the characters preceding headings are all ^I (set nolist) turns this off.
Fix
- Manually replace spaces with tabs with assistance from set list
- Use substitution to replace all problematic lines. This could also be linked to an autocommand that automatically checks and fixes problems
General Advice
- See this vimtip on highlighting unwanted white space In the case of VimOutliner, this would be any space before the first printed character in a line.

How to Ask Me a Statistics Question

2011-03-30T15:36:00.004+11:00

This post includes instructions on how to ask me a written statistics question.

The old way

I receive a lot of statistics questions by email. Answering such questions by email is inefficient.

A lot of questions re-occur.
Answers only help a single person.
Senders only get my suggestions, and not suggestions from other experts.
Sender do not get much feedback on how to improve in the art of writing effective questions.

A better way

A better model involves using http://stats.stackexchange.com . It offers many benefits.

Many experts have an opportunity to provide an answer.
The questions can be improved and refined.
Questions and answers are accessible from search engines. Thus, the question and answers become an ongoing Internet resource that are typically read by many hundreds of people.
The answers are under a Creative Commons licence which means that they can be reused in other forms.
Learning to write effective questions on question and answer sites is an important skill.

Procedure for asking a question

Go to http://stats.stackexchange.com/ .
Create an account
- You can create an account with a non-identifying user name if you prefer.
Read the FAQ.
Click ASK QUESTION.
- Enter your question title, description, and any relevant tags.

Important things to remember:

Be prepared to edit your question to provide additional information in response to comments; be prepared to respond to people who propose answers and explain why the answer is helpful or not. Asking a question is only the beginning of the process. The people answering the question are volunteers who are often motivated to create a question and answer combination that will be helpful for others in the future. You should try to facilitate this process.
When asking questions, show what initial research you have done to answer the question (e.g., Google searches, reading a textbook).
Try to extract the general question out of the specifics of your project, so that answers will be of greater use for future readers.
Include background information that might be needed in order to answer your question effectively.
If your question includes multiple parts, then you may need to ask separate questions.
If you are concerned about anonymity, create a user name that is non-identifying, and, if necessary, tweak your question to make any project-specific details a little more abstract.
Your question may be clearer if you break up your question into (a) context; (b) questions; and where appropriate (c) your initial answers.
If your background is psychology, be aware that users on stats.stackexchange have diverse backgrounds. Clearly explain any psychological terms or concepts.

Asking me a question

If you are a research student or academic where I work at the Melbourne Business School, then send me the URL to the question by email, and I will do my best to answer your question.
For all others, if you don't get a good answer in a few days and you think that I'd be well suited to answering the question, feel free to send me an email including the web address for the question, or post it below. Of course, I can't promise that I'll have time to answer the question.

R Optimisation Tips using Optim and Maximum Likelihood

2011-02-21T13:45:00.000+11:00

This post summarises some R modelling tips I picked up at AMPC2011.

I got some tips from a tutorial on parameter estimation put on by Scott Brownfrom the Newcastle Cognition Lab. The R code used in the tutorial is available directly here or from the conference website.

The main tips I took from the tutorial were:

Consider using BIC as a model comparison criterion.
When modelling reaction times, consider modelling data as a mixture model of two processes. One process is the main process of experimental interest and another is a secondary process that otherwise contributes noise. The secondary process is used to capture what would otherwise be outliers that flow, particularly, from very slow reaction times observed when participants get distracted. Probability assigned to the two processes can be specified a priori based on knowledge of the experimental phenomena. In the specific example that Scott showed, the outlier process was given a probability of 0.03 and this was treated as a uniform distribution between 0 and the trial time-out time.
Consider transformations model parameters for the purpose of estimation and then converting the transformed parameters back to their original scale. This can facilitate estimation and also assist in enforcing psychologically meaningful constraints on parameter values (e.g., ensuring that asymptotic reaction time is greater than zero).
The combination of the R function optim and a custom created objective function, such as a minus log-likelihood function provides a powerful tool for parameter estimation of custom models.
- Scott Brown's tutorial includes an example of this.
- Ajay Shah has an example of writing a likelihood function and then getting a maximum likelihood estimate using optim.
- Benjamin Bolker has great material available on the web from his book Ecological Models and Data in R. PDFs, Rnw, and R code for early versions of the chapters are provided on the website. Chapter 6 (likelihood and all that) , 7 (the gory details of model fitting), and 8 (worked likelihood estimation examples).
- Brian Ripley has a set of slides on simulation and optimisation in R. In particular it provides a useful discussion of the various optimisation algorithms available using optim.
I also got the impression that it will soon be time to dive into WinBUGS.

R versus Matlab in Mathematical Psychology

2011-02-21T13:33:00.001+11:00

I recently attended the 2011 Australasian Mathematical Psychology Conference. This post summarises a few thoughts I had on the use of R, Matlab and other tools in mathematical psychology flowing from discussions with researchers at the conference.

I wanted to get a sense of the software used by researchers in mathematical psychology. What was popular? Why was it popular? From the small-n, non-random sample of conference attendees that I spoke to over coffee and cake, I concluded:

Many experienced math psych researchers know a bit of both R and Matlab, but most specialised in one.
Matlab seemed to be substantially more popular than R in math psych.
The general attitude seemed to be that both tools offered similar functionality.
Reasons given for using Matlab:
- Consistency: several researchers commented that functions are highly consistent in Matlab, making it easier to return to coding in Matlab after a break.
- Superior built-in documentation: There was a sense that Matlab documentation was more user-friendly.
- Historical precedent: researchers grew up on Matlab and then taught it to their graduate students.
- Existing packages and models: it seems like Matlab is well established in cognitive psychology where substantial existing code to guide subsequent researchers.
- University pays: Thus, while R is free, Matlab is effectively free to the academic if the academic's university has a site licence.
- User friendly IDE: In R it seems that most users pretty quickly start playing around with alternative editors, whether it be ESS, Vim and R, Eclipse, Tinn-R or something else. In Matlab, the built-in IDE seemed popular. While these external editors can be configured to create a really powerful data analytic environment, Matlab users appreciated having something that was productive out-of-the-box.
- Matlab is user friendly for implementing matrix algebra based calculations.
Reasons given for using R:
- Free (as in beer)
- Open source: A few people talked about this. However, I got the sense that the ideology of open source technology could be encouraged further.
- Sweave: Even amongst Matlab users, there was a respect and interest in the idea of Sweave in R
- R's packages: The sheer number of packages particularly for statistics is one of R's great strengths.
- Superior graphics
A few people also spoke positively of Python (see this summary of useful Python packages for statistics by Christophe Lalanne.
All the above links into general discussion of the relative merits of R, Matlab, and Python on SO.

From my discussions, I saw no need for me to personally switch from R to Matlab. Sweave, graphics, and all the R packages are fantastic. The community around R is also one of its great strengths.

Finally, open source just aligns better with science.

Open and freely modifiable source code
Freely available psychological measurement tools
Freely available data
Reproducible research documents using technologies such as Sweave
Open-access journals

It all combines to support scientific disciplines in sharing and building knowledge through accountability and trust. This applies both to sharing between researchers as well as communicating with the broader community.

I get a bad feeling when I think of researchers and interested community members who can't afford Matlab being excluded from research.

However, it was interesting to consider how issues like user-friendly documentation, development environments, and consistency could be facilitated in a massive and distributed open source project such as R.

...END RANT...

Getting Started with Sweave
Learning R in Psychology
Links to Example Sweave Documents
David Hiebeler's Matlab to R translation document.

Reasons for Transitioning to Vim: Bringing LaTeX, R, Sweave and More under One Roof

2011-01-11T15:30:00.002+11:00

This post describes the reasons for my transition to Vim.

Brief Background

Over the years I've used a lot of different text editors on Windows. In general, I've used whatever text editor came with a program. When I started using R, I moved from Rgui through Tinn-R, JGR, and others, finally settling in to really enjoy the StatET plugin in Eclipse. When I started using LaTeX, I tried TexnicCenter, WinEdt, and Texlipse. Each program had the standard Windows shortcut keys and often a few extra ones for tasks such as deleting lines and toggling comments. However, these novel shortcut keys varied between programs.

I've acquired some fairly good skills in using Windows' admittedly limited set of standard text editing features. For example, I was comfortable pressing Ctrl+Shift+Right to select a word. I even developed and learnt a custom set of key bindings that allowed me to use the navigation and cursor keys from the home key position (see my earlier post).

Nonetheless, this wasn't enough. I wanted something more powerful, more consistent, more "awesome".

Reasons for Adopting Vim

Frustration with Previous Tools

I was hitting a wall in Eclipse. The StatET plugin for Eclipse has some excellent features which make working with R really nice (see my earlier blog post However, there were a few tasks that I could not easily automate. For example, I couldn't get decent code folding or an outline view for Sweave files. I couldn't develop a quick keyboard shortcut to send the R code in \Sexpr{} to the R console. There was no key to switch consoles. I could go on. I could see no easy way to introduce the desired customisation. No doubt the customisations were possible, but it meant diving into a whole world of complexity or hoping that one day someone else would write a plugin to meet my particular needs. I could describe a similar experience with WinEdt.

Readiness for Vim

I'd already adopted a plain-text--command-line workflow. LaTeX for documents; R for data analysis; Beamer for presentations; Make for building programs; Markdown for writing blog posts. This meant that I could do most of my daily tasks in a text editor. There was also the possibility that with Vim I could bring in other tasks into a more integrated environment (e.g., outlining, task lists, etc.). When I was first learning LaTeX, R, the command-line, and so on, it was useful to have software that held my hand a little bit. After getting comfortable with such things, Vim is just one additional step.
The main markup and programming languages that I use already had decent support in Vim (e.g., R, LaTeX, etc.).

Vim Matched My Orientation

I like the philosophy of Vim. I touch type. I learn shortcut keys. I use AutoHotKey. I despise unnecessary movement of the hands to the cursor keys or the mouse. Vim is built for efficiency. Vim is built for using the keyboard. I'm happy to invest a little effort to get this efficiency.
I watched Derek Wyatt's awesome, funny, and informative videos on Vim. http://www.derekwyatt.org/vim/vim-tutorial-videos/vim-novice-tutorial-videos/ I related to what Derek was saying. I related to the "Notepad-wall". I liked an editor that saw the mouse as an "evil" "rat-tailed beast". I liked an editor that saw moving the hands from the home key position as a health hazard.

Great Features of Vim

Vim was invented in 1991 and Vi goes back much further (see Wikipedia)). It's still very popular. Whenever I learn shortcut keys and introduce software customisations, I am paying a present cost for future time savings. Future time savings come both from the fact that the software is still supported and that I don't drop the software because I find something better. Vim has been around for a long time, is open source, and has an active developer community. Thus, it feels like it will be around for a long time to come. Also, given that many coding gurus use Vim, I'm hoping I wont be switching to another tool any time soon (but I guess we'll have to see).
The Pragmatic Programmer says that you should learn at least one powerful text editor really well. The main two options seem to be Vim and Emacs. At this point I've been lured by the elegance of the mode-based approach to text editing in Vim. However, as with a lot of things, I feel the more important point is that I move my workflow to either Emacs or Vim.
The system for saving settings in Vim gives me more confidence that I can roll over my customisations easily to other machines. Eclipse always felt a bit mysterious regarding how and where customisations would be stored.
Vim appears to lack "deal-breakers". By deal-breakers I refer to something about a piece of software that is so frustrating that you just can't use it. While most software can be frustrating at times, it should be simple enough to customise the software to overcome the problem. With Vim customisation is a natural extension of the editor. With Eclipse, some useful customisation is possible, but there also feels like there is a wall.
Building on the previous point customisation is a natural extension of the software. Vim is similar to R in this respect. While a lot of programs have a more advanced scripting language to add customisations, they don't all facilitate the gradual transition towards greater use of this scripting language.

Conclusion

Expect to see several upcoming posts on Vim as I begin to incorporate Vim into my workflow using R, LaTeX, and other tools.

Other Posts on Vim

2010 in Review: 99 Most Popular Pages on Jeromy Anglim's Blog

2010-12-28T18:55:00.000+11:00

In the spirit of taking stock of the year that's been, and in the spirit of festive television repeats, this post reviews 2010 for this blog.

2010 in Review

It has been a satisfying year of blogging. Subscribers have grown from 70 to around 390. Annual site visits have grown from 10,000 in 2009 to around 94,000 in 2010.

I've also enjoyed getting involved with CrossValidated, Melbourne R Users, R-Bloggers, github, and Twitter.

I'm looking forward to posting more in 2011 on Sweave, reproducible research, R packages, unit testing, and more.

To all the readers of this blog, all the best for New Years and the year ahead.

Question and Answer Sites on Statistics, Computing, Research, and Psychology

2010-12-28T17:02:00.000+11:00

This post sets out some of my favourite question and answer sites on topics related to statistics, computing, research, and psychology.

Overview

Question and answer sites have been a part of the internet for a long time. However, more recent sites such as typified by the StackExchange Model are improving the situation substantially.

Results are readily searchable on Google.
Good answers get voted to the top
Subscriptions are not required
Incentives are given to posters to provide good answers
Questions and answers can be edited and improved
Community and social networks are facilitated

Forums by Topic

The following are some of my current favourite question and answer sites for various types of questions.

Statistics

Cross Validated: A StackExchange site for questions related to statistics and data analysis.

R

R Tag on StackOverflow: A good option for anything related to programming in R
Cross Validated: This is a good option if the question concerns statistical elements of R.
R Help Mailing List: Another option for asking R related questions.

SPSS

SPSSX-L Listserv: An SPSS Mailing list
Cross Validated: Add the SPSS tag to the question. The question is likely to be better received if it concerns statistics more than it concerns SPSS.

General Question and Answer Sites

Quora: Quora is relatively new and is designed to cover any question imaginable.

Psychological Research

A Cognitive Science StackExchange has been proposed. If this interests you, commit to the proposal.

Additional Resources

How to ask questions the smart way

Video of Reproducible Research with R: Melbourne R Users 1st Dec 2010

2010-12-13T23:40:00.002+11:00

As previously mentioned I gave a talk at Melbourne R Users Group titled "Reproducible Research and R Workflow". It covered technologies including LaTeX, Sweave, R, make, Eclipse, and git. This post shares the video.

I'd like to thank Pedro Olaya for filming the session, and Drew Conway for preparing, uploading, and hosting the video along with the many other great existing videos from R User Groups from around the world.

If you follow Drew's Blog ZIA you may have already seen the video. If you don't follow his blog, you should definitely check it out. Here are three of my favourite posts by Drew:

For those who did not catch Drew's original post, the videos are shown below.

My Talk

For a copy of the slides, R Code, and more see the previous post. The video is embedded below (If you are viewing this post in a feed reader, and the embedded video is not displaying click here to go to the original post):

Geoff's Talk

The other talk from the session was by Geoff Robinson who discussed several useful strategies for working with R.

The slides and R Script are available for download.

The talk is embedded below:

Melbourne R User Group

Yuval Marom kindly asked me to become a co-organiser of the Melbourne R user Group (MelbURN). While most people who read this blog are from overseas, if you are in Melbourne, feel free to join the group. If you know something about R, are passing through Melbourne, and would like to give a talk on R, let Yuval or me know (see the Contact Us Button), and we'll see if we can organise something.

Related Resources

R Workflow: Slides from a Talk at Melbourne R Users (1st Dec 2010)

2010-12-02T00:03:00.000+11:00

I gave a presentation at Melbourne R Users on the topic of R Workflow. The presentation covered R code organisation, and useful R related tools including Eclipse, StatET, Git, make, Sweave and LaTeX. Also, the slides from the presentation provide links to four complete examples of using R, Sweave, LaTeX, and make.

PDF of Slides

A PDF of the slides is available in fullscreen and four-to-a-page handout format. Most slides contain links to resources for learning more. Thus, it should be possible to understand the main ideas of the talk just by just reading the slides and reading the links.

Video

A video of both my talk and the previous talk by Geoff Robinson will hopefully be made available in the not too distant future (I'll post when it is available).

Github Repository

If you want to learn more about beamer, LaTeX and make, you can check out the source code used to produce the PDF of the presentation. The repository is available on GitHub:

https://github.com/jeromyanglim/RMeetup_Workflow

Thanks

I'd like to thank Yuval Marom for all the work he has put into kickstarting and organising the Melbourne R User Group and Deloitte for providing such a great venue.

Links Presented in the Talk

Sweave Tutorial 3: Console Input and Output - Multiple Choice Test Analysis

2010-11-30T22:23:00.000+11:00

This post provides an example of using Sweave to perform an item analysis of a multiple choice test. It is designed as a tutorial for learning more about using Sweave in a mode where console input and output is displayed. Copies of all source code and the final PDF report is provided.

Overview

The repository with all source files is available at:

https://github.com/jeromyanglim/Sweave_Item_Analysis/.

A copy of the resulting PDF can be viewed here.

For general information on program requirements and running the code see this earlier post

Sweave Documents that Display Console Input and Output

I find it useful to distinguish between different kinds of Sweave documents. One key distinction is between

reports that display the console and reports that do no display the console.

Reports that display the console are suited to distinct applications, including:

R tutorials
Personal analyses
Analyses provided to experts who understand R and the Project

It would be further possible to distinguish between reports that do and do not show the console input (echo=true).

Developing Sweave reports that display the console still benefit from thoughtful variable names, selective display of output and so forth. However, naturally less time is invested in putting the polish on figures, tables, and inline text.

The previous Sweave Tutorials were examples of Sweave documents that do not display the console. Tutorial 1 was a data driven document Tutorial 2 was a set of batch polished reports.

The present tutorial is an example of a Sweave document that displays console input and output.

The remainder of the post discusses various aspects of the source code.

Source Code

Folder and File Structure

.gitignore records the folder where derived files are stored by make.
makefile is similar to that explained and used in Sweave Tutorial 1. This similarity has been obtained through (a) the use of variables in the makefile (b) the fact that both projects are driven by the Rnw file; thus, make calls Rnw, which in turn imports data, and so forth.
README.md is a file in markdown. Markdown is the markup language used on Github. The file is automatically displayed on the repository home page.
data: This folder contains a file with the responses to the 50 multiple choice questions.
meta: This folder contains a file with information about each of the 50 multiple choice questions including the text, response options, and the supposedly correct response.
backup: This folder contains a copy of the resulting PDF. Although this is a derived file and as such should not generally be monitored by Git, it's helpful to include a copy for easy access.
Sweave.sty: I find it easier and more portable to just include this LaTeX style file required by Sweave in with the project.

Item_Analysis_Report.Rnw

Library loading and data import

<<initial_settings, echo=false>>=
options(stringsAsFactors=FALSE)
options(width=80)
library(psych) # used scoring and alpha
library(CTT) # used for spearman brown prophecy
@


<<import_data, echo=false>>=
cases <- read.delim("data/cases.tsv")
items <- read.delim("meta/items.tsv")
items$variable <- paste("item", items$item, sep="")
@

options(width=80) ensures that the console width is suitable for the printed page
options(stringsAsFactors=FALSE) means that character variables imported using read.delim are left as character variables and not converted into factors. In general I find this a more useful default behaviour. In particular I often use the actual text, particularly in metadata, to generate variable names, print text and so forth. Leaving variables as character is better for this.

Using data before it has apparently been generated

...
The example involves performing an item analysis of 
responses of \Sexpr{nrow(cases)} students 
to a set of \Sexpr{nrow(items)} multiple choice test items.
...

<<>>=
<<initial_settings>>
<<import_data>>
@

In the above code I wanted to be able to write the number of cases before showing the code for importing settings and data. Thus, I first ran the code chunks with echo=false to prevent display. Then, afterwards, these code chunks were rerun inside a code chunk using the syntax <<name_of_code_chunk>> (i.e., without the = sign at the end of the opening. This time they were displayed.

Scoring multiple choice tests

<<score_test>>=
itemstats <- score.multiple.choice(key = items$correct, 
            data = cases[,items$variable])
@

score.multiple.choice is a function in the psych package for scoring multiple choice tests. key is a vector of integers representing the correct response. data is a matrix or data.frame of responses from a set of respondents.
the example shows how metadata can be used to simplify code. items$variable includes the name of the 50 personality test items items$correct includes the vector of correct responses.

...

Figures in Sweave

<<plot_mean_by_r, fig=true>>=
plot(r ~ mean , itemstats$item.stats, type="n")
text(itemstats$item.stats$mean, itemstats$item.stats$r, 1:50)
abline(h=.2, v=c(.5, .9))
@

Code chunks can produce single figures. the fig=true key-value pair is required.
type="n" is used to not show points and then text(...) is used to plot the item numbers on the plot.
Because the document is an informal document designed to display the console, the figure is not wrapped in a figure float. A float would involve more typing and might even be annoying if it moved around the document.

...

Using Sweave to Better follow the DRY (Don't Repeat Yourself) Principle

<<flag_bad_items>>=
rules <- list(
        tooEasy = .95,
        tooHard = .3,
        lowR = .15)
oritemstats$item.stats$tooEasy <- 
    oritemstats$item.stats$mean > rules$tooEasy
...
@

\begin{itemize}
\item \emph{Too Easy}: mean correct $>$
\Sexpr{rules$tooEasy}.
\Sexpr{sum(oritemstats$item.stats$tooEasy)}
items were bad by this definition.
... 
\end{itemize}

The above abbreviated version of the actual code highlights how Sweave can be used to prevent repetition and facilitate modifiability.
The code flags items as too easy if more than 95% of participants get the item correct. This value (.95) is stored in a variable. It's then subsequently used both in the code to flag items as too easy and also used in the text where the rule is described in plain text (i.e., \Sexpr{rules$tooEasy}).
This is a particularly powerful use of Sweave whereby any text in a document that might be repeated or any text that describes details of a data analytic algorithm is a good candidate for simplification using Sweave.

...

\Sexpr{} and formatting

The formula suggests  that in order to obtain
an alpha of \Sexpr{sbrown$targetAlpha},
\Sexpr{round(sbrown$multiple, 2)} times as many items are required.
Thus, the final scale would need around
\Sexpr{ceiling(sbrown$refinedItemCount)} items.
Assuming a similar number of good and bad items,
this would require an initial pool of around
\Sexpr{ceiling(sbrown$totalItemCount)} items.

The above code highlights a couple of examples of how inline formatting of numbers can be done, and is often required when including inline text. In this case, ceiling and round functions were used.

Sweave Tutorial Series

This post is the third installment in a Sweave Tutorial Series:

Sweave Tutorial 2: Batch Individual Personality Reports using R, Sweave, and LaTeX

2010-11-29T17:48:00.001+11:00

This post documents an example of using Sweave to generate individualised personality reports based on responses to a personality test. Each report provides information on both the responses of the general sample and responses of the specific respondent. All source code is provided, and selected aspects are discussed, including makefiles use of \Sexpr, figures, and LaTeX tables using Sweave.

Overview

All source code is available on GitHub:

https://github.com/jeromyanglim/Sweave_Personality_Reports

Three examples of compiled PDF reports can be viewed as follows: ID1 ID2 and ID4.

The resulting report is a simple proof of concept example.

Discussion of Source Code

makefile

outputDir = .output
backupDir = .backup

test:
    -mkdir $(outputDir)
    Rscript --verbose run1test.R  

test5:
    -mkdir $(outputDir)
    Rscript --verbose run5test.R  


runall:
    -mkdir $(outputDir)
    Rscript --verbose runAll.R  

clean:
    -rm $(outputDir)/*

backup:
    -mkdir $(backupDir)
    cp $(outputDir)/Report_Template_ID*[0123456789].pdf --target-directory=$(backupDir)

outputDir stores the name of the folder used to store derived files (e.g., tex files, images, and compiled document PDFs)
backupDir stores the name of the folder where document PDFs are to be stored
test: is the default goal. Running make in the project directory will run run1test.R which will build one report.
the --verbose option shows the progress of R when run as a script. It's useful for seeing progress and debugging.
test5: compiles five reports.
runall: compiles all reports. Further information on each of the Run... .R files can be obtained by inspecting these files. In general they source Run.R and specify which ids to run reports on.
clean: removes all the files from the output directory (i.e., all the derived files)
backup: copies reports to the backup folder; i.e., it separates the finished documents from all the other derived files.
To run test5, runnall etc., type make test5 or make runnall etc.

main.R

main.R loads external functions and packages, imports data, imports metadata and processes the data.

# Import Data
ipip <-read.delim("data/ipip.tsv")
ipipmeta <-read.delim("meta/ipipmeta.tsv")
ipipscales <- read.delim("meta/ipipscales.tsv")

When importing the data, I have adopted the useful convention (which I observed from John Myles White's ProjectTemplate Package) of naming objects and data file names the same. The file extension also clearly indicates the file format (i.e., tab-separated-values).
I often have separate data and meta folders. Importing metadata often makes for more manageable code than when incorporating metadata by hard coding it into the R script.

Test scores are calculated using the function score.items.

ipipstats <- psych::score.items(ipipmeta[,ipipscales$scale], 
        ipip[,ipipmeta[,"variable"]],
        min = 1, max = 5)

The psych package has a number of useful functions for psychological research. score.items is particularly good. It enables the creation of means and totals for multiple scales. It handles item reversal. It also returns information related to the reliability of the scales.

Run.R

source("main.R", echo = TRUE)
id <- NULL
exportReport <- function(x) {
    id <<- x
    fileStem <- "Report_Template"
    file.copy("Report_Template.Rnw",
            paste(".output/", fileStem, "_ID", id, ".Rnw", sep =""),
            overwrite = TRUE)
    file.copy("Sweave.sty", ".output/Sweave.sty", overwrite = TRUE)
    setwd(".output")
    Sweave(paste(fileStem, "_ID", id, ".Rnw", sep =""))
    tools::texi2dvi(paste(fileStem, "_ID", id, ".tex", sep =""), pdf = TRUE)
    setwd("..")
}

The above code provides the function to run Sweave on each individualised report
the code is a little bit messy, contains a few hacks, and is not especially robust.
the exportReport function takes an id value as an argument x. Note the use of the alternative assignment operator. (See ?assignOps)
The code is designed to keep derived files away from source files by copying files into the .output folder and even changing the working directory to that directory.
The code creates an individualised copy of the Rnw file; Runs Sweave on the report to produce a tex file, and then runs texi2dvi with pdf=TRUE to produce the final pdf.

Report_Template.Rnw

The Rnw file contains interspersed chunks of LaTeX and R code.
Because the Rnw file is called from within R, all the R objects and data processing code does not need to be called at the start of the Rnw file. This approach is one way of reducing the time it takes to run a set of Sweave reports all based on a common data source.
The \Sexpr{} command is used to incorporate in-line text. (... sample of \Sexpr{nrow(ipip)} students ...). In the example above, it prints the actual number of cases into the ipip data.frame (i.e., the sample size).

Incorporating a figure using Sweave

\begin{figure}
<<plot_scale_distributions, fig=true>>=
plotScale <- function(ipipscale) {
    ggplot(ipip, aes_string(x=ipipscale["scale"])) + 
        scale_x_continuous(limits=c(1, 5),
            name = ipipscale["name"]) +
        scale_y_continuous(name = "", labels ="",   breaks = 0) +
        geom_density(fill="green", alpha = .5) +
        geom_vline(xintercept = ipip[ipip$id %in% id, ipipscale["scale"]],
                size=1) 
}


scaleplots <-   apply(ipipscales, 1, function(X) plotScale(X))

arrange(scaleplots[[1]], 
        scaleplots[[2]],
        scaleplots[[3]],
        scaleplots[[4]],
        scaleplots[[5]],
        ncol=3)
@
\caption{Figures show distributions of scores of each personality factor
in the norm sample.
Higher scores mean greater levels of the factor. 
The black vertical line indicates your score.}
\end{figure}

The first R code chunk produces a figure using ggplot2.
The code above takes a while to run (perhaps around 10 seconds on my machine). But the resulting plot is more attractive than what I could easily get with base graphics.
<<plot_scale_distributions, fig=true>>= indicates the start of an R code chunk. fig=true lets Sweave know that it has to produce code to include a figure.
The R code chunk is substituted with \includegraphics{Report_Template_ID10-plot_scale_distributions} in the tex file and the pdf and eps figures are created. Thus, if you want a float with captions and labels, you have to add them around the R code chunk.
the plotScale function is used to generate a ggplot2 figure of the distribution of scores on each personality scale along with a marking of the respondent's score on each scale.
The arrange function is used to layout multiple ggplot2 figures on a single plot. The source code is in the lib/vp.layout.R and was taken from a [post by Stephen Turner( http://gettinggeneticsdone.blogspot.com/2010/03/arrange-multiple-ggplot2-plots-in-same.html)

Preparing a formatted table in R for LaTeX

<<prepare_table>>=
ipiptable <- list()
ipiptable$colnames <- c("item", "scaleF", "text", "meanF",
        "sdF", "is1F", "is2F", "is3F", "is4F", "is5F")
ipiptable$cells <- ipipsummary[,ipiptable$colnames ]
ipiptable$cells$item <- paste(ipiptable$cells$item, ".", sep="")

# assign actual respones to table
ipiptable$cells[,c("is1F", "is2F", "is3F", "is4F", "is5F")] <-
        sapply(1:5, function(X)
        ifelse(as.numeric(ipip[ipip$id %in% id, ipipmeta$variable]) == X, 
                paste("*", ipiptable$cells[[paste("is", X, "F", sep ="")]], sep =""),
                ipiptable$cells[[paste("is", X, "F", sep ="")]]))


ipiptable$cellsF <- as.matrix(ipiptable$cells) 

ipiptable$cellsF <- ipiptable$cellsF[order(ipiptable$cellsF[, "scaleF"]), ]

ipiptable$row1 <- c("", "Scale", "Item Text", 
        "M", "SD", "VI\\%", "MI\\%", "N\\%", "MA\\%", "VA\\%")

ipiptable$table <- rbind(ipiptable$row1, ipiptable$cellsF)
ipiptable$tex <- paste(
        apply(ipiptable$table, 1, function(X) paste(X, collapse = " & ")), 
        "\\\\")
for(i in c(41, 31, 21, 11, 1)) {
    ipiptable$tex <- append(ipiptable$tex, "\\midrule", after=i)
}
ipiptable$tex1 <- ipiptable$tex[c(1:34)]
ipiptable$tex2 <- ipiptable$tex[c(1,35:56)]

I often find it useful to split R code chunks for table preparation and table presentation. In general this allows any text that appears before the table to include \Sexpr{} commands incorporating figures from the analyses which generate the table. In the present case, it was useful because the table was split over two pages.
The code shows some of the general logic I use for customised table creation. In hindsight I could probably refactor it into a function so that I don't have to always type ipiptable which would make things a little more concise
The general process of table creation involves: (a) extracting information on cells with cells often grouped into types which will receive common formatting treatment (b) formatting cells (e.g., rounding, decimals, and so on) (c) assembling the cells typically using a combination of the functions rbind and cbind (d) Inserting tex column and end of row separators with something like: paste(apply(x, 1, function(X) paste(X, collapse = " & ")), "\\\\") where x is the matrix of table cells.

Don't Repeat Yourself Principle using R and Sexpr{}

ipiptable$caption <-
        "Response options were 
1 = (V)ery (I)naccurate, 
2 = (M)oderately (I)naccurate,
3 = (N)either Inaccurate nor Accurate, 
4 = (M)oderately (A)ccurate
5 = (V)ery (A)ccurate.
Thus, VI\\\\% indicates the percentage of the norm sample 
        giving a response indicating that the item is a Very Inaccurate
description of themselves.
Your response is indicated with an asterisk (*)."

This text was used in both tables. Thus, this text can then be called using \Sexpr{ipiptable[["caption"]]}. This follows the DRY principle (Don't Repeat Yourself). Thus, if the caption needs to be modified, it only needs to be modified in one place.

Incorporating the tex formatted table using R Code chunks

\begin{table}
\begin{adjustwidth}{-1cm}{-1cm}
\caption{Table of results for (A)greeableness, (C)onscientiousness
and (E)motional (S)tability items.
\Sexpr{ipiptable[["caption"]]}} 
\begin{center}
\begin{tabular}{rrp{4cm}rrrrrrr}
\toprule
<<table_part1, results=tex>>=
cat(ipiptable$tex1, sep="\n") 
@
\bottomrule
\end{tabular}
\end{center}
\end{adjustwidth}
\end{table}

The tables are then incorporated into the tex file.
The R code only generated some of the required tex for the table. Thus all the other desired elements such as the table environment and captions are written either side of the R code chunk.
the R code chunk uses the option results=tex in order to enter the output from the cat function verbatim into the resulting tex file.
cat(ipiptable$tex1, sep="\n") includes a vector of tex. With the newline separator simply making the resulting tex more readable.

Additional Resources

Getting Started with Git, EGit, Eclipse, and GitHub: Version Control for R Projects

2010-11-29T13:55:00.000+11:00

This post provides information on
(a) installing Git using the Eclipse plugin Egit. (b) uploading repositories to GitHub, and (c) links to resources on Git, Git and LaTeX, and Git and R. The focus is on version control for people working on R, Sweave, and LaTeX related projects.

Overview

Version control works really well with R, Sweave, and LaTeX projects.

Benefits of Version Control

There are many benefits to version control for the data analyst. Version control allows you to:

Rewind a project or a file to a previous state, which in turn encourages experimentation
Ensure there is a record of changes
Facilitate collaboration
Facilitate backup
Show changes between files
Facilitate code sharing and reproducibility
and much more... See this question on StackOverflow for further discussion.

I also found that adopting version control facilitated several conceptual benefits. It encouraged greater consideration of:

the distinction between source and derived files
the nature of dependencies:
- dependencies between elements of code
- dependencies between files within a project
- and dependencies with files and programs external to the repository
the nature of a repository and how repositories should be divided
the nature of committing and documenting changes and project milestones

Choosing a Version Control System and Workflow

Why Git?

There are many version control systems (see Plastic SCM for a discussion). I've chosen to use Git for the following reasons:

Git can work well with Eclipse and Windows using Egit and many other tools
Git is one of the popular version control systems
Git enables uploading to Github
Git has good documentation and support material
Experts, who know a lot more about version control than I do, use Git (e.g., Hadley Wickham); the designer of Git is Linus Torvalds.

Finally, the big difference is between using a version control system and not using a version control system.

Why EGit?

EGit is a Git plugin for Eclipse. I use Eclipse and StatET to write R code and Sweave documents. I found EGit a particularly easy tool for getting started with Git and version control. The documentation is straightforward and the interface is easily integrated into my Eclipse workflow.

Getting Started with EGit and Git in Eclipse

There are many ways to interact with Git.

Installing EGit in eclipse involves using the update manager. Vogella.de has a tutorial.

To get started with your first Git repository in Eclipse, check out the EGit user Guide. When I was first getting started I used a simple R project rather than a Java Hello World application.

GitHub

GitHub is one of several sites for sharing git repositories (for example, see Hadley Wickham's baby names analysis, or my own example of using Sweave to write Multiple Choice Questions). It also has many useful social networking features.

Uploading a repository to GitHub from Eclipse

Set up a free account on https://github.com/
Work through the tutorial on creating a repository at GitHub

While the above tutorial briefly mentions SSH Configuration, it does not go into detail. When setting up my SSH key, I did the following:

Click Eclipse -- Window -- Preferences -- General -- Network Connections -- SSH2 -- Key Management
Click on Generate DSA Key
Type in a passphrase (i.e., a long and robust password)
Click Save Private Key (I saved it to a new folder under my user account)
Go to this new folder and open "id_dsa.pub" as a plain text file and copy the contents of the file to the clipboard.
Go to github.com -- Account Settings -- SSH Public Keys and click Add another public key.
Paste the public key into the box and give it a name

Using gist.github

Gists provide a quick way to get started with GitHub. Gists are useful for storing and sharing snippets of code. The result can be embedded into blog posts. To get formatted R code, give the file name a ".r" file extension (e.g., "test.r") (thanks to Hadley Wikham)

A simple example of an embedded gist is shown below:

Interesting R GitHub repositories

Good examples of people sharing R projects on GitHub include:

See the suggestions on Stats.SE. I also have an account in case you are interested.

Additional Resources

General Git

Git and LaTeX

Discussion on the Academic Productivity Blog on the topic of version control and LaTeX
Tex.SE has several questions on LaTeX and git
Article on tools for collaborative writing of scientific latex documents
vc package on CTAN supports Git

Git and R

Sweave Tutorial 1: Using Sweave, R, and Make to Generate a PDF of Multiple Choice Questions

2010-11-27T00:31:00.006+11:00

In this post I present an example of using Sweave to prepare a PDF of formatted multiple choice questions. More broadly the example shows how to use Sweave to incorporate elements of a database into a formatted LaTeX document. It aims to be useful to anyone wanting to learn more about the almost magical powers of make, Sweave, and R.

Overview

The repository with all source files is available at:

https://github.com/jeromyanglim/Sweave_MCQ.

The repository allows you to download all files as an archive or view the files individually on the web. A copy of the final PDF generated from the process is available here

I ran the code on Windows with the following programs installed.

R: For the R code
Rtools: For make and the sh commands in Make and for Sweave to run on the command line
MikTeX: For compilation of the PDF using texify and the default downloading of the exam document class

It should run on MAC and Linux with appropriate R, make, and LaTeX tools installed.

Assuming you have the above installed, to run the code

Download the repository from github
Uninstall to a directory
Open the shell in that directory
Type: make

The remainder of this post explains the code in each of the main files in the repository.

The Makefile

The makefile is used to build the PDF from the Rnw Source. It also performs other useful tasks. A copy of the makefile is shown below:

output = .output
rnwfile = Sweave_MCQ
backup = .backup

all:
    R CMD Sweave $(rnwfile).Rnw
    -mkdir $(output)
    -cp *.sty $(output)
    -mv *.tex *.pdf *.eps $(output)
    cd $(output); texify --run-viewer --pdf $(rnwfile).tex 

tex:
    cd $(output); texify --run-viewer --pdf $(rnwfile).tex

clean:
    -rm $(output)/*

backup:
    -mkdir $(backup)
    cp  $(output)/$(rnwfile).pdf $(backup)/$(rnwfile).pdf

I recently posted on the benefits of makefiles when developing Sweave documents.

The make file starts with three variables.

output stores the name of the folder where where all derivative files are placed.
rnwfile stores the name of the Rnw source file without the .Rnw extension. This is also the base of the resulting .tex and .pdf files.
backup stores the name of the folder where a copy of the pdf is placed if this is desired.

The file then has four goals.

The default goal is called all:. If make is called without argument from the command line in the project directory, the recipe immediately below all: is run. Note that all apparent indentations are tab indentations (a set of spaces would cause an error).

The first line runs Sweave on the Rnw file from the command line. On windows this R CMD Sweave requires installation of Rtools Note how $(rnwfile).Rnw will actually be Sweave_MCQ.Rnw after variable substitution.
The second line creates a directory that corresponds to the value in the variable output. The hyphen at the start of the line ensures that any errors, such as if the folder already exists, do not stop make from running. Note that commands such as mkdir, cp, cd, and rm are based on the sh shell. These commands are supported on Windows if you have Rtools installed.
The third line copies Sweave.sty into the output folder.
The fourth line copies tex, pdf and eps files (i.e., those generated by the Sweave command) into the output folder. This is done to ensure that the root directory only includes source files. This has several benefits. (a) It makes version control easier;
(b) it makes it easy to see the source files and (c) it reduces the risk of accidentally deleting source files when deleting derived files.
the fifth line changes the home directory to the output directory and then runs texify on the tex file generated from Sweave. The flags ensure that a pdf is generated and that the default viewer is initiated. This command could be modified to something like pdflatex or some other latex program.
By changing the directory, all the derived latex files are kept in the output folder.

The tex: goal can be called by running make tex at the command line. I use it in case I want there is an error in the when compiling the pdf from the tex file. Sometimes its easier to work out where the bug is by manipulating the intervening tex file. Of course once the problem has been identified, it needs to be incorporated into the Rnw source.

The clean: goal removes all files in the output directory (i.e., all the derived files)

The backup: goal copies the resulting pdf into the backup folder. I figured this might be useful in order to include a copy of the final product in the repository.

.gitignore

/.output
.project

The .gitignore file prevents all files in the /.output directory (i.e., the derived files) and the file .project from being placed under version control in git.

I'm preparing a post on version control, git, and github which will be posted shortly.

Sweave_MCQ.Rnw

Sweave_MCQ.Rnw is the R noweb file that contains chunks of LaTeX and R code. When Sweave is run on this file, the R code chunks are converted into tex and, potentially, image files are generated.

LateX Preamble

\documentclass[12pt, a4paper]{exam}
\usepackage[OT1]{fontenc}
\usepackage{Sweave}
\SweaveOpts{echo=FALSE}
\usepackage{hyperref}            
\hypersetup{pdfpagelayout=SinglePage} % http://www.tug.org/applications/hyperref/ftp/doc/manual.html
\setkeys{Gin}{width=0.8\textwidth}
\pagestyle{headandfoot} % every page has a header and footer
\header{}{Sample Multiple Choice Questions}{}
\footer{}{Page \thepage\ of \numpages}{}

The latex preamble is mostly general code that ensures proper display of the reuslting document.

The exam document class is great for writing a variety of exam style documents in LaTeX. See CTAN - exam for documentation.
hyperref is used to display hyperlinks and allows the resulting pdf to open in SinglePage format.
\setkeys... controls the width of Sweave figures relative to the paragraph width. There are no figures in this documents; so it is not really required.
\pagestyle{headerandfoot}... These three lines ensure the display of header and footer information on each page.

First R Code Chunk

<<prepare_data>>=
items <- read.csv("data/items.csv", stringsAsFactors = FALSE)

writeQuestion <- function(x){
    c("\\filbreak",
            paste("\\question\n", x["itemText"]),
            "\\begin{choices}",
            paste("\\choice", x["optionA"]),
            paste("\\choice", x["optionB"]),
            paste("\\choice", x["optionC"]), 
            paste("\\choice", x["optionD"]), 
            "\\vspace{10 mm}",
            "\\end{choices}\n\n")
}

itemText <- apply(items, 1, function(X)  writeQuestion(x = X))

answers <- paste(items$item, "=",
        LETTERS[as.numeric(items$correctAnswer)],
        sep ="")
answersText <- paste(answers, collapse = "; ")
@

R code chunks in Sweave are commenced by <<>>= and ended by @. These need to appear in the first column of the text file.
<<prepare_data>>=: The first non-keyword placed in the opening tags provides a name for the R code chunk. A short descriptive title is useful both when reading the source and when debugging Sweave compilation.
items...: this line reads in a csv file into a data frame with 40 cases. Each case is a multiple choice question with fields such as the question text, the text for the four response options, and the correct answer.
writeQuestion...: This is a function which is designed to take a row of data from the items data frame and return a latex formated character vector, where each element is ultimately be printed on its own line of the tex file. Note how in order to produce one backslash in LaTeX, two backslashes, need to be written. The \question, \choices, and \choice commands are part of the exam document class and are used for formatting multiple choice questions.
apply... takes the items data frame and for each row (1=rows) runs the function writeQuestion on the row.
answers... and answersText create a formatted string that shows item numbers and letters for correct answers, all drawn from the items data frame.

Remaining code

\begin{questions}
<<print_items, results=tex>>=
cat(itemText, sep = "\n")
@
\newpage
\section*{Answers}
<<print_answers, results=tex>>=
cat(answersText) 
@

\end{questions}

The second R code chunk has a descriptive name print_items. It uses the key-value pair results=tex. This ensures that Sweave interprets the text outputed using cat as raw tex.
cat... prints the long character vector itemText containing all the latex for the questions. sep="\n" means that each element is printed on a new line which makes the resulting tex file easier to read.

Summary and Related Resources

The combination of make, R, Sweave, and LaTeX is tremendously powerful. Hopefully, this post encourages a few more people to have a play. To learn more check out some of the following posts and pages:

Getting Started with Sweave
makefiles for Sweave, R and LaTeX using Eclipse on Windows
Question on Stats.SE 'Complete substantive examples of reproducible research using R'
UPDATE: After posting this article on my blog, Christophe Lalanne let me know about a paper by Bettina Grün and Achim Zeileis called 'Automatic Generation of Exams in R' in JSS. It also uses Sweave, LaTeX, and the exams document class.

makefiles for Sweave, R and LaTeX using Eclipse on Windows

2010-11-23T16:09:00.003+11:00

This post provides a brief introduction to make and makefiles. In particular it describes how to set up make on Windows with an emphasis on using make in Eclipse on projects involving R, Sweave, and LaTeX.

Overview

make is software that uses makefiles to build projects. make has many benefits.

One click build
Reliable build (always build derived files that need to be built; i.e., ensure build is up to date)
Fast build (i.e., don't build things that don't need to be built)
Clean up a project

make has won the ACM Software System Award. It shares this honour with two of my other favourite tools: TeX and S.

make has also encouraged me to think more about dependencies, one-click builds, and project file structures. Specifically, I use make in conjunction with R, Sweave, LaTeX, and other command line tools to build reproducible research output. I'll post some examples in the near future.

Installation on Windows

Option 1: Install Cygwin
Option 2: Install Rtools and add Rtools to the Windows path (installation can do this for you);

In order to run Sweave on the command line on Windows using: R CMD Sweave foo.Rnw:

Add R binaries to the Windows Path (e.g., C:\Program Files\R\R-2.11.1\bin; see here for instructions)
Have Rtools installed

Introduction to R has more information about running R at the command line.

Running makefiles in Eclipse

I use Eclipse to edit R and Sweave files. To configure make in Eclipse:

Run - External Tools - External Tools Configurations
New
- Name: make
- Location: location of make file; e.g., C:\Program Files\GnuWin32\bin\make.exe
- Working Directory: ${Project_loc}

Thus, with a makefile in the root directory of a project, Run - External Tools - Make runs the makefile.

Also note:

Whenever a directory is added to the Windows Path, Eclipse needs to be restarted for this to take effect. Sometimes it seems that Windows itself has to be restarted.
Editing makefiles in Eclipse requires the use of tabs. The following setting enables tabs instead of space substitution (Window - Preferences - General - Editors - Text Editors - Insert Space for Tables (uncheck)); or edit the makefile in another text editor such as notepad.
For LaTeX I have MikTeX installed; this should be on the Windows path

Resources for Getting Started with makefiles for Sweave

Ross Ihaka has a quick introduction to makefiles focusing on Sweave, R, and LaTeX
Charlie Gibbons wrote a document with LaTeX, Sweave, and Beamer examples of makefiles
Gregor Gorjanc shows an example Sweave make file using his custom Sweave command
Documentation on GNU Make
Running command line programs in Windows
Getting Started with Sweave: R, LaTeX, Eclipse, StatET, & TeXlipse

Getting Started with Markdown: Benefits, Installation, Learning, and Conversion

2010-11-12T20:49:00.000+11:00

I've recently adopted Markdown as a markup language for editing my blog posts. This post discusses (a) the benefits of Markdown, (b) installing a Markdown editor, (c) learning Markdown, and (d) converting between formats.

Overview

Great features of Markdown are:

It's plain text (e.g., support for version control, create with code)
It's simple to learn and use
The source is highly readable (much more so than HTML or LaTeX; and arguably more so than other markup languages)
It's quicker to type than HTML
It's easy to convert to other formats
Several websites such as Stack Exchange and Github use it
HTML code can be interspersed when features such as tables are not supported

It's not for everything, but it's a good tool for certain tasks. I'm still exploring the options, but in the case of blog posts and readme files it works well.

Installation

I chose to install the Markdown Editor plugin for Eclipse developed by Daniel Winterstein. Cool features include:

Outline View
Syntax highlighting
Markdown HTML Preview
Integration with Eclipse

Installation instructions can be found on the winterwell website.

Learning the Syntax

The markup is simple to learn. I first installed the Markdown Editor in Eclipse
and associated .md files with the Markdown editor.

I then:

created a file called test.md and opened it in the Markdown editor in Eclipse
opened the Markdown HTML Preview: Window -- Show View -- Other -- Markdown -- Markdown HTML Preview
worked through the Markdown syntax guide creating simple proof of concept examples, bouncing between the Markdown Editor and the Markdown HTML Preview.

Conversion

Markdown to HTML

Using the Eclipse Markdown plugin, a HTML version of the Markdown file can be obtained in several ways.

Right click the Markdown HTML Preview and click "View Source"
When in the Markdown editor, go to File -- Export HTML. This creates a HTML version of the file

Other options

Pandoc appears to be a good option for converting between document formats such as Markdown, LaTeX, RTF, PDF, OpenDocument, and more.

Copy of this Post in Markdown

The code below shows the Markdown used to create this post (except of course for the Markdown shown below).

I've recently adopted Markdown as a markup language for editing my blog posts.
This post discusses 
(a) the benefits of Markdown,
(b) installing a Markdown editor,
(c) learning Markdown, and
(d) converting between formats.

<!-- more -->
### Overview

Great features of Markdown are:

* It's plain text (e.g., support for version control, create with code)  
* It's simple to learn and use
* The source is highly readable (much more so than HTML or LaTeX; and arguably more so than other markup languages)
* It's quicker to type than HTML 
* It's easy to convert to other formats
* Several websites such as Stack Exchange and Github use it
* HTML code can be interspersed when features such as tables are not supported 

It's not for everything, but it's a good tool for certain tasks.
I'm still exploring the options, but in the case of blog posts and readme files it works well.


### Installation
I chose to install the Markdown Editor plugin for Eclipse developed by Daniel Winterstein. 
Cool features include:

* Outline View
* Syntax highlighting
* Markdown HTML Preview
* Integration with Eclipse

Installation instructions can be found on the [winterwell] website.

[winterwell]: http://www.winterwell.com/software/markdown-editor.php 


### Learning the Syntax
The markup is simple to learn.
I first installed the Markdown Editor in Eclipse  
 and associated `.md` files with the Markdown editor.

I then:

1. created a file called `test.md` and opened it in the Markdown editor in Eclipse
2. opened the Markdown HTML Preview: 
`Window -- Show View -- Other -- Markdown -- Markdown HTML Preview`
3. worked through the [Markdown syntax guide][msg]
  creating simple proof of concept examples, bouncing between
  the Markdown Editor and the Markdown HTML Preview.

[msg]: http://daringfireball.net/projects/markdown/syntax


### Conversion
#### Markdown to HTML
Using the Eclipse Markdown plugin, a HTML version of the Markdown file
 can be obtained in several ways.

* Right click the Markdown HTML Preview and click "View Source"
* When in the Markdown editor, go to File -- Export HTML. 
  This creates a HTML version of the file

#### Other options
[Pandoc] appears to be a good option for converting between document 
 formats such as Markdown, LaTeX, RTF, PDF, OpenDocument, and more. 

[pandoc]: http://johnmacfarlane.net/pandoc/

Jeromy Anglim's Blog: Psychology and Statistics

Example Reproducible Report using R Markdown: Analysis of California Schools Test Data

Preliminaries

Load packages and data

Q 1 What does the CASchools dataset involve?

Q. 2 To what extent does expenditure per student vary?

Q. 3a What predicts expenditure per student?

Q. 4 what is the relationship between district level maths and reading scores?

Q. 5 What is the relationship between maths and reading after partialling out other effects?

Q. 6 What fraction of a computer does each student have?

Q. 7 What is a good model of the combined effect of other variables on academic performance (i.e., math and read)?

Now, let's look at some weird stuff.

Q. 8.1 What are common words in Californian School names?

Q. 8.2 Is the number of letters in the school's name related to academic performance?

Q. 8.3 Is the number of words in the school name related to academic performance?

Q. 8.4 Are schools with nice popular nature words in their name doing better academically?

Review

Getting Started with R Markdown, knitr, and Rstudio 0.96

Getting started

Prepare for analyses

Basic console output

Plots

Simple plot

Multiple plots

ggplot2 plot

lattice plot

R Code chunk features

Create Markdown code from R

Create Markdown table code from R

Control output display

Control figure size

Cache analysis

Basic markdown functionality

Dot Points

Equations

Tables

Hyperlinks

Images

Code

Quote

Conclusion

Questions

Annoying <br/>'s

Temporarily disable caching

Equivalent of Sexpr

Image format

How to plot three categorical variables and one continuous variable using ggplot2

1. Create Data

2. Produce Plot

Getting Started with JAGS, rjags, and Bayesian Modelling

What is JAGS?

Why JAGS?

JAGS Installation

Information on JAGS

Examples JAGS Scripts

Asking questions

Interesting Psychological Applications of Bayesian Modelling

New Psychology and Cognitive Science Question and Answer Site: COGSCI.SE

A little history

Reasons to participate as an academic

Reasons to participate as a student researcher

Getting Started

Tips for Undergraduates Interested in a Career in Organisational Psychology: Australian Perspective

Overview of Organisational Psychology

What is the profession called?

What is organisational psychology?

Learning more about the profession:

Organisational Psychology in Australia

Registration

Finding organisational psychology university programs in Australia:

Groups and networking opportunities

Salary Surveys

A career in academia

PhD on a topic related to organisational psychology or related area

Examples of eminent organisational psychology academics

Example academic websites

Journals to read

Additional Resources

Correlation Resources: SPSS, R, Causality, Interpretation, and APA Style Reporting

Introductions to correlation

`ggplot2` plot

`lattice` plot

Annoying `<br/>`'s