Jeromy Anglim's Blog: Psychology and Statistics

Beamer presentations using pandoc, markdown, LaTeX, and a makefile

2012-07-23T23:20:00.000+10:00

This post discusses the creation of beamer presentations using a combination of markdown, pandoc, and LaTeX. This workflow offers the potential to reduce typing and increase readability of beamer presentation source code. Source code for an example presentation is provided containing markdown and LaTeX source code along with a makefile for building the beamer PDF.

Motivation

I've used beamer quite a lot to prepare slides for both research and teaching purposes (e.g., this 2010 presentation on R Workflow). I've also written up a guide to getting started with beamer and a simple beamer template.

Nonetheless, for some time I've been concerned about the high ratio of markup to content in beamer presentations. I even asked a question on TeX.SE on strategies for dealing with this issue. I find that beamer markup is a burden. It interferes with content creation. Creating, editing, and re-arranging slides is more difficult than it needs to be. The high quantity of markup also interferes with readability.

Several ways of dealing with this.

Use LaTeX macros: I.e., to shorten common environments. However, this reduces readability if it is ad hoc.
Org Mode in Emacs. This sounds good, but I'm more experienced with Vim.
Code Snippets. Code snippets partially solve the typing issue, but they don't solve the readability issue.

In the end, I decided to explore the combination of pandoc, markdown, and LaTeX to create a beamer presentation. The reasons for this included that:

Markdown is a really intuitive markup format that is easy to read and easy to modify.
When pandoc converts markdown to LaTeX, any LaTeX is passed straight through. Thus, it is possible to obtain customisation beyond the basic options provided by markdown.

Existing resources on combining Beamer, markdown and pandoc

John MacFarlane, author of pandoc, has some relevant documentation on slide production Important points:

The basic compilation command is: pandoc -t beamer my_source.md -o my_beamer.pdf
The post explains the slide separation rules.
You can have incremental lists by pre-pending dot points with the greater than symbol
Beamer Themes can be used via the -V option e.g., (-V theme:Warsaw)
It shows how the first few lines of the file pre-pended by % are incorporated into the title slide

Stephen Sinclair has a tutorial. Relevant points include:

Latex gets passed directly through
- Equations can be passed directly through
- Image size and placement can be controlled in detail with latex e.g., \centerline{\includegraphics[height=2in]{my_image.pdf}}
- You can use bibtex for citations
He also mentions a number of other options for compiling the document involving templates, regular expressions, and so on.

My approach

My approach involved running a makefile which converted a markdown file into a tex file, which was then incorporated into another tex file and then converted into a pdf.

The repository containing all files is available on github: https://github.com/jeromyanglim/rmarkdown-rmeetup-2012/tree/master/talk
The presentation PDF

To use the approach you would need the following software:

LaTeX distribution with beamer package
pandoc
support for make: On Linux, make is installed by default; on Windows, you may need something like cygwin or Rtools.

makefile

The makefile was as follows:

pdf:
    pandoc talk.md --slide-level 2 -t beamer -o talk.tex
    pdflatex main.tex
    pdflatex main.tex
    -xdg-open main.pdf

pandoc converted talk.md into a beamer latex file talk.tex
--slide-level 2 meant that level 1 markdown headings (i.e., lines preceded with a single hash: # Section name) represented section headings used in the presentation, and level 2 headings (i.e., lines preceded with two hashes ## Slide Title) represented new slides and their title.
The line -xdg-open main.pdf opens the resulting pdf file on Linux, but xdg-open could be replaced by the name of pdf viewer (e.g., on a different operating system).

Preamble LaTeX file: `main.tex`

I had a main LaTeX file (main.tex) as follows:

\documentclass[t]{beamer}  
\usetheme{Berkeley}                 
\setbeamertemplate{navigation symbols}{}
\title{MY_TALK_TITLE}
\subtitle{MY_TALK_SUBTITLE}
\author{MY_NAME}
\institute{MY_INSTITUTION}
\date{DATE_OF_MY_TALK}

% more preamble...

\begin{document}
\begin{frame}
\titlepage
\end{frame}

\begin{frame}
\frametitle{Outline}
\tableofcontents
\end{frame}

\input{"talk.tex"}

\end{document}

The file is completely in latex and includes the preamble the document environment, some opening slides with particular features, and the input command which reads in the file talk.tex.
talk.tex is generated by pandoc from talk.md and contains all the individual content slides.
I prefer to exclude navigation symbols.
Naturally, usetheme could be altered to some other theme (see the beamer theme matrix), such as default.

Body markdown slide file: `talk.md`

talk.md contained all the individual markdown slides.

For example a basic slide might look as follows:

# NAME OF A SECTION
## SLIDE TITLE
* Some point to make
    * Another point
    * Another point
* Some point to make
    * Another point
    * Another point

The first line adds a section title. This is not part of the slide, but can be used to generate table of contents, and in slide navigation.
The second line starts a new slide with the content to the right of the double hash constituting the slide title.
And then subsequent lines generate a two-level list represented in LaTeX using the itemize environment.

In general, markdown is converted by pandoc into sensible beamer content. See the actual markdown file talk.md and resulting tex file talk.tex. However, pandoc passes any LaTeX through as is, and this is sometimes required.

For incorporating images, I found the default markdown image command led to an excessively large image. Thus, I used LaTeX for images. I'd like to think that there is a way of making default images work well, but I didn't work it out. Thus, instead, I used commands like the following:

## SLIDE TITLE
\includegraphics[width=4in]{FIGURE_FILE_NAME.PNG}

I often had to tweak the image width to get it the right size.
I also read about some other options, which I discuss here.

Obviously there are many reasons that you might want to fall back to LaTeX. In my talk, I tried to keep things simple, so the main instances were:

Images as shown above
Small text for footnotes often with a url inside: e.g., \tiny{some text and a link: \url{http://jeromyanglim.blogspot.com}}
Large text at the end of the talk: e.g., \begin{center} \LARGE{Questions?} \end{center}

Conclusion

Overall, there were pros and cons to the approach I adopted.

By incorporating markdown and pandoc, there was an extra layer of complexity to think about to ensure that the conversion process had the desired effect. Error messages were sometimes more difficult to diagnose.
There were a lot of situations where you might want to have more control over slide content than what you get by default with Markdown.
There is an argument to suggest that slide creation is best in a WYSIWIG environment where you can manually tweak image positioning and layout.

Nonetheless, I really liked how easy it was to create, edit, and read dot points, nested dot points, frames, sections, and basic formatting, and in general it was relatively easy to incorporate LaTeX when required. I also like the idea of where possible using open plain-text file formats to take advantage of easier programmability, version control, incorporating into a powerful text editor, simpler conversion, and so on.

Other aspects

The following are a few other aspects that might interest some readers, particularly Vim users.

Syntax highlighting of markdown+LaTeX in Vim

There is a Vim plugin for pandoc that provides many features including syntax highlighting for documents that combine multiple markups including markdown and LaTeX. I found it best to install the latest version available on github: https://github.com/vim-pandoc/vim-pandoc

Code folding of markdown-Beamer

I also have the following script in my .vimrc file. The great thing about it is that it allows code folding if you use hash style markdown headings. It is setup to only fold on headings 1 and 2. This corresponds to sections and slides in my pandoc setting for beamer markdown documents. To increase the level, change it to MarkdownLevel(3), etc.

function! MarkdownLevel(maxlevel)
    if a:maxlevel >= 1 && getline(v:lnum) =~ '^# .*$'
        return ">1"
    endif
    if a:maxlevel >= 2 && getline(v:lnum) =~ '^## .*$'
        return ">2"
    endif
    if a:maxlevel >= 3 && getline(v:lnum) =~ '^### .*$'
        return ">3"
    endif
    if a:maxlevel >= 4 && getline(v:lnum) =~ '^#### .*$'
        return ">4"
    endif
    if a:maxlevel >= 5 && getline(v:lnum) =~ '^##### .*$'
        return ">5"
    endif
    if a:maxlevel >= 6 && getline(v:lnum) =~ '^###### .*$'
        return ">6"
    endif
    return "=" 
endfunction

au BufEnter *.md  setlocal foldexpr=MarkdownLevel(2)  
au BufEnter *.md  setlocal foldmethod=expr     
au BufEnter *.md  setlocal autoindent

Then the following Vim commands in normal model make folding, navigation, and getting a sense of structure really easy:

zx show current line and necessary headings; close other headings
zc close heading
zj and zk to move down and up headings

Showing backticks and single quotes properly in code

I often need to show code, and backticks and single quotes weren't showing properly. The following code in my LaTeX preamble drawn from this TeX.SE question solved the problem:

% enables straight single quote
\makeatletter
\let \@sverbatim \@verbatim
\def \@verbatim {\@sverbatim \verbatimplus}
{\catcode`'=13 \gdef \verbatimplus{\catcode`'=13 \chardef '=13 }} 
\makeatother

% enables backticks in verbatim
\makeatletter
{\catcode`\`=13
\xdef\@verbatim{\unexpanded\expandafter{\@verbatim}\chardef\noexpand`=18 }
}
\makeatother

Converting Sweave LaTeX to knitr LaTeX: A case study

2012-06-10T00:20:00.000+10:00

The following post documents the steps I needed to take in order to convert a project using Sweave LaTeX into one using knitr LaTeX.

Additional Resources

It is fairly straightforward to convert a document from Sweave LaTeX to knitr LaTeX. Yihui Xie on the knitr website provides the following useful resources:

Transition to Sweave from knitr: This document describes knitr specifically from the perspective of what is the same as Sweave and what is different from Sweave.
knitr options: This includes discussion of the many R code chunk options in knitr. Many are the same as Sweave, but there are some new ones, and some modifications.
knitr minimal examples: These are useful for getting started with different types of knitr document including LaTeX.

My conversion from Sweave to knitr

The following documents the steps I needed to do in order to convert a journal article that was in Sweave LaTeX into a knitr LaTeX document. Most of this was documented in the above mentioned links on the knitr website, but there were still a few little surprises.

Rnw to tex conversion: Convert R CMD Sweave myfile.rnw to Rscript -e "library(knitr); knit('myfile.nw')" in makefile (see this SO question ).
global options: Replace \SweaveOpts{echo=FALSE} with \Sexpr{opts_chunk$set(echo=FALSE)}; This needed to appear before the first R code chunk in order to affect all code chunks in the file.
case on R code chunk options: Update true and false to TRUE and FALSE in r code chunk options.
results option: Update results=tex to results='asis' and in general ensure that text values in R code chunks are surrounded by quotation marks.
message option: I needed to prevent the display of messages when certain packages were loaded using \Sexpr{opts_chunk$set(message=FALSE}}.
These messages did not previously display under sweave.
hiding output: I had some R code chunks with options print=FALSE, term=FALSE; I replaced this with results='hide'.
methods package: I had a densityplot() (i.e., a lattice plot) that didn't display properly. It instead showed an error: Error using packet 1 could not find function "hasArg"; apparently this is caused by the fact that the methods package doesn't load by default when using Rscript; thus I needed to put require(methods) in the first R code chunk.
Sweave.sty: I removed Sweave.sty from my project directory and removed the line \usepackage{Sweave} from my rnw file as both things are not needed in knitr.
caching: Although there are packages for enabling caching, I'd never adopted any of them. knitr makes caching very simple. I just added cache=TRUE to the global chunk options (i.e., \Sexpr{opts_chunk$set(echo=FALSE, message=FALSE, cache=TRUE)}. This reduced the time to build the PDF from around 5 seconds to 1 second. I'm also planning to incorporate some Bayesian analyses with JAGS and rjags, where I'm expecting analyses will take several minutes or longer to run. At that point, I'll really appreciate the speed benefits of caching.
to make or not to make: I had a custom makefile on the project that kept everything neat and tidy, copying source files into a build directory, running all necessary commands to convert from rnw to tex and then to pdf, and then opening the pdf in a viewer. This still works well. However, the default "Compile to PDF" option in RStudio was also quite good (after setting tools - options - Sweave - Weave Rnw files using knitr). In particular, I liked the synctex support for Sweave that allows you to move from a position in the source to the corresponding position in the PDF viewer. Also, RStudio in combination with knitr seems to do a reasonable job of keeping the main project directory tidy. A few auxiliary files are added, but not too many. I also appreciate the simplicity that a simple button brings to getting started with analyses. However, a makefile does make things more portable.

My main conclusion from this process is that converting an ongoing Sweave LaTeX document to knitr LaTeX is fairly straightforward, and there are a number of useful benefits that arise. In particular, I really appreciate simple caching and not having to worry about Sweave.sty. Great work Yihui Xie!

Additional Resources

Transition to Linux and Ubuntu: The Experience of a Quantitative Social Scientist

2011-05-23T16:14:00.000+10:00

This post discusses my experience migrating to Ubuntu from Windows from the perspective of a quantitatively oriented researcher in psychology. It discusses (a) my general transition to open source tools for doing research; (b) examination of Cygwin; (c) choosing a Linux distribution; (d) preparing for installation of Ubuntu; (e) initial reflections on installing Ubuntu; and (f) a long list of various solutions to specific problems that I encountered whilst getting started with Ubuntu.

Context of transition

Anyone following this blog for a while will know that I've been progressively migrating to using open source software for doing research. My first foray into open source started in 2007 when I decided to replace SPSS with R.

As is the case with many open source tools, they have a habit of mutually reinforcing each other. The path went something like this (with links pointing to associated blog posts):

LaTeX replaced Word as my scientific document format.
Markdown replaced Word as my document format for note taking and informal documents.
Bibtex, Jabref, and other tools replaced Endnote as a bibliographic reference management system.
Vim replaced a host of other IDEs, and text editors; it even replaced MS Word, in that it became the editor for the plain text file formats (e.g., LaTeX, Sweave, Markdown, etc.) that I use the most.
git replaced ad hoc version control systems.
make replaced various manual processes for building final products.

For a general summary of this workflow (albeit, prior to the adoption of Vim), see this video that I gave at Melbourne R Users in late 2010.

However, I still needed to cut ties from Windows, the operating system I'd been using for 15 years. I'd been putting off adopting Linux for several reasons. I was concerned about:

hardware support for my main computer,
the temporary drop in performance that results from adopting a new operating system,
difficulties collaborating with other researchers operating in the Windows world,
lack of support for various key programs (e.g., Outlook; Inquisit), and
the possibility that some tasks would be less efficient under Linux.

Despite these concerns, I decided to finally dive in.

Initial foray into Cygwin

About a month prior to making the switch I'd installed Cygwin on Windows. This gave me access to many Linux command-line programs. However, ultimately, while Cygwin whet my appetite, it failed to satisfy.

By default, Cygwin does not always play nicely with non-Cygwin programs. File paths often get confused. If it can't play nicely with Windows programs, I might as well go all the way.
There are issues of where to store files (Cygwin home or Windows user home).
Some Linux programs are not included.
The speed of the terminal and shell was often poor.
In general, interacting with other programs was a frustrating experience.

While some of these challenges probably could have been overcome, it felt silly investing time in learning the idiosyncrasies of a system that only goes half-way to creating a genuine unix experience.

Choosing a Linux distribution

The first decision involved choosing a Linux distribution. I decided on Ubuntu for several reasons.

It's a popular Linux distribution.
It seems like a good distribution for people new to Linux.
Hardware support looked encouraging.
I'd heard it was well suited as a personal desktop
I'd also heard that ultimately Linux is configurable and thus, the distribution does not provide major constraints; furthermore basic unix command-line programs should work in all distributions.

Preparing to install Ubuntu

I approached the installation of Ubuntu with trepidation.

I read a lot about partitioning and dual-booting on Ubuntu documentation and psychocats.
I backed up my data (of course) before starting installation.
I thought a lot about setting up my partitions. In the end I decided on a dual-boot partition. I have a solid state drive which is really fast, but capacity is not that high (only 120gb). Thus, I had to be careful with assigning space. I gave 40gb to Windows and 80gb to Ubuntu. This required the removal of many unwanted files from the Windows partition.
I created and trialled the Ubuntu Live USB to check for hardware support, and everything seemed to be working surprisingly well on my Dell Latitude laptop.
I then installed Ubuntu, letting the Ubuntu installer handle the hard drive partitioning.

To my pleasant surprise, the installation was smooth, involving minimal choices. The Ubuntu installation successfully partitioned my hard drive, and the core features of my laptop still worked (sleep, hibernate, wifi, volume buttons, graphics card, and so on). Furthermore, I was still able to boot into Windows for when I really needed to use Windows-only software.

Reflection on the installation

Switching operating systems is painful even when the new operating system is better than the old one. Many little tasks that were once automatic need to be automated in a new way. Furthermore, to fully grok the new system, it is often necessary to do more than map the old workflow onto the new system.

For the most part the switch to Ubuntu has been smoother than I was expecting. The first week involved dealing with a wide range of fundamental issues related to software installation and system configuration. However, in general, these issues have been resolved by a few Internet searches and asking questions on Ask Ubuntu (a Stack Exchange Q+A site).

The following were some of the main issues I encountered:

Linking with my university email / calendar Exchange server has been painful. After two weeks, this still remains my biggest pain; The email program Evolution basically works, but is painfully slow at times, often freezes, and often produces errors.
Configuring the Unity desktop (the default behaviour of the 11.04 launcher is awful) took a bit of searching.
Configuring so that I could move between laptop monitor configurations required some fiddling.
Configuring my vimrc took some work. I had a number of file format issues related to text files having inappropriate line feeds (i.e., Windows text files instead of Unix text files).

Despite these issues I'm really enjoying using Ubuntu. Adopting a Vim + command-line workflow is so much easier, and I'm gradually incorporating some of the great Linux tools into my workflow.

Some reading

As part of the transition, I've also been reading a few books:

Harley Hahn's guide to Unix and Linux - Harley Hahn
- This one was quite pedagogical in flavour and a bit wordy at times, but did provide a helpful introduction to Linux.
UNIX power tools - Jerry Peek, Tim O'Reilly, Mike Loukide
- This book is massive.
- It provides heaps of short articles outlining useful tricks for getting more out of Unix / Linux.
Learning the bash Shell - Cameron Newham, Bill Rosenblatt

Looking forward

As I get settled into the new operating system I'm looking forward to really incorporating the tools available into my research workflow.

Some of the things I'm looking forward to (which I may blog about at some point, see here for Blog RSS subscription options) include:

Writing some bash scripts and Vim commands to enable me to link bibtex citations to stored pdfs of journal articles;
Taking advantage of the command-line for file manipulation, running R, creating LaTeX documents, and more; and
Dabbling in sed, awk, Python for better text manipulation.

Now with the first two weeks of initial adjustment out of the way, the subtler phase of tweaking and refinement begins. If you've made the switch to Linux particularly within the context of research and data analysis, I'd be keen to hear your experience.

Assorted Resolved Issues

Overview

The rest of this post documents a wide range of problems that I encountered when getting started with Ubuntu and the solutions that I adopted. The caveat applies that I'm pretty new to Ubuntu, so the solutions may not be best practice. Furthermore, they may not make for interesting reading unless perhaps you've stumbled onto this post through Google searching for the specific problem.

Overhaul file system

Question

What are the implications of Linux for how my file system should be organised?

Discussion

I used the transition to Linux to overhaul my file system. I was keen to use the terminal to access files. Good file names for terminal access are different to that for a point-and-click environment.

Some general principles that guided the new file system

Remove spaces from file names
- It's a pain to have to escape spaces (\)
- I replaced spaces with hyphen "-"
Convert file names to lower case where possible
- Linux is case sensitive
- It's quicker to type lower case
- Consistency reduces confusion
Prefer file and folder names unique in the first few letters
- This facilitates tab completion
Prefer short, descriptive, and memorable file names

I also found it particularly useful to do a general reflection on my file structure. I cleaned up the hierarchy so that all my core activities had a location in the file system.

Accessing Windows Drive from Ubuntu

Question

Are there any issues with accessing the Windows OS partition from Ubuntu?

Discussion

It's easy. It shows up as a mounted drive called "OS". It's path was /media/OS

Monitoring processes and use of system resources

Question

What is the Ubuntu equivalent of Ctrl+Alt+Delete for viewing active processes and memory and processor usage?

Discussion

There is a program called "System Monitor"

Command prompt for terminal colour

Question

It seems to me that it might be useful to make the command prompt a different colour to help make it clear where one command starts and output begins a little clearer.

How can this be done?
What are good colour options?

Discussion

Sites discussing the issue
- cyberciti
- archlinux wiki
The .bashrc file that came with Ubuntu had a setting that could be enabled to have a colour prompt.

Opening common binary files from command-line

Question

How can common binary files (e.g., PDF, Word, Excel, PowerPoint) be opened from the command line?

Discussion

gnome-open
- This command can be followed by filenames, web addresses, directories, and will typically do some appropriate default action.
- gnome-open . opens the current directory in Nautilus
- alias go=’gnome-open’ or equivalent may be useful

Keyboard shortcuts for size window left or right half of screen

Question

I tend to work on a large monitor with space for Windows on the left and right side of the screen. I like to have a keyboard shortcut which sizes a window to the left or right of the screen.

How can this be done on Ubuntu?

Discussion

Ubuntu does support half Desktop sizing by dragging a window with the mouse to the left or right most side of the desktop.
keyboard shortcuts offers some options for moving windows, including maximise vertically or maximise horizontally, but no option for positioning window on left half of screen.
Ctrl+Alt+NumPad Key
- I learnt the following on askubuntu
- By default Ubuntu supports shortcut keys for window manipulation (e.g., Ctrl+Alt+NumPad4 maximises a window vertically on the left half of the screen and Ctrl+Alt+NumPad6 does the same but to the right.
- The main annoying aspect of this is that these keys are not suitable for touch typing.
Solution
- Install Advanced Desktop Effects settings (Compiz with Compiz Config)
- Start CompizConfig
- Go to Window Manager - Grid
- Change the key bindings (I chose the following, which have a certain consistency with Vim movement keys)
  - Super + h (left)
  - Super + L (right)
  - Super + K (maximise)
  - Super + J (restore)

Shortcut key to initiate web search

Question

On windows I had a global shortcut key (Windows+Z) to initiate a Google search.

How can this be done on Ubuntu?

Discussion

googlizer
- This can be used to create command line programs for searching.
Chrome
- Start chrome
- Type search term in address bar
- Making google.com the home page can speed this up
- Evaluation:
  - The responsiveness of this is not adequate (a search box is instant; chrome takes around 1 second to start up)
  - It results in the loss of the default Chrome homepage
AutoKey
- A program modelled on AutoHotKey might provide the desired functionality.
I asked the question on askubuntu
- Google Desktop Search with Desktop search disabled was a good choice.
- I changed the default shortcut key to Ctrl+F12

Using a laptop with multiple external monitors

Question

How can a laptop monitor be disabled and an external monitor enabled?
I use two external monitors, one at home, and one at work. How can Ubuntu be configured to automatically detect which one is connected?

Discussion

When I configure to just use my external monitor and not my laptop monitor and then unplug the external monitor, the laptop monitor is not activated. Thus, I can't use the mouse to reactivate the laptop monitor. It would be good if Ubuntu could automatically detect the removal of the external monitor, or alternatively I could have a shortcut key to activate laptop only mode.
ubuntugeek describes configuring Ubuntu so that Ctrl+Alt+Backspace can force a restart of X
A user asks about automatic switching of monitor configurations on Ubuntu forums
More suggestions
I asked the question on askubuntu
- I added my answer to the question
- Essentially it involves using a program called disper
- disper -s and disper -S activate primary and secondary monitors respectively. I've configured these commands to be triggered by shortcut keys using CompizConfig.

Launching programs with keyboard shortcuts

Question

What is a good strategy for launching programs from keyboard shortcuts?

Discussion

CompizConfig makes it easy to create commands and assign shortcut keys.
The Unity Launcher enables shortcut keys combining the Super key and a number or typing the Super key and typing a key word.

Restart to Windows

Question

My bootloader defaults to Ubuntu.

Is there a way to shutdown Ubuntu and boot to Windows without having to make a selection from the bootloader?

Discussion

I asked the question on Ask Ubuntu
- I received a good answer, and I added further details on AskUbuntu.

Default terminal size

Question

What is a good default Terminal size?

Discussion

I can always press F11 to get a full-screen terminal. However, sometimes it is useful to be able to have a reasonable size terminal come up that does not take over the whole screen.

I've configured the default profile to be 95 columns by 40 rows. I would choose more rows, but I want the setup to work on my smallest monitor (i.e., my laptop).

Prevent Unity menu from showing

Question

At first I thought the Unity menu was okay. However, I don't like:

the screen real estate that launcher consumes
the way that it defaults to placing left aligned windows indented with space to the launcher
The size of the launcher icons (although at least this can be reduced)
The way that it stays in focus sometimes for a period of time

Discussion

I asked the question on askubuntu and received a good suggestion.

In Compix Config set Unity - Hide Launcher to Auto-hide.

Configuring Ubuntu for running basic scripts

Question

What is a basic strategy for running scripts in Ubuntu?
How do permanently add the custom script to the path?
Where should I store throw-away scripts?
What is a good way to run custom scripts?

Discussion

There is a discussion of where to store scripts here.
A common place seems to be ~/bin

I did the following:

mkdir ~/bin
added export PATH="$PATH:~/bin" to .bashrc

Find file by name

Question

How can I find a file by a name or part of its name within a specified directory (or subdirectories)?

Discussion

Cyberciti provides a page explaining several options.
find command
- The following command would find all files with the txt extension in ~/dir
- find ~/dir -name "*.txt"

Accessing Ubuntu drive from Windows on dual-boot machine

Question

I store my data on my Ubuntu drive.

How can I access my Ubuntu drive from Windows?

Discussion

This page has three options, the last of which, looks like what I want.

Ext2 Installable File System for Windows
- This did not work on Windows 7 (perhaps this will change in the future)
DiskInternals Linux Reader
- Evaluation
  - This worked adequately
  - This has both the benefit and the problem that it does not alter the Linux partition
- How did it work?
  - Start Windows 7
  - Install and Start DiskInternals
  - Save selected Linux partition files onto Windows partition (I had a special section)
  - View, edit, and change files
  - Restart Ubuntu and copy files from Windows partition back into appropriate location

Script fur renaming files in folder

Question

I sometimes download files sent to me and want to convert the file names into more appropriate ones.

The main conversions that I'd like to see are:

replace spaces with dashes ("-")
replace uppercase with lower case

Others may arise as time goes by.

The script should take a list of file names.

How can this be done?

Discussion

I wrote the following script and called it cleannames

#!/bin/bash
for file in * ; do mv "$file" `echo "$file" | sed 's/ /-/g'`; done
for file in * ; do mv "$file" `echo "$file" | sed 's/.*/\L&/g'`; done

Minimise current window with keyboard shortcut

Question

What is the keyboard shortcut to minimise a Window?

Discussion

Alt+F9 minimises
Alt+F10 maximise
Launcher - System Settings - Shortcuts allows for custom shortcut keys

Terminator Terminal

Question

Should I adopt Terminator Terminal?

Discussion

At present I'm using Gnome Terminal.
Various reviews suggest that Gnome terminal is a reasonable option in terms of speed and features.

Preventing automatic shading when watching flash videos

Question

I had an ABC video on full screen and after a few minutes, Ubuntu shaded the screen.

How can this be prevented?

Discussion

Power Management has some options
I disabled the screensaver. That seemed to fix the problem.

Batch renaming of files

Question

Unix is case sensitive. It also encourages substantially greater use of the command-line. As such, it is useful to adopt file naming conventions that make it easier to work with files on the command-line.

How can I batch rename files
- replace underscore with dash
- replace upper case letter with corresponding lower case letter

Discussion

Some options
- KRename
- for loop
  - Tutorial on using for loops
- rename

Convert underscore to dash:

`for file in * ; do mv $file `echo $file | sed 's/_//'` ; done`

Convert upper case letter This tutorial shows how to use the convmv command.

Kill a program

Question

Windows has Ctrl+Alt+Delete which facilitates killing a process.

What does Ubuntu involve?

Discussion

The system monitor lists processes and enables killing processes
Ctrl+Alt+Backspace kills X (once this feature is activated)

Better console / terminal colours

Question

The default background for Ubuntu terminal is a type of dark purple. While it's "pretty", I find the contrast to be relatively low.

What are better settings?

Discussion

I applied the following settings:

Edit - Preferences:
- Colors - Built-in schemes= white on black
- Background = solid colour
export TERM="xterm-256color" enables better colour support for Vim

Problems displaying full screen video

Question

I find that full screen video on YouTube seems to freeze. How can this be fixed?

Discussion

There is a tutorial on YouTube on how to fix this.

sudo mkdir /etc/adobe
sudo su
sudo echo "OverrideGPUValidation = 1" >> /etc/adobe/mms.cfg"

Reducing Grub loader times

Question

I think it would be useful to reduce the default grub display time to something like 1 or 2 seconds. I almost always boot to Ubuntu, and if I want to boot to Windows, I have a command from within Ubuntu that changes the default boot for one occasion.

Is this a good idea?
How can I do this?

Discussion

I think 2 seconds would make sense as this is long enough to easily enter manual mode if I am watching carefully.
sudo vim/etc/default/grub
I modified the value of GRUB_TIMEOUT and made it GRUB_TIMEOUT=2
sudo update-grub

Set up git on Ubuntu

Question

How do I configure git on Ubuntu?

Discussion

General information on configuring git and github on Linux is available here

Video of Reproducible Research with R: Melbourne R Users 1st Dec 2010

2010-12-13T23:40:00.002+11:00

As previously mentioned I gave a talk at Melbourne R Users Group titled "Reproducible Research and R Workflow". It covered technologies including LaTeX, Sweave, R, make, Eclipse, and git. This post shares the video.

I'd like to thank Pedro Olaya for filming the session, and Drew Conway for preparing, uploading, and hosting the video along with the many other great existing videos from R User Groups from around the world.

If you follow Drew's Blog ZIA you may have already seen the video. If you don't follow his blog, you should definitely check it out. Here are three of my favourite posts by Drew:

For those who did not catch Drew's original post, the videos are shown below.

UPDATE: The videos were taken down so I've replaced them with YouTube version.

My Talk

For a copy of the slides, R Code, and more see the previous post. The video is embedded below (If you are viewing this post in a feed reader, and the embedded video is not displaying click here to go to the original post):

Geoff's Talk

The other talk from the session was by Geoff Robinson who discussed several useful strategies for working with R.

The slides and R Script are available for download.

The talk is embedded below:

Melbourne R User Group

Yuval Marom kindly asked me to become a co-organiser of the Melbourne R user Group (MelbURN). While most people who read this blog are from overseas, if you are in Melbourne, feel free to join the group. If you know something about R, are passing through Melbourne, and would like to give a talk on R, let Yuval or me know (see the Contact Us Button), and we'll see if we can organise something.

Related Resources

R Workflow: Slides from a Talk at Melbourne R Users (1st Dec 2010)

2010-12-02T00:03:00.000+11:00

I gave a presentation at Melbourne R Users on the topic of R Workflow. The presentation covered R code organisation, and useful R related tools including Eclipse, StatET, Git, make, Sweave and LaTeX. Also, the slides from the presentation provide links to four complete examples of using R, Sweave, LaTeX, and make.

PDF of Slides

A PDF of the slides is available in fullscreen and four-to-a-page handout format. Most slides contain links to resources for learning more. Thus, it should be possible to understand the main ideas of the talk just by just reading the slides and reading the links.

Video

A video of both my talk and the previous talk by Geoff Robinson will hopefully be made available in the not too distant future (I'll post when it is available).

Github Repository

If you want to learn more about beamer, LaTeX and make, you can check out the source code used to produce the PDF of the presentation. The repository is available on GitHub:

https://github.com/jeromyanglim/RMeetup_Workflow

Thanks

I'd like to thank Yuval Marom for all the work he has put into kickstarting and organising the Melbourne R User Group and Deloitte for providing such a great venue.

Links Presented in the Talk

Sweave Tutorial 3: Console Input and Output - Multiple Choice Test Analysis

2010-11-30T22:23:00.000+11:00

This post provides an example of using Sweave to perform an item analysis of a multiple choice test. It is designed as a tutorial for learning more about using Sweave in a mode where console input and output is displayed. Copies of all source code and the final PDF report is provided.

Overview

The repository with all source files is available at:

https://github.com/jeromyanglim/Sweave_Item_Analysis/.

A copy of the resulting PDF can be viewed here.

For general information on program requirements and running the code see this earlier post

Sweave Documents that Display Console Input and Output

I find it useful to distinguish between different kinds of Sweave documents. One key distinction is between

reports that display the console and reports that do no display the console.

Reports that display the console are suited to distinct applications, including:

R tutorials
Personal analyses
Analyses provided to experts who understand R and the Project

It would be further possible to distinguish between reports that do and do not show the console input (echo=true).

Developing Sweave reports that display the console still benefit from thoughtful variable names, selective display of output and so forth. However, naturally less time is invested in putting the polish on figures, tables, and inline text.

The previous Sweave Tutorials were examples of Sweave documents that do not display the console. Tutorial 1 was a data driven document Tutorial 2 was a set of batch polished reports.

The present tutorial is an example of a Sweave document that displays console input and output.

The remainder of the post discusses various aspects of the source code.

Source Code

Folder and File Structure

.gitignore records the folder where derived files are stored by make.
makefile is similar to that explained and used in Sweave Tutorial 1. This similarity has been obtained through (a) the use of variables in the makefile (b) the fact that both projects are driven by the Rnw file; thus, make calls Rnw, which in turn imports data, and so forth.
README.md is a file in markdown. Markdown is the markup language used on Github. The file is automatically displayed on the repository home page.
data: This folder contains a file with the responses to the 50 multiple choice questions.
meta: This folder contains a file with information about each of the 50 multiple choice questions including the text, response options, and the supposedly correct response.
backup: This folder contains a copy of the resulting PDF. Although this is a derived file and as such should not generally be monitored by Git, it's helpful to include a copy for easy access.
Sweave.sty: I find it easier and more portable to just include this LaTeX style file required by Sweave in with the project.

Item_Analysis_Report.Rnw

Library loading and data import

<<initial_settings, echo=false>>=
options(stringsAsFactors=FALSE)
options(width=80)
library(psych) # used scoring and alpha
library(CTT) # used for spearman brown prophecy
@


<<import_data, echo=false>>=
cases <- read.delim("data/cases.tsv")
items <- read.delim("meta/items.tsv")
items$variable <- paste("item", items$item, sep="")
@

options(width=80) ensures that the console width is suitable for the printed page
options(stringsAsFactors=FALSE) means that character variables imported using read.delim are left as character variables and not converted into factors. In general I find this a more useful default behaviour. In particular I often use the actual text, particularly in metadata, to generate variable names, print text and so forth. Leaving variables as character is better for this.

Using data before it has apparently been generated

...
The example involves performing an item analysis of 
responses of \Sexpr{nrow(cases)} students 
to a set of \Sexpr{nrow(items)} multiple choice test items.
...

<<>>=
<<initial_settings>>
<<import_data>>
@

In the above code I wanted to be able to write the number of cases before showing the code for importing settings and data. Thus, I first ran the code chunks with echo=false to prevent display. Then, afterwards, these code chunks were rerun inside a code chunk using the syntax <<name_of_code_chunk>> (i.e., without the = sign at the end of the opening. This time they were displayed.

Scoring multiple choice tests

<<score_test>>=
itemstats <- score.multiple.choice(key = items$correct, 
            data = cases[,items$variable])
@

score.multiple.choice is a function in the psych package for scoring multiple choice tests. key is a vector of integers representing the correct response. data is a matrix or data.frame of responses from a set of respondents.
the example shows how metadata can be used to simplify code. items$variable includes the name of the 50 personality test items items$correct includes the vector of correct responses.

...

Figures in Sweave

<<plot_mean_by_r, fig=true>>=
plot(r ~ mean , itemstats$item.stats, type="n")
text(itemstats$item.stats$mean, itemstats$item.stats$r, 1:50)
abline(h=.2, v=c(.5, .9))
@

Code chunks can produce single figures. the fig=true key-value pair is required.
type="n" is used to not show points and then text(...) is used to plot the item numbers on the plot.
Because the document is an informal document designed to display the console, the figure is not wrapped in a figure float. A float would involve more typing and might even be annoying if it moved around the document.

...

Using Sweave to Better follow the DRY (Don't Repeat Yourself) Principle

<<flag_bad_items>>=
rules <- list(
        tooEasy = .95,
        tooHard = .3,
        lowR = .15)
oritemstats$item.stats$tooEasy <- 
    oritemstats$item.stats$mean > rules$tooEasy
...
@

\begin{itemize}
\item \emph{Too Easy}: mean correct $>$
\Sexpr{rules$tooEasy}.
\Sexpr{sum(oritemstats$item.stats$tooEasy)}
items were bad by this definition.
... 
\end{itemize}

The above abbreviated version of the actual code highlights how Sweave can be used to prevent repetition and facilitate modifiability.
The code flags items as too easy if more than 95% of participants get the item correct. This value (.95) is stored in a variable. It's then subsequently used both in the code to flag items as too easy and also used in the text where the rule is described in plain text (i.e., \Sexpr{rules$tooEasy}).
This is a particularly powerful use of Sweave whereby any text in a document that might be repeated or any text that describes details of a data analytic algorithm is a good candidate for simplification using Sweave.

...

\Sexpr{} and formatting

The formula suggests  that in order to obtain
an alpha of \Sexpr{sbrown$targetAlpha},
\Sexpr{round(sbrown$multiple, 2)} times as many items are required.
Thus, the final scale would need around
\Sexpr{ceiling(sbrown$refinedItemCount)} items.
Assuming a similar number of good and bad items,
this would require an initial pool of around
\Sexpr{ceiling(sbrown$totalItemCount)} items.

The above code highlights a couple of examples of how inline formatting of numbers can be done, and is often required when including inline text. In this case, ceiling and round functions were used.

Sweave Tutorial Series

This post is the third installment in a Sweave Tutorial Series:

Sweave Tutorial 2: Batch Individual Personality Reports using R, Sweave, and LaTeX

2010-11-29T17:48:00.001+11:00

This post documents an example of using Sweave to generate individualised personality reports based on responses to a personality test. Each report provides information on both the responses of the general sample and responses of the specific respondent. All source code is provided, and selected aspects are discussed, including makefiles use of \Sexpr, figures, and LaTeX tables using Sweave.

Overview

All source code is available on GitHub:

https://github.com/jeromyanglim/Sweave_Personality_Reports

Three examples of compiled PDF reports can be viewed as follows: ID1 ID2 and ID4.

The resulting report is a simple proof of concept example.

Discussion of Source Code

makefile

outputDir = .output
backupDir = .backup

test:
    -mkdir $(outputDir)
    Rscript --verbose run1test.R  

test5:
    -mkdir $(outputDir)
    Rscript --verbose run5test.R  


runall:
    -mkdir $(outputDir)
    Rscript --verbose runAll.R  

clean:
    -rm $(outputDir)/*

backup:
    -mkdir $(backupDir)
    cp $(outputDir)/Report_Template_ID*[0123456789].pdf --target-directory=$(backupDir)

outputDir stores the name of the folder used to store derived files (e.g., tex files, images, and compiled document PDFs)
backupDir stores the name of the folder where document PDFs are to be stored
test: is the default goal. Running make in the project directory will run run1test.R which will build one report.
the --verbose option shows the progress of R when run as a script. It's useful for seeing progress and debugging.
test5: compiles five reports.
runall: compiles all reports. Further information on each of the Run... .R files can be obtained by inspecting these files. In general they source Run.R and specify which ids to run reports on.
clean: removes all the files from the output directory (i.e., all the derived files)
backup: copies reports to the backup folder; i.e., it separates the finished documents from all the other derived files.
To run test5, runnall etc., type make test5 or make runnall etc.

main.R

main.R loads external functions and packages, imports data, imports metadata and processes the data.

# Import Data
ipip <-read.delim("data/ipip.tsv")
ipipmeta <-read.delim("meta/ipipmeta.tsv")
ipipscales <- read.delim("meta/ipipscales.tsv")

When importing the data, I have adopted the useful convention (which I observed from John Myles White's ProjectTemplate Package) of naming objects and data file names the same. The file extension also clearly indicates the file format (i.e., tab-separated-values).
I often have separate data and meta folders. Importing metadata often makes for more manageable code than when incorporating metadata by hard coding it into the R script.

Test scores are calculated using the function score.items.

ipipstats <- psych::score.items(ipipmeta[,ipipscales$scale], 
        ipip[,ipipmeta[,"variable"]],
        min = 1, max = 5)

The psych package has a number of useful functions for psychological research. score.items is particularly good. It enables the creation of means and totals for multiple scales. It handles item reversal. It also returns information related to the reliability of the scales.

Run.R

source("main.R", echo = TRUE)
id <- NULL
exportReport <- function(x) {
    id <<- x
    fileStem <- "Report_Template"
    file.copy("Report_Template.Rnw",
            paste(".output/", fileStem, "_ID", id, ".Rnw", sep =""),
            overwrite = TRUE)
    file.copy("Sweave.sty", ".output/Sweave.sty", overwrite = TRUE)
    setwd(".output")
    Sweave(paste(fileStem, "_ID", id, ".Rnw", sep =""))
    tools::texi2dvi(paste(fileStem, "_ID", id, ".tex", sep =""), pdf = TRUE)
    setwd("..")
}

The above code provides the function to run Sweave on each individualised report
the code is a little bit messy, contains a few hacks, and is not especially robust.
the exportReport function takes an id value as an argument x. Note the use of the alternative assignment operator. (See ?assignOps)
The code is designed to keep derived files away from source files by copying files into the .output folder and even changing the working directory to that directory.
The code creates an individualised copy of the Rnw file; Runs Sweave on the report to produce a tex file, and then runs texi2dvi with pdf=TRUE to produce the final pdf.

Report_Template.Rnw

The Rnw file contains interspersed chunks of LaTeX and R code.
Because the Rnw file is called from within R, all the R objects and data processing code does not need to be called at the start of the Rnw file. This approach is one way of reducing the time it takes to run a set of Sweave reports all based on a common data source.
The \Sexpr{} command is used to incorporate in-line text. (... sample of \Sexpr{nrow(ipip)} students ...). In the example above, it prints the actual number of cases into the ipip data.frame (i.e., the sample size).

Incorporating a figure using Sweave

\begin{figure}
<<plot_scale_distributions, fig=true>>=
plotScale <- function(ipipscale) {
    ggplot(ipip, aes_string(x=ipipscale["scale"])) + 
        scale_x_continuous(limits=c(1, 5),
            name = ipipscale["name"]) +
        scale_y_continuous(name = "", labels ="",   breaks = 0) +
        geom_density(fill="green", alpha = .5) +
        geom_vline(xintercept = ipip[ipip$id %in% id, ipipscale["scale"]],
                size=1) 
}


scaleplots <-   apply(ipipscales, 1, function(X) plotScale(X))

arrange(scaleplots[[1]], 
        scaleplots[[2]],
        scaleplots[[3]],
        scaleplots[[4]],
        scaleplots[[5]],
        ncol=3)
@
\caption{Figures show distributions of scores of each personality factor
in the norm sample.
Higher scores mean greater levels of the factor. 
The black vertical line indicates your score.}
\end{figure}

The first R code chunk produces a figure using ggplot2.
The code above takes a while to run (perhaps around 10 seconds on my machine). But the resulting plot is more attractive than what I could easily get with base graphics.
<<plot_scale_distributions, fig=true>>= indicates the start of an R code chunk. fig=true lets Sweave know that it has to produce code to include a figure.
The R code chunk is substituted with \includegraphics{Report_Template_ID10-plot_scale_distributions} in the tex file and the pdf and eps figures are created. Thus, if you want a float with captions and labels, you have to add them around the R code chunk.
the plotScale function is used to generate a ggplot2 figure of the distribution of scores on each personality scale along with a marking of the respondent's score on each scale.
The arrange function is used to layout multiple ggplot2 figures on a single plot. The source code is in the lib/vp.layout.R and was taken from a [post by Stephen Turner( http://gettinggeneticsdone.blogspot.com/2010/03/arrange-multiple-ggplot2-plots-in-same.html)

Preparing a formatted table in R for LaTeX

<<prepare_table>>=
ipiptable <- list()
ipiptable$colnames <- c("item", "scaleF", "text", "meanF",
        "sdF", "is1F", "is2F", "is3F", "is4F", "is5F")
ipiptable$cells <- ipipsummary[,ipiptable$colnames ]
ipiptable$cells$item <- paste(ipiptable$cells$item, ".", sep="")

# assign actual respones to table
ipiptable$cells[,c("is1F", "is2F", "is3F", "is4F", "is5F")] <-
        sapply(1:5, function(X)
        ifelse(as.numeric(ipip[ipip$id %in% id, ipipmeta$variable]) == X, 
                paste("*", ipiptable$cells[[paste("is", X, "F", sep ="")]], sep =""),
                ipiptable$cells[[paste("is", X, "F", sep ="")]]))


ipiptable$cellsF <- as.matrix(ipiptable$cells) 

ipiptable$cellsF <- ipiptable$cellsF[order(ipiptable$cellsF[, "scaleF"]), ]

ipiptable$row1 <- c("", "Scale", "Item Text", 
        "M", "SD", "VI\\%", "MI\\%", "N\\%", "MA\\%", "VA\\%")

ipiptable$table <- rbind(ipiptable$row1, ipiptable$cellsF)
ipiptable$tex <- paste(
        apply(ipiptable$table, 1, function(X) paste(X, collapse = " & ")), 
        "\\\\")
for(i in c(41, 31, 21, 11, 1)) {
    ipiptable$tex <- append(ipiptable$tex, "\\midrule", after=i)
}
ipiptable$tex1 <- ipiptable$tex[c(1:34)]
ipiptable$tex2 <- ipiptable$tex[c(1,35:56)]

I often find it useful to split R code chunks for table preparation and table presentation. In general this allows any text that appears before the table to include \Sexpr{} commands incorporating figures from the analyses which generate the table. In the present case, it was useful because the table was split over two pages.
The code shows some of the general logic I use for customised table creation. In hindsight I could probably refactor it into a function so that I don't have to always type ipiptable which would make things a little more concise
The general process of table creation involves: (a) extracting information on cells with cells often grouped into types which will receive common formatting treatment (b) formatting cells (e.g., rounding, decimals, and so on) (c) assembling the cells typically using a combination of the functions rbind and cbind (d) Inserting tex column and end of row separators with something like: paste(apply(x, 1, function(X) paste(X, collapse = " & ")), "\\\\") where x is the matrix of table cells.

Don't Repeat Yourself Principle using R and Sexpr{}

ipiptable$caption <-
        "Response options were 
1 = (V)ery (I)naccurate, 
2 = (M)oderately (I)naccurate,
3 = (N)either Inaccurate nor Accurate, 
4 = (M)oderately (A)ccurate
5 = (V)ery (A)ccurate.
Thus, VI\\\\% indicates the percentage of the norm sample 
        giving a response indicating that the item is a Very Inaccurate
description of themselves.
Your response is indicated with an asterisk (*)."

This text was used in both tables. Thus, this text can then be called using \Sexpr{ipiptable[["caption"]]}. This follows the DRY principle (Don't Repeat Yourself). Thus, if the caption needs to be modified, it only needs to be modified in one place.

Incorporating the tex formatted table using R Code chunks

\begin{table}
\begin{adjustwidth}{-1cm}{-1cm}
\caption{Table of results for (A)greeableness, (C)onscientiousness
and (E)motional (S)tability items.
\Sexpr{ipiptable[["caption"]]}} 
\begin{center}
\begin{tabular}{rrp{4cm}rrrrrrr}
\toprule
<<table_part1, results=tex>>=
cat(ipiptable$tex1, sep="\n") 
@
\bottomrule
\end{tabular}
\end{center}
\end{adjustwidth}
\end{table}

The tables are then incorporated into the tex file.
The R code only generated some of the required tex for the table. Thus all the other desired elements such as the table environment and captions are written either side of the R code chunk.
the R code chunk uses the option results=tex in order to enter the output from the cat function verbatim into the resulting tex file.
cat(ipiptable$tex1, sep="\n") includes a vector of tex. With the newline separator simply making the resulting tex more readable.

Additional Resources

Getting Started with Git, EGit, Eclipse, and GitHub: Version Control for R Projects

2010-11-29T13:55:00.000+11:00

This post provides information on
(a) installing Git using the Eclipse plugin Egit. (b) uploading repositories to GitHub, and (c) links to resources on Git, Git and LaTeX, and Git and R. The focus is on version control for people working on R, Sweave, and LaTeX related projects.

Overview

Version control works really well with R, Sweave, and LaTeX projects.

Benefits of Version Control

There are many benefits to version control for the data analyst. Version control allows you to:

Rewind a project or a file to a previous state, which in turn encourages experimentation
Ensure there is a record of changes
Facilitate collaboration
Facilitate backup
Show changes between files
Facilitate code sharing and reproducibility
and much more... See this question on StackOverflow for further discussion.

I also found that adopting version control facilitated several conceptual benefits. It encouraged greater consideration of:

the distinction between source and derived files
the nature of dependencies:
- dependencies between elements of code
- dependencies between files within a project
- and dependencies with files and programs external to the repository
the nature of a repository and how repositories should be divided
the nature of committing and documenting changes and project milestones

Choosing a Version Control System and Workflow

Why Git?

There are many version control systems (see Plastic SCM for a discussion). I've chosen to use Git for the following reasons:

Git can work well with Eclipse and Windows using Egit and many other tools
Git is one of the popular version control systems
Git enables uploading to Github
Git has good documentation and support material
Experts, who know a lot more about version control than I do, use Git (e.g., Hadley Wickham); the designer of Git is Linus Torvalds.

Finally, the big difference is between using a version control system and not using a version control system.

Why EGit?

EGit is a Git plugin for Eclipse. I use Eclipse and StatET to write R code and Sweave documents. I found EGit a particularly easy tool for getting started with Git and version control. The documentation is straightforward and the interface is easily integrated into my Eclipse workflow.

Getting Started with EGit and Git in Eclipse

There are many ways to interact with Git.

Installing EGit in eclipse involves using the update manager. Vogella.de has a tutorial.

To get started with your first Git repository in Eclipse, check out the EGit user Guide. When I was first getting started I used a simple R project rather than a Java Hello World application.

GitHub

GitHub is one of several sites for sharing git repositories (for example, see Hadley Wickham's baby names analysis, or my own example of using Sweave to write Multiple Choice Questions). It also has many useful social networking features.

Uploading a repository to GitHub from Eclipse

Set up a free account on https://github.com/
Work through the tutorial on creating a repository at GitHub

While the above tutorial briefly mentions SSH Configuration, it does not go into detail. When setting up my SSH key, I did the following:

Click Eclipse -- Window -- Preferences -- General -- Network Connections -- SSH2 -- Key Management
Click on Generate DSA Key
Type in a passphrase (i.e., a long and robust password)
Click Save Private Key (I saved it to a new folder under my user account)
Go to this new folder and open "id_dsa.pub" as a plain text file and copy the contents of the file to the clipboard.
Go to github.com -- Account Settings -- SSH Public Keys and click Add another public key.
Paste the public key into the box and give it a name

Using gist.github

Gists provide a quick way to get started with GitHub. Gists are useful for storing and sharing snippets of code. The result can be embedded into blog posts. To get formatted R code, give the file name a ".r" file extension (e.g., "test.r") (thanks to Hadley Wikham)

A simple example of an embedded gist is shown below:

Interesting R GitHub repositories

Good examples of people sharing R projects on GitHub include:

See the suggestions on Stats.SE. I also have an account in case you are interested.

Additional Resources

General Git

Git and LaTeX

Discussion on the Academic Productivity Blog on the topic of version control and LaTeX
Tex.SE has several questions on LaTeX and git
Article on tools for collaborative writing of scientific latex documents
vc package on CTAN supports Git

Git and R

Sweave Tutorial 1: Using Sweave, R, and Make to Generate a PDF of Multiple Choice Questions

2010-11-27T00:31:00.006+11:00

In this post I present an example of using Sweave to prepare a PDF of formatted multiple choice questions. More broadly the example shows how to use Sweave to incorporate elements of a database into a formatted LaTeX document. It aims to be useful to anyone wanting to learn more about the almost magical powers of make, Sweave, and R.

Overview

The repository with all source files is available at:

https://github.com/jeromyanglim/Sweave_MCQ.

The repository allows you to download all files as an archive or view the files individually on the web. A copy of the final PDF generated from the process is available here

I ran the code on Windows with the following programs installed.

R: For the R code
Rtools: For make and the sh commands in Make and for Sweave to run on the command line
MikTeX: For compilation of the PDF using texify and the default downloading of the exam document class

It should run on MAC and Linux with appropriate R, make, and LaTeX tools installed.

Assuming you have the above installed, to run the code

Download the repository from github
Uninstall to a directory
Open the shell in that directory
Type: make

The remainder of this post explains the code in each of the main files in the repository.

The Makefile

The makefile is used to build the PDF from the Rnw Source. It also performs other useful tasks. A copy of the makefile is shown below:

output = .output
rnwfile = Sweave_MCQ
backup = .backup

all:
    R CMD Sweave $(rnwfile).Rnw
    -mkdir $(output)
    -cp *.sty $(output)
    -mv *.tex *.pdf *.eps $(output)
    cd $(output); texify --run-viewer --pdf $(rnwfile).tex 

tex:
    cd $(output); texify --run-viewer --pdf $(rnwfile).tex

clean:
    -rm $(output)/*

backup:
    -mkdir $(backup)
    cp  $(output)/$(rnwfile).pdf $(backup)/$(rnwfile).pdf

I recently posted on the benefits of makefiles when developing Sweave documents.

The make file starts with three variables.

output stores the name of the folder where where all derivative files are placed.
rnwfile stores the name of the Rnw source file without the .Rnw extension. This is also the base of the resulting .tex and .pdf files.
backup stores the name of the folder where a copy of the pdf is placed if this is desired.

The file then has four goals.

The default goal is called all:. If make is called without argument from the command line in the project directory, the recipe immediately below all: is run. Note that all apparent indentations are tab indentations (a set of spaces would cause an error).

The first line runs Sweave on the Rnw file from the command line. On windows this R CMD Sweave requires installation of Rtools Note how $(rnwfile).Rnw will actually be Sweave_MCQ.Rnw after variable substitution.
The second line creates a directory that corresponds to the value in the variable output. The hyphen at the start of the line ensures that any errors, such as if the folder already exists, do not stop make from running. Note that commands such as mkdir, cp, cd, and rm are based on the sh shell. These commands are supported on Windows if you have Rtools installed.
The third line copies Sweave.sty into the output folder.
The fourth line copies tex, pdf and eps files (i.e., those generated by the Sweave command) into the output folder. This is done to ensure that the root directory only includes source files. This has several benefits. (a) It makes version control easier;
(b) it makes it easy to see the source files and (c) it reduces the risk of accidentally deleting source files when deleting derived files.
the fifth line changes the home directory to the output directory and then runs texify on the tex file generated from Sweave. The flags ensure that a pdf is generated and that the default viewer is initiated. This command could be modified to something like pdflatex or some other latex program.
By changing the directory, all the derived latex files are kept in the output folder.

The tex: goal can be called by running make tex at the command line. I use it in case I want there is an error in the when compiling the pdf from the tex file. Sometimes its easier to work out where the bug is by manipulating the intervening tex file. Of course once the problem has been identified, it needs to be incorporated into the Rnw source.

The clean: goal removes all files in the output directory (i.e., all the derived files)

The backup: goal copies the resulting pdf into the backup folder. I figured this might be useful in order to include a copy of the final product in the repository.

.gitignore

/.output
.project

The .gitignore file prevents all files in the /.output directory (i.e., the derived files) and the file .project from being placed under version control in git.

I'm preparing a post on version control, git, and github which will be posted shortly.

Sweave_MCQ.Rnw

Sweave_MCQ.Rnw is the R noweb file that contains chunks of LaTeX and R code. When Sweave is run on this file, the R code chunks are converted into tex and, potentially, image files are generated.

LateX Preamble

\documentclass[12pt, a4paper]{exam}
\usepackage[OT1]{fontenc}
\usepackage{Sweave}
\SweaveOpts{echo=FALSE}
\usepackage{hyperref}            
\hypersetup{pdfpagelayout=SinglePage} % http://www.tug.org/applications/hyperref/ftp/doc/manual.html
\setkeys{Gin}{width=0.8\textwidth}
\pagestyle{headandfoot} % every page has a header and footer
\header{}{Sample Multiple Choice Questions}{}
\footer{}{Page \thepage\ of \numpages}{}

The latex preamble is mostly general code that ensures proper display of the reuslting document.

The exam document class is great for writing a variety of exam style documents in LaTeX. See CTAN - exam for documentation.
hyperref is used to display hyperlinks and allows the resulting pdf to open in SinglePage format.
\setkeys... controls the width of Sweave figures relative to the paragraph width. There are no figures in this documents; so it is not really required.
\pagestyle{headerandfoot}... These three lines ensure the display of header and footer information on each page.

First R Code Chunk

<<prepare_data>>=
items <- read.csv("data/items.csv", stringsAsFactors = FALSE)

writeQuestion <- function(x){
    c("\\filbreak",
            paste("\\question\n", x["itemText"]),
            "\\begin{choices}",
            paste("\\choice", x["optionA"]),
            paste("\\choice", x["optionB"]),
            paste("\\choice", x["optionC"]), 
            paste("\\choice", x["optionD"]), 
            "\\vspace{10 mm}",
            "\\end{choices}\n\n")
}

itemText <- apply(items, 1, function(X)  writeQuestion(x = X))

answers <- paste(items$item, "=",
        LETTERS[as.numeric(items$correctAnswer)],
        sep ="")
answersText <- paste(answers, collapse = "; ")
@

R code chunks in Sweave are commenced by <<>>= and ended by @. These need to appear in the first column of the text file.
<<prepare_data>>=: The first non-keyword placed in the opening tags provides a name for the R code chunk. A short descriptive title is useful both when reading the source and when debugging Sweave compilation.
items...: this line reads in a csv file into a data frame with 40 cases. Each case is a multiple choice question with fields such as the question text, the text for the four response options, and the correct answer.
writeQuestion...: This is a function which is designed to take a row of data from the items data frame and return a latex formated character vector, where each element is ultimately be printed on its own line of the tex file. Note how in order to produce one backslash in LaTeX, two backslashes, need to be written. The \question, \choices, and \choice commands are part of the exam document class and are used for formatting multiple choice questions.
apply... takes the items data frame and for each row (1=rows) runs the function writeQuestion on the row.
answers... and answersText create a formatted string that shows item numbers and letters for correct answers, all drawn from the items data frame.

Remaining code

\begin{questions}
<<print_items, results=tex>>=
cat(itemText, sep = "\n")
@
\newpage
\section*{Answers}
<<print_answers, results=tex>>=
cat(answersText) 
@

\end{questions}

The second R code chunk has a descriptive name print_items. It uses the key-value pair results=tex. This ensures that Sweave interprets the text outputed using cat as raw tex.
cat... prints the long character vector itemText containing all the latex for the questions. sep="\n" means that each element is printed on a new line which makes the resulting tex file easier to read.

Summary and Related Resources

The combination of make, R, Sweave, and LaTeX is tremendously powerful. Hopefully, this post encourages a few more people to have a play. To learn more check out some of the following posts and pages:

Getting Started with Sweave
makefiles for Sweave, R and LaTeX using Eclipse on Windows
Question on Stats.SE 'Complete substantive examples of reproducible research using R'
UPDATE: After posting this article on my blog, Christophe Lalanne let me know about a paper by Bettina Grün and Achim Zeileis called 'Automatic Generation of Exams in R' in JSS. It also uses Sweave, LaTeX, and the exams document class.

makefiles for Sweave, R and LaTeX using Eclipse on Windows

2010-11-23T16:09:00.003+11:00

This post provides a brief introduction to make and makefiles. In particular it describes how to set up make on Windows with an emphasis on using make in Eclipse on projects involving R, Sweave, and LaTeX.

Overview

make is software that uses makefiles to build projects. make has many benefits.

One click build
Reliable build (always build derived files that need to be built; i.e., ensure build is up to date)
Fast build (i.e., don't build things that don't need to be built)
Clean up a project

make has won the ACM Software System Award. It shares this honour with two of my other favourite tools: TeX and S.

make has also encouraged me to think more about dependencies, one-click builds, and project file structures. Specifically, I use make in conjunction with R, Sweave, LaTeX, and other command line tools to build reproducible research output. I'll post some examples in the near future.

Installation on Windows

Option 1: Install Cygwin
Option 2: Install Rtools and add Rtools to the Windows path (installation can do this for you);

In order to run Sweave on the command line on Windows using: R CMD Sweave foo.Rnw:

Add R binaries to the Windows Path (e.g., C:\Program Files\R\R-2.11.1\bin; see here for instructions)
Have Rtools installed

Introduction to R has more information about running R at the command line.

Running makefiles in Eclipse

I use Eclipse to edit R and Sweave files. To configure make in Eclipse:

Run - External Tools - External Tools Configurations
New
- Name: make
- Location: location of make file; e.g., C:\Program Files\GnuWin32\bin\make.exe
- Working Directory: ${Project_loc}

Thus, with a makefile in the root directory of a project, Run - External Tools - Make runs the makefile.

Also note:

Whenever a directory is added to the Windows Path, Eclipse needs to be restarted for this to take effect. Sometimes it seems that Windows itself has to be restarted.
Editing makefiles in Eclipse requires the use of tabs. The following setting enables tabs instead of space substitution (Window - Preferences - General - Editors - Text Editors - Insert Space for Tables (uncheck)); or edit the makefile in another text editor such as notepad.
For LaTeX I have MikTeX installed; this should be on the Windows path

Resources for Getting Started with makefiles for Sweave

Ross Ihaka has a quick introduction to makefiles focusing on Sweave, R, and LaTeX
Charlie Gibbons wrote a document with LaTeX, Sweave, and Beamer examples of makefiles
Gregor Gorjanc shows an example Sweave make file using his custom Sweave command
Documentation on GNU Make
Running command line programs in Windows
Getting Started with Sweave: R, LaTeX, Eclipse, StatET, & TeXlipse

Getting Started with Writing Mathematics in LaTeX

2010-10-23T20:22:00.000+11:00

LaTeX and mathematics go well together. At the risk of preaching to the converted, this post sets out (a) reasons to learn to write mathematics in LaTeX, (b) a few free internet guides on learning to write mathematics in LaTeX, (c) several internet resources which can facilitate writing mathematics in LaTeX.

Reasons to Learn to Write Mathematics in LaTeX

There are many good reasons to learn how to write mathematics in LaTeX.

LaTeX encourages mathematical writing. It's just plain text. It's easy to write, copy, and paste formulas. That which is easy gets done.
Many websites such as Wikipedia, Stackexchange, Wordpress blogs, and so on use LaTeX to render mathematics. Thus, learning mathematics and LaTeX pays dividends in many places.
LaTeX demystifies the complexity of mathematical texts. Before learning LaTeX, my mind boggled at the apparent complexity of mathematical documents. All the numbering, cross-references, and symbols that appeared so complex, appear relatively straightforward from the perspective of LaTeX.
LaTeX fosters skills in verbalising mathematics. The markup is similar to verbalised mathematics. Along with pronunciation guides and mathematical video courses, writing LaTeX can be particularly useful for the self-taught social scientist.

Getting Started

If you're completely new to LaTeX, check out this guide by David R. Wilkins. In regards to getting started with writing mathematics in LaTeX, the internet is filled with resources. The following are two goods guides:

Setting up a Learning Environment

The following links provide an environment of rapid feedback useful when learning to write mathematics in LaTeX.

A text editor that provides lists of mathematical symbols: I use WinEdt, which has a GUI for many mathematical symbols. But many text editors provide such support.
Codecogs has a real time online LaTeX equation editor.
Uni Colorado provides a quick to load extensive list of symbols
Scott Pakin provides a big download with a comprehensive list of LaTeX symbols
DeTeXify allows you to draw the desired symbol online and the program attempts to guess the LaTeX symbol.
Tex.SE is a great site for getting answers to LaTeX related questions.

Basic Working LaTeX file

The following provides the simplest of working environments which can be useful when first learning.

\documentclass{article}
\usepackage{amsmath}
\begin{document}

\end{document}

Getting Started with Beamer: Tips and Tricks for LaTeX Presentations

2010-08-25T22:26:00.001+10:00

This post provides a guide to getting started with Beamer, a popular LaTeX package for preparing slide presentations. The post: (a) Lists some of the benefits of Beamer in comparison to PowerPoint; (b) Links to tutorials and suggestions for learning Beamer for both people who do and do not know LaTeX; (c) Documents problems that I encountered when learning to use Beamer and the solutions that I developed to overcome them.

Overview Of Beamer

Benefits of Beamer

In contrast to Microsoft PowerPoint, the following are some of the benefits of Beamer:

Outputs to PDF which allows for consistent appearance across versions and Operating systems
Free as in no cost and free as in the ability to view and modify the source code
Simple generation of slide navigation tools
Robust incorporation of mathematical formulas
Text based format enables programmatic incorporation of slides and use of text manipulation tools to alter presentation
Easier to create different versions of slides for handouts, articles, and the main presentation
As with LaTeX in general, it is easier to create reproducible research
Encourages good practice in slide construction and formatting (e.g., consistent formatting; useful defaults; strong support for navigation; encourages a focus on content rather than fancy effects; etc.)
Sequencing and highlighting of presentation text is easy (i.e., overlays)

Getting Started

The starting point for learning Beamer depends on whether you already know LaTeX.

If you don't know LaTeX:

Then you'll want to learn about LaTeX before getting started with Beamer. You'll need:
(a) A text editor. I currently use WinEdt 6.0, which integrates with MikTeX particularly well. TeXnicCenter also integrates with MikTeX well. See this discussion on StackOverflow regarding good LaTeX editors.
(b) A LaTeX distribution. I use MikTeX. MikTeX appears to be a popular option on Windows. It does a good job of automatically downloading packages, such as beamer.

Then spend a couple of weeks learning LaTeX by reading materials about LaTeX and applying the ideas to a document you are currently working on. There's heaps of good materials online, such as:

If you have the money or a good library, its worth reading a few books.

If you already know LaTeX:

If you already know LaTeX but you don't know Beamer, I'd suggest the following steps.

1. Get Started:

This tutorial on Math-Linux shows a very basic beamer presentation. Run the code in your text editor to ensure your installation is working. Get a flavour for the essential elements of a Beamer document.

2. Get Oriented:

Charles Batts has a Beamer Tutorial. It introduces many important ideas. It also has the benefits of being in Beamer
Sascha Frank has provided a set of eight example slides showing LaTeX source and PDF. Reverse engineering these slides is a good way to learn how Beamer works.
Norm Matloff also has a useful quick tutorial on Beamer.
If you like videos, you might also want to check out GM-RAM's introductory videos on Beamer.

3. Learn the detail

The Beamer user guide is excellent. It is over 200 pages. It's worth reading in its entirety.

More tips, if you want them

tex.stackexchange.com is a great place to ask latex-related questions including Beamer questions.
Rob Hyndman gives some tips for beamer conference presentations.

well structured beamer presentations

Rouben Rostamian has a great Beamer guide.
Krzysztof Fleszar has a beamer tutorial that has lots of examples highlighting the effect of particular code.
Matthias Pospiech presents an extensive beamer example.

General Reflections

I've used the transition to Beamer to reflect on my slide creation strategies.

Use sections and subsections wisely. Think of the document in terms of a hierarchical structure.
Incorporate Tables of contents to show the outline of the talk. In long talks remind the audience of where you are in the structure.
In longer talks, use the visual section display tools
Use slide text to support the talk, not supplement the talk. Only include extended passages of text, where the exact wording is particularly important, such as for quotes and definitions. If extended passages of text are important enough to include, they should generally be read out verbatim, so that the audience is not conflicted between whether to listen to the speaker's words or read the slides. Most of the time, extended passages of text should be avoided. The words of the talk should extend the text displayed on the screen.
If a more comprehensive handout is required such as for future reference or for people who did not attend the talk, a separate document should be prepared. Beamer makes this relatively easy with handout and article modes.
Prefer more slides to fewer slides with too much information. If there are a large number of points on a slide, the slide should be split into multiple slides.

Challenges Resolved

The following reflects a few of the problems that I encountered when first learning Beamer and the solutions I adopted. Many of the solutions were generated by reading the Beamer User Guide.

1. Fitting text on a slide

Problem: PowerPoint automatically adjusts the size of text to fit on a slide. What is the Beamer approach to this problem?

Solution: Users of Beamer and LaTeX actively discourage such ad hoc adjustment of font sizes. In such situations the slide should be restructured. However, shrink=5 can be added as a frame option (see UMBC). Alternatively, the allowframebreaks is an option of the frame environment. It can be used to extend text over multiple slides.

2. Handouts with only a subset of slides

Problem: When I create presentations, I often include slides that I don't want included in the handout. For example, I may wish to include a series of photos in the presentation, but leave them out of the handout.

Solution: See nabble for a discussion. I add handout:0 to the frame as follows:

\begin{frame}<handout:0>{Title of Frame}

To print handouts the document class is changed as follows:

\documentclass[handout, t]{beamer}

A second option is to add a line in the preamble like this:

\includeonlyframes{handout}

Then each frame that is to be included in this version should have the additional label.

\begin{frame}[label=handout]

3. Full Screen Images

Problem: How do you create slides with full screen images?

Solution: It is useful to know that the default frame size is 128mm (width) by 96mm (height). One solution is suggested on this news group. The beamer user guide also mentions the textpos package as a tool for positioning figures absolutely.

The following strategy worked, but not when I had \mode* in operation.

{
\usebackgroundtemplate{\includegraphics[width=\paperwidth]{figure}}
\begin{frame}[plain]
\end{frame}
}

The following seems to work when I have \mode* operating in the document.

\mode<all>
{
\usebackgroundtemplate{\includegraphics[width=\paperwidth]{figure}}
\begin{frame}[plain]
\end{frame}
}
\mode<all>{\usebackgroundtemplate{}}
\mode*

4. Incorporating Notes

Problem: In general, when I prepare a presentation, I like to write out the main points that I want to make. When it comes time to giving the presentation, I don't read the notes, but the act of writing the notes, ensures that I have thought about the points that I want to make. If I need to give the presentation a second time, I'll read over the notes again. In some settings, I'll make a copy of the presentation available that includes both the slides and the notes. Thus, in Beamer I was looking for a good system that (a) enabled the writing of notes in an around the latex source for each slide (b) enabled easy toggling between PDF versions of the presentation with and without notes.

Solutions:
A. Beamer does allow for the \notes command.

B. However, my preferred solution is to type normal sentences after each slide; i.e., after \end{frame}. I then add \mode* just after \begin{document}. In general this means that text outside frames is not displayed in the presentation. The benefit of this approach are that (a) these notes can easily be converted into a written text using the article mode; and (b) text can be written in a natural way without the need to enclose it in a particular environment or command. More information about \mode* and \mode is available in the beamer user guide. The user guide also has advice (21.2.2 Workflow) on how to manage the production of an article and a presentation version of a document.

5. Hyperlinks within the document

Problem: How do you set up buttons and links from one slide to another?

Solution:

Section 11 of the Beamer user guide has a section on Adding Hyperlinks and Buttons.
UMBC has a simple example of using hyperlink buttons.

6. Removing Superfluous materials

Problem: Many of the standard templates include many extra lines above and below the slide that reduce the space available for slide content. How should this be dealt with?

Solution: The advice on most beamer-related websites regarding navigation is as follow. If the presentation is short and simple, don't include navigation. If the presentation is long or complex, include navigation. The loss of slide-space to navigational elements is worth it. Material can be restructured to use more slides.

Similarly, the footlines are typically used to put things like presenter's name, institutional affiliation, title, page number, and so on. The general advice on beamer-related websites is that if your audience knows all this information already (e.g., lecture in a university subject), then don't included it. If your audience doesn't know this information (e.g., conference presentation) include it.

It is possible to temporarily remove remove headlines, footlines and sidebars by using the [plain] option in the frame environment. This is particularly useful for slides that show a single large image.

The following code in the preamble can be used in order to create a sparse style:

\usetheme{default}

7. Forward and Inverse Search with PDF

Problem: I'd previously been using latex and a DVI viewer for compiling documents. I decided to make the transition to pdflatex. pdflatex makes it easier to work with graphics supporting png, jpg, and pdf, as opposed to latex which supports eps. Unfortunately, Adobe Acrobat does not support forward and inverse search which makes it a lot easier to edit latex source.

Solution: I downloaded and installed Sumatra PDF. Configuring was simple in WinEdt 6.0.

Options - Execution Modes - PDF Viewier - Sumatra PDF

There is further discussion here.

In WinEdt the default keys are: Ctrl+Shift+R to show PDF, and Shift+F8 to do forward search. Double clicking in the PDF goes to the latex source.

8. Improving the WinEdt 6.0 Tree View

Problem: WinEdt 6.0 has a tree view which by default shows various aspects of the structure of latex documents, such as sections and subsections. I wanted to be able to see frames underneath sections and subsections. This would allow me to navigate more easily to particular slides. It would also help in seeing where I was in the hierarchical structure of the presentation.

Solution: I added the following code to tree.ini. I simply adapted it from an existing example. It works, but I am not sure what all the lines of code are doing. It also does not work with some less common frame declarations.

  ITEM="\begin{frame}{?}"
    MODE="TeX"
    CASE_SENSITIVE=1
    BEGINNING_OF_LINE_ONLY=1
    CURRENT_DOCUMENT_ONLY=1
    ALL_OPENED_DOCUMENTS=0
    COMPLETE_PROJECT_TREE=1
    ICON="Paragraph"
    LEVEL=26
    CAPTION="%?"
    BALANCED="{}\"
    MAX_LINE_SPAN=3
    ON_CTRL_CLICK="GlobalMark;TreeTrack(2);"
    ON_CTRL_DBL_CLICK="Relax;"
    ON_CLICK="TreeTrack(2,1);"
    ON_DBL_CLICK="TreeTrack(2,2);"
    ACTION="Find"
      IMAGE="Find"
      MACRO="TreeTrack(2);"
    SUBSTITUTION="\label{*}"
      SUB_FIND="\\label\{*\}"
      SUB_REPLACE=""
      SUB_REG_EX=1
      SUB_CASE_SENSITIVE=1
    SUBSTITUTION="  "
      SUB_FIND=" +{ }"
      SUB_REPLACE=" "
      SUB_REG_EX=1
      SUB_CASE_SENSITIVE=1
    SUBSTITUTION="\ldots"
      SUB_FIND="\ldots"
      SUB_REPLACE="..."
      SUB_REG_EX=0
      SUB_CASE_SENSITIVE=1

9. Hyperlink Navigation within a Presentation

Problem: How can hyperlinks be included which can be clicked in order to navigate between slides. Solution: The beamer user guide discusses this. Matthias Pospiech presents a few examples of using buttons and hyperlinks. Go to the page and search for "hyperlink".

A simple option is to add a label to the target frame. E.g.,

\begin{frame}[label=thetargetlabel]
...

Then add within the linking frame

\hyperlink{thetargetlabel}{\beamergotobutton{Button Text}}

Then press Alt+Left to navigate back to the linking frame if viewing the slide show in Adobe Acrobat.

10.Creating handouts

Problem: Handouts are typically supplied with multiple slides per page. How is this effect achieved in Beamer?

Solution:
A. pdfnup is meant to be well suited to the task of placing n, pdf pages up onto one page.

B. I'm currently using the pgfpages package to do the job. I quite like the results that I get from four per page (i.e., 2 by 2) in landscape mode using the code below.

\mode<handout>{
\usepackage{pgfpages}
\pgfpagesuselayout{4 on 1}[a4paper, border shrink=5mm, landscape]
}

The code

\mode<handout>{...}

requires the documentclass to have the handout option to be specified like this: \documentclass[handout, t]{beamer}

11. Minimising Typing

Problem: The markup for beamer is quite verbose. All the frame, block, columns, and itemize environments and the need to add item to the start of each dot point leads to a lot of non-content related typing.

Solutions
I received some helpful suggestions when I asked the question about minimising typing on tex.stackexchange.com.

A. There are several shorthand notations to make creating beamer presentations simpler.

However, I'm reluctant to adopt these approaches. Using non-standard notation often introduces compatibility issues.

B. I have also set up template text of the most common types of slides. I place these in a separate file, which I can copy and paste into the main presentation file as required. I'm considering using some form of code snippet system. I present this template in a separate post.

12. Preventing Beamer markup from interfering with thinking

Problem: Beamer formatting tags are fairly verbose. For example, you have to remember to open and close environments and for each dot point its necessary to write "\item". Thus, producing a slide involves a fair amount of typing. Similarly, modifying a slide also takes time. When the content of the slide is still being refined, prematurely entering formatting can be a waste of time. In contrast to LaTeX documents, the markup to text ratio in Beamer presentations is particularly high. I found that writing Beamer markup often interfered with the content creation process.

Solution:
A. With practice, the markup becomes more natural. In addition, there are tricks for using slide templates to speed up the process of entering markup.

B. There are also various wiki-like markup languages which aim to make the markup less verbose.

C. Initially I found it useful to use my own loose markup style when writing the draft of the presentation. My process involved entering the details of the slide text on a first draft and then going over the slides in a second iteration to put in the markup. My basic markup looked like this:

\section{This is the first section}
* Slide 1 Title text
- point 1 on slide 1
- point 2 on slide 1

* Slide 2 Title text
% insert someImageFileName

D. In general over time I've found that typing basic beamer commands becomes a lot quicker and less distracting with practice. I've also found that I do a fair amount of initial brainstorming of the structure and content of a presentation in a separate outlining tool such as Outline View in Word.

13. Resolving Beamer Error Messages

Problem: Beamer error messages can be particularly mysterious. What are common error messages? What strategies speed up diagnosis and resolution of error messages?

Solution:
Common errors include:

Forgetting to include the end environment. e.g., \end{block}
Problems with braces }: e.g., forgetting to include a brace; typing the wrong character, such as a parenthesis; problems with nested braces, etc.
Forgetting to add at least one \item in an itemize environment
Forgetting to add a title to a block environment
Typographical mistakes in common command names (e.g., ietm or iteM instead of item
Putting begin where end should be placed and vice versa.
Forgetting to escape (i.e., "\") special characters such as the ampersand, dollar sign, and percentage sign particularly when pasting text from another source

General strategies for debugging:

In some instances, the error message will indicate the text that is related to the error.
The LaTeX compiling information often indicates the number of the slide at which an error occurred. e.g., you will see [1][2][3]...
In other situations progressively move the \end{document} command around the document. The document will only compile up to that point. This should isolate the slide where the error is occurring.
Frequently compile, especially when first learning, to minimise the possible error locations.
Once the probable frame has been identified, look for common errors in the frame as mentioned above.
Use an editor that flags common errors such as non-matching environments or mistyped command names

Simple Beamer Template for Getting Started and Reducing Typing

2010-08-25T21:18:00.001+10:00

For those who don't already know, Beamer is a useful package in LaTeX for preparing slide presentations. I have a Beamer template of preamble and slide templates. I found having a template was useful: (a) when first learning Beamer commands, and (b) in order to save typing. Thus, this post shares and explains the template in case it was of interest to others. It includes both my standard preamble and templates for individual slides. It is designed so that it is easy to compile both a presentation and a 2 x 2 handout.

Overview of Using the Code

At the risk of stating the obvious, I use the template as follows.

When beginning a new presentation three text files are created (substituting the word "content" for the name of the talk): (a) handout_content.tex, (b) presentation_content.tex; (c) content.tex.
- presentation_content.tex contains just the two lines as shown under (1) below.
- handout_content.tex contains just the two lines as shown under (2) below.
- content.tex contains the preamble as shown under (3) below.
The slide templates under (4) are copied and pasted into content.tex as needed to prepare the presentation.
presentation_content.tex is compiled while preparing the talk. handout_content.tex is compiled only once the presentation is finalised.

The Beamer Template Code

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 1. Presentation header file (in separate file)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[t]{beamer}
\input{texfile} % replace texfile with main content file name

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 2. Handout header file (in separate file)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[handout, t]{beamer}
\input{texfile} % replace texfile with main content file name




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 3. Main content preamble and opening slides
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\mode<beamer>{
\usetheme{Frankfurt}
\setbeamertemplate{navigation symbols}{}
}
\mode<handout>{
\usepackage{pgfpages}
\pgfpagesuselayout{4 on 1}[a4paper, border shrink=10mm, landscape]
\usetheme{default}
}

\title{}
\subtitle{}
\author{}
\institute{}
\date{}

\begin{document}
\mode*
\begin{frame}
\titlepage
\end{frame}

\begin{frame}
\frametitle{Outline}
\tableofcontents
\end{frame}


\end{document}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 4. Templates for individual slide types
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Slide at the beginning of a section
\begin{frame}{Outline}
\tableofcontents[currentsection]
\end{frame}


% Simple slide
\begin{frame}{}
\begin{itemize}
\item
\end{itemize}
\end{frame}


% Slide with a block
\begin{frame}{}
\begin{block}{}
\begin{itemize}
\item
\end{itemize}
\end{block}
\end{frame}


% Slide with example (different colour to Block)
\begin{frame}{}
\begin{example}
\begin{itemize}
\item
\end{itemize}
\end{example}
\end{frame}


% Slide with two columns
% (0.5 can be adjusted but should generally sum to 1.0)
\begin{frame}{}
\begin{columns}
\begin{column}{0.5\textwidth}
\end{column}
\begin{column}{0.5\textwidth}
\end{column}
\end{columns}
\end{frame}


% Verbatim
\begin{frame}[fragile]{}
\begin{verbatim}
% code
\end{verbatim}
\end{frame}

% slide with title and figure
\begin{frame}{}
\begin{center}
\includegraphics[width=11cm]{files/figure}
%\caption
\end{center}
\end{frame}


% Full screen image
\mode<all>
{\usebackgroundtemplate{\includegraphics[width=\paperwidth]
{files/figure}} %replace files/figure with file path and name
\begin{frame}[plain]
\end{frame}}
\mode<all>{\usebackgroundtemplate{}}
\mode*


% Basic centred table with alternating colour rows
% \usepackage{xcolor}
% \documentclass[xcolor=pdftex,dvipsnames, table]{beamer}
% adapted from http://www.tug.org/TUGboat/Articles/tb26-1/mertz.pdf
\begin{frame}{}
\begin{center}
\rowcolors{1}{\RoyalBlue!20}{\RoyalBlue!5}
\begin{tabular}{lll}\hline
A & B & C \\
\end{tabular}
\end{center}
\end{frame}


% Frame with incrementally displayed dot points with an alert
\begin{frame}{}
\begin{itemize}[<+-| alert@+>]
\item
\end{itemize}
\end{frame}

WinEdt 6.0 for LaTeX: Features, Configuration, and Resources

2010-04-21T23:06:00.000+10:00

This post discusses my experience with WinEdt 6.0 as a text editor for writing documents in LaTeX. In this post I outline: (a) Why I have chosen to use WinEdt; (b) The role of WinEdt in my workflow; (c) Various customisations of WinEdt which I have found useful; (d) links to additional resources for getting the most out of WinEdt.

Overview

I've recently been examining a number of text editors for writing LaTeX documents in a Windows Environment. These include TeXnicCenter (v1.0), Texlipse for Eclipse, and Notepad++. I read a discussion on Stack Overflow discussing the Best LaTeX editor for Windows. Each editor that I tried had good features. Each was free. However, as I tried to get the configuration I wanted, I frequently felt that I was having to implement a lot of custom configurations. Also, while these programs had many great features, there were always features missing. I did not feel that I would be truly comfortable working with these editors for extended periods of time on complex documents.

As a side point, Emacs/Xemacs and Vi/vim are also meant to be very good for editing LaTeX documents. I think the text editing shortcut keys in these programs are superior to those on Windows. In particular, I would like to never have to let my fingers leave the home keys in order to edit text. I can also see the benefit of having a single text editor that you use for all programming, emailing, and document creation tasks. It's also great to have an open source text editor supported by a huge community of technical users. However, the shortcut keys used in these programs are completely different to those used in other Windows programs. When working with both Windows programs and editors like Emacs, there is a negative transfer which disrupts automaticity. Thus, until I'm willing to adopt Linux, I'm not inclined to learn these editors.

In particular I wanted usability. I wanted to be able to navigate my document comfortably. I wanted the display to reinforce the document structure. I wanted the program to support my workflow organising my task related comments. I wanted to be able to easily compile and debug my LaTeX source in a comfortable way. I wanted intelligent defaults with the option to customise if a default annoyed me. I wanted the program to have thought about usability issues that I hadn't thought about.

I'm currently using WinEdt 6.0. It looks like it will be the editor for me. It has been designed with someone like me in the mind: A researcher who wants to write technical documents. It's not free, but it's not expensive.

Features that I like

In particular there were several Features in WinEdt that I liked at first glance:

Tree View can be customised
Automatically display your current location in TOC
Colour coding that aids usability
Intelligent defaults
Options to customise almost anything
Easy configuration interface with MikTeX
One click build process for LaTeX documents
Intuitive default shortcut keys and intuitive alt menu letters
Ctrl+Up and Ctrl+Down navigates between paragraphs.

The remainder of this post discusses: configuration, features, and set up ideas.

Configuration

The following sets out a few alterations to the default WinEdt 6.0 configuration that I have found useful.

Configuring Inverse Search with Yap

WinEdt includes documentation on how to configure Inverse search in Yap.

Configuring with JabRef

In JabRef, I changed the following:
Preferences - External Programs - WinEdt: I had to change the default file location.
Once this was set, Ctrl+L sends the selected citation to WinEdt and returns focus to WinEdt.

Changing Settings

The method of changing settings in WinEdt 6.0 differs from most Windows programs. The configuration is based on a large number of *.ini files. Ini files are arranged categorically.
To make changes:
(1) edit the ini file,
(2) save the ini file, and
(3) then click the "load current script" button.

The options are reasonably self-documenting. In particular:
0 = No or Off or False
1 = Yes or On or True

You can backup and restore your settings. This is particularly useful if you corrupt your ini files.

Spelling

I needed to customise Australian spelling. This meant enabling a number of dictionaries (e.g., "colour", "ise", "yse", "centre") and disabling the corresponding American spellings.

Dual Monitor Issues

Some dialog boxes default to the top-left of the primary monitor (e.g., Forward Search, spelling). I have a dual monitor setup where the main monitor I work on is the secondary monitor. My Start Menu is displayed vertically from top-left to bottom-left of my primary monitor. When several dialog boxes display, they are obscured by the Start Menu. I still have not resolved this issue for several dialog boxes. However, for some dialog boxes, settings like the following work:

Incremental Search:
LOCKED_POSITION=0

Word Wrapping

Word wrapping is an important topic. It's also a matter of personal preference. WinEdt provides many options for word wrapping. Wrapping can be turned off and on each document by clicking on the status bar. I'm still thinking about the best configuration for this.

A freedom of writing in HTML and LaTeX is that you can arrange you have greater freedom in how text is arranged on the page. The following command does a reasonable job of creating new lines at the end of each sentence. Find: "[.] " Replace: ".<" Regular Expressions enabled

Disable Sound Effects

I work in an office. Thus, dings and dongs are unwanted.

Sound Effects:
DISABLE_SOUND_EFFECTS=1

Auto Saving

My main aim was to customise the setting to ensure frequent auto-saving (e.g., at least every 5 minutes). I like my backup files in a separate folder as opposed to cluttering the primary folder.

backup.ini BACKUP_FILENAME="c:\temp\%n%t.bak"

AutoSave.ini AUTO_SAVE_FILENAME="c:\temp\%n%t.sav"

Error Processing in LaTeX

When in draft stages of preparing large document, I find warnings related to "overfull and underfull boxes" to be distracting. Thus, I disabled their bookmarked Inclusion in the Error log. They are still shown in the log, but clicking "Next" or "Previous" does not go to them.

TYPE="Type II:  ?...<From>--<To>"
  WARNING=0
  ENABLED=0
  FILTER="TeX"
    TRIGGER="Overfull \"
    TRIGGER="Underfull \"

Fonts and such

I modified the display of HTML (Font Schemes - Fonts.ini). I prefer a fixed-width font for writing HTML, such as "Courier New". I also modified the TAB_LENGTH to 4 instead of 8.

FONT_SCHEME="HTML Documents"
  MODE="HTML|HHC;XWE"
  USE_IN_DIALOGS=1 // Use in Spelling or Complete Dialogs...
  FONT_NAME="Courier New"
  FONT_SIZE=10
  FONT_BOLD=0
  FONT_ITALIC=0
  FONT_CHARSET=1
  EXTRA_LINE_SPACING=1
  EXTRA_CHAR_SPACING=0
  TAB_LENGTH=4
  TAB_OPT_LEADING=0
  TAB_OPT_NON_LEADING=0
  FREE_CARET=0

Display

As far as I can tell, all the standard toolbar buttons are available through menus. And the combination of the Alt Key and appropriate keyboard accelerators is quicker than using the mouse. Thus, I right clicked around the toolbars to hide toolbars. This has the benefit of creating additional space to view text.

Getting Things Done

Getting a sense of Context

The importance of context When working with large documents it is important to have a sense of context. I want to know where I am in the hierarchical structure of a document. In projects with multiple files, I want to know which file I am in.

The following two commands are great for providing context:
Project - Auto-Select TOC Branch
Project - Auto-Select Tree Files

Navigation within Document

Basic Cursor Movement

Between Paragraphs
Ctrl+Up and Ctrl+Down: Move cursor between paragraphs (a nice feature for a coding editor).

Standard Windows Editing Keys

Most standard text editing keys work like you would expect, if you have grown up in a Windows world. e.g., Ctrl+Delete; Ctrl+Backspace; PageUp; PageDown; Ctrl+Left; Ctrl+Right

Tree View

Tree View provides: Files, TOC, Bibliography, and Labels The mouse can be used to select objects (note my frustration at the absence of keyboard navigation). There's also a slight problem with keeping the tree current (particularly the TOC). It can be manually refreshed by:
(a) clicking the "Build Tree" button; or
(b) opening the main file of the project, which in a single file project should already be opened, and selecting Project - Set Main File (Alt, p, e)

Navigating Between Files

Open File

Ctrl+O Brings Up an Open File Box. It tends to be in the folder of the main project file. The file name of the tex file can then be entered.

Next and Previous Window

Ctrl+Comma and Ctrl+Shift+Comma move to the next or previous open document based on the order of the tabs (tab-order). This is in contrast to MDI-order which is based on the order that the windows have bene last accessed. When opening a document, the new tab is added after the tab for the current open document. Tabs can be rearranged using drag and drop. By opening a set of documents for a project in a particular order (e.g., main, preamble, chapter1, chapter2, chapter3, etc.) the tab-order shortcut keys can be used to navigate through the files that make up the sections of your document. The tree view also provides a tool to open and close all files in a project or at a given heading level.

Open Input and include documents

Double click on an input or include command to go to the file referenced by the command. Double clicking hyperlinks in HTML files open the link in the default web browser.

Global Mark and Global Return

Ctrl+F10 sets a mark in an open document. Ctrl+Shift+F10 returns the cursor to this mark. This works even if you are working in a document other than the marked document. Thus, it is useful when editing multiple documents.

Bookmarks

The "Search - Bookmarks" sub-menu lists a number of options for working with bookmarks. They appear to be only effective within a file. If you want to use bookmarks to navigate between files, you need to use the Global Mark option. Up to 9 separate bookmarks can be set using Ctrl+Shift+1, Ctrl+Shift+2, etc to set the bookmark, and Ctrl+1, Ctrl+2, etc to go to the bookmark.

When should you use bookmarks? If you are writing at line X, but need to check something in line Y:
(1) bookmark line X, Ctrl+Shift+1
(2) go to line Y,
(3) bookmark line Y Ctrl+Shift+2
(4) check whatever you wanted to check in line Y
(5) return to line X, Ctrl+1
If you then need to check line Y again, Ctrl+2 Macros for navigation I would like to be able to navigate the outline in the Tree View using keyboard short cuts. It seems that the macro language might be the best course of action. Tree and Gather Macro Interfaces are provided. Scroll Lock Navigation In "editor.ini", the following setting turns the cursor keys into a tool to navigate when Scroll Lock is activated. ENABLE_SCROLL_LOCK_KEY=1

Customised Tree View

Paragraph descriptions in Outline

When I write LaTeX documents, I often write what I call paragraph descriptions above each paragraph. Paragraph descriptions are typically less than the length of a line and state summarise the contents of the paragraph. They are useful for highlighting and checking the structural flow of a document. I incorporate them in latex using a custom command that does not print in the PDF. The syntax that I use for the command is \pd{Text describing title} A particular appealing feature of WinEdt is that these paragraph descriptions can be added to the Table of Contents Outline. To enable this I Added the following option. It is a copy of the "\paragraph" item with a couple of changes (e.g., level is deeper)

  ITEM="\pd{?}"
    MODE="TeX"
    CASE_SENSITIVE=1
    BEGINNING_OF_LINE_ONLY=1
    CURRENT_DOCUMENT_ONLY=1
    ALL_OPENED_DOCUMENTS=0
    COMPLETE_PROJECT_TREE=1
    ICON="Paragraph"
    LEVEL=26
    CAPTION="%?"
    BALANCED="{}\"
    MAX_LINE_SPAN=3
    ON_CTRL_CLICK="GlobalMark;TreeTrack(2);"
    ON_CTRL_DBL_CLICK="Relax;"
    ON_CLICK="TreeTrack(2,1);"
    ON_DBL_CLICK="TreeTrack(2,2);"
    ACTION="Find"
      IMAGE="Find"
      MACRO="TreeTrack(2);"
    SUBSTITUTION="\label{*}"
      SUB_FIND="\\label\{*\}"
      SUB_REPLACE=""
      SUB_REG_EX=1
      SUB_CASE_SENSITIVE=1
    SUBSTITUTION="  "
      SUB_FIND=" +{ }"
      SUB_REPLACE=" "
      SUB_REG_EX=1
      SUB_CASE_SENSITIVE=1
    SUBSTITUTION="\ldots"
      SUB_FIND="\ldots"
      SUB_REPLACE="..."
      SUB_REG_EX=0
      SUB_CASE_SENSITIVE=1

There are many further possibilities for customisation. includegraphics: I also added a level value to the \includegraphics command in the tree.

Spelling

I disabled:
Confirm Deletion
Confirm Addition
Confirm Capital Addition

When first starting with a new text editor, there are lots of words that I use that are not spelling mistakes. Thus, the following steps can be applied to a word incorrectly classified as a spelling error:
Ctrl+Enter, Alt+A, or
Right Mouse Click, Alt+A

Customising Gather

In my workflow, I have a number of custom commands where it would be useful if these could be incorporated into the gather functionality. \todo{...} I use the todo command from the todo package to list tasks that in the document. Navigating this list is useful way of working through a todo list.

PAGE="todo"
  // Change Mode from TeX to ? to disable this item
  PAGE_MODE="TeX|DTX;INS;STY"
  PAGE_SORTED=0
  PAGE_CASE_SENSITIVE=1
  PAGE_IGNORE_COMMENTS=1
  PAGE_BEFORE_BUILD="Relax;"
  PAGE_AFTER_BUILD="Relax;"
  PAGE_ON_ACTIVATE="GlobalMark;"
  PAGE_ON_SHOW="GlobalMark;"
  PAGE_ON_ENTER="GDIKillFocus;"
  PAGE_ON_CANCEL="GlobalReturn;GDIKillFocus;"

  ITEM="\todo{?}"
    MODE="TeX"
    CASE_SENSITIVE=1
    BEGINNING_OF_LINE_ONLY=0
    CURRENT_DOCUMENT_ONLY=1
    ALL_OPENED_DOCUMENTS=0
    COMPLETE_PROJECT_TREE=1
    RETURN_AT_EOF=0
    ICON="ArrowOrange"
    LEVEL=0
    CAPTION="%!?"
    MAX_LINE_SPAN=2
    ON_CLICK="GDITrack;"
    ON_DBL_CLICK="GDIKillFocus;"
    ACTION="Find"
      IMAGE="Find"
      MACRO="GDITrack;"

Word Counts

Document - Word Count (Alt, D, W) can be used to estimate the number of words in a *.tex file. If you select text

Excluding text from word count If you have custom commands that you use often that do not print text, you may wish to exclude them from word counts. For example, I use the \todo command to store tasks that need to be done. I use my own \pd command to store a non-printing paragraph descriptions. I use my own \OUTPUT command to indicate where I need to insert a specific table, figure, or results text. These commands and their contents can be excluded from word counts by editing the WordCount.ini file (Options - Options - Formatting - Word Count). The built-in examples make it pretty clear what needs to be done to add additional tags.

WORD_COUNT_SET="TeX"
   ...
  SKIP="\pd{?}"
  SKIP="\todo{?}"
  SKIP="\OUTPUT{?}"

Other word count options Note that the word count option does not work for multi-file projects. Other options exist for word count calculation:

The main challenge with word counts in both MS Word, in LaTeX, and in PDFs is that many word count limits imposed in academic writing exclude various forms of content. For example, the word count for a thesis might exclude: references, tables, figures, appendices, headers, footers.

LaTeX Build Profiles

Configuring various LaTeX build options is really easy with WinEdt. Under Options - Execution Modes, I have the following options selected:
Process Flow: Wait for execution to finish; Start Viewer; Forward search; No interaction [Apply to All]
I generally use TeXify to automatically determine how many times the document needs to be built.

Using Template Text

WinEdt can facilitate the insertion of template text. For example, there is a template for inserting a figure. The template use a character that looks like "*" for information that needs to be replaced by the user. Next Bullet (Ctrl+Space) and Previous Bullet (Ctrl+Shift+Space) can be used to navigate between bullets.

You can even create your own templates using Ctrl+Alt+Space and whatever surrounding template text is required. You may want to create a set of common templates and store it in a separate file.

Problems with WinEdt 6.0

The following represents frustrations that I have experienced with WinEdt 6.0. WinEdt has many customisation options. Thus, in some cases, perceived problems may represent my lack of knowledge about how to configure WinEdt to operate in the way that I wish.

No keyboard access to certain features

There are some commands which do not appear to be accessible to the keyboard, including: TOC View, File View, Gather View, Toolbar Buttons, Find in Files, LaTeX Error View. I find this very frustrating. These are fundamental navigational tools that are frequently used. They should not require the mouse. There may be workarounds using macros.

The placement of some dialog boxes

Some dialog boxes are not well-placed if you use a dual monitor configuration and work principally on your second monitor. Hopefully this will be corrected in future versions.

Resources

Using Macros

WinEdt provides good support for macros. Here is an Introductory guide to marcos Programs that interest me:

www.winedt.org

There are a great set of resources on http://www.winedt.org/. These are a few things that have interested me that I haven't yet tried.

Macros

Plug-ins

TeXtab

Add-ons

Additional Resources

Additional Configuration ideas for LaTeX and WinEdt

Exporting PDFs from JabRef using a Batch File

2010-04-12T17:33:00.001+10:00

If you do research, JabRef is a great tool for managing a personal repository of PDFs. JabRef allows you to link your citations with full text PDFs. This post discusses one way of exporting a subset of your PDF repository using JabRef, Excel, and a Windows Batch file. This is useful (copyright permitting) in many situations.
For example:
(a) You need to extract a subset of references for a subject you are teaching.
(b) You need to give a colleague or research student a subset of your references.
(c) You want to transfer a subset of your references to another device or computer.

Option 1: Open and Save As

When there are fewer than five PDFs, it is quicker just to open the PDFs individually and save them in the desired location.

Option 2: Batch File Export

When there are more than five PDFs, it may be quicker to adopt a procedure like the following. The procedure involves creating a batch file to copy specified PDFs to a designated export folder.
The procedure is as follows:

1. In JabRef, Select PDFs of References that you want to export (see Figure 1).

2. In JabRef, Copy Selection (Edit - Copy)

3. In Excel, Paste into first column. Note: You could use a different Spreadsheet application, such as Open Office Calc.

4. In Excel, Sort the column alphabetically

5. In Excel, Highlight cells beginning with the word "file". Note: the "file" field is where JabRef stores the relative path for the PDF.

6. In Excel, Text to columns delimited by ":" Note: JabRef separates the data into file name, path name, and file type separated by colons. This step extracts the path name into it's own cell. If your repository does not include subfolders, file name will be equivalent to path name.

7. In Excel, Create the string to copy the batch file (see Figure 2).
7.a create the string "copy filePath destinationFolder" e.g., ="copy "&B24&" c:\temp", where B24 is the cell with the path name
7.b copy and paste this formula for the remainder of the cells.

8. In text editor, Create the batch file (see Figure 3).
8.a Open Notepad or another text editor
8.b the first line changes the directory to the root folder of your PDF repository. e.g., cd /d D:\My Documents\2Repository\filed
8.c Copy and paste the copy commands from Step 7 below this line.
8.d Save as a .bat file (e.g., copyPDFs.bat)

9. In Windows Explorer, Run the batch file. i.e., just double click the batch file.

Figure 1. Screen Shot of JabRef showing references with connected PDFs

Figure 2. Screen Shot of Excel showing Formula for creating copy command

Figure 3. Screen Shot of Notepad showing example of complete batch file

TeXnicCenter Customisations

2010-03-19T22:16:00.000+11:00

Over the last few weeks I've been exploring text editors for writing LaTeX documents. I wrote these notes when I was using TeXnicCenter (version 1.0). I have since switched to using WinEdt for writing LaTeX documents. If you do any substantial writing in LaTeX and you are choosing between free TeXnicCenter and shareware WinEdt, buy WinEdt. I'm planning a post soon on WinEdt. Nonetheless, I thought I'd post these rather rough notes I made while playing around with TeXnicCenter in case they are of interest.

Shortcut Keys

I added the following keyboard customisations to the default set. The keys were chosen with the aim of (1) being consistent with other programs, and (2) not conflicting with existing shortcut keys.

Find Next: Ctrl+PageDown
Find Previous: Ctrl+PageDown
Navigation Items - Go to: Ctrl+Shift+D
Window - Close: Ctrl+Shift+W
Find in Files: Ctrl+Shift+F
Select Output Profile: F12
Tools - Windows Explorer: Ctrl+Shift+E

Document navigation

The following are some strategies that can be used to facilitate document navigation in TeXnicCenter.

Structure, Objects, Files
Press one of the following keys, then navigate to the target, and press Ctrl+Shift+D (requires prior shortcut).

Activate Structure: Ctrl+G
Activate Objects: Ctrl+U
Activate Files: Ctrl+D

Incremental Search
1. Ctrl+I
2. Get to target by typing identifying letters or combine semi-identifying letters with additional Ctrl+I or Ctrl+Shift+I
3. Enter
4. To remove selection: Movement key or Ctrl+I Enter

Variant on Incremental Search
1. Highlight Text
2. Ctrl+F Esc
3. Find Next or Find Previous (F3 and Shift+F3; or Ctrl+PageUp and Ctrl+PageDown with my modified short cut keys)

Getting an overview

Structure View

TeXnicCenter has Structure View. It shows heading titles down to the subsubsection level. It shows tables and figures in floats and separated equations. Ctrl+G activates Structure View. Double clicking or using a shortcut can take you to the corresponding location in tex source. Left, Right, Up, and Down serve navigational purposes

One frustrating element is that pressing Ctrl+G when in tex source, does not take you to the corresponding location in the Structure View. Thus, if you are trying to get a sense of where you are in the structure, you need to navigate to the location.

A second frustration is that you can't control how the Structure View is displayed. It is not possible to expand all headings. It is not possible to show a given depth. It is not possible to add custom tags.

Note: I've since discovered that WinEdt supports these features.

Spelling

To download a dictionary in a language other than US English (i.e., Australian English)

Download a dictionary from from: Open Office
Copy unzipped dictionary files into the "Language" folder of your TeXnicCenter Installation. e.g., "C:\Program Files\TeXnicCenter\Language".

I also set up a custom dictionary in a suitable location.

Working on two files

It is sometimes useful to be able to work on two *.tex files at the same time. For example, I may be in Chapter 2 citing a reference in Appendix C. Thus, I'll have both chapter2.tex and appendix3.tex open. I achieve this effect by:

1. While in one of the files (e.g., chapter2.tex): Window - Close all but Active
2. Window - Open the other desired File (e.g., appendix3.tex)
3. Window - Tile Vertically

Window Layout

On a 24 inch monitor I have the left two-thirds of the monitor with TeXnicCenter and the right one-third of the monitor with Yap.

Work Flow:

When learning to write in LaTeX, it is important to make repetitive tasks efficient.

Edit-Build-Check-Edit Cycle:
1. Build Output: Ctrl+F5 in TeXnicCenter:
2. Show LaTeX source in TeXnicCenter using Inverse Search: E-Key in Yap or Double click on document;
It can be useful to show source links so that it is clear where the cursor will go when you activate inverse search. (in Yap: Options - User Interface - Show source links). Double clicking on one of the circles overlaid by Yap tends to return the cursor to a specific expected location.3.

Return Focus to TeXnicCenter
Several options exist:
(a) Close Yap (q)
(b) Use mouse to select TeXnicCenter
(c) Alt+Tab
(d) Use an AutoHotKey Script: I set up a script using AutoHotKey that toggles between TeXnicCenter and Yap.
; Script toggles focus between Yap and TeXnicCenter
;Script assumes both are open
;and that only one copy is open
#1:: ;Windows+1 is a shortcut key
if WinActive, TeXnicCenter
WinActivate, Yap
else
WinActivate, TeXnicCenter

APA Style References in LaTeX

2010-03-15T17:29:00.000+11:00

This post discusses my experience getting APA style references in LaTeX. This includes both in-text citations and the end of document references list. It focuses on the use of the apacite package.

Setting up a BibTeX Bibliographic Database

Regardless of what citation style you want, you need to have your references formatted in BibTeX format. JabRef is a powerful reference manager for working with a BibTeX database. Also, see my previous post for migrating an Endnote database into BibTeX format and migrating a Word Document with Endnote references into LaTeX format.

APA Style References Several options exist:

apalike style in natbib
apacite package: See the PDF documentation for details.

From my initial examination apacite looks like it has more features and is more compatible with the nuances of APA style. However, natbib is somewhat of a standard in LaTeX. Thus, using natbib citation commands may enable easier conversion between citation styles. The remainder of this post focuses on my experience with the apacite package.

Using apacite

For comprehensive information on using apacite, see the documentation. The following sets out some basic details:

1. Ensure that you have a BibTeX database for the document. Your document will be more portable if the database is placed in either the same folder as the LaTeX files or in a subfolder of the LaTeX files.

However, you may prefer to maintain a general Bibliographic database that can be used across documents. If you wish to have a general database, you can use absolute file references to the database. Then when you need to share the BibTeX database, copy the database into a LaTeX project folder or subfolder and change the file reference in the bilbiography command .

2. Reference the apacite package in the preamble.

\usepackage{apacite}

3. Place the bibliography in the desired location in the document (usually after the main text) using the following code:

 \bibliographystyle{apacite} 
 %http://www.ctan.org/tex-archive/biblio/bibtex/contrib/apacite/apacite.pdf 
 \bibliography{bibtex/thesis} % database is "thesis.bib" located in a "bibtex" subfolder

4. To insert citations in the text apacite has a range of commands (see "The citation commands" in the Manual). Table 1 in the manual lists some of the main commands. I find myself using the following ones most often:

cite: This inserts the author names and years in brackets. This is the standard methods. E.g., "(Anglim, 2002)".
citeA: Inserts the author names in the text and years in brackets. I use this to insert the authors names in the text e.g., "Anglim (2002)".
citeyear: inserts just the years in brackets. This is useful when I want to include the author's names in the text in a non-standard way. E.g., I've already written "Anglim's" then citeyear will insert "(2002)".

Each command takes a BibTeX reference as an argument and can have pre- and post- text. For example:

cite<e.g.,>{Ackerman1990JoEPLMaC}

displays as "(e.g., Ackerman, 1990)"

5. To produce the document with references included, the document typically needs to be built three times in order for all updates to occur.

Additional Challenges and Resolutions

French formatting
Problem: For some reason my document was using a French language version of apacite. This resulted in many unwanted customisations, such as the French word for references (Références) being displayed at the start of the reference list.
Solution: I inserted the following code into the preamble to restore the English mode of apacite:

\usepackage[english]{babel}

Additional Resources

William Revelle's notes on configuring LaTeX for APA style.
Ista Zahn's document Learning To Sweave in APA style.

Converting a Microsoft Word Document into a LaTeX Document

2010-03-15T17:01:00.001+11:00

This post discusses my experience converting a large MS Word document into a LaTeX document using Word-to-LaTeX. Along the way I encountered several challenges. I thought I'd document them in case it may be of interest to others.

Overview of Options

Having a good conversion method is important when transitioning existing Word documents to LaTeX and when colloborating with others who are not familiar with LaTeX. Wilfred Hennings provides a great page that summarises options for converting documents from PC Wordprocessors to LaTeX. The page list options and provides recommendations. Wilfred's Quick Comparison List is particularly useful.

My Experience with Word-To-LaTeX:

I started my conversion journey with Michal Kerbt's Word-to-LaTeX (Word-to-XML) Convertor. It's free software but Michal accepts donations. I used it in it's stand alone form. It provides many options for converting documents. While it generally worked well, the following discusses various challenges in the conversion process with a description of what I did in response.

Convert *.docx to *.doc format:
Problem: The convertor did not appear to work with Word 2007 files (*.docx).
Solution: Save As Word 97 (*.doc) format.

Set up a PostScript printer:
Problem: In order to convert graphics files a postscript printer needs to be setup.
Solution: (Warning: I've had some problems with the EPS files generated; I'm not sure if its related to this printer setup)

Install a postcript printer: Adobe sets out one way to set up a virtual printer. I followed these instructions. Here is a direct link to the downloads. In short it involves installing a PostScript printer driver with a PPD specification.
Specify in Word-to-LaTeX: This printer is then specified in the Word-to-LaTeX configuration: Figures/Eq/Documents - Figures - PostScript printer.

Big complex documents:
Problem: Long and complex documents can take a while to run (e.g., 15 minutes for a 60,000 word document with many styles and tables on a 2007 laptop).
Solution: Hey. Who cares! Just let it run. It's quicker than trying to do it manually.

Security Alert Over Macros:
Problem: The program installs a macro in the Word Startup folder. My version of Word (Word 2007) disabled this by default.
Solution: It is possible to enable all macros. However, this is not particularly safe. I decided to delete the file from: "C:\Program Files\Microsoft Office\Office12\STARTUP" and just run the program through its stand-alone desktop interface.

Tidying Up
Problem: The *.tex document was not exactly what I wanted.
Solution: Several options presented themselves.

Change input: I could alter the Word document. I could remove styles, remove unwanted fonts, and so on.
Change process: Word-to-LaTeX presents many configuration options which I could play with.
Post-process: I could apply various replacement operations on the *.tex created by Word-to-LaTeX.

The solution I adopted combined all three approaches. For example, I converted hidden text in the Word document to a particular style. This meant that it was enclosed in a command in *.tex that was easy to find and replace in post-processing.

Start Up Problem
Problem: When I ran Word-to-LaTeX, I obtained the following error:

Conversion started.
Fatal error: Call was rejected by callee.
   at Word.DocumentClass.Activate()
   at WordToLatex.WLConvertor.Convert()
   at WordToLatex.Bin.WLApplication.Main(String[] args)

Solution I closed Word-to-LaTeX. I closed Word. I then pressed control+alt+delete and ended any WINWORD processes that were running. I then restarted Word-to-LaTeX. As an additional point it was sometimes necessary to close Word-to-LaTeX

Conversion Problem
Problem: I obtained the following error.

Converting document fields.
Unknown error: Object reference not set to an instance of an object.
   at WordToLatex.WLProcessFields.FieldHyperlink(Field field)
   at WordToLatex.WLProcessFields.ProcessField(Field field)
   at WordToLatex.WLProcessFields.ProcessAllFields()
   at WordToLatex.WLConvertor.ConvertInner()
   at WordToLatex.WLConvertor.Convert()

Solution:

Divide and conquer: Dividing a long document into smaller parts to identify which parts could be processed was one strategy. If you do this, it may be good to put the files in separate folders, otherwise image files from one subdocument may be overridden by a latter subdocument.
Paste into Fresh Document: Another trick that worked for me was to copy and paste the contents of the document into a new document. I'm not sure why this worked. Perhaps it worked because it removed a number of custom styles I had.

Problems importing EPS files

Problem: I let Word-to-LaTeX convert the images to EPS. I could view these images in an editor and they had indeed been converted. However, when I added them in LaTeX, only white space was shown.

Solution: For pictures derived from R I just created them again, this time using the postscript driver.

Opening the image in Adobe Acrobat Professional and saving as EPS was one option for the other pictures.

Export from Endnote to BibTeX, JabRef, and LaTeX

2010-03-15T16:47:00.002+11:00

This post sets out a procedure that I used to migrate a large set of Endnote references in a Word Document to a Latex document with BibTeX references in JabRef. In particular, it sets out (1) how to export an Endnote database into BibTeX format ready for inclusion in a LaTeX document; and (2) how to modify an existing document with Endnote citations into a document that cites the BibTeX database.

Exporting Endnote database into BibTeX format

Josh explains how to do this using JabRef. However, if you want to alter a document with a large number of existing Endnote citations, you may want to adopt a modified procedure. Below I set out the procedure that I adopted. However, I don't claim it to be the most elegant procedure. Nonetheless, it may give you some ideas.

Changing Endnote Citations to BibTeX format

If you already have a document with Endnote citations, you are going to need to convert these citations to BibTeX format. If the document only includes a few citations, this conversion can be done manually with little effort. However, if you have a book or thesis with several hundred citations, you may want a quicker and more reliable method. The remainder of this post sets out a procedure that I adopted to transition my own Word document with around 500 unformatted Endnote citations to BibTeX format.

This post assumes that:

you have exported your document from a word processor into some preliminary LaTeX format.
you have unformatted Endnote citations in the document
you have JabRef installed
you have some understanding of how Endnote, LaTeX, JabTeX, and BibTeX work.

Background

An unformatted Endnote citation typically looks like this:

Single Author and Year: {Author, Year #Record Number} : e.g., {Anglim, 2008, #231}
Multiple Author and Year: {Author, Year #Record Number; Author, Year #Record Number} : e.g., {Anglim, 2008, #231; Smith, 2009, #100}
Just Year: {, Year #Record Number} : e.g., {, 2008, #231}
Author and Year with preceding text: {introductory text \Author, Year #Record Number} : e.g., {see \Anglim, 2008, #231}
Author and Year with following text: {Author, Year #Record Number, following text} : e.g., {Anglim, 2008, #231, p.73}

The key in the Endnote database is called a record number. A BibTeX key is a text string which it is typically some automatically generated function of the record such as AuthorYear (e.g., "Anglim2008"). The first step to any updating procedure is to ensure that there is database that matches the Endnote key with the BibTeX key.

Importing Endnote Record Numbers into JabRef

Step 1.

Read Josh's protocol but don't follow it just yet. The problem with this procedure is that it does not export the Record Number.

Step 2.

Instead of following Josh's protocol exactly, the following modifies the protocol to include exporting of the Endnote Record Number. Create a copy of "BibTeX Export to JabRef.ens" and name it "BibTeX Export to JabRef RecordNumbers.ens"

Step 3.

Modify the output style:

In Endnote go into edit this style "BibTeX Export to JabRef RecordNumbers.ens".
Go to bibliography templates.
For each Reference Type (e.g., Journal, Book, etc.) add an extra line with the following text:`endnotekey = {`Record Number`},`

Paste the above text just after the "URL" line is one option. E.g., it might look something like this in the area around the pasting:

   `Abstract = {`Abstract`},`
   `Note = {`Notes`},`
   `URL = {`URL`},`
   `endnotekey = {`Record Number`},`
   `EndNoteRefType = {Journal Article},`

Step 4.

Return to Josh's protocol but use "BibTeX Export to JabRef RecordNumbers.ens" to export from Endnote. I.e.,

Highlight all rows of your Endnote Database and Export the references
Use "BibTeX Export to JabRef RecordNumbers.ens" and File Export
after exporting change the file extension of the exported file to ".bib".
open the bib file in JabRef

Step 5.

After opening the BibTeX file in JabRef, add "endnotekey" as a field:

Menu: Options - Set General Fields
Add the following to the end of the line that starts with "General: " ";endnotekey"

Step 6.

To enable you to see the Endnote Record Number in the main viewing table in JabRef:

Options - Preferences Menu
Entry table columns
Add (Plus)
add the field "endnotekey"

Step 7.

Auto-generate a BibTeX key for all the imported references. In a previous post I discuss some of the issues regarding choosing a BibTeX auto-generation key.

Manually Update Endnote Citations

At this point each unformatted endnote reference could be manually updated so that it refers to the appropriate BibTeX key. E.g., If you had an unformatted Endnote citation with Endnote Record Number like: {Bryan, 1897 #1366}, you could do the following:

Search for this record number in JabRef

Control + F (to open Search in JabRef)
"endnotekey=1366" (to get the specific article) OR simply "1366" (this may not always be unique but saves a little typing)

Copy the BibTexKey: Control + K would copy to the clipboard: \cite{Bryan1897}.
Paste over the unformatted Endnote citation in your LaTeX document.

Automatically Update Endnote Citations to BibTeX format

A more advanced alternatively involves automatically replacing [endnotekey] with [bibtexkey] throughout the LaTeX document. There are many ways that this could be done. The following is a fairly inelegant method using a combination of Excel and Word (see the bottom of this post for alternatives). The post assumes that the hash symbol (#) followed by a number only occur in relation to Endnote citations.

1. Create a copy of your ".bib" file

call it "copy.bib"

2. Copy the "endnotekey" field into the "note" field.

This is required because the "endnotekey" field is not included in a standard export.

Open "copy.bib"
Tools - Set/Clear/Rename Fields
Field name: "endnotekey" and Rename field to "note"
OK

3. Export the JabRef Database to a Spreadsheet readable format.

File - Export
File of type: "OpenOffice CSV (*.csv); let's call it "bib.csv"

4. Open "bib.csv" in Excel

The following steps would presumably work in most other spreadsheet programs like Open Office, perhaps with some modification to the function.

5. Create in Excel the code required for the VBA Word Procedure

The general format of the procedure is:

ReplaceEndnoteRef EndnoteKey, BibtexKey

where ReplaceEndnoteRef is the name of the procedure, EndnoteKey is the number of the endnote key, and BibtexKey is the BibtexKey. For example, this could be:

ReplaceEndnoteRef 724, "Smith2008"

To create this in your spreadsheet (with bib.csv) enter the following function in an extra column (it assumes that the "note" field is in column M and the bibtex identifier is in column C):

="ReplaceEndnoteRef "&M2&", "&CHAR(34)&C2&CHAR(34)

ReplaceEndnoteRef 1294, "Abraham1992AoDPS"

Copy and paste the function so that it applies to all rows.
Sort all rows by the "note" field in descending order. The code needs to be run in this order to avoid inappropriate replacements.
Paste a copy of the LaTeX text into a New Word Document. WARNING: do not run this macro on your document without a backup.
Apply a macro in Word

Paste your latex text into a new word document

Open up the Visual Basic Editor and paste the function below (insert a new module if needed)

Sub ReplaceEndnoteRef(EndnoteKey, BibtexKey)
    EndnoteKey = "#" & EndnoteKey
    Selection.Find.ClearFormatting
    Selection.Find.Replacement.ClearFormatting
    With Selection.Find
        .Text = EndnoteKey
        .Replacement.Text = BibtexKey
        .Forward = True
        .Wrap = wdFindContinue
        .Format = False
        .MatchCase = False
        .MatchWholeWord = False
        .MatchWildcards = False
        .MatchSoundsLike = False
        .MatchAllWordForms = False
    End With
    Selection.Find.Execute Replace:=wdReplaceAll
End Sub

Create a new procedure and paste the commands from Excel. Here are the first few lines of such a procedure

Sub ReplaceRefs()
ReplaceEndnoteRef 1495, "Sternberg1979AP"
ReplaceEndnoteRef 1494, "Thoresen2004JoAP"
ReplaceEndnoteRef 1493, "Lievens2002MEa"
ReplaceEndnoteRef 1492, "Lievens2002ME"
ReplaceEndnoteRef 1491, "Goldberg2006JoRiP"
ReplaceEndnoteRef 1490, "Carroll1865"
ReplaceEndnoteRef 1489, "Walsh2009CP"
ReplaceEndnoteRef 1488, "Zimbardo1973NYTM"
ReplaceEndnoteRef 1487, "Anglim2000"
ReplaceEndnoteRef 1486, "Fry"
ReplaceEndnoteRef 1485, "Anglim2007"
ReplaceEndnoteRef 1484, "Zou2007PM"
' And many more references
end sub

run the ReplaceRefs() procedure on your document. F5 is a shortcut key when the cursor is in the procedure. This should automatically replace endnotekeys with bibtexkeys.

This still leaves a fair amount of text in the citation to tidy up, but it is at least a reasonable start. In addition the remaining text is likely to be relevant to how the text should appear (i.e., whether there is preceding text, or following text, and whether the authors names should appear).

If you continue to use this version of the database you may want to clear the "note" field so that the endnote key is no longer present. Otherwise it may appear in references. This can be done using the Tools - Set/Clear/Rename Fields menu option in JabRef.

ALTERNATIVE OPTIONS

After devising the above procedure, I discovered Noah Hoffman's Approach involving Python Scripts. It might also be worth a look.

Choosing an Auto Generation Pattern for BibTeX Keys in JabRef

2010-03-08T22:08:00.005+11:00

This post discusses the issue of choosing a default pattern for the BibTex key generator in JabRef.

THE CONTEXT

If you haven't already heard, JabRef is an open source reference manager built on Java, and BibTeX is a file format for storing references. I've recently been making the transition from Word to LaTeX and from Endnote to BibTeX and JabRef (although you can use JabRef with MS Word).

Once I had imported my references from Endnote into JabRef, I needed to generate Bibtexkeys. These keys could then be used to link references in the BibTeX database to citations in a document. Jabref facilitates the task using an auto key generator. This generator takes information from references to create the keys. A typical key pattern might be [auth][year] which might create a key like "Smith2000" for an article by Smith in the year 2000.

However, this presented a challenge. Once created, keys should not be changed. If the choice of key mattered and I chose a poor key, it would be a big issue once I had many documents integrated with many citations all using these key formats. The key can also be used to link article PDFs if you rename your files to match the key. Thus, I decided to think about whether the choice of key mattered and if so, what makes a good key?

The remainder of this post discusses my thoughts and sets out the auto-key generation pattern that I adopted.

DECIDING ON A KEY

A good key is:
Unique: Within the database a key must be unique in order to link citations to the appropriate reference. While Jabref automatically appends letters to duplicates in the database (i.e., Smith2000a, Smith2000b), if you need to combine two databases from two different researchers, it would be better if there were no identical keys.

Short: Short references are easier to type. They take up less space when cited in LaTeX source.

Readable in LaTeX: If the key suggests the reference, this can make the LaTeX text more readable. It also means that if the database is lost, damaged, or not available, a reasonable guess can be made about what was the intended citation, at least if you know the area of research.

Excludes problematic characters: Certain characters would prevent the key from being used in LaTeX or prevent it from being used as a file name. Using only letters and numbers and starting with a letter seems like a safe option.
Never Changes: A good key never changes. If it were to change, connections to the reference in documents and files would also need to be updated.

Decision:
In the end I adopted the following algorithm

[auth][year][journal:abbr]:
This key prints first author, year and initials of the journal. It is of moderate length (perhaps 10 to 20 characters), quite highly unique, and quite recognisable. It also represents how I file articles in my head. The choice may also reflect my training in APA format, which adopts an author-year style of citation. Thus, when speaking to others about an article, I might refer to it as "Smith 2000" or "Smith's 2000 JAP article".

I also had to overcome one extra issue:

Avoiding Problematic Characters

Problem: I found that some of my journal names included Ampersands (&) (e.g., Memory & Cognition; Journal of Personality & Social Psychology). This led to Bibtexkeys with Ampersands (&), which is problematic for LaTeX.

Solution: I changed the journal names to use "and" instead of "&". For example, "Memory and Cognition" replaced "Memory & Cognition". I also added a field to the replace regular expression in JabRef.
Replace (regular expression): \&
by:
i.e., the replacement was the "&" preceded by an escape character and the "by" field was left blank indicating that such a character should be removed.

Here's a screen shot of my BibTex key generator preferences:

What happens if the details of the reference change?
Once a key has been generated it should almost never change. Thus, even if a field in the reference used to generate the key was incomplete or had a typographical error, the key should not be regenerated. Similarly when references are imported that use a different key, they should not be altered.
In short, I think its better to not have to think about it too much. In JabRef I also specify "Do not overwrite existing keys" to further prevent the loss of keys.

LOOKING UP THE KEY
I have JabRef open while I edit my documents. My procedure:
1. alt-tab to bring up JabRef
2. Control + F (Find) type identifying information e.g., author year
3. Control + Shift + E (Focus entry table)
4. Down Key (to highlight first reference)
5. Control + Shift + K (to copy BibTex key) (or Control + K to include the \cite{...} text)
6. Alt + Tab to return to text editor
7. paste citation into text editor

I believe some editors have even better integration with JabRef than the one that I use.

Getting Started with Sweave: R, LaTeX, Eclipse, StatET, & TeXlipse

2010-02-23T17:30:00.011+11:00

Being able to press a single button that runs all your statistical analyses and integrates the output into your final report is a beautiful thing. If you have not already heard, this is what Sweave can do for you. However, getting your computer to run Sweave can be a little bit fiddly. Thus, this post: (1) sets out the benefits of Sweave; (2) sets out how to install and configure R, Sweave, and Eclipse on Windows; (3) lists resources for people wanting to learn more about how to use LaTeX and Sweave; and (4) lists some specific resources relevant to researchers in psychology wanting to use these tools.

OVERVIEW

What is Sweave?

To Sweave is to weave in S. To weave is to combine data analysis code and standard formatted text into a single self-describing document. R is a dialect of S. Thus, if you use R to do your statistical analyses and you want to automate the importation of analyses in R into your reports, Sweave may be the tool for you. For a longer description, see Friedrich Leisch's (2002) Sweave: Dynamic Generation of Statistical Reports Using Literate Data Analysis.

Why Sweave?

Reproducibility: The most important reason to adopt a tool like Sweave is to make your research more reproducible. The R code sets out exactly how the raw data is transformed into publication output. The Sweave document links this R output with the final report.
Efficiency: Statistical output is automatically incorporated into your report. There is no need to copy and paste output from your statistical analysis program into your report. If your data or analyses change, you can update your report with a single click instead of having to manually update every table and figure.
Reliability: The integration of analyses with the report reduces the chance of errors entering in through copying and pasting of statistical output into documents.
Education & Communication: By providing data analysis code for a report, this teaches others how to do similar analyses.
For an extended discussion, see Anthony Rossini and Friedrich Leisch's (2003) working paper Literate Statistical Practice.

Common Use Cases

Statistics Instructional Materials
Empirical reports, journal articles, book chapters, theses, etc.

Data sharing, literate programming, reproducible research, weaving: This is future of data analysis. Why not get on board now?

MY SWEAVE INSTALLATION AND CONFIGURATION

A strength of R, Sweave, and LaTeX is that they are cross platform tools that can be integrated together to support powerful data analytic workflows. These tools run on Windows, Linux, and Mac OS with a range of text editors and command line options. However, the flexibility in configurations presents a challenge. There is no single click installation file like "setup.exe". The tools need to be assembled. This section sets out how to install and configure a system for writing Sweave documents based around the Eclipse IDE and a Windows Operating System. It's not the only way to assemble a system to support Sweave, but for someone entrenched in the Windows world, I think its a good start.

1. Download and install R

R Project. I'm assuming you already use R, but if not you may wish to read my post on Getting Started with R.

2. Download and install a Latex distribution

There are several LaTeX distributions. I installed MikTex.

3.a Download and install Eclipse and the StatET and TeXlipse plugins

See the StatET Installation page for instructions on how to install StatET and Eclipse.
See also the links under "Getting Started" with StatET and Eclipse.

3.a Download and install Eclipse

See StatET Installation page; This assumes you have Java installed.

3.b Install the StatET plugin and the TeXlipse plugin

See StatET Installation page

3.c Configure the StatET plugin

See Longhow Lam's Eclipse and the R plug-in StatET

3.d Configure the TeXlipse plugin

The TeXclipse homepage lists general information. See specifically, the configuration page. My configuration could be abbreviated to: Go to Window - Preferences in Eclipse; Then, TeXlipse - Builder Settings; Then, enter the appropriate directory for your Bin directory of TeX distribution. In my case this was "D:\MiKTeX 2.8\miktex\bin" .

3.e Configure Sweave

Sweave.sty: Sweave is a built-in function in R. However, when you run Sweave, your LaTeX distribution needs to be able to find a file called "Sweave.sty". The file is stored in your R program files (e.g., "C:\Program Files\R\R-2.9.1\share\texmf"). A quick way to make it accessible is to place the file in your Eclipse project folder where the Rnw file is located. See this R-Help post for tips. UMN has some additional tips. (UPDATE: Bernd referred me to some additional material on linking Sweave.sty with MikTeX.
External Tools: Go to Run -- External Tools - External Tools configurations;
Sweave Document Processing (R/LaTeX); Click New Button; Give it a name like "Sweave-PDF"
Under the LaTeX tab change output format to pdf build commands pdflatex.exe

LEARNING LATEX

Using Sweave assumes that you know how to use LaTeX. If you just want to write LaTeX documents using Eclipse (without R code), you can go to File - New Project (Texlipse - LaTeX Project). Once you have a basic working environment, it's easy to experiment with all the details of LaTeX. Here are some web guides among the many that are available.

LEARNING SWEAVE

Sweave is fairly straightforward. In Eclipse you can start a new R Project and add an *.Rnw file to write your Sweave document. Then use the Document menu to convert the Sweave file to a TEX file, PDF file, etc. There are many more general resources on Sweave:

Friedrich Leisch's Resources Page: This includes an overview of Sweave, Sweave documentation; example Sweave documents, and discussion of literate programming
Learning to Sweave in APA Style: A useful introduction for those new to LaTeX, Sweave, and R.
Nicola Sartori's Sweave demo: 30 slide presentation introducing Sweave.
Sarah Haile's Overview: 17 slide introduction to Sweave.
Charles J. Geyer's Demonstration: Tutorial and notes with examples on using Sweave.
Vinh Q. Nguyen's UCI Seminar on Sweave: Slides introducing Sweave: Note particularly, the discussion of the cacheSweave package for computationally intensive code that you don't want to re-run every time you refresh your document.
Benjamin Bolker's website for his book Ecological Models and Data in R. The site has earlier drafts from his book and problem exercises in pdf and Sweave-ready format (see the Rnw files).
Drew Conway referred me to Michael Malecki's Sweave Template

LATEX, SWEAVE, AND PSYCHOLOGY

Adopting LaTeX and Sweave presents several challenges related to somewhat discipline specific needs. These pertain particularly to the various style conventions expected for journal submission. The following are some useful resources:

Introduction: Learning to Sweave in APA Style is A useful introduction if you are new to LaTeX, Sweave, and R from the perspective of a psychology researcher.
APA Style: William Revelle provides a page on Writing a research methods paper in APA style using LaTeX
APA Style assumes decimal point alignment in tables: dcolumn may be useful.
Reference Manager for latex: bibtex is the tool; apacite provides APA style instructions to bibtex.
Slide Creation: beamer is a commonly used tool to produce slides using LaTeX
Spell checker for LaTeX in Eclipse: Martin Homik provides some suggestions.
R and Psychology: My post on Getting Started with R for researchers in Psychology
Options for Collaborating with Non Latex Users:

Export Sweave document to HTML (e.g., R2HTML).
Use odfWeave. It's like Sweave, but it's for Open Office.
Convert LaTeX generated pdf to HTML.

Jeromy Anglim's Blog: Psychology and Statistics

Beamer presentations using pandoc, markdown, LaTeX, and a makefile

Motivation

Existing resources on combining Beamer, markdown and pandoc

My approach

makefile

Preamble LaTeX file: main.tex

Body markdown slide file: talk.md

Conclusion

Other aspects

Syntax highlighting of markdown+LaTeX in Vim

Code folding of markdown-Beamer

Showing backticks and single quotes properly in code

Converting Sweave LaTeX to knitr LaTeX: A case study

Additional Resources

My conversion from Sweave to knitr

Additional Resources

Transition to Linux and Ubuntu: The Experience of a Quantitative Social Scientist

Context of transition

Initial foray into Cygwin

Choosing a Linux distribution

Preparing to install Ubuntu

Reflection on the installation

Some reading

Looking forward

Assorted Resolved Issues

Overview

Overhaul file system

Question

Discussion

Accessing Windows Drive from Ubuntu

Question

Discussion

Monitoring processes and use of system resources

Question

Discussion

Command prompt for terminal colour

Question

Discussion

Opening common binary files from command-line

Question

Discussion

Keyboard shortcuts for size window left or right half of screen

Question

Discussion

Shortcut key to initiate web search

Question

Discussion

Using a laptop with multiple external monitors

Question

Discussion

Launching programs with keyboard shortcuts

Question

Discussion

Restart to Windows

Question

Discussion

Default terminal size

Question

Discussion

Prevent Unity menu from showing

Question

Discussion

Configuring Ubuntu for running basic scripts

Question

Discussion

Find file by name

Question

Discussion

Accessing Ubuntu drive from Windows on dual-boot machine

Question

Discussion

Script fur renaming files in folder

Question

Discussion

Minimise current window with keyboard shortcut

Question

Discussion

Terminator Terminal

Question

Preamble LaTeX file: `main.tex`

Body markdown slide file: `talk.md`