Trestle Technology, LLC

On Being Reliable

Wed, 26 Oct 2022 07:13:04 +0000

For many of us, adulthood comes with new sources of responsibilities. As a kid, it didn’t take much coordination to stay on top of a handful of classes or social groups that had expectations of me. But as an adult, I quickly discovered that new responsibilities seemingly emerge daily – each with different levels of urgency. There are things that I have to do today: pick up the kids, finish up the project I told my boss I’d get done, or pay that bill that’s due. Then there’s a class of things that I really should do soon: the yard’s overgrown and needs mowing, I need to decide on a strategy for the big project at work, and I keep putting off getting my flu shot. Then there are other things that are longer-term goals that I really want to make time for: I’d like to get a better handle on our finances, exercise more regularly, or learn to play guitar.

A few years ago, I got to a point where I was not faithfully executing the tasks in any of these categories and felt behind on everything. So I decided to start investing in finding strategies that allowed me to be more consistent with the things that I needed/wanted to do. For me, this journey can all be distilled down to two things: organization and discipline.

Organization

I’m speaking here of being organized when it comes to your responsibilities – not of keeping your room clean. If someone has their responsibilities organized, we might consider that person “reliable” or “dependable.”

It turns out that it doesn’t take much effort to become above-average at organizing your responsibilities. I’ll often hear suggestions like “we should make sure we’re prepared to start that project by next summer” or “It would be fun to go camping this fall”. In most circles I’ve been in, no one in the room actually expects that someone would follow through on those suggestions. Which means that being able to actually deliver on those ideas is perceived as a super-power.

I genuinely believe that anyone is capable of achieving this. We all have ways to remember these kinds of things if we genuinely cared to (e.g. put a reminder in your calendar), we just don’t take the time to do it. Becoming a more reliable person doesn’t require sophisticated strategies; the biggest determinant here is just deciding that you want to cultivate this skill.

There are likely hundreds of different tools and systems that you can use for managing tasks. I spent countless hours shopping for different systems that would help here and ultimately my conviction is that it doesn’t really matter which tool you use. What matters is that you have something that you’re regularly paying attention to. If I were starting this adventure all over again, I think I’d start with pen and paper. I wasted a lot of time getting distracted by various software tools and how to use each effectively. What really matters is that you have a list of tasks and when they’re due. (If you’d prefer to use something digital, look at todo.txt for a simple approach.) Then once per day, review your list to see what’s due today and what’s coming up that you need to get started on.¹

The one other pitfall I experienced was trying to use too many different systems to track my responsibilities. I’ve typically had a system at work where I was expected to track my assignments, then we all have email inboxes, or incoming Slack messages, etc. Wherever possible, I’d recommend consolidating all of these into one or at most two systems for tracking your responsibilities. I found it completely overwhelming to have emails marked as unread plus Slack reminders plus a todo list plus a task board at work. I was consistently inattentive to at least one of those systems. So transfer any incoming work to your “authoritative” task list as quickly as possible. If I receive an email that requires more than a few minutes of work, I write it on my task list and mark the email as read.

If you’re curious, I’m using OmniFocus these days. It’s a perfectly fine tool, but I don’t think there’s any magic in it that doesn’t exist in any of its alternatives. The magical bit is that I look at this single tool every day and use it as my primary source-of-truth for my responsibilities.

If you’re consistently willing to invest a couple of minutes a day in this practice, you’ll quickly find that you now have a mechanism for staying on top of any number of tasks and priorities.

Discipline

As I attempted to get my tasks more organized, I cycled through a variety of tools and none of them stuck. They eventually became a backlog of tasks that were hopelessly overdue. So I’d start over in a new tool and repeat the process. I blamed the tools, but I eventually realized that the common problem in all of these tools was the user (me) and that the underlying issue here was one of discipline.

I’ve never considered myself a terribly disciplined person. My wife, on the other hand, is the most disciplined person I’ve ever met. She would laugh to hear that I was giving advice on being disciplined. If you’re naturally a disciplined person, you might be able to skip right over this section. For the rest of you, hopefully my learnings as I struggled to become more disciplined will be of use to you.

No matter how well I organized my todo list, I was consistently getting behind. The tasks I’d neglect usually fell into one of these categories:

Tedious – “Manually copy and paste 100 items from this system to that system.”
Anxiety-provoking – “Have a difficult conversation with Jim ahead of his performance review.”
Overwhelming – “Learn to play the guitar.”

No matter how organized my todo list was, or how often I reviewed it, I never got around to completing these types of tasks. The two approaches that I’ve found that help here are decomposing tasks, and having a more strict adherence to the list.

The decomposition suggestion is probably self-explanatory. Instead of a task like “learn to play the guitar,” maybe the first step is “find a guitar teacher” or “order an instructional guitar book.” Break down the larger task into steps that you can complete in a few minutes. You could also schedule a recurring task of practicing guitar for 10 minutes every Monday (or whatever duration/frequency you know you’d be able to actually complete as your first step).

And then you just have to arbitrarily stick to doing what you said you were going to do. It won’t feel comfortable or pleasant to complete some of these tasks, but try to instill in your brain that if it’s on your todo list for today, you’re just going to get it done without delay or excuse.

As an aside, I had an interaction with a friend when I was younger that stuck with me. I was dreading doing a big project for school and complaining that I didn’t want to get started on it. He asked me sincerely: “Is it going to be easier to do it tomorrow?” My answer, of course, was: “No, I guess not.” To which he replied: “Then I guess go ahead and do it today.” I find that asking myself that question is helpful when I’m facing some of these unpleasant tasks. But to be clear, I’m still far from perfect in implementing this recommendation.

You can also combine these two techniques to suit your personality. If you’re naturally quite disciplined, you may be able to power through large, unpleasant or overwhelming tasks. I find that the more unpleasant the task is, the more I need to decompose it. In taking the example of the “difficult conversation” above, I might instead break the task down into:

Monday: collect examples of Jim’s problematic behavior.
Tuesday: sketch out some talking points on how to articulate the feedback
Wednesday: put a meeting on the calendar with Jim
Thursday: rehearse giving the feedback with my boss to get their input

These are tasks that I would actually be able to complete. It may take a bit of practice for you to identify the right way to chop up tasks that suits your personality. But if you find yourself not being reliable with certain things, pause and ask yourself why you’re not willing to do this task. Is it because it frightens you? Or because you don’t know how to start? Or because you don’t actually think it’s worth doing? Once you’ve identified why you don’t want to complete a task, you’re much closer to actually completing it (or deciding that it’s not worth completing).

Conclusion

Many smarter and more successful people than I have written on this topic and I’m sure I haven’t uncovered any new ground here. But I hope this testimony convinces you that even those of us who aren’t naturally disciplined or organized can still develop a reliability that stands out should we choose to.

Often people start by using a calendar to manage their tasks. I think this is a reasonable place to start if you’re only intending to track a handful of todos per week. In my experience, if you’re trying to track multiple deliverables per day in your calendar, things will quickly fall apart. If When you miss a deadline, your calendar won’t easily surface that overdue item for you. And you won’t be able to meaningfully review your tasks because they’re intermingled with actual events on your calendar.↩

Announcing the Plumber v0.4.4 Release

Mon, 04 Dec 2017 16:11:04 +0000

Plumber is a package which allows you to create web APIs from your R code. If you're new to Plumber, you can find out more at www.rplumber.io.

We're excited to announce the v0.4.4 release of Plumber! This release adds a handful of oft-requested features and cleans up a few issues that came out of the major refactor that took place in the 0.4.2 release. We've also continued to expand the official Plumber documentation. We'll mention the highlights below, but you can see the full release notes for v0.4.4 here. The release is on CRAN now, so you can update using:

install.packages("plumber")

Plumber v0.4.4 Highlights

Support customized image sizes on the @png and @jpeg annotations. More details here.
Support expiration, HTTPOnly, and Secure flags on cookies as discussed here (but see the "Known Bugs" section below for an important note about cookie expiration in v0.4.4).
Restore functionality of PlumberStatic routers (#156).
Support arguments sent from clients to nested subrouters.
For APIs deployed using DigitalOcean, set the working directory to the root of the API before starting.
Restore functionality of setErrorHandler.
Ignore capitalization when searching for plumber.r and entrypoint.r files when plumb()ing a directory.
Support query strings with keys that appear more than once (#165)
Fix the validation error warning at the bottom of deployed Swagger files which would have appeared any time your swagger.json file was hosted in such a way that a hosted validator service would not have been able to access it. For now we just suppress validation of swagger.json files. (#149)
Support for floating IPs in DNS check that occurs in do_configure_https()
Make adding swap file idempotent in do_provision() so you can now call that function on a single droplet multiple times.
Support an exit hook which can define a function that will be evaluated when the API is interrupted. e.g. pr <- plumb("plumber.R"); pr$registerHook("exit", function(){ print("Bye bye!") })
Fixed bug in which a single function couldn't support multiple paths for a single verb (#203).

Known Bugs

At the time of writing, one bug has already been fixed since the v0.4.4 release. Cookie expiration times were not properly being sent to clients which caused most clients to ignore these times altogether and revert back to using session cookies. If you wish to set an expiration time on a cookie, you will need to use the development release of Plumber which you can install using:

devtools::install_github("trestletech/plumber")

Preparing for the Plumber v0.4.0 Release

Thu, 20 Jul 2017 09:13:04 +0000

Plumber is a package which allows you to create web APIs from your R code. If you're new to Plumber, you can take a look at www.rplumber.io to learn more about how to use the package to create your APIs.

Over the years, we've noticed a handful of things that we wished we'd done differently in the development of Plumber. In particular, there were some decisions that prioritized convenience over security which we wanted to roll back. We've decided to bite the bullet and make these changes before more users start using Plumber. The v0.4.0 release of Plumber includes a handful of breaking changes that mitigate these issues all at once. Our hope in getting all of these out of the way at once is to ensure that users building on Plumber moving forward have confidence that any breaking change under consideration has already been made and that we shouldn't have this kind of churn again anytime soon.

If you're already using Plumber, we strongly encourage you to read the v0.4.0 migration guide below. As you'll see, most of these changes affect the hosting or low-level interactions in Plumber. The dominant mode in which people use Plumber (adding comments to their existing R functions) has not changed; there are no breaking changes in how those files are interpreted.

v0.4.0 of Plumber will be deployed to CRAN in the coming days. We strongly encourage you to try out the v0.4.0 version of Plumber ahead of time to make sure that you've migrated your APIs correctly.

devtools::install_github("trestletech/plumber")

Alternatively, you can continue using the last release of Plumber which didn't include any of these breaking changes (v0.3.3) until you're ready to upgrade by using the following command:

devtools::install_github("trestletech/plumber", ref="v0.3.3")

You can see the full release notes for v0.4.0 here.

Plumber v0.4.0 Migration Guide

There are a number of changes that users should consider when preparing to upgrade to plumber v0.4.0.

Plumber no longer accepts external connections by default. The host parameter for the run() method now defaults to 127.0.0.1, meaning that Plumber will only listen for incoming requests from the local machine on which it's running -- not from any other machine on the network. This is done for security reasons so that you don't accidentally expose a Plumber API that you're developing to your entire network. To restore the old behavior in which Plumber listened for connections from any machine on the network, use $run(host="0.0.0.0"). Note that if you're deploying to an environment that includes an HTTP proxy (such as the DigitalOcean servers which use nginx), having Plumber listen only on 127.0.0.1 is likely the right default, as your proxy -- not Plumber -- is the one receiving external connections.
Plumber no longer sets the Access-Control-Allow-Origin HTTP header to *. This was previously done for convenience but given the security implications we're reversing this decision. The previous behavior would have allowed web browsers to make requests of your API from other domains using JavaScript if the request used only standard HTTP headers and were a GET, HEAD, or POST request. These requests will no longer work by default. If you wish to allow an endpoint to be accessible from other origins in a web browser, you can use res$setHeader("Access-Control-Allow-Origin", "*") in an endpoint or filter.
Rather than setting the default port to 8000, the port is now randomly selected. This ensures that a shared server (like RStudio Server) will be able to support multiple people developing Plumber APIs concurrently without them having to manually identify an available port. This can be controlled by specifying the port parameter in the run() method or by setting the plumber.port option.
The object-oriented model for Plumber routers has changed. If you're calling any of the following methods on your Plumber router, you will need to modify your code to use the newer alternatives: addFilter, addEndpoint, addGlobalProcessor, and addAssets. The code around these functions has undergone a major rewrite and some breaking changes have been introduced. These four functions are still supported with a deprecation warning in 0.4.0, but support is only best-effort. Certain parameters on these methods are no longer supported, so you should thoroughly test any Plumber API that leverages any of these methods before deploying version 0.4.0. Updated documentation for using Plumber programmatically is now available.

EigenCoder: Programming Stereotypes

Wed, 09 Mar 2016 08:15:04 +0000

There are a lot of stereotypes in the programming community. "Swift is used by a bunch of bearded hipsters." "C++ is for old people." "No one likes coding in Java." Well it turns out that some of these might be true.

Approach

GitHub is likely the most popular open-source hosting platform in use today. GitHub has many open-source repositories for code written in wide variety of languages. They also provide "trending" pages which show you repositories in a particular language that are particularly popular right now.

These popular repositories also include the profile pictures of some of the most prolific committers on these projects. Meaning that we can easily get a few dozen profile pictures of some of the busiest contributors to popular projects for any given language.

We also have access to a neat resource from Microsoft's Project Oxford called the "Face API" which, among other things, can detect a face in a given image and estimate some properties about the face. Is the subject smiling? What gender and age are they? Do they have facial hair?

Combined, we can get an estimate of some interesting properties about the profile pictures of programmers who are working in various languages. Let's get started!

It should be noted that this is super non-scientific. Who knows how accurate the Face API is or how accurately a user's GitHub profile picture maps to any aspect of their personality/identity. It's also unclear whether the most prolific contributors to popular repositories accurately represent a community. Also, small sample sizes. Etc., etc.

All code used in this post is available here: https://github.com/trestletech/eigencoder

Data

GitHub lists 25 repositories on its trending page and shows the top 5 committers for each. Some projects don't have 5 contributors, so fewer are shown. We remove duplicated usernames then send each of these profile pictures (up to 125 per language) to be analyzed by the Face API. Of course, not all pictures have (detectable) faces in them.

In total, we get the following:

Lang	FacesDetected
ruby	71
r	38
javascript	60
java	47
html	59
go	53
cpp	34
c	24
python	49
php	66
perl	45
swift	49
csharp	61

Gender

One of the properties returned by the Face API is a prediction of the gender of the subject of the photo. The results are pretty discouraging for anyone who's not a chauvinist...

Age

Age is an interesting trend. Some people assume that "old-school" languages are only used by old people and that new, trendy languages are used by hipsters. It turns out that's not always true; Java, for instance, has the lowest median age.

Barplot
Density

Smiles

Every programmer has a language that makes them miserable. So miserable, perhaps, that you can't even muster a smile for your GitHub profile picture.

The Face API returns a score from 0 to 1 approximating the amount that you're smiling. Programmers using certain languages seem happier than others. Maybe R programmers are just smiling about the crazy market for data scientists in this economy...

Barplot
Density

Facial Hair

If you've been coding for any length of time, you've met at least one mustached fellow riding a fixie and wearing skinny jeans who won't stop talking about Swift. Turns out that's a real stereotype.

I did not normalize for gender here.

Barplot
Density

Or we can look at each facial hair property returned by the Face API individually.

Conclusion

I guess to grow a mustache if you want to contribute to a successful C++ project?

In reality, you'd need a much larger sample size and some guarantees around the accuracy of the Face API before you could have any real confidence about the conclusions you draw from this data.

plumber — Convert R Code to a Web API

Wed, 24 Jun 2015 12:00:04 +0000

I’m excited to announce a new R package: plumber, a package that enables you to convert your existing R code into web APIs by merely adding a couple of special comments.

EDIT: This package was originally named “rapier” and has since been renamed to “plumber”.

Take a look at an example:

# myfile.R

#' @get /mean
normalMean <- function(samples=10){
  data <- rnorm(samples)
  mean(data)
}

#' @post /sum
addTwo <- function(a, b){
  as.numeric(a) + as.numeric(b)
}

These comments allow plumber to convert your R code into a web API with just a couple of commands:

> library(plumber)
> r <- plumb("myfile.R")  # Where 'myfile.R' is the location of the file shown above
> r$run(port=8000)

You can visit this URL using a browser or a terminal to run your R function and get the results. Here we’re using curl via a Mac/Linux terminal.

$ curl "http://localhost:8000/mean"
 [-0.254]
$ curl "http://localhost:8000/mean?samples=10000"
 [-0.0038]

plumber Magic

As you might have noticed, there are a couple of neat tricks plumber uses to save you time. First, it identifies all the API endpoints in your code by looking for special annotations like @get and @post.

Second, plumber converts all query string parameters (the samples=10000 in http://localhost:8000/mean?samples=10000) and URL-encoded POST bodies (often used when submitting forms online) to be available as parameters in your plumber functions. This trick allows us to connect the samples value given by the API user with the parameter in the R function without you having to do any additional wiring. This makes it much simpler to actually start using your API.

Demos

Webhooks are just one example of what you can do once you’ve exposed R to the web using plumber. A Webhook is a pattern that allows an API endpoint to subscribe to updates or notifications from some service. GitHub, Dropbox, Google, and many others offer Webhooks with many of their services; we chose to demonstrate the use of a GitHub Webhook in this example.

You can see that plumber allows us to give a “target” for the GitHub Webhook we setup, allowing us to execute whatever R code we desire in response to this event. In this case, we install the latest version of the R package when we receive the notification, resulting in a machine that is running the up-to-the-second latest version of an R package.

Behind The Scenes

The two functions you’ve seen above are examples of plumber “endpoints.” You can read more about endpoints on the endpoints documentation page.

In addition to endpoints, you can also use plumber filters, which are a layer of middleware in your web service. You can use filters to do things like require user authentication or pre-process some value before the request gets to an endpoint.

Much more detail is provided at the plumber website.

Summary

The extent of what’s possible once you’ve exposed your R code to the web using plumber is limitless. We’re excited to see all the different ways the community can leverage this tool.

The package is open-source (MIT) and maintained on GitHub at trestletech/plumber and the project page is available at https://rplumber.io.

Let us know what you think!

Todo.txt++ Analysis

Wed, 29 Oct 2014 05:12:08 +0000

Todo.txt++ is a hosted version of an open-source application built around the todo.txt protocol. Todo.txt++ has been online for a few months now, and has had many visitors. At the moment, we have 390 todo.txt lists in our cache.

We can perform some basic analysis on these todo.txt lists to see how people are using their todo.txt lists. This information can provide insight into which features are most popular with users which can guide what features we work to get into Todo.txt++.

Below are histograms of various features of the Todo.txt protocol or add-ons that show how many people are using each.

It looks like contexts and projects are commonly used.

We can now look to see how many tasks have been given a priority:

or the percent that start with a creation date.

Some users use the DUE: syntax to show when a task is due…

We can do some analysis on TODOs that are or are not done, but since we don’t yet support done.txt, we’re obviously going to be missing many people’s dones.

There are quite a few lists with little-to-no completed tasks. Specifically, 58.46 % of lists have zero completed tasks; presumably they’re using a done.txt file. If we exclude those, the graph looks a bit different:

It’s a bit more reasonable of a distribution.

Conclusion

It looks like not as many people use the DUE: syntax as expected, so perhaps resources would be better spent working on other features like sorting by priority, or adding done.txt support which it looks like most users are using.

If you have other features you’d like to see or would like to vote on existing feature requests, drop us a line on our Issues page.

shinyStore – Persistent Client-Side Storage in Shiny

Fri, 12 Sep 2014 02:38:59 +0000

We’re thrilled to announce the availability of shinyStore, an R package that enables HTML5 Web Storage from Shiny, an interactive web application framework for R.

A live demo of an example application is available here. Set a text value then refresh the page, or close the tab and come back in a new tab. You’ll find that any tab in that browser will always remember the last text value you’d saved. You can imagine using this functionality to remember a user’s preferences or inputs for your application.

The package is open-source (MIT) and the code and more details can be obtained from the GitHub page; if you encounter any problems or feature requests, feel free to open a ticket there.

To get started with the package, you’ll need to installed it from GitHub, as it’s not yet available on CRAN. The easiest way to do this is using the devtools package which you should install if you haven’t already:

install.packages("devtools")
library(devtools)

Now you’re ready to install shinyStore and load up one of the examples:

# Load shiny (I'm assuming it's already installed)
library(shiny)

# Run one of the included examples in the shinyStore package.
runApp(system.file("examples/01-persist", package="shinyStore"))

We’re interested in hearing ways that you might be able to use this package, or ways that it could be improved to better accommodate your use cases!

shinyTree: jsTree + shiny

Fri, 05 Sep 2014 01:10:13 +0000

We’re happy to announce the release of our latest R package, shinyTree. shinyTree is an integration of the jsTree library with the Shiny interactive web framework for R, which makes it simple for R developers to create web applications without knowing anything about HTML or JavaScript.

You can view a live demo of one application here.

shinyTree offers Shiny developers a way to create interactive trees that can be selected and interactively expanded/collapsed with convenient ways to access the user’s current selections and any other property about how the tree is currently presented. You can view more details or download the source code from the GitHub page; if you encounter any problems or feature requests, feel free to open a ticket there.

install.packages("devtools")
library(devtools)

Now you’re ready to install shinyTree and load up one of the examples:

# Load shiny (I'm assuming it's already installed)
library(shiny)

# Run one of the included examples in the shinyTree package.
runApp(system.file("examples/04-selected", package="shinyTree"))

We’re considering features like client-side drag-and-drop support within the trees as well as adding search capabilities to trees; let us know in the comments on or GitHub what you’d be most interested in using with your projects. We’re also interested in ways to make this package more performant.

Ace Code Editor in Shiny (shinyAce)

Wed, 06 Nov 2013 03:42:40 +0000

The Ace code editor is an elegant, full-featured, browser-based code editor used in such products as RStudio’s IDE. We’re announcing a new R package called shinyAce (now available on CRAN) which integrates the Ace editor into the Shiny web framework.

For those who may be unfamiliar, Shiny is an R package which makes it trivial to create interactive web applications in R — even without knowing any of the languages typically used to create websites (like HTML or JavaScript).

shinyAce exposes all of the color schemes and programming languages (called “modes” in Ace) that Ace supports, meaning you can customize the look and feel of the editor and support syntax highlighting for R, JavaScript, Python, and dozens of other languages. The editor is treated as just another input in Shiny, making it simple to reactively access the text currently contained in the editor. There is also an update function that allows you to update the text (or mode/theme) once the editor has already been created.

See the GitHub page to download the code, view installation instructions, or see details on the examples that come with the package. These examples showe how to create Shiny applications that use shinyAce to allow users to edit and run R code, R Markdown, or even another Shiny user interface.

To install the package:

# Download and install shinyAce
install.packages('shinyAce')

To run an example of shinyAce that features all the available modes and themes on your own machine now:

# Install and load Shiny and ShinyAce
install.packages('shinyAce')
library(shiny)

runApp(system.file("examples/01-basic", package="shinyAce"))

You can view a description of all the examples (and instructions for running them) on the GitHub page. Feel free to let us know if you have any issues or feature requests on our Issues page.

Interactive 3D in Shiny (shinyRGL)

Mon, 14 Oct 2013 03:30:20 +0000

The rgl package has been used to produce rich, interactive 3D graphics within R using the OpenGL framework for some time. The rgl authors also added the ability to export to WebGL, allowing R authors to produce interactive 3D graphics that were accessible from within any modern web browser.

This functionality can now be leveraged within the Shiny framework using the new shinyRGL package — allowing users of R who may have no prior web development experience to create interactive web applications which incorporate 3D graphics. Examples such as this 3D cube filled with variable numbers of points or this modification of the ‘bivar’ example included in rgl can be produced with minimal effort. (The source code for both of these applications is available in the /examples/ directory of the shinyRGL package; both are under 100 lines of code and are written entirely in R.)

The package was just released on CRAN and should now be available from your preferred CRAN mirror. You can install using the command:

install.packages("shinyRGL")

Alternatively, you can download the bleeding-edge version of the code, view more details about the implementation, or track the development of the package on our GitHub repository. Please use GitHub to notify us of any issues or feature requests you may come across, as well.

Below is a sample of the kinds of graphics that can be produced using shinyRGL by writing only R code. Be sure to pan the scene by clicking and dragging, zoom using the scroll wheel, and even change the point of view by clicking and dragging with the right mouse button.

You must enable Javascript to view this page properly.

Reproducing R: Scripts, Documents, and Packages

Sat, 29 Jun 2013 05:19:17 +0000

I’m sharing the slides from the talk I’ll be giving at the Dallas R Users Group on creating R packages (and other techniques for reproducing R). I’ll introduce R scripts, reproducible R documents, and R packages. We’ll use the knitr, devtools, and roxygen2 packages in the examples.

Download the slides here.

If you’re unable to make the talk, feel free to watch a video recording of the steps involved in creating an R package here.

Dallas R Users: Creating R Packages this Saturday, 6/29

Wed, 19 Jun 2013 03:53:32 +0000

I’ll be presenting at the Dallas R Users Group next Saturday at 10:00AM at the University of Dallas on how to reproduce your R code. We’ll review how to use R scripts, how to embed R code in reproducible documents, and then introduce how to create your own R packages based on your R code. At the end of the afternoon, you’ll be familiar with the entire process of creating an R package to be shared on a repository like CRAN or Bioconductor. No programming background is required – the entire process will be done within RStudio and the R code we’ll be using will be very basic.

Attendance is, as always, free and all in the Dallas/Ft. Worth metroplex are welcome! Details and directions can be found on the Meetup page below.

http://www.meetup.com/Dallas-R-Users-Group/events/124213832/

See you there!

Package-Wide Variables/Cache in R Packages

Mon, 08 Apr 2013 23:00:03 +0000

It’s often beneficial to have a variable shared between all the functions in an R package. One obvious example would be the maintenance of a package-wide cache for all of your functions. I’ve encountered this situation multiple times and always forget at least one important step in the process, so I thought I’d document it here for myself and anyone else who might encounter this issue. I setup a simple project on GitHub to demonstrate the various attempts you may take to solve the problem and, ultimately, a solution: https://github.com/trestletech/RCache. The rest of this article will presume some knowledge of authoring R packages. If that’s not you, check out RStudio’s guide to authoring R packages.

The fundamental problem comes down to R’s management of environments. (I’ll introduce the basics here, but for a thorough discussion on the topic, be sure to check out Hadley Wickham’s writings on the topic.) An environment is responsible for mapping data to named objects. When I execute

> a <- "hello"
> a
[1] "hello"

there was an environment responsible for initially associating the length-one character vector containing the text hello with the variable named a. Later, when I go to reference a variable by name, an environment is responsible for retrieving the data I had associated with this variable. Functions have their own environments in which they run, which is why you observe behavior like:

> x <- 1
> foo <- function(){ 
  print (x) 
  x <- 2 
  print(x) 
  x 
} 
> x 
[1] 1
> foo() 
[1] 1 
[1] 2 
[1] 2
> x
[1] 1

You can see that the variable named x, depending on which environment it’s in, can take one of two values in this example. Inside of the foo function, the x variable initially only existed in the parent environment, so it took on the value of x in that environment and was 1. Later, it was assigned a value of 2 and retained that value when it was returned. Outside of the function, however, x was associated with the value 1, and this binding was maintained despite another variable named x being created inside of a function’s environment.

This is a powerful feature of R, when properly understood. This allows you to create variables that only exist inside of a function or, more relevant to today’s discussion, only within a particular package. So it should be possible to leverage these environments to create a variable named cache which is accessible to all the functions in my package, but won’t accidentally likely be overwritten or modified by the user or, even more likely, collide with another variable named cache used in some other package.

Example Package – rev. a804

For the rest of the demonstration, we’ll use the example of creating an R package which merely downloads files from the Internet. Of course, it might be appropriate to cache data in such a package so that the same URL won’t have to be retrieved remotely multiple times. One simple approach would be to use a list to associated character strings (such as URLs) with some data (such as the content of the web page). We can first create a function which will download data without using any cache:

#' Download a file.
#'
#' @importFrom httr GET
#' @importFrom httr content
#' @export
download <- function(url){
  content(GET(url))
}

Example Package – rev. a81e

Creating a variable that is available within all functions of your package is as simple as binding a variable to data outside of any functions in a .R file in your package.

cache <- list()

One might think we could then put data into that cache variable by assigning named elements within to it from packages that can access it. For instance:

download <- function(url){
  if (!is.null(cache[[url]])){
    return(cache[[url]])
  }

  file <- content(GET(url))
  cache[[url]] <- file

  file
}

Unfortunately, this, like our first example, is assigning data to a separate variable named cache which exists only inside of this function. If you were to build the package and run it, you’d find that the code:

down <- download("http://www.gutenberg.org/cache/epub/2500/pg2500.txt")
RCache:::cache

would successfully download the book, Siddhartha, but the package cache would be empty. (If you’re unfamiliar with the ::: operator, it checks inside of the package — named on the left — for a variable — named on the right and returns it. So we can inspect the cache variable from code outside of our package.)

Special Assignment – rev. 5548

In order to alter a variable created in the parent environment, one must use the special assignment operator, <<-. This will adjust a binding not in the current environment, but in a parent environment. So we can adjust our line in which we assign a value to the cache variable to use this operator:

file <- content(GET(url))
  cache[[url]] <<- file

However, if we run this, we’ll find that we get an interesting error:

down <- download("http://www.gutenberg.org/cache/epub/2500/pg2500.txt")
Error in cache[[url]] <<- file :
  cannot change value of locked binding for 'cache'

What’s the deal? R has a concept of “Locked Bindings” which allow you to ~~forbid~~ discourage changes in variables by locking either a particular variable binding or an entire environment. In this case, the cache binding has been locked and can’t be altered by constituent functions. So we’ll need to take a different approach altogether.

Environments – rev. 320e

It seems we can properly access a package-wide variable from within a function of a package, but we’re not allowed to overwrite it (or create new variables at that level from within a function). Perhaps we could leverage environments to solve this problem. As it turns out, environments were likely a cleaner solution to our problem all along. Instead of creating a cache list, we can create it a cache environment:

cacheEnv <- new.env()

As long as this environment is created in one of our package’s .R files and not inside of a function, it will be accessible across our entire package. We can do all the things with this environment that we’re used to doing in our regular R environment (whether we knew we were using environments or not): create new variables (assign), modify existing variables (assign), remove variables (rm), retrieve data associated with a variable (get), list the variables in an environment (ls), etc.

All of the functions mentioned above accept an envir argument which specifies in which environment you’d like to perform the operation. The default is your current environment, but you could just as easily point these functions at your new environment to do something like assign(url, file, envir=cacheEnv) to assign the value currently stored in the file variable to a new variable who’s name is the value currently contained in the url variable within our cache environment. Then we could use get(url, envir=cacheEnv) to get the variable who name matches the current value of the url variable in the cache environment.

For instance:

> url <- "http://mytext.com"
> file <- "This is the content I downloaded"
> cacheEnv <- new.env()
> assign(url, file, envir=cacheEnv)
> get(url, envir=cacheEnv)
[1] "This is the content I downloaded"

Now we can incorporate this into our package by changing the download function to use these facilities:

download <- function(url){
  if (exists(url, envir=cacheEnv)){
    return(get(url, envir=cacheEnv))
  }

  file <- content(GET(url))
  assign(url, file, envir=cacheEnv)

  file
}

And we finally have a working cache. When a URL is requested via our package’s download function, the data will first be stored in this package’s cacheEnv environment before being returned. The next time that URL is requested, the cacheEnv environment will be checked to see if we already downloaded the content of that URL. Because a variable by that name already exists in our cacheEnv environment, the value will be pulled from there and returned rather than retrieved remotely.

Conclusion

Hopefully you’ve learned a thing or two about environments in R and how to use them from within R packages. Environments can be very valuable tools for advanced R programmers. They’re one of the tried-and-true ways to replicate “pass-by-reference” programming in R (though that’s typically unexpected — thus discouraged — behavior for an R object). They also have some unique lookup properties being built on hashes that allow them to more expediently map character strings to data when there is a very large number of possible keys.

Happy packaging!

Graphical Tools (rgl) on a Headless Shiny Server

Thu, 28 Feb 2013 01:15:18 +0000

If you’ve encountered errors such as

Warning in rgl.init(initValue) : RGL: unable to open X11 display 
Warning in fun(libname, pkgname) : error in rgl_init

Error: rgl_dev_getcurrent

when trying to use a graphically based package like RGL with Shiny Server, then you’re in the right spot. The issue is likely that you’re running R or Shiny Server on a server which has no monitor attached to it (a “headless” server). Tools such as rgl require a graphical “X” environment in order to function properly. Since you have no such graphical system, the tools won’t work properly.

But you’re in luck! More intelligent programmers than you and I have blazed this trail before and created XVFB, or the X Virtual FrameBuffer. This system emulates a graphical setup without requiring that any screen is even attached to the server on which you’re running.

If you’re the DIY type, check out the instructions below. If you prefer the ready-made solution, you can try out one of our free, public AMIs on Amazon EC2. We have the base Shiny Server installation on Amazon Linux (described thoroughly in our previous post), or our new AMI with the installations and configurations described below already completed: ami-ec2dbf85, “ShinyServer-RGL-XVFB”. Whichever path you choose, make sure that you have the proper firewall ports open (3838, by default) to access Shiny Server remotely.

Installation

Most Linux distributions already have a pre-built library for XVFB which can be installed using your package manager (likely yum or aptitude, if you’re not sure). If you’re using our Shiny Server AMI on Amazon’s EC2 (setup described here), then you already have the necessary dependencies installed.

Once you have XVFB setup, you’re ready to install rgl (or whatever other graphical package you’re looking for). If you’re on Ubuntu, you can find more detailed instructions about dependencies for rgl at our post on that topic in particular.

You should now be able to test your setup by starting XVFB manually, setting an environment variable for your display, then starting Shiny Server manually. (If your server is already running Shiny Server, you’ll need to stop that instance first. Try service shiny-server stop or stop shiny-server). Now you’re ready.

Xvfb :7 -screen 0 1280x1024x24 &
        export DISPLAY=:7
        shiny-server

which will start a virtual X session using XVFB on :7, set your DISPLAY environment variable accordingly, then start shiny server (assuming all of these tools are in your PATH). You can now try loading an RGL-based shiny application (you can try ours if you don’t have one). If things go well, then you have all the necessary ingredients to do some cool 3D apps with Shiny!

Automation & Configuration

The final step is automating this configuration so that it will load automatically on boot. If you’re using the older init.d style scripts, unfortunately, you’re on you’re own. You can likely hack a few lines into an rc.local file to get XVFB to run automatically on boot. If you’re using the newer Upstart system, then it’s a bit easier; you’ll just need to slightly tweak your /etc/init/shiny-server.conf file. Look at the script section of the file below for guidance.

# shiny-server.conf

description "Shiny application server"

#start on stopped networking

start on stopped rc RUNLEVEL=[S3]
stop on runlevel [016]

limit nofile 1000000 1000000

script
        Xvfb :7 -screen 0 1280x1024x24 &
        export DISPLAY=:7
        exec /usr/local/bin/shiny-server >> /var/log/shiny-server.log 2>&1
end script
respawn

This file will preface the shiny-server execution with the Xvfb startup lines we saw previously. This way, Shiny Server will always have access to that Xvfb screen when running its apps.

Now go make some cool apps! A couple of examples to get you started: color plots, and arbitrary 3d points (both with source code available on GitHub). Let us know in the comments if you’re able to build anything interesting!

Analysis of Public .Rhistory Files

Thu, 21 Feb 2013 00:06:25 +0000

GitHub recently launched a more powerful search feature which has been used on more than one occasion to identify sensitive files that may be hosted in a public GitHub repository. When used innocently, there are all sorts of fun things you can find with this search feature.

Inspired by Aldo Cortesi's post documenting his exploration of public shell history files posted to GitHub, I was curious if there were any such .Rhistory files. For the uninitiated, .Rhistory files are just logs of commands entered into the interactive console during an R session. Some recent IDEs, such as RStudio, automatically create these files as you work. By default, these files would be excluded from a Git repository, but users could, for whatever reason, choose to include their .Rhistory files in the repository.

Using this search function, combined with the Python script Mr. Cortesi had put together to download the files associated with a GitHub search, I was able to download 638 .Rhistory files from public GitHub repositories (excluding forks). What follows is an exploration of those files.

Load Data

Trimming out the 0-line .Rhistory files leaves us with a total of 531 non-empty files, totaling 157265 commands entered into R.

First, I was curious about the length of these files.

It seems that many of these files represent very brief (and likely unpleasant) interaction with R. For instance:

exit
exit
ls
exit

(if you're out there, you were likely looking for the 'q()' command). Others represent quite extensive projects; the maximum was 7268 lines long.

Package Usage

More interesting to me was how these users were using R – what the details contained in these history files represent in terms of the user's interaction with R. For starters, which packages were the users using? We can identify packages loaded via the library() or require() functions.

There were 3068 such calls to load packages in the scripts. The top 10 packages loaded in this set were:

Package Name	Count
`ggplot2`	291
`plyr`	81
`GREBase`	59
`xtable`	59
`reshape`	52
`reshape2`	48
`devtools`	43
`igraph`	41
`RGreenplum`	40
`lattice`	39

(Of course it's likely worth noting the selection bias from examining only R commands which were included in GitHub projects. I would imagine that the usage for devtools, for instance, is certainly inflated among GitHub projects over the general populace.)

Function Use

I was also curious which functions were most widely executed. We can get a rough identification of most function names by looking for a sequence of valid characters followed by an ( symbol.

This gives us a total of 100190 function calls of 8028 unique function names. The 20 most popular functions executed in this set were:

Function Name	Count
`source`	5191
`plot`	2552
`c`	2448
`library`	2416
`function`	1711
`for`	1138
`summary`	1107
`if`	1062
`read.csv`	955
`rep`	887
`length`	880
`head`	828
`lm`	766
`sum`	753
`View`	722
`print`	661
`install.packages`	648
`mean`	606
`setwd`	569
`names`	562

It should also be possible to identify for which functions the help/manual pages were viewed by identifying lines beginning with a “?” or arguments inside of a call to help().

I can identify 2409 requests for help on 1101 different function names. The top 10 most prevalent functions for which users request help follow.

Function Name	Count
`plot`	43
`hist`	31
`lm`	25
`writePage`	24
`order`	20
`sort`	20
`cor`	17
`apply`	16
`read.csv`	16
`matrix`	15

Conclusion

Of course, there are all sorts of different types of analysis one could perform on this dataset. Post any suggestions you have in the comments; I imagine there's at least one more post of interesting finds in this data. Check out the source code on GitHub.

Introduction to Shiny Slides

Sat, 09 Feb 2013 15:58:33 +0000

Below you can download the slides presented on February 9, 2013 at the Dallas R Users Group on RStudio’s Shiny framework which allows you to create interactive web applications in R. No knowledge of HTML or JavaScript is required!

I introduce the alternatives and history of R programming on the web, the reactive programming model, and get into some sample applications.

Adobe PDF Slides

PowerPoint Slides

All sources used are available on GitHub: https://github.com/trestletech/shiny-sandbox

Dallas R Users: Learn Shiny this Saturday, 2/9

Wed, 06 Feb 2013 00:14:02 +0000

Just a heads-up for any R users in the Dallas/Fort Worth Metroplex: I’ll be presenting at the Dallas R Users Group this Saturday, 2/9/2013 at 10:00AM at the University of Dallas (1845 East Northgate Drive, Irving, TX). I’ll be talking about how to use RStudio’s new Shiny framework to create R-powered web applications.

For the uninitiated, Shiny makes it trivial to create web applications in R using only R code — I’ll presume no knowledge of HTML, JavaScript, or any other web development tools. If you’re looking to share some of your R code publicly, Shiny may be a good solution to easily get your application into a browser and in front of your users. A couple of examples of what I’ve been doing in R include:

Reconstruct Gene Regulatory Networks (Shiny + D3) – full post here.
Color Palettes in Interactive 3D (Shiny + RGL) – full post here.

For more information and directions/parking instructions, checkout the Meetup page: http://www.meetup.com/Dallas-R-Users-Group/events/101898022/ . I’ll be sure to post the slides online afterwards for those interested.

See you there!

Interview with DecisionStats

Sat, 02 Feb 2013 17:04:50 +0000

I was contacted by Ajay from DecisionStats for an interview about my work in R. I share some details about my background in R and the future of Trestle.

Read the interview here!

Color Palettes in Interactive 3D (Shiny+RGL)

Tue, 22 Jan 2013 17:30:46 +0000

You can view the application online at http://trestletech.com:3838/rgl/!

_(Bear with us as we try out the very early release of Shiny Server. There may be some downtime and manual restarts of the service required in the next couple of weeks. Please drop us a line if it’s not working as you expected.)

This is an adaptation of some previous analysis exploring the progression of color palettes through three-dimensional RGB space. Thanks to RStudio’s wonderful new Shiny application, this analysis can now be made more interactive. You can now explore all of the color palettes available in RColorBrewer and optionally even calculate/display the principal components for each of those palettes.

As always, the source is up on Github.

Acknowledgements

All analysis done in R
Interactive 3D graphics generated using rgl version 0.92.879
Code hosted by GitHub in trestletech/shiny-sandbox/
Color palettes obtained from Cynthia Brewer’s ColorBrewer2.0 service

Reconstruct Gene Networks Using Shiny

Thu, 06 Dec 2012 05:44:39 +0000

Check out the application! http://trestletech.com:3838/grn/

We’ve been experimenting with RStudio’s new Shiny software as a way to quickly and easily create interactive, responsive web applications which are able to leverage complicated analytics back-ends built in the R programming language.

We created a simple interface which can infer the structure of an underlying Gene Regulatory information based on gene expression patterns; the application is available at http://trestletech.com:3838/grn/. (At the time of writing, Shiny’s file upload functionality is highly unstable and may not work from your machine — hopefully improvements to the project will resolve the issues shortly.)

Technical Details

I’m using Shiny/Glimmer to handle all web traffic, and ENA to wrap GeneNet to do the actual network inference (more methods could easily be added later). I used D3’s Force Layout to provide some interactive network visualization — I don’t know that it’s really the most practical way to view the data, but it’s some fun eye-candy.

I had stumbled through a few un-documented features of Shiny (app-specific www directories, JS to handle custom Shiny outputs, fileInput() for uploading data, etc.) — if you find yourself in the same boat, all my source is available on Github here if anyone’s interested.

I had created a custom reactive function in Shiny to handle an adjacency matrix (the preferred format for the ENA package), which will translate the adjacency matrix into an adjacency list which can be properly handled by RJSONIO, which is how Shiny transfers data. I think created the client-side portion in Javascript which is able to parse that JSON and display it in a D3 Force-Directed Graph.

Install RGL in Ubuntu

Sun, 14 Oct 2012 10:10:28 +0000

I had a few hurdles to get through when trying to install the “rgl” R package in Ubuntu. In summary, here was my solution:

From the terminal:

sudo apt-get install r-cran-dev xorg-dev libglu1-mesa-dev

Then from R:

install.packages("rgl")

Detailed Log

Thanks to Dirk Eddelbuettel for the following tip. He points out that, rather than all the guess-and-check below, he points out:

If you want to install from newer sources, ‘sudo apt-get build-dep r-cran-rgl’ will install all dependencies needed to builds the package saving you the guesswork.

I’ll be sure to do that next time! For posterity’s sake, here was the detailed log of my first attempt.

The first snag I hit was the following implying I was missing some necessary X code:

checking for X... no
configure: error: X11 not found but required, configure aborted.
ERROR: configuration failed for package ‘rgl’

You’ll need the xorg-dev package to get around this issue. I then hit an error regarding a GL library:

checking for X... libraries , headers
checking GL/gl.h usability... no
checking GL/gl.h presence... no
checking for GL/gl.h... no
checking GL/glu.h usability... no
checking GL/glu.h presence... no
checking for GL/glu.h... no
configure: error: missing required header GL/gl.h
ERROR: configuration failed for package ‘rgl’

A quick Google showed that the libglu1-mesa-dev package contained these headers. Once I had installed those package, the install process went smoothly!

Note, too, that there may be a precompiled package may be available for your distribution under the name r-cran-rgl. In my case, I needed a more recent version of the package than was offered there, hence this exploration.

Color Palettes in HCL Space

Fri, 12 Oct 2012 17:38:41 +0000

This is a quick follow-up to my previous post about Color Palettes in RGB Space. Achim Zeileis had commented that, perhaps, it would be more informative to evaluate the color palettes in HCL (polar LUV) space, as that spectrum more accurately describes how humans perceive color. Perhaps more clear trends would emerge in HCL space, or color palettes would more closely hug their principal component. For the uninitiated, a good introduction to HCL color spaces is available at this site, or from Mr. Zeileis' own paper here. We'll start by loading in some code written previously (or slightly modified to support HCL). We can plot out an image of the different color palettes we're evaluating once again.

source("../Correlation.R")
source("loadPalettes.R")
par(mfrow = c(6, 3), mar = c(2, 1, 1, 1))
for (i in 1:18) {
    palette <- getSequentialLuv(i, allSequential)
    image(mat, col = hex(palette), axes = FALSE, main = i)
}

HCL-Space The fundamental question for this analysis was how these color palettes move through HCL space, as opposed to RGB space, which was considered last time.

You must enable Javascript to view this page properly.

An interactive visualization of palette #2 in 3-Dimensional HCL space

R² Values As we did in the previous analysis, we can use the principal component of each palette, to compute the R

² value to quantify how well the data aligns to this component.

You must enable Javascript to view this page properly.

Palette #2 with the Principal Component.

We'll recycle some of the old functions, and create a new one that calculates the R

² values in HCL space.

#' Compute the proportion of variance accounted for by the given number of components
#'
#' @author Jeffrey D. Allen email{Jeffrey.Allen@@UTSouthwestern.edu}
propVar <- function(pca, components=1){
  #proportion of variance explained by the first component
  pv <- pca$sdev^2/sum(pca$sdev^2)[1] 
  return(sum(pv[1:components]))
}

#' Calculate the R-squared values of all 18 color palettes
#'
#' @author Jeffrey D. Allen email{Jeffrey.Allen@@UTSouthwestern.edu}
calcR2Sequential <- function(){
  library(rgl)
  R2 <- list()
  for (i in which(sequential[,2] == 9)){
    palette <- RGBToLuv(sequential[i:(i+8), 7:9])
    pca <-plotLuvCols(palette[,"L"],palette[,"C"], palette[,"H"], pca=1)    
    R2[[length(R2)+1]] <- propVar(pca,1)
  }
  return(R2)
}

calcR2RGBSequential <- function() {
    library(rgl)
    R2 <- list()
    for (i in which(sequential[, 2] == 9)) {
        palette <- sequential[i:(i + 8), 7:9]
        pca <- plotCols(palette$R, palette$G, palette$B, pca = 1)
        cat(i, ": ", propVar(pca, 1), "n")
        R2[[length(R2) + 1]] <- propVar(pca, 1)
    }
    return(R2)
}

Comparison of R² Values Let's compare the R

² values computed from RGB space to those computed in HCL space

# R2 Values in HCL Space
r2 <- calcR2Sequential()
r2 <- unlist(r2)
names(r2) <- 1:18

# R2 Values in RGB Space
rgbr2 <- calcR2RGBSequential()

## 34 :  0.981 
## 76 :  0.9097 
## 118 :  0.9541 
## 160 :  0.9847 
## 202 :  0.9552 
## 244 :  0.9663 
## 286 :  0.9674 
## 328 :  0.9273 
## 370 :  0.9344 
## 412 :  0.9593 
## 454 :  0.9049 
## 496 :  0.9007 
## 538 :  0.9954 
## 580 :  0.9752 
## 622 :  0.9846 
## 664 :  0.9292 
## 706 :  0.9311 
## 748 :  1

rgbr2 <- unlist(rgbr2)
names(rgbr2) <- 1:18

# plot the comparison
plot(rgbr2 ~ r2, ylab = "RGB R2 Values", xlab = "HCL R2 Values", main = "RGB vs. HCL R2 Values")

pv <- anova(lm(rgbr2 ~ r2))$"Pr(>F)"[1]

As seen clearly in the plot, these two variables are not correlated (p-value of

0.5232), so there's definitely a difference in how these palettes move through HCL vs. RGB space.

Color Ratings One hypothesis of this analysis was that because HCL space corresponds more directly to our perception of color, perhaps a smoother or more linear path through HCL space would have greater consequence on the visual appeal of the color palette than it would in RGB space. To test this, we can repeat the same analysis as we had before to see if a small R

² value is significantly correlated with the visual appeal of a color palette.

colorPreference <- read.csv("../turk/output/Batch_790445_batch_results.csv", 
    header = TRUE, stringsAsFactors = FALSE)
colorPreference <- colorPreference[, 28:29]
colorPreference[, 1] <- substr(colorPreference[, 1], 44, 100)
colorPreference[, 1] <- substr(colorPreference[, 1], 0, nchar(colorPreference[, 
    1]) - 4)
colnames(colorPreference) <- c("palette", "rating")

prefList <- split(colorPreference[, 2], colorPreference[, 1])
prefList <- prefList[order(as.integer(names(prefList)))]

R² Values We can calculate the R

² values for each palette as previously discussed and compare to see if it's associated with the aesthetic appeal of a palette.

r2

##      1      2      3      4      5      6      7      8      9     10 
## 0.9614 0.9794 0.9799 0.9388 0.9708 0.8902 0.9651 0.9579 0.9725 0.9842 
##     11     12     13     14     15     16     17     18 
## 0.9325 0.9129 0.9603 0.9358 0.9568 0.9611 0.9785 1.0000

plot(avgs ~ r2, main = "Linearity of Color Palette vs. Aesthetic Appeal", xlab = "R-squared Value", 
    ylab = "Average Aesthetic Rating")
abline(lm(avgs ~ r2), col = 4)

pv <- anova(lm(avgs ~ r2))$"Pr(>F)"[1]

Oddly, this result contradicts the previous result; it seems that adhering to a linear path through HCL space is actually

inversely related with the aesthetic appeal on these palettes. We should note, of course, that the p-value for this correlation is not significant, as it stands (p-value = 0.6598). Regardless, the negative trend seems fairly obvious on inspection and, if you were to exclude the two "outlier" palettes on the left side which have stark breaks in HCL space on either end of the spectrum, you get a significant (0.01) negative correlation. Indeed, these same two palettes were the same outliers in the previous plot comparing RGB to HCL R² values. Obviously, as it stands, the conclusion of the analysis is that there is no correlation in these palettes between linearity in HCL space and aesthetic appeal, but it is curious that the hinted correlation is actually negative.

Dimensional Spread Another point of interest is the "spread" or coverage of a palette across one particular axis. For instance, it may be interesting to compare a color palette which varies only in luminance to one which varies only in chroma to see if one type of progression is more aesthetically appealing. We can summarize such a phenomenon by calculating the distance between each color in a palette along a given axis (luminosity, for instance), which we'll label "ΔLuminosity" the . Extracting the median of these points may provide some insight into the movement of a particular palette along an axis.

plotDimensionalScoring <- function(palettes, scoring, ...) {

    ranges <- data.frame(L = numeric(0), C = numeric(0), H = numeric(0))

    for (i in 1:length(palettes)) {
        colPal <- RGBToLuv(palettes[[i]])

        # get the differences between each color on each axis
        colPal <- diff(colPal)
        colPal <- abs(apply(colPal, 2, median))

        ranges[i, ] <- colPal

    }

    plot3d(0, 0, 0, xlab = "L", ylab = "C", zlab = "H", type = "n", xlim = c(-0.5, 
        max(ranges[, 1] + 1)), ylim = c(-0.5, max(ranges[, 2] + 1)), zlim = c(-0.5, 
        max(ranges[, 3] + 1)), ...)

    cols <- heat_hcl(n = 101)

    for (i in 1:nrow(ranges)) {
        plot3d(ranges[i, 1], ranges[i, 2], ranges[i, 3], type = "p", add = TRUE, 
            pch = 19, size = 10, col = cols[101 - (round(scoring[i], 2) * 100)], 
            ...)
    }
    ranges
}

normAvgs <- avgs - min(avgs)
normAvgs <- normAvgs/max(normAvgs)
ranges <- plotDimensionalScoring(allSequential, normAvgs)

Each palette now has one median Δ value for each axis. We can plot these in 3D space to see where the palettes fall. We'll go ahead and color-code these based on their average visual appeal to see if we can observe any trends or "hotspots" in which palettes are consistently rated as visually appealing based only on their median movement along an axis.

You must enable Javascript to view this page properly.

A plot of all color palettes' median change in HCL space.

We're descending into fairly non-scientific analysis here, but I noticed a pattern when viewing these points along the chroma and luminosity axes. It seems like there's something of a pattern resulting in consistently positioned high-ranked palettes. I interpolated a heatmap based on these ratings on the C and L axes.

library(akima)

resolution = 64
cSeq <- seq(min(ranges[, 2]), max(ranges[, 2]), length.out = resolution)
lSeq <- seq(min(ranges[, 1]), max(ranges[, 1]), length.out = resolution)

a <- interp(x = ranges[, 1], y = ranges[, 2], z = normAvgs, xo = lSeq, yo = cSeq, 
    duplicate = "mean")

filled.contour(a, color.palette = heat_hcl, xlab = "Luminance", ylab = "Chroma", 
    main = "Visual Appeal Across Chroma & Luminance")

maxL <- lSeq[which.max(apply(a$z, 2, max, na.rm = TRUE))]

## Warning: no non-missing arguments to max; returning -Inf

maxC <- cSeq[which.max(apply(a$z, 1, max, na.rm = TRUE))]

## Warning: no non-missing arguments to max; returning -Inf

You can see the peak of visual appeal at luminance =

6.374 and chroma = 8.332. A few examples of palettes using this "optimized" chroma and luminance spacing...

par(mfrow = c(6, 3), mar = c(2, 1, 1, 1))
for (i in 1:18) {
    barplot(rep(1, 9), col = sequential_hcl(9, h = (20 * i), c. = c(100, 33.6), 
        l = c(40, 84.59), power = 1))
}

Individual Axis Analysis There is one palette that seems to be a bit of an outlier in most regards. Of all the palettes, the greyscale palette has the lightest L value, and the lowest C and H values. If you calculate the Z-scores on each color axis, this palette has consistently higher scores; below is a plot of the average Z-score across the 3 axes.

barplot(apply(abs(scale(ranges)), 1, mean), col = c(rep("#BBBBBB", 17), "#CC7777"), 
    xlab = "Palette #", ylab = "Average Z-score Across {H, C, L} Axes")

Because of this, I decided to exclude it from all further calculations of significance. Where appropriate, I'll still plot it graphically but distinguish it from the other palettes. Rather than relying on a 3D plot, we can examine each axis individually to see if there's a significant association between a palette's median variation on that access and its visual appeal. (The greyscale plot will be plotted with a triangle symbol in these plots, and was excluded to p-value and regression calculations).

filRanges <- ranges[1:17, ]
filNormAvgs <- normAvgs[1:17]

plot(normAvgs ~ ranges[, "H"], pch = c(rep(1, 17), 2), main = "Hue vs. Visual Appeal", 
    xlab = expression("Median " * Delta * Hue), ylab = "Average Aesthetic Rating")
abline(lm(filNormAvgs ~ filRanges[, "H"]), col = 2)
pvH <- anova(lm(filNormAvgs ~ filRanges[, "H"]))$"Pr(>F)"[1]
text(3, 0.1, paste("p-value =", round(pvH, 3)))

plot(normAvgs ~ ranges[, "C"], pch = c(rep(1, 17), 2), main = "Chroma vs. Visual Appeal", 
    xlab = expression("Median " * Delta * Chroma), ylab = "Average Aesthetic Rating")
abline(lm(filNormAvgs ~ filRanges[, "C"]), col = 2)
pvC <- anova(lm(filNormAvgs ~ filRanges[, "C"]))$"Pr(>F)"[1]
text(3, 0.1, paste("p-value =", round(pvC, 3)))

plot(normAvgs ~ ranges[, "L"], pch = c(rep(1, 17), 2), main = "Luminance vs. Visual Appeal", 
    xlab = expression("Median " * Delta * Luminance), ylab = "Average Aesthetic Rating")
abline(lm(filNormAvgs ~ filRanges[, "L"]), col = 2)
pvL <- anova(lm(filNormAvgs ~ filRanges[, "L"]))$"Pr(>F)"[1]
text(5, 0.1, paste("p-value =", round(pvL, 3)))

As shown on the plots, there is no observable correlation with regards to movement through hue with visual appeal; there is a significant negative correlation between median variation in chroma and the visual appeal; and there seems to be a negative (though non-significant) correlation between median variation in luminance and visual appeal.

Conclusion

Comparison to Colorspace Palettes It was mentioned that colorspace generates their palettes in HCL space, so the trends should more obviously emerge for such palettes. As it turns out, the palettes in HCL not necessarily linear, even in HCL space. Many palettes are fairly linear on at least one axis, but often have at least one color at either end which is completely non-linear, causing the R2 values to often be lower than they were in colorbrewer palettes. For instance:

csp <- as(hex2RGB(sequential_hcl(n = 9)), "polarLUV")@coords
pca <- plotLuvCols(csp[, "L"], csp[, "C"], csp[, "H"], 1)

You must enable Javascript to view this page properly.

A plot of the "sequential_hcl" color palette from the colorspace package.

After inspecting the code, the hue value in this color palette never changes. So it seems that we're encountering a substantial rounding error when you get the the dark/light edges of the color palette.

Summary I'm still a bit perplexed by the results I observer, but I suppose this analysis hints at a few things. First of all, I realized that it very much matters which color space you're working in when designing color palettes. As the lack of a correlation between the R

² values from RGB space and HCL space show, the movement of different palettes through these spaces are drastically different and don't seem to be correlated. Second, it looks like there is some potential to optimize the spacing of colors in a given palette by finding these "hotspots" in the Luminance vs. Chroma plot. On the other hand, movement along the hue axis for a given palette doesn't seem to have much of an effect on the visual appeal. Finally, it seems that the rounding error on hex codes and 255-value RGB values becomes significant for especially dark or bright ends of a palette. This may put into question the effectiveness of such analysis in 3D space. Of course, the hue value represents a complete circle, so a hue of 359 is very close to a hue of 0, which wouldn't be captured in a naive 3D analysis. Even accounting for this problem, there are other nuances of this color space which can't be captured in 3D space, and may jeopardize the effectiveness of PCA in such a space.

Future Work As mentioned previously, it will be interesting to analyze not just the visual appeal of a color palette, but the effectiveness of a palette at communicating information. Hopefully the next post will be able to capture some of that information. As always, comments are welcome! I'm well outside the scope of my training and expertise, so I'd be happy to hear any critiques or concerns about methodology.

Acknowledgements

All analysis done in R
All ratings generated using Amazon Mechanical Turk
Interactive 3D graphics generated using rgl version 0.92.879
Report generated using knitr
Code hosted by GitHub in trestletech/RGB-Space
Color palettes obtained from Cynthia Brewer's ColorBrewer2.0 service
Idea from Achim Zeileis

Color Palettes in RGB Space

Wed, 20 Jun 2012 07:27:54 +0000

Introduction I've recently been interested in how to communicate information using color. I don't know much about the field of Color Theory, but it's an interesting topic to me. The selection of color palettes, in particular, has been a topic I've been faced with lately.

I downloaded 18 different sequential color palettes from Cynthia Brewer's ColorBrewer2 website to use as suggested color palettes. I was struck by the various movements through the spectrum of some of these palettes and wanted to poke around at quantifying some of that movement. This is the result of that analysis.

Using the 18 different sequential color palettes, I generated heatmaps to try to gauge the aesthetic appeal of each palette. Using Amazon's Mechanical Turk, I was able to ask workers to rate a palette on a scale from 1 to 5. I had each palette rated 20 times to generate a sufficient amount of data to start determining statistical significance. The 360 ratings were completed by 28 workers in a couple of hours.

This initial study didn't consider anything about how clear the heatmaps are, only how aesthetically pleasing they are. The 18 matrices in the 18 available palettes are displayed below.


  source("../Correlation.R")
  source("loadPalettes.R")
  par(mfrow = c(6, 3), mar = c(2, 1, 1, 1))
  for (i in 1:18) {
      palette <- allSequential[[i]]
      image(mat, col = rgb(palette/255), axes = FALSE, main = i)
  }

RGB-Space One question I had was about the motion of a "visually attractive" color palette through the 3-dimensional space of all RGB values. My assumption was that most palettes can be represented as a straight line through

this space.

You must enable Javascript to view this page properly.

An interactive visualization of palette #2 in 3-Dimensional RGB space

R² Values One way to quantify this is to calculate the principal component of each palette, which represents the line in RGB space which best fits the palette. The R

² value can then be used to quantify how well the data aligns to this component. An R² value of 1 indicates that the palette aligns perfectly to a straight line through RGB space.

You must enable Javascript to view this page properly.

Palette #2 with the Principal Component. Note the curve of the palette around the component.


  #' Compute the proportion of variance accounted for by the given number of
  #' components
  #' 
  #' @author Jeffrey D. Allen \email{jeff.allen@@trestletech.com}
  propVar <- function(pca, components = 1) {
      # proportion of variance explained by the first component
      pv <- pca$sdev^2/sum(pca$sdev^2)[1]
      return(sum(pv[1:components]))
  }
  
  #' Calculate the R-squared values of all 18 color palettes
  #' 
  #' @author Jeffrey D. Allen \email{jeff.allen@@trestletech.com}
  calcR2Sequential <- function() {
      library(rgl)
      R2 <- list()
      for (i in which(sequential[, 2] == 9)) {
          palette <- sequential[i:(i + 8), 7:9]
          pca <- plotCols(palette$R, palette$G, palette$B, pca = 1)
          cat(i, ": ", propVar(pca, 1), "\n")
          R2[[length(R2) + 1]] <- propVar(pca, 1)
      }
      return(R2)
  }

Path Length An alternative way to consider the palette in RGB space is to consider the "length" of the palette through RGB space by simply calculating the distance between each color represented as a point in RGB space. My thought is that this may better capture the "movement" of a palette around this space. The R

² value doesn't encompass any notion of how much space is covered by a palette, but only the arrangement of the colors relative to their principal component.


  #' Calculate the path length for all sequential palettes.
  #' 
  #' @author Jeffrey D. Allen \email{jeff.allen@@trestletech.com}
  calcPathLength <- function() {
      plen <- array(dim = sum(sequential[, 2] == 9, na.rm = TRUE))
      p <- 1
      for (i in which(sequential[, 2] == 9)) {
          palette <- sequential[i:(i + 8), 7:9]
          cat(i, ": ", getPathLength(palette), "\n")
          plen[p] <- getPathLength(palette)
          p <- p + 1
      }
      return(plen)
  }
  
  #' Calculate the length of a path through RGB space of a given palette.
  #' 
  #' Sums the distance from all adjacent colors.  @author Jeffrey D. Allen
  #' \email{jeff.allen@@trestletech.com}
  getPathLength <- function(palette) {
      pd <- apply(palette, 2, diff)
      pl <- sqrt(apply(pd^2, 1, sum))
      return(sum(pl))
  }

Comparison Let's compare the R

² values to the path length values


  r2 <- calcR2Sequential()
  



  ## 34 :  0.981 
  ## 76 :  0.9097 
  ## 118 :  0.9541 
  ## 160 :  0.9847 
  ## 202 :  0.9552 
  ## 244 :  0.9663 
  ## 286 :  0.9674 
  ## 328 :  0.9273 
  ## 370 :  0.9344 
  ## 412 :  0.9593 
  ## 454 :  0.9049 
  ## 496 :  0.9007 
  ## 538 :  0.9954 
  ## 580 :  0.9752 
  ## 622 :  0.9846 
  ## 664 :  0.9292 
  ## 706 :  0.9311 
  ## 748 :  1 
  



  r2 <- unlist(r2)
  names(r2) <- 1:18
  
  pl <- calcPathLength()
  



  ## 34 :  404.8 
  ## 76 :  455.2 
  ## 118 :  377.5 
  ## 160 :  407.5 
  ## 202 :  418.1 
  ## 244 :  400.5 
  ## 286 :  389.5 
  ## 328 :  405.6 
  ## 370 :  430 
  ## 412 :  420.7 
  ## 454 :  424.7 
  ## 496 :  430 
  ## 538 :  342.9 
  ## 580 :  374.1 
  ## 622 :  400.2 
  ## 664 :  400.3 
  ## 706 :  430.6 
  ## 748 :  441.7 
  



  plot(r2 ~ pl, main = "Path Length vs. R-Squared Value", xlab = "Path Length",
      ylab = "R-Squared Value")
  abline(lm(r2 ~ pl), col = 2)
  






  pv <- anova(lm(r2 ~ pl))$"Pr(>F)"[1]

The p-value (

0.0318) is significant in the negative correlation between the two variables, meaning that, as expected, the closer the palette stays to its principal component, the shorter the path through RGB space is (on average). So either scheme could be used to quantify a palette's non-linearity.

Color Ratings The output of the Mechanical Turk trial is available in a stored file in this project. We'll read it in and filter out the peripheral information.


  colorPreference <- read.csv("../turk/output/Batch_790445_batch_results.csv",
      header = TRUE, stringsAsFactors = FALSE)
  colorPreference <- colorPreference[, 28:29]
  colorPreference[, 1] <- substr(colorPreference[, 1], 44, 100)
  colorPreference[, 1] <- substr(colorPreference[, 1], , nchar(colorPreference[,
      1]) - 4)
  colnames(colorPreference) <- c("palette", "rating")
  
  prefList <- split(colorPreference[, 2], colorPreference[, 1])
  prefList <- prefList[order(as.integer(names(prefList)))]

We can then visualize the results, as well.


  boxplot(prefList, main = "Ratings of Color Palettes", xlab = "Palette Number",
      ylab = "Rating")
  






  avgs <- sapply(prefList, mean)

We can then check to see if there's a significant association between the palette and the rating of the palette, or if we've just got noise.


  fit <- anova(lm(colorPreference[, 2] ~ as.factor(colorPreference[,
      1])))
  pv <- (fit$"Pr(>F)")[1]

With a p-value of

1.6911 × 10-4, you can see that there is a significant difference between the different palettes.

We can list out the most attractive palettes in order, as well:


  sort(sapply(prefList, mean), decreasing = TRUE)
  



  ##   14   13    4    6    3   15    1    7   18    5    8    2   16    9   10 
  ## 4.30 4.10 3.85 3.75 3.65 3.65 3.55 3.45 3.45 3.40 3.40 3.35 3.15 3.10 3.10 
  ##   17   12   11 
  ## 2.95 2.85 2.65

So the palettes, in order of visual appeal are:


  par(mfrow = c(6, 3), mar = c(2, 1, 1, 1))
  for (i in order(avgs, decreasing = TRUE)) {
      palette <- allSequential[[i]]
      image(mat, col = rgb(palette/255), axes = FALSE, main = i)
  }

Warm Palettes The first thing I noticed was that the cooler palettes were rated more highly than the warmer palettes. To try to quantify this, we can plot out the "redness" (the strenght of the red channel in each palette) against the average rating.


  redness <- apply(sapply(allSequential, "[[", "R"), 2, mean)
  plot(avgs ~ redness, main = "Warmth of Palette vs. Aesthetic Appeal",
      xlab = "\"Redness\"", ylab = "Average Aesthetic Rating")
  abline(lm(avgs ~ redness), col = 2)
  






  pv <- anova(lm(avgs ~ redness))$"Pr(>F)"[1]

Indeed, the p-value of this correlation is significant for this data (

3.6602 × 10-4) indicating that -- among these palettes and in this context -- cooler palettes are more visually appealing.

R² Values We can calculate the R

² values for each palette as previously discussed and compare to see if it's associated with the aesthetic appeal of a palette.


  r2
  



  ##      1      2      3      4      5      6      7      8      9     10 
  ## 0.9810 0.9097 0.9541 0.9847 0.9552 0.9663 0.9674 0.9273 0.9344 0.9593 
  ##     11     12     13     14     15     16     17     18 
  ## 0.9049 0.9007 0.9954 0.9752 0.9846 0.9292 0.9311 1.0000 
  



  plot(avgs ~ r2, main = "Linearity of Color Palette vs. Aesthetic Appeal",
      xlab = "R-squared Value", ylab = "Average Aesthetic Rating")
  abline(lm(avgs ~ r2), col = 4)
  






  pv <- anova(lm(avgs ~ r2))$"Pr(>F)"[1]

Again, the p-value of this association is significant (

3.2224 × 10-4). So it seems that adhering a color spectrum to a straight line through RGB-space is visually appealing.

Similarly, for the path length, the p-value is significant (though not as strongly as with the R² values).


  anova(lm(avgs ~ pl))$"Pr(>F)"[1]
  



  ## [1] 0.001476

Summary This analysis answered a couple of questions for me. First, it showed that, in general, linear paths through RGB space create more aesthetically pleasing color palettes. Second, it demonstrated that, in this narrow study, palettes with cooler color schemes were preferred as more "aesthetically pleasing." Finally, it gave some concrete recommendations regarding which of the available color palettes to use if the goal is purely aesthetic.

Future Work Of course, aesthetics are not to only goal behind color palette selection. Generally, the goal of heatmaps such as these is to convey information. If no legend is given, we're hoping to convey relative "strengths" of some phenomena to the viewer. If a legend is given, we're additionally hoping to support some quantification to these data, as well. So merely determining which color palettes are best to look at will likely not be the most important consideration in determining which palettes to use. We should do further analysis to determine which palettes convey such information most efficiently, and then likely make some compromise between efficient communication and aesthetics, depending on the application.

Acknowledgements

All analysis done in R
All ratings generated using Amazon Mechanical Turk
Interactive 3D graphics generated using rgl version 0.92.879
Report generated using knitr
Code hosted by GitHub in trestletech/RGB-Space
Color palettes obtained from Cynthia Brewer's ColorBrewer2.0 service

Sermon Sentiment Analysis

Tue, 22 Nov 2011 01:15:52 +0000

Matt Chandler vs. Mark Driscoll

I came across an interesting API from Viral Heat which is capable of “Sentiment Analysis.” This analysis is designed to capture the sentiment of a statement by ranking it on a scale from -1 to 1. For instance, a chipper sentence like “The smell of roses makes me giddy!” is rated a solid 0.82 (a very positive score), while a downer such as “She was distraught by the genocide” gets rated a very negative -0.91. Of course, there are many blips which get rated incorrectly (at least in my mind), but there seems to be some truth underneath the noise. This same sentiment analysis engine was used by OpenBible.info to map out the sentiment of the entire Bible.

I was intrigued by the idea and decided to play with it a bit myself. I sought out some repositories of transcripts (speeches, scripts, etc.) but after a bit of searching, I wasn’t able to find a repository of speeches by single speakers; I settled on using sermon transcripts — at least for some initial experimentation. I grabbed a couple dozen transcripts from Matt Chandler of

The Village Church and Mark Driscoll of Mars Hill Church and got to work. I ended up downloading 19 sermons from Matt covering multiple years from 2004 – 2010 and 7 of Mark’s recent sermons (all 2011).

First I was interested in looking at individual sermons to see the pattern or trends in sentiment over the course of a single sermon. I then looked at which words were most likely to be found in positive-sentiment sentences and which were likely to be found in negative ones. Here are a couple of examples:

Next, I investigated which words were used most commonly by each pastor (after removing common words like “it,”me,” etc. I observed the following list, showing the word, and the average percentage of the time that word is used:

Commonly Used By Both

Jesus(3.2%)
God (2.6%)
People (1.5%)
Love (0.87%)
Life (0.82%)
Bible (0.72%)
Church (0.64%)
Time (0.57%)
World (0.51%)
Look (0.49%)
Tell (0.45%)
Day (0.42%)

Frequently Used by Matt

Little (0.64%)
Christ (0.60%)
Pray (0.50%)
Heart (0.44%)
Lot (0.38%)
Verse (0.36%)
Week (0.36%)
Maybe (0.35%)

Frequently Used by Mark*

Peter (0.81%)
Luke (0.71%)
Judas (0.69%)
Serve (0.54%)
Sin (0.51%)
Book (0.47%)
Times (0.47%)
Temple (0.46%)

*I’m guessing many of these were present only because he was recently in a series on Luke. A bigger sample may change these numbers significantly.

We can also look at how the sentiment of an average sermon for each speaker progresses by aggregating the trends across all of the sermons covered in this analysis. Below are the results with the average trend highlighted in blue.

You may notice that Mark’s trendline seems to be higher than Matt’s, implying that, on average, Mark has a more positive sentiment throughout his sermon. Indeed, it appears that there is a statistically significant difference (p=0.027, excluding the selection bias) between the two speakers’ sentiments, with Mark’s average sermon having a serntiment of 0.197 and Matt having one of 0.096, as shown below.

Feel free to comment if you have any thoughts/critiques or any ideas for further analysis!

Edit: I added my preliminary code on GitHub: https://github.com/trestletech/Sermon-Sentiment-Analysis

Matchbox 2.1 Released

Tue, 09 Aug 2011 23:17:50 +0000

We just launched MatchBox, one of our initial internal projects for public consumption. MatchBox is a project that aims to understand patterns and rhythms in your typing habits. This release is a Firefox Add-On that keeps starts learning your habits for you to use later. Once we’ve collected some data, we’ll be able to do some interesting analysis such as adding a layer of security to your browser based on whether or not the person typing matches your patterns, or show you a realistic “Words-Per-Minute” count, or show you how you compare to other typists of your gender/age.

You can view more information about the project here, or go straight to the download page.

New: Web Hosting

Sun, 17 Apr 2011 23:05:08 +0000

Attention to our web development customers:

We’re now offering our own web hosting services! By integrating web hosting into our offerings, we can now offer you a single point of contact for all of your website needs. No more need to keep up with your logins and payments with some other web hosting company. Most importantly, no more need to worry while making changes on those stressful configuration pages. The highlights of this new model include:

A personal representative to manage any change you need to make to your web hosting setup — no need to guess about how to setup your website or lookup confusing acronyms.
Faster equipment on the back-end gives you faster page load times. Your visitors will spend less time waiting on the page to load, and more time looking at your content.
Comparable pricing to what you’d pay for a slower, impersonal hosting service!

You can get more information at our Web Hosting page, or checkout our Prices.

The Beginning

Mon, 17 May 2010 22:50:25 +0000

Trestle Technology, LLC was formed in May of 2010.

Product-specific developments will be announced as they occur. Subscribe to this site to ensure you always have the latest news on Trestle.

Trestle Technology, LLC

On Being Reliable

Organization

Discipline

Conclusion

Announcing the Plumber v0.4.4 Release

Plumber v0.4.4 Highlights

Known Bugs

Preparing for the Plumber v0.4.0 Release

Plumber v0.4.0 Migration Guide

EigenCoder: Programming Stereotypes

Approach

Data

Gender

Age

Smiles

Facial Hair

Conclusion

plumber — Convert R Code to a Web API

plumber Magic

Demos

Behind The Scenes

Summary

Todo.txt++ Analysis

Conclusion

shinyStore – Persistent Client-Side Storage in Shiny

shinyTree: jsTree + shiny

Ace Code Editor in Shiny (shinyAce)

Interactive 3D in Shiny (shinyRGL)

Reproducing R: Scripts, Documents, and Packages

Dallas R Users: Creating R Packages this Saturday, 6/29

Package-Wide Variables/Cache in R Packages

Example Package – rev. a804

Example Package – rev. a81e

Special Assignment – rev. 5548

Environments – rev. 320e

Conclusion

Graphical Tools (rgl) on a Headless Shiny Server

Installation

Automation & Configuration

Analysis of Public .Rhistory Files

Load Data

Package Usage

Function Use

Conclusion

Introduction to Shiny Slides

Dallas R Users: Learn Shiny this Saturday, 2/9

Interview with DecisionStats

Color Palettes in Interactive 3D (Shiny+RGL)

Acknowledgements

Reconstruct Gene Networks Using Shiny

Technical Details

Install RGL in Ubuntu

Detailed Log

Color Palettes in HCL Space

HCL-Space The fundamental question for this analysis was how these color palettes move through HCL space, as opposed to RGB space, which was considered last time.

R2 Values As we did in the previous analysis, we can use the principal component of each palette, to compute the R

Comparison of R2 Values Let's compare the R

R2 Values We can calculate the R

Conclusion

Summary I'm still a bit perplexed by the results I observer, but I suppose this analysis hints at a few things. First of all, I realized that it very much matters which color space you're working in when designing color palettes. As the lack of a correlation between the R

Acknowledgements

Color Palettes in RGB Space

Introduction I've recently been interested in how to communicate information using color. I don't know much about the field of Color Theory, but it's an interesting topic to me. The selection of color palettes, in particular, has been a topic I've been faced with lately.

RGB-Space One question I had was about the motion of a "visually attractive" color palette through the 3-dimensional space of all RGB values. My assumption was that most palettes can be represented as a straight line through

R2 Values One way to quantify this is to calculate the principal component of each palette, which represents the line in RGB space which best fits the palette. The R

Comparison Let's compare the R

Color Ratings The output of the Mechanical Turk trial is available in a stored file in this project. We'll read it in and filter out the peripheral information.

Warm Palettes The first thing I noticed was that the cooler palettes were rated more highly than the warmer palettes. To try to quantify this, we can plot out the "redness" (the strenght of the red channel in each palette) against the average rating.

R2 Values We can calculate the R

Acknowledgements

Sermon Sentiment Analysis

Matt Chandler vs. Mark Driscoll

Commonly Used By Both

Frequently Used by Matt

Frequently Used by Mark*

Matchbox 2.1 Released

New: Web Hosting

The Beginning

R² Values As we did in the previous analysis, we can use the principal component of each palette, to compute the R

Comparison of R² Values Let's compare the R

R² Values We can calculate the R

R² Values One way to quantify this is to calculate the principal component of each palette, which represents the line in RGB space which best fits the palette. The R

R² Values We can calculate the R