Demian Neidetcher

Book Review Functional Programming For Java Developers

2013-05-14T00:00:00-05:00

First off, this is a small book. 72 pages including a glossary. I was able to read it on a plne ride from Louisiana to Denver. Like the tagline says, I hope I never need this book.

As the title suggests, the book is about making Java more functional. On my team we spend half our time in Groovy and half on Scala. I suppose we have done similar things in Groovy. The support that Groovy has for closures makes it easier than it would have been in Java.

He also covers what is coming in Java for more FP features like closures. The syntax looks clunky but I could see living with it if I had no choice.

I found the book useful in reviewing and appreciating where I’m at. How things were in Java and how nice things are in Scala. Even if you don’t need to do functional programming in Java right now I think this book is a good read. If my current gig fell apart and I had to do Java fulltime then I would grab this book to keep my sanity.

April 2013 Month In Review

2013-04-27T00:00:00-05:00

Home Gym

It all started with some bro leisurely doing super-sets that included chain curls in the only available squat rack that uses standard weights (non metric). It was my squat day, oh man. I lifted at Red’s for a year and it’s alright, but their equipment setup is goofy. No wonder they have so many machines as opposed to barbells and plates; no one puts anything away at Red’s.

So I started to put together a home gym. I have a horrible barbell that I got for free. I have a set of hi-temp bumper plates (recycled tires) from Rogue and a bench and squat stand from Valor. I got some small standard weights for Ryder’s birthday this year. Those are good for shoulder dumbell work. Also I had a few kettlebellls. I’m able to get in everything I need with this equipment, it’s awesome.

Workouts are going great. The weather in Louisiana is awesome this time of year we’ll see how it is when it gets very hot this summer and when it gets chilly in the following winter. What’s especially cool is on week 4 of 5/3/1 where you’re de-loading. I don’t have to drive 15 minutes each way to the gym just to kind of go through the motions and not push myself too hard.

Too Aggressive

With 5/3/1 you’re supposed to up your upper body lifts by 5 lbs and lower body lifts by 10 lbs every month. For April I decided to up my deadlift by 15 lbs and my press by 10 lbs. For those exercises I was hitting 10 reps for week 3 of 5/3/1 regularly and I thought I should bump it up more quickly.

April numbers

pres: 170x6 (197 C1RM)
deadlift: 330x3 (349 C1RM)
bench: 270x5 (304 C1RM)
squat: 270x3 (286 C1RM)

Squat

I still get video of my squat and put it online, especially on the 5/3/1 week. I really want to keep myself honest here. This shot is kind of low, off of my bench but I can see my thigh make a brief appearance above my hip crease so I think I’m in good shape.

In Praise Of The Full Time Employee

2013-03-17T00:00:00-05:00

As software engineeers we have to turn our attention to thought leaders. These might be consultants, blog authors, book authors, speakers, software authors. Usually they have short engagements with companies. Their skills run deep into their chosen technologies.

I think it’s right for us to look up to these guys and to use them as resources. But I also think we tend to overlook the FTE of the software world. Their careers might not be as sexy, they might not work with the latest technologies but they deserve some recognition.

From the outset here I gotta say this is self serving. I’m a working stiff FTE. I’ve thought about contracting a few times in my career but I always found a reason to chicken out. So this is in no way a condemnation of consultants that specialize in specific technology for short term engagements.

What’s more, I consider many of those very guys friends, occasional drinking buddies and dare I say mentors. Scott Davis, Tim Berglund, Matthew McCullough.

Intellectual Property

When companies talk about their intellectual property they probably don’t think of their software engineers. I think this is a mistake. In many aspects of the business no one knows how things get done like software engineers. No one knows the algorithms, where the data comes from and who the data goes to like the software engineers. If you run a business, your software engineers that are full time employees are your intellectual property.

Fixing the Disfunction

Full time employees know that just as they are going to have to live in a code base they are also going to have to live in an organizational chart. They care more about hiring the right team members. They care about pulling other team members up. They also care about establishing long-term relationships within the organization that benefit the company.

Sleeping in Your Own House

If you’ve had a career of living with your own software, design, architecture and coding decisions then you go about these decisions differently. By ‘living with’ I mean that you are present for the inception of a project, the initial development and you are around to support the software for a few years. If you’ve done that a few times in your career I think you get an appreciation for the impact of decisions.

Summary

I hate to take the steam out of my argument but I have to say that in each one of these areas I’ve seen short-term contractors have a positive effect.

Smoke Tests For Your System

2013-02-24T00:00:00-06:00

Me and another team member took our most recent lab week to do something incredibly un-sexy. Our resumes have no new libraries, buzz-words or languages that we can add. We decided to use some old-school tools to create a very tactical set of smoke tests for our system and for the back-end systems that we depend on.

Our system is used by many clients. When we get requests we then turn around and talk to about 20 different back-end systems. Many times we are caught in the middle when there are anomalies found by our clients. When this happens we have to turn to the logs to see if there’s a problem with a system we depend on. We also didn’t have much visibility when our systems were not acting as they should. The principle is this. Create a shell script that will run a bunch of other shell scripts in a directory. If the script sends back an exit status that is bad (other than 1) we will consider that test failed. The runner has a simple check for that and will echo PASS/ FAIL based on what we get back. We also color the text green or red based on the results. 3/4 of my career has been about making red things green.

The output is so simple that my wife looked at a screen with a bunch of red and said, “that doesn’t look good.” I’m not sure if we’ll opensource it, there isn’t much intellectual property to speak of. We decided to set up the scripts to run in a cron job once an hour and generate an HTML report showing the same type of data.

We set a low bar on the smoke tests, sometimes they use cURL to POST a SOAP request and we just look at the response to see a specific piece of information is present. Sometimes we just use NMAP to convince us that the port is warm on the far end and NMAP at least thinks it’s the right type of application. The idea is that these tests should be written in 5 minutes.

We’re not sure how these will be used. We hope that folks that support operations (including developers) will see new problems that come up not only as an opportunity to diagnose the matter at hand but to also add another smoke test that will give us a more clear indication of the problem the next time around.

This was an incredibly tactical bit of software that won’t win us any design awards but it has already paid off a handful of times. I regret that we didn’t put something together like this sooner.

Maven3 Maven2 Mixup On Mac

2013-01-05T00:00:00-06:00

At first my Mac was all:

  ~/opt/maven/bin>mvn --version  
  Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/plexus/classworlds/launcher/Launcher  
  Caused by: java.lang.ClassNotFoundException: org.codehaus.plexus.classworlds.launcher.Launcher  
  at java.net.URLClassLoader$1.run(URLClassLoader.java:202)  
  at java.security.AccessController.doPrivileged(Native Method)  
  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)  
  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)  
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)  
  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

And then I’m like (in my ~/.bashrc)

  export M3_HOME=$OPT/maven  
  export M2\_HOME=$M3\_HOME  
  export PATH=$M3_HOME/bin:$PATH

So now it’s all:

  ~/opt/maven/bin>./mvn --version  
  Apache Maven 3.0.4 (r1232337; 2012-01-17 02:44:56-0600)  
  Maven home: /Users/demian/opt/maven  
  Java version: 1.6.0_37, vendor: Apple Inc.  
  Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home  
  Default locale: en_US, platform encoding: MacRoman  
  OS name: "mac os x", version: "10.8.2", arch: "x86_64", family: "mac"

Java Jdk 1.7 On Mac 1.8.2

2012-12-14T00:00:00-06:00

I have been doing Scala and Groovy at the day job for about a year now. For some side work I thought I’http://neidetcher.blogspot.comd use Java, there obviously aren’http://neidetcher.blogspot.comt enough semi-colons in my life. I haven’http://neidetcher.blogspot.comt touched 1.7 yet and setting it up on my Mac wasn’http://neidetcher.blogspot.comt incredibly intuitive.

I had to do more poking around to arrive at this than I wish I had to.

Finding the actual download from Oracle is easy.

After installing Java 1.7 from Oracle I added this to my ~/.bashrc to actually use it. Not sure if the install should’http://neidetcher.blogspot.comve done this or if I missed some steps I was supposed to take.

export JRE_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/
export JAVA_HOME=$JRE_HOME
export JDK_HOME=$JRE_HOME
export PATH=$JAVA_HOME/bin:$PATH

And holy cow Oracle does not make it easy to find out how to download the JDK 1.7 API documentation. Got to have it local for coding on an airplane.

Another Look At Play2

2012-11-19T00:00:00-06:00

Neidetcher: Another Look at Play2

Matt Raible has some good writeups on Play2. We switched to Scala about a year ago. For lab week, a few of us evaluated Play earlier this year. At the time I mostly worked with the persistence layer, anorm. Our conclusion was that it wasn’t quite ready for what we wanted to do. Just recently I finished a lab week and decided to take another look at Play. Here are some conclusions.

Overall
The big question is, compared to what. At work we use Grails on the web side and Scala on the back-end. The reality is that we leverage front-end engineers to help us make Time Warner Cable web sites look and behave well. In our situation I don’t think I can recommend it. But if you’re in a hardcore Scala shop and your Scala engineers can do enough of the front-end work then Play might be a good option.

Working with routes and templates felt a lot like Android development. You work in a non-code DSL to create your routes and that turns into code that you write against. Same for templates, they just turn into functions. sbt run fires up their own netty server, it’s solid. It does a great job picking up changes. I didn’t restart the play development server for a few days and it didn’t miss anything. Play also gives you helpful error messages in the browser when there are problems.

Routes
They’re handled in a text file. I guess it’s good because there’s no cruft surrounding the definition of your routes. It’s bad because I think Scala is the perfect language for defining an internal DSL to handle routes. It’d be great to have IDE support while working with routes.

Controllers
Everything feels very idiomatic to Scala. For example optional parameters are (drum-roll) Options. Play doesn’t have you use servlet filters, you just wrap functions. The controllers themselves are not easy to test but there are workarounds such that you can do good unit tests.

Templating
It’s familiar to someone used to Scala. Templates are just functions and have input parameters with types. This gets statically checked and your IDE will bark at you if you send the wrong data type into a template. I think that’s awesome. The templating is not as rich as Groovy Server Pages. For the actual templating work there is no good IDE support. It’d be great if TypeSafe took this on. The templating that Play gives you would be painful for front-end engineers that are only familiar with HTML, CSS and JavaScript.

Testing
They have utilities to help with testing. All the examples use Specs, I haven’t got the Specs memo yet, still using jUnit (get off my porch). The biggest problem is that the controllers are objects so it’s hard to do good unit testing. I found a workaround that someone suggested. Basically your controller is a class, in this case taking a dependency via the constructor. Then the object that you tell Play about in the routes extends that class. This seems to work in my toy example.

object MockUp extends controllers.MockUp(new ProductionGuidCryptoServiceProvider()) {}

: http://www.playframework.org/assets/images/logos/normal.png

The Setup

2012-11-16T00:00:00-06:00

Neidetcher: The Setup

One of my favorite blogs is The Setup. They feature various people that use computers in one way or another to do their job and simply ask them what their setup is. Well since they haven’t asked me what my setup is yet (not realizing that I’m kind of a big deal) I’m going to just post this on my own.

#

Who are you, and what do you do? I’m a software engineer. I have worked mostly for telecommunications companies.

The most challenging job I had was leading up a small but capable development team responsible for routing phone calls, including geo-spatial 911 traffic. This was at a startup without much support.

Now I’m a Senior Engineer at Time Warner Cable. My home office is in Denver, Colorado but I work remote in Lafayette, Louisiana. I fly back about once a month. The team I’m on works on the services that IP devices use to get streaming television content.

#

What hardware do you use? My main computer is a MacBookPro 15” from 2010. I also have a MacPro Desktop. Each computer has a 27” Thunderbolt Display. I was skeptical at first but the Apple Magic Mouse is an amazing piece of equipment. I use 1 standard keyboard for the desktop but primarily I use the bluetooth keyboard. I wouldn’t mind having the bigger size but the travel from home keys to the mouse seems like an eternity compared to the bluetooth keyboard.

I also have an iPad 2 for work. I use it to validate software releases, cable channel updates. I also have an Asus Transformer to do the same thing on the Android platform. I don’t write the client code but the team I’m on is responsible for many of the back end services that these devices talk to.

My phone is an LG P999 Android phone. Works great, has all the apps I need; Evernote, Pandora, Dropbox, Google Reader.

#

And what software? I run MacOS on both machines. The laptop is 10.6, the desktop is 10.7. I hear so much griping about upgrading the OS that I don’t mind waiting a while until the kinks get worked out. Before going to the Mac a few years ago I was strictly a Linux user. It has been a smooth transition to the Mac. I still feel comfortable whenever I go back to Linux. If Apple gets too stupid I’ll probably switch back.

I spend most of my time in the IDE. I run Eclipse with the Scala-IDE plugin. I also use a code coverage plugin for Eclipse that can handle Scala called eCobertura.

Despite having an IDE I spend a lot of my time in the console, for that I use iTerm. Being a long-time VIM user I’m thankful for MacVim, it’s a great distribution of VIM. I use Homebrew for installing commandline tools that are missing from the Mac. At work and in my personal development we use git exclusively. On my desktop I run Jenkins for continuous integration. It polls the git repository on my laptop and our remote GitHub Enterprise servers at work. It continuously compiles our code, runs tests and generates reports.

When using a browser for development I will typically use Chrome. The Chrome plugins I use are for either maintaining privacy or doing software development. I use Poster, Edit this Cookie, AdBlock, Chrome Nanny, Ghostery, Time Tracker, Evernote and HTTPS Everywhere.

I use ShareMouse to make both computers and monitors feel like one big computer. The price is right (free in my case) but it’s occasionally glitchy.

For remote work these come in handy: TimeZone, Skype and join.me for impromptu screen sharing

I use Evernote to keep track of information, DropBox to sync files, Pomodoro on occasion to focus, f.lux to save my eyes when working after dark, Alfred as an app launcher and searcher, Marked for previewing MarkDown files. MindNode to create mind maps, PandaBar to make listening to Pandora radio not depend on Adobe Air (WTF) and ShiftIt to easily organize windows.

#

What would be your dream setup? Pretty much my same setup but with more RAM. Maybe some SSDs. The Scala compiler does a lot of work for you so you need top end hardware when you’re doing any Scala project that is beyond trivial. Although our own informal analysis at work seems to indicate that SSDs don’t buy you too much for building Scala code, seems that good old fashioned RAM and processor speed is what you need..

Other than that I’d like to have a screaming ThinkPad or AlienWare laptop running some Debian Linux variant.

Functional Programming Principles In Scala

2012-11-09T00:00:00-06:00

Functional Programming Principles in Scala

So here’s the deal, I was dreading the thought of doing another assignment in the Odersky Scala course on Coursera. I have other projects I want to move on to and man I’m not built for the academic side. Then they send out an email telling everyone the minimum number of points we need to complete the course. Alt-tab over to a Google Doc spreadsheet, do some quick math and presto, I have way more than enough points. So, I punted on the last assignment. Yep, I’m that guy, I wrapped up my CS degree with a D in post-calc stats to get the diploma. That’ll work. Besides my weak finish the course was awesome. If you’re in the Scala space and you are anything short of a wizard I don’t understand why you wouldn’t jump on the opportunity to get instruction from Martin Odersky himself.

The Course The focus of the course, as you would imagine, is functional programming. They take much inspiration from

SICP. I don’t know how much of it was SICP or Martin Odersky himself but the course laid out the proper motivation and revelation of information such that my brain was willing and able to accept it. This was a well thought out, professional course. 50,000 people signed up for the course. I don’t know how many people finished it. The course developers created an automated system that analyzed your code to make sure it passed their own tests along with some other code analysis. For example I had some code implemented that passed tests but it didn’t use the groupBy capability in collections. That was the point of the exercise so they dinged me. Fair enough, they’re checking lots of stuff.

Because of the automated nature, the programming assignments have to structured in a particular way. They have to lead you down a path. They can’t just give you an assignment and a blank canvas. I think this is okay. But it’s a different approach to programming assignments and worth recognizing.

#

To Those That Come After Here are some things I recommend to someone taking the course when it is offered next.

Get the SICP book and refer to it, it helped a few times. Odersky mentions it and I think it’s fair game to refer to it.
I didn’t do any of the video quizes, I don’t think it hurt me.
Leverage the Scala-IDE worksheet, dump code in there and play, I think it’s better than the REPL.
The forums were sometimes (not very often) helpful.
Submit early and often. You get great feedback. Some co-workers found that if you put print statements in your code you’ll see the output in the online feedback, this can give you some indications of the additional test scenarios they are using to evaluate your code.

The Cheating 9 co-workers and myself have our own Skype room dedicated to the course. No one wanted to cheat, we all wanted to get a lot out of the class. We even made sure that no one gave away

spoilers before the projects were due. But we also knew that we would benefit greatly by looking at each others work when we were done with assignments. I actually learned some cool stuff and it was a great time for me to accept the information since I just got done spending hours trying to solve the same problem. So I was one of the guys that had his assignments up on github. Then we get the email from the admins that we need to take it down. That’s all cool but they cited how we signed an agreement acknowledging we wouldn’t post our stuff or whatever. I wish you could see my face but let’s just say, no one fuckin reads EULAs. If you don’t already know that, especially if you’re in software I’m not sure what to say. So I took it down from github and put it on a private repo on bitbucket. If you don’t already know, you get free private git repos from bitbucket, good stuff.

On From Here I’m doing things out of order here but I’m going to go through the Scala Koans from here. Just to loop back and go thru the basics some more. In the forums the course administrators say they plan on offering additional Scala classes through Coursera. I look forward to seeing what they offer.

Stairway To Scala

2012-10-31T00:00:00-05:00

The team I am on at Time Warn Cable has been doing Scala for almost a year now. 5 of us flew out to San Francisco to attend Stairway to Scala training.

The training was great, I really enjoyed the style of having 2 instructors. One of them doing traditional lessons and the other one would work on the REPL. To engineers things are more real when you can see running code. It’s obvious that Dick and Bill put lots of effort into the curriculum.

I recommend the training. Dick Wall and Bill Venners are awesome instructors. The Time Warner Cable crew was pretty punchy and the instructors managed to keep their cool.

Spare Box Ci

2012-09-17T00:00:00-05:00

I have 2 employer provided computers. One is a Mac Book Pro, the other is a Mac Pro. I do my day to day development on the MBP because I want a smooth transition when I’m on the road. I do this even though the desktop is a beast. At home I use ShareMouse (a software KVM) so that the computers almost seem as one. The desktop streams music from Pandora, has all the IM and email clients.

I still wanted to leverage the desktop for my workflow. The setup I have allows me to run CI unit tests against my local (laptop) git repo. I’ll make changes, commit them and then the desktop will poll my laptop to see if there is anything new. If there is something new then it does a full build and unit test. I can continue doing other tasks on the laptop while the desktop is churning through our scala project (compiling scala code is resource intensive). Then when I see that the build is in good shape I’ll do a push to our team GitHub Enterprise Server.

On your laptop

Tell git it’s okay to export your project by touching this file: <path-to-project>/.git/git-daemon-export-ok
Serve up my local repo git daemon --verbose --base-path=<path-to-project>

On your desktop

Install Jenkins on the desktop
Point desktop Jenkins to your git repo: git://<your-network-ip>/.git

The end result looks something like this:

(http://www.flickr.com/photos/neidetcher/7997187470/)

Scala Encoding Fun

2012-03-15T00:00:00-05:00

I pulled a toy Scala project into Eclipse and saw this problem:

IO error while decoding [file].scala with UTF-8 Please try specifying another one using the -encoding option

This guy didn’t have the exact solution but he got me pointed in the right direction. The problem is that I must have copied in some non-UTF-8 characters into my source code. At this point the Scala compiler got confused.

Here’s a bunch of shell commands I did to fix the problem:

Figure out the character set of the file we’re having trouble with

~/code/scalakata>file -I ./src/test/scala/com/neidetcher/sfti/Chapter1Test.scala
./src/test/scala/com/neidetcher/sfti/Chapter1Test.scala: text/x-java; charset=iso-8859-1

Yep, the old iso-8859-1, that’s not what we want. That’s what we use as the from encoding.

Make a copy of the offending file UTF-8

~/code/scalakata>iconv -f iso-8859-1 -t utf-8 ./src/test/scala/com/neidetcher/sfti/Chapter1Test.scala > ./src/test/scala/com/neidetcher/sfti/Chapter1Test.scala.utf8

The diff command will show you what the offending line was

~/code/scalakata>diff ./src/test/scala/com/neidetcher/sfti/Chapter1Test.scala ./src/test/scala/com/neidetcher/sfti/Chapter1Test.scala.utf8
134c134
< 		// 4) Scala lets you multiply a string with a number?try out "crazy" * 3 in the REPL. What does this operation do? Where can you find it in Scaladoc?
---
> 		// 4) Scala lets you multiply a string with a numberÑtry out "crazy" * 3 in the REPL. What does this operation do? Where can you find it in Scaladoc?

Now actually replace the old file

~/code/scalakata>mv ./src/test/scala/com/neidetcher/sfti/Chapter1Test.scala.utf8 ./src/test/scala/com/neidetcher/sfti/Chapter1Test.scala

After that I was back in business.

Current Lifting Program

2012-01-03T00:00:00-06:00

Over the past couple years I have gotten back into weight lifting. It has been great, I’m stronger than I was when I was in the Marines 20 years ago. I’m nowhere near as fast though, maybe we’ll work on that in the future.

Kettlebell

When I first got back into lifting I picked up the kettlebell at the recommendation of Bryan Noll. This was a good place to start. If I was stranded on a desert island and could only have one piece of equipment it would be a 53 pound kettlebell. The simplicity of the device makes it easy to get going.

I read Enter the Kettlebell, it’s a great resource to start. It’s probably enough just to buy yourself a kettlebell and watch some YouTub videos.

5x5

Eventually I wanted more body composition changes so I put together a 5x5 program for myself. I was inspired by the Stong Lifts e-book but didn’t use his program. He lays out a full-body system where every day you work all muscle groups using old-school multi-joint barbell lifts.
I think my body doesn’t recover quickly enough to lift a few times in a week. I have always been that way even though I respond well to lifting weights.

I created a split routine that was push, pull and legs. With a handful of exercises that were all 5 sets of 5 reps. I got great results with this.

5/3/1

In the fall of 2011 I started Jim Wendlers 5/3/1 program. 5/3/1 is an undulating progression routine where you increase weights for 3 weeks then take a week to do light work. The 4 weeks you do the same gradual progression but with 5-10 more pounds. A program like 5/3/1 makes sense when you are no longer able to make linear progression of 5 pounds for a lift between workouts.

I can’t hold a candle to the time under the bar and intelligence that Jim Wendler has but after a couple cycles I quit doing 5/3/1 strictly.

There are a couple reasons I didn’t like 5/3/1 strictly: * my military press or overhead press is already in great shape and I don’t need an entire day for shoulder work. * I didn’t like the abscence of a day spent on back and pulling. Although the 5/3/1 deadlift day would probably be seen as a back day also. 5/3/1 also suggests doing pull-ups between all push movements, that would get you lots of time pulling.

I’ll probably give strict 5/3/1 another chance but for now this is what I’m doing.

Modified 5/3/1

Leg

5/3/1 squat
3x10 leg curl
3x10 leg extension
3x20 weighted hyper-extension
3x10 squat

Press

5/3/1 bench
3x10 military press
3x10 dumbell press
3x10 arnold press
3x10 tricep press

Pull

3 sets of pull ups, assisted
3x10 bent row
3x10 upright row
3x10 db row
3x10 curl machine
3x10 hang clean

Jekyll Introduction

2011-12-29T00:00:00-06:00

This Jekyll introduction will outline specifically what Jekyll is and why you would want to use it. Directly following the intro we’ll learn exactly how Jekyll does what it does.

Overview

What is Jekyll?

Jekyll is a parsing engine bundled as a ruby gem used to build static websites from dynamic components such as templates, partials, liquid code, markdown, etc. Jekyll is known as “a simple, blog aware, static site generator”.

Examples

This website is created with Jekyll. Other Jekyll websites.

What does Jekyll Do?

Jekyll is a ruby gem you install on your local system. Once there you can call jekyll --server on a directory and provided that directory is setup in a way jekyll expects, it will do magic stuff like parse markdown/textile files, compute categories, tags, permalinks, and construct your pages from layout templates and partials.

Once parsed, Jekyll stores the result in a self-contained static _site folder. The intention here is that you can serve all contents in this folder statically from a plain static web-server.

You can think of Jekyll as a normalish dynamic blog but rather than parsing content, templates, and tags on each request, Jekyll does this once beforehand and caches the entire website in a folder for serving statically.

Jekyll is Not Blogging Software

Jekyll is a parsing engine.

Jekyll does not come with any content nor does it have any templates or design elements. This is a common source of confusion when getting started. Jekyll does not come with anything you actually use or see on your website - you have to make it.

Why Should I Care?

Jekyll is very minimalistic and very efficient. The most important thing to realize about Jekyll is that it creates a static representation of your website requiring only a static web-server. Traditional dynamic blogs like Wordpress require a database and server-side code. Heavily trafficked dynamic blogs must employ a caching layer that ultimately performs the same job Jekyll sets out to do; serve static content.

Therefore if you like to keep things simple and you prefer the command-line over an admin panel UI then give Jekyll a try.

Developers like Jekyll because we can write content like we write code:

Ability to write content in markdown or textile in your favorite text-editor.
Ability to write and preview your content via localhost.
No internet connection required.
Ability to publish via git.
Ability to host your blog on a static web-server.
Ability to host freely on GitHub Pages.
No database required.

How Jekyll Works

The following is a complete but concise outline of exactly how Jekyll works.

Be aware that core concepts are introduced in rapid succession without code examples. This information is not intended to specifically teach you how to do anything, rather it is intended to give you the full picture relative to what is going on in Jekyll-world.

Learning these core concepts should help you avoid common frustrations and ultimately help you better understand the code examples contained throughout Jekyll-Bootstrap.

Initial Setup

After installing jekyll you’ll need to format your website directory in a way jekyll expects. Jekyll-bootstrap conveniently provides the base directory format.

The Jekyll Application Base Format

Jekyll expects your website directory to be laid out like so:

.
|-- _config.yml
|-- _includes
|-- _layouts
|   |-- default.html
|   |-- post.html
|-- _posts
|   |-- 2011-10-25-open-source-is-good.markdown
|   |-- 2011-04-26-hello-world.markdown
|-- _site
|-- index.html
|-- assets
    |-- css
        |-- style.css
    |-- javascripts

_config.yml
Stores configuration data.
_includes
This folder is for partial views.
_layouts
This folder is for the main templates your content will be inserted into. You can have different layouts for different pages or page sections.
_posts
This folder contains your dynamic content/posts. the naming format is required to be @YEAR-MONTH-DATE-title.MARKUP@.
_site
This is where the generated site will be placed once Jekyll is done transforming it.
assets
This folder is not part of the standard jekyll structure. The assets folder represents any generic folder you happen to create in your root directory. Directories and files not properly formatted for jekyll will be left untouched for you to serve normally.

Jekyll Configuration

Jekyll supports various configuration options that are fully outlined here: https://github.com/mojombo/jekyll/wiki/Configuration

Content in Jekyll

Content in Jekyll is either a post or a page. These content “objects” get inserted into one or more templates to build the final output for its respective static-page.

Posts and Pages

Both posts and pages should be written in markdown, textile, or HTML and may also contain Liquid templating syntax. Both posts and pages can have meta-data assigned on a per-page basis such as title, url path, as well as arbitrary custom meta-data.

Working With Posts

Creating a Post
Posts are created by properly formatting a file and placing it the _posts folder.

Formatting
A post must have a valid filename in the form YEAR-MONTH-DATE-title.MARKUP and be placed in the _posts directory. If the data format is invalid Jekyll will not recognize the file as a post. The date and title are automatically parsed from the filename of the post file. Additionally, each file must have YAML Front-Matter prepended to its content. YAML Front-Matter is a valid YAML syntax specifying meta-data for the given file.

Order
Ordering is an important part of Jekyll but it is hard to specify a custom ordering strategy. Only reverse chronological and chronological ordering is supported in Jekyll.

Since the date is hard-coded into the filename format, to change the order, you must change the dates in the filenames.

Tags
Posts can have tags associated with them as part of their meta-data. Tags may be placed on posts by providing them in the post’s YAML front matter. You have access to the post-specific tags in the templates. These tags also get added to the sitewide collection.

Categories
Posts may be categorized by providing one or more categories in the YAML front matter. Categories offer more significance over tags in that they can be reflected in the URL path to the given post. Note categories in Jekyll work in a specific way. If you define more than one category you are defining a category hierarchy “set”. Example:

---
title :  Hello World
categories : [lessons, beginner]
---

This defines the category hierarchy “lessons/beginner”. Note this is one category node in Jekyll. You won’t find “lessons” and “beginner” as two separate categories unless you define them elsewhere as singular categories.

Working With Pages

Creating a Page
Pages are created by properly formatting a file and placing it anywhere in the root directory or subdirectories that do not start with an underscore.

Formatting
In order to register as a Jekyll page the file must contain YAML Front-Matter. Registering a page means 1) that Jekyll will process the page and 2) that the page object will be available in the site.pages array for inclusion into your templates.

Categories and Tags
Pages do not compute categories nor tags so defining them will have no effect.

Sub-Directories
If pages are defined in sub-directories, the path to the page will be reflected in the url. Example:

.
|-- people
    |-- bob
        |-- essay.html

This page will be available at http://yourdomain.com/people/bob/essay.html

Recommended Pages

index.html
You will always want to define the root index.html page as this will display on your root URL.
404.html
Create a root 404.html page and GitHub Pages will serve it as your 404 response.
sitemap.html
Generating a sitemap is good practice for SEO.
about.html
A nice about page is easy to do and gives the human perspective to your website.

Templates in Jekyll

Templates are used to contain a page’s or post’s content. All templates have access to a global site object variable: site as well as a page object variable: page. The site variable holds all accessible content and metadata relative to the site. The page variable holds accessible data for the given page or post being rendered at that point.

Create a Template
Templates are created by properly formatting a file and placing it in the _layouts directory.

Formatting
Templates should be coded in HTML and contain YAML Front Matter. All templates can contain Liquid code to work with your site’s data.

Rending Page/Post Content in a Template
There is a special variable in all templates named : content. The content variable holds the page/post content including any sub-template content previously defined. Render the content variable wherever you want your main content to be injected into your template:

...
<body>
  <div id="sidebar"> ... </div>
  <div id="main">
    {{content}}
  </div>
</body>
...

Sub-Templates

Sub-templates are exactly templates with the only difference being they define another “root” layout/template within their YAML Front Matter. This essentially means a template will render inside of another template.

Includes

In Jekyll you can define include files by placing them in the _includes folder. Includes are NOT templates, rather they are just code snippets that get included into templates. In this way, you can treat the code inside includes as if it was native to the parent template.

Any valid template code may be used in includes.

Using Liquid for Templating

Templating is perhaps the most confusing and frustrating part of Jekyll. This is mainly due to the fact that Jekyll templates must use the Liquid Templating Language.

What is Liquid?

Liquid is a secure templating language developed by Shopify. Liquid is designed for end-users to be able to execute logic within template files without imposing any security risk on the hosting server.

Jekyll uses Liquid to generate the post content within the final page layout structure and as the primary interface for working with your site and post/page data.

Why Do We Have to Use Liquid?

GitHub uses Jekyll to power GitHub Pages. GitHub cannot afford to run arbitrary code on their servers so they lock developers down via Liquid.

Liquid is Not Programmer-Friendly.

The short story is liquid is not real code and its not intended to execute real code. The point being you can’t do jackshit in liquid that hasn’t been allowed explicitly by the implementation. What’s more you can only access data-structures that have been explicitly passed to the template.

In Jekyll’s case it is not possible to alter what is passed to Liquid without hacking the gem or running custom plugins. Both of which cannot be supported by GitHub Pages.

As a programmer - this is very frustrating.

But rather than look a gift horse in the mouth we are going to suck it up and view it as an opportunity to work around limitations and adopt client-side solutions when possible.

Aside
My personal stance is to not invest time trying to hack liquid. It’s really unnecessary from a programmer’s perspective. That is to say if you have the ability to run custom plugins (i.e. run arbitrary ruby code) you are better off sticking with ruby. Toward that end I’ve built Mustache-with-Jekyll

Static Assets

Static assets are any file in the root or non-underscored subfolders that are not pages. That is they have no valid YAML Front Matter and are thus not treated as Jekyll Pages.

Static assets should be used for images, css, and javascript files.

How Jekyll Parses Files

Remember Jekyll is a processing engine. There are two main types of parsing in Jekyll.

Content parsing.
This is done with textile or markdown.
Template parsing.
This is done with the liquid templating language.

And thus there are two main types of file formats needed for this parsing.

Post and Page files.
All content in Jekyll is either a post or a page so valid posts and pages are parsed with markdown or textile.
Template files.
These files go in _layouts folder and contain your blogs templates. They should be made in HTML with the help of Liquid syntax. Since include files are simply injected into templates they are essentially parsed as if they were native to the template.

Arbitrary files and folders.
Files that are not valid pages are treated as static content and pass through Jekyll untouched and reside on your blog in the exact structure and format they originally existed in.

Formatting Files for Parsing.

We’ve outlined the need for valid formatting using YAML Front Matter. Templates, posts, and pages all need to provide valid YAML Front Matter even if the Matter is empty. This is the only way Jekyll knows you want the file processed.

YAML Front Matter must be prepended to the top of template/post/page files:

---
layout: post
category : pages
tags : [how-to, jekyll]
---

... contents ...

Three hyphens on a new line start the Front-Matter block and three hyphens on a new line end the block. The data inside the block must be valid YAML.

Configuration parameters for YAML Front-Matter is outlined here: A comprehensive explanation of YAML Front Matter

Defining Layouts for Posts and Templates Parsing.

The layout parameter in the YAML Front Matter defines the template file for which the given post or template should be injected into. If a template file specifies its own layout, it is effectively being used as a sub-template. That is to say loading a post file into a template file that refers to another template file with work in the way you’d expect; as a nested sub-template.

How Jekyll Generates the Final Static Files.

Ultimately, Jekyll’s job is to generate a static representation of your website. The following is an outline of how that’s done:

Jekyll collects data.
Jekyll scans the posts directory and collects all posts files as post objects. It then scans the layout assets and collects those and finally scans other directories in search of pages.
Jekyll computes data.
Jekyll takes these objects, computes metadata (permalinks, tags, categories, titles, dates) from them and constructs one big site object that holds all the posts, pages, layouts, and respective metadata. At this stage your site is one big computed ruby object.
Jekyll liquifies posts and templates.
Next jekyll loops through each post file and converts (through markdown or textile) and liquifies the post inside of its respective layout(s). Once the post is parsed and liquified inside the the proper layout structure, the layout itself is “liquified”.
Liquification is defined as follows: Jekyll initiates a Liquid template, and passes a simpler hash representation of the ruby site object as well as a simpler hash representation of the ruby post object. These simplified data structures are what you have access to in the templates.
Jekyll generates output.
Finally the liquid templates are “rendered”, thereby processing any liquid syntax provided in the templates and saving the final, static representation of the file.

Notes.
Because Jekyll computes the entire site in one fell swoop, each template is given access to a global site hash that contains useful data. It is this data that you’ll iterate through and format using the Liquid tags and filters in order to render it onto a given page.

Remember, in Jekyll you are an end-user. Your API has only two components:

The manner in which you setup your directory.
The liquid syntax and variables passed into the liquid templates.

All the data objects available to you in the templates via Liquid are outlined in the API Section of Jekyll-Bootstrap. You can also read the original documentation here: https://github.com/mojombo/jekyll/wiki/Template-Data

Conclusion

I hope this paints a clearer picture of what Jekyll is doing and why it works the way it does. As noted, our main programming constraint is the fact that our API is limited to what is accessible via Liquid and Liquid only.

Jekyll-bootstrap is intended to provide helper methods and strategies aimed at making it more intuitive and easier to work with Jekyll =)

Thank you for reading this far.

Next Steps

Please take a look at or jump right into Usage if you’d like.

Book Review My Job Went To India

2006-03-19T00:00:00-06:00

i bought the pdf version of the book. if you haven't bought a pdf book from the pragmatic programmers they give you a customized book complete with your name on each page. to do this processing it takes a few minutes before you can actually download your book. your name in the book is to discourage you from throwing it up on some p2p system i'm sure.

the book was great, i felt the same way as when i read the pragmatic programmer. there were lots of moments where the author said things that seem obvious but it's always good to get validation. even tho a lot of the information should be a given to anyone in the tech business the book serves as a kick in the pants to motivate you into change.

the book was well worth the price for the information that was new to me. it wasn't so much new information but insightful things you can do to improve yourself.

here are some good points from the book:

learn some obscure technologies: you can get a new perspective and differentiate yourself from the herd.
learn your domain: if you're in telecom, learn your vertical, the coder ignorant of his business won't last long in the future.
be a generalist: be a 1 man shop, networking, dba, coder, tester, architect
be a specialist: if you are a java developer know java. know how the classloader works, know serialization, know how to break the jvm, know garbage collection, know the libraries.
don't become tiffany: remember her? it's better to be like madonna, change with the times. do an audit of your skills and where they are in the marketplace. look for new trends.

it's worth mentioning that the author has reverence for people in india. he appreciates the culture and reminds us that they are intelligent as their stateside counterparts. he reminds the reader that most indian workers are highly motivated to improve their standard of living. the author also gives you insight into establishing a good relationship with distributed teams.

there's lots more wisdom in the book. in the process of reading the book i've made a checklist of things i want to do to improve myself professionally; books to read, tasks to do and things to learn. ...yeah, i know i'm so cool, seriously tho, that might be a good way to approach the book. as i said, the book is worth the price especially if you're getting the pdf ($13).

My Job Went to India: And All I Got Was This Lousy Book (Pragmatic Programmers)