Jenkins CI

NYC Jenkins User Conference Recap

rtyler — Mon, 21 May 2012 17:11:26 +0000

Editor's Note: The following is a write-up courtesy of Jesse Farinacci

This past week I had the pleasure of attending the Jenkins User Conference in NYC. A hundred other like-minded continuous integration enthusiasts and I packed a very posh Marriott Marquis for a full day of Jenkins excitement.

Famed Hudson and Jenkins founder Kohsuke Kawaguchi delivered the opening address to a crowded room.

I'm sure everyone knows the statistics by now, that Jenkins adoption and development continues at an unbridled pace. Pushing past all the mailing list users and posts, the JIRAs opening and closing, the Twitter followers, the five Jenkins User Conferences scheduled for this year, the unprecedented number of installations reporting anonymous usage, the native availability for nine different OSes, in pushing past all of that..

For me, the most impressive number was that on average there was about 1 plugin created every day over the past year. Let me reiterate that: 1 plugin created every day for a year. If that isn't the best testament to the versatility, extensibility, and just plain usefulness of a piece of software, then I don't know what would be!

Announced at the conference was the general availability of CloudBees BuildHive, this is a mechanism for quickly and easily obtaining access to cloud-based Jenkins. If you have projects on GitHub, you can effortlessly log in to BuildHive via GitHub OAuth, import your projects with literally a single click, and start benefiting from the powerful promise of the cloud. You'll no longer have to worry about managing infrastructure, you'll just get all that great Jenkins CI capability for your projects immediately.


With BuildHive enabled projects, if a user of your project creates a pull request, then it will automatically pre-test that request before it hits the master tree -- commenting inside the pull request whether or not the change broke the build. If a developer wants to test some changes before pushing them to the master tree, there's the Validated Merge feature, which can be used to screen commits before they land in master. Both of these features are wickedly awesome, to not use them would be a mistake for any F/OSS community.

I would encourage you to try out BuildHive, it's already active for most/all of the Jenkins plugins. There's also an existing RFC thread which is in discussion for making this the de facto location for Jenkins plugin on Jenkins testing. Please give it a shot and then post your comments.

Aside from this major announcement, I think the most interesting perspective I took away from the conference was the sheer diversity of Jenkins exploitation. People are using it on all sorts of hardware platforms, from mobile to cloud, on all sorts of languages from Java and C/C++ to Python and Ruby. I was also impressed with how many teams are using Jenkins as part of their mission critical business process.

Finally, I'd like to thank all the Jenkins developers and users for making it the best continuous integration software available today. Keep up the good work!

Jenkins User Conference Paris Summary

kohsuke — Thu, 10 May 2012 13:00:00 +0000

The first stop of Jenkins User Conference world tour this year was Paris, where there's a considerable concentraion of Jenkins developers and users (sometiems those of us on the other side of the Atlantic call them "the French gang") The event was held a day before Devoxx France, in the hope that we attract more attendance.

I believe there are 100+ people that actually showed up, and we had a full day divided in two tracks, talking all things about Jenkins. While many are French, some of the attendees come from all over the Europe. I was able to see some familiar faces, as well as those who I've only known by their names.

I tried to get in and out of both tracks to get the sense of what's going on, so that I can report them later, and here's my notes.

I kicked off the whole day with a keynote, looking back what we've done since we became Jenkins. I've looked into various activities in the community, such as LTS, Jenkins CIA, Ruby plugin development, and UI enhancements. I updated my adoption statistics slides (we are happy to report that we crossed 40K installations in our tracking), and reported that JFrog is now hosting our repositories that we rely on for the development. I showed some of what we've been lately working on at CloudBees — such as the upcoming version of Jenkins Enterprise by CloudBees that support high-availability, our giving away the folder plugin for free (as in beer), and previews of some not quite public yet features, which is a treat only for those who came!

In the first slot, Gregory Boissinot went through a plugin development workshop. This was actually something I really wanted to understand, so that I get the objective view on where the pitfalls are. Even though the talk was in French, I did understand the code he was showing, and I took some notes about having some kind of skeleton code generator — for example, there's a common pattern for building an UI bound model object (for asking the user to enter data that has structures, persisting them, and so on), and having a code generator command line tool (like jenkins.rb has) could be really handy.

In another room, Nicolas and Mathieu were showing their "build flow" plugin, which lets you write a workflow in Groovy DSL. Choreographing a complex workflow that involves multiple jobs is a commoon challenge among any Jenkins users, and so this talk was well attended, and I'm really looking forward to seeing this plugin mature (there's a separate effort to integrate BPMN workflow into Jenkins, see more about that here.) One thing I learned about Groovy DSL since then is the AST transformation. I'm thinking it might allow us to convert the DSL workflow script into a continuation passing style so that you can suspend/resume workflow at arbitrary point.

The day was so packed that we didn't even waste the lunch time! While attendees are eating, we had lightening talks in the room. Olivier showed off how Apache runs Jenkins, which is quite sizable, then I pitched in for Domonik, who couldn't make it to the conference, and covered the scriptler plugin. Vincent followed and covered the similar Groovy system console. Harpreet then closed off the lunch lightening talks by showing the templates plugin in Jenkins Enterprise by CloudBees.

In the afternoon, Arnaud, one of our French gangs, showed how you can set up the iOS development on Jenkins (from code change to test to the delivery of the binaries to actual phones.) Bruno then did a demo of how he uses DEV@cloud and RUN@cloud to quickly set up continuous deployment for Java webapps. For system integraters that deal with lots of projects, I think it is a great combination (for example allowing you to hand over the entire development environment to the customer when the project is over.)

While all that is going on in one room, in another room Lars Kruse showed off how the old meets the new — where you take ClearCase UCM and use it to do validated merge, in which only the changes tested by Jenkins become visible to the rest of the team. I personally don't know much about ClearCase, but it was very interesting that emerging techniques like validated merge can be applied on more traditional SCM tools. He also said his company works with clients to develop custom Jenkins plugins. I always felt that any big company adopting Jenkins need some custom glue plugins, and I regularly come across those companies, but CloudBees can only help so many. It's great to see that there are more help available now!

The talk that followed was from Julien Carsique from Nuxeo, discussing how he manages and improves the CI environment for his organization. Now, I regret I didn't take all the notes about details, but I think this was one of the best presentations of the day for me. I remember thinking that if we had the best Jenkins administrator award for those who push things to the limit and beyond, he would be my top pick. IIRC, he had a major Maven projects that span across different repos and all. He set up Jenkins such that any change triggers a cascade of new builds of downstream jobs, which later then fan out to cross-platform test jobs, then he made the whole thing visualized so you can track exactly where the time is spent and how those changes propagate. I think this was very inspirational to many other fellow Jenkins users, and I hope he will put his slides somewhere so that other people can mimic what he's done.

Back to the big room, my fellow colleagues Stephen and Harpreet did the only introductory talk in the whole day, going through check lists of production Jenkins deployments, recapping why you want CI, etc. (And I always forget that there are still many who don't know much about Jenkins!)

It was also great to see and hear Sebastian Bergmann, the guy behind Jenkins PHP, talk about Jenkins and PHP integrations. I wish we had more of those people who bridge our community to different communities and help us spread the ideas. He even kindly gave me his Jenkins/PHP book and signed it for me!

Aside from talks, food was great, especially for those of us who came from the U.S. I've got some good inspirations about where we need to work. And I also managed to implement the search filter in the update center during the day, in response to the valid complaint from Sebastian. For virtual communities like ours, it's really good to meet people in the meat space and put faces on names. Build automation engineers are often somewhat lonely in their respective organizations — there just aren't that many people who get excited about automating things away, and so having so many of like-minded folks in one room was by itself a great experience.

On the things to improve side, I felt that workshops was tricky to do in a limited time and in a big room. Maybe it would work out better if there's a smaller room where smaller number of people can gather and hack away (probably some time slots designated for some specific topics), then we can collectively merge pending important pull requests, teaching how to develop plugins, or ask others to look at their plugins, etc. There also can be a valid discussion about JUC, run nicely in exchange of admission fee, vs JUC run cheaply but free.

In any case, I think the quality of presentations were very good, and knowing local Jenkins developers/users would help expand your horizon. As I said in the beginning, we are takin JUC around the world this year. The one in New York is just in the next week, followed by Herzelia (Israel), Tokyo, San Francisco, and Antwerp. Please register while seats are still available!

Continuous Information vol.2

kohsuke — Fri, 13 Apr 2012 22:06:02 +0000

Because I work on Jenkins day in day out, it's easy for me to forget that most people don't pay /that/ much attention to Jenkins. If you fit that category, and if you want to stay on top of the latest happenings in Jenkins, don’t miss Volume 2 of Continuous Information, the CloudBees Newsletter for Jenkins.

This issue...

Features details about the 6 upcoming Jenkins User Conferences (don’t miss these)
Announces the new Jenkins CIA Program (join us to promote Jenkins around the globe)
Shows you where to find in-depth information about the latest Jenkins UI improvements and featured plugins (cool stuff)
Highlights the importance of Jenkins Security Advisories (install these regularly)
Tells you why Jenkins has blue balls instead of green ones (seriously)
Shows you the latest Jenkins Usage Stats (still growing super-fast)
… and more great stuff, including a bit of Jenkins humor (courtesy of our friends at Geek and Poke)

View this issue in full or sign up to receive future newsletters directly or to stay on top of the latest Jenkins goodness.

On somewhat unrelated note, call for Papers for upcoming JUC 2012 is open! Please help us spread the word...

Announcing the Jenkins CIA

rtyler — Mon, 26 Mar 2012 17:01:18 +0000

For years, we've been hearing about covert installations of Jenkins by groups of developers within larger companies. Rogue engineers, frustrated by the lack of continuous integration would download jenkins.war and run it off their workstation. As time went on, word-of-mouth within the organization spread Jenkins far and wide.

Today we announce an initiative to help support these rogue agents: the Jenkins CIA. CIA being short for Continuous Integration Ambassador of course.

If you're going to be speaking at a JUG or another event where you will have the opportunity to promote and teach people about Jenkins, you too can join the CIA:

Send us an email telling us about the event and how many people you expect
Write us a guest blog post ahead of time, talking about the event
We dispatch Jenkins stickers and a CIA Agent shirt for you to wear.
Write up a summary blog post about the event afterwards
Repeat!

In the coming months, we'll start collaborating and creating standard presentations that can be easily re-used to introduce people not only to Jenkins, but continuous integration in general, so stay tuned.

If you're not the speaking type but instead prefer to work behind the scenes, you can join the OSS by checking out the Beginner's Guide to contributing to Jenkins.

- Agent Dero, over and out.

This message will self-destruct in 5.

poof.

Why does Jenkins have blue balls?

rtyler — Tue, 13 Mar 2012 15:00:00 +0000

It is interesting having an open source project that is sufficiently old to start generating "lore" of some form or another. Jenkins is starting to get to be that age, having been started over 6 years ago.

One of the most commonly asked questions, is about Jenkins' use of "blue balls" to indicate success by default. This is enough of an "issue" for some users that the Green Balls plugin is in the list of top 10 installed plugins.

The reason behind our use of blue to indicate success has its basis in Kohsuke's Japanese upbringing. The cultural differences were enumerated in a bug report comically titled "s/blue/green/g" (JENKINS-369):

This response Kohsuke cited was taken from this Q&A thread

Q. "Why do Japanese people say that they have blue traffic lights when they are really green?" --Question submitted by John Sypal

A: According to the book, Japan From A to Z: Mysteries of Everyday Life Explained by James and Michiko Vardaman, the first traffic signals in Japan were blue instead of green, but the blue lights were difficult to see from a long distance away so they were replaced with green ones. Vardaman says that the custom of referring to traffic lights is a holdover from those days.

This sounds like a good explanation, but the problem with it is that you will hear Japanese people refer to other green things (like cucumbers, spinach, and sometimes grass) as being blue as well. This is because historically, Japanese people considered green to be a shade of blue. For example, the Chinese character for blue, pronounced ao is made up of two characters, iki (life) and i (well) and refers to the colour of plants which grow around a well, a colour between green and blue. When Chinese people see the character, they say it means green, but Japanese people say it means blue.

Japanese books on colours tell us that there are four tertiary colours: red, blue, white and black, and that all others are shades of those four main ones. Ao, therefore, is a sort of ideal blue, halfway between green and blue. The sky is said to be blue, but it is a different shade of ao than a traffic light is. Tree leaves are said to be green, but green is a shade of ao, like crimson is a shade of red.

In another interesting cultural difference relating to colour, Japanese children always colour the sun red instead of yellow.

(here's a direct link to Kohsuke's comment)

Unfortunately it's not for color blind users, although that's a pretty convincing explanation. Jenkins has blue balls because in Japan, red means stop and blue means go!

Image courtesy of this site

Critical security advisory in Jenkins core

kohsuke — Tue, 06 Mar 2012 16:00:00 +0000

We've identified and fixed a critical security vulnerabilities in
Jenkins core. This affects all the releases of Jenkins to date (main line releases up to 1.452 and LTS up to 1.424.3.) Please upgrade to the new releases at your earliest convenience, especially if your Jenkins is internet facing.

For more details about the vulnerabilities, affected versions, and so on, please consult the security advisory.

(See our Wiki page about security advisories about how we do these.)

Sponsor a Jenkins User Conference!

rtyler — Tue, 21 Feb 2012 15:30:00 +0000

With the JUC Paris 2012 call for papers open, it's important to mention that we are also looking for sponsors for the various Jenkins User Conferences that are being planned around the world right now!

Currently there are four conferences being put together right now:

If your company is heavily invested in Jenkins or interested in reaching the kind of audience that will be at a JUC (highly technical, motivated) then you should consider becoming a sponsor one of these conferences (link below)!

Sponsor the Jenkins User Conference

FOSDEM 2012 Recap

rtyler — Tue, 21 Feb 2012 13:00:00 +0000

(Editor's note: Apologies for the delay in getting this wrap-up out, it's been quite a busy month!)

This year has already been full of milestones, the first of which being our first birthday as an open source project. The second major milestone for the project was that we went to FOSDEM 2012, arguably the largest volunteer-organized and operated open source conference on the planet.

We had a couple of things going on at FOSDEM that merit a mention:

The Jenkins project had a stand in the K building, the same building where the Free Java, Config and Systems Management, and a few other pertinent dev rooms were located
The Jenkins project gave away 2 free copies of John Smart's book: "Jenkins: The Definitive Guide" (thanks to O'Reilly!)
Community member R. Tyler Croy gave a talk on running the Jenkins project infrastructure with Puppet
The O'Reilly folks brought 10+ Jenkins books to sell at their stand.
Project founder Kohsuke Kawaguchi and a number of project members held a constructive UI Enhancements discussion.

We were very fortunate to have so many Jenkins contributors in attendance, who all helped with the Jenkins stand, introducing people to Jenkins and much more.

FOSDEM in their own words

(after the break)

Nicholas de Loof:

FOSDEM was for me an opportunity to meet other Jenkins contributors I only knew by IRC nickname. Those two days were awesome to discuss with users on our stand, joke and socialize, as well as having some more technical debates and encourage folks to get involved in the community. Even I couldn't attend the talks I selected on the conference agenda due to room being full, I really enjoyed this 100% geek weekend.

Christopher Orr:

The people, the talks, the social events, the sheer size of FOSDEM all make for a pretty inspiring weekend. And incredibly, it's all for free.

You can not help but feel motivated after attending. You always learn something new, discover myriad projects -- in niches you never knew existed -- and talk to smart folk from all over.

Talking to people at the Jenkins stand was no different. Though in a striking number of cases, people had already heard of Jenkins, were big fans and took a clutch of stickers back home for their colleagues.

Speaking with those who weren't yet using Jenkins was equally
interesting. My favourite was talking to one guy who described a
particularly complex workflow; at each step he asked if Jenkins could do it, and I was able to cheerily reply "yes" to every single one.

Getting to put faces to names of Jenkins developers was also a huge plus, and resulted in numerous great conversations.

Domi:

It was great to get some faces to the short names on IRC. Talking to other commiters was awesome, I thought they are cool before I went to FOSDEM, but now I know!

Its was great to talk to people who are using the tools you are working on, there where so many just coming up to say thank you! (?and there where/are by far more then I thought!) I know I'm standing on the shoulders of a giant - but I also do feel that my commitment is of value for others too.

Feels great to be part of this community!

R Tyler Croy:

I spent so much time at the stand telling people about Jenkins or showing them, that I only ended up seeing a couple actual sessions the whole weekend. The kinds of people who came to talk to us almost entirely developers of one kind or another, which was really great to talk about how Jenkins can be used effectively for Perl shops, for Python, C++, C#, Java (of course) or even for deployment automation. The spread was a pretty big endorsement, I think, of the extensible nature of the Jenkins plugin ecosystem. In planning for FOSDEM I had urged Kohsuke to order thousands of stickers for the event, and when all was said and done I think we had given away around 1000 stickers to new Jenkins fans, old Jenkins fans and a few folks in the community who were looking forward to going back to their local JUG to share.

I'm looking forward to making the trip to the bitter cold of Brussels in February again next year.

Fred G:

FOSDEM had lots of interesting talks and was very well organized (from my point of view) and best of all... it's free! Apart from the talks and nice lineup of speakers, it has been a great opportunity to meet people. People I already knew, some in person, some only from IRC, the mailing list, or as a maintainer of a plugin; but also people thatcame up to the Jenkins stand.

From the people that I talked to: - 60% knew Jenkins and use it every day ("Yeah, I know Jenkins. The whole company uses it and nothing works without it!") - 35% had heard of it or were very interested ("I know Jenkins, but we bought the Atlassian package and now we have to use Bamboo, Jira and Confluence." [me mentioning the Jira integration of Jenkins] "Wow, I really have to try that out and convince our team to switch to Jenkins!". - 5% Weirdos and WTF!? (disgusted "Is this another one of these
projects funded by Red Hat? They fund everything!")

Meeting Tom Huybrechts without knowing it (at first) was a big
suprise. He has created or contributed to some of the best plugins
(eg. the parameterized-trigger plugin) and I see his name at least
once whenever I browse through core source code. During the impromptu UI enhancement meetings he showed us another three plugins that he wrote but never made public just because he doesn't have time to support them all.

Then at the stand he casually mentioned that he is administering
around 3000 jobs on 100+ build machines. At the same time he seems
like a very humble and low key character.
To sum it up, the best thing about an open source project like Jenkins is the community. Working together with nice people from all over the world to create the best CI server has been a great experience. The FOSDEM weekend was another event that proved that. I'll definitely come back.

Kohsuke Kawaguchi:

FOSDEM is one of the few conferences that has a distinctive hand-made geek-for-geek feeling to it. No marketing people, no bullshit flyers, but lots of technical folks and good beer. I really enjoyed talking to users, as always, but above all I was very happy to see developers and project members in the Jenkins community come out in full force, and I felt they enjoyed it just as much. I'm really hoping that we'll now keep this going for years to come.

I think it's safe to assume we'll be back next year for FOSDEM 2013, hope to see you there!

Jenkins User Conference 2012 Paris

ndeloof — Mon, 13 Feb 2012 08:01:00 +0000

(English text follows the French text)

Après le succès de la Jenkins User Conference l’an dernier à San Francisco et à l’intérêt qu’elle a soulevé, nous organisons cette année la JUC dans quatre grande villes à travers le monde. La premiére étape de cette tournée est la JUC 2012 à Paris, le 17 avril. La conférence aura lieu la veille de Devoxx France dans les mêmes locaux. La date a été spécialement choisie pour que vous puissiez faire d’une pierre deux coups, ou plutôt deux confs !

Les inscriptions sont ouvertes. Inscrivez-vous dès maintenant pour bénéficier de la réduction réservée au plus enthousiastes. Vous pouvez également proposer un sujet ou bien soutenir la conférence en tant que sponsor. Comme il est bien connu que les Français aiment voir leurs compatriotes sur scène, j’espère que nos nombreux développeurs jenkins francophones (les meilleurs, soit dit en passant) proposeront un sujet !

Cette année, nous demandons une contribution pour les inscriptions (avec un tarif réduit avant le 18 février), afin de couvrir les frais d’organisation, mais nous vous offrirons en contrepartie le T-shirt officiel ainsi que des autocollants Jenkins. Si on se base sur la JUC 2011, SCALE 10x, ou le FOSDEM, ce sera une opporunité exceptionnelle pour rencontrer et discuter avec les dévelopeurs majeurs du coeur ou des plugins, ainsi que les nombreux utilisateurs prêts à partager leur expérience et à répondre à toutes vos questions.

Thanks to the success of the Jenkins User Conference last year at San Francisco and high interest, this year we are bringing JUC to 4 cities around the world. And the first stop is JUC 2012 in Paris, on April 17. This is one day before Devoxx France, and in the same venue. The date is specifically chosen so that you can kill two conferences in one stone!

Register for JUC Paris now (http://www.cloudbees.com/juc2012.cb) and get the Early Bird discount - which is a significant reduction in the registration fee! The conference is accepting registrations as well as looking for talk submissions and sponsors. I get the impression that French people like other fellow French speaking, so I hope our French-speaking plugin developers will submit talks.

This year, we are charging a small amount of money in the hope of covering the expense, but we'll give out T-shirts (which were really hot last year!) as well as stickers. And if the experience at JUC 2011, SCALE 10x, and FOSDEM was any indication, this is a great opportunity to meet and talk with plugin/core developers, and other fellow users with whom you can discuss your experience/questions.

Happy birthday Jenkins!

rtyler — Thu, 02 Feb 2012 09:08:25 +0000

On February 2nd, 2011 the first release of Jenkins, version 1.396, was made available for public consumption. Thus marking a new beginning for many of us who had come to rely on this very versatile piece of software and wanted to see it continue to thrive.

Along with some other bug fixes, the 1.396 release of Jenkins included a very important changelog item:

Fixed a trademark bug that caused a considerable fiasco by renaming to Jenkins

On behalf of the core Jenkins team and the governance board I would like to extend a extremely large Thank You! to all of the plugin developers, bug filers, wiki page editors, book authors and the users who have helped grow Jenkins into the project it is today.

Some of the tidbits from our highlight reel:

As of this writing there have been 54 releases of Jenkins
Jenkins now supports writing plugins in Ruby as well as Java (more languages in the process)
We have 7 high-speed mirrors streaming Jenkins packages to users around the world.
There are now over 450 different plugins available for Jenkins
Over 80 donors participated in our end of year fundraising drive
5 "Long Term Support" releases have been published by the Jenkins community, offering users a slower moving upgrade target (supported even further by CloudBees' Enterprise Jenkins product)
Public project governance meetings are held and recorded (almost) every couple of weeks.
More than 340 individuals contribute on GitHub to the project in some form or another.
About 750 members of the developers mailing list and around 1700 on the users mailing list

There are many other impressive sounding numbers I could rattle off, but the list is far too long to be interesting.

The project isn't perfect and nor is the software, but we're off to a fine start and I hope you'll join us in making this next year of Jenkins even better than the first.