www.thoughtworks.com aggregator

Patrick Farley: Getting The very best Offer When Shopping Online

Wed, 10 Dec 2014 13:15:38 +0000

Whether you have to get a vehicle or some groceries, anything can be purchased through the Web it appears. You are likewise able to patronize a brand name’s website, merchant sites, and auctions online. New or utilized, you can purchase almost everything at a discount rate. Follow this info to discover and benefit from good deals online.

A terrific suggestion to keep in mind when you’re considering online shopping is to ensure the website is official. You do not wish to be providing your charge card info away to somebody who might possibly rip-off you. Constantly take a look at the website to inform if it’s expert and safe to utilize.

Try to find discount coupons prior to buying an item online. Many online establishments constantly have some advertising offers going on. If you do not discover any details about existing offers on the website of the online establishment, view other websites. Clients often share their voucher codes on message boards or couponning blog sites.

Make sure where you send your charge card details. When you are going shopping online, constantly utilize safe and secure websites. These are websites that have actually taken steps to secure your important info. A protected web site will certainly start with HTTPS instead of the basic HTTP. There will certainly likewise be a locked icon, either in your address or condition bar, depending upon which browser you make use of.

If you intend on buying a product that you could wish to return, patronize a shop that provides totally free returns or exchanges. Lots of online shops desire your company so they will certainly provide things like totally free returns or exchanges. This alternative is fantastic if you are not sure if a product will certainly fit your requirements.

If you’re paying too much with expedited shipping online, attempt to be client by utilizing basic shipping. The speed of conventional shipping simply may shock you. Plus, you conserve a lot cash by waiting simply a couple days more.

If an offer appears too great to be real, it is. Remember that any individual can introduce an online establishment and not all establishments all trustworthy. Do more study on various establishments prior to choosing where you wish to acquire your items from. Do pass by a really inexpensive item if it is offered by an establishment with a bad credibility and reputation.

Just store at online retail websites that you trust. It is all too simple for somebody to simply produce a store on the internet with some item info to offer product. Do some study on the seller’s credibility and reputation prior to you supply any charge card details. This will certainly assist you prevent any frauds out there.

Keep in mind to think about not just the product’s real expense, however likewise just how much will certainly be charged for shipping. A product could cost a couple of dollars more at one website, however if they provide totally free shipping, it may wind up being cheaper total. If shipping costs aren’t provided in advance, you can examine them by starting the checkout procedure. The shipping expense will certainly be revealed prior to the order is full.

In order to minimize shipping costs, it is best to obtain everything you require in one order. If you do refrain it by doing this, you will certainly need to pay a different shipping cost for each order you make. This will certainly wind up costing you a fortune in the long run.

Pick online shops which provide a live chat alternative. Live chat is a quicker, easier method to deal with a concerns or concerns you may have. In addition, you might be able to work out a much better offer using Live Chat. Numerous websites provide such discount rates or totally free shipping if you make your order right then, as opposed to waiting.

When we go to the establishment to acquire a product, we typically end up leaving with a couple of impulse purchases also. These products can swiftly build up and take a toll on your checking account! Purchasing things online can be an exceptional method to reduce the quantity of impulse purchases that you make.

Purchasing presents and vacation shopping can be made a lot simpler using the web. You can acquire a product and have it delivered to the recipient without ever laving your house. You can likewise decide to purchase a gift-card to a site that you understand has products that they will certainly such as. This can conserve you money and time and guarantee that the recipient will certainly enjoy your present and consideration.

If you lead a busy way of life, or work odd hours, it can frequently be tough to make it to the establishment prior to they close. Online shopping, nevertheless, is something that you can do at any hour of the day (or night!), and might for that reason be a more practical option for you.

Swirlio desert maker. Buy now

There’s constantly brand-new techniques to discover, no matter how skilled a consumer you are. A little details can make a huge distinction in the kind of offer you have the ability to discover. Read this short article and others like it if you wish to discover brand-new and interesting strategies to refine your strategy to online shopping.

Amazon.com best online store

The post Getting The very best Offer When Shopping Online appeared first on Save Big Online Shopping.

Trisha Gee: New Blog Post: New Year, New Adventures

Wed, 10 Dec 2014 11:16:00 +0000

New Blog Post: New Year, New Adventures

Brian Oxley: Java validation

Tue, 09 Dec 2014 21:54:00 +0000

Martin Fowler posted Replacing Throwing Exceptions with Notification in Validations discussing alternatives to data validation than throwing exceptions. There are off-the-shelf solutions such as Commons Validator (XML driven) or Bean Validation (annotation driven) which are complete frameworks.

There is more to these frameworks than I suggest, but to explore Fowler's post better I quickly coded up my own simple-minded approach:

public final class ValidationMain {
    public static void main(final String... args) {
        final Notices notices = new Notices();
        notices.add("Something went horribly wrong %d time(s)", 1);
        try {
            foo(null);
        } catch (final Exception e) {
            notices.add(e);
        }
        notices.forEach(err::println);
        notices.fail(IllegalArgumentException::new);
    }

    public static String foo(final Object missing) {
        return missing.toString();
    }
}

Output on stderr:

lab.Notices$Notice@27f8302d
lab.Notices$Notice@4d76f3f8
Exception in thread "main" java.lang.IllegalArgumentException: 2 notices:
 Something went horribly wrong 1 time(s)
 at lab.ValidationMain.main(ValidationMain.java:21)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:483)
 at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
 Suppressed: java.lang.IllegalArgumentException: Only 1 reason(s)
  at lab.ValidationMain.main(ValidationMain.java:14)
 Suppressed: java.lang.IllegalArgumentException
  at lab.ValidationMain.main(ValidationMain.java:18)
 Caused by: java.lang.NullPointerException
  at lab.ValidationMain.foo(ValidationMain.java:25)
  at lab.ValidationMain.main(ValidationMain.java:16)

And the Notices class:

public final class Notices
        implements Iterable<Notice> {
    private final List<Notice> notices = new ArrayList<>(0);

    public void add(final String reason, final Object... args) {
        // Duplicate code so stack trace keeps same structure
        notices.add(new Notice(null, reason, args));
    }

    public void add(final Throwable cause) {
        // Duplicate code so stack trace keeps same structure
        notices.add(
                new Notice(cause, null == cause ? null : cause.getMessage()));
    }

    public void add(final Throwable cause, final String reason,
            final Object... args) {
        // Duplicate code so stack trace keeps same structure
        notices.add(new Notice(cause, reason, args));
    }

    public <E extends Exception> void fail(
            final BiFunction<String, Throwable, E> ctor)
            throws E {
        final E e = ctor.apply(toString(), null);
        notices.forEach(n -> e.addSuppressed(n.as(ctor)));
        final List<StackTraceElement> frames = asList(e.getStackTrace());
        // 2 is the magic number: lambda, current
        e.setStackTrace(frames.subList(2, frames.size())
                .toArray(new StackTraceElement[frames.size() - 2]));
        throw e;
    }

    @Override
    public Iterator<Notice> iterator() {
        return unmodifiableList(notices).iterator();
    }

    @Override
    public String toString() {
        if (notices.isEmpty())
            return "0 notices";
        final String sep = lineSeparator() + "\t";
        return notices.stream().
                map(Notice::reason).
                filter(Objects::nonNull).
                collect(joining(sep,
                        format("%d notices:" + sep, notices.size()), ""));
    }

    public static final class Notice {
        // 4 is the magic number: Thread, init, init, addError, finally the
        // user code
        private final StackTraceElement location = currentThread()
                .getStackTrace()[4];
        private final Throwable cause;
        private final String reason;
        private final Object[] args;

        private Notice(final Throwable cause, final String reason,
                final Object... args) {
            this.cause = cause;
            this.reason = reason;
            this.args = args;
        }

        public Throwable cause() {
            return cause;
        }

        public String reason() {
            return null == reason ? null : format(reason, args);
        }

        private <E extends Exception> E as(
                final BiFunction<String, Throwable, E> ctor) {
            final E e = ctor.apply(reason(), cause);
            e.setStackTrace(new StackTraceElement[]{location});
            return e;
        }
    }
}

Comments:

I manipulate the stack traces to focus on the caller's point of view. This is the opposite of, say, Spring Framework.
I haven't decided on what an intelligent toString() should look like for Notice
Java 8 lambdas really shine here. Being able to use exception constructors as method references is a win.

Mark Needham: R: Cleaning up and plotting Google Trends data

Tue, 09 Dec 2014 18:14:45 +0000

I recently came across an excellent article written by Stian Haklev in which he describes things he wishes he’d been told before starting out with R, one being to do all data clean up in code which I thought I’d give a try.

My goal is to leave the raw data completely unchanged, and do all the transformation in code, which can be rerun at any time.

While I’m writing the scripts, I’m often jumping around, selectively executing individual lines or code blocks, running commands to inspect the data in the REPL (read-evaluate-print-loop, where each command is executed as soon as you type enter, in the picture above it’s the pane to the right), etc.

But I try to make sure that when I finish up, the script is runnable by itself.

I thought the Google Trends data set would be an interesting one to play around with as it gives you a CSV containing several different bits of data of which I’m only interested in ‘interest over time’.

It’s not very easy to automate the download of the CSV file so I did that bit manually and automated everything from there onwards.

The first step was to read the CSV file and explore some of the rows to see what it contained:

> library(dplyr)
 
> googleTrends = read.csv("/Users/markneedham/Downloads/report.csv", row.names=NULL)
 
> googleTrends %>% head()
##                   row.names Web.Search.interest..neo4j
## 1 Worldwide; 2004 - present                           
## 2        Interest over time                           
## 3                      Week                      neo4j
## 4   2004-01-04 - 2004-01-10                          0
## 5   2004-01-11 - 2004-01-17                          0
## 6   2004-01-18 - 2004-01-24                          0
 
> googleTrends %>% sample_n(10)
##                   row.names Web.Search.interest..neo4j
## 109 2006-01-08 - 2006-01-14                          0
## 113 2006-02-05 - 2006-02-11                          0
## 267 2009-01-18 - 2009-01-24                          0
## 199 2007-09-30 - 2007-10-06                          0
## 522 2013-12-08 - 2013-12-14                         88
## 265 2009-01-04 - 2009-01-10                          0
## 285 2009-05-24 - 2009-05-30                          0
## 318 2010-01-10 - 2010-01-16                          0
## 495 2013-06-02 - 2013-06-08                         79
## 28  2004-06-20 - 2004-06-26                          0
 
> googleTrends %>% tail()
##                row.names Web.Search.interest..neo4j
## 658        neo4j example                   Breakout
## 659 neo4j graph database                   Breakout
## 660           neo4j java                   Breakout
## 661           neo4j node                   Breakout
## 662           neo4j rest                   Breakout
## 663       neo4j tutorial                   Breakout

We only want to keep the rows which contain (week, interest) pairs so the first thing we’ll do is rename the columns:

names(googleTrends) = c("week", "score")

Now we want to strip out the rows which don’t contain (week, interest) pairs. The easiest way to do this is to look for rows which don’t contain date values in the ‘week’ column.

First we need to split the start and end dates in that column by using the strsplit function.

I found it much easier to apply the function to each row individually rather than passing in a list of values so I created a dummy column with a row number in to allow me to do that (a trick Antonios showed me):

> googleTrends %>% 
    mutate(ind = row_number()) %>% 
    group_by(ind) %>%
    mutate(dates = strsplit(week, " - "),
           start = dates[[1]][1] %>% strptime("%Y-%m-%d") %>% as.character(),
           end =   dates[[1]][2] %>% strptime("%Y-%m-%d") %>% as.character()) %>%
    head()
## Source: local data frame [6 x 6]
## Groups: ind
## 
##                        week score ind    dates      start        end
## 1 Worldwide; 2004 - present     1   1 <chr[2]>         NA         NA
## 2        Interest over time     1   2 <chr[1]>         NA         NA
## 3                      Week    90   3 <chr[1]>         NA         NA
## 4   2004-01-04 - 2004-01-10     3   4 <chr[2]> 2004-01-04 2004-01-10
## 5   2004-01-11 - 2004-01-17     3   5 <chr[2]> 2004-01-11 2004-01-17
## 6   2004-01-18 - 2004-01-24     3   6 <chr[2]> 2004-01-18 2004-01-24

Now we need to get rid of the rows which have an NA value for ‘start’ or ‘end':

> googleTrends %>% 
    mutate(ind = row_number()) %>% 
    group_by(ind) %>%
    mutate(dates = strsplit(week, " - "),
           start = dates[[1]][1] %>% strptime("%Y-%m-%d") %>% as.character(),
           end =   dates[[1]][2] %>% strptime("%Y-%m-%d") %>% as.character()) %>%
    filter(!is.na(start) | !is.na(end)) %>% 
    head()
## Source: local data frame [6 x 6]
## Groups: ind
## 
##                      week score ind    dates      start        end
## 1 2004-01-04 - 2004-01-10     3   4 <chr[2]> 2004-01-04 2004-01-10
## 2 2004-01-11 - 2004-01-17     3   5 <chr[2]> 2004-01-11 2004-01-17
## 3 2004-01-18 - 2004-01-24     3   6 <chr[2]> 2004-01-18 2004-01-24
## 4 2004-01-25 - 2004-01-31     3   7 <chr[2]> 2004-01-25 2004-01-31
## 5 2004-02-01 - 2004-02-07     3   8 <chr[2]> 2004-02-01 2004-02-07
## 6 2004-02-08 - 2004-02-14     3   9 <chr[2]> 2004-02-08 2004-02-14

Next we’ll get rid of ‘week’, ‘ind’ and ‘dates’ as we aren’t going to need those anymore:

> cleanGoogleTrends = googleTrends %>% 
    mutate(ind = row_number()) %>% 
    group_by(ind) %>%
    mutate(dates = strsplit(week, " - "),
           start = dates[[1]][1] %>% strptime("%Y-%m-%d") %>% as.character(),
           end =   dates[[1]][2] %>% strptime("%Y-%m-%d") %>% as.character()) %>%
    filter(!is.na(start) | !is.na(end)) %>%
    ungroup() %>%
    select(-c(ind, dates, week))
 
> cleanGoogleTrends %>% head()
## Source: local data frame [6 x 3]
## 
##   score      start        end
## 1     3 2004-01-04 2004-01-10
## 2     3 2004-01-11 2004-01-17
## 3     3 2004-01-18 2004-01-24
## 4     3 2004-01-25 2004-01-31
## 5     3 2004-02-01 2004-02-07
## 6     3 2004-02-08 2004-02-14
 
> cleanGoogleTrends %>% sample_n(10)
## Source: local data frame [10 x 3]
## 
##    score      start        end
## 1      8 2010-09-26 2010-10-02
## 2     73 2013-11-17 2013-11-23
## 3     52 2012-07-01 2012-07-07
## 4      3 2005-06-19 2005-06-25
## 5      3 2004-12-12 2004-12-18
## 6      3 2009-09-06 2009-09-12
## 7     71 2014-09-14 2014-09-20
## 8      3 2004-12-26 2005-01-01
## 9     62 2013-03-03 2013-03-09
## 10     3 2006-03-19 2006-03-25
 
> cleanGoogleTrends %>% tail()
## Source: local data frame [6 x 3]
## 
##   score      start        end
## 1    80 2014-10-19 2014-10-25
## 2    80 2014-10-26 2014-11-01
## 3    84 2014-11-02 2014-11-08
## 4    81 2014-11-09 2014-11-15
## 5    83 2014-11-16 2014-11-22
## 6     2 2014-11-23 2014-11-29

Ok now we’re ready to plot. This was my first attempt:

> library(ggplot2)
> ggplot(aes(x = start, y = score), data = cleanGoogleTrends) + 
    geom_line(size = 0.5)
## geom_path: Each group consist of only one observation. Do you need to adjust the group aesthetic?

As you can see, not too successful! The first mistake I’ve made is not telling ggplot that the ‘start’ column is a date and so it can use that ordering when plotting:

> cleanGoogleTrends = cleanGoogleTrends %>% mutate(start =  as.Date(start))
> ggplot(aes(x = start, y = score), data = cleanGoogleTrends) + 
    geom_line(size = 0.5)

My next mistake is that ‘score’ is not being treated as a continuous variable and so we’re ending up with this very strange looking chart. We can see that if we call the class function:

> class(cleanGoogleTrends$score)
## [1] "factor"

Let’s fix that and plot again:

> cleanGoogleTrends = cleanGoogleTrends %>% mutate(score = as.numeric(score))
> ggplot(aes(x = start, y = score), data = cleanGoogleTrends) + 
    geom_line(size = 0.5)

That’s much better but there is quite a bit of noise in the week to week scores which we can flatten a bit by plotting a rolling mean of the last 4 weeks instead:

> library(zoo)
> cleanGoogleTrends = cleanGoogleTrends %>% 
    mutate(rolling = rollmean(score, 4, fill = NA, align=c("right")),
           start =  as.Date(start))
 
> ggplot(aes(x = start, y = rolling), data = cleanGoogleTrends) + 
    geom_line(size = 0.5)

Here’s the full code if you want to reproduce:

library(dplyr)
library(zoo)
library(ggplot2)
 
googleTrends = read.csv("/Users/markneedham/Downloads/report.csv", row.names=NULL)
names(googleTrends) = c("week", "score")
 
cleanGoogleTrends = googleTrends %>% 
  mutate(ind = row_number()) %>% 
  group_by(ind) %>%
  mutate(dates = strsplit(week, " - "),
         start = dates[[1]][1] %>% strptime("%Y-%m-%d") %>% as.character(),
         end =   dates[[1]][2] %>% strptime("%Y-%m-%d") %>% as.character()) %>%
  filter(!is.na(start) | !is.na(end)) %>%
  ungroup() %>%
  select(-c(ind, dates, week)) %>%
  mutate(start =  as.Date(start),
         score = as.numeric(score),
         rolling = rollmean(score, 4, fill = NA, align=c("right")))
 
ggplot(aes(x = start, y = rolling), data = cleanGoogleTrends) + 
  geom_line(size = 0.5)

My next step is to plot the Google Trends scores against my meetup data set to see if there’s any interesting correlations going on.

As an aside I made use of knitr while putting together this post – it works really well for checking that you’ve included all the steps and that it actually works!

Joanne Cranford: Let your users direct the tech

Tue, 09 Dec 2014 01:32:54 +0000

I was trying to explain to my co-founder Liz last night the changes that I’d been making to Cake, the system that we use to process enquiries from our website, when she frowned and said, “that sounds really complicated.”

It stopped me in my tracks – technology is supposed to be enabling our company to move faster! She went on, “Trello is so simple in comparison!” She’s right on that – Trello IS simple but it only covers one part of of the process, while Cake is trying to do all the things. But it was time to stop ignoring that niggling feeling that I was losing my direction on what I was building, and revisit the system with the person who actually uses it every day.

So co-founder-2 (Phil) and I sat down today, hooked his laptop up to a large TV screen and started putting orders through on the staging site, all the way from the customer placing them, through to the supplier accepting them, and the customer confirming and paying. I played the part of the customer, and Phil took on the two hats of the supplier and himself, doing his daily job of processing orders for You Chews. It was a bit confusing – would have been better with Liz to act as supplier!

Things started off slowly. I’m using a Kanban system to pull changes through to production, and I’m releasing changes sometimes several times a day. For Phil, that means that the system and processes he uses are changing every day – and even when it seems to me like it’s for the better, sometimes it’s just a PITA. It’s not until you see somebody else using the system that you built that you start to see whether or not it will actually work!

The first order was a bit of a struggle. It became obvious fairly quickly that Cake needs to update the order status automatically, that having that as an extra step makes it significantly more confusing.

With the second order, we discovered some bugs in the steps to email the supplier – luckily they were just config issues, so I fixed them, and we were off again.

The third and fourth orders started to go more smoothly. Phil was following the steps he had noted down for himself, knew that there would be upcoming changes to automate the status updates, and asked loads of questions. He consistently deleted one part of the email that would go to suppliers, so it was really clear that it wasn’t needed. Towards the end of the meeting, he was even getting excited – “This is great! This is great!” – he even said at one point. We discovered a ton of places where small changes like wording could make the process smoother, and I walked away with a much clearer vision of what I need to create to make the system useful for him. Phil even left saying that he’d enjoyed the meeting – we spent about two hours in that room, so believe me that is quite an acheivement!

It reinforced a few things for me. Firstly, and most obviously, test the system with real users. They can be customers or staff – and don’t just do a quick test, let them use it a few times, see if you can provide help and direction and if that makes a difference, and watch what they do. The more they use the system, the more useful feedback they’ll be able to provide you.

Secondly, if you’re working in a fast-paced environment, constantly pushing through changes, it’s a good thing to stop and review the bigger picture every so often. It doesn’t need to be a big, formal process – I spent five minutes sketching a diagram of what I was trying to do, and fifteen minutes talking it through, and was far more confident that I was heading in the right direction.

Lastly: if you are going to do user testing, expect to walk away with a longer to-do list!

Mark Needham: R: dplyr – mutate with strptime (incompatible size/wrong result size)

Mon, 08 Dec 2014 19:02:46 +0000

Having worked out how to translate a string into a date or NA if it wasn’t the appropriate format the next thing I wanted to do was store the result of the transformation in my data frame.

I started off with this:

data = data.frame(x = c("2014-01-01", "2014-02-01", "foo"))
> data
           x
1 2014-01-01
2 2014-02-01
3        foo

And when I tried to do the date translation ran into the following error:

> data %>% mutate(y = strptime(x, "%Y-%m-%d"))
Error: wrong result size (11), expected 3 or 1

As I understand it this error is telling us that we are trying to put a value into the data frame which represents 11 rows rather than 3 rows or 1 row.

It turns out that storing POSIXlts in a data frame isn’t such a good idea! In this case we can use the as.character function to create a character vector which can be stored in the data frame:

> data %>% mutate(y = strptime(x, "%Y-%m-%d") %>% as.character())
           x          y
1 2014-01-01 2014-01-01
2 2014-02-01 2014-02-01
3        foo       <NA>

We can then get rid of the NA row by using the is.na function:

> data %>% mutate(y = strptime(x, "%Y-%m-%d") %>% as.character()) %>% filter(!is.na(y))
           x          y
1 2014-01-01 2014-01-01
2 2014-02-01 2014-02-01

And a final tweak so that we have 100% pipelining goodness:

> data %>% 
    mutate(y = x %>% strptime("%Y-%m-%d") %>% as.character()) %>%
    filter(!is.na(y))
           x          y
1 2014-01-01 2014-01-01
2 2014-02-01 2014-02-01

Mark Needham: R: String to Date or NA

Sun, 07 Dec 2014 19:29:01 +0000

I’ve been trying to clean up a CSV file which contains some rows with dates and some not – I only want to keep the cells which do have dates so I’ve been trying to work out how to do that.

My first thought was that I’d try and find a function which would convert the contents of the cell into a date if it was in date format and NA if not. I could then filter out the NA values using the is.na function.

I started out with the as.Date function…

> as.Date("2014-01-01")
[1] "2014-01-01"
 
> as.Date("foo")
Error in charToDate(x) : 
  character string is not in a standard unambiguous format

…but that throws an error if we have a non date value so it’s not so useful in this case.

Instead we can make use of the strptime function which does exactly what we want:

> strptime("2014-01-01", "%Y-%m-%d")
[1] "2014-01-01 GMT"
 
> strptime("foo", "%Y-%m-%d")
[1] NA

We can then feed those values into is.na..

> strptime("2014-01-01", "%Y-%m-%d") %>% is.na()
[1] FALSE
 
> strptime("foo", "%Y-%m-%d") %>% is.na()
[1] TRUE

…and we have exactly the behaviour we were looking for.

Daniel Wildt: Desconf 2014? Sim!

Sun, 07 Dec 2014 11:00:00 +0000

Yes! Teremos Desconf 2014 em Porto Alegre. A data é dia 16 de dezembro, a partir das 19h, no Bar Carmelita (na Cidade Baixa). Teremos 15 ou mais palestrantes, show de música e janta para o networking acontecer.

Quer garantir a sua vaga e ver mais informações? Veja aqui.

Quer ver palestras de 2010?

Tagged: desconf, empreendedorismo, evento

Brian Oxley: Writing your own lombok annotation

Sat, 06 Dec 2014 04:59:00 +0000

It took me quite a while to get around to writing my own lombok annotation and processor. This took more effort than I expected, hopefully this post will save someone else some.

tl;dr — Look at the source in my Github repo.

Motivation

Reading the excellent ExecutorService - 10 tips and tricks by Tomasz Nurkiewicz, I thought about tip #2, Switch names according to context, which recommends wrapping important methods and code blocks with custom thread names to aid in logging and debugging.

"This is a great use case for annotations!" I thought. The code screams boilerplate:

public void doNiftyThings() {
    final Thread thread = Thread.currentThread();
    final String oldName = thread.getName();
    thread.setName("Boy this is nifty!");
    try {
        // Do those nifty things - the actual work
    } finally {
        thread.setName(oldName);
    }
}

The whole point of the method is indented out of focus, wrapped with bookkeeping. I'd rather write this:

@ThreadNamed("Boy this is nifty!")
public void doNiftyThings() {
    // Do those nifty thing - the actual work
}

Bonus: simple text search finds those places in my code where I change the thread name based on context.

Writing the annotation

Ok, let's make this work. I started with cloning the @Cleanup annotation and processor, and editing from there. First the annotation, the easy bit. I include the javadoc to emphasize the importance of documenting your public APIs.

/**
 * {@code ThreadNamed} sets the thread name during method execution, restoring
 * it when the method completes (normally or exceptionally).
 *
 * @author <a href="mailto:binkley@alumni.rice.edu">B. K. Oxley (binkley)</a>
 */
@Documented
@Retention(SOURCE)
@Target({CONSTRUCTOR, METHOD})
public @interface ThreadNamed {
    /** The name for the thread while the annotated method executes. */
    String value();
}

Nothing special here. I've made the decision to limit the annotation to methods and constructors. Ideally I'd include blocks but that isn't an option (yet) in Java, and you can always refactor out a block to a method.

Writing the processor

This is the serious part. First some preliminaries:

I have only implemented support for JDK javac. Lombok also supports the Eclipse compiler, which requires a separate processor class. I have nothing against Eclipse, but it's not in my toolkit.
I'll discuss library dependencies below. For now pretend these are already working for you.
I'm a big fan of static imports, the diamond operator, etc. I don't like retyping what the compiler is already thinking. You should note List below is not java.util.List; it's com.sun.tools.javac.util.List. Yeah, I don't know this class either.
The implementation is hard to follow. Most of us don't spend much time with expression trees, which is how most compilers (including javac) see your source code. A language like LISP lets you write you code as the expression tree directly, which is both nifty and challenging (macros being like annotation processors).

Without further ado:

/**
 * Handles the {@code lombok.ThreadNamed} annotation for javac.
 */
@MetaInfServices(JavacAnnotationHandler.class)
@HandlerPriority(value = 1024)
// 2^10; @NonNull must have run first, so that we wrap around the
// statements generated by it.
public class HandleThreadNamed
        extends JavacAnnotationHandler<ThreadNamed> {
    /**
     * lombok configuration: {@code lab.lombok.threadNamed.flagUsage} = {@code
     * WARNING} | {@code ERROR}.
     * <p>
     * If set, <em>any</em> usage of {@code @ThreadNamed} results in a warning
     * / error.
     */
    public static final ConfigurationKey<FlagUsageType>
            THREAD_NAMED_FLAG_USAGE = new ConfigurationKey<FlagUsageType>(
            "lab.lombok.threadNamed.flagUsage",
            "Emit a warning or error if @ThreadNamed is used.") {
    };

    @Override
    public void handle(final AnnotationValues<ThreadNamed> annotation,
            final JCAnnotation ast, final JavacNode annotationNode) {
        handleFlagUsage(annotationNode, THREAD_NAMED_FLAG_USAGE,
                "@ThreadNamed");

        deleteAnnotationIfNeccessary(annotationNode, ThreadNamed.class);
        final String threadName = annotation.getInstance().value();
        if (threadName.isEmpty()) {
            annotationNode.addError("threadName cannot be the empty string.");
            return;
        }

        final JavacNode owner = annotationNode.up();
        switch (owner.getKind()) {
        case METHOD:
            handleMethod(annotationNode, (JCMethodDecl) owner.get(),
                    threadName);
            break;
        default:
            annotationNode.addError(
                    "@ThreadNamed is legal only on methods and constructors"
                            + ".");
            break;
        }
    }

    public void handleMethod(final JavacNode annotation,
            final JCMethodDecl method, final String threadName) {
        final JavacNode methodNode = annotation.up();

        if ((method.mods.flags & Flags.ABSTRACT) != 0) {
            annotation.addError(
                    "@ThreadNamed can only be used on concrete methods.");
            return;
        }

        if (method.body == null || method.body.stats.isEmpty()) {
            generateEmptyBlockWarning(annotation, false);
            return;
        }

        final JCStatement constructorCall = method.body.stats.get(0);
        final boolean isConstructorCall = isConstructorCall(constructorCall);
        List<JCStatement> contents = isConstructorCall
                ? method.body.stats.tail : method.body.stats;

        if (contents == null || contents.isEmpty()) {
            generateEmptyBlockWarning(annotation, true);
            return;
        }

        contents = List
                .of(buildTryFinallyBlock(methodNode, contents, threadName,
                        annotation.get()));

        method.body.stats = isConstructorCall ? List.of(constructorCall)
                .appendList(contents) : contents;
        methodNode.rebuild();
    }

    public void generateEmptyBlockWarning(final JavacNode annotation,
            final boolean hasConstructorCall) {
        if (hasConstructorCall)
            annotation.addWarning(
                    "Calls to sibling / super constructors are always "
                            + "excluded from @ThreadNamed;"
                            + " @ThreadNamed has been ignored because there"
                            + " is no other code in " + "this constructor.");
        else
            annotation.addWarning(
                    "This method or constructor is empty; @ThreadNamed has "
                            + "been ignored.");
    }

    public JCStatement buildTryFinallyBlock(final JavacNode node,
            final List<JCStatement> contents, final String threadName,
            final JCTree source) {
        final String currentThreadVarName = "$currentThread";
        final String oldThreadNameVarName = "$oldThreadName";

        final JavacTreeMaker maker = node.getTreeMaker();
        final Context context = node.getContext();

        final JCVariableDecl saveCurrentThread = createCurrentThreadVar(node,
                maker, currentThreadVarName);
        final JCVariableDecl saveOldThreadName = createOldThreadNameVar(node,
                maker, currentThreadVarName, oldThreadNameVarName);

        final JCStatement changeThreadName = setThreadName(node, maker,
                maker.Literal(threadName), currentThreadVarName);
        final JCStatement restoreOldThreadName = setThreadName(node, maker,
                maker.Ident(node.toName(oldThreadNameVarName)),
                currentThreadVarName);

        final JCBlock tryBlock = setGeneratedBy(maker.Block(0, contents),
                source, context);
        final JCTry wrapMethod = maker.Try(tryBlock, nil(),
                maker.Block(0, List.of(restoreOldThreadName)));

        if (inNetbeansEditor(node)) {
            //set span (start and end position) of the try statement and
            // the main block
            //this allows NetBeans to dive into the statement correctly:
            final JCCompilationUnit top = (JCCompilationUnit) node.top()
                    .get();
            final int startPos = contents.head.pos;
            final int endPos = Javac
                    .getEndPosition(contents.last().pos(), top);
            tryBlock.pos = startPos;
            wrapMethod.pos = startPos;
            Javac.storeEnd(tryBlock, endPos, top);
            Javac.storeEnd(wrapMethod, endPos, top);
        }

        return setGeneratedBy(maker.Block(0,
                        List.of(saveCurrentThread, saveOldThreadName,
                                changeThreadName, wrapMethod)), source,
                context);
    }

    private static JCVariableDecl createCurrentThreadVar(final JavacNode node,
            final JavacTreeMaker maker, final String currentThreadVarName) {
        return maker.VarDef(maker.Modifiers(FINAL),
                node.toName(currentThreadVarName),
                genJavaLangTypeRef(node, "Thread"), maker.Apply(nil(),
                        genJavaLangTypeRef(node, "Thread", "currentThread"),
                        nil()));
    }

    private static JCVariableDecl createOldThreadNameVar(final JavacNode node,
            final JavacTreeMaker maker, final String currentThreadVarName,
            final String oldThreadNameVarName) {
        return maker.VarDef(maker.Modifiers(FINAL),
                node.toName(oldThreadNameVarName),
                genJavaLangTypeRef(node, "String"),
                getThreadName(node, maker, currentThreadVarName));
    }

    private static JCMethodInvocation getThreadName(final JavacNode node,
            final JavacTreeMaker maker, final String currentThreadVarNAme) {
        return maker.Apply(nil(),
                maker.Select(maker.Ident(node.toName(currentThreadVarNAme)),
                        node.toName("getName")), nil());
    }

    private static JCStatement setThreadName(final JavacNode node,
            final JavacTreeMaker maker, final JCExpression threadName,
            final String currentThreadVarName) {
        return maker.Exec(maker.Apply(nil(),
                maker.Select(maker.Ident(node.toName(currentThreadVarName)),
                        node.toName("setName")), List.of(threadName)));
    }
}

Wasn't that easy?

Dependencies

Of course the code depends on lombok. I'm using version 1.14.8. It also needs tools.jar from the JDK for compiler innards like expression trees. (An Eclipse processor needs an equivalent.)

Unfortunately lombok itself uses "mangosdk" to generate a META-INF/services/lombok.javac.JavacAnnotationHandler file for autodiscovery of processors. I say 'unfortunately' because this library is not in maven and is unsupported. Happyily Kohsuke Kawaguchi wrote the excellent metainf-services library a while back, maintains it, and publishes to Maven central. If you're new to annotation processors it's a good project to learn from.

Conclusion

Ok, that was not actually so easy. On the other hand, finding a starting point was the biggest hurdle for me in writing a lombok annotation. Please browse my source and try your hand at one.

UPDATE — A little bonus. This code:

@ThreadNamed("Slot #%2$d")
public void doSomething(final String name, final int slot) {
    // Do something with method params
}

Produces the thread name "Slot #2" when called with "Foo", 2. Strings without formatting or methods with params treat the annotation value as a plain string.

Felix Leipold: Managing Configuration

Fri, 05 Dec 2014 04:21:01 +0000

The Problem

In almost every serious software development effort that entails integrating different systems multiple environments are being used throughout development, testing and the production use of the system. Each of these environments typically contains a number of systems, that interact with each other. Each system needs to know how to talk to the systems it depends on in a particular envirnoment. Also, there are other parameters such as timeouts and feature flags. that might change across different environments.

The classical solution to that problem is to externalise the configuration of upstream systems into a separate file, so that the installed artefact can be made to talk to different systems depending on where it is being installed.

Frameworks like spring and puppet allow the use of placeholders that get assigned different values for different environments as a means to parameterise these files.

A Stroll down the Garden Path

On a recent project I started going down this route. Spring would load a .properties file which in turn contained placeholders which puppet would then populate differently for different environments. This was the approach all services in our department were using so, in the interest of consistency, we just adopted it.

Let’s look at an example. Our system talks to a system called System A. The interaction is encapsulated into a client abstraction taking the configuration as a a constructor parameter:

@Bean
@AutoWired
class SystemAClient(@Value("${dependencies.system.a.url}) systemAUrl: String) {
 
    // ... implementation
 
}

Spring in turn used a .properties file to fill in the right hand side values. For day-to-day development on the developers’ workstations we used a file such as the following:

dependencies.system.a.url=http://a.development.envs.company.com
dependencies.system.b.url=http://b.development.envs.company.com
connect.timeout=1000

The trouble is, that for our acceptance test environment we needed a different version of this file as we did for our continuous integration build, our demo system and most importantly production.

This is were puppet came into play. The operations team created a template for the configuration file as part of their scripting of the deployment:

dependencies.system.a.url=<%= @dependecies_system_a_url %>
dependencies.system.b.url=<%= @dependecies_system_b_url %>
connect.timeout=<%= @connect_timeout %>

Dealing with optional values was extremely awkward and involved an if statement inside the placeholder expression.

The values for these variables were defined in a yaml file that was far away from the code in a repository that belonged to the operation team.

Here an example for the testing environment:

classes:
    company::myservice:
        dependecies_system_a_url: http://a.testing.envs.company.com
        dependecies_system_b_url: http://b.testing.envs.company.com
        connect_timeout: 1000

Trouble Strikes

This is of course a massively complicated system. We ran into several issues:

Introducing new configuration parameters became hard, because
- The new parameter had to be introduced at several layers.
- The change of the puppet config had to be in lockstep.
Textual substitutions can easily lead to escaping and type errors.
Understanding the whole mechanism caused a lot of cognitive burden, as we quickly learnt when introducing new team members.
Releases could easily fail, because there was no easy way to test this until the actual deployment using puppet, because some of it relied on puppet functionality and some of it relied on the spring configuration parameter resolution which both couldn’t easily be tested together without an actual deployment.

We actually experienced quite a few failed attempts to relese because of issues with the configuration.

Surely we could have introduced a complicated test harness for this set-up, but we felt that this would be throwing good money after bad, so we had a brainstorming session to come up with our procedural requirements for the way we handle configuration.

What we Needed

The following requirements emerged during our session:

A workflow that enables developers to do all the changes associated with a new parameter in a single commit, so that they don’t go out of sync.
Minimise the number of changes required to introduce a new configuration parameter.
Ensure that parameter values are provided for every environment.
Ensure that no values for parameters that have been removed are provied to avoid zombie values, that sometimes got updated for quite a while, before everyone realised that they were no longer used.
Ensure that trivial validations for the values have been performed.
A central place that tells us what configuration parameters are available and whether they are optional.
A mechanism to change configuration values in an emergency without rebuilding the application.

The Solution

All configuration lives in the same repository as the code. There is a special place in the source directory where all the config files live. There name is the name of the environment they describe.
The configuration was bundled with the application so that they would move together.
When configuring the application, puppet only told the application which file to pick up.
Instead of using two layers of templating we managed the actual files as deployed to the server in the source repository. To make sure this remained manageable we stripped everything out of the configuration that is not changing across environments.
For the actual config files as loaded by the app we moved from .properties files to .json files, which we mapped to a couple of classes that actually represented the configuration. Using json4s we could easily express constraints such as: this is a number, this is a url, or this parameter is optional.
We wrote a unit test that tries to load up all files and check a number of properties
- Whether the json is valid json.
- Whether it can be deserialised into a configuration object. This in turn checked whether all non-optional values were supplied for every environment and whether URLs and numbers could be parsed as such.
- Whether the file has the canonical format with defined indentation and field order. If it wasn’t, a correctly formatted file was written to the target folder along with an error message that contained a commandline mv statement that would replace the file in question with the properly formatted version.
A little script was build that would allow the operations team to replace the config that came bundled

The configuration had been turned into proper objects:

/** Environment specific configuration for my service. */
case class MyServiceConfig(
    /** Configuration */
    dependendcies: DependenciesConfig,
    /** The connect timeout for all upstream systems in ms.*/
    connectTimeout: Int
)
 
case class DependenciesConfig (
    /** We rely on Service A, which is maintained by the foo-group */
    systemAUrl: Url,
    /** Service B is not required in all deployments, so it is optional. */
    systemBUrl: Option[Url]
)

Here an example for the configuration for the testing environment:

{
  "dependencies": {
    "systemAUrl": "http://a.testing.envs.company.com"
  },
  "connectTimeout": 1000
}

The configuration for a developer workstation on the other hand could look like this:

{
  "dependencies": {
    "systemAUrl": "http://a.development.envs.company.com",
    "systemBUrl": "http://b.development.envs.company.com"
  },
  "connectTimeout": 1000
}

This solution worked beautifully and suddenly introducing a new configuration parameter was no longer a dreaded task.

With the MyServiceConfig class there now also was a canonical place to look at things that can be configured as well as to elaborated a little bit on the parameters, which was a much better place than a separate wiki page.

Going further we could have also written tests on top of the MyServiceConfig abstraction, e.g. to ensure that all the urls point to our companies domain or that certain patterns are being followed across environments.

On my next project I will push hard for using this approach as soon as the problem of configuration comes up.