Mailinator(tm) Blog

Mailinator now supports POP3

2012-01-13T08:12:00.000-08:00

POP3 support means that you can now subscribe to Mailinator inboxes and get the email in your existing email client (i.e. Gmail, Yahoo Mail, Outlook, Thunderbird, etc.).

Of course you can subscribe to any inbox you like and unsubscribe at will. A big use-case for Mailinator is for people to sign-up for newsletters and daily deal websites like GroupOn or LivingSocial. They can get those great deal emails without giving out their real email addresses, and by quickly switching addresses, effectively (and definitively) "unsubscribe" if they wish.

Until now, they needed to use RSS or be prompt about checking the Mailinator inboxes. Now, instead you can simply point your POP3 enabled email client at the Mailinator inbox of your choice and you'll never lose an email. It comes right into your regular inbox!

We'll be putting up an FAQ and some helpful instructions but here's the simple configuration information for the initiated:

username: anything you want! This is mailinator remember?
password: 8char random password. Can be anything, but must be there
server: pop.mailinator.com
port: 110
encryption: off

The system is for now in beta but has been working out pretty well so far. Enjoy!

RickRolling 1000 people a day

2011-12-05T07:31:00.001-08:00

I wrote previously about the voracity of some folks to discover Mailinator's list of alternate domains. If you recall, the point of alternate domains are, to me anyway, to alleviate the worry that websites might 'ban' email addresses ending in the mailinator.com domain.

People use Mailinator when they want to use a website but not give out their primary email for fear of future spam. Preventing that use is asking the customer to reassess a decision they've already made (not wanting to give out their real email) which of course might result in their answer being "nah".

Or, of course they could decide to simply go create a fake yahoo account. That takes a few minutes and before Mailinator, when I tried that I often found myself losing interest in the whole idea before I was done.

In my last blog post I mentioned we get emails similar to this from time to time:

Hi! Love your service. Can you send me the exhaustive, comprehensive, and complete list of alternate domains so I can pick a nice one that suits my individual personal style? kThxBai

And when I say from "time to time", I mean like last week too. Giving out that list of course would be completely counterproductive to the point of having the alternate domains.

I did some minor front-page spruce-ups this weekend (like the dinosaur eating spam?) including giving a relatively real-time count of the amount of email Mailinator gets in the last day (15 million when I just glanced at the homepage), I made the randomly-generated names more readable, and matched them up with alternate domains as examples.

In addition, I added a bunch more alternate domains. Also if you recall, you can point your own spare domains at Mailinator and send email to those - then you have your own Mailinator alternate domain!

At this time there are literally several hundred (maybe over 1000, have to check) alternate domains.

It occurred to me though, that I might be able thwart some of the people emailing asking for a full alternate-domain list by actually giving them a link where they can see the only version of the full list I'm willing to publish. You'll see it on the homepage as:

Want to see a list of ALL Mailinator alternate domains?

I made the link small and didn't expect many people to click it. Maybe just the curious few. Well, to my surprise, the curious few turned out to be about 1000 a day. Nice. I surely underestimated the number of curious people out there.

Feel free to click yourself, but don't expect to actually see a list of any domains Oh, and enjoy the video. :)

How to get gmail.com banned - not that I did this

2011-07-01T16:40:00.000-07:00

Sorry I haven't posted in awhile - I've been heads down working on our new (free) ad-trading network Clickochet. If you like Mailinator, you'll love Clickochet - give it a try!

When I started Mailinator, a LOT of people told me it wouldn't work because websites would ban it right away. Ban it with reckless abandon. Ban it like the new thing on the internet was to just sit around and ban Mailinator all darn day long.

As it turns out, that didn't happen. Sure, some sites do ban Mailinator and some are even really (really) excited about the idea, but in the grand scheme of things, it's not really very many. Thousands of people use Mailinator everyday, so clearly, its a useful tool that many sites accept.

Back in the day however, I sadly fell prey to the words of doom that I was being fed. I mean, holy mackerel - what if sites DO ban it? What then?

So I drew up a plan. A plan, that at this time I can say I may not be fully proud of. A plan that involved guile, wit, a few domain names, and some rate-limiting (thread-safe) data structures.

I write this now because, well, for the most part the war is over and Gotham has grown past needing Batman anymore. Mailinator is not really the rogue tool it once was. Heck, hotmail supports disposable email now. It's mainstream.

Typically there are two reasons people want to ban Mailinator. A few years ago, people really had some sort of notion that your email somehow equated to your identity. Given the radically insecure setup of email in general, that was really a ridiculous technical assumption. Nonetheless it was pervasive.

Secondly, people banned Mailinator for fear of people abusing their website. Now keep in mind, anything you can do with Mailinator, you can also do with YahooMail or Hotmail. Its just that Mailinator lets you do it faster, but Yahoo is plenty happy to let you sign-up for 100 email accounts.

I get occasional emails from people asking me to have Mailinator stop accepting email from their site. Usually for the reason of stopping abuse. If they're nice and it makes sense, I almost always do it. But in my experience, usually when the existence of Mailinator is pinpointed as a cause of abuse, it is in truth merely an avenue that is already inherent to the internet or your website. Even shutting Mailinator down wouldn't solve the problem. The bad-guys just go somewhere else and keep on abusing.

Any sort of abuse is needless to say, no fun for anyone. Mailinator has specific code built-in to detect scripts and stop them.

In truth, Mailinator's system for detecting and shutting-down scripts and abuse really only serves one purpose. Its like that silly metal bar people put on the steering wheels of their cars. Let's be real, if a thief really wants your car, some dinky metal bar on the steering wheel isn't going to do diddly to stop him.

Same with Mailinator's anti-abuse code - it won't stop a determined person - but it does make it more of a pain than simply using something else.

So -Dear hacker bad-guy abuser dudes - other disposable email sites probably don't have that sneaky anti-script pain-in-the-butt hacker-stop code like Mailinator. Go use them.

There....

1) Solved hacking and abuse on the internet ? --> Not even maybe
2) Solved a little of the hacking and abuse on the internet possibly for me? ---> DING!

Ok, back to the story - as I said, in the beginning, the idea of wide-spread Mailinator banning scared me a lot. So what did I do? I bought some additional domains for Mailinator.

To this day, you can email bob@thisisnotmyrealemail.com and it will end up at mailinator (in the bob inbox) just like bob@mailinator.com.

Cool. Alternate domains. Problem solved.

Wait a second. How exactly do I tell the world about the alternate domains without telling the people that want to ban them all?

Every few weeks I get an email like:

Hi! Love your service. Can you send me the exhaustive, comprehensive, and complete list of alternate domains so I can pick a nice one that suits my individual personal style? kThxBai

At first, I was like "Neat! People love Mailinator and want to...heeeyy.. waaiiit a second".

If I give them the whole list, then they will, um, have the whole list. And then they can ban the whole list.

Ok. I know. I'll list one random alternate domain on the homepage every time you visit. No one will have the whole list. Just one here, one there. Perfect !

There problem solved. Again. Well, sort of.

Soon after I put up this "one random alternate domain per homepage load" system - the scrapers started. Every now and then I'd notice several hundred homepage loads from the same IP in a very short period of time.

They were scripts; scripts that were loading the homepage over and over and scraping out the random alternate domain that was shown. Sneaky. By doing this they could eventually formulate the entire list of alternate domains.

Drat. Now what. For awhile, nothing. I just let them go. A few months later however, I got an email from a Russian guy (sorry Russian guy, I don't remember your name).

You are dumb. Your homepage is easy to scrape and doesn't change so its easy to scrape your alternate domain. You are dumb.

He was right. Well, I'm not sure about the dumb part, but my homepage was easy to scrape. Someone could probably write a script to scrape it in short order. Probably just took a few minutes.

Could I make it harder to scrape? Well, I could, but wouldn't really slow anyone down much.

It was then however, I had a flash. An idea of simply epic proportions. A thought so crazy - that dad-burn-it, it just might work.

Let's not make the page scraping harder - let's make it EASIER.

I removed the bit of code that displayed the alternate domain and put it in its own (teensy) webpage. That "webpage" had absolutely nothing in it, except the text for the randomly chosen alternate domain itself.

Then, I embedded my new tiny webpage into the homepage (so it showed as before). Basically, to the viewer of the homepage - nothing was different. You saw the homepage and a randomly generated alternate domain, just where it was.

But to the folks that had been scraping my site, things looked plenty different. In fact, I probably broke all their scrapers (Sorry nice people trying to get all my alternate domains just to ban them! (ok, not really)).

Now here is a finer point of semantics. If you go to the Mailinator homepage, there is some text that says "Here is an alternate domain" followed, by, well, a randomly chosen alternate domain.

However, now that I split off that tiny little webpage with JUST the alternate domain in it - you could go there too by typing in the url directly. And you'd see nothing BUT the alternate domain. No surrounding text. No text saying "this is an alternate domain". That little page showed a domain, but made no claim about what it was displaying.

For your browsing pleasure, here's the only direct link to that page that I know of: Go ahead, reload the page a few times. (You can see this also on the Mailinator homepage on the lower left).

After the script guys got over the minor annoyance of their scripts breaking because o f my new setup, I'm sure there were office parties across the nation. Mailinator! Now even easier to scrape!

Now for the record, the rest of this post is hypothetical. An unimplemented idea if you will. Who knows - I'll bet nothing you read here on out ever happened. Just random thoughts. Musings. One big theory. Consider it random daydreams of guy who runs a fun email service.

Remember all that script-detecting code from the anti-abuse system? Well, what if I put that in here too I thought. Let's "detect" when a script is hitting our weensy alternate-domain page.

And, what if we also detected when the little web page is being viewed but not "in" the homepage - but by itself (just like the link above). And what if after about 30 page hits from the same script (or so), stop displaying actual alternate domains and start sprinkling in some other things. Hmm... but what other things?

I know - how about "gmail.com". Or, um "hotmail.com". Or maybe, "yahoo.com".

What, in our completely and totally hypothetical situation, would that do?

Well, let's see. There are these folks out there running scripts against Mailinator collecting all my alternate domains. Those scripts probably put results in a database or something and connects to their website. When one of their users tries to sign-up on their site using one of my alternate domains, it's in their database as a banned site and its immediately rejected.

Now imagine the wacky fun if somehow, some way, (totally theoretically speaking) some silly person snuk "gmail.com" in that list. I'd guess banning your users trying sign-up with "gmail.com" addresses is probably not what you want.

And, hypothetically speaking if you had code that would sneak in these non-alternate-domains in the page they weren't supposed to accessing anyway, when would be the best time to set it into action?

Well, those scripts ran at many different times, but just after midnight seemed like a popular time-slot.

If such code existed, making it active Sunday morning from Midnight to 2am seems nice. I mean heck, if my website stopped accepting signups from "gmail.com" on some Sunday morning, I'm sure I'd be downright chipper to hop into the office and find out why.

Boy. If all that stuff happened - I wonder what kind of email conversations I'd have on that Sunday afternoon? I bet they'd be like:

Your alternate domain list displayed 'gmail.com'!
Hi Fred, no it doesn't. Just reloaded the homepage 10 times, nothing like that. all the best.

or I bet another would be like:

Yahoo.com? What is this some kind of joke?
Sorry, did you mean to email this to Carol Bartz? Not sure what you're talking about.

Phew. Well, that's surely a fun thought experiment. As you can see from the link above however, it surely doesn't do anything like that. Honestly these days, most of the scrapers are gone. I think simply that the internet evolved and more of them simply lost interest in the fight.

Every now and then I'm still asked what I think about banning Mailinator. I've mellowed a lot since the early days and I pretty much always give the same answer. If you think banning Mailinator going to solve your problem, go ahead. In my experience, it won't. And by asking I am guessing that you are making some assumptions in your site that will surface as issues in other ways.

And of course, script writers, you now have the direct link to the alternate domain page above. Scrape away. But keep in mind, the best way not to trigger any Mailinator abuse systems is to not do anything "too fast". Those script detectors are pretty fickle little beasts. It's not a bad idea to try and stay on their good side.

Why you see the Ads you see

2011-06-15T11:05:00.001-07:00

Since we've launched Clickochet (as in "Click Ricochet", an Ad Trading Network), I get a rather constant flow of questions about how we do ad targeting.

Honestly, I've never been a person to use an Ad-blocker and the reason is that if I go to a website that brings me value, it's a small price to pay to see ads in exchange. The value that Google search brings me EVERY day is immense - and all I have to do is view ads that once in awhile are actually what I was looking for anyway? Deal. Reddit actually thanks its users for not using ad-block.

The trick here however is to bring people "good", "quality" and "relevant" ads. And it is of course, our goal at Clickochet is to try to bring you the most relevant ads to you as we can.

So, after writing a few responses to nice Clickochet users about how our ad-targeting works, I've decided instead to simply - let you see for yourself. At least for now, let you into the belly of the beast.

After spending a good number of months staring at ads, I'm a bit obsessed with them now. It's rather rare that I don't go to a page with Clickochet ads and push my nose to the screen wondering "Why is THAT ad showing THERE?!"

I usually (usually) then continue that self-conversation with, "oh. oh. ok. that makes sense." But to be able to answer that for myself, Clickochet uses some diagnostic cookies to turn on extra information about why you see the ads you see. And actually - more important for me - it has facilities to tell me why you don't see the ads you don't see.

A few disclaimers about what you're about to see. Pretty much, 100% of this stuff will change. Ad targeting is an exercise in perpetual tweaking and change - and given the infancy of our network, you can count that statement as double. Also, keep in mind that Clickochet is a new network. We're working furiously to build the best network for our customers that we can - but our targeting mechanisms are relatively boilerplate at this time. Finally, this stuff isn't for general consumption - in other words, it's not meant to be "pretty" nor am I guaranteeing it works across browsers (so use firefox or chrome).

So to begin, in a separate browser window or tab, hop over to one of Clickochet's favorite yoga sites, Qi-Yoga.

http://www.qi-yoga.com

Nice ads huh? And chances are you'll see an ad like this:

I guess you'll see that particular ad because:

1) I chose "yoga" specifically because we have a few, but not too many yoga sites in the system (at the moment) so that determinism helps writing a semi-predictable blog post.
2) This ad is contextually relevant to qi-yoga and our contextual system realizes that. (the two sites are not otherwise related)
3) I don't know the fine folks over at HealthAndYogaRetreats.com but I slipped them a nice pile of extra impression credits so that their ad won't "run out" in the middle of my examples.

Ok - keep qi-yoga.com up in a browser window and open yet another browser window (welcome to my world). This time go here:

http://www.clickochet.com/pages/diag

You don't have to sign-up or be logged in. Again, this is arguably an internal page. Press the "Turn On" button for both options. Note: The 2nd option CAUSES A POPUP for every page you load that has Clickochet ads. Again, this is a diagnostic page no one sees unless the turn on that cookie. And incidentally, to follow along, you'll need to tell your browser to "allow popups" from qi-yoga.com I'd guess.

Great. Now refresh the qi-yoga.com page.

Welcome to the diagnostic view. Ad titles are now color coded and have been suffixed by a single character. That character signifies the targeting mechanism used that turned that ad "on".

The popup window is a list of most of the ads that were considered for showing on this page. (I say "most" because some might have been short-circuited once we found 3 acceptable ones.)

The popup you see should look something like this:

The first 3 ads in the diagnostic ad window are the ones that were shown. The rest (and sometimes the list is long, are the ones that were rejected). The lists will perpetually change as you hit refresh (the system knows when you hit refresh over and over and no one is charged or awarded credits for that behavior). Also, keep in mind that refreshing the page a few times will help our examples here but aren't a normal ad viewer use-case. People typically see one set of ads and then leave the page. So you'll likely start getting less relevant ads the more refresh.

So, what are the title suffixes you might see? The represent the different ad targeters and specifically which ad targeter decided to show you that particular ad.

(C) - A "contextual" ad, for some broad definition of "contextual". Back to my theme that "everything will change", the contextual engine is finding its sea legs. It sometimes expands its search and sometimes contracts it to find ads that could be relevant (it then keeps track and hopefully finds a happy place). Hence, you might see "contextual" ads you don't think are all that contextual. Don't worry - if that's true, they'll cycle themselves out eventually. Again, I chose the yoga theme here because of the median set size of what the contextual targeting finds.

(A) - A "site affinity" ad. Refresh the page a few times if needed to see HealthAndYogaRetreats again. You'll notice it is an affinity site. Yes its contextual, but more importantly, people have clicked from qi-yoga to healthAndYogaRetreats a few times. One click won't do much - but a pattern of clicking and the system takes notice. Those sites become (spiritually) linked in the system.

This is a very important feature beyond contextual advertising. The classic example is that "Budweiser" ads might do very well on a "Nascar" website. But contextual analysis might not determine that (in fact, drinking and driving might be a negative contextual correlation!). But of course let's hope the budweiser drinkers are watching the races, not participating. So when they click the budweiser ads, the system realizes "huh, for some reason these two very unrelated things have an affinity for each other".

Affinity relationships perpetually degrade and require, again, a pattern of clicking to keep them alive.

(U) - An underserved ad. Simply put, there are ads in the system that haven't found a solid niche to display in. These ads can be pretty non-relevant at times but the idea is that they won't individually show up all that often on any given site. And showing them serves two purposes -

1) to get underserved ads some impressions, and
2) remember site affinities from above? How do we 'discover' site affinities if we only show direct contextual ads. Showing ads from a larger pool gives the affinity system a chance to discover unexpected affinities.

You'll see more underserved ads if you refresh the page a lot - which again, isn't the "normal" use case.

(1) - actually, any numerical digit you see means we're running an experiment. Again, every last thing here is subject to change, but experiments are even more transient than that. One experiment we're working on is transitive relationships between site affinities. So if Budweiser ads are good on Nascar sites, and then we discover Nascar ads are good on fishing sites - are Budweiser ads good on fishing sites? (I'm guessing that's an affirmative)

If you'll notice, none of the targeting above has to do with YOU (or more specifically your browser since we never know about "you"). All those targeting mechanisms have to do with the sites, ads, and contexts involved. Site affinity is based off clicks but it's really about all clicks in the system - not really about you.

That being said - it's purely possible that although you like both budweiser and nascar - you might like yoga too. And although you are an awesome beer-drinking race-car-driving yogi - that's probably not a common combination. Showing a yoga ad on a nascar site as a general rule is probably wrong - but for you specifically, it might be right.

So, what we can do for you specifically is try to find things that you like. This is of course where people get worried about advertisers finding out private information about users. Clickochet does not keep personally identifiable information about anyone who views the ads (please see our privacy policy on the site). We keep a cookie noting where a browser goes (and note that if people share a browser, that browser becomes a collection of all browsing destinations, regardless of any given user).

The goal again is to show you the most relevant ads possible. That brings the most value to people viewing ads and the most value to people showing ads. Of course, Clickochet has an opt-out cookie (see the front page of Clickochet for a link or go directly to http://www.clickochet.com/pages/optout to turn it on). But! Don't turn it on just yet or the rest of this blog article won't work!

Ok. Let's go to another Clickochet ad site - oh, wait, you're already on one. This blog has come Clickochet ads on it in the right-hand side-bar. Ok, note the ads you see there and .. hit refresh. Here, I'll bookmark this blog entry for you so you can pick up reading here.

BOOKMARK FOR WHEN YOU REFRESH THIS BROWSER WINDOW TO SEE RETARGETING

Ok .. hit refresh.

BOOKMARK FOR WHEN YOU REFRESH THIS BROWSER WINDOW TO SEE RETARGETING

Ok.. great. Now.. what ads shown? There's a high probability (note: EVERYTHING involved here is a probability) that you'll see a red ad for Qi-yoga followed by the suffix: (R). (if not refresh the page once or twice)

(R) retargeting. It seems almost counter-intuitive but the ad industry has plenty of documentation that people will click on things they already have seen - and/or simply want to see again. The system now knows you went to qi-yoga.com (because I told you to earlier in this case). And thinks, "Hey.. this person likes that site" - so it gives a reasonable probability to show ads for qi-yoga to get you to go there again.

There's surprising advantages to this. If you liked qi-yoga and saw it last week, you might be interested in seeing if the site is updated. Qi-yoga likes this because as with all websites, there is a lot of bounce - people come, go, and never come again, This reminds you of that great site you saw. Clickochet likes this because it keeps you within our community of sites that are helping each other exchange traffic.

And then.. we can run experiments based off that correlation. Maybe there is a site affinity between this blog and qi-yoga. Or, maybe its simply a personal preference of yours and maybe we can help you discover other yoga sites. The possibilities are endless.

If this all seems odd that all this is happening in the background, I'll note that Clickochet is a newbie in this arena. Every ad system out there does this and more (goto zappos.com, browse some shoes and then browse around the web - see how many zappos ads you then see).

As I said - you can opt out of this type of browser-based targeting on our opt-out page if you like. But there's nothing more nefarious there than simply trying to show you ads for things you might be interested in.

Again, our goal is help our community of sites trade traffic for traffic. As we say in our FAQ - give an ad, get a few - the system really does work. And it will work better and better with the more value we can bring to everyone involved including publishers, advertisers, and web-passers-by.

Sign-up (it's free!) at http://www.clickochet.com

Introducing Clickochet: A (free) Ad trading community

2011-05-10T12:12:00.000-07:00

I have a lot of ideas for apps/websites/start-ups. No seriously - I mean a lot. Some of them aren't all that bad either. And some are too - really bad, like "waking up in the morning and thinking 'what the heck were you thinking'"-bad too. I mean really.

Alas, usually even with ones I think are awesome - I typically end up finding some fatal flaw. It could be business model or timing or ... who knows.

There's a lot of factors that make me consider an idea for more than a few minutes much less actually work on it:

1) Is it in my skillset
2) Is it the right scope - too small and its a weekend project, if its too big, am I the right guy to assemble the infrastructure (team, investors, etc) to bring it together.
3) How hard will customer acquisition be?
4) Does it have a viable business model?

There's a lot more subtle questions, but these are some of the bigger disqualifiers.

Once in awhile however I get one that sticks. One that I say "why didn't I think of that !" - and then I say "Oh. I did!". Mailinator was one. And this - this is another.

Last fall my girlfriend started a celebrity news blog. I'm not much into celebrities but she was pretty passionate about it and it was exciting to share her enthusiasm.

As with any new site, she was trying to gain traffic. So to help with that effort, I bought her $100 worth of adwords. I've bought adwords for many different companies and projects, but I was still rather surprised with the results. For $100 I got a 3-day (or so) ad campaign that yielded 100 visitors.

The interesting part was that all this time, she was running Adsense - that is, ads to make money on the site. Now, ask anyone trying to make money with Adsense why they don't "buy" ads to gain traffic and they'll tell you, it's losing proposition. And no wonder - that's the crux of how ad networks make money.

Sure, they sell ads. But for a moment, consider a website that does both - it's both an advertiser and a publisher. In that scenario, ad networks sell you ad impressions for money - and then trade you money for ad impressions. Simply put, there is an exchange rate on ad-impressions to money and just like the kiosk at the airport which changes currency, they take a "cut" in both directions.

And the cut can be pretty big. In the case of my girlfriend's site, I successfully managed to convert $100 into 15cents by changing it into ad-impressions and back again (Actually its even worse because the girlfriend gets to keep the 15cents, but we don't bring that up).

Now my assumption above is for sites that are both advertisers and publishers. Many small sites are publishers, but not so many are advertisers. So my assumption is, well, rather assumptive.

But why aren't they advertisers? They want traffic too right? Advertising is a way to get it right?

The answer is that those sites would do both, but its non-economical to do so. Buying ads to show ads makes no sense. Even just buying ads is a very expensive method of traffic acquisition - but what if ads were really (really) cheap. Then everyone would buy them wouldn't they? I mean, it would be a cheap method of traffic acquisition.

Ad networks (BTW, I don't mean to pick on Google above, I just had been using their service because they are one of the most accessible ad networks for small sites) provide a valuable service. Clearly - the ad industry is highly alive - they are bringing value to lots of people.

But it occurred to me - how do I make an ad system that brings value to smaller sites. The under-served billions of websites where buying ads makes no sense today. How about I make a system where you trade ad-impressions for ad-impressions. That simple. And, instead of making it where you give something and get a lot less back, what if we fliped that. That is, give one ad impression and get a few back?

I call it Clickochet. Like "Click Ricochet" because that's what it is.

So how do you get more back than you put in? Well, consider your normal banner text ad. In reality, its one ad impression on your website, but it contains ads for 3 different websites within. When you show one "ad", three other sites benefit.

Clickochet is built as a community. We connect it closely with social networks that encourage social interaction and membership. A Social Ad Trading Network. This is key - you're not trading money and ads with a big company, but you're trading ads with each other. Other website owners looking for traffic just like you. Of course we strictly filter out porn and malicious sites, but being a good community member becomes an important factor on how much you get out of the system.

So, in a perfect world, you'd give one banner (keep in mind smaller ad creatives have less ads inside) and get 3 ad impressions. But it became quickly apparent, the world isn't perfect. If someone puts your ad at the bottom of their page, I doubt you'd be happy giving them an ad at the top of your page in return. Or what if their ad quality was low? Or conversely, what if their website had a huge click-through profile? In other words, not all ad impressions are created equal.

All these factors matter. So instead converting ad-impressions to real currency, we convert them to virtual currency. An exchange rate exists, but its minimal and its based on a lot of things - much of it tied to people showing good ads. If you show a banner ad, you won't get 3 ad impressions for it, but that isn't a far off guess either. You'll very likely get more out than you put in.

You are the publisher - you show ads on your site and earn credits.

And you are the advertiser - you spend credits and your ads show all across the network.

You can spend your virtual "impression credits" the instant you make them. Or you could bank them and save them for your new site's impending launch 6 months later. Or take your impressions (from a site like Mailinator) and spend them on your friend's site (like my girlfriend's).

This isn't about replacing existing ad networks. There's no need for that. If you're making money running ads - don't stop.

But unlike buying an ad campaign which costs a lot and lasts a few days (which you surely can still do), you simply give up whatever amount of page real estate you want for Clickochet ads and leave them there. And every day, from here on out, all across the network your page views are giving your a multiple of ad impressions on other sites.

To "prime the pump" I'm giving this virtual economy a boost - an "economic stimulus" if you like. I'm injecting all of Mailinator's ad impressions into the economy without taking them back out. Active users (i.e. you must be earning your own credits somewhat) will get lots of free ad impressions. (note: when I say "all" ad impressions, I mean "most" as I plan to advertise Clickochet on the Clickochet network somewhat).

Of course, it's free to sign up. Free to use. We may introduce a premium service at some point, but you can count on being a member of this community staying free.

Notes from "The Programming Leap to Multithreading"

2011-05-01T08:09:00.000-07:00

I gave a (techinical) keynote at the ICIS conference in Shanghai a year or two ago. It was a fun talk and I was recently asked for the notes so I thought I'd post them.

When I was at Google I spent some of my 20% time simply surfing through the Java code base looking for race conditions. It was a lot of fun and often I was patching code that was stuff that hadn't been changed in forever or was completely out of my normal stomping grounds.

After submitting the code changes with the comment 'Fix a race condition', I can remember at least one code reviewer responding, "Approved. Cool. But, what race condition?"

This talk covers a bunch of the common cases I encountered.

LeapToMultithreading.pdf

How I sped up my server by a factor of 6

2010-06-02T08:57:00.000-07:00

(with one linux command)

Subtitle: IO-bound and CPU-bound applications are common - here's maybe a "memory contention bound" app

I've written a lot of servers. If you've read the architecture of mailinator or benchmarking talkinator or "blocking faster than non-blocking" you probably already have that idea.

I was working on a new server infrastructure recently I needed for a new product. The server has a novel internal architecture to me that seeks to never voluntarily block threads or cause context switches - but at the same time is highly multi-threaded and creates new ones whenever it needs to.

The server's nature isn't totally important but you can think of it along the lines of a twitter server. It gets messages from one place, collates them, and sends them to (potentially many) other places. I suppose it could be used directly as a "twitter server" but of course, they already have those and my start up focus is rather orthogonal to that.

I usually write servers in Java on linux so keep that in mind for the ideas raised here. At Google, I worked on servers in C++ and Java. And especially after that experience, I'll stay with Java. To hopefully avoid language wars, the #1 reason I write in Java is that I'm way better at Java than I am at C++ or Ruby or Python or whatever. In other words, it was a practical (and definitely reasonable) choice in this case.

Also, the benchmarking here isn't contrived - I'll save the details, but I truly am asking the server to do precisely what it would do in the wild except at a higher rate.

Now, given the nature of the server, I needed 2 things to help benchmark. A "producer" (a system that produces messages) and a "consumer" (a system that consumes them). The producer produces a sequenced message set, and the consumer verifies it receives every message intact *and* in-order.

I chose message sizes at somewhat less than twitter size (msg size = 100 bytes) to induce a reasonable amount of activity in the server (server purpose is not necessarily large messages and large messages tend to just start saturating bandwidth without causing contention or server cpu activity). My custom protocols add a few bytes and TCP has a 40byte header - so overall I'm guessing that I'm using an average 200 byte messages counting everything.

Producer -> Server -> Consumer

The server system is pretty flexible and in effect, location transient. That is, a server process might live on one machine today and find itself on another machine tomorrow. Or a few new server processes may join the mesh. Needless to say, the idea is to create a scalable, flexible system.

One ramification of that is that it's possible and even very likely that some producers, servers, and consumers could be on the exact same machine. Socket communication over localhost changes a lot of assumptions. Firstly, TCP fusion can occur to reduce overhead significantly. Secondly, there is not an effective bandwidth limitation - there isn't a real network involved, it's a virtual (and fast) one.

Running all on the same server, I would expect this benchmark to be CPU-bound given that I/O is now virtual and effectively a CPU operation. Like I said, loopback is a real scenario I need but I initially think I benchmarked on a single machine mostly as a matter of convenience.

So, with all 3 processes running on the same machine, I ran the test. The CPU was a Intel Core I7 920. It has 4 hyperthreaded cores that act like 8. (note: I've tested all of this on a Core2Quad (non-I7) cpu and CoreDuos and results are effectively the same)

Here's the image of htop during the test with all 3 processes running. (if you run linux and use top, upgrading to htop is highly recommended).

Don't get hung up on reading the numbers or text. The graph in the upper left gives you a sense of how busy the CPUs are. In this picture, they're all at least a little busy. That's no surprise given we're running many tens of threads.

Notice that none of the cores are "maxed". This benchmarked showed the server to receive and re-send about 120,000 messages per second (that's 120k from producer to server and the same 120k from server to consumer - so 240k "messages transmissions" but only 120k messages - this would be analogous to queries-per-second for a webserver).

Why aren't the active cores maxed?

It occurred to me that I was running 3 CPU-bound processes on the same machine and that the processes might be stepping on each other's toes. It's possible that if the server is running on core 4 one second and the producer is running there the next, the level 1 cache of that core could be ruined for the server the next time it ambles over there.

The simple solution was to assign cores to the processes. In linux, you do this with command: taskset

Taskset is followed by a bitmask value to assign CPUs - so I ran (in separate xterms):

taskset 0x3F java manyfeedServer
taskset 0x40 java theProducer
taskset 0x80 java theConsumer

and off they went. The server is highly multithreaded so I gave it 6 cores. The producer and consumer each got one.

The result? 270,000 messages/second! Wow. If my cache assumption was right (and I'm not claiming to know if it was) - it REALLY worked. One way or another though - something worked - we got better than a 2x speedup.

So you might be thinking the moral of this story is:

1) If you're in linux, install HTOP
2) If you're sharing a computer amongst CPU-bound processes, isolating the processes might (and very well "might not") be beneficial.

And ok, those are fine ideas. But what bugged me was that htop showed me that no CPUs were maxed yet. Again, what was slowing my application down ahead of CPU power?

I then tried limiting the server to 2 (hyperthreaded) cores. (I also tried keeping the producer and consumer on the same hyperthreaded "core" and given that I had cores to spare, also tried separating them, but the result was the same).

taskset 0x03 java manyfeedServer

Now, we get 530,000 messages/second. Nice. Reducing the cores from 6 to 2 nearly doubles our msgs/sec again. Here's the htop now:

You can see that cores 1 & 2 are plenty busy. Cores 3,4,5,6 are idle as expected. Core 7 (the producer) is pretty busy and so is Core 8 (the consumer).

530k msgs/second is nothing to sneeze at but.. um.. again, no core is maxed. Why - not? What's the bottleneck?

Obviously.. the last test is to throw the whole server on ONE cpu. Apart from the fact that I very purposely and meticulously coded this server be highly multi-threaded, fewer CPUs seem to make it happier. I am a conservative thread-safety-inducer. That is, I'm only really dangerous with firearms and synchronized blocks. But I'm by no means afraid to use the latter when needed.

taskset 0x01 java manyfeedServer

And finally, we hit 100% utilization on CPU 1 at 790,000 messages per second.

Here's the TL;DR of this blog post:

Some multi-threaded java applications apparently run faster on 1 core than on multiple cores.

Note the very non-committal phrasing in an attempt to make this a rather defensible statement. A graph showing the ManyFeed server's performance per number of cores:

So.. if you are a CPU or JVM guru and want to tell me your thoughts on what's going on, I'd love to hear it.

A long time ago, I benchmarked a bunch of different CPU types on doing contended memory stores. To my surprise, a little single-core Dothan processor did amazing in a few tests and I had no idea why. The discussion above is really the same idea. (I've retested that benchmark by the way on the Core I7 - and it has very different characteristics - probably a testament to the I7's new memory architecture).

Note that if you have a pre-I7 Intel multi-core cpu, you can reproduce my "1 core runs faster" results here using the code in that blog post (note: this code *is* a micro-benchmark, built to create a situation that causes extreme and unrealistic threading competition - for more details read that blog post). This gives a clearer picture of what's going on. My theory is that with 2 cores and 2 threads - each gets a core and every store operation competes for the memory bus in order to do its operation. Threads spend lots of time "competing" and less time doing real work. On one core (and 2 threads) - only one thread runs at a time - so every time it tries to store in memory, it can.

The Core-I7 is different (and maybe AMD cpus too) - it doesn't suffer from this memory contention problem at all. (Although - given that my server still runs (way way) better on 1 cpu even on I7, then maybe the contention is elsewhere in my server).

Here's that benchmark running for Static Volatile Store on a Core Duo CPU with 2 cores and again with 1 core. No performance loss on just one core, otherwise same exact run.

(numbers along the bottom are the number of threads in that run - it's not until we have 2 threads that contention creeps in and hurts us).

Oh.. and how does my little nifty manyfeed server do on a real network? Testing on 1 CPU on my local 1Gbps LAN I get about 300,000 messages per second.

With some quick (and hopefully not silly) math, this looks about right.

1Gbps = 1,000,000,000 bits/sec = 125,000,000 bytes/sec

125,000,000 bytes/sec divided by 200 bytes/message = 625,000 messages.

half for producer send and half for the server send (to consumer) = 312,500 each

On a real LAN, the server saturates bandwidth (i.e. becomes IO-bound) and that's no surprise (i.e. until I have access to 10Gbps networks, I don't need to speed up my server any more).

Interestingly I tested this idea of "1-core'ing-it" using apache tomcat and apache bench and saw no improvement. I also saw far fewer qps (loading a single, tiny, web-page over and over) and 100% core utilization even on multi-core. It'd be my guess that tomcat isn't contention-bound but truly cpu-bound.

In other words, don't follow this path without testing this yourself. The good news is that its extremely easy to try and you don't even have to change a line of code. Just try "1-core'ing" it.

Mailinator and (not) Death-by-Popularity

2010-03-01T09:11:00.000-08:00

As far as I know, Mailinator was the first website of its kind. A website to accept any email at all, no sign-up, no registration. Obviously, email websites surely existed before but not with this "no one owns any account" twist. And what a twist it turned out to be. Check out the original web page in 2003 at the Wayback Machine (including the "its like flicking a booger at spam!" original tagline!)

Needless to say, copycat websites showed up fast. And that's a decent indicator that a new idea is a good one. Or at least an interesting one. As I've (and many others have) said before ideas by themselves are basically worth nothing. And any good idea is destined be copied, stolen, and taken credit for. Execution is the key.

If you don't believe me - here is part of Mailinator's terms of service present on every inbox page:

"You agree to hold ManyBrain, Inc./Mailinator harmless from any damages caused by loss of emails, content within emails, damage to your computer or innocence from viewing emails, direct or indirect use of this system, or anything else you can think of. Use at your own risk."

I wrote those words a bunch of years ago. I'm no lawyer so I tried to come up some simple words to get the message across.

Now. Go to google and search on (quoted) "computer or innocence". Here's a screen shot of what I got.

Almost all entries on this page are Mailinator copycat services that not only "borrowed" the idea of Mailinator - a few even "borrowed" its terms of service !! Talk about uncreative copying. :)

For the public record - again, I'm no lawyer so if you unauthorizedly copied my terms of service and get in trouble - you can't blame me.

Now keep in mind here - theft of ideas is all part of the game. It is, for better or worse, part of human nature. And as I outlined here - the execution and evolution of Mailinator was mine, but the idea of no-sign-in email wasn't. It was my old roommate's (who at times, has helped out on Mailinator copy and such).

Google didn't invent the idea of internet search. Facebook didn't start the social web. They just made it better or more usable - or something. But they won. Sometimes taking a great idea and twisting it just a little, turns it into something great.

Mailinator however is an interesting beast. It is a hard business to monetize. And that's just fine, so long of course, it doesn't cost a ton to run.

But therein lies the rub.

There's a strong sentiment in the web industry that performance, at many levels, doesn't matter. That you simply can "buy another server" and solve many performance problems. And that's true. And generally that's a good idea. I mean, wasting a lot of developer time (who could instead be creating new features) on performance optimization is dubious. Especially if you can throw down 2 or 3 thousand dollars and simply buy another machine to solve the problem.

I wrote here that the initial incarnation of Mailinator started to die at around 800,000 emails per day. If you want to make a Mailinator copy - its really not very hard. You can do it with almost all off-the-shelf (and free) software. Sendmail to receive and some sort of webmail front-end to view.

That's it. And that's largely what Mailinator was when it started and it took about a weekend to setup.

But then... I ran into that 800,000 a day problem. This was a performance problem of epic proportions. That is, the site was perpetually crashing. I needed a solution.

One solution, as I pointed out was to "buy another server" but that would have cost the 2 grand plus the ongoing monthly cost to pay for it. And Mailinator was not making enough to cover that at all. I would have had to start paying for Mailinator out-of-pocket to keep it going.

And maybe worse - that would have just solved the problem temporarily until I needed yet another server.

So instead, I did the wrong thing. I rewrote the system from scratch. Threw out the off-the-shelf stuff and built a software system that was customized for Mailinator. Again, financially this was a poor decision (i.e. cost of my development time) but luckily Mailinator is a hobby too so I wrote it off as just that.

As soon as I brought up the new software email jumped to 3,000,000 a day. The old system was not only choking at 800,000 it was refusing connections. I then upgraded the network at the time from 10mbs to 100mbs and the email again leaped to 6,000,000 a day. That is, the new software was fast enough to expose that we were now saturating bandwidth.

(Mailinator has now seen >25,000,000 emails per day)

So that's all fun stuff - but remember those copycat services? Over the years, I've seen 3 of them rise in popularity - and then die (as in literally, site gone, blog explaining it became too expensive to run).

That's what I meant when I said Mailinator is an interesting beast. Its a paradigm that's extremely easy to reproduce - so long as you don't actually get popular (or you find a great way to monetize).

Seems like the web has a lot of opportunity for services like this. Cool ideas that in many senses are just too costly to keep running. Then again, the price of tech comes down - and performance increases.

A Sendmail/Webmail Mailinator today on a modest machine could probably handle a fair bit more than 800,000 a day.

Every few years I try to re-evaluate every old idea I dropped because I felt it wasn't technically/financially feasible. There's a perpetual convergence of faster tech and cheaper prices.

New things become feasible every day that weren't feasible yesterday.

Got any ideas?

Mailinator for Testing

2009-11-11T11:10:00.001-08:00

Recently I implemented a captcha system in Mailinator.

If you're a normal user - you've probably never seen it. That's because it doesn't get activated until emails with the same subject get read more than like 10 times in a minute. Needless to say, if you're using Mailinator to sign-up for something here and there, that's not a normal use case.

But if you're a script - or a person signing up for some website over and over and over (and over) - you hit this pretty fast. It has done a fantastic job of stopping scripts pretty succinctly and slowing down humans zealously working over some site. The main problem with the latter is that many of those sites don't like that and may then ban Mailinator. That's not good for anyone.

What surprised me about the captcha system is how many people emailed me that I broke their test scripts. That is, they had email system tests (i.e. "Thanks for registering!") that use Mailinator as an end point. Their test then (with variation per user of course) checks the Mailinator inbox that their system correctly sent the email.

ManyBrain, Inc. (the company that owns Mailinator) has offered for a long time testing packages that completely bypass Mailinator's abuse system for high-volume testing or other emailing.

I'd be interested in however in improving this service and making it more mainstream. If you use Mailinator for testing - or would like to - I'd like to hear from you.

How would you use it? What kind of volume? I can't believe anyone is excited about scraping HTML (that wasn't particularly designed to be scraped) to get test results. Mailinator already has a (secret, shhh) JSON interface thats not public. That would seem to be the way to go.

My thoughts is a JSON-based SOAP/REST/whatever API that allows testing scripts to read emails. Whats a reasonable method and threshold of use to start charging? Possibly a sister site to Mailinator itself would be a better home for the testing service.

If there's enough interest to support the development, I'd be excited to get this up and running.

Email me at paul@manybrain.com

More Alternate Domains !

2009-09-21T00:45:00.000-07:00

I've added a few more alternate domains. If you've never used them, they are simply other domains (for example, sogetthis.com) that forward all email to Mailinator.

Simply, if you send email to fred@mailinator.com - or - fred@sogetthis.com it works exactly the same way and your email will arrive in the "fred" inbox at Mailinator both ways.

You can find alternate domains (including the new ones mixed in) on the front page of Mailinator on the left column.

I'll be sneaking more into the rotation soon!

A Beautiful Race Condition

2009-06-09T05:53:00.000-07:00

I recently gave a keynote at the ICIS 2009 conference in Shanghai. The topic was why multithreaded programming seemed so easy, yet turned out to be so hard. The fun part was I investigated (per my last post and this one) several old, personal concurrency demons I knew existed but wanted to know more about.

One of those was, indeed, my favorite race condition. It doesn't escape me that its probably wholly unhealthy to even *have* a favorite race condition (akin to having a favorite pimple or something) - but nonetheless, the elegance of this one still makes my heart aflutter.

The scenario of this race is that we assume, not terribly inaccurately, that race conditions at times, can cause corrupted data. However, what if we have a situation where we sort of don't mind some corrupted data? A "good enough" application as it were.

The dangerous part of all this is if we assume (without digging in) what kind of data corruption can happen. As you'll see, you might just not get the type of data corruption you were hoping for (which is one of the sillier sentences I've ever written).

The particular instance of this kind of happy racing I've encountered is where someone uses a java.util.HashMap as a cache. I've never done such a thing myself, but I heard about this race and thus this analysis. They may use it with a linked-list or maybe just raw, but the baseline is that they figure a synchronized HashMap will be expensive - and in their case, a race condition inside the HashMap will just lose (or double up on) an entry now and then.

That is - a race condition between two (or more) threads might accidentally drop an entry causing an extra cache miss - no biggie. Or, it may cause one thread to re-cache an entry that didn't need it. Also no biggie. In other words, a slightly imprecise, yet very fast cache is ok by them. (of course, this assumption is dead wrong - don't do that - read on for why!)

So they setup a HashMap in some global manner, and allow any number of nefarious threads bang away on it. Let them put and get to their hearts content.

Now if you happen to know how HashMap works, if the size of the map exceeds a given threshold, it will act to resize the map. It does that by creating a new bucket array of twice the previous size, and then putting every old element into that new bucket array.

Here's the core of the loop that does that resize:

1:  // Transfer method in java.util.HashMap -
2:  // called to resize the hashmap
3:
4:  for (int j = 0; j < src.length; j++) {
5:    Entry e = src[j];
6:    if (e != null) {
7:      src[j] = null;
8:      do {
9:      Entry next = e.next;
10:     int i = indexFor(e.hash, newCapacity);
11:     e.next = newTable[i];
12:     newTable[i] = e;
13:     e = next;
14:   } while (e != null);
15:   }
16: }

Simply, after line 9, variable e points to a node that is about to be put into the new (double-wide) bucket array. Variable
next holds a reference to the next node in the existing table (because in line 11, we'll destroy that relation).

The goal is that nodes in the new table get scattered around a bit. There's no care to keep any ordering within a bucket (nor should there be). HashMap's don't care about ordering, they care about constant time access.

Graphically, let's say we start with the HashMap below. This one only has 2 buckets (the default of java.util.HashMap is 16) which will suffice for explanatory purposes (and save room).

As our loop starts, we assign e and next to A and B, respectively. The A node is about to be moved, the B node is next.

We have created a double-sized bucket array (in this case size=4) and migrate node A in iteration 1.

Iteration 2 moves node B and Iteration 3 moves node C. Note that next=null is the ending condition of our while loop for migrating any given bucket (read that again, its important to the end of the story).

Also important to the story, note that the migration inverted the order of Node's A and B. This was incidental to the smart idea of inserting new nodes at the top of the list instead of traversing to find the end each time and plunking them there. A normal put operation would still have to check that its inserting (and not replacing) but given a resize can't replace, this saves us a lot of "find the end" traversals.

Finally, after iteration 3, our new HashMap looks like this:

Our resize accomplished precisely the mission it set out to. It took our 3-deep bucket and morphed it into a 2-deep and 1-deep one.

Now, that's all well and good, but this article isn't about HashMap resizing (exactly), its about a race condition.

So, let's assume that in our original happy HashMap (the one above with just 2 buckets) we have two threads. And both of those threads enter the map for some operation. And both of those threads simultaneously realize the map needs a resize. So, simultaneously they both go try to do that.

As an aside, the fact that this HashMap is unsynchronized opens it up to a scary array of unimaginable visibility issues but that's another story. I'm sure that using an unsynchronized HashMap in this fashion can wrack evil in ways unlike man has ever seen, I'm just addressing one possible race in one possible scenario.

Ok.. back to the story.

So two threads, which we'll cleverly name Thread1 and Thread2 are off to do a resize. Let's say Thread1 beats Thread2 by a moment. And let's say Thread1 (by the way, the fun part about analyzing race conditions is that nearly anything can happen - so you can say "Let's say" all darn day long and you'll probably be right!) gets to line 10 and stops. Thats right, after executing line 9, Thread1 gets kicked out of the (proverbial) CPU.

1:  // Transfer method in java.util.HashMap -
2:  // called to resize the hashmap
3:
4:  for (int j = 0; j < src.length; j++) {
5:    Entry e = src[j];
6:    if (e != null) {
7:      src[j] = null;
8:      do {
9:      Entry next = e.next;
     // Thread1 STOPS RIGHT HERE
10:     int i = indexFor(e.hash, newCapacity);
11:     e.next = newTable[i];
12:     newTable[i] = e;
13:     e = next;
14:   } while (e != null);
15:   }
16: }

Since it passed line 9, Thread1 did get to set its e and next variables. The situation looks like this (I've renamed e and next to e1 and next1 to keep them straight between the two threads as both threads have their own e and next).

Again, Thread1 didn't actually get to move any nodes (by this time in the code, it did allocate a new bucket array).

What happens next? Thread2, that's what. Luckily, what Thread2 does is simple - let's say it runs through the full resize. All the way. It completes.

We get this:

Note that e1 and next1 still point to the same nodes. But those nodes got shuffled around. And most importantly the next relation got reversed.

That is, when Thread1 started, it had node A with its next as node B. Now, its the opposite, node B has its next as node A.

Sadly (and paramount to the plot of this story) is that Thread1 doesn't know that. If you're thinking that the invertedness of Thread1's e1 and next1 are important, you're right.

Here's Thread1's next few iterations. We start with Thread2's bucket picture because thats really the correct "next" relations for our nodes now.

Everything sort of looking ok.. except for our e and next at this point. The next iteration will plunk A into the front of the bucket 3 list (it is after all, next). And will assign its next to whatever happens to already be there - that is, node B.

Woah. Thar she blows.

So right about now Thread1 goes into, what we like to call in the biz, an "infinite loop". Any subsequent get operations that hit this bucket start searching down the list and, go into, yep - an infinite loop. Any put operation that first needs to scan the nodes to see if its going to do a replace, will, you guessed it, go into an infinite loop. Basically, this map is a pit of infinite loopeness.

If you remember we noted that race conditions cause data corruption. Well, yeah, thats what we have here - just very unlucky data corruption on pointer structures. I'm the first to admit this stuff is tricky - if you found errors in my analysis I'll happily fix this post.

Now I had the happy fortune for a time of sharing an office with Josh Bloch who wrote java.util.HashMap. When I innocently mentioned he had a bug in his code given this behavior, he did indeed (to use Josh's word's) go non-linear on me.

And he was right. This is not a bug. HashMap is built specifically for its purpose and this implementation is not intended as threadsafe. There's a gaggle of ways to make it threadsafe, but in plain, vanilla, (and very fast) form - its not. And needless to say, you shouldn't be using it that way.

Race conditions are nothing to mess with and the worst ones are the ones that don't crash your program but let it wander down some unintended path. Synchronization isn't just for fun you know.

And nefarious uses of HashMap aside, I still attest - this is, indeed, a beautiful race.

Addendum: I've been yelled at a few times for calling any race condition "beautiful". I'll defend myself by our apparently human nature to generally call intricate complexity beautiful (i.e. waves crashing on a shore, nature in general).

Most races end up being about data-corruption. This one is data-corruption that manifests as control-flow-corruption. And it does so fantastically without an error (infinite loops notwithstanding).

As the evolution analogy goes, if you drive a needle into a pocket watch - chances are you'll simply destroy it. But there's a tiny chance you'll actually make it a better watch (clearly, not the case here). And another tiny chance you'll simply make it something "different" - but still, per se, functioning.

Again, my use of "beautiful" might be more appropriate as "a complex mutation with surprising non-destruction" :)

On Java Visibility

2009-06-08T08:00:00.000-07:00

A semi-famous example of broken Java synchronization is this:

class SomeClass {

  private boolean keepGoing = true;

    public boolean get() {
      return keepGoing;
    }

    public synchronized boolean set(boolean x) {
      keepGoing = x;
    }
}

I believe this example is in Josh Bloch's book "Effective Java" (which I now notice I somehow lost my copy in my most recent move - don't tell Josh).

The idea is here that someone (ostensibly) thought that they'd save some performance by not synchronizing a getter and just synchronizing the setter. There definitely is a cost to synchronization and needless to say, getting doesn't change anything - so why bother paying that cost for gets?

As has been pointed out long before this post, without a synchronize on a getter, there is no guarantee of visibility when doing the get. That is, if one thread calls set(false), there's no guarantee that any other thread will know it happened.

Consider code that might use the code above:

class SomeOtherClass {

  SomeClass keepGoing = new SomeClass();

  class Thread1 implements Runnable {
    public void run() {
      while (keepGoing.get()) x++;
      System.out.println("done1");
    }
  }

class Thread2 implements Runnable {
    public void run() {
    keepGoing.set(false);
  }
  }
}

Let's say you start a Thread1 running. And of course, keepGoing.get() is true, so it just keeps looping along. Then lets say an eternity (or maybe 2 seconds) later you start Thread2.

If we had reliable visibility, Thread1 would end the moment after Thread2 sets keepGoing to false.

If you've read Josh's book, its no surprise that it doesn't. Specifically, Thread1 doesn't end. It keeps going. Thread2 ends happily and Thread1 never ends.

The only interesting part to me was that this always worked. Always. Adding to the complexity of visibility concerns is that memory is "leaky". That is, even without guaranteed visibility you often get unreliable visiblity.

So, I dug a little deeper. If you're adventurous enough to grab a debug-version of the JDK and run it with the -XX:+PrintOptoAssembly option. You get to see the optimizations the JIT are making. That is, you see the assembly code version of your Java code - post-optimization. Check out Koshuke Kawaguchi's Blog for some instructions.

So here's Thread1's loop code after JIT optimization at runtime:

02c movq R10, precise klass manybrain/test/Main: 0x0000000040a50968:Constant:exact * # ptr
036 movsbl R8, [R10 + #596 (32-bit)] # byte ! Field manybrain/test/Main.keepGoing
03e testl R8, R8
041 je,s B4 P=0.000000 C=147944.000000
041
043 B2: # B3 <- B1 Freq: 1
043 movl R8, [R10 + #592 (32-bit)] # int ! Field manybrain/test/Main.x
043
04a B3: # B3 <- B2 B3 top-of-loop Freq: 1e-35
04a incl R8 # int
04d movl [R10 + #592 (32-bit)], R8 # int ! Field manybrain/test/Main.x
054 testl rax, [rip + #offset_to_poll_page] # Safepoint: poll for GC # manybrain.test.Main$Thread1::run @ bci:14 L[0]=_
# OopMap{r10=Oop off=84}
05a jmp,s B3
05a
05c B4: # N53 <- B1 Freq: 4.76837e-07
05c movl RSI, #27 # int
061 nop # 2 bytes pad for loops and calls

If you're old school, welcome home.

If you're not, then this might look like a lot of goop. So let's parse out just the interesting parts.

Line 41 is an conditional jump. Basically, if keepGoing (per the comment in line 36) is false, we jump to line 5C (label B4) and end the code segment. You and I know that keepGoing started true, so basically that jump isn't followed.

Lines 43-5a are the loop that does x++.

And checkout line 5a. That is an unconditional jump back to the top of the loop. So what does all this mean? That the JIT did some very aggressive optimization. In fact, remember our original loop from Thread1?

while (keepGoing.get()) x++

The JIT has optimized this to:

if (keepGoing.get()) {
  while (true) x++;
}

No wonder the loop never ends. You've got bigger problems now than a little leaky visibility issue. I'm positive I'm oversimplifying, but effectively the JIT saw your get method wasn't synchronized and made the assumption that it could optimize as such. If you didn't guarantee visibility, it didn't need to either. Obviously, add the synchronization modifier to the get method and all this badness won't happen.

Moral of the story is much like the inevitable topics discussed at a lunch with Jeremy Manson - you can't optimize away correct synchronization; as all you'll probably do is optimize the "correctness" part away.

How long before an email is removed from the system?

2009-01-26T10:38:00.000-08:00

I get this question a fair bit. How long, exactly, is an email available to be read after it enters the system?

Its hard to answer because its literally dependent upon the incoming rate at which email arrives. In effect, new email "pushes out" old email. That's a bit simplified as some email is thrown away based upon spam rules (if you get 100,000 emails with the same subject, you can probably say they aren't all going to be read).

So, whats the average? This weekend, email was lasting around 10 hours before getting "pushed out" of memory. I've seen that jump down to an hour - but mostly its around 5-7 hours.

I'll note that the primary Mailinator use case is that people tend to read email soon after it arrives. In other words, if email only lasted 15 minutes, we'd actually fill the needs of many users.

(I'd be *very* interested in hearing ways you use Mailinator that need mail to stick around longer.)

No, Mailinator didn't spam you

2009-01-08T22:04:00.000-08:00

I must say, it's rare these days but I still now and then get emails from folks that think Mailinator spammed them. As it says on our contact page, this is pretty unlikely. I say that because Mailinator is custom software and that software contains no specific way to actually have a user "send" email. There's no chance of Mailinator being an open-relay as it stands. And of course, there is no place at all on the site to accept an email for sending.

I suppose its possible, but it would involve a hacker breaking into the server, installing some other email server (which would likely conflict with mailinator itself), configuring it, and then start pummeling it for their nefarious purposes. Given that any self-respecting spammer has a billion zombies at their disposal and that this would definitely be discovered very quickly (my colo vendor loves to watch my bandwidth), it doesn't seem like an efficient way to spam.

In any case, I still get accused of spamming at times. And all that accusation takes is for a spammer to forge the return address as a mailinator address. Let me tell you, forging a return address is stunningly easy. Here's 3 million or so guides how to do it if you're wondering.

Below is an actual email header someone sent me.

The interesting parts are really the first two lines. As you see the forged return path is ronb@mailinator.com. Now if you know mailinator, you know ANYONE can check that box. It belongs to no one and everyone (as outlined in the FAQ - Mailinator guarantees NO PRIVACY. All emails are viewable by ANYONE).

The 2nd line (i.e. Received:) shows the IP (and dns) of the server that actually sent the email. Something at abac.net. That looks like a hosting company somewhere. One thing I can tell you though is that that server has zero to do with mailinator. The spam email never ever touched the mailinator server. So even if I devoted my life to stopping this email, there's nothing I could do.

Return-Path: ronb@mailinator.com
Received: from 216.55.169.94 (216-55-169-94.dedicated.abac.net
216.55.169.94)
by smtpin4.mail.de.uu.net (8.14.1/8.14.1) with SMTP id n083RPV6001157;
Thu, 8 Jan 2009 03:27:26 GMT
Message-Id: 200901080327.n083RPV6001157@smtpin4.mail.de.uu.net
From: "RON" ronb@mailinator.com
Reply-To: "RON" ronb@mailinator.com
To: xxxxxxxxxxxxxx --> edited

This is sort of similar to a phishing attack. Someone gets an email from their bank, then goes to the phish site, then loses all their money. In truth their bank had nothing to do whatsoever with any of that but the bank still gets blamed.

The saddest part for me is that even after I respond to people showing them the real culprit, its not uncommon for them to stay mad at me. I suppose its because they then don't know who they're going to yell at now and I'm still available for the job.

Mailinator is about letting you protect your real email address. It might be to prevent spam but at times it might even be to receive spammy email they really want (just not at their primary address).

Regardless what you use it for, it won't email you. It just doesn't do that.

Plenty of people threaten to blacklist Mailinator from ever sending them email again. Yes, please do! As I've said in the past, feel free to put mailinator.com on the tippy-tippy-top of all your spam blacklists. Mailinator doesn't send any email at all - so you can be sure any email that looks like it came from a mailinator address is forged. And I'll sleep just fine if such email gets blacklisted.

Dear World, email addresses are not identity

2008-12-08T15:34:00.000-08:00

Its no secret that part of putting up any website or service is the consideration of security measures to stop people from abusing the system. And as you can imagine, those particular issues are probably doubled or tripled with a site like Mailinator (and of course Talkinator). Needless to say, I've gotten pretty good at unorthodox security.

Anonymity does indeed breed bravery.

The normal Mailinator use case is that you have a need for a quick email address to sign-up for some web service. Some people however, use Mailinator as an actual primary email repository. With the RSS feed, it acts as a convenient dropbox for newsletters or other semi-private email needs.

Unfortunately, there are some wacky funsters out there that think it might be fun to sign-up for some website over and over and over and over (and over). Often this is for some site that has "vote for your favorite band" or "sign up for a free gift" or some such thing.

The primary flaws of these systems is not Mailinator - its that these websites equate the idea of identity with email addresses. Seriously, long before Mailinator existed I had 3 or 4 email accounts that I actively used and another 10 I had probably abandoned. I'm not sure what those site designers were thinking - Mailinator or not, email addresses are FREE.

If you give someone something that has any value at all in exchange for free email addresses, they're going to ask for lots. I'm probably in a unique position to view this, but I see this idea as incredibly broken.

One option I had was to simply ignore this idea. Let crafty script-writers create systems that sign up for Wink accounts 1000 times an hour. (As I write this, I'm watching some automated system using several hundred IPs trying to sign-up for wink.com using Mailinator - and watching the Mailinator system dutifully send each request to the abuse page).

The problem with this is that someday, wink.com will catch on. And they'll ban Mailinator. This is sadly, a wonderfully broken solution to a still existing broken site design.

The problem for me is that I likely have legitimate users that want to sign up for Wink - and I want them able to do so (and I imagine Wink might want more users too, so by extension they'll lose some or all of the ones I lost). What's insanely broken for sites banning Mailinator is that there are tens of Mailinator-copy-cat disposable web services out there. Or even worse, someone with access to a server and a domain, who can install sendmail and create a few thousand accounts. Simply put, banning mailinator is like catching a single mouse and thinking you've solved the mouse problem.

You stop the bad guys, but for about a day until they implement a new system.

I had an interesting discussion with an acquaintance recently. During the conversation I described Mailinator to him. His mouth gaped open and told me he would look into it and probably ban it from his site. I asked what he would be banning it "from". He said he had a trial piece of software that people could sign-up for and download. And he wanted their real information to email them later (i.e. I did my best not to say that he was sending "spam") to see if they wanted to buy.

I noted that sometimes when I download software to try, I do want to enter my real email. I'm interested enough to want to be registered. But other times, I'm just in browsing mode. If given the chance I'd download and check it out, but if you give me too much impedance I'll probably just go check out his competitor.

In those cases when I'm just browsing, I'll use mailinator.

In other words, there are 3 types of potential customers. Those that don't care about his software. Those who really love the idea of trying his software and will do anything to do so. And those who are on the fence.

For obvious reasons, Mailinator is my "on-the-fence" tool of choice. If he banned it, he'd be refusing some subset of those potential customers. So it basically comes down to the question - whats better?

1) Definitely get user information you can spam later - or
2) get your product in front of as many eyeballs as possible.

Also noting the fact that NO email insures any relation to an actual person whatsoever (including yahoo, gmail, hotmail, etc.) - whats the point?

We continued our discussion and agreed that from a marketing perspective, you actually don't want to remove the email sign-up altogether. It actually brings value to some customers. If you remove it or make it optional, most everyone will skip it just to get to the goodies. But by leaving it and knowing that some people, using Mailinator or Yahoo or whatever, will give you temporary email addresses, you're maximizing your potential customer base.

It didn't hurt my argument to mention a few other disposable email services that he'd have to ban too. I sure don't know them all - they seem to come and go a lot. And that surely doesn't count ones that run semi-privately. Basically, it would be a fulltime job to keep up.

Oh. So, back to our script kiddies above. Mailinator includes a system to stop scripts from signing up for websites over and over. I love fun algorithms/data-structures so your homework can be to design something like Mailinator's abuse trigger system - a key-value datastruct that ages with time and is refreshed by lookups that come in at some notable (and tweaked) rate (in the same ballpark as a LRU cache, but definitely more dynamic).

Its unlikely a human will set-off the triggers but its possible. The sad part (for script writers) is that the algorithm doesn't trigger until their script gets going, so its probably a bit heart-breaking to spend a few hours perfecting a script to scrape Mailinator and then have Mailinator detect it only once it gets going and shut it down hard.

The first level is the Abuse Page. If you push it, Mailinator will ban IP addresses - but only under certain conditions. That's rather an imprecise way of stopping abuse. In addition, it looks for patterns of mailbox usage regardless of IP. An obvious one is that if one subject "Welcome to Wink!" shows up a lot in the read emails. Sadly, its difficult to distinguish valid users trying to sign-up for wink amongst the botnet hitting right now - so they'll probably get the abuse page too for the time being.

Potential site abusers taught me a lot and hardened the site considerably. Abuse attempts are still a common occurrence but far less normal than a few years ago. I assume many scripters went to less caring disposable services.

I often get asked if I care if sites ban Mailinator. I don't really. In some cases its prudent if you really do need to email people that use your service. In most cases however, its simply a knee-jerk reaction attempting to patch an otherwise flawed system. Not only is it a sure way to eliminate some potential customers, the flaw will show up again soon when the abusers shift to another method - and probably another method without Mailinator's facilities to stop scripts.

In the end, there is no real identity on the Internet. At least none past an IP address and a subpoena. At best, email is optional identity. And prudently, it should probably be treated that way.

Reserved Names in Talkinator

2008-12-04T10:04:00.000-08:00

Talkinator now includes a beta feature (ok, all features are beta) to allow reserved names. Each room can have only one and the person owning the site (and cut and pasting the code) designates the reserved name and the password to access it.

If you already have a Talkinator in your site, just click the </> icon (i.e. get the code!) in the upper right of any talkinator chatbox and create your Talkinator with your reserved name!

Please let us know how it works and especially if you find any bugs! (email to support@manybrain.com)

So, Mailinator's core-duo server now gets 3 Terabyte a month

2008-11-11T15:55:00.000-08:00

Its been awhile since I've updated my article The Architecture of Mailinator. That was a long time ago and really and tons has changed. To save you reading that article if you're in a hurry, the crux was that Mailinator was originally based on the unix email program sendmail, which pretty much died at around 800,000 emails a day. And then it discusses the design of the fully custom SMTP server designed for mailinator.

In that article, Mailinator was running on a single AMD cpu and processing 3 million emails a day or so. Well, somewhere in there I got a deal with my hosting company and was running a dual dual-core opteron machine. Mailinator ran on that for about a year as our email load doubled a time or two.

Eventually I needed a server for another project and realized that giving Mailinator that dual-opteron was pretty overkill. So I "downgraded" the machine to a core-duo. Which is where it runs now. In the aforementioned article we were getting 3 million emails a day - now I sometimes see 2 million an hour.

For the geeks out there, this is still of course a highly-multithreaded java implementation often running 4000 simultaneous threads. Pure Java I/O (not that stinky and slow NIO.

Today I got an email from my hosting server that I "surpassed my allotted bandwidth". I'm pretty used to getting subpoena's and such, but not so many of these.

Apparently, Mailinator received 3.1 Terabytes of email in the last 4 weeks.

Thats (ballpark) 480 million emails or so in that 4 weeks. As I've said before, that doesn't compare to any bigger email service, but then again, I bet they have more than one server :) I'll also emphasize that numbers I usually throw up here were measured or calculated by me - that 3T number comes from my provider so I hope its accurate as they want money for it !

Thats a LOT of spam (sure there's web traffic in there which might be hefty in other circumstances, but it doesn't hold a candle against the mail traffic).

I've priced things out and although my server still has plenty of computing capacity left, I'm clearly hitting an artificial (i.e. pay for more) bandwidth limit. It makes far more economical sense to simply get another server (per all pricing schemes I find) than to buy more bandwidth on this one.

So, someday I'll get around to writing the next "Architecture of Mailinator" article and although it almost sounds funny to me after all this time, I'll likely be talking about "both" Mailinator servers. Not just our trust little core duo.

The Revolutionary Fuzzy-Logical Super-Duper Mailinator Captcha System(tm)

2008-11-03T07:52:00.000-08:00

So, after many years of resistance for various reasons, Mailinator has finally implemented a DELETE button. That's right. The number one frequently-asked-question that isn't in the FAQ has been answered.

The reasons for its delay are many. In the early days I was too worried that someone would write a script that would attack the system and delete every email out there. Then after a year or two of people writing scripts to attack Mailinator in every other way - I had enough AI built into the system to stop most of it. (As always, I define "attack" as something that could bring the system down or ruin its use for other users - spamming Mailinator is really not on the radar, its not our place to define what is spam in this context).

Subsequently, I looked at implementing it a few times but tended to back off. If you've read this blog awhile you know that Mailinator and Talkinator at least partially existed as a testbed for implementing server ideas (some earlier articles: The Architecture of Mailinator and Benchmarking Talkinator).

And while implementing a delete function might sound simple - it had some interesting complexities in the architecture. In any case I'm happy to say - its finally here.

And here's the best news. To thwart all those rascally script-kiddies (who probably have seen the rather hidden but sometimes funny Mailinator Abuse Page) we're proud to announce the "Revolutionary Fuzzy-Logical Super-Duper Mailinator Captcha System(tm)" !!!!!

Thats right. The great service and ironclad spam (fighting/loving/tolerating, your call) abilities of Mailinator are now in a REVOLUTIONARY captcha system! We could have used a pre-written captcha system but that would have been sadly inferior and very non-mailinatory!

Just look at it and you'll know that this is going to take some major AI, image processing and statistical analysis to crack past this sucker.

Oh. And one more thing.... (got that from Steve Jobs)

We at Mailinator are happy to announce that this Captcha system is just TOO GOOD not to share with the world. So, you guessed it, we're releasing it as a web service !

You can now include The Revolutionary Fuzzy-Logical Super-Duper Mailinator Captcha System(tm) in your websites too !!!

Here's the highly-technical, low-level URL:

(edit: url momentarily removed due to immense user respo.. ok.. just a bug... be bak soon)

(where yourDestination.html is where you want the captcha to go after very securely and precisely verifying the humanness (and also non-computerness) of your user).

Put this in your web pages and sleep well!

(Disclaimer: This captcha system, although highly-advanced and chock-full of the latest advances in modern artificial intelligence does a better-than-expected job of verifying humans are humans. However we've noticed an eensy bug that seems to prevent it from sometimes (or "usually" really (actually its probably more like "mostly" (sometimes its pretty "always" tho))) figuring out that computers aren't humans. So consider this and "alpha" release.. or maybe "pre-alpha" or something like that. A "prototype", yeah, its a "prototype").

PS: Have your own idea for a captcha? Make the image 80x24 pixels and send it to support@manybrain.com !

PPS: The delete function is in beta. Please let us know of any bugs.

The Developer's Mailinator

2008-10-23T10:17:00.000-07:00

I get a surprising number of emails from developers testing some sort of emailing system and wondering if its "ok" to send those emails to Mailinator.

Of course - the answer is yes!

In fact, thanks to an email from one of those developers named Dave, I've actived port 587 on Mailinator to be an active receiving port (in addition to port 25).

Mailinator has always accepted email addressed to any domain - (i.e. have a domain name? point your MX record to the mailinator server and it will arrive just fine) so it really is a pretty solid testbed. Remember, you can see the raw email (headers and all) by clicking the "text" link on any mailinator show-email page.

The only caveat here is that Mailinator's defenses don't come down ever - even for testing. If you want to send a few thousand emails to Mailinator, it very well might start throwing emails away. As is sort of our goal - we don't mind spam (actually - given the nature of Mailinator - who are we to even define what spam is?). But we ARE interested in the Mailinator service staying up and running. Any legal (and nice - please read terms of service) use is fine by us - but if Mailinator thinks you're being overzealous, we'll probably side with it :)

Mailinator for your IGoogle Homepage

2008-09-02T08:12:00.000-07:00

Frank Ellermann has created a very pretty integration of Mailinator into the iGoogle homepage. It now installs with just a click!

Thanks Frank!

See it here: Mailinator iGoogle.

Bob Cringely on Talkinator

2008-07-30T09:12:00.000-07:00

Bob Cringely, well-known computer industry guy wrote a nice article about the "Five percent solution". He uses Talkinator as prime example!

I like this idea - I remember many years ago someone telling me about Google and I thought they were nuts - "I have Altavista - why do I need this Google thing? (silly name by the way)". Or even a bigger one for me was when I first heard of YouTube - so some little guy is going against an already established Google Video with their only real distinguishing feature is that they only allow "short videos"?

Bob's right - sometimes a very established idea simply needs a 5% twist to take it to the next level.

Thanks Bob !

Talkinator notes

2008-07-24T18:06:00.001-07:00

When Mailinator came out, people simply couldn't get over the fact that there was no login - I mean, how can you have email that doesn't have a login and password? It was a very strong paradigm shift for some folks.

For Talkinator, this paradigm shift has so far been the shock of "all of a sudden" being in a live discussion in a Talkinator room. Again, they didn't sign in or realize it would happen. Its rather common for people to not believe its actually live (often they assume its some sort of advertisement).

It took awhile for people to get-it as far as how Mailinator works. Paradigm shifts come slow - but they're often a lot of fun.

Mobile Mailinator

2008-04-12T01:33:00.000-07:00

Great idea - a nice write-up on how to use Mailinator on your mobile phone!

http://ergo.rydlr.net/?p=63

How fast is Java Volatile? or Atomic? or Synchronized?

2008-03-30T14:18:00.001-07:00

This is a question I've wondered awhile. And I've gone ahead and made a stab at finding out. One question that I had was I had always heard that AMD chips have an on-cpu memory controller and Intel's don't (at least not until they're upcoming Nehalem line). Now I'm first to admit I'm talking above my knowledge (please correct profusely).

But the idea was that some operations (notably I've heard volatile reads which I didnt test here) are very similar regardless of volatility.

I spent a few hours making up a benchmark. I'm publishing the benchmark below for one important reason - to get it right. This is big disclaimer: Don't trust these results - at least with any precision. I do conclude that contended synchronization is expensive - thats a pretty expected result so hopefully we're somewhere on the right track.

However, Java benchmarks are very hard to right. So hard in fact that they do a hardness-overflow and end up looking easy - which is precisely what makes them so hard to get right. Hence - given that only I have seen this benchmark, its pretty likely to be broken some (and hence why its published here for public review to help fix).

With all that...

What I've tested here are storage operations. Specifically:

Storage to a local variable (mostly used as a control)
Storage to an instance variable
Storage to a static variable
Storage to a static volatile variable
Storage to a static AtomicLong variable
Storage to a static variable protected by syncrhonization

I tested this on 3 different CPUs, all with JDK1.6 and variously on Linux and Windows. One very important point is do *not* compare the absolute numbers in the graphs directly. In no case are comparing the same CPU across 2 operating systems. The CPUs are wildly different in capability.

So we can explain more with a graph:

This is the run on a dual-core dual-Opteron machine. Note the blue-bar is local variable storage. As you'd expect it simply destroys everything else. This is very probably because the smart JVM went in and realized it could optimize the assignment out of the loop. Note the blue-bar just gets happier and happier as we add cores. The threads are completely unaffected by each other and we get actual linear increases in power. Even at 5 threads (note we have 4 cores) we get some speedup.

The orange and yellow bars are instance variable and static variable stores respectively. Some level of indirection or inability to optimize jumps in and makes things a bit more sane. Surprising we get little or no improvement as we add threads.

The three thread-visible methods of store are so slow they barely show up. This brings up an important point, let's assume you wrote an application and made it correctly thread safe. Using synchronization or volatiles or whatever. You *cannot* optimize away synchronization in the name of performance. It simply can't be done. (conversely, if you could then it wasn't correctly thread-safe to start). You can however, at times, replace one type of synchronization primitive with another.

That being said, lets take a closer look at the same graph with just the barrier memory store operations.

Volatile and Atomic are neck-and-neck. Synchronization is still a big loser, especially after we contend giving it two threads.

One other anecdote - while running the non-memory-barrier benchmarks, my CPU meter showed 100% user space usage. With these benchmarks however, it went to 30-40% kernel cpu usage.

Ok, jumping to the Core2Quad Extreme CPU. Definitely a faster processor but with a different memory architecture.

Local store again goes flying. Oddly the instance store doesn't do much but the static store increases nicely with the cores. And, once again, we can't even fricken see the barriered writes.. so here they are.

Look how great the JVM does knowing that just one thread is out there. It can really optimize the code to elide or at least marginalize the impact. Surprisingly, synchronization still gets nailed across the board.

Note that according to what I have so far, a synchronized static store is something 50 times slower than a simple static store. And 10 or so times slower than a volatile static store.

One more.. and its a fun one. Windows XP running on a Dothan. (Funny, I realized I hadn't directly known that Dothan was single core, but I just assumed it after I saw the graphs.)

Crazy. Add a few threads and performance doesn't even budge. Of course, with only one core a system can completely avoid all the complex architecture keeping cores in sync. Why its local writes are worse as compared to instance beats me.

Also, although I said don't compare graphs, this little single core Dothan beats the Core2Quad in all barrier-ed writes after 2 threads. Note that on the local writes, the Core2Quad is something like 50 time faster. But even on the simple static volatile store - at 2 threads, the Dothan is now twice as fast.

So. Pardon my informalness here - I'm actually quite expecting feedback to break this code in grand ways and force me to redo all the runs and rewrite all the text (which I'll be happy to do if we get a clean bench out of this). I have no intention of making very specific claims once this benchmark is firmed up - but I would like to have a "sense" of the costs of these different operations.

Source code for the Benchmark.

Introducing: Alternate Inbox Names

2008-03-27T22:26:00.000-07:00

Mailinator helps thousands of people every day avoid spam. It's been a great success.

One point that always sort of bothered me however was that by giving out a Mailinator address, you were basically telling people not only how to email you, but also how they can check *your* email. If I say "Email me at binkypop@mailinator.com", the whole world knows where to send and read my email!

Well, this is no longer a problem. Today we added "Alternate Inboxes" to Mailinator. We've always had alternate domains, but this is quite different.

Now, if you check any inbox on Mailinator (say, binkypop), listed on the page is the alternate inbox name. All alternate inbox names start with "M8R-". For example, for binkypop, the alternate inbox name is M8R-yg1hkn@mailinator.com.

So simply put, if you email binkypop@mailinator OR M8R-yg1hkn@mailinator.com it doesn't matter. Both emails would end up in the binkypop inbox (and nothing in the M8R-yg1hkn inbox). The only way to find out an alternate inbox name is to check the inbox here first - thus if you give out the alternate address, there is no way for anyone to guess that it actually goes to binkypop.

So.. pick yourself a favorite Mailinator inbox name (make it big, long, and hard-to-guess please!), go check the inbox to find out the alternate inbox name, and hand it out all over the web (of course, alternate domains work on alternate inboxes too!).

Then check the box (or RSS it) knowing only you know thats the real destination. Also, keep in mind - you don't have to remember the alternate inbox name ever - you just have to remember the real destination address. You can always go to Mailinator and get the alternate anytime just by checking the inbox. And of course, the regular think-up-on-the-fly-and-use Mailinator you know and love hasn't changed a bit. This feature is purely optional (I'm surprised how fast people started using it!)

(This feature is in beta and we might end up changing the alternate address scheme some)

This is cool. Alternate inboxes really do up the ante in Mailinator's spam fighting abilities. Enjoy !