So, what is Postmark?

Postmark helps web apps deliver email effectively, while giving you the tools to monitor and track delivery. It’s specifically focused on transactional email, so messages like welcome emails, invoices, comment notifications or any other emails that are triggered by an action from your site. Through our existing knowledge of managing large volume email in Newsberry, Postmark provides a more reliable solution to make sure your emails are received. If there are issues with delivery, Postmark gives you the tools to correct the problems and improve.

The service works entirely through a robust API (xml and json), which you can use to replace your existing SMTP code. From the start, we’ll have code samples for many languages (Rails, .NET, etc) to make integration as easy as possible. We’re also looking for help from our readers, so if you have experience with PHP, Java, ColdFusion or want to write plugins for other apps, please email us.

Why is this better than a local mail server?

If you’ve ever built or launched a web application, you know that setting up an SMTP server is pretty easy. The basic steps can have you up in running in minutes. What you may not know, is that doing it correctly is complex. For instance:

• Setting up authentication like SPF and DomainKeys
• The importance of Reverse DNS
• Managing connection and message rules for each ISP
• Applying for ISP whitelisting and feedback loops
• Accreditation with ISIPP and ReturnPath
• Tracking bounces and spam complaints
• Understanding volume over time

The list is longer, but hopefully you get the point. With Postmark, we bring these benefits directly to your application so those importance emails get to your customers or members. If delivery problems arise, you’ll know right away.

When will it be ready?

We’re in the final stages of launching our private beta. The beta will be available to a small group, then opened up more as we learn and tune the product. If you are interested, please sign up for the beta.

Where free succeeds

Where the “free” model plays an important role is not the fact that it is free, but the ability to prove your product. The “try before you buy” concept is not new. By offering a free plan, you give your potential customers a risk-free opportunity to try your product. This can go a very long way compared to “30 day trials” or a well designed marketing site.

The other area where free has value is word of mouth. The majority of our accounts are free, while this puts load on our system and support, it also creates a loyal base of people who rave about our product. We’ve seen many examples of freelance developers on free plans who then convince their clients to use Beanstalk on a paid plan.

Finally, the last point of value is about creating movement. When we started Beanstalk, our goal was to convince more people to use source control and improve their process. It amazed us how many people still did not have proper source control for their projects. By offering free plans, each account is a small triumph for our goal. The “greater good” value eventually brings back more value to the business as more people use source control.

Where free fails

With all that said, there are definitely cases where free does not work. I was doing a quick review of our stats and noticed something interesting. We get a nice number of visitors from stackoverflow.com. One post in particular sends the most traffic, which is titled Free Online Private SVN repositories. Here are some stats for a given period:

• Visitors: 2,617
• Free sign ups: 728
• Paid upgrades: 6

By breaking these numbers down into conversion rates we see some interesting results. First, the conversion rate from visitor to sign up is huge, almost 28%. The conversion from sign up to paid on the other hand is awful, only 0.8%.

What does this tell us? It’s pretty obvious, people who are looking for free will most likely not pay in the long run anyway. If we don’t offer enough resources at the free level, they will probably just go somewhere else. Knowing this, it helps focus attention on the people who are looking for value and willing to pay for it and service them as best as possible.

Learning from the numbers

We are just beginning to understand the details and still have a lot to learn. We’ll continue to review results and study conversions to help make our products more profitable. In the long run, this helps us grow our business and expand our offering for our customers. We’ll be sure to write the results here as well.

Do you run a web product? We’d love to hear your results as well. Please share in the comments.

Beanstalk already has pretty amazing conversion rates from free to paid (almost 10%!). Of course, we always want to improve. One attempt was our recent public site update, which included customer logos on the homepage, a buzz page, and a more detailed partners page. Building credibility and showing that others already use your product successfully goes a long way, especially if you have some recognizable brands. So far we’ve noticed great results.

Pricing page redesign

Our most recent update is the pricing page. While the site explains the purpose and features, the pricing page helps explain plans and the value offering between free and paid. It’s also meant to reassure visitors about what they get and that they are making the right decision (or not).

If you remember our previous sign up page, it was quite busy. The chart helped show the features, but it was hard to read. As we added new plans, it became even more complex. With our new pricing page, we decided to prioritize the items. First, people want to know the cost. We added a simple block to the top that displays each option, including the free plan in plain sight. The second priority was to show the difference between free and paid. Slightly below the chart, we designed easy to read blocks that reiterate the features between free and paid. This allowed us to get rid of the complex matrix and focus on detailed descriptions for each major benefit.

We struggled a bit on whether we should highlight or downplay the free plan. The trend lately is to downplay it, but we still feel free plans offer huge value for word of mouth and eventually grow into loyal customers. In the design process, we created some other revisions that we will test. Our goal is to give it a try for a week or two and measure the results. When it comes to improving conversion rates, it’s all about testing and revising. I will post a follow up with some more detailed statistics once we get through the tests.

New pricing page

Old pricing page

As most of you know, the time it takes to design a feature is not equal to the time it takes to implement and support it. For instance, designing the screens for Git support in Beanstalk did not take long at all, but getting it up and running is a different story. With that imbalance, our designers often have some extra time for playing around or client work.

So, on a limited basis, we’ll be offering design iterations in weekly increments for one flat rate. With this, you get one full-time designer, myself, and Igor to QA test. We’ll put together a plan, set priorities, and deliver results.

Interested? Learn more on the hire us page.

Refactoring of course is not an easy task, and often you need to get a bird’s eye overview of the system before starting to improve it. You should see all classes, methods, modules, how they are structured and interact with each other. In an ideal world you would use a gigantic display for that task that would allow you to see all the stuff simultaneously without scrolling. This would make reviewing big chunks of code much easier. But unfortunately we don’t have such displays yet. Even 30” is not enough for this task. So why not use the space of your desk instead?

So, here is the simple trick. Try printing your code on paper and arrange the sheets on your desk in a way that all of them are visible at once. Now, when you can see all of your classes and methods, you can start re-arranging stuff. Use a red marker to move methods and strike out deprecated code. Don’t hesitate to write-down your comments on the sheets as well. Check out the photo of my recent paper refactoring:

This method always works for me, hopefully it will help you too.

Preparing data

When you are testing existing or new features of your application, it’s very important to have adequate testing data. The more data you have, the more bugs can be spotted early in the testing/development stage.

Having good test data is very important for both manual and automated testing. Before writting the tests, it’s wise to think through and see which data you will need to run the tests.

Some of the questions you need to ask yourself before starting automated testing are:

• What functionality do you want to test?
• Will you be able to test the functionality (limits of automated testing tools, etc)?
• What data do you need for the testing?
• How do you organize the test cases to be reusable and easy to use?

Before you start writting the test cases, think of all the data you will need and the stages of the testing which will require it. This can save a lot of time and make your automated tests consistent. You will need to have data which is needed in all stages of testing (like login credentials), and data which would be used during testing. Managing data will probably include 4 steps:

• Prepare data needed in all stages of testing
• Prepare test data which would be used during testing
• Delete all data used during testing
• Delete all data needed in all stages of testing

By following the steps mentioned above, you will have the data you need during the tests, and you will leave the application data intact – by deleting all data you used during testing, you will get the system back in the state in which it was before testing. You should always make sure to set the application back to the original state, this way your automated tests can be run unlimited number of times and would not leave the application in an undesired state.

Reusability

When writing test cases, though you don’t need to be an expert programmer, try not to repeat the code too often. You would probably want to run those test cases daily, and updating a bunch of repeating code in your test cases can be hard and time consuming.

Always try to separate the parts which you will reuse often in your test cases.  For example, a developer changed a part of the code, which you test often. If you did not repeat your code, you would probably need to update your test cases in only one place. Same goes for data. I usually store data like login credentials and automated testing settings in separate files, so I can easily update them later.

Use meaningfull names for the test cases

Using meaningfull names for the test classses and test cases can be handy too. It will allow you to find the code you need to update easier. I usually call the test classes by the name of the section I am testing. For example, in web applications, I use the name “Account” for test class which will contain all related test cases to “Account” section in web application. This way I can also find the test class easier when I want to include/exclude it from the test suite for the next run.

Always be prepared

The most important part of automated testing, as for any other type of testing is being prepared: being familiar with the system which needs to be tested. Thinking through all the actions needed to create automated tests and organizing data and test cases will help you create a test suite (test cases) of high quality.

When we decided to partner with Engine Yard, I can say it was one of the best things that happened to Beanstalk. We let them take care of our infrastructure while we worked on strategy, design, usability, code, and improving the product overall. When it comes to Rails, no one does it better than Engine Yard. The problem is, Beanstalk is a lot more than just a Rails app, we’re a hosting service on our own. As we’ve grown, our need for a unique hosting environment became more apparent.

Making the decision

I have to admit, it was really freaking hard to move away from Engine Yard. They literally take care of everything and their support team is awesome (and a cool group of people). At the same time, we needed more control and a specific configuration for things like svn, storage, apache, virtualization, and so on. So, we decided to go with the leader, Rackspace. They are well known for their network, support, and are able to deliver the managed complex hosting environment that we needed.

Getting help where it is needed

Like I said, we know how to build web apps, but we are not server admins. Moving the amount of data and configurations we have to a new host is a scary process. To make it smoother, we reached out to the experts. For help with setting up our Rackspace environment, we contacted Tom Copeland at InfoEther. He’s been a big part of the process for setting up our new environment, tuning our servers, and understanding our current bottlenecks. It’s obviously a big help that he already runs RubyForge. When it comes to Subversion, we contacted Blair Zajac from Orcaware. Blair has provided some advice for us in the past. This time around, we had him review our Apache and Subversion setup to make sure we covered everything.

Better performance and stability

Beanstalk has had its share of performance problems in the past. The majority of these problems were due to our storage and IO performance. In our environment at Rackspace, we have invested heavily in the best hardware and storage options available. This should reduce most of the headaches when it comes to the speed of checkouts and commits in Subversion. I’m sure we’ll have some kinks to work out after the migration, but the new environment will benefit all of our customers (free and paid).

The migration

Planning the migration was tricky, but we’re confident we have it figured out. Over the next couple weeks we will be migrating groups of accounts over to the new Rackspace environment. We’ll have plenty of notices before it happens and we expect little downtime for each account. With the migration, we also have some really nice improvements like Subversion 1.6 support.

Want to know more about our setup?

This has been a really detailed and exhausting process, so I’d like to share more about the actual server architecture in a future post. Once we get through the migration process, I will post some more details about the new environment. We’re really excited about it.

I am a programming language, development tool, and platform junkie with many interests, but at the moment I am serving as a .NET developer for Newsberry. Newsberry is an old project with a huge codebase. And guess what? I have to learn it. I’ve been doing that for some time now, and I daresay I have been making good progress. Here I’d like to share some tips and techniques both on the technological and philosophical level that have proven themselves useful.

Keep your thoughts organized and don’t forget the big picture

I know many people just get the source code, compile it and start running it in the debugger, trying to get an idea about what is going on. This may be a good strategy for smaller projects and one-off scripts that you find on the web, but it does not scale well. At best, you will achieve good knowledge about a part of the system (the portion of code you have read through), while you will have no idea about the rest. Don’t forget that there is a good chance that you don’t understand anything as the code can only answer the “How is this application built?” question. You will not get the “Why” part, and that is often crucial in order to understand the “How” bit.

My first steps were to get an account over at http://www.newsberry.com/ and start playing with the system. I had to first learn how it works as a user, so that I know what problems it solves. Once I felt confident I knew it at a reasonable level, I got to the source code part. I had to learn a lot about the problem domain – I was an email marketing newbie, and the body of knowledge on the subject is huge.

After you know enough about the business domain, you can start playing with the source code. To me the most important part in a system is its deployment story. You have to know the various components in the system: what are the jobs they perform, how they interact, etc. Newsberry is pretty complex: it has several web sites, Windows services, mail servers, and a couple of custom tools. Getting to know those components is essential. Exploring their interactions will quickly expose holes in your understanding of the business domain, and you have the duty and the opportunity to fill them. There are two tools that helped me here:

Mind mapping

I have been using mind maps for years, and I am used to drawing them on paper or with a computer program. They are a great tool to organize your thoughts when learning something. My guess is that they boost your understanding by stimulating the creative parts of your brain. They trigger that by making you produce something instead of passively reading up on a subject. Anyway my theory really doesn’t matter – all I care is that mind maps work for me, and I recommend everyone try them too. So, how do I use mind maps when learning Newsberry? Using my favorite mapping program, Freemind, I started creating maps for the email marketing process, Newsberry system deployment, data flow throughout the system such as email generation and transformation. As an example, here is a screenshot of my deployment map:

You can use maps to store all kinds of information: from the high-level overviews down to the nitty-gritty. Again, the most important thing is not the map itself, but the process of creating the map. Try it – It works!

Virtualization

Now that you are pretty comfortable with the deployment architecture, you want to get your hands dirty. Of course, you can’t do that on the production server. You can’t do it on the staging server too – I didn’t want to do something stupid and kill the staging environment or destroy the CI process just because I don’t know the system yet. You can’t really test on your machine either, because… well because you have to be able to deploy your program everywhere and it is too easy to hardcode settings that work on your machine only. That is why, apart from running the system on my development machine, I decided to recreate the production environment in a virtual machine. My virtualization solution of choice is VirtualBox. It is fast, easy to use and offers tons of advanced features too. I quickly got a Windows 2003 server OS installed on a VM, and started playing. I configured my virtual network cards with the same IP addresses that the server had. I edited my hosts file to redirect domains to the local machine. I installed SQL Server and imported a test DB full of usable data. I got an SMTP server running that was able to deliver mail to several fake domains. I could then fetch that mail using POP3 and inspect it for errors or write automated tests. I even went ahead and duplicated the CruiseControl.NET setup so that I could test CI builds as if they were running on the real build machine.This looks like a lot of work, and it really is. I am glad I did it because it helped me learn how the system operates. I went through the horrors of administering an SMTP server, configuring IIS, fighting SQL Server, and, at the end, I came out stronger. I really recommend using virtualization to safely play with a new, unknown system. One VirtualBox tip: use the snapshots feature as often as possible. If you manage to destroy your system, just revert to the previous snapshot, and you’ll be running again in seconds.

Use code exploration tools

One word here: Resharper. Many think it is a refactoring tool, but it is really much more. Load all your code in a Visual Studio solution, and start poking around. I use Resharper’s “Find Usages” feature to find who calls a given method, what code uses a property or sets a new value. The tool is indispensable.

Another tool that deserves mention here is Reflector. It does not work with source, but decompiles assemblies instead. It has great code analysis features, and you can quickly understand how a method is used, who instantiates a given class, etc. Not needing the source code makes Reflector the perfect detective tool that allows you to unobtrusively poke even at your production code.

The ultimate method

Well really there is no such thing as the one and only codebase learning technique. You can’t use purely a technical approach or a purely theoretical one. Creating software is a complex collaborative process, and you can’t understand how a large system works using only of the two approaches.

I am sure there are a lot of useful techniques one can use to learn a new codebase quickly that I have not covered here. What are your tools and methods of choice? Share them in the comments!

The features we miss

We have been using scrum for a while and we could not adapt it to Basecamp. It’s missing a few features that are basics for agile:

• Backlog – a place where we could store tasks that we are not working on currently
• Task states – we could not change the state of the tasks (started, accepted, delivered, etc)
• Prioritizing tasks – todo lists in Basecamp are not well suited for grouping tasks. If we add tasks to todo lists, we cannot prioritize them well, since some tasks in one todo list should have a higher priority than other tasks in another todo list

Not to mention that we don’t have burn-down charts, velocity, etc. These things are not top priority though.

Seeking a different issue tracker

The problem we had while searching for a different solution was that many systems out there try to improve things based on Basecamp. Then Petyo suggested that we try out Pivotal Tracker.

I had a look at and it seems that it is the closest solution to the agile principles and scrum I’ve seen so far.

Pivotal Tracker is much more complex than Sifter which I reviewed a week earlier (Review coming soon – Sifter’s biggest advantage is the simplicity, great dashboard, task type states and filters, though some features are missing like proper backlog). On the other hand, Pivotal Tracker has a very interesting approach to task management, and all the functionality needed to track tasks properly.

The main screen of Pivotal Tracker contains columns which can be changed based on what you need and want to see. Tasks are separated in iceboxes, backlog and current task list.

The good

Iceboxes

Iceboxes are place where you create tasks. In the icebox you have all the tasks that you plan to work on someday, and they are not ordered.

Backlog

Backlog contains all the tasks you plan to work on, but you did not start to work on yet. Backlog is ordered, and tasks can shift from it to current milestone dynamically.

Current tasks

The current tasks list is the place where you can find tasks which you are working on in the current iteration.

Velocity

One more interesting feature of Pivotal Tracker is velocity. It is the average number of points accepted per iteration, based on recently completed iterations. It measures progress, and allows Tracker to predict when milestones will be completed based on past performance.

Everything is in task lists

Pivotal Tracker is task oriented. Everything you need to see in Pivotal Tracker is related to tasks (releases, comments, backlog, current tasks) and can be seen in task lists named current, backlog, icebox etc. Even release date looks like a task. It is created in the same way as the tasks only it’s marked with a different type.

Not all tasks can have points in Pivotal Tracker, only features do, which is logical, since the estimates are related to the features, and when they will be finished, so bugs, chores do not have points.

Burn-down chart

Another great feature in Pivotal Tracker is the burn down chart which is generated automatically based on the work completed. It can be reviewed in the same way you would review tasks.

Search, group tasks

Pivotal Tracker also contains a way to have proper todo lists, by adding tags. Every task can have a tag which gives you an easy way of searching and grouping tasks.

Report generating

It contains simple report generating, although as far as we’ve seen, it’s disadvantage is that you can’t export them.

The bad

Well, on the bad side, Pivotal Tracker is strictly a tracker, so there is no way to use it for messaging, project specifications etc.. What we love about  Basecamp is that you have separate sections like messaging and writeboards, which can be used for communication and project specifications. But when you take a closer look, that is not a purpose of Pivotal Tracker, it’s not meant to be used as an all-in-one solution.

One more thing to note is that Pivotal Tracker is complex at first glance at it and requires time to get used to it. It seems it’s developed in a way so you have to follow the rules, which is a good thing.

The feature we would miss most from Basecamp is the communication between more than one person on a task. Pivotal tracker has it but it works in a way that only the owner, requester and all members who have posted comments are emailed when a new comment is posted. There is no way to choose users to notify. Also reply from email would be nice. This is the feature we seek most, since this could make it hard for us to communicate with the client directly about a task.

I must say that Pivotal Tracker is interesting, and a nice approach to management. We are still reviewing it, and who knows, maybe this will be our next task management system.

Review is still in progress

We’re trying Pivotal Tracker on a small project for now to see if it fits. In the mean time, we’re also trying some others tools to get a feel of what is out there. It’s almost impossible to tell if a tool is right unless you try it on a real project. Stay tuned for more reviews in the coming weeks.

We are in search of the good solution which could improve our process, feel free to recommend any other task management systems.

Since I heard of the concept (and the company) I was extremely excited. The only problem was that I already had a lease, so other than the occasional use from my wife, we never really experienced it. Once we moved to center city and gave up our lease, we decided to give PhillyCarShare a try. We’ve never looked back.

For those of you who don’t have car sharing in your city, here’s how it works:

1. Go on the website and reserve a car close to you with the time frame you need
2. Head over to the car and unlock it with your rfid card
3. Drive it around and return it when it is due back

I think everyone can agree that the concept rocks, but let’s get to the point of the post. Not only was this a great concept, but PhillyCarShare (the company) was extremely impressive when it came to customer service, availability of cars, and attention to detail. I really can’t think of many companies that have impressed me so much, which quickly made me a loyal customer.

Then Zipcar came along

When Zipcar showed up, PhillyCarShare had its first competition. Being the loyal customer that I was, I immediately viewed Zipcar as an outsider; a national company that had no chance in Philadelphia. They seemed less community driven than PhillyCarShare and frankly, I didn’t even give them a chance.

Philly Car Share gets new management

Within the past few months, we’ve noticed some drastic changes from PhillyCarShare. From what I hear the original owners left (or something) and new management stepped in. Since that time, I have never seen such a sudden change in quality and customer service. First, they decided to remove free plans (hourly rates only) by sending an email on Friday evening, which took effect the following Tuesday (shady). Then, we started to notice less cars, cheaper cars, and even cars that are beat up and not fixed for a long time. It was extremely disappointing.

Considering other options

With all the problems, I went from a fanatically loyal customer to a “shopping for options” consumer. It amazed me how fast a company can slip, without much room or options to get those customers back. As I started looking at their competitor for the first time, Zipcar seemed like a great option. They now have excellent cars, a great reservation system, and many automated ways to interact with your reservation.

We signed up for Zipcar but decided to keep PhillyCarShare while we evaluate. As it stands now, Zipcar seems to be the better option.

You can’t afford to slip

What I learned from this is that no matter how good you are today, you still have to work your ass off to keep that brand loyalty. Small things can add up over time that leave a negative impression, but are hard to see from the inside. It’s important to listen to your customers and learn their overall experience, and most of all, focus on doing every small thing as best as possible.

What does Wildbit mean?

This was actually the first question I asked Chris when we started working on the new identity. It’s a shame that I lost the chatlog from that day, but the answer was pretty close to this:

I don’t know :)

Yes, I hear a lot of people crying right now, but I’m sure there at lot of company names which don’t have a real meaning either. We even thought about a new name, but we ran into a wall quickly and we really like the word itself.

To define the emotional direction of the company, we asked our team members about what Wildbit means to them. Some quotes from that discussion:

I want Wildbit to be screaming-cool, hot and a bit geeky.

We should be fun, easy to communicate, but at the same time really serious about our business.

No infantile stuff like frogs and octopussies, right?

With some of this information in mind, the process started. We pretty much agreed that the colors should have a more earthy tone since the word “Wildbit” pretty much nails down a connection between the wild, or wilderness. So, after approximately 107442709 logo variations (that’s how it felt to me), I came up with the final result. Problem was, we could not decide on a color scheme because they all looked so great. We then decided to use them all because they work so well together.

The idea is that the stripes represent blades of grass and each stripe is perfectly aligned to the typeface. Hell yeah I love perfectionism.

I hope this removes the question mark above your heads. Watch our for the next post where I explain how our website design evolved.

Selenium IDE

There are many different ways to automate tasks and Selenium IDE is just one of them. Selenium IDE has been around now for some time. It’s a small add-on for firefox which allows you to run and record tests for your web application. The reason I use Selenium IDE is because you can use it without almost any interference in your process of manual testing. Selenium IDE is lightweight and very easy to use and it’s constantly improved.

When I do manual testing, I cover the areas that I wante to test, and as I go, I turn on Selenium IDE and record tasks. I try to record scenarios that I will use later in other tests. These small Selenium IDE test cases are recorded so I will not waste time on common tasks, which would be repeated numerous times. Automating small tasks can speed up testing tremendously. This way you would spend time on reviewing parts of the application which are important, rather than spending time on common, everyday tasks.

Here is a simple scenario of my usual review of new functionality on our Newsberry project: Recently we released monthly plans, which required a lot of sign up testing. During testing of the sign up process, I recorded some scripts in Selenium IDE (simple html scripts), for signing up users, adding senders and activating them. This is the usual process you would use when you are creating a new account. Now, these scripts were used a lot during the review of monthly plans, but they were used a lot later during other reviews too. By recording small tasks, you can simply re-run them later, and not worry about doing the same process manually over and over again.

The beauty of the process is that it takes only couple of minutes to do this. These are not complicated test cases, test suites, no, these are very simple scripts, which everyone can run. You can simply send them over to developers, designers, and they can use it the same way you did, as long as they have firefox installed on their computers.

I’m not kidding about eighty languages here, you can see a full list at the Pygments web-site. It includes stuff like a full family of C languages, Ruby, Python, Java, Scala, Haskell, Scheme, JavaScript, Io, Matlab, Erlang, HTML and many many more.

It also has lovely bits like “.html.erb”, that our fellow Rails-developers use every day in their projects, and will have both HTML and ERB highlighted properly. And much anticipated support for CSS for all the designers out there.

We also updated the diff viewer so that it will now display line numbers correctly for file comparisons.

So, jump into your Beanstalk account (or grab one for free) and check how nifty your code looks with the new Code Viewer. But please keep in mind that as with any other new cool feature there may be some raw edges, such as problems with various character encodings out there. Don’t hesitate to contact us and provide feedback that will help us make the Code Viewer even better.

Overview

When you log in to your account you will notice a new search field at the top right corner of the interface. It will always be there across all pages. If you are on the Dashboard, it will search for commits across all repositories. If you are in repository Activity page or Browser, or any other repository-related page, it will search only in that repository. When you enter something in that field you will be redirected to a Search Results page with a list of all revisions found for your search term.

Right now Search is only capable of finding your commits, searching for files contents is coming later. You can use Search to find by commit message, name of file or directory changed in commit, author name and revision number. Search is using stemming so it will treat words like “fishing”, “fish” and “fisher” as the same word.

And the search is blazing fast, because it’s using an industry standard Sphinx search engine.

Search Scopes

Ok, let’s do some advanced searching now. On the Search Results page you noticed a few options to adjust your search: select boxes for repository and user. Use these to search for commits only in some repository or made by a specific user. If you leave the search field empty and select some user, you will get a list of all commits made by him (or her). Very useful to get information about a user’s activity.

Tricks

There are two special triggers that you can use in the search field to adjust your search results: +by and +rev. +by is the same as the users select box on the Search Results page but it allows you to find by users that are not registered in your Beanstalk accounts. You can use that trigger like that (without quotes): “updating CSS +by gilbert”. Or you can use it without search term to get the list of all commits made by user: “+by ilya”.

The +rev trigger is my favorite. Using it you can quickly jump to a specific revisions in your repository. Say you are on Activity page and you want to see a changeset for revision 400 of your repository. Just enter this in the search field and you will get what you want: “+rev 400″ (again, without quotes). If you do that on the Dashboard you will get a list of revision 400 changesets for all repositories in your account.

How do you like it?

This implementation of Search is just iteration number one so it’s probably not ideal and won’t suit all your needs, but as we move forward we will continue improving it according to your feedback. So we would like to hear from you! Let us know how search works for you and what interesting ideas do you have about making it even better.

In addition to this, our OpenID support was never really complete. We lacked the proper verification and over time, it became a little difficult to maintain. This month we discussed internally whether we should finally improve OpenID implementation in Beanstalk. As we discussed the effort, we decided that it may be better to just remove it. This creates less maintenance for us and does not remove too much value for our customers since a user/pass is still required for SVN.

We’re hoping we can revisit this in the future as we improve integration options, especially if SVN gets OAuth support.

At Engine Yard, we have a group of slices that manage SVN, Web Servers, and our backend processes for things like deployments, integration tools, and so on. All of these slices use a shared storage system called GFS. This allows us to scale to many slices while still having access to essential data.

The issue

When the outage occurred, one of our slices had a severe memory problem. We attempted to fix it with Engine Yard, but due to the nature of GFS we had to first make sure there we no data corruptions. This forced us to shut down the entire environment while we checked the disks. It’s basically a precaution to make sure we don’t reboot with data corruption.

Since we have so much data, the disk checks took a really long time, which resulted in a full environment outage. It was the best approach to make sure we safely brought the environment back, but obviously not ideal for those who needed access to their code. After the disk checks and reboot all was back to normal and the data was verified.

Future avoidance

When things like this happen, we always need to reflect and figure out ways to avoid them from happening again. In this situation, we need to make sure we have enough resources to handle severe spikes that might cause full environment problems. I am confident we added the necessary resources and can control it in the future.

More updates coming

We recently got back from Railsconf. I have to say that hanging out with Engine Yard was the most valuable part of being there. We had a great opportunity to sit down with the team and discuss better ways for us to scale Beanstalk. There are many options, so choosing the right path can be tricky. We feel like we came up with a good plan to not only scale the service, but also isolate our shards (nodes) to avoid system wide issues when things go wrong. As of now, many of you should have already noticed SVN speed improvements.

We have tons of cool features to work on, but our first priority is performance. It’s amazing how challenging and rewarding a successful web product can be at the same time. We’re excited about our growth and ready to take on the challenge to continue scaling the service. Thanks for being patient and helping us grow.

The Navigation

We have to admit that the navigation was not entirely easy to understand on the previous version. The biggest problem was that the hierarchy of the different navigation elements was torn appart. The new version does a much better job at grouping all the actions and presenting them with much better visual weight.

Have a look at the Subscribers page for example. I’ve marked the different navigation levels with numbers, with 1 being the the highest level and 4 being the lowest. As you can see, the new design even got rid of the fourth level, making it much easier for navigating through the application. At the same time, your eye does not need to jump from left to right/right to left when you move between the main areas (Dashboard, Lists and Templates).

Yes, we removed the left sidebar

Some customers might be a bit confused that we removed the left sidebar. But we did this in favor of consistency. For instance, a Camapaign was created within the Campaigns section of the application. Lists and Templates were created via the sidebar. Sounds wrong, doesn’t it?

Now, we basically had two options:

• Add a “Create Campaign” button to the sidebar
• Remove the sidebar and add “Create List” and “Create Template” buttons, just like the “Create Campaign” button

As you probably noticed, option two won. Removing the sidebar really helped to clean up the overall design, which results in less confusion for the viewer.

We really hope that these changes help to speed up your interaction with Newsberry and make the overall experience more fun.

So for me, RailsConf was all about people. Meeting Railsists and Rubyists as I am, sharing experiences, talking about our projects and just bragging about day-to-day shit. It was a very inspiring and interesting experience. RailsConf talks and tutorials were a good bonus to all this :)

I really enjoyed the talks by Obie Fernandez, DHH and Chris Wanstrath. Too bad I missed the talk about Smalltalk and Ruby by Robert Martin.

We also attended New Relic’s customers party. Free food and drinks, you can’t ignore something like that :) New Relic seriously rocks.

Probably the most useful thing we did at RailsConf is talk to the Engine Yard guys. We solved many many problems from our current Beanstalk setup. Hopefully this will make our scaling pain go away soon and make Beanstalk faster. We are working really hard on this one.

And of course, Vegas itself is a super-crazy place to visit! Probably the craziest city in the world. Super amazing shows, lot’s and lot’s of fun and beautiful stuff to see.

We also visited NYC and Philadelphia during the trip. It was also very interesting. I liked Philadelphia very much. Calm and beautiful place to live. I plan to move there for a couple of months to try living the US way.

So, RailsConf 09 was a blast, I really enjoyed it and will definitely attend next year. I hope we will make some small Beanstalk users meet-up also :)

Don’t forget to check out some photos that I took at the conference.

TeamSupport is a SaaS application that integrates a Help Desk with a Bug Tracking system. They feel that most software companies struggle with internal communication since they have different applications for their customer service group and their development/QA teams. Having one tool that both groups can use will facilitate internal communication and also lead to better customer satisfaction.

Of course, the Beanstalk integration is of most interest to the developers and designers using TeamSupport, but it is nice that the customer service group can also see that a particular ticket is addressed by a revision in Beanstalk.

When a new version is committed into Beanstalk, we send out a WebHook configured for TeamSupport. TeamSupport receives this and scans the description field for “version” and “tickets” tags. They then match up the tickets that the new version addresses and they add a note to the ticket with information about the commit, including the link back to their repository on Beanstalk so they can review the actual code.

Hope you like it. Be sure to give us feedback and let us know what you think.

So, what’s different?

We have been thinking about a way to keep the old functionality intact, but adding a new fresh look to it. Thanks to our designer Gilbert, Newsberry got a completely new face. If you have been using Newsberry before and you liked it, we’re sure you will love it now!

What have we done?

• Moved Subscriber Lists to their own section at the top to make them more easily accessible.
• We’ve moved the tabs for Account and Help to the top so they are always visible.
• Moved our Help System to an outside application called Tender. This will allow our customers to post questions, comments and feedback as well as post private issues for our customer service.

These are only small changes which you would notice when you first log into your Newsberry account, but it does not stop here. During the makeover, we have completely redesigned navigation, buttons, tabs, modalboxes and much more.

For example, take a look at the redesigned “Create a Campaign” page:

Want to see for yourself? Login to your account or signup now!
 
The adventures of Newsberry do not end here. We have a lot more cool stuff planned for 2009. Stay tuned and follow us on Twitter to get the latest updates.