<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" media="screen" href="/~d/styles/atom10full.xsl"?><?xml-stylesheet type="text/css" media="screen" href="http://feeds.feedburner.com/~d/styles/itemcontent.css"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en-US">
  <title>entp hoth blog - Everything ENTP</title>
  <id>tag:hoth.entp.com,2009:mephisto/</id>
  <generator uri="http://mephistoblog.com" version="0.8.0">Mephisto Drax</generator>
  
  <link href="http://hoth.entp.com/" rel="alternate" type="text/html" />
  <updated>2009-06-23T08:36:21Z</updated>
  <link rel="self" href="http://feeds.feedburner.com/activereload-articles" type="application/atom+xml" /><entry xml:base="http://hoth.entp.com/">
    <author>
      <name>rick</name>
    </author>
    <id>tag:hoth.entp.com,2009-06-23:14906</id>
    <published>2009-06-23T08:36:00Z</published>
    <updated>2009-06-23T08:36:21Z</updated>
    <category term="Tender Support" />
    <link href="http://hoth.entp.com/2009/6/23/new-in-tender-enhanced-search" rel="alternate" type="text/html" />
    <title>New in Tender: Enhanced Search</title>
<content type="html">
            &lt;p&gt;It’s true. Until now, Tender’s search capabilities have been in need of some work.  Over the weekend, we deployed the first major update to overhaul the search system.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://img.skitch.com/20090622-pcnb2k4pjkq2g8199941w2rcb6.png" alt="Advanced Search screen" /&gt;&lt;/p&gt;

&lt;p&gt;Design-wise, we didn’t like how the search was so comment-centric.  Searching for anything that matches anywhere in a comment body will now return the whole discussion just once.  You can also search for names and emails to pull up discussions that the user has participated in.&lt;/p&gt;

&lt;p&gt;Also, we wanted the search to be unified on a single page.  You can now search Discussions, FAQs, and users (if you’re a support member) from a single page.  We’re working on a proper FAQ + Discussion search (heavily weighted to prefer FAQs) as well.&lt;/p&gt;

&lt;p&gt;On the backend, we’ve added filters for searching just the title or by discussion state.  We’ll also be adding filters for the published/updated dates, as well as configurable sorting.&lt;/p&gt;

&lt;p&gt;Please check out the updated search and let us know how it works out for you.  If you have any feedback, please let us know on &lt;a href="http://help.tenderapp.com" title="Tender Support"&gt;http://help.tenderapp.com&lt;/a&gt; or ping us on &lt;a href="http://twitter.com/tenderapp" title="tenderapp"&gt;twitter&lt;/a&gt;.&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>giles</name>
    </author>
    <id>tag:hoth.entp.com,2009-06-23:14908</id>
    <published>2009-06-23T07:52:00Z</published>
    <updated>2009-06-23T07:52:49Z</updated>
    <category term="Tumbles" />
    <category term="quote" />
    <link href="http://hoth.entp.com/2009/6/23/no-thirst-software-on-tender" rel="alternate" type="text/html" />
    <title>No Thirst Software On Tender</title>
<content type="html">
            &lt;blockquote&gt;&lt;p&gt;&lt;a href="http://kevinhoctor.blogspot.com/2009/06/growing-pains.html"&gt;It's only been a week of working with Tender and a few days of having it live for our customers to use, but &lt;strong&gt;I'm thrilled with it&lt;/strong&gt;. The discussion forum is much more structured than our old Google Groups forum—we can mark issues as open or resolved, assign priority queues to issues, and, best of all, support emails go to the forum so they can't get lost in a cluttered inbox. It's not as large and in charge as FogBugz, but that's part of its appeal. We're even more committed to offering timely and effective support as our customer base continues to grow, and Tender will help us stay on top of our game. My next blog post will talk more about this customer support tool and why &lt;strong&gt;it fits No Thirst Software so well&lt;/strong&gt;.&lt;/a&gt;&lt;/p&gt;&lt;/blockquote&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>rick</name>
    </author>
    <id>tag:hoth.entp.com,2009-05-23:14809</id>
    <published>2009-05-23T11:30:00Z</published>
    <updated>2009-05-23T18:31:06Z</updated>
    <category term="Tender Support" />
    <link href="http://hoth.entp.com/2009/5/23/job-queue-issues-it-s-not-lupus" rel="alternate" type="text/html" />
    <title>Job Queue Issues:  It's Not Lupus</title>
<content type="html">
            &lt;p&gt;Yow, what happened to the queue?  We had two separate issues on Thursday night and Friday night that have since been fixed.  &lt;/p&gt;

&lt;p&gt;The first one had to do with a strange error from parsing one of the many emails we receive.  A bug in our email parsing library raises a critical exception that kills the queue process.  I found the issue, deployed a workaround, and then &lt;a href="http://github.com/mikel/tmail/issues/#issue/1"&gt;submitted a patch to the library author&lt;/a&gt;.  Typically, our queue catches and logs exceptions, instead of dying.  However, the bug caused it to mistakenly raise a fatal SyntaxError, which is typically reserved for a typo in a script that prevents the application from even running.  &lt;/p&gt;

&lt;p&gt;The second problem has to do with a new job we deployed on Thursday.  One of the problems we saw with accepting anonymous comments on Tender, is that you lose all connection to those comments when you create a new profile.  The new job automatically links them to you when you create a profile on Tender.  The problem is, these jobs started taking a bit longer than anticipated to run, holding up more critical tasks.  I’ve since started using a priority system to ensure that critical tasks like spam checks or comment notifications are run before these jobs.  &lt;/p&gt;

&lt;p&gt;On a final note, it’s clear that the job queue is of utmost importance for Tender. We actually have a munin server setup for monitoring, but we’re still working with Engine Yard on getting the firewall port issues worked out.  This incident has moved that ticket to our top priority so we can get on top of these issues sooner.  Sorry for the inconvenience, everyone!&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>kyle</name>
    </author>
    <id>tag:hoth.entp.com,2009-05-15:14744</id>
    <published>2009-05-15T19:32:00Z</published>
    <updated>2009-05-15T19:33:02Z</updated>
    <category term="Tumbles" />
    <category term="image" />
    <link href="http://hoth.entp.com/2009/5/15/extreme-programming" rel="alternate" type="text/html" />
    <title>Extreme Programming</title>
<content type="html">
            &lt;p&gt;&lt;img src="http://share.kyleneath.com/captures/skitched-20090515-123210.jpg" alt="Extreme Programming" /&gt;&lt;/p&gt;

&lt;p&gt;Rick &amp;amp; Towski practice some Extreme Programming.&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>kyle</name>
    </author>
    <id>tag:hoth.entp.com,2009-05-13:14734</id>
    <published>2009-05-13T19:56:00Z</published>
    <updated>2009-05-13T19:58:49Z</updated>
    <category term="Tender Support" />
    <link href="http://hoth.entp.com/2009/5/13/new-in-tender-autoresponders" rel="alternate" type="text/html" />
    <title>New in Tender: Autoresponders</title>
<summary type="html">&lt;p&gt;In our continued dedication to rolling out new features for Tender, we’ve recently deployed &lt;a href="https://help.tenderapp.com/faqs/email-integration/how-do-autoresponders-work"&gt;Autoresponders&lt;/a&gt;.  Autoresponders allow you to automatically send an email to users after they submit an issue.&lt;/p&gt;

&lt;p&gt;But we didn’t just implement your run-of-the-mill “We’ll get back to you soon” kind of autoresponders.  We’ve given you power with some &lt;a href="https://help.tenderapp.com/faqs/email-integration/autoresponder-syntax"&gt;special variables&lt;/a&gt;.  These variables let you use things like the customers name, email, the title &amp;amp; URL of the discussion, and even suggested FAQs! &lt;/p&gt;</summary><content type="html">
            &lt;p&gt;In our continued dedication to rolling out new features for Tender, we’ve recently deployed &lt;a href="https://help.tenderapp.com/faqs/email-integration/how-do-autoresponders-work"&gt;Autoresponders&lt;/a&gt;.  Autoresponders allow you to automatically send an email to users after they submit an issue.&lt;/p&gt;

&lt;p&gt;But we didn’t just implement your run-of-the-mill “We’ll get back to you soon” kind of autoresponders.  We’ve given you power with some &lt;a href="https://help.tenderapp.com/faqs/email-integration/autoresponder-syntax"&gt;special variables&lt;/a&gt;.  These variables let you use things like the customers name, email, the title &amp;amp; URL of the discussion, and even suggested FAQs! &lt;/p&gt;
&lt;div class="figure"&gt;&lt;img src="http://s3.amazonaws.com/entp-tender-production/assets/fa002d18adf81a36cb487f654b9de286ffda67d5/MyCompany-Support_-Thanks-for-your-inquiry-%E2%80%94-Inbox_normal.jpg" /&gt;&lt;/div&gt;

&lt;p&gt;This means users who submit issues to you via email can get suggestions from your Knowledge Base emailed back to them.  I personally think this is pretty awesome – it helps to cut down on the number of support requests dramatically.&lt;/p&gt;

&lt;p&gt;As a bonus, you can set up different autoresponders for each category, so customers submitting billing issues get a different autoresponder from those suggesting new features. You can find the new functionality in &lt;code&gt;Admin -&amp;gt; Site Settings -&amp;gt; Email -&amp;gt; Autoresponders&lt;/code&gt;.  Hope you enjoy them as much as we are!&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>kyle</name>
    </author>
    <id>tag:hoth.entp.com,2009-05-04:14712</id>
    <published>2009-05-04T06:36:00Z</published>
    <updated>2009-05-04T06:38:20Z</updated>
    <category term="Tumbles" />
    <link href="http://hoth.entp.com/2009/5/4/headed-to-railsconf-or-cabooseconf" rel="alternate" type="text/html" />
    <title>Headed to RailsConf or CabooseConf?</title>
<content type="html">
            &lt;p&gt;Rick, Towski, Courtenay, Kyle, Justin and Giles will be hanging out mostly at CabooseConf in Vegas this week. Follow &lt;a href="http://twitter.com/entp"&gt;@entp&lt;/a&gt; to see where we’re hanging out and come say hi!&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>rick</name>
    </author>
    <id>tag:hoth.entp.com,2009-04-29:14701</id>
    <published>2009-04-29T19:03:00Z</published>
    <updated>2009-04-29T20:06:26Z</updated>
    <category term="Lighthouse" />
    <link href="http://hoth.entp.com/2009/4/29/what-s-new-in-lighthouse-bulk-edit" rel="alternate" type="text/html" />
    <title>What's New in Lighthouse: Bulk Edit!</title>
<content type="html">
            &lt;p&gt;Earlier this week we deployed a hugely requested feature: ticket bulk editing.  &lt;/p&gt;

&lt;div class="figure"&gt;&lt;img src="/assets/2009/4/29/Tickets_-_Lighthouse_-_entp.jpg" alt="Screenshot" /&gt;&lt;/div&gt;

&lt;p&gt;Using that form, you can now update the state, milestone, assigned user, tags, and project for a group of tickets.  The best part is that it’s applicable for just a few tickets selected with checkboxes, or the whole result of a query.  &lt;/p&gt;

&lt;p&gt;Try it out, and &lt;a href="http://help.lighthouseapp.com"&gt;let us know what you think&lt;/a&gt;.&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>rick</name>
    </author>
    <id>tag:hoth.entp.com,2009-04-28:14697</id>
    <published>2009-04-28T07:24:00Z</published>
    <updated>2009-04-28T23:49:29Z</updated>
    <category term="Tender Support" />
    <link href="http://hoth.entp.com/2009/4/28/tender-spam-woes" rel="alternate" type="text/html" />
    <title>Tender Spam Woes</title>
<content type="html">
            &lt;p&gt;A patch that was deployed late Friday afternoon angered the spam gods, and most incoming discussions were marked as spam.  You may have noticed your Tender site being suspiciously quiet this weekend.  In light of this, we’ve done these 3 things:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Added a switch to the site settings to disable spam.  This might be a better option if you’re not receiving much spam as it is.&lt;/li&gt;
&lt;li&gt;Refined the spam rules so that the third party spam service is used as little as possible.  Your site’s comments skip the spam checker if they are emailed in, or if the user has posted more than 3 valid comments.  Three comments is the point that a user is ‘trusted’ and no longer subjected to spam or captcha checks.&lt;/li&gt;
&lt;li&gt;Added a site-wide spam log in our admin tools so we can monitor the incoming spam.  This will help us track and fix future problems much faster.  We’re currently going through the pages of spam and clearing or restoring everything.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;What was the original bug?  When we receive comments through email, we lose out on 3 key attributes that spam checkers use: user IP, user agent, and HTTP referrer.  These are all values sent by the user’s browser, but not through email.  As a result, we started using default values for everything.  Reports of invalid spam were on the rise, so I took out the defaults thinking that spammy emails might be tainting those default values.  Since all emails were using the same fake user IP, 5 bad emails could suggest to the spam checker that everything from that IP is bad.  However, removing those 3 value completely must’ve put a big target over every email.  This is why we’ve stopped checking spam for emails.&lt;/p&gt;

&lt;p&gt;We apologize for the inconvenience.  We take spam very seriously.  The open nature of the service makes it a challenge to keep all of the laptop auction emails off your site, and all of the new support questions out of your spam queue.  I’m hoping that the new tweaks tonight should help considerably.  If not, we’ll be watching everyone’s spam and fine tuning as necessary.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Update:&lt;/em&gt;  There’s a bug related to sites with disabled spam checking that now send all anonymous issues to the Trash queue.  We’re deploying a quick hotfix, and then we’re going to restore all the issues for you.&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>kyle</name>
    </author>
    <id>tag:hoth.entp.com,2009-04-13:14650</id>
    <published>2009-04-13T22:13:00Z</published>
    <updated>2009-04-13T22:17:05Z</updated>
    <category term="Lighthouse" />
    <link href="http://hoth.entp.com/2009/4/13/lighthouse-for-iphone" rel="alternate" type="text/html" />
    <title>Lighthouse for iPhone </title>
<summary type="html">&lt;p&gt;A weird thing happened recently. Several people got the urge to build iPhone apps for Lighthouse. Whatever it was that was in the air, we caught it too. And since we’re the people who actually make Lighthouse, we were able to do some back-end tweaks to make it turn out just right (in our ever-so-humble opinions, of course).&lt;/p&gt;

&lt;p&gt;Presenting &lt;a href="http://iphone.lighthouseapp.com"&gt;Lighthouse for iPhone&lt;/a&gt;.&lt;/p&gt;</summary><content type="html">
            &lt;p&gt;A weird thing happened recently. Several people got the urge to build iPhone apps for Lighthouse. Whatever it was that was in the air, we caught it too. And since we’re the people who actually make Lighthouse, we were able to do some back-end tweaks to make it turn out just right (in our ever-so-humble opinions, of course).&lt;/p&gt;

&lt;p&gt;Presenting &lt;a href="http://iphone.lighthouseapp.com"&gt;Lighthouse for iPhone&lt;/a&gt;.&lt;/p&gt;
&lt;div class="figure"&gt;
&lt;img src="http://share.kyleneath.com/captures/lh_iphone-20090413-144640.jpg" alt="Lighthouse for iPhone Screenshots" /&gt;
&lt;/div&gt;

&lt;p&gt;Check it out! &lt;a href="http://iphone.lighthouseapp.com"&gt;http://iphone.lighthouseapp.com&lt;/a&gt;&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>kyle</name>
    </author>
    <id>tag:hoth.entp.com,2009-04-10:14657</id>
    <published>2009-04-10T03:33:00Z</published>
    <updated>2009-04-10T03:36:45Z</updated>
    <category term="Tender Support" />
    <link href="http://hoth.entp.com/2009/4/10/what-s-new-in-tender-queues" rel="alternate" type="text/html" />
    <title>What's new in Tender: Queues </title>
<summary type="html">&lt;p&gt;One of the first big new features to hit Tender are Queues.  We actually launched this feature a few weeks ago, but forgot to tell everyone about it! (silly us).  Queues are a way to organize issues on the admin-side.  They’re generic notification-enabled buckets that hold issues. It’s much easier to show than to tell, so we have some screenshots to walk you through it all.&lt;/p&gt;</summary><content type="html">
            &lt;p&gt;One of the first big new features to hit Tender are Queues.  We actually launched this feature a few weeks ago, but forgot to tell everyone about it! (silly us).  Queues are a way to organize issues on the admin-side.  They’re generic notification-enabled buckets that hold issues. It’s much easier to show than to tell, so we have some screenshots to walk you through it all.&lt;/p&gt;
&lt;div class="figure"&gt;&lt;img src="http://hoth.entp.com/assets/2009/4/10/queues_admin.jpg" alt="Screenshot" /&gt;&lt;/div&gt;

&lt;ol&gt;
&lt;li&gt;Next to each queue (and next to standard state-based buckets) is a pin icon.  Clicking that icon sticks a queue with you, as seen in…&lt;/li&gt;
&lt;li&gt;After you stick a queue, you’ll get this yellow tab that follows you around everywhere on the site.  It’s a great way to work through a queue without going back and forth to the admin.&lt;/li&gt;
&lt;li&gt;You can name your queues whatever you like.  They can be user-specific (the same as assigning an issue to someone), action-specific, or general buckets.&lt;/li&gt;
&lt;li&gt;After you add an issue to a queue, it’ll be tagged in the listings. You can see at a glance which issues don’t have any queues.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;Adding issues to queues&lt;/h2&gt;

&lt;p&gt;Adding issues to a queue is simple.  From the admin pages you can drag issues to the queues in the sidebar to the left.  Or, if you’re on a discussion page, you can add them there as the screenshot below shows.&lt;/p&gt;

&lt;div class="figure"&gt;&lt;img src="http://hoth.entp.com/assets/2009/4/10/discussion_queues.jpg" alt="Screenshot" /&gt;&lt;/div&gt;

&lt;ol&gt;
&lt;li&gt;Clicking the “+” next to each queue adds the discussion to the queue, clicking the “-” removes it from the queue, and clicking the pin sticks the queue with you.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Notice the queue we stuck previously carried through to the front-end.  Clicking the yellow tab shows the list of issues in that bucket right on the screen.&lt;/p&gt;

&lt;h2&gt;Filtering Queues&lt;/h2&gt;

&lt;p&gt;When we were playing with queues internally, we found out that sometimes we like queues to show open issues but not resolved ones, while other times we just didn’t care what the state was. Enter filtering.&lt;/p&gt;

&lt;div class="figure"&gt;&lt;img src="http://hoth.entp.com/assets/2009/4/10/queue_queue.jpg" alt="Screenshot" /&gt;&lt;/div&gt;

&lt;p&gt;When you click on a queue in the sidebar, you get the screen shown above.  The light blue bar above the issue lets you choose which issues show up in the queue.  We still save any issues that are filtered out, but only the ones that match your preferences will show in the interface.&lt;/p&gt;

&lt;h2&gt;Email Notification&lt;/h2&gt;

&lt;p&gt;The last little bit of magic for queues is how email notification works.  When you’re on the edit page of a queue (after clicking manage custom queues, then edit), you get to choose who is notified when an issue is thrown into a queue.  Alternatively, you can set which queues you are notified on in your ‘My Profile’ page.&lt;/p&gt;

&lt;h2&gt;What do you think?&lt;/h2&gt;

&lt;p&gt;We hope you like queues.  They’ve been working out great for us – working as a filtering mechanism from all the issues we get, so that each person on our team can see the issues they can help with easily.&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>will</name>
    </author>
    <id>tag:hoth.entp.com,2009-04-02:14633</id>
    <published>2009-04-02T06:00:00Z</published>
    <updated>2009-04-02T14:08:10Z</updated>
    <category term="Tender Support" />
    <link href="http://hoth.entp.com/2009/4/2/a-little-tender-logic" rel="alternate" type="text/html" />
    <title>A little Tender logic</title>
<summary type="html">&lt;p&gt;Spam is a never ending battle for just about any web based system that allows anonymous user discussions.  The nature of &lt;a href="http://tenderapp.com"&gt;Tender&lt;/a&gt; is to let users easily post discussions, which means that we too are susceptible to spam.&lt;/p&gt;

&lt;p&gt;For some Tender accounts, spam has become a problem.  We've managed to filter a great deal of spam coming into the system, but a handful of spam has been making it through those filters.&lt;/p&gt;</summary><content type="html">
            &lt;p&gt;Spam is a never ending battle for just about any web based system that allows anonymous user discussions.  The nature of &lt;a href="http://tenderapp.com"&gt;Tender&lt;/a&gt; is to let users easily post discussions, which means that we too are susceptible to spam.&lt;/p&gt;

&lt;p&gt;For some Tender accounts, spam has become a problem.  We've managed to filter a great deal of spam coming into the system, but a handful of spam has been making it through those filters.&lt;/p&gt;
&lt;p&gt;Spam is a never ending battle for just about any web based system that allows anonymous user discussions.  The nature of &lt;a href="http://tenderapp.com"&gt;Tender&lt;/a&gt; is to let users easily post discussions, which means that we too are susceptible to spam.&lt;/p&gt;

&lt;p&gt;For some Tender accounts, spam has become a problem.  We've managed to filter a great deal of spam coming into the system, but a handful of spam has been making it through those filters.&lt;/p&gt;

&lt;p&gt;The painful truth wasn't so much the fact that spam was making it through, but that when a spam comment was added to an open discussion, it was emailed out to all of those users.  Not cool. Not cool at all.&lt;/p&gt;

&lt;p&gt;After some debate on how to best handle the problem, we've just deployed a logic spam buster to curb off the remaining spam. This feature is purely experimental for the time being.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://img.skitch.com/20090402-mmy9eyhqdmaf6ufq2rqaptujq6.png" alt="img" /&gt;&lt;/p&gt;

&lt;p&gt;Users who are not logged in to Tender will see an additional line in the discussion forms.  This line presents the user with a general logic question, such as "What is five times five?".&lt;/p&gt;

&lt;p&gt;While it's not a complex feature to add, it's still fairly unpolished.  The form will be cleaned up in the coming days to make it more aesthetically pleasing while we continue to refine it to help stop the remaining spam that is slipping into Tender.&lt;/p&gt;

&lt;p&gt;Please let us know if you come across any issues.&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>will</name>
    </author>
    <id>tag:hoth.entp.com,2009-03-30:14583</id>
    <published>2009-03-30T23:54:00Z</published>
    <updated>2009-04-02T14:08:32Z</updated>
    <category term="Lighthouse" />
    <category term="Tender Support" />
    <link href="http://hoth.entp.com/2009/3/30/attack-of-the-killer-monday-from-outer-space" rel="alternate" type="text/html" />
    <title>Attack of the KILLER Monday from Outer Space</title>
<summary type="html">&lt;p&gt;Let's start off by saying that I woke up this morning with a case of the Mondays.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://img.skitch.com/20090331-mmgsmdjji4bihdmf3ef8pc37ea.png" alt="Monday" /&gt;&lt;/p&gt;

&lt;p&gt;Our servers decided to hop on board that boat as well. Go figure.&lt;/p&gt;</summary><content type="html">
            &lt;p&gt;Let's start off by saying that I woke up this morning with a case of the Mondays.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://img.skitch.com/20090331-mmgsmdjji4bihdmf3ef8pc37ea.png" alt="Monday" /&gt;&lt;/p&gt;

&lt;p&gt;Our servers decided to hop on board that boat as well. Go figure.&lt;/p&gt;
&lt;h2&gt;A rundown of events which occurred on Monday, Mar. 30, 2009:&lt;/h2&gt;

&lt;p&gt;At 9:15 a.m. PDT, both our &lt;a href="http://lighthouseapp.com"&gt;Lighthouse&lt;/a&gt; and &lt;a href="http://tenderapp.com"&gt;Tender&lt;/a&gt; services went down.  We immediately contacted our host, &lt;a href="http://engineyard.com"&gt;Engine Yard&lt;/a&gt;, who was aware of the problem. &lt;/p&gt;

&lt;p&gt;A number of other services were also affected, including &lt;a href="http://github.com"&gt;GitHub&lt;/a&gt;, &lt;a href="http://www.pivotaltracker.com"&gt;Pivotal Tracker&lt;/a&gt; and Engine Yard's personal website.&lt;/p&gt;

&lt;p&gt;Engine Yard support contacted their West Coast data center, &lt;a href="http://www.heraklesdata.com"&gt;Herakles&lt;/a&gt;, which was reporting upstream issues with their connectivity.&lt;/p&gt;

&lt;p&gt;At 11:56 a.m. the issue was elevated to Cisco engineers who came in to assist in the matter.&lt;/p&gt;

&lt;p&gt;At 12:22 p.m. I noticed 42 new gray hairs and was filing an insurance claim for premature joint disease due to clicking refresh in my browser 36,000 times while restlessly waiting for updates from the data center.&lt;/p&gt;

&lt;p&gt;At 2:49 p.m. service had been partially restored.  Both Lighthouse and Tender were up and running but continued to experience intermittent connectivity issues while engineers at the data center were finalizing the repairs.&lt;/p&gt;

&lt;p&gt;At 3:45 p.m. Engine Yard gave the final green light on the stability of service.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The initial report states that the outage was due to a failed hardware card in a pair of redundant access switches.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;Continued...&lt;/h2&gt;

&lt;p&gt;While service is stable at the moment, there will be a scheduled maintenance window to bring the systems redundancy back online and ensure the integrity of the repairs. We'll post an announcement if there is going to be any scheduled downtime.&lt;/p&gt;

&lt;p&gt;We sincerely apologize about the inconvenience.  When our services are down it affects not only your work, but our own client work as well.&lt;/p&gt;

&lt;p&gt;For future reference, you can follow the Lighthouse Status blog, &lt;a href="http://lhstatus.com"&gt;http://lhstatus.com&lt;/a&gt;, as well as the &lt;a href="http://twitter.com/lighthouseapp"&gt;@lighthouseapp&lt;/a&gt; and &lt;a href="http://twitter.com/tenderapp"&gt;@tenderapp&lt;/a&gt; twitter accounts for status updates and announcements.&lt;/p&gt;

&lt;p&gt;We would like to thank Engine Yard for the prompt response times and frequent status updates while the situation was being handled. They have been an amazing supporter of the community and provided great service for us since we launched Lighthouse two years ago. If today was stressful on us, we can only imagine how it was on their end.&lt;/p&gt;

&lt;p&gt;We'll be present for the weekly Engine Yard conference call in hopes to discuss measures that will be taken to prevent such an outage from occurring in the future.  Monday wasn't fun for anyone.&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>kyle</name>
    </author>
    <id>tag:hoth.entp.com,2009-03-27:14577</id>
    <published>2009-03-27T21:29:00Z</published>
    <updated>2009-03-27T21:37:10Z</updated>
    <category term="Tender Support" />
    <category term="Tumbles" />
    <category term="image" />
    <link href="http://hoth.entp.com/2009/3/27/one-more-reason-to-look-forward-to-railsconf" rel="alternate" type="text/html" />
    <title>One more reason to look forward to RailsConf...</title>
<content type="html">
            &lt;p&gt;&lt;img src="http://hoth.entp.com/assets/2009/3/27/tendershirt.jpg" alt="Tender Shirt" /&gt;&lt;/p&gt;

&lt;p&gt;Coming to RailsConf?  You might be able to snag yourself some of our freshly delivered &lt;a href="http://www.flickr.com/photos/courtenay/3390859198/"&gt;Tender T-Shirts&lt;/a&gt;&lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>justin</name>
    </author>
    <id>tag:hoth.entp.com,2009-03-23:14559</id>
    <published>2009-03-23T19:50:00Z</published>
    <updated>2009-07-02T23:22:24Z</updated>
    <category term="Lighthouse" />
    <category term="Tumbles" />
    <category term="image" />
    <link href="http://hoth.entp.com/2009/3/23/lighthouse-fluid-icon" rel="alternate" type="text/html" />
    <title>Lighthouse Fluid Icon</title>
<content type="html">
            &lt;p&gt;&lt;img src="http://files.getdropbox.com/u/358452/lighthouse_fluid_icon.png" height="300" width="300" /&gt;&lt;/p&gt;

&lt;p&gt;Recently, Kyle posted a beautiful new &lt;a href="http://hoth.entp.com/2009/3/18/tender-fluid-icon"&gt;Fluid Icon for Tender&lt;/a&gt;.  Lighthouse users need not despair! Get your &lt;a href="http://files.getdropbox.com/u/358452/lighthouse_fluid_icon.png"&gt;Lighthouse Fluid Icon here&lt;/a&gt;. &lt;/p&gt;
          </content>  </entry>
  <entry xml:base="http://hoth.entp.com/">
    <author>
      <name>kyle</name>
    </author>
    <id>tag:hoth.entp.com,2009-03-18:14521</id>
    <published>2009-03-18T20:50:00Z</published>
    <updated>2009-06-23T04:55:04Z</updated>
    <category term="Tender Support" />
    <category term="Tumbles" />
    <link href="http://hoth.entp.com/2009/3/18/tender-fluid-icon" rel="alternate" type="text/html" />
    <title>Tender Fluid Icon</title>
<content type="html">
            &lt;p&gt;&lt;img src="http://files.getdropbox.com/u/358452/tender_fluid_icon.png" height="300" width="300" /&gt;&lt;/p&gt;

&lt;p&gt;Use &lt;a href="http://tenderapp.com"&gt;Tender&lt;/a&gt; with &lt;a href="http://fluidapp.com"&gt;Fluid&lt;/a&gt;?  We’ve just whipped up a quick &lt;a href="http://hoth.entp.com/assets/2009/3/18/tender_fluid.png"&gt;Fluid Icon&lt;/a&gt; for you!&lt;/p&gt;
          </content>  </entry>
</feed>
