Astrails - Home

astrails-safe-0.2.2 - sftp, timing + another rotation bugfix

2009-07-07T16:01:00Z

There is a new 0.2.2 version of astrails-safe on github.

If you don’t know it yet: astrails-safe is an easy to use backup solution for Unix like operating systems. It supports filesystem, mysql, postgresql, and subversion backups to local filesytem, Amazon S3 and remote SFTP.

The biggest change in this version is the support for SFTP as a storage option (contributed by Adam). It can be used instead of or in addition to S3. Note that is is still somewhat experimental and was not tested much in production. Please report any problems in the issues tracker

‘local’ storage option is still mandatory since S3 requires knowing the file size before the upload and keeping all in memory is not a good option :).

Does anyone really need an option of using SFTP w/o the local storage? If so please comment and I will look into possibility of implementing it.

Another change is that astrails-safe will now print timing statistics when running in a verbose mode (-v) (contributed by Neer)

Also another bug with rotation code was fixed (contributed by Layton). Rotation code would mistakenly match any file starting with the name of the current backup. e.g. cleanup for ‘foo’ would cleanup ‘foobar’ as well. I also added a test that should have caught this bug :)

Clicktale + Rails = Better Usability

2009-06-03T14:43:00Z

Clicktale is a service that allows you to record and later playback behavior of your users while they are using your site. And Rails is Rails, you know. And those two are getting along just fine, until the user logs in. After that clicktale service is cut out of the html pages this user gets and can’t record the session. But it just started to get interesting…

This plugin brings back the connection between Clicktale and Rails even for those closed pages. You’re going to get your better usability after all.

Clicktale

Clicktale is nice. I suggest you to head over to their site and check out the short(1:15) promotional video they have to get a feeling of what they do.

It records user sessions, allows form optimization by showing how much users drop out on any form field, does landing page optimization and other goodness.

Anyways, i find it very useful. It should be installed from the very first users new service is starting to get. It is an easy way to approximate field usability testing. It doesn’t replace the actual usability testing, but it is as close as you get without dragging yourself from the comfort of your Aeron chair.

Bottom line is that can help you make lives of your users a bit easier.

They are on pricey side of the internet with cheapest plan at $99/month, but they have a free plan. Very limited, but enough to see what it is about.

Rails Integration

I was really disappointed to find out that in our rails projects i could record user sessions only on outer parts of the site.

You see, clicktale works by combining two sides:

A small javascript snippet inserted into your rendered page to record user behavior
A copy of rendered HTML that clicktale stores on it’s servers to allow playback of recorded user actions on top of it.

Now, the first part in not a problem. The second part is. When a user hides behind a login, clicktale is no longer able to get the HTML. So it still knows where user clicks and where the mouse was moved, but this information is useless without the actual page your user sees at this moment.

To continue to enjoy the “big brother” feeling even after users log in, we need to supply this HTML to clicktale. Fortunately clicktailers allowed this by adding an option to their javascript to provide a URL to this HTML file that is different from the current page’s URL.

To close this circle, i use rails caching mechanism to store rendered page on the disk, and handle a path to the resulted file to clicktale.

Ok, nice, now how do i…

Install the plugin

./script/plugin install git://github.com/astrails/clicktale.git

Head to http://clicktale.com and signup for a free account. Or not free. Your choice.

Get a tracking code from clicktale. It should look something like this:

<!-- ClickTale Bottom part -->
<div id="ClickTaleDiv" style="display: none;"></div>
<script src="http://s.clicktale.net/WRb.js" type="text/javascript"></script>
<script type="text/javascript">
  if(typeof ClickTale=='function') ClickTale(<u><project_id>,<ratio>,<param></u>);
</script>
<!-- ClickTale end of Bottom part -->

Replace project_id, ratio and param in the autogenerated config/clicktale.yml with values from clicktale tracking code.

Add clicktale partials into layout inside the ‘body’ tag:

<body>
  <%= clicktale_top %>
  ...
  <%= yield %>
  ...
  <%= clicktale_bottom %>
</body>

Add a cron job(crontab -e, right?) that will take care of the old cached files

*/30 * * * * find /path/to/your/application/public/clicktale/ -type f -mmin +30 -exec rm {} \;

Note: The plugin works by leveraging rails caching mechanism, which is by default only enabled in production environment. To enable the plugin in the development environment do the following:

set enabled=true in config/clicktale.yml (development section)
set config.action_controller.perform_caching=true in config/environments/development.rb

Another Note: As of this writing, clicktale service ignores existance of Safari browser. I hope it will someday.

Options

Not much for now. But you can:

Add clicktale method on class level in your controller to change the clicktale project for specific controller
```
class UsersController < ApplicationController
  clicktale :project_id => ANOTHER_PROJECT_ID
  ...
end
```

Call the same method to tag this controller’s actions in clicktale records

class UsersController < ApplicationController
  clicktale :project_id => ANOTHER_PROJECT_ID, :tag => :specific_tag
  ...
end

You can call the same method with same parameters on the action level to control project id and tag for this specific action

Where’s everything

The code is on github.

Please submit issues also to github.

As always, suggestions are welcome, code contributions are even more welcome.

Now, go see what your users are doing.

Astrails-safe: 0.1.9 - BUGFIX release

2009-05-22T11:40:00Z

I just pushed new version 0.1.9 of astrails-safe to github.

The main difference is a fix to an embarrassing bug in the S3 backup rotation code. Thanks to Thuvarakan Tharmalingam for reporting.

Again, the reason it escaped was the fact that we don't yet have full test coverage. We are getting there though....

PostgreSQL and Subversion support for astrails-safe s3 backup

2009-05-21T01:47:00Z

It looks like our astrails-safe gem is quite popular :). People started to contribute new features:

Mark Mansour contributed PostgreSQL backup support
Richard Luther contributed Subversion repository dump support

I just released new version 0.1.8 on the github.

In addition to postgre/svn support this release also includes a long overdue test suite.

Note: I personally don’t use Subversion or PostgreSQL so I can’t actually test it. If you use it you can now easily backup it to local filesystem or Amazon S3 and if you find any problem please report here ;-)

If you don’t know what is astrails-safe, you can read the original announcement. In a nutshell its a simple backup script that supports file backups (with tar), MySQL backup (with mysqldump), PostgreSQL backup (with pg_dump) and Subversion backup (with “svnadmin dump”), all that with GPG encryption and backup rotation and Amazon S3 support.

See README for installation instructions.

The backup can be configured in minutes, so now you don’t have the excuse not to do it ;-).

Importing Mephisto comments into Disqus

2009-05-20T23:06:00Z

Mephisto commenting system is… how do i put it … outdated :)

And we wanted something more engaging for our blog. Looking around the web we found that Disqus was used all over the place, so we decided to integrate it into our blog instead of the native comments system.

The integration itself was fairly painless, they provide simple html/js snippets that we had to insert into Mephisto liquid templates for our blog.

The problem is that we wanted to completely remove the old commenting system but we didn’t want to loose existing comments on the blog.

So we needed a way to import existing comments to Disqus.

Google to the resque…

And we found this. Jim Mulholland wrote a small sinatra app that accepts blog comments RSS feed and imports the comments to Disqus. Which we didn’t have.

So we decided to write a direct export script instead, after all we don’t need RSS, we have mephisto db right here.

Below is the script. You just need to edit a couple of constants at the beginning and run it from your mepthisto directory like this:

./script/runner path/to/the/importer/script/disqus_import

If you use something other than Mephisto, you can still find it usefull, but it’s up to you to replace the code that retrieves comments from your blog engine.

Enjoy

Yes We Can. "require" over HTTP, That Is.

2009-05-12T15:24:00Z

Wouldn’t it be cool if you could just require “http://my-host/my-lib.rb” in ruby?

Now You Can! Using our “http_require” gem! :-)

test.rb:

require "http_require"
# this will download bar.rb and eval it
require "http://example.com/foo/bar.rb"

If a remote file (or one of its local dependencies) requires something that can’t be found locally, it will try to find it remotely from the same location as the parent.

Example

http://example.com/test/foo.rb:

# this will load "http://example.com/test/foo/bar.rb"
# if "foo/bar" is not available locally
require "foo/bar"

Stacktrace

http_require properly sets filename on eval so that the file’s URI appears in the stacktrace:

http://example.com/foo/foo.rb:

puts :foo
require 'bar'
def foo
  bar
end

http://example.com/foo/bar.rb

puts :bar
def bar
  raise
end

$ irb
>> require 'http_require'
=> true
>> require 'http://example.com/foo/bar.rb'
foo
bar
=> nil
>> foo
RuntimeError:
    from http://localhost:2000/bar.rb:3:in `bar'
    from http://localhost:2000/foo.rb:5:in `foo'
    from (irb):3
>>

Installation

sudo gem install astrails-http_require --source http://gems.github.com/

Sources

You can find sources on github

UPDATE:

There seems to be lots of similar comments that I’d like to answer here:

Q: This is a HUGE security hole
A: No it isn’t. running it directly from the web is no less secure then downloading it and then running locally. you can use same security protections, for example SSH tunnel, or SSL like you would for any other kind of ‘code delivery’ e.g. rsync, scp etc. If you control the source and the ‘tunnel’ then this is no less secure, and if you don’t, then no other method is secure unless you start encrypting/signing files.
Q: Why on earth would you do something crazy like this?
A: It is kind of cool :) Seriously though I do have a real usage in mind for this (more on that later), meanwhile consider rails app templates (rails -m app_template.rb) which do support running templates form the web, and no one seems to be crying out laud about a huge security hole :) Unfortunately though rails templates do not support (not out of the box) breaking down such remote templates into subfiles. you will need to do manual path mangeling (see app_lego for example). I guess http_require can be used to do it cleaner.

Simple backups can be simple!

2009-04-06T17:09:00Z

Everyone needs a backup, right? Unfortunately almost no one does though. Why?!

We needed something for ourselves and our customers. Something simple, free, configure-and-forget. Most of the time there is no need for something fancy, a simple tar + mysqldump can do the job for many small/medium sites.

We did some reasearch but every solution we found had one of the following problems:

too complicated to use/configure
not open source
does only filesystem or only mysql backup but not both
no Amazon S3 support
no backup rotation support

So we wrote our own. Basically we cleaned up and refactored some custom backup scripts that we had for ages on our own servers.

We had the following requirements in mind when we wrote it:

opensource :)
simple to install and configure
support for simple ‘tar’ backups of directories (with includes/excludes)
support for simple mysqldump of mysql databases
support for symmetric or public key encryption (see README for instructions)
support for local filesystem and Amazon S3 for storage
support for backup rotation. we don’t want backups filling all the diskspace or cost a fortune on S3

So lets dive right in:


# gem install astrails-safe --source http://gems.github.com/
Successfully installed astrails-safe-0.1.4
1 gem installed
Installing ri documentation for astrails-safe-0.1.4...
Installing RDoc documentation for astrails-safe-0.1.4...

# astrails-safe my-backup.conf
ERROR: Created default /root/my-backup.conf. Please edit and run again.

For configuration file format we use Ruby DSL (yeah, we probably got too excited writing it and went a little overboard with the implementation, a simple hash probably would suffice here :), but we actually like the result.

Check it out (you can see more configuration options in the generated template config file):



safe do
  local :path => "/backup/:kind/:id" 

  s3 do
    key "...................." 
    secret "........................................" 
    bucket "backup.astrails.com" 
    path "servers/alpha/:kind/:id" 
  end

  gpg do
    # symmetric encryption key
    # password "qwe" 

    # public GPG key (must be known to GPG, i.e. be on the keyring)
    key "backup@astrails.com" 
  end

  keep do
    local 2
    s3 30
  end

  mysqldump do
    options "-ceKq --single-transaction --create-options" 

    user "root" 
    password "............" 
    socket "/var/run/mysqld/mysqld.sock" 

    database :blog
    database :servershape
    database :astrails_com
    database :secret_project_com

  end

  tar do
    archive "git-repositories", :files => "/home/git/repositories" 
    archive "dot-configs",      :files => "/home/*/.[^.]*" 
    archive "etc",              :files => "/etc", :exclude => "/etc/puppet/other" 

    archive "blog-astrails-com" do
      files "/var/www/blog.astrails.com/" 
      exclude ["/var/www/blog.astrails.com/log", "/var/www/blog.astrails.com/tmp"]
    end

    archive "astrails-com" do
      files "/var/www/astrails.com/" 
      exclude ["/var/www/astrails.com/log", "/var/www/astrails.com/tmp"]
    end
  end
end

UPDATE: There is new updated version with PostgreSQL and svn support.

RRDtool and Ruby bindings on OSX Leopard

2009-04-01T14:46:00Z

RRDtool is the OpenSource industry standard, high performance data logging and graphing system for time series data. Use it to write your custom monitoring shell scripts or create whole applications using its Perl, Python, Ruby, TCL or PHP bindings.

Let’s run it with Ruby on Leopard.


sudo port install rrdtool

Default ports installation comes without ruby bindings. Get full installation from here:


cd ~/tmp
wget "http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.3.6.tar.gz"

It’s likely to download the same version that has been installed with ports :-)


open "rrdtool-1.3.6.tar.gz"

“open” not “tar” – we’re on macs after all :-)


cd rrdtool-1.3.6/bindings/ruby
ARCHFLAGS="-arch i386" ruby extconf.rb --with-rrd-dir=/opt/local
make
sudo make install

Enjoy:


irb
>> require "RRD" 
=> true
>> RRD
=> RRD

JRuby on Rails on Glassfish with Mysql

2009-03-19T12:29:00Z

Thanks a lot to Amit Hurvitz for providing a file of Virtual Disk Image (VDI) of VirtualBox, containing an up and running JRuby on Rails on Glassfish with Mysql. Image also contains some examples (actually solutions to the code camp exercises), all running on top of an OpenSolaris guest OS (can be run on many host systems).

Grab the image ~1.5GB archive.

Grab the exercises ~9.7MB archive.

flushing OS X DNS cache

2009-03-03T16:43:00Z

We recently moved our DNS to dnsmadeeasy.com from godaddy.com name servers.

After the transfer some internal CNAME records had a problem. So after fixing the problem and checking in the terminal that the changes propagated to the DNS server (host xxx.astrails.com) I tried to type the address in the browser, but it kept giving me the “can’t find host” error.

The solution was to flush OSX built in DNS cache (apparently the ‘host’ utility bypasses it, and browsers do not).

Fortunately it is very easy to do:

On Leopard: dscacheutil -flushcache
On Tiger (didn’t check it, no tigers here :): lookupd -flushcache

Non relational MySQL

2009-02-27T09:53:00Z

Nice idea and implementation of a schema-less data store on top of MySQL .

JRuby on Rails with GlassFish Code Camp

2009-02-26T11:48:00Z

We participated in JRuby on Rails with GlassFish Code Camp hosted by Sun Microsystems Inc. I was speaking about the framework in general trying to infect Java developers with Ruby On Rails. Slides are available.

Amit Hurvitz gave exciting presentation about GlassFish and short introduction into DTrace. Find out more details about the Code Camp.

Rediscovering HAML

2009-02-11T15:24:00Z

I was the last person in our company working with ERB to render templates. While all the rest switched to HAML. At the beginning it was quite hard for me to read HAML comparing to ERB. HAML looked for me like some completely alien thing with weird percent marks all over the place and the significant whitespace never did it for me. On the other hand ERB felt like warm home after years we spent together.

Until I did the switch.

Now I would compare HAML with Mac. Once you’ve switched you never get back. Guys from HAML say that “HAML is based on one primary principle. Markup should be beautiful.”, they even go as far as call it markup haiku. Which is true. Together with “beautiful” you get less code that is more readable and secure.

Now, writing rails views that are readable, maintainable and concise was never an easy job. If you let them to get out of hand you quickly find yourself with 400 lines of… not code, no… let’s call it a mess. We actually seen it happens to less careful colleagues, and cleaning this mess was not a fun and easy thing to do.

HAML is here to help you with that.

You obviously can get to the same mess level with HAML, but if you on a task to avoid it, HAML is there to support you.

Simply put, If you write something that doesn’t look beautiful you know that you’re doing something wrong and it is a time to stop for a minute and think about refactoring.

long views with deep nesting – you can check 2 things here
- whether you need to create some partials for duplicating parts
- or you can check your HTML markup (table layout will look really really bad in HAML)
if your lines are too long – create helpers and get your ruby code out of views
inline javascript doesn’t really looks fit – check out the unobtrusive javascript, which will bring along another nicetohaves.

Another big bounty you get is that HTML escaping is opt out vs. the opt in in ERB after the following magic. Put this in in config/environment.rb (after Rails::Initializer.run):

Haml::Template.options[:escape_html] = true

So you have to make an informed decision that you want this piece of code unescaped which hopefully will make your code more secure out of the box with next to nothing additional work on your hands. So long, nasty XSS.

This line should not get your XSS alarm go off

%h2= @product.title

This one should, but i’m sure you know it, cause you went to an extra step of putting ”!” there

#products
  != render(:partial => @products)

Generally, i can’t find a good reason for leaving all your strings unescaped by default (wink-wink ERB). Do I miss something?

Another year in consulting

2009-02-04T21:43:00Z

2008 was the year when we finally switched to full time consulting. And like all consulters we faced the problem of correct pricing. There are two well-known ways to charge a customer: per-hour rate and fixed bid quote, and several combinations of them.

Per-hour rate.

Everything is quite clear here. You just charge per hour rate no matter what. Your customer wears all risks. Per-hour rate is good for short-term consulting projects, like deployments, launch reviews, etc. If you’re going to work on a project that takes more then 10-15 hours, you will need to estimate the required hours to accomplish the tasks. And, obviously, meet this estimation.

So, you calculate estimations. Actually, you always do an estimation, it may be time, it may be money. But you always estimate. Because every customer needs to be sure that she has a budget to hire you.

Fixed bid.

The Dark Art of Realistic Time Estimation is something that anyone can learn. The only question is how much time and money are wasted until you learn it. Because you wear risks in such a case. With fixed bid you can get more money then working on per-hour basis. But also you can loose money, or actually time, working for free because of wrong estimation. Give an expensive quote and customer goes to other developers, give underestimated quote and work for free. Sounds like a dangerous and loose-loose situation.

Sometimes, preparing a realistic fixed bid quote requires several hours or even days of research and planning. At the beginning we did a time consuming planing for free, just to give some number to a potential customer. Sometimes customers just ask for a quote and don’t mean to work with you, they’re just curious about your quote.

Save your time.

We’ve found a way to save our time and filter such customers. We make very very rough estimation that sometimes just a price range, which takes 2-4 hours maximum. If the price range matches customer expectations we proceed with exact quote. This is the first-level filter. Obviously, to issue an exact quote we make general software designs and discuss implementation of complex parts. If a quote preparing takes more then 5-7 hours we charge our customer for working on it. Customers that don’t agree to pay for issuing fixed bid quote don’t pass second-level filter.

In such a way we get only serious customers that really want to hire us and have their projects done. And we always want to work with them. Often, we can reduce a final price by reconsidering our approaches, changing some requirements, reworking flows, etc. We always find out a way to work with these customers, and they always find out a way to work with us. These are high quality customers, and we enjoy every minute of working with them.

Experimental approach.

There is another approach that sounds quite fair. Risks are shared between customer and consulter. Let’s say the original estimation is 10 days, and you charge say $800 per day. The total price of $8000 is split into 2 parts: per day part, say $3500 ($350 per day), and final part $4500. Final part is paid at the end of the job. If something bad is happened and you work 12 days instead of 10 days – customer pays you extra of 2x$350, final part remains unchanged. If you complete the project in 8 days instead of 10 days – customer pays you 2x$350 less, and final part also remains unchanged. So if things got complicated, or you underestimate the tasks you get lower average per-hour rate and customer pays a bit more. If you work faster – you get higher average per-hour rate and customer pays less and gets the project early.

Slow and expensive.

We heard a lot that large companies charge 5 times more then we do, work 3 times slower, and produce ugly unmaintainable code. And they still get customers. So what’s the secret here? I can think about only one reason: they’re big. Sometimes customers think that if you’re small you will disappear in a nearest future, and no one will be able to take care of the project. This is ridiculous when you’re talking about Ruby On Rails.

Are there other reasons?

Securing Panda AMI instance for production

2009-01-22T10:12:00Z

Recently we looked for video transcoding/hosting solution to use in one of our client’s projects.

The best thing we’ve found is Panda. It runs on Amazon stack of services including ec2, s3, and simpledb.

Using amazon has many advantages. no contracts, pay as you go, easy and fast scaling in case your site explodes :)

Unfortunately the image that is refered in the Getting Started (ami-05d7336c) is not safe for production – it has openssh version with a serious security bug, but don’t worry, we will explain how to fix it.

Actually, you should always apply latest security patches and change ssh keys when using public AMI images in production. Or even better, bundle your own AMI image with all your changes and never use public ones in production.

OK, now lets get to work.

start ami instance


$ ec2-run-instances ami-05d7336c -k EC2_PUBLIC_KEY_NAME
RESERVATION    r-xxxxxxxx    xxxxxxxxxxxx    default
INSTANCE    i-xxxxxxx    ami-05d7336c            pending    your_key    0        m1.small    2009-01-22T08:44:12+0000    us-east-1c

EC2_PUBLIC_KEY_NAME is the name of the keypair that you generated using ec2-add-keypair command. (see Amazon EC2 Getting Started Guide for details)

wait until it starts


$ ec2-describe-instances
RESERVATION    r-xxxxxx    xxxxxxxxxxxxxx    default
INSTANCE    i-xxxxxxx    ami-05d7336c    xxxxxxx.compute-1.amazonaws.com    ip-xxxxxxxxx.ec2.internal    running    your_key    0        m1.small    2009-01-22T08:44:12+0000    us-east-1c

login into the instance


$ ssh -i key-c2a root@xxxxx.compute-1.amazonaws.com
The authenticity of host 'xxxx.compute-1.amazonaws.com (xxxxx)' can't be established.
RSA key fingerprint is c4:e0:8e:cc:6a:b6:6f:63:8c:c2:5d:13:7e:77:36:a2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'xxxx.compute-1.amazonaws.com,xxxxx' (RSA) to the list of known hosts.
Last login: Wed Sep 24 10:43:28 2008 from zzzz
Linux xxxxx.compute-1.amazonaws.com 2.6.16-xenU #1 SMP Mon May 28 03:41:49 SAST 2007 i686

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
panda:~#

update apt

The current source list


panda:~# apt-get update
...
Reading package lists... Done
panda:~#

upgrade packages

Those are bug security fixes:


panda:~# apt-get upgrade -u
Reading package lists... Done
Building dependency tree... Done
The following packages have been kept back:
  openssh-client openssh-server
The following packages will be upgraded:
  bsdutils cpio debconf debconf-i18n dpkg dpkg-dev dselect e2fslibs e2fsprogs file findutils git-core initscripts irb1.8 libblkid1 libc6 libc6-amd64 libc6-dev libc6-dev-amd64
  libc6-xen libcairo2 libcomerr2 libdbi-perl libfreetype6 libgnutls13 libkrb53 liblcms1 libmagic1 libmysqlclient15off libopenssl-ruby1.8 libpam-modules libpam-runtime libpam0g
  libpcre3 libpcre3-dev libpcrecpp0 libpq4 libpulse0 libqt3-mt libreadline-ruby1.8 libruby1.8 libruby1.9 libspeex1 libss2 libssl-dev libssl0.9.8 libtiff4 libuuid1 libvorbis-dev
  libvorbis0a libvorbisenc2 libvorbisfile3 libxine1 libxml2 locales login mount mplayer mysql-client-5.0 mysql-common openssl passwd perl perl-base perl-modules postfix python2.4
  python2.4-minimal rdoc1.8 rsync ruby1.8 ruby1.8-dev ruby1.9 sysv-rc sysvinit sysvinit-utils tar tzdata unzip util-linux
80 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
Need to get 77.1MB of archives.
After unpacking 1220kB disk space will be freed.
Do you want to continue [Y/n]? y
Get:1 http://security.debian.org etch/updates/main login 1:4.0.18.1-7+etch1 [797kB]
Get:2 http://ftp.debian.org etch/main bsdutils 1:2.12r-19etch1 [68.5kB]
Get:3 http://ftp.debian.org etch/main dpkg 1.13.26 [2034kB]
Get:4 http://security.debian.org etch/updates/main perl-modules 5.8.8-7etch6 [2328kB]
Get:5 http://ftp.debian.org etch/main libc6-dev 2.3.6.ds1-13etch8 [2718kB]
Get:6 http://security.debian.org etch/updates/main perl 5.8.8-7etch6 [3599kB]                                                                                                       
Get:7 http://ftp.debian.org etch/main libc6-dev-amd64 2.3.6.ds1-13etch8 [2015kB]                                                                                                    
Get:8 http://security.debian.org etch/updates/main perl-base 5.8.8-7etch6 [763kB]                                                                                                   
Get:9 http://ftp.debian.org etch/main libc6-amd64 2.3.6.ds1-13etch8 [3327kB]                                                                                                        
Get:10 http://security.debian.org etch/updates/main libssl-dev 0.9.8c-4etch4 [2094kB]                                                                                               
Get:11 http://security.debian.org etch/updates/main libssl0.9.8 0.9.8c-4etch4 [2721kB]            
...
Get:79 http://ftp.debian.org etch/main unzip 5.52-9etch1 [152kB]                                                                                                                    
Get:80 http://ftp.debian.org etch/main postfix 2.3.8-2+etch1 [1090kB]                                                                                                               
Fetched 77.1MB in 2m36s (494kB/s)                                                                                                                                                   
Extracting templates from packages: 100%
Preconfiguring packages ...
...
Unpacking replacement tzdata ...
Setting up tzdata (2008e-1etch3) ...
Running 'tzconfig' to set this system's timezone.
Your current time zone is set to Unknown
Do you want to change that? [n]: y

Please enter the number of the geographic area in which you live:

    1) Africa            7) Australia

    2) America            8) Europe

    3) US time zones        9) Indian Ocean

    4) Canada time zones        10) Pacific Ocean

    5) Asia                11) Use System V style time zones

    6) Atlantic Ocean        12) None of the above

Then you will be shown a list of cities which represent the time zone
in which they are located. You should choose a city in your time zone.

Number: 12

GMT GMT+0 GMT+1 GMT+10 GMT+11 GMT+12 GMT+2 GMT+3 GMT+4 GMT+5 GMT+6 GMT+7
GMT+8 GMT+9 GMT-0 GMT-1 GMT-10 GMT-11 GMT-12 GMT-13 GMT-14 GMT-2 GMT-3
GMT-4 GMT-5 GMT-6 GMT-7 GMT-8 GMT-9 GMT0 Greenwich UCT UTC Universal Zulu

Please enter the name of one of these cities or zones
You just need to type enough letters to resolve ambiguities
Press Enter to view all of them again
Name: [] UTC
Your default time zone is set to 'Etc/UTC'.
Local time is now:      Thu Jan 22 09:03:14 UTC 2009.
Universal Time is now:  Thu Jan 22 09:03:14 UTC 2009.

...
Setting up perl (5.8.8-7etch6) ...

Setting up dpkg-dev (1.13.26) ...
Setting up mysql-client-5.0 (5.0.32-7etch8) ...
panda:~#

Notice 2 packages that were not upgraded: openssh-client and openssh-server

upgrade openssh


panda:~# apt-get install openssh-server 
Reading package lists... Done
Building dependency tree... Done
The following extra packages will be installed:
  openssh-blacklist openssh-client
Suggested packages:
  ssh-askpass xbase-clients rssh molly-guard
The following NEW packages will be installed:
  openssh-blacklist
The following packages will be upgraded:
  openssh-client openssh-server
2 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 3006kB of archives.
After unpacking 4096kB of additional disk space will be used.
Do you want to continue [Y/n]? y
...
Setting up openssh-server (4.3p2-9etch3) ...
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Restarting OpenBSD Secure Shell server: sshd.

panda:~#

During the installation it created new sshd server keys.

check for vulnerable ssh keys


panda:~# ssh-vulnkey  -a
Not blacklisted: 2048 25:7b:b5:cf:ae:44:87:00:9d:b6:62:39:f9:4e:10:7d /etc/ssh/ssh_host_rsa_key.pub
Not blacklisted: 1024 e2:26:98:52:ad:9f:2c:43:77:45:71:e9:87:17:7c:08 /etc/ssh/ssh_host_dsa_key.pub
Not blacklisted: 2048 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx your-key
panda:~#

update known_hosts file

you just changed server’s ssh key, so next time you try to login ssh will complain:


$ ssh -i your-key root@xxxxxxxxxxxxxxxxxxx.compute-1.amazonaws.com
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Please contact your system administrator.
Add correct host key in /Users/vitaly/.ssh/known_hosts to get rid of this message.
Offending key in /Users/vitaly/.ssh/known_hosts:68
RSA host key for xxxxxxxxxxxxxxxxxx.compute-1.amazonaws.com has changed and you have requested strict checking.
Host key verification failed.

Remove line with the old host key (filename and line number is given by ssh) and you will be able to login again

install panda

Just follow Panda Getting Started Guide starting with “Grab Panda”

bundle new AMI

Now you can create a new Amazon AMI image from your current instance. Refer to Amazon EC2 Getting Started Guide (section “Creating an Image”)