A Single Programmer's Blog - Home

"piston" on Google returns piston.rubyforge.org

2008-05-14T04:23:00Z

Wow, without my ever doing anything special, searching for “piston” on Google returns this:

Thanks to everyone who linked to Piston and made this happen.

Piston 2.0 Progress: info, lock, unlock

2008-05-13T22:38:00Z

Tapajos has been helping me shaping Piston into shape. And I’m working on update now, so expect a 1.9.3 RSN

Piston 2.0 Progress: native or ruby patching ?

2008-05-09T15:57:00Z

piston update is coming. I have the high-level workflow completed. Conceptually, updating is pretty simple:

module Piston
  module Commands
    class Update < Piston::Commands::Base
      # +wcdir+ is the working copy we're going to change.
      # +to+ is the new target revision we want to be at after update returns.
      def run(wcdir, to)
        working_copy = Piston::WorkingCopy.guess(wcdir)

        logger.debug {"Recalling previously saved values"}
        values = working_copy.recall

        repository_class = values["repository_class"]
        repository_url = values["repository_url"]
        repository = repository_class.constantize.new(repository_url)
        from_revision = repository.at(values["handler"])
        to_revision = repository.at(to)

        logger.debug {"Validating that #{from_revision} exists and is capable of performing the update"}
        from_revision.validate!

        logger.info {"Updating from #{from_revision} to #{to_revision}"}
        working_copy.apply_differences(from_revision.differences_with(to_revision))
      end
    end
  end
end

Obviously, the devil’s in the details… Notice the last line above:

1	working_copy.apply_differences(from_revision.differences_with(to_revision))

from_revision will calculate a set of differences between itself and to_revision. In Subversion speak, that would probably mean an svn log followed by an svn diff, to get all changes (copies + diffs).

What Piston 1.x does is copy the newer file over the original file, and then applies the changes between the last changed revision of the local files and the working copy. This ensures changes that were made are kept in the new revision.

I know I can do it under Subversion, as I have already done it, but what about git ? I can probably use a combination of git-format-patch and git-apply to get the job done. That would certainly work.

I also thought about finding / using a patch implementation in Ruby. I wonder if that would be another acceptable road ? Anybody out there has / knows about a patch implementation in Ruby ?

Super simple git-rebase explanation

2008-05-06T21:14:00Z

James Bowes has a super simple git-rebase explanation: git rebase: keeping your branches current.

If you have any difficulty understanding the concept, or want an easy way to explain to someone, go ahead.

I’m just starting to use git-svn and git-rebase, and the combination rocks.

Capistrano Tip'n'Trick

2008-05-03T05:32:00Z

If like me you didn’t know, you can do that with Capistrano:

12	$ cap HOSTS=web0.myapp.com deploy:update_codecap ROLES=web,app deploy

Not terribly useful for the tasks demonstrated above, but what about this ?

1	$ cap ROLES=app monit:summary

Incidentally, where are these documented ? I dimly remembered reading this somewhere once, and tried it. Works like a charm!

New Piston Contributor: Josh Nichols

2008-04-21T20:38:00Z

I just received 2 contributions from Josh Nichols:

New—repository-type option on piston import to force the repository backend to use (instead of letting Piston guess), and for cases where Piston is unable to guess: ea958dd
Test suite reorganization: 1cef7b6 and 9cfa8f3

Both contributions were accepted and are now part of Piston’s master branch. Thank you very much, Josh, for your work.

If you want to help, do not fear !

$ git clone git://github.com/francois/piston.git
$ # make changes
$ git commit
$ # fork piston's repository
$ git remote add github git@github.com:YOURNAME/piston.git
$ git push github master
$ # Send me a pull request

Cliaws SQS: Command-line access to Amazon Simple Queue Service

2008-04-17T20:57:00Z

Have you ever wished you could access your Amazon Simple Queue Service from the command line ? Now you can:

$ AWS_ACCESS_KEY_ID=<your access key>
$ AWS_SECRET_ACCESS_KEY="<the secret access key>"

$ clisqs create my-queue
Queue my-queue was created.

$ clisqs list
my-queue

$ clisqs push --data "this is the message" my-queue
Pushed 19 bytes to queue my-queue

$ cat README.txt | clisqs push my-queue
Pushed 2687 bytes to queue my-queue

$ clisqs push my-queue README.txt
Pushed 2687 bytes to queue my-queue

$ clisqs size my-queue
3

$ clisqs pop my-queue
this is the message

$ clisqs delete --force my-queue
Queue my-queue was deleted.

Installation

1	$ sudo gem install cliaws

Direct-code access

require "rubygems"
require "cliaws"

Cliaws.sqs.push("my-queue", "the data")
the_size = Cliaws.sqs.size("my-queue")
the_message = Cliaws.sqs.pop("my-queue")

S3 ?

This gem also works with S3. See my prior release announcement: Cliaws: command-line access to S3

Cliaws: command-line access to S3

2008-04-15T01:43:00Z

I just released a new gem on RubyForge: cliaws. Using it, you have access to nice command line shortcuts:

$ AWS_ACCESS_KEY_ID=<your access key>
$ AWS_SECRET_ACCESS_KEY="<the secret access key>"
$ clis3 list BUCKET/PATH-PREFIX
$ clis3 put BUCKET/FULL/PATH/NAME /path/to/local/file
$ clis3 get BUCKET/FULL/PATH/NAME /path/to/local/file
$ clis3 cat BUCKET/FULL/PATH/NAME

This is not rocket science, but with the kind of work I’m doing at the moment, having quick command-line access to S3 is a boon.

Installation

1	$ gem install cliaws

Direct-code access

require "rubygems"
require "cliaws"

# Retrieve and store in a variable
value = Cliaws.s3.get("BUCKET/FULL/PATH/NAME")

# Retrieve and stream to specified file
File.open("/path/to/local/file", "w") do |f|
  Cliaws.s3.get("BUCKET/FULL/PATH/NAME", f)
end

# Returns an Array of full names
contents = Cliaws.s3.list("BUCKET")

Repository

The repository is hosted on GitHub, with a mirror on RubyForge.

The repository’s URLs are:

git://github.com/francois/cliaws.git
git://rubyforge.org/cliaws.git

Go forth and fork !

Is Windows a good environment for Ruby/Rails ? It certainly should !

2008-03-26T13:53:00Z

I just stumbled upon Is Windows a First Class Platform for Ruby? by Peter Cooper and Is Windows a supported platform for Ruby? I guess not by Luis Lavena.

I have to admit, I did use Windows daily for 3 years before I switched permanently to Ubuntu, about 6 months back. Initially, I was using IntelliJ’s IDEA as my development platform, then I switched to the e Text Editor. I never had many problems with gems that didn’t/couldn’t/wouldn’t install. There are many kind souls in the community that keep these gems up to date. I’m talking specifically about Tim Hunter (RMagick), Luis Lavena (Mongrel) and others.

Just before I made the final switch, I was using Linux in a VMware image to run XLsuite. The application was unbearably slow in Windows, but acceptable in a virtual machine. I was using the e Text Editor as my editor, accessing the code through a Samba share.

My own experience was pretty positive. Now that I’m on Ubuntu, I wouldn’t go back though. What made me switch ? Better performance on the same hardware, mostly; the novelty of the experience. I do not dislike Windows, nor do I think Redmond is a bad place. Windows is a fine platform.

Just to contrast, look at the excellent support Java enjoys on Windows. Windows is the 2^nd platform for Java (with Solaris being the 1^st). We, the Ruby community, should be learning from Sun. There are many, many more Windows machines than Mac or Linux machines out there. There are literally millions of people who could learn to write Ruby, but are on Windows. Dr Nic said it all: <q cite="http://www.rubyinside.com/is-windows-a-first-class-platform-for-ruby-823.html#comment-35759">… People Use Windows Too</q>. Whether you want to or not, Windows isn’t going away soon.

Personally, I have made the switch. But just on my small team, Windows users outnumber other platforms 1 to 1. Here’s the breakdown:

4 Windows (2 coders, 1 designer, 1 sponsor)
2 Ubuntu (1 coder, 1 designer)
1 Mac (1 ExtJS coder)

Is Windows a good platform for Ruby ? Yes. Is Windows a great platform for Ruby ? No, but there’s no reason why it shouldn’t be. Just look at Why’s work on Hackety Hack. I’m really impressed and happy that I will be able to show Ruby to my daughters. The catch ? Hackety Hack is for Windows only right now.

Please, let’s keep and increase Windows support. Once they’re hooked, they might switch, who knows ?

Piston 2.0 Progress: Much saner logging

2008-03-25T04:25:00Z

Instead of using Logger, I’m now using Log4r. This is a departure for me, as I initially gave myself the goal of not depending on too many libraries. But since I’m already depending on Main (which itself has 2 dependencies) and open4, I thought, “why not another one ?”

But this new dependency gives me much greater freedom in logging. I’m not done coding all of this, but—verbose won’t just be a flag. It will represent a level, and the higher the level, the more logging will be done. Obvious, but much more interesting.

Anyway, here’s what 1.9.1 was logging for a simple svn/git pistonization:

$ ruby -I lib bin/piston import http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement
D, [2008-03-25T00:41:13.494826 #13759] DEBUG -- : Piston::Commands::Import with options {:verbose=>false, :force=>false, :quiet=>false, :lock=>false, :dry_run=>false}
D, [2008-03-25T00:41:13.495078 #13759] DEBUG -- : Guessing the repository type of "http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement"
D, [2008-03-25T00:41:13.495386 #13759] DEBUG -- : git ls-remote --heads http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement
D, [2008-03-25T00:41:13.495543 #13759] DEBUG -- : > "git ls-remote --heads http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement"
D, [2008-03-25T00:41:13.721569 #13759] DEBUG -- : > #<Process::Status: pid=13760,exited(1)>, success? false, status: 1
D, [2008-03-25T00:41:13.722096 #13759] DEBUG -- : svn info http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement
D, [2008-03-25T00:41:19.142407 #13759] DEBUG -- : Path: ssl_requirement
URL: http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement
Repository Root: http://dev.rubyonrails.org/svn/rails
Repository UUID: 5ecf4fe2-1ee6-0310-87b1-e25e094e27de
Revision: 9088
Node Kind: directory
Last Changed Author: bitsweat
Last Changed Rev: 8780
Last Changed Date: 2008-02-02 00:16:53 -0500 (Sat, 02 Feb 2008)


D, [2008-03-25T00:41:19.142810 #13759] DEBUG -- : Guessing the working copy type of #<Pathname:repository>
D, [2008-03-25T00:41:19.142950 #13759] DEBUG -- : Asking Piston::Git::WorkingCopy if it understands repository
D, [2008-03-25T00:41:19.143063 #13759] DEBUG -- : git status on repository
D, [2008-03-25T00:41:19.143490 #13759] DEBUG -- : git status on .
D, [2008-03-25T00:41:19.143681 #13759] DEBUG -- : git status
D, [2008-03-25T00:41:19.143848 #13759] DEBUG -- : > "git status"
D, [2008-03-25T00:41:19.166951 #13759] DEBUG -- : > #<Process::Status: pid=13772,exited(1)>, success? false, status: 1
D, [2008-03-25T00:41:19.167193 #13759] DEBUG -- : # On branch my1.9.1
nothing to commit (working directory clean)

D, [2008-03-25T00:41:19.167443 #13759] DEBUG -- : Initialized on repository
D, [2008-03-25T00:41:19.167920 #13759] DEBUG -- : svn checkout --revision HEAD http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement .repository.tmp
D, [2008-03-25T00:41:25.100301 #13759] DEBUG -- : A    .repository.tmp/test
A    .repository.tmp/test/ssl_requirement_test.rb
A    .repository.tmp/lib
A    .repository.tmp/lib/ssl_requirement.rb
A    .repository.tmp/README
Checked out revision 9088.

D, [2008-03-25T00:41:25.100986 #13759] DEBUG -- : svn ls --recursive .repository.tmp
D, [2008-03-25T00:41:30.056625 #13759] DEBUG -- : README
lib/
lib/ssl_requirement.rb
test/
test/ssl_requirement_test.rb

D, [2008-03-25T00:41:30.057107 #13759] DEBUG -- : Copying README to repository/README
D, [2008-03-25T00:41:30.058074 #13759] DEBUG -- : Copying lib/ssl_requirement.rb to repository/lib/ssl_requirement.rb
D, [2008-03-25T00:41:30.058994 #13759] DEBUG -- : Copying test/ssl_requirement_test.rb to repository/test/ssl_requirement_test.rb
D, [2008-03-25T00:41:30.059800 #13759] DEBUG -- : svn info --revision 9088 http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement
D, [2008-03-25T00:41:34.750474 #13759] DEBUG -- : Path: ssl_requirement
URL: http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement
Repository Root: http://dev.rubyonrails.org/svn/rails
Repository UUID: 5ecf4fe2-1ee6-0310-87b1-e25e094e27de
Revision: 9088
Node Kind: directory
Last Changed Author: bitsweat
Last Changed Rev: 8780
Last Changed Date: 2008-02-02 00:16:53 -0500 (Sat, 02 Feb 2008)


D, [2008-03-25T00:41:34.751037 #13759] DEBUG -- : Remembering {"piston:remote-revision"=>9088, "piston:root"=>"http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement", "piston:uuid"=>"5ecf4fe2-1ee6-0310-87b1-e25e094e27de"}
D, [2008-03-25T00:41:34.752256 #13759] DEBUG -- : Calling #after_remember on repository/.piston.yml
D, [2008-03-25T00:41:34.752475 #13759] DEBUG -- : git add .
D, [2008-03-25T00:41:34.752605 #13759] DEBUG -- : > "git add ."
D, [2008-03-25T00:41:34.758728 #13759] DEBUG -- : > #<Process::Status: pid=13785,exited(0)>, success? true, status: 0
D, [2008-03-25T00:41:34.758993 #13759] DEBUG -- : Removing temporary directory: .repository.tmp

And here’s the current master branch:

$ ruby -I lib bin/piston import http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement
 INFO main: Guessing the repository type
 INFO main: Guessing the working copy type
 INFO main: Checking out the repository
 INFO main: Copying from Piston::Revision(http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement@9088)
 INFO main: Checked out "http://dev.rubyonrails.org/svn/rails/plugins/ssl_requirement" r9088 to "ssl_requirement"

Obviously, in 1.9.1, I hadn’t configured the logger at all, and everything was logged. Not so anymore !

Piston 1.9.1: Import goodness

2008-03-23T18:22:00Z

Well, with much more testing, I’m releasing another preview release of Piston. This release should import Subversion or Git repositories into Subversion or Git working copies just fine. There’s one slight problem, which is when you use piston import URL (without a target directory), it will import into a folder named repository, which isn’t what it’s supposed to do. I’m missing a couple of tests, is all.

How to grab this release ? Two ways:

git clone git://github.com/francois/piston.git
Grab a tarball

Once you have the code, run rake install_gem to install the gem. Enjoy !

Piston 2.0 Progress: New integration tests

2008-03-23T03:25:00Z

I have new integration tests, and they work just beautifully. I’m missing a couple more, but things are looking very good.

Thanks to Paul Watson for finding and fixing two bugs in the Git/Git case.

Finally, I have faisal which offered looking into adding SVK support for the working copy.

Piston 1.9.0: Official Preview Release

2008-03-17T03:29:00Z

I’m making it official. I’m releasing 1.9.0 today:

http://github.com/francois/piston/tree/1.9.0

I’ll put up a gem tomorrow, but if you want to play with Piston now, the best way is to either clone the repository, or grab a tarball directly from GitHub.

What’s implemented in this release ?

piston import

But you can import from either SVN or Git, into either SVN or Git. All Piston metadata is stored in a .piston.yml file in the pistonized dir’s root.

If you have questions, problems, comments, go ahead and comment right here, or use Piston’s tracker

Piston 2.0 Progress: Piston can import all four cases

2008-03-15T19:03:00Z

As I outlined in Piston will get Git support, the four cases below are now supported (at least for importing):

Repository	Working Copy	Strategy
Subversion	Subversion	Use current strategy of storing data in Subversion properties
Subversion	Git	Clone the Git repository, copy the files over and store the metadata as Subversion properties. Use Git to handle the merging for Piston (Yay!)
Git	Subversion	svn export the data and use a hidden YAML file to store the metadata in the pistonized directory
Git	Subversion	Use Git submodules perhaps ? Or git clone + copy + YAML

I’m not in fact using git submodules, or anything fancy. I’m cloning the repository, and copying manually from there. So, nothing fancy. But adding new repository and working copy handlers is so easy:

samples/import_svn_svn.rb

#!/usr/bin/env ruby
#
# Import an SVN repository into an SVN working copy.
require File.dirname(__FILE__) + "/common"

@root = @root + "tmp/git_git"
@root.rmtree rescue nil
@root.mkpath

@tmp = @root + "plugin.tmp"

@plugin = @root + "plugin"
@plugin.mkpath
File.open(@plugin + "README", "wb") {|f| f.puts "Hello World"}
File.open(@plugin + "init.rb", "wb") {|f| f.puts "# Some init code"}
Dir.chdir(@plugin) do
  git :init
  git :add, "."
  git :commit, "-m", "initial commit"
end

@wc = @root + "wc"
@wc.mkpath
File.open(@wc + "README", "wb") {|f| f.puts "My local project"}
Dir.chdir(@wc) do
  git :init
  git :add, "."
  git :commit, "-m", "initial commit"
end

repos = Piston::Git::Repository.new("file://" + @plugin.realpath)
commit = repos.at(:head)
commit.checkout_to(@tmp)

wc = Piston::Git::WorkingCopy.new(@wc + "vendor")
wc.create
wc.copy_from(commit)
wc.remember(commit.remember_values)
wc.finalize

samples/import_git_svn.rb

#!/usr/bin/env ruby
#
# Import a Git project into a Subversion working copy.
require File.dirname(__FILE__) + "/common"

@root = @root + "tmp/git_svn"
@root.rmtree rescue nil
@root.mkpath

@repos = @root + "repos"
@wc = @root + "wc"

@plugin = @root + "plugin"
@tmp = @root + "plugin.tmp"

svnadmin :create, @repos
svn :checkout, "--quiet", "file://" + @repos.realpath, @wc

@plugin.mkpath
File.open(@plugin + "README", "wb") {|f| f.puts "Hello World"}
File.open(@plugin + "init.rb", "wb") {|f| f.puts "# Some initialization code here"}
Dir.chdir(@plugin) do
  logger.debug {"CWD: #{Dir.getwd}"}
  git :init
  git :add, "."
  git :commit, "-m", "initial commit"
end

repos = Piston::Git::Repository.new("file://" + @plugin.realpath)
commit = repos.at(:head)
commit.checkout_to(@tmp)
wc = Piston::Svn::WorkingCopy.new(@wc + "vendor")
wc.create
wc.copy_from(commit)
wc.remember(commit.remember_values)
wc.finalize

Do you see the differences ? They’re all in the setup code. Once we hit commit.checkout_to, everything else is the same.

I’m almost ready to release a release candidate. This will be 1.9.0, and only support the import subcommand. It will at least expose the code to more testing than just what I have.

Oh, and no more Piston 1.3.3: Now with specifications. This version of Piston was tested right from the start.

Git status exit status ?

2008-03-15T02:46:00Z

$ git status
# On branch master
nothing to commit (working directory clean)
$ echo $?
1

Shouldn’t a successful git status in a git repository return a status code of 0 ? Doing it in a random folder returns a sensible value:

$ git status
fatal: Not a git repository
Failed to find a valid git directory.
$ echo $?
128

Can anybody shed some light on this ?

I found Possible bug in ‘git status’ exit code is 1 instead of 0. What is the rational for this ? It goes against everything Unix ?!?