Comments for Development Blog: Viget Labs http://www.viget.com/extend/ en David Eisinger david.eisinger@viget.com Wed, 08 Jul 2009 20:21:17 -0400 Comment on How (& Why) to Run Autotest on your Mac by Andrew Cox http://feedproxy.google.com/~r/extendcomments/~3/sKurylBd_G0/ http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#9012 Great stuff - the first post I found that actually worked.  It sure is easy to find outdated information about autotest :)

One thing that’s really bugging me though is that autotest isn’t rerunning all of my tests after passing a failed test. According to the strategy on autotest’s home page, it looks like it’s supposed to rerun all tests once passing a subset of tests:

1. Find all files and associate them from impl <-> test.
2. Run all tests.
3. Scan for failures.
4. Detect changes in ANY (ruby?. file, rerun all failures + changed files.
5. Until 0 defects, goto 3.
6. When 0 defects, goto 2.

Is it supposed to do that still? If it doesn’t do that anymore, then is there a hook I can add to make it do that? Because that seems really useful. I’m using ZenTest (4.1.3), autotest-fsevent (0.1.1), and autotest-rails (4.1.0).

]]> Andrew Cox Wed, 08 Jul 2009 20:21:17 -0400 http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#9012 Comment on Getting Started with Webrat, Selenium, Rails and Firefox 3 by Javier Alvarez http://feedproxy.google.com/~r/extendcomments/~3/OiMeRJS01DA/ http://www.viget.com/extend/getting-started-with-webrat-selenium-rails-and-firefox-3/#9010 Hi Brian:
After a horrible afternoon pulling my hair trying to make rails work with selenium and cucumber I have stumbled with your Post. Thank you so much.
I have a question though… Which of the 1000+ possible base setups did you use for your app?? I you could point me to the right url, I’m sure that will increase you karma.
Thanks in advance.

]]> Javier Alvarez Tue, 07 Jul 2009 22:26:00 -0400 http://www.viget.com/extend/getting-started-with-webrat-selenium-rails-and-firefox-3/#9010 Comment on Introducing Garb: Access the Google Analytics Data Export API with Ruby by Tony Pitale http://feedproxy.google.com/~r/extendcomments/~3/EUo5qjDOShE/ http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8996 Hi folks, sorry it took so long! The gem should be up on Rubyforge momentarily.

]]> Tony Pitale Sun, 05 Jul 2009 19:34:34 -0400 http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8996 Comment on Introducing Garb: Access the Google Analytics Data Export API with Ruby by Bruce Williams http://feedproxy.google.com/~r/extendcomments/~3/xIjpGhkpMo0/ http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8995 I’d love to see that new gem on Rubyforge too; also running into this issue.

]]> Bruce Williams Sun, 05 Jul 2009 18:51:51 -0400 http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8995 Comment on Rails, Internationalization, and Tú by Francesc Esplugas http://feedproxy.google.com/~r/extendcomments/~3/O_VXE9FnwZk/ http://www.viget.com/extend/rails-internationalization-and-tu/#8985 For translating strings you can use Rails I18n backend instead of using inflectors.

The `typus_human_name` is a patch to fix a problem in `human_name` [1].

[1] https://rails.lighthouseapp.com/projects/8994/tickets/2120-humanize-and-human_name-dont-separate-words

]]> Francesc Esplugas Fri, 03 Jul 2009 09:08:54 -0400 http://www.viget.com/extend/rails-internationalization-and-tu/#8985 Comment on Test Drive mod_rewrite Rules With Test::Unit by Patrick Reagan http://feedproxy.google.com/~r/extendcomments/~3/LhXC8-GCqFE/ http://www.viget.com/extend/test-drive-mod-rewrite-rules-with-testunit/#8983 @Dean: This isn’t an approach to do controller testing.  It’s simply providing a DSL to *externally* test your web server by treating it as a black box and examining the responses returned.  In my case, I was using Apache, but there’s no reason that this approach wouldn’t work against Litespeed, lighttpd, Ngnix, or others.

Check out Matt’s gem for a quicker way to get started using this.

]]> Patrick Reagan Thu, 02 Jul 2009 14:27:20 -0400 http://www.viget.com/extend/test-drive-mod-rewrite-rules-with-testunit/#8983 Comment on Test Drive mod_rewrite Rules With Test::Unit by Dean http://feedproxy.google.com/~r/extendcomments/~3/S0M7b7J6Uoo/ http://www.viget.com/extend/test-drive-mod-rewrite-rules-with-testunit/#8981 I must have missed what about the ruby class (or your environment) makes sure that Apache is in the loop and processing rewrite rules.

My scenario is that we use litespeed (with a mod_rewrite), but my impression was that controller testing didn’t require (or use) a running web server.

Thanks for any help..

]]> Dean Thu, 02 Jul 2009 12:39:19 -0400 http://www.viget.com/extend/test-drive-mod-rewrite-rules-with-testunit/#8981 Comment on Rails, Internationalization, and Tú by Tony Pitale http://feedproxy.google.com/~r/extendcomments/~3/xXFumswgkVE/ http://www.viget.com/extend/rails-internationalization-and-tu/#8980 Through advice from Patrick, I’ve used translations to swap simple content in pseudo-white-labeled sites. It worked very well in cases where content_for would have been painful and messy, and partials would be complete overkill.

]]> Tony Pitale Thu, 02 Jul 2009 09:47:29 -0400 http://www.viget.com/extend/rails-internationalization-and-tu/#8980 Comment on Introducing Garb: Access the Google Analytics Data Export API with Ruby by Philipp http://feedproxy.google.com/~r/extendcomments/~3/zEDCHiW0J_k/ http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8979 Please, can you post it, when the gem is up to date?…
Thank you very much!

]]> Philipp Thu, 02 Jul 2009 08:04:25 -0400 http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8979 Comment on Introducing Garb: Access the Google Analytics Data Export API with Ruby by Tony Pitale http://feedproxy.google.com/~r/extendcomments/~3/p-ndps5dv2E/ http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8970 Google changed their API for requesting accounts. We’ve updated the gem on github, and will be pushing a new gem to rubyforge shortly.

]]> Tony Pitale Wed, 01 Jul 2009 10:17:25 -0400 http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8970 Comment on Introducing Garb: Access the Google Analytics Data Export API with Ruby by Philipp http://feedproxy.google.com/~r/extendcomments/~3/5ZpViOTp0TE/ http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8969 Thank for this article. But i have a problem with this GEM:

after i do:
profile = Garb::Profile.all.first
i become this error:
RuntimeError: “Accounts URL did not end in /default”

What is the error here? A few days ago it still worked?!

Hope you can help.
Thx

]]> Philipp Wed, 01 Jul 2009 07:22:12 -0400 http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8969 Comment on How (& Why) to Run Autotest on your Mac by David Eisinger http://feedproxy.google.com/~r/extendcomments/~3/JrLL4Iz4e_Q/ http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8934 @George: to run Autotest on Linux, you’ll want to skip steps four through six. If you’d like to get Growl-style notifications, this page looks like a good place to start.

]]> David Eisinger Wed, 24 Jun 2009 10:03:00 -0400 http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8934 Comment on How (& Why) to Run Autotest on your Mac by George http://feedproxy.google.com/~r/extendcomments/~3/B56-3LONAJg/ http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8932 Hey thanks for the short way procedure of Auto test.
But i have Linux,Is it applicable to it?

]]> George Wed, 24 Jun 2009 08:06:54 -0400 http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8932 Comment on How (& Why) to Run Autotest on your Mac by David Eisinger http://feedproxy.google.com/~r/extendcomments/~3/Sb5ddrW--kM/ http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8914 @Kevin: cool, I’ll definitely pull that down.

@Scott: Thanks! As far as I know, you can run Autotest on Windows or Linux, you just won’t be able to run the fsevent & growl add-ons. Not that I don’t recommend switching to a Mac, of course :)

]]> David Eisinger Sun, 21 Jun 2009 13:21:28 -0400 http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8914 Comment on How (& Why) to Run Autotest on your Mac by Scott Radcliff http://feedproxy.google.com/~r/extendcomments/~3/y5zBPXPTymE/ http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8913 I’m still quite new to Ruby and building Rails apps, and recently discovered cucumber. I will definitely look into Autotest when I switch to a Mac (I’m jumping the PC ship in a few weeks). Thanks for the primer, you guys have a great blog.

]]> Scott Radcliff Sun, 21 Jun 2009 11:43:35 -0400 http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8913 Comment on How (& Why) to Run Autotest on your Mac by Kevin http://feedproxy.google.com/~r/extendcomments/~3/jwuWcnzyDDY/ http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8911 You can color the autotest results (green/red for pass/fail) by adding this line to your ~/.autotest:

require ‘autotest/redgreen’

You may also have to run this first, in case you don’t have the RedGreen gem installed:

gem install --remote Redgreen

This gem will basically just add a red or green bar to the test results. This helps me glance over at my terminal window to see whether or not the changes I made to my code had any effect on the tests.

]]> Kevin Sat, 20 Jun 2009 00:16:10 -0400 http://www.viget.com/extend/how-why-to-run-autotest-on-your-mac/#8911 Comment on Hoptoad: in Brief by vijay http://feedproxy.google.com/~r/extendcomments/~3/LoYZiZlxanI/ http://www.viget.com/extend/hoptoad-in-brief/#8894 Commenting on my own post.. I spoke too soon.. Looks like there is an API to get the errors.. This doubles my excitement about this tool.

]]> vijay Wed, 17 Jun 2009 03:04:49 -0400 http://www.viget.com/extend/hoptoad-in-brief/#8894 Comment on Hoptoad: in Brief by vijay http://feedproxy.google.com/~r/extendcomments/~3/Wwc2IayPvoM/ http://www.viget.com/extend/hoptoad-in-brief/#8893 This is a great tool. It would really help if there was a webservice around it to retrieve the error information and use it for a custom dashboard. Not sure if there is one already..

]]> vijay Wed, 17 Jun 2009 02:57:16 -0400 http://www.viget.com/extend/hoptoad-in-brief/#8893 Comment on Backup your Database in Git by Bram Schoenmakers http://feedproxy.google.com/~r/extendcomments/~3/-qvT-ltndiM/ http://www.viget.com/extend/backup-your-database-in-git/#8847 There are some configuration options for git which should decrease the disk usage of the repository:

* core.compression = 9 : Flag for gzip to specify the compression level for blobs and packs. Level 1 is fast with larger file sizes, level 9 takes more time but results in better compression.
* repack.usedeltabaseoffset = true : Defaults to false for compatibility reasons, but is supported with Git >=1.4.4.
* pack.windowMemory = 100m : (Re)packing objects may consume lots of memory. To prevent all your resources go down the drain it’s useful to put some limits on that. There is also pack.deltaCacheSize.
* pack.window = 15 : Defaults to 10. With a higher value, Git tries harder to find similar blobs.
* gc.auto = 1000 : Defaults to 6700. As indicated in the article it is recommended to run git gc every once in a while. Personally I run git gc --auto everyday, so only pack things when there’s enough garbage. git gc --auto normally only triggers the packing mechanism when there are 6700 loose objects around. This flag lowers this amount.
* gc.autopacklimit = 10: Defaults to 50. Every time you run git gc, a new pack is generated of the loose objects. Over time you get too many packs which waste space. It is a good idea to combine all packs once in a while into a single pack, so all objects can be combined and deltified. By default git gc does this when there are 50 packs around. But for this situation a lower number may be better.

]]> Bram Schoenmakers Tue, 09 Jun 2009 03:36:17 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8847 Comment on Named Scope Caching by Adam http://feedproxy.google.com/~r/extendcomments/~3/-VXcAPbpqFE/ http://www.viget.com/extend/named-scope-caching/#8834 Thanks for pointing out the multi-app caching gotcha with this technique.  I’ll be running more than one instance of my app in the future so I’ll probably use memcache.  Your code example definitely helped me figure out how to handle this type of stuff in Rails though!

]]> Adam Fri, 05 Jun 2009 15:58:12 -0400 http://www.viget.com/extend/named-scope-caching/#8834 Comment on Named Scope Caching by Brian Landau http://feedproxy.google.com/~r/extendcomments/~3/Y_QS-3xmm2A/ http://www.viget.com/extend/named-scope-caching/#8831 Adam:
Glad, you’ve found this useful. And yes you can just throw drop that gist into your /lib directory.

Andrew:
You’re absolutely correct. If you’re running more than one instance of the app this method will fail. In that case instead of caching to a class variable you would cache to memcache or it’s equivalent.

]]> Brian Landau Fri, 05 Jun 2009 13:15:59 -0400 http://www.viget.com/extend/named-scope-caching/#8831 Comment on Named Scope Caching by Andrew Chase http://feedproxy.google.com/~r/extendcomments/~3/gN9slvmz9vE/ http://www.viget.com/extend/named-scope-caching/#8830 I’ve used a similar approach to this in the past, but as soon as you deploy the app across across multiple mongrels (or whatever other app container you’re running) the cache invalidation logic fails. One mongrel may receive a save/create/delete that triggers local cache invalidation, but meanwhile all the other instance still think their cache data is legit. I agree that there are times when caching this type of data can be useful, but the data being cached should either never change or a more sophisticated (shared) caching layer should be used (e.g. memcache).

Cheers,
Andrew

]]> Andrew Chase Fri, 05 Jun 2009 12:27:12 -0400 http://www.viget.com/extend/named-scope-caching/#8830 Comment on Named Scope Caching by Adam http://feedproxy.google.com/~r/extendcomments/~3/FO3zLNOSY_E/ http://www.viget.com/extend/named-scope-caching/#8828 I’ve been looking for an elegant way to solve this problem for awhile and you nailed it!  I had a lot of constants defined in various models throughout my app that hit the database to retrieve their values.  Ideally, I just wanted to run these queries once (when the app was initialized) and then have them cached on subsequent calls.  Needless to say, I ran into lots of problems running migrations because the queries would try execute before the tables were created.  Even tried config.after_initialize with no luck.  It was hard to find a ‘best practice’ for handling this type of situation until I came across your blog.

What’s the best way to use this in my app?  As a plugin or just drop it in /lib?

Thanks!

]]> Adam Thu, 04 Jun 2009 17:16:57 -0400 http://www.viget.com/extend/named-scope-caching/#8828 Comment on Updated Garb: Even Easier Access to the Google Analytics API by Ryan http://feedproxy.google.com/~r/extendcomments/~3/zL-04kyBFyo/ http://www.viget.com/extend/updated-garb-even-easier-access-to-the-google-analytics-api/#8821 Thanks for the quick response! Just created an issue on Github here: http://github.com/vigetlabs/garb/issues#issue/6

]]> Ryan Tue, 02 Jun 2009 16:08:56 -0400 http://www.viget.com/extend/updated-garb-even-easier-access-to-the-google-analytics-api/#8821 Comment on Updated Garb: Even Easier Access to the Google Analytics API by Tony Pitale http://feedproxy.google.com/~r/extendcomments/~3/0ndAOjNFh48/ http://www.viget.com/extend/updated-garb-even-easier-access-to-the-google-analytics-api/#8820 I am not sure why that is. If you could open an issue on Github and leave the stack trace I’ll be sure to look at it ASAP.

]]> Tony Pitale Tue, 02 Jun 2009 15:52:46 -0400 http://www.viget.com/extend/updated-garb-even-easier-access-to-the-google-analytics-api/#8820 Comment on Updated Garb: Even Easier Access to the Google Analytics API by Ryan http://feedproxy.google.com/~r/extendcomments/~3/d0xcY1_rgdQ/ http://www.viget.com/extend/updated-garb-even-easier-access-to-the-google-analytics-api/#8819 I’ve been having trouble getting the start_date and end_date params to work for me. After struggling with it a bit, I’ve basically just copied and pasted your code in, but I’m still getting one of these: “ArgumentError (wrong number of arguments (1 for 0)):”

Any ideas? Thanks!

]]> Ryan Tue, 02 Jun 2009 14:46:00 -0400 http://www.viget.com/extend/updated-garb-even-easier-access-to-the-google-analytics-api/#8819 Comment on Reusing Contexts in Shoulda with Context Macros by Morgan http://feedproxy.google.com/~r/extendcomments/~3/p3WTMW9hapM/ http://www.viget.com/extend/reusing-contexts-in-shoulda-with-context-macros/#8795 thanks.

]]> Morgan Wed, 27 May 2009 18:29:49 -0400 http://www.viget.com/extend/reusing-contexts-in-shoulda-with-context-macros/#8795 Comment on Rack Support in Rails: Why It Matters by Integrum http://feedproxy.google.com/~r/extendcomments/~3/seDuT_HHqtk/ http://www.viget.com/extend/rack-support-in-rails-why-it-matters/#8786 Excellent overview of the importance of Rack in rails.  I wish that we would look at Python and other implementations of frameworks more often and see where we can make improvements to rails.

]]> Integrum Mon, 25 May 2009 22:23:51 -0400 http://www.viget.com/extend/rack-support-in-rails-why-it-matters/#8786 Comment on Named Scope Caching by Lenart Rudel http://feedproxy.google.com/~r/extendcomments/~3/no5EhN56p2U/ http://www.viget.com/extend/named-scope-caching/#8784 I’ve been looking for similar solution. I wanted to use Rails.cache instead of class variable. I’ve noticed that caching named_scope does not work! Does anyone else have similar experience?

Here’s the code: http://pastie.org/488836

]]> Lenart Rudel Mon, 25 May 2009 04:15:28 -0400 http://www.viget.com/extend/named-scope-caching/#8784 Comment on Rack Support in Rails: Why It Matters by Neeraj http://feedproxy.google.com/~r/extendcomments/~3/27F1dS9I-38/ http://www.viget.com/extend/rack-support-in-rails-why-it-matters/#8762 Thank you for the write up. I attended you presentation at Railsconf and it was well delivered.  Since I returned from the conference I have been playing with Rack a little bit and it is such an amazing too.

Thanks for advocating and promoting the usage of Rack.

]]> Neeraj Tue, 19 May 2009 14:50:55 -0400 http://www.viget.com/extend/rack-support-in-rails-why-it-matters/#8762 Comment on Introducing Garb: Access the Google Analytics Data Export API with Ruby by Charlie Plexus http://feedproxy.google.com/~r/extendcomments/~3/yy44j3uX0I0/ http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8754 Thanks Tony,
Apparently my problem is specific to Mac OS X 10.5.6—I had to remove libxml2 and recompile it from source (v. 2.7.2), then re-install the libxml-ruby gem. That fixed it. Just putting it up here in case anyone else has the same problem.

Thanks for writing the gem, I’ve been messing around with it over the weekend and trying to familiarize myself with the Analytics API. Great job!

]]> Charlie Plexus Mon, 18 May 2009 09:34:57 -0400 http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8754 Comment on Introducing Garb: Access the Google Analytics Data Export API with Ruby by Tony Pitale http://feedproxy.google.com/~r/extendcomments/~3/YYgxhNTgZv4/ http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8741 Libxml-ruby built without issue? My first guess is that the libxml C library is not installed. This would generally cause libxml-ruby gem to fail. My first action would be attempt to reinstall both of these components.

]]> Tony Pitale Thu, 14 May 2009 16:34:59 -0400 http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8741 Comment on Introducing Garb: Access the Google Analytics Data Export API with Ruby by Charlie Plexus http://feedproxy.google.com/~r/extendcomments/~3/vr9glfkbgyo/ http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8740 Has anything changed with the gem itself recently? I installed the gem but when I try to require it or use it I get the following error:
dyld: NSLinkModule() error
dyld: Symbol not found: _htmlNewParserCtxt
Referenced from: /usr/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.9.8/lib/libxml_ruby.bundle
Expected in: flat namespace

Any suggestions?

]]> Charlie Plexus Thu, 14 May 2009 16:15:17 -0400 http://www.viget.com/extend/introducing-garb-access-the-google-analytics-data-export-api-with-ruby/#8740 Comment on JSConf 2009 Recap: Javascript at the Edge by Neeraj http://feedproxy.google.com/~r/extendcomments/~3/B0Nob6Saido/ http://www.viget.com/extend/jsconf-2009-recap/#8711 Thanks for the nice recap.

]]> Neeraj Tue, 12 May 2009 10:45:08 -0400 http://www.viget.com/extend/jsconf-2009-recap/#8711 Comment on Backup your Database in Git by sanatate http://feedproxy.google.com/~r/extendcomments/~3/74_lWjgYuBk/ http://www.viget.com/extend/backup-your-database-in-git/#8702 Nice website! :)

]]> sanatate Mon, 11 May 2009 17:26:24 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8702 Comment on Backup your Database in Git by Strony_Internetowe_Szczecin http://feedproxy.google.com/~r/extendcomments/~3/_oeLqlNpcWQ/ http://www.viget.com/extend/backup-your-database-in-git/#8690 By the way, CVS and Subversion also only store deltas. I dont know that Git invent that.

]]> Strony_Internetowe_Szczecin Sun, 10 May 2009 15:17:38 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8690 Comment on Backup your Database in Git by Ryan Findley http://feedproxy.google.com/~r/extendcomments/~3/r347J-_ZtM0/ http://www.viget.com/extend/backup-your-database-in-git/#8688 Oops, sorry for the misinfo. Thanks Jonathan!

]]> Ryan Findley Sat, 09 May 2009 18:41:00 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8688 Comment on Backup your Database in Git by Tom Anderson http://feedproxy.google.com/~r/extendcomments/~3/ksXtBtyj1CM/ http://www.viget.com/extend/backup-your-database-in-git/#8687 To address the issue pointed out by @Dan Grossman, you might want to prune old revisions out of your branch.  For example, this command should remove the commit from 7 days ago:

git rebase --onto master~8 master~7

]]> Tom Anderson Sat, 09 May 2009 15:58:42 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8687 Comment on Backup your Database in Git by Jonathan Wright http://feedproxy.google.com/~r/extendcomments/~3/9yjq128y8xQ/ http://www.viget.com/extend/backup-your-database-in-git/#8684 @Ryan Findley: The simple Git repository format does store it’s blobs in gzipped files, but the packed files are more sophisticated. David mentioned running git gc every day or so. Git gc packs up this simple gzip blob format into the deltified pack format where the real savings are made.

]]> Jonathan Wright Sat, 09 May 2009 10:45:12 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8684 Comment on Backup your Database in Git by Ryan Findley http://feedproxy.google.com/~r/extendcomments/~3/fUP_in4Pf3E/ http://www.viget.com/extend/backup-your-database-in-git/#8683 Careful guys, Git doesn’t store diffs / deltas, it stores the whole file bzipped (this is a blob). See here for more info: http://eagain.net/articles/git-for-computer-scientists/
Blobs are stored in Git’s internal DB using the SHA-1 hash of the blob, and can be inspected using “git show”. I highly recommend the Git Internals PDF Peepcode.

]]> Ryan Findley Sat, 09 May 2009 10:38:18 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8683 Comment on Backup your Database in Git by Alexandre Dulaunoy http://feedproxy.google.com/~r/extendcomments/~3/IJosbodYMtw/ http://www.viget.com/extend/backup-your-database-in-git/#8682 I like the idea. A small but important note : Git is very good when you have a collection of content, git is not tracking files or metadata (http://git.or.cz/gitwiki/ContentLimitations). If you want an optimal approach for storing database,IMHO it’s better to dump each table in a file. Like that, if you have only updates on a specific table, git is only storing/updating the updated tables.  But only the mysqldump need to be updated to make a dump per table of each database.

]]> Alexandre Dulaunoy Sat, 09 May 2009 08:55:22 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8682 Comment on Backup your Database in Git by Jonathan Wright http://feedproxy.google.com/~r/extendcomments/~3/pj0g0JDo9WM/ http://www.viget.com/extend/backup-your-database-in-git/#8678 I stumbled upon this approach a while ago, and have been storing backups of a little database for six months or so. It works wonderfully (for this database).

The system was setup to scrape a website, store some data and trace a few things. Initially the git repo was setup to store the daily scrapes. I was worried that a month or two down the road I’d discover a bug in the data extraction from the pages, and have lost a few months of data. By keeping the original scrapes, I could rerun the analysis after fixing problems. It quickly became obvious that storing a dump of the database didn’t add to the size much.

The cool thing is six months of history is stored in less size than the single uncompressed database dump and the last scrape. That’s just cool.

As Dan Grossman points out, this solution isn’t for everyone. The repo size grows over time, rather than being fixed size. For databases that are big, or have large frequent changes, a growing repo size isn’t acceptable. However, for small, yet important, databases where the compressed/delta’ed size grows a small amount each day, it’s a wonderful solution.

]]> Jonathan Wright Sat, 09 May 2009 03:44:57 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8678 Comment on Backup your Database in Git by David Eisinger http://feedproxy.google.com/~r/extendcomments/~3/vd3h9RTHwak/ http://www.viget.com/extend/backup-your-database-in-git/#8672 @Matt Graham: for the databases I work with, which have auto-increment primary keys on all tables, mysqldump outputs the rows in a predictable order.

]]> David Eisinger Fri, 08 May 2009 22:38:16 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8672 Comment on Backup your Database in Git by Dan Grossman http://feedproxy.google.com/~r/extendcomments/~3/aitqM2pozH4/ http://www.viget.com/extend/backup-your-database-in-git/#8671 By the way, CVS and Subversion also only store deltas. Git didn’t invent that.

]]> Dan Grossman Fri, 08 May 2009 22:26:43 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8671 Comment on Backup your Database in Git by Dan Grossman http://feedproxy.google.com/~r/extendcomments/~3/ZyzCiyD4Yd4/ http://www.viget.com/extend/backup-your-database-in-git/#8670 1GB a day in new, changed or deleted rows is a 1GB delta from the previous day’s dump. The repository grows by 30GB a month, where rotating backups are constant size.

]]> Dan Grossman Fri, 08 May 2009 22:22:32 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8670 Comment on Backup your Database in Git by Preston Marshall http://feedproxy.google.com/~r/extendcomments/~3/7AEsH8q6a2g/ http://www.viget.com/extend/backup-your-database-in-git/#8669 @Dan Grossman:
This would definitely be the case with SVN, but Git’s repositories only track deltas from commit to commit.  The git-gc command can also be run to collect random garbage in the repository.  Your experience would most definitely be different with Git than with SVN, or another VCS like CVS.

]]> Preston Marshall Fri, 08 May 2009 21:59:54 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8669 Comment on Backup your Database in Git by Dan Grossman http://feedproxy.google.com/~r/extendcomments/~3/6ZLBxq9qKTs/ http://www.viget.com/extend/backup-your-database-in-git/#8668 I tried this some years ago, and it was a horrible solution. The repository grew much faster than the database, since the repository tracks every row that ever was, while in reality, some data gets deleted each day.

Keeping a rotating daily, weekly and monthly backup would’ve used a constant couple gigs of space, while keeping a single backup on a code repository would grow by gigs each week.

]]> Dan Grossman Fri, 08 May 2009 20:50:29 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8668 Comment on Backup your Database in Git by Joaquin Bravo Contreras http://feedproxy.google.com/~r/extendcomments/~3/B7xqBufr1UA/ http://www.viget.com/extend/backup-your-database-in-git/#8667 Have you tried using the --tab option of mysqldump. Its output is much clearer and makes for very nice change history when using git or any other version control software.

Ask Brian Aker =P http://krow.livejournal.com/593424.html

]]> Joaquin Bravo Contreras Fri, 08 May 2009 19:38:47 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8667 Comment on Backup your Database in Git by Matt Graham http://feedproxy.google.com/~r/extendcomments/~3/R8tP17VHEVI/ http://www.viget.com/extend/backup-your-database-in-git/#8666 Does MySQL always dump rows in stable order or does it move them around from day to day?

]]> Matt Graham Fri, 08 May 2009 18:11:51 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8666 Comment on Backup your Database in Git by David Eisinger http://feedproxy.google.com/~r/extendcomments/~3/9fwVYS3sIGk/ http://www.viget.com/extend/backup-your-database-in-git/#8664 @Ren & Brandon: thanks, guys!

@Tony: good point – this technique is by no means specific to MySQL and git, although you’d miss out the flexible push/pull nature of a DVCS doing this with something like SVN.

]]> David Eisinger Fri, 08 May 2009 16:43:33 -0400 http://www.viget.com/extend/backup-your-database-in-git/#8664