PayPlay.FM development.log

metal and activerecord

noreply@blogger.com (Evgeniy K) — Tue, 24 Mar 2009 14:56:00 +0000

I've tried to move some small actions to Metal to make it faster but it actually became even slower.

Benchmarking localhost (be patient).....done

Server Software: Mongrel
Server Hostname: localhost
Concurrency Level: 1
Complete requests: 25
Failed requests: 0
Write errors: 0
Requests per second: 0.83 [#/sec] (mean)

Percentage of the requests served within a certain time (ms)
50% 5
66% 6
75% 45
80% 5007
90% 5008
95% 5009
98% 5010
99% 5010
100% 5010 (longest request)

At first I thought something was wrong with my code (as it usually happens) but numbers show that metal became equally slow after first several super fast calls. Commenting parts of my code made clear that it has something with AR because code without find() worked fine.

The answer told by Google was to call

ActiveRecord::Base.clear_active_connections!

before returning resulting array from call.

Metal wins with ~240 req/s when ActionController was almost twice slower.

what is Rails now?

noreply@blogger.com (Evgeniy K) — Tue, 10 Feb 2009 15:06:00 +0000

I thought about this post long ago, but yesterday found great similar post by Josh Susser.

I started writing rails sites about four years ago. It was something like 0.12 version and I still have CHANGELOG file in RAILS_ROOT of my first working rails site. I don't mean I am a guru/old-school hacker- I'm not. I just want to say that I can see difference between Rails start and where it is now.

Rails was simple and it was its most promoted feature - you could watch screencast, read couple of brochures like "rails in 4 days" and that was all - you were able to begin working with rails. Even without seeing ruby before.

Remember picture with pack of java books against two rails manuals? Many things changed since then.

Now to start new rails app, lots of decisions need to be taken: choose template engine, choose testing suite, choose ORM, choose javascript library. Rails became a skyscraper where every stage can be different. With Rails 3.0, when merb is merged in, things will be even more complicated.

You can't specify application environment just by calling it Rails. From now you need to clarify what your Rails is (haml,mocha,datamapper,prototype) or (sequel,erb,rspec,jquery) or whatever else you want to use.

Yes, you can use default options. But you still need to know other options if you want to be competitive. I think this 'flexibility' might be scary for beginners - there are too many options, too many ways of doing similar things.

I don't want to say that freedom of choice is bad thing. I just think rails is too fragmentized now. Maybe it is growth problem and it will be solved in future with something like common interfaces or by abandoning some technologies.

mediatemple: mtr start

noreply@blogger.com (Evgeniy K) — Mon, 19 Jan 2009 12:33:00 +0000

If you see messages like this

"application started, but pid file was not found, check 'mtr status' "

when doing mtr start/restart, it could be caused by several reasons: first, you need to read RAILS_ROOT/log/*, if it does not help - check your config/environment.rb for initializing RAILS_GEM_VERSION - rails version on (mt) can be different from your dev server.

Next thing to check is whether your gs-container is capable of working with your app. When you try to run several apps on default free 64M Container (and that was my case), the earliest process will be killed and you'll get no messages but this default 'pid was not found'.

apache and %2F

noreply@blogger.com (Evgeniy K) — Fri, 28 Nov 2008 11:06:00 +0000

If you are running rails on passenger and sometimes get apache internal 404 errors, this can be because by default apache doesn't handle urlencoded chars like '%2F' (/). In this case it won't even get to rails and will only appear in access_log with 404 status.

Solution is pretty simple - just add AllowEncodedSlashes On to virtual host section.

Mailer Callbacks: hooks for ActionMailer

noreply@blogger.com (Evgeniy K) — Fri, 21 Nov 2008 11:10:00 +0000

I've just published tiny library that I wrote on weekend and gonna use on production.
It is an extension to ActionMailer that gives two useful callbacks for your mailer classes: before_deliver and after_deliver. "before_deliver" helps to cancel delivery under certain conditions, "after" filter is usually for saving/logging sent emails.

http://github.com/kelyar/mailer_callbacks

That was the first release but it is stable enough to work.

@page_title

noreply@blogger.com (Evgeniy K) — Tue, 18 Nov 2008 11:37:00 +0000

I don't think that populating @page_title in controller's action, that can be seen in almost every controller, is the 100% right thing. Why controller if it surely belongs to view?

capture helper looks much more reasonable.

passenger

noreply@blogger.com (Evgeniy K) — Tue, 09 Sep 2008 13:01:00 +0000

right now I'm in the middle of setting up new payplay server with completely new configuration.

For a long-long time PayPlay.FM was running under lighty + fcgi/mongrel_cluster.
It worked but had a lot of problems starting with lighty's poor mod_proxy (resulting in 100% CPU load just because it can't re-enable crushed proxy) and ending with huge memory eating. We had like 5 mongrels, 200 Mb each. Awful. If you add 1G for memcached, and 700Mb for mysqld, you'll find that you've got not so much memory as you expected.

So here is what I change now:

1. enterprise ruby authors claim it has 33% memory benefit which is great.
2. passenger aka mod_rails

Some notes for future:

1. if you can't see your assets (css,images, 403 forbidden) - just add "allow from all" to the end of your VirtualHost's Directory section.
2. remember that passenger will run under user that is owner of config/environment.rb (+ one root spawner)

StumbleUpon, StumbleVideo and now StumbleAudio?

noreply@blogger.com (Evgeniy K) — Fri, 18 Jul 2008 15:01:00 +0000

StumbleAudio launched beta of its social music discovery site this week heading into direct competition with Last.fm, Pandora and Jango. TechCrunch dubbed it "like a Pandora for indie music".

rails 2.1 problems

noreply@blogger.com (Evgeniy K) — Wed, 04 Jun 2008 11:47:00 +0000

be aware that there were a lot of changes in AR especially with associations and eager loading.
we had couple of hude broken sql both with :joins and :include.

UPD next thing is IP spoofing attack prevention that thinks yahoo crawler is hacker and raises 500 on every 'request.remote_ip' call. still not sure how to turn this thing off, rollbacked to 2.0.2

UPD2 there is something with sweepers where nil.controller_name is called

elite networks

noreply@blogger.com (Evgeniy K) — Mon, 26 May 2008 13:41:00 +0000

Launching beta of your startup, telling everyone about it and trying to get as much customers as possible - I truly believe it is no longer a good strategy.

For the last couple of years people became much more lazy. They are tired of all these fancy social networks, etc. They don't want to sign up just to check it out.

I am not trying to say that people won't register anymore. If idea is great you will succeed anyway. But it became much more complicated to run 'yet another social network'.

I think it's time to create sites not for everyone. To deny, not allow. Let them fight to get in. Denial is always intriguing and can mean different things. For example, for photosite it could be evil moderators that pass only high quality images (like 500px.com), or registration by invitation only.

missing flash

noreply@blogger.com (Evgeniy K) — Thu, 24 Apr 2008 14:34:00 +0000

I've just spent almost an hour trying to fix strange bug in signup procedure.
"flash" message that I sent from one action to another disappeared somewhere between requests.

The problem was in the multiple ajax requests that I completely forgot of)
flash.keep if request.xhr? fixes it

kubuntu update on Mar 13th

noreply@blogger.com (Evgeniy K) — Fri, 14 Mar 2008 14:00:00 +0000

I am pretty sure it is already a history, but here is how I restored my system after broken libc update.

1. boot with livecd.
2. mount /dev/sda2 /media/disk
3. wget http://launchpadlibrarian.net/11110565/libc6_2.7-5ubuntu2_i386.deb
4. sudo cp libc6_2.7-5ubuntu2_i386.deb /media/disk/tmp
5. sudo chroot /media/disk dpkg -i /tmp/libc6_2.7-5ubuntu2_i386.deb
6. reboot

git

noreply@blogger.com (Evgeniy K) — Tue, 05 Feb 2008 15:03:00 +0000

I have todo list called "if I ever had unlimited free time". Yesterday new item was added - moving payplayfm source from svn to git. I really like doing some programming at home where internet connection is very limited, and btw it has its own positive sides ;)
Thus distributed system with easy merging is definitely what I'd like to have. It's a pity I haven't found gui for my windows coworkers, but as Linus said you can simulate svn behavior while seating behind git system. Isn't it great?

Webmaster tools

noreply@blogger.com (Evgeniy K) — Fri, 07 Dec 2007 15:11:00 +0000

This month I found that MS corporation is not as dead as I thought before. Yes, its online presence is not that big, but at least they try to change it.

Why? Because this month I stumbled upon products.live.com, system like Froogle or Y! Shopping.
For the last couple of years, Yahoo and Microsoft both tried to copy Google's tools but with different success. I can't compare their strategies or decisions,the only thing I can compare is a quality of their tools.

While Yahoo! Product Submit sucks (reports with no details, broken categorization, strange format, one file data.txt,etc), his brother at Product Upload even in beta has some features even Google Products does not have. And it even supports google base txt format.

Today I found another clone from this trinity: Google Webmaster Tools and Yahoo Site Explorer, meet Live Search Webmaster Center. Almost same things - upload/ping sitemaps, see top search position, check robots.

several comments on rails2.0 preview and the things that appeared to be broken on payplay.fm after small test

noreply@blogger.com (Evgeniy K) — Thu, 04 Oct 2007 11:21:00 +0000

auto_complete that we use on search suggest page is now out of ActionPack.
acts_as_tree and all acts_as_* are also absent - thus if you see messages about missing methods - just install them with script/plugin.

AssertPackager had different signature for compute_public_path that's why image_tag didn't work.

memcached

noreply@blogger.com (Evgeniy K) — Wed, 13 Jun 2007 09:00:00 +0000

An interesting reading about Facebook internals - looks like they have 200 (!) machines with no hard drives that are dedicated memcached servers.

Another quote:

For Bloglines, we commonly have pages that will do 200 to 1000 object fetches from memcache. Getting to 6 million+ aggregate memcache requests per second is definitely feasible.

Reading about really big systems can be shocking sometimes =)

PS Facebook is built in PHP, C++, Perl, Python, Java, and even a little bit of ML—and it all works together

World's largest MP3 store launches

noreply@blogger.com (Evgeniy K) — Wed, 30 May 2007 14:06:00 +0000

PayPlay has just launched "the world's largest MP3 download store" -- a store selling more than 1.3 million indie music tracks, with a search engine that allows you to search for your fave artists and get similar indie artists in the catalog.

We managed to give MP3s before iTunes. Great.

read more

JS/CSS caching in Rails 2.0

noreply@blogger.com (Evgeniy K) — Thu, 24 May 2007 14:29:00 +0000

In order to remove extra HTTP connections because of small files, we use Asset Packager plugin. It makes one big JS or CSS file from all your assets making it faster to load for clients.
Bad thing is that I had to reset all HTML cache every time after deployment because svn revision is added to name of base.js. I bet it can be easily changed, but I had no time to check.

I've just found that this behavior will be standard in Rails 2.0 - if you are in production mode just add :cache=>true to javascript_include_tag. That's cool.
Plugins are great but it is always better to have it out of the box.

feeds VS emails

noreply@blogger.com (Evgeniy K) — Thu, 17 May 2007 09:21:00 +0000

Last week I "burnt" some our feeds on feedburner.com
Great service that can give you some nice stats about your feeds for free. In two days we had about 40 subscribers for one of most interesting RSS - Customer Reviews

They have thing that instantly captured my attention - almost everything is a feed. For example, they don't send you emails about strange site behavior - you should subscribe to your FeedMedic RSS.

What I thought about was a way of displaying system logs. When you have big system with a lot of users, many actions are being happened: user registered, someone put money on his account, someone requested some service, etc. All these actions are logged of course - directly like payments or indirectly like user signup.
Feed of these events sometimes can be more useful that statistics.

And error feeds also have their advantages - you don't need to rely on mail server, you don't need to maintain list of developers to send them errors. Just one (or several if you need) feed, maybe protected with password for security. Opera Mini supports feeds so anyone with cellphone can read it.

The bad thing is a limit of items in RSS. So if your system is overwhelmed by errors only latest events will be accessible. But in such case you'll get notified anyway =)

1'000'000 tracks and even more short news

noreply@blogger.com (Evgeniy K) — Sat, 05 May 2007 18:52:00 +0000

PayPlay.FM finally reached very remarkable milestone - it officially has 1'000'000 tracks in WMA format! We have another million in MP3 and it scares me to count all these terabytes =)

Another great news - PayPlay.FM is now "Certified Windows Media Player 11 Online Store". It means if you open Windows Media Player and go to music stores - you'll see red icon of payplay.fm )
I didnt believe we'd pass all tests but we did.
(I would would be even more happy if I had Windows ;)

Even more - we support Google Checkout now. I tried to find something already written on ruby but there were only shopping carts (which are 5minutes task to write) and huge modules for Drupal/OSCommerce on PHP. All I need was simple integration module and I did it myself. If you need something to begin with and that is small but works - you can find it here http://gcheckout.rubyforge.org/

PS. Thing I need to do when I had enough time - extend ActiveMerchant to add Google Checkout support.

rails 1.2

noreply@blogger.com (Evgeniy K) — Thu, 18 Jan 2007 16:24:00 +0000

congrats with new release! i've just installed new gem and i wonder if payplay.fm will run with no changes ;)

noreply@blogger.com (Evgeniy K) — Thu, 07 Dec 2006 08:26:00 +0000

Anti-DRM day

noreply@blogger.com (Evgeniy K) — Tue, 03 Oct 2006 09:50:00 +0000

As you probably know, payplay.fm helps independent artists selling their audio records. One of the ways to do this is using DRM (BTW, the thing I just did is called "googlebomb".hehe ) But if you think we are DRM fans - you are wrong. All we want is to make both artists and customers happy. DRM has its unpleasant moments and everyone knows it. That's why we'll try to provide new way of selling tracks - old good mp3. It will take some time to implement but now you know our plans.

So we celebrate Anti-DRM too,why not? You cat get any song on site for free if you register today (Oct 3rd).

Rails schema

noreply@blogger.com (Evgeniy K) — Fri, 29 Sep 2006 15:16:00 +0000

Just found unpleasant bug while doing rake db:schema:dump - it appears that RoR doesn't save table charsets in schema.rb. So when you do load, your tables will be vanilla latin1. Not really cool. Promised to fix in 1.2

Credit card validation

noreply@blogger.com (Evgeniy K) — Thu, 21 Sep 2006 16:02:00 +0000

Hot news: from now we support AmEx and Discover.

I had to write some JS validations for AmEx cause they have 4 digits in CVV and while writing code realized that I wrote smth as boring as this simple JS like 3 years ago. This is so 90s! Definitely not web2.0 style ;)
Pephaps if I weren't so lazy, I could write ajax-based validations. hehe. It reminds me another crazy guy who did "web2.0 redirect function" using ajax ;) Sometimes boredom makes me do crazy things like this too =)