Lifescaler

MySQL performance tunning thoughts

Bogdan — Fri, 24 Sep 2010 11:39:22 +0000

There are a lot of posts and discussions on the matter, but here are some tips:

- replacing distinct with a group by on an unique column can/would speed up the query quite a lot

- there is a variable called query_cache_min_res_unit. Use: show status like ‘qc%’; in the mysql console and especially check the Qcache_lowmem_prunes. If there are a lot of them, it means that the memory gets fragmented, as min_res_units are too big. Gives quite a big performance penalty, especially under a lot of concurrent requests and a cache invalidation

- some order by queries might be improved by using force index(…) with the proper index. Sometimes, it clears the filesort that gets used by default.

Writing dynamic fixtures in Symfony

Bogdan — Wed, 02 Jun 2010 10:35:56 +0000

Whenever writing dynamic fixtures in Symfony, be careful about putting new line characters, after each property, but not after the main object declaration. Second, make sure you don’t start with any whitespace, and that the entire yml file does not contain unnecessary new lines or spaces. To debug, simply enable query logging in your database, and you’ll see that, when something is wrong, parts of the query will be mushed-up. Simple example (just look where “\n” is and where it isn’t):

(...)
Read the rest of Writing dynamic fixtures in Symfony (25 words)

Non linear management – fostering creativity / design driven development

Bogdan — Mon, 01 Feb 2010 18:01:10 +0000

Please read the first post on the subject of non linear management before continuing.

Level 2 of this journey starts with learning how to setup an environment that would catalyze creativity. In other words, we’re going to talk about design driven development and most importantly, how to actually implement it in a corporate environment.

There has been a lot of hype these past days about the launch of iPad and Apple’s decisions on this product. Whether or not this will be a successful device, it remains to be seen. However, the whole process of getting it to the market is really appealing. They are and have always been trend setters, and that is the basic, most important feature of design driven development. The idea behind it is very simple. Don’t use the old way, of market driven development, in which each new product or feature is a direct response of market needs. Instead, get a team of experts, create a community of requirements gathering – whether a closed or open one, whether a linear of hierarchical one, and use that to TELL your future customers what they want, give them what YOU think they need. It’s like going to a hotel and finding a free bottle of water on the table each day. You would never think to ask for/wish something like this, but if it’s there, your reaction towards that hotel is definitely more positive. Thus, you are in the avant-garde of creation, of satisfying your customers, and last but not least, of CREATING NEW NEEDS.

(...)
Read the rest of Non linear management – fostering creativity / design driven development (755 words)

Nonlinear management: creative thinking

Bogdan — Thu, 21 Jan 2010 18:14:33 +0000

It’s times like these that should make people realize the importance of adaptability. But as I experience the stubbornness in keeping a dissipated conservative view on problems by most managers, I’m hoping to share some more light on what the vanguard of management is.

Firs topic, creative thinking. This concept has been around for a while, but has yet to be adopted on large scale. One could start wondering why is that? - why would people not see that getting the knowledge on how to come up with great ideas is equally, if not more important than building a business around them? Creative thinking is a bit of an abstract term. In business in general, in management in particular, it refers to the set of skills required to gain a holistic view of a problem. To do that, one must train his mind on several levels. Each level is hierarchically more important and as you reach the top, you are able to see at 360 degrees. But unlike in a forest, where you have to get to the top of the highest tree to get an overview of your situation, creative thinking is more of a pyramid, in which as you construct towards the top, you gain height and more overview based on what you’ve constructed underneath.

(...)
Read the rest of Nonlinear management: creative thinking (932 words)

When Doctrine leftJoin fails

Bogdan — Fri, 02 Oct 2009 12:17:20 +0000

I’ve stumbled across a weired problem in Symfony, using Doctrine. Whenever I wanted to use a leftJoin, stuff would fail. So, here are some tips to help you pass this problems:

1. Check that the leftJoin is issued like $q->(a.RelationshipName r);
where “a” is the root alias of the query, RelationshipName is the NAME of the relationship defined in the yaml file of the database.

2. Make sure you use the actual relationship name from the yaml DB schema, and not the class name or something else. For instance, let’s say you have 2 classes: MysiteArticle and MysiteCategory. To fetch articles, you would build your query like: Doctrine_Query::create()->from(’MysiteArticle a’);
But, to fetch articles together with the category, you would do something like:
Doctrine_Query::create()->from(’MysiteArticle a’)->leftJoin(’a.Category c’); if your relationship name is “Category”, in the schema.yml, or Doctrine_Query::create()->from(’MysiteArticle a’)->leftJoin(’a.MysiteCategory c’); if your relationship name is MysiteCategory.

Getting the cursor position inside a text-field and smart ways to build form validators

Bogdan — Thu, 18 Dec 2008 09:24:51 +0000

I have always wanted a nice clean way to build form validators based on any kind of rule i would choose, however most of the times I stumbled upon a simple problem: How to get the cursor position inside a text-field? It’s easy in Mozilla, but what about IE?
The challenge was weird, but I finally got a way around, starting an idea here.
What I’ve done is I used Prototype to extend the Form Element Methods and simply add a version of selectionStart and selectionEnd for IE. After that, I just put them to work in a nice input wrapper class.

(...)
Read the rest of Getting the cursor position inside a text-field and smart ways to build form validators (463 words)

Javascript regular expressions quick guide

Bogdan — Thu, 18 Dec 2008 09:11:53 +0000

Quick quide to regular expressions in JavaScript

1. The form of an expression:
var a = /dog/gim
//dog = the expression to match, g = search all matches, i = case insensitive, m=multiple line
The cool thing is that /dog/ is an object, so you can, for example, do something like /dog/.test(something)

(...)
Read the rest of Javascript regular expressions quick guide (504 words)

Database sharding unraveled - part IV

Bogdan — Thu, 11 Dec 2008 15:25:31 +0000

Continuing the series about Database Sharding, I’m going to to talk about the software/hardware architecture. This post started from an excellent read, MySQL Database Scale-out and Replication for High Growth Businesses.

The first order of business is MySQL replication. Replication is needed to offer redundancy and to distribute even further the load on the system. In a typicall shard environment, the database is split among multiple servers, with data being unique to each server. If one of the servers goes down, all that data will become unavailable, and even though the system will continue working, parts of some scenarios will fail. This is where replication comes to stage.

(...)
Read the rest of Database sharding unraveled - part IV (739 words)

Smart trick to check some bottlenecks in your DB

Bogdan — Tue, 07 Oct 2008 09:20:52 +0000

There’s a nice tool called iostat to check the HDD related info, like number of reads/writes, amount of data processed, etc. It’s a must have for any good sysadmin, as it allows you to identify some of the bottlenecks in the DB.

Together with the vmstat tool - allows a user to see statistics for the virtual memory usage - form a powerful duo to use, especially when your DB is running very slow, but the processors are not fully used.

The tools have enough explanations on the man pages.

So, the only thing remaining is to start them up:

Open 2 terminal windows. The first one would run something like iostat -dx 10 (will display the device extended report, refreshed every 10 seconds - you can increase/decrease this number to suite your needs - too small is not very good, as it’s better to have stats over a longer period). The second one should run vmstat 10.

Last but not least, to get them you need to install the sysstat package (vmstat is in the procps package, installed by default). For ubuntu, type: sudo apt-get install sysstat.

Database sharding unraveled - part III

Bogdan — Mon, 16 Jun 2008 16:56:24 +0000

Before continuing, please read the first parts of the database sharding adventure:
Database sharding unraveled - part I
Database sharding unraveled - part II

Chapter 1. The small guys

Before really diving into high scalability principles, I want to take a moment to talk about why database sharding has an important role even in small startups or medium sized web-sites (5 - 30k unique visitors/day).

It is equally important and benefic for a smaller web business to prepare itself from the beginning to tackle large amounts of users cheap. If it’s not obvious enough, think about what happens to a web-page that gets some plain old Digg attention. The server quickly collapses and the user experience immediately turns from positive to mega negative.
As I’ve explained before, the whole purpose of sharding is to be able to use an unlimited number of cheap machines topped by an open-source database. As experience taught me, the web server will rarely die. Instead, the DB server will choke easily when having to deal with many simultaneous connections.
The database doesn’t even have to be very big.

(...)
Read the rest of Database sharding unraveled - part III (717 words)