There are two parts to speeding up JavaScript on your website. The first addresses how quickly your page loads, and the second how well it responds to the user. There are several things you can do to improve the former, while the latter is mostly at the mercy of the web browser’s Javascript engine.

When I set up my new systems, I set up centralized logging via rSyslog. I chose rSyslog because It allowed me to use standard Syslog on other systems, so I didn’t have to install anything special on my other hosts. It also allowed me to log into MySQL. Unfortunately, I didn’t do the smart thing and set up MySQL partitioning, so that data got unwieldy. I also had no reliable front end or analyzer that could take advantage of MySQL, so I did away with that component.

I do administer Red Hat machines professionally, but not personally. One thing I like about Red Hat machines is LogWatch. It has succinct log summaries with some analysis and is nice to read. It is analogous to the FreeBSD system and security log summaries, which I also like.

Still, LogWatch wasn’t enough for me. Enter OSSEC from TrendMicro. It has a lot of security functionality, but the one that’s most useful to me is the log analysis. It tells me about possible security issues, system errors, and miscellaneous issues. It runs on my centralized log host, so I only need to install and maintain it once. It can be a little noisy by default, but it often tells me important things that I should know. It has notified me of misconfiguration, Apache eating all available memory on hosts, processes that have run amok. All easy things to fix, and logged, but hard to find amongst the tons of data on my multiple hosts. Fortunately, no security issues have come up, but I’m sure I’ll get a good idea that something is wrong should there be one. Remember, your servers and firewalls are *always* under attack.

It’s good to have a tool that can help put my paranoid mind at ease a little. Some of the default alerts are a little… verbose. I love getting the “multiple attempts from a blacklisted host” when a server on an RBL does a dictionary attack against one of my spam-trap domains. That alert is fairly reasonable, only firing off after a few attempts.

Now, to get an IPS system working…

From VMware’s VCP site:

This industry-recognized certification requires completion of a VMware-authorized training course and hands-on experience with VMware technologies. Achieving the VCP certification confirms that you have the education needed to successfully install, deploy, scale and manage VMware vSphere environments, as well as the skills obtained by a minimum of six months experience with VMware infrastructure technologies.

ipHouse employees hold many VMware certifications:

• 3 have VCP certificates: Mike, Nick, and Ben
• 2 have VMware Technical Sales Professional certificates: Ben and Nick
• 7 have VMware Sales Professional certificates: Aileen, Alex, Andrew, Ben, Dan, Mike, and Nick

This means that ~60% of ipHouse employees are certified in VMware products with 3 employees holding multiple certifications.

What does this mean to you?

ipHouse employees can help design a virtualization solution for your business whether it is a single server web site to a multi-server Microsoft Remote Desktop Services solution and everything in between. Look at our vmForge offerings and give us a call at 612-337-6320 and let us help you.

In the middle of January I helped someone move away from colocation into our vmForge VDC service offering.

Think in the cloud but without the variable monthly billing or non-persistent storage.

“virtual data center” has a nice ring to it doesn’t it?

My customer had 6 aging servers in a colocation rack in our Minneapolis, MN data center.

At ipHouse, a colocation rack with 10A (8A usable) of power is $300 per month, or $600 for a 20A (16A usable) circuit (1 year term). This customer had the larger sized circuit (old servers use more power it seems). There is also the added fee for the Internet bandwidth.

We (ipHouse) were able to move this customer into a virtual data center for far less than the 1/2 rack was per month and the customer has room to grow and scale. Today, they are purchasing more storage than they need so they can resize their servers during deployment and add a 7th in the near future. This would have been more expensive and take more time in the physical server realm as adding more storage to a server is usually more involved than just plugging in that USB cable from the portal hard disk. Also, servers like a little more performance than what a normal USB device can offer.

From 6 physical servers of different vintages with a myriad of storage configurations to a vmForge VDC with 16 GiB RAM, 8 Ghz CPU, 500 GiB of storage, and 250 GiB of transfer.

The cost? $450.00 per month. That’s over $150.00 per month savings for this customer because their Internet bandwidth had been in addition to their colocation fees.

What if they added an 8th server to their virtual data center? If it fits within their current resource allocation then there is no added cost for them. (that isn’t a complete truth as it really depends on the operating system choice as some have costs)

Over 12 months my customer will save at least $1,800.00.

Think about the following questions:

What are you paying for colocation?

Could a virtual data center save you money every month vs physical data center services?

If you answer anything but ‘no‘ to the second question then call or email us right away. Let our knowledge help your business thrive.

Changing your disk scheduler on a Linux virtual machine to increase performance.

First some background of what we do with storage at ipHouse in our VMware environments.

We really like NFS. Architecturally it’s simpler than block based storage; you just need a good local area network and a storage system that can export a file based protocol. There’s no need for specialized hardware or intelligent host bus adapters, just let the storage array handle the storage. Virtualization lends itself to file based storage quite well. VMDKs are just files after all. I kind of snickered when VMware first came out with their VAAI storage extensions. It seemed, to me, like they were enhancing block-level storage devices to do a lot of what NAS based storage already does.

While I was taking my VCP4 class my colleges, most of whom were from big companies, snickered when I mentioned that our storage was on a NAS. A “filer” for them was a place for document sharing and storage. There was “no way” it would ever be fast enough, or good enough to backend their virtualized infrastructure. I’ve seen that notion fade more and more as ZFS has opened the doors for storage startups; and the big players are fighting back with their own specialized NAS devices. There are some really cool ideas floating around: NAS devices that are scale-out, that are optimized for virtualization, and that can do in-line deduplication of data.

That being said…

I have learned that there are some OS level tweaks that can enhance performance on virtual machines. Most x86 operating systems seem to be optimized for single disks, or internal RAID setups. Understandable as that has traditionally been the bulk of their install base. This means that the OS can manage disk queuing better that the dumb RAID card, or the dumber hard drive. CFQ, the default disk scheduler as of kernel 2.6.18 does this. As I understand it CFQ breaks synchronous read/write requests into queues, and assigns timeslices to each queue, weighted by IO priority. The effect is that higher priority processes get longer queues which keeps IO requests from the same process close together. Great idea when the OS has direct access and is managing the storage. Not so great when the storage is handled remotely; the array on the other side is doing the scheduling. All of that optimization is ostensibly ignored. So for a virtual machine it’s better to switch to a simpler algorithm and let the storage array handle the write queuing.

From my reading (and testing) It’s better to switch to the noop scheduler. Noop simply shoves all requests into a first-in-first-out (FIFO) queue and can merge requests. It is simple, fast, and is great for flash storage (no mechanical latency) or for situations where optimization is handled by another device. Like a NAS! Perfect for virtualization!

I discovered this after getting a snippet of a shell script to try from Mike (who got it from a potential vendor that is a big storage geek). This wasn’t new information as Mike had mentioned this almost 18 months ago in passing but neither he nor myself ever tested it. After giving me the info, again, he suggested that I “test this out, and let me know if it works.”.

I’m still testing it, so caveat emptor, but I thought I’d share it with you.

***WARNING DO NOT DO THIS ON A VM WITH SNAPSHOTS***

#!/bin/sh

grep '' /sys/block/sd*/queue/scheduler
for d in /sys/block/sd*; do
echo noop > $d/queue/scheduler
done
grep '' /sys/block/sd*/queue/scheduler

This switches the scheduler from cfq to noop on all “SCSI” disks in the virtual machine.

He also added the following tweak to increase the read-ahead from 256 sectors to 1000 sectors, which caches more disk data for faster read times, after printing what the OS has mounted.

#!/bin/sh

mount
blockdev --getra /dev/sd?
blockdev --setra 10000 /dev/sd?
blockdev --getra /dev/sd?

Again, I’m still testing this on my personal stuff, but, qualitatively, things feel a lot faster. If anything, I haven’t crashed my Linux systems.

Anyways, I hope that helps!

MinneDemo happens two to three times a year and I think this has to be one of the most exciting MinneStar events (although, I have yet to attend MinneBBQ).

As you enter the area of presentations you feel a great surge of energy from the presenters and the crowd. There is an excellent mix of different age groups that attend. The first hour is for networking, eating, and of course – drinking. Once you have had your fill of food and conversation, you head into a medium-sized auditorium to take your seat, wait, and feel the anticipation build.

All of the presenters have a spark that ignites the entire audience sucking them into the presentation.

What are people presenting at MinneDemo?

Presenters show off real, working technology products created locally in Minnesota. One not only feels energy but a sense of pride in our great state!

I am just going to touch on two of the different presenters that I enjoyed wholeheartedly but before I do that I would like to congratulate Code 42 software, the company behind CrashPlan for landing a $52.5M growth capital investment round.

In case you want to catch the entire show there is video footage of the event and more great links on the video page.

Brahmageddon is an iOS game that is a lot like wack-a-mole but, with a spice of Hindu mythology. The company that makes the game, Bust Out Solutions, is based in Minneapolis, Minnesota. It was quite entertaining to watch the game being played and hearing the Hindu music in the background. The graphics are magical and I would love to meet the artist who created the Hindu demons. The game takes on the same effect that Angry Birds.What I mean by this is it is a fast paced game that is easy to understand with a pinch of challenge to keep the players enthralled. It is quite simple and an easy way to acquire entertainment on the go. I could easily see myself playing this game on the bus ride home from work. The audience for the game seems like it would cover all demographics. It was also encouraging that the creators still find the game quite fun to play.

Red Stamp’s presentation takes the quirky, stylish greeting cards that you would buy at say a Paper Source and puts it into a mobile correspondence. It was a pleasure to watch a company that moved with the times and did not just stay in the paper world. I think Red Stamp represents a beautiful evolution in the greeting card. Red Stamp is really catering to the crowd that just does not have time to stop by a store or order a card and have it shipped their way. I think this crowd of busy people is growing and Red Stamp is growing right along with them. It is refreshing to see a company that instead of saying “no we won’t do that” said “hey, we are up to the challenge and can make this happen”.

If you would like a kick in the butt to remember what dreams you used to have and what it is like to strive to make those dreams come true, you should check out MinneDemo.

And again, if you would like to see all the presentations, check out the video footage.

First, its become common practice for many pages to have at least three stylesheets; one for all media, one for screen-specific instructions, and another for print. Other stylesheets might be imported for specific pages or resources, such as a lightbox library or HTML form suite. Instead of putting each stylesheet existing in a separate file which must be loaded and parsed separately, consider consolidating them as much as possible by using @media directives within the CSS file. For example,


// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
// combined.css
// these styles apply everywhere, all the time

body { color: black; }
h1 { font-size: 150%; font-weight: bold; }

// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
// these styles only apply on-screen

@media screen {
body { background: url('background.jpg'); }
div.nav a:hover { font-weight: bold; }
}

// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
// these styles only apply in-print

@media print {
div.nav { display: none; }
p { text-align: justify; }
}

// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


Placing all your stylesheets in the <head> section of your web page is best practice, and also improves apparent performance since the web browser can load and parse the style information before it begins rendering the page body. Associating stylesheets via <link> tags instead of @import is better, since not all browsers handle @import directives promptly.

Also consider using Expires headers to instruct web clients to cache CSS files for a long time. In apache, this is easily achieved through the mod_expires instructions,


ExpiresActive On
ExpiresByType text/css "access plus 1 year"


If you need to retain the ability to change your CSS files more frequently, add a version number to the filename and update it as necessary. For example, combined-1.0.css

I look at many cloud providers and I see the opposite. Their services were designed for expedience instead of permanence. They make it hard and, at times, very expensive to actually keep data around. Usually you have to attach a “disk” (or “volume”) to any machine that has data you want to keep and you have to pay for that privilege. You also better have backups because you have no idea about the underlying storage or data retention policies.

Any data that you absolutely need could mean you’re paying two or three times what you’d expect in order to keep it.

To my hoarder eyes the cloud is one big data furnace. It’s a dangerous place for your information to stay.

Enterprise data storage is expensive. I’ve often joked that virtualization is a scheme to sell storage arrays. It’s a tricky game of performance, space, and redundancy. Disks fail, flash is expensive, you never have enough RAM or CPU. There are dozens of types of arrays for hundreds of applications, retention policies, regulations; it’s a mess! When you have a service that has hundreds of thousands of customers then it may make sense that you discourage persistent data. You want people to consume your resources, pay their bill, and move on. Expedience instead of permanence. I’ve often been asked: Why online storage is so expensive when hard drives are so cheap? Well, this is why.

We built the ipHouse vmForge product with the idea that a virtual data center (VDC) replaces co-located infrastructure. The storage is persistent from the get-go. Is it any wonder that Mike has been loath to call it a ‘cloud service’?

This means that there are severe implications for any storage array that we put in place. We have to make sure that anything we put in place not only performs well but also goes the distance. It’s still a very good idea to do backups, though they probably will not be nearly as large, as most customers just need to back up a few key files or the database dumps that happen regularly. (you are backing up your database, right?)

Well, that’s my opinion anyways. Now I’m going to go back home and work on my basement.

• Centralized management over several to many access-points.
• Unified access policies.
• Ease of deployment.
• Rogue AP scanning for PCI/DSS compliance.

The lesson I learned: Feature freeze is a good thing. Know when to stop fixing.

Now early in the project, I had a pretty good idea of what pieces needed to go together but I did not have a very good idea of how to get there. I put down a quick design and while I was doing that I started to see problems..

• Pieces did not fit together.
• Some things were missing.
• This was not going to work.

Time to start learning. I love learning.

So this is the good part. This is the fun part. This is where things go click. More on that in a moment.

I strive to make every day a boring day on my production servers. That doesn’t mean that they don’t do cool stuff. That doesn’t mean I don’t like my job or that I find it dull. I don’t. I just like to be beyond the point of being surprised, pleasantly or otherwise, when I am doing something for a client. That’s the goal.

Now technology moves too fast to be expert in everything. There will always be opportunities to learn something new, but my goal is to make the systems I run not scratch that itch for new learning. All that is to say that I have a craving for learning new stuff that is not and should not be filled by the day-to-day work I fit my learning projects around.

So there I am in the middle of doing a general something new. I have a general idea of how it goes, but there is a part missing or a process that I don’t know how to do. I know the next step, but not quite how to get there. Then I learn how, or I learn it is not going to work and I find a new way. My favorite learning is when I learn something new that brings two formerly unrelated pieces together in my mind and they fit together. That click is one of my favorite experiences in life.

After a while of this, things started to make a lot more sense. I had something that would mostly work. There were still some things to optimize, and some things to work out. More of the fun part. Here’s the problem: It gets addicting to learn stuff. If you’ve ever gone to Wikipedia and seen hours magically vanish, you know what I am talking about.

At some point though, it is time to stop fixing for a while. Freeze the design and commit to finish the thing, even though it is broken. Make version 1.0. There are a lot of things you can learn from seeing the finished system even with its flaws that you can’t see by looking at the parts.

I’m not saying leave it broken. Make version 2, and version 3, if you want but at some early point, freeze the specification, stop making changes, stop making fixes, go finish. Exercising the discipline to stop following new opportunities to improve my project, stop fixing, and stop learning (for a moment) meant the difference between having an imperfect but completed project and having a whole ton of good ideas.

So I stopped and did the drudge work. I finished. I made something imperfect. My reward? In addition to the problems I deliberately ignored, I see lots of little details that need fixing, which I would never have seen without finishing.

Off to version 2.0. I love learning.