rackAID Blog

rackAID clients don't have to worry about hosting.

2009-05-20T16:30:03Z

Recently rackAID sent out a survey to clients using our managed services. Though there is always room for improvement, the survey shows that we still maintain very high marks in the areas of customer satisfaction, staff expertise, and problem resolution. We thank you for your praises and will continue to work diligently to improve our services. What I was really happy to see were comments like these: "Freeing up my time so I don't have to make sure my server is secure, updated and always running smoothly." "Not having to have expert IT staff." "I am a web designer and I don't have the knowledge or expertise to handle issues when something goes wrong with a server. I can relax knowing you guys have everything under control and if anything goes wrong, it's not up to me to solve it." These comments reflect the key values that rackAID delivers. We will continue to deploy new products and services and refine old ones to assure we continue to deliver. We Check for Updates Every Hour! We asked "How often do you think rackAID checks for patches on your server?" Nobody answered this questions correctly. We use automated systems to monitor your server for updates. Should your server have a new patch available, we detect in under an hour. In most cases, those patches are applied in under 24 hours. Dozens of Patches Per Month We also asked how many patches do you think we apply to your server each month. This varies considerably form just a few patches to more than 100 when major updates roll out. On average, we estimate we apply between 20-50 patches per month on newer OS versions. This does not even count the patches for control panels. We have few issues updating servers because our experience has taught us which packages are problematic and which ones apply safely. We maintain lists of these troublesome updates and use special procedures to apply them when they arrive. This approach results in fewer post-upgrade issues. Security is Paramount Based on some survey questions and comments, security tops the list in service areas in which we need solutions. Over the next few months, watch for announcements regarding:

Email Security Solutions
Server Security Testing/Scanning
Web Application Firewalls

We have all of these in the pipeline. If anyone is interested in our new Email security services or server security testing services, please drop us a note in our help desk. We are in the soft launch phase of those services now.

How to Remove Google or Firefox Site May Harm You Warning

2009-04-15T16:19:20Z

"This site may harm your computer."

Google and Firefox both provide safe browsing features. These tools try to identify potentially harmful sites by working with groups like StopBadware. If you attempt to visit a site listed as harmful, Google and Firefox will display a warning message. In Google's search results, you will actually see something the "This site may harm your computer." Removing your site from the malware list, requires that you first fix your site security and then use Google's webmaster tools for a review of your listing.

IFRAME Exploits

Before we go over what the block looks like and how to remove that "This site may harm your computer." warning, I want to go over why your site may be listed. Over the past week, we have seen a rise in IFRAME based exploits. We are still gathering data, but the issue appears to be a trojan that hijacks any FTP accounts found on your system. The trojan is either piggybacking on open FTP connections or getting the passwords from the registry. Once this happens, the virus logs into your server and modifies index.*, home.* or default.* files with an IFRAME exploit. These worms link to sites like: goooogleadsence.biz, google-ana1yticz.com, hostyapics.net, and others. These sites in turn either redirect or have malicious code on them. As a result, your site will get flagged in Firefox or Google search results. In Firefox, you will see: In Google, you will see:

Remove the "This site may harm your computer." Warning"

To remove the warning, you must first clean up your site. The IFRAME exploit is just one of many possible items. You will need to dig through Google's pages to see why you are listed. In another case, we found a third party ad-network had an infected ad. The infected ad was triggering the block. Once you identify the source of the block, you will want to remove it, and if possible identify how the malicious code entered your site in the first place. Typically, we find compromised passwords, end-user desktops or insecure web applications to be the vector for the exploit. You need to identify the cause to prevent your site from becoming infected again. These web parasites can quickly wreck your site's reputation, so getting to the source of the issue is critical.

Google's Webmaster Tools

Once you have cleaned up your site, you need to have a webmaster tools account with Google. Once you have your account, you will need to add your web site. This requires you adding your site to their dashboard and then completing the site verification process. You verify your are the site owner by placing a page on your site. Google will provide you with the name of the page. Once you have verified your site. You can then view it. On the overview page, you should notice a note about malware being on your site: Click on the request a review link. You will typically get a response within 48 hours.

Summary

Following this procedure should get your removed from the block list. If you find that you still are blocked after these steps, then you may want to consider hiring someone to help you out. The key is to eliminate the method the attackers are using to modify your site. As we continue to get more feedback on the IFRAME issue, I will send some updates.

How to Remove your Email Server IP from Earthlink's Blacklist

2009-04-10T15:34:19Z

This is a second post in a series dealing with removing your IP from various email blacklists. In the first post, I covered how to remove your IP from Yahoo's blacklist. I recommend reading the first post as there is some details on there about how to proceed that are applicable to any blacklist removal process. For Earthlink, the process involves:

Emailing Earthlink's postmaster at a special address
Reading the response email
Cleaning up any blacklist issues if you have them
Submitting your IP for removal from Earthlink's blacklist

As mentioned with Yahoo!, this procedure will not help you if you do not fix the problem. If you've fixed the problem, then read on to learn how to get removed from Earthlink's blacklist.

Blocked by "blockedbyearthlink@abuse.earthlink.net"

If you get a bounce with the following:

Delivery to the following recipient failed permanently: [person]@earhtlink.com Technical details of permanent failure: PERM_FAILURE: SMTP Error (state 8): 550 550 Dynamic/zombied/spam IPs blocked. Write blockedbyearthlink@abuse.earthlink.net

The you have likely been blocked by Earthlink's spam filtering system. Over the years, Earthlink acquired many smaller ISPs. As a result, there are many domains that may be impacted by having your email server IP listed on Earthlink's list.

Call the Postmaster

Earthlink's postmaster information is a bit scattered. Right now, the best resource is a knowledgebase article, which will provide you with links to key policies and procedures for email. In most cases, you will want to email blockedbyearthlink@abuse.earthlink.net, you will get a reply within a couple of minutes with the following:

Thank you for writing blockedbyearthlink@abuse.earthlink.net. This is an automated response to the email you sent to this address.

PLEASE READ THE ENTIRE EMAIL TO RESOLVE YOUR ISSUE This email address is only for those issues where EarthLink appears to be blocking a mail server and mail is returned with the following error messages: "550 Dynamic/zombied/spam IPs blocked. Write blockedbyearthlink@abuse.earthlink.net". EarthLink may be blocking your mail server because the server is/was listed in a dynamic IP range, an open proxy, a compromised machine, or major source of spam. In order to help you with your issue we will need some information from you. If you are the administrator of this mail server, please follow the instructions below before emailing us back. If you do not know what IP address your server is sending with, please contact the mail administrator of your company or your ISP and have them follow the instructions below. If you are the administrator of the email server being blocked, please know you may be able to expedite unblocking by checking to see if your mail server is listed in the following lists and resolving before emailing us: The CBL http://cbl.abuseat.org/lookup.cgi This is a list of IPs that may be currently infected and sending spam unknowingly. Spamhaus http://www.spamhaus.org/sbl/index.lasso From their website you will be able to query your IP to see if it listed in any of their 3 lists that track dynamic IPs, zombied IPs, and IPs that are known to purposely send spam. EarthLink blocks IPs known to be dynamic, zombied, and purposely sending spam. If your mail server/gateway IP is not on these lists, is not dynamic, and is not an open relay/proxy/zombie, please reply back to blockedbyearthlink@abuse.earthlink.net with 'BLOCKED ' in the subject line with the mail server/gateway IP inserted. Blockedbyearthlink@abuse.earthlink.net does not block any mail sent to it; so, if possible, please email us from the server in question. All reports sent to this address, not using this format will receive this auto response. This process will allow us minimize the amount of time to resolve any blocking issues. Please note, any emails you send to us without the formatted subject lines are not kept. If you included important information in that first unblock request, please make sure that information is included in your subject-formatted response. Thank you for your cooperation, EarthLink Abuse

As they say, read the entire email and simply comply with their request. At rackAID, we use DNSStuff to do RBL lookups. If you are clear of the RBLs, then you can email Earthlink your server's IP address. Be sure to format your email exactly as they request or it will not work:

'BLOCKED ' in  the subject line with the mail server/gateway IP inserted.

Bulk Senders

I could not find any specific information regarding sending bulk email to Earthlink accounts. Yahoo! has a special form to fill out. The most information I could find was a policies page , which has a useful list of all of Earthlink's domains.

Review

To get out of Earthlink's blacklist, simply email your IP to the blockedbyearthlink@abuse.earthlink.net email address. Be sure that your system is clean before you do so. You certainly do not want to get listed again and lower your sender reputation.

TMP Directory Hardening Increasingly Ineffective

2009-04-10T14:54:27Z

Yesterday, I commented on how hardening host.conf file provides very little security. Today, I want to focus on another item often found on "server hardening" checklists: TMP directory hardening. While TMP directory hardening still has its place, I feel it has lost its effectiveness in today's threat landscape.

Before TMP Directory Hardening

After applying these changes to your /etc/fstab file, you will end up with something like this:

/dev/sda2   /tmp                    ext3  defaults 1 1

After TMP Directory Hardening

/dev/sda2   /tmp                    ext3  rw,nodev,nosuid,noexec 1 1

What's being done?

TMP directory hardening involves restricting activities on the /tmp and /var/tmp directories by use of mount options. The motivation for this on web hosting servers is to prevent attackers from executing code within the /tmp directory. Since /tmp is world writable, if an attack can gains access to the system, they can store and execute files from within /tmp. For filesystem hardening, the CIS benchmarks focus on three key restrictions: nodev - prevent devices from being created on the filesystem nosuid - ignores the suid bit on the filesystem noexec - do not execute files on this filesystem These restrictions should be used of various directories, such as /tmp, to restrict activity on the server. It is also recommend that you symlink /var/tmp to /tmp to prevent hardlinks from within /tmp to data contained in /var. cPanel has a script called securetmp that can be ran on boot to secure the tmp directory (but it does not apply a nodev restriction).

Analysis of TMP Hardening

While I still recommend this step when hardening a server, I think the usefulness of it has been lost. Attackers used to dump files into /tmp via a web application exploit. They would then try to execute these files. Switching to using noexec on /tmp directories prevented this type of attack. Unfortunately, there is an easy way around these restrictions. You can simple call your scripts via a perl, php or similar program instead of putting an executable in /tmp. For example, if you put a hello.pl script into /tmp and try to run it directly with the noexec option enabled, the script will fail. However, if you simply run it as perl /tmp/hello.pl it will work. More recently, most web application exploits have involved XSS exploits which directly runs code from within the script. As a result, the /tmp hardening procedures do little to protect you from these attacks.

Conclusion

While I still recommend doing /tmp hardening, I place little value in it. I've seen numerous incidents where /tmp was secured but attackers still ran code from the directory through various wrapper approaches. I've also seen situations where /tmp was hardened but /var/tmp and /dev/shm left open. The latter two directories are popular targets as well. So while this step certainly does not hurt, I find it does little to improve security on a web hosting sever. I often find it listed in the laundry list of items included in server hardening checklists. Currently, I recommend securing /tmp only when your server already has a dedicated /tmp partition. Given the low value of this technique in today's threat landscape, I do not feel it necessary to use workaround approaches, such as cPanel's securetmp script, to get /tmp hardened.

Host.conf Hardening of Little Value to Web Hosts

2009-04-09T20:14:35Z

If you've ever looked at Linux server management companies, you often find a laundry list of "security" items that they apply to your servers. Many of these items are nothing more than standard practices while others are simply popular items gleamed from forums. Many of these "tweaks" have no real testing behind them; they are often applied with no real information as to why they are done. Over the next few weeks, I will discuss some of these server hardening practices and try to determine which ones provide real benefit. This way, when you look at long list of server hardening items, you will know what is valuable or not. As I noted before, the number of security updates typically declines as a OS matures. Keeping your system standardize by not tweaking things unnecessarily improves long term server stability and security by making ongoing management more coherent.

Host.conf Hardening

A quick check revealed more than a dozen companies listing "host.conf hardening" as part of their security package. Let's review what's actually being done, and then discuss if there is any value in doing this. We will be using a standard CentOS5/RHEL 5 system for this comparison. For the "hardened" versions, I am using ones that I commonly see in various how-to's and forums.

Before Host.conf Hardening

The host.conf file resides in /etc/host.conf. This is it before hardening:

order hosts,bind

After Host.conf Hardening

order bind,hosts
multi on
nospoof on

What's Being Done?

A quick look at the man page for the host.conf file gives you a good idea of what this does: "The file /etc/host.conf contains configuration information specific to the resolver library." In other words, this file controls, in part, how your system looks up DNS information. The order directive tells the system in what order it should lookup information. By default, the system first checks the host file and then uses DNS information. In the hardened version, the system first checks DNS and then uses the hosts file. The 'multi on" directive simply allows the resolver, the system that translates hostnames into IP addresses, to return multiple listings if they are found in the hosts file. The "nospoof on" setting will compare the IP address returned by a hostname lookup to the hostname returned by an IP address lookup. If the forward and reverse lookups do not match, then a spoof warning is generated.

Analysis

The Center for Internet Security (CIS) publishes a 130 page plus document on best security practices for Red Hat Enterprise Linux. Not once in these pages does it mention hardening host.conf. One of the touted benefits of this tweak is to "Prevent IP Spoofing." This tweak does nothing to prevent IP spoofing. What is does do is provide additional checks when using weak authentication mechanisms based on the host name or address of the remote computer. These "r-commands" should not be used anyway. With SSH widely available, rlogin, rsh, rcp commands should not be used. Another benefit I've seen discussed says that it prevents someone from editing your hosts file and redirecting DNS. If someone can edit your hosts file, they have root, and they can do what they want anyway.

Conclusion

Host.conf hardening provides little value to the average server deployment used by SMB's and web hosting firms. As such, you can pretty much scratch it off the list of items in the "server hardening" laundry list. Some may say host.conf hardening does not hurt, so why not do it. I disagree. If the stock settings provide sufficient security for your needs, then do not change them. Keeping a system free of unneeded "tweaks" has inherit value to long term management.

Forwarded Emails May Cause Backscatter Spam Complaints

2009-04-03T19:50:50Z

I've previously written on how to stop email backscatter. In case you missed that post, email backscatter is when your server bounces and email to an unknown user. Since the reply-to fields can be spoofed, this allows spammers to bounce emails off of your server, thus getting their spam delivered. Instead of sending these non-delivery reports (NDRs), you can set your server to reject email to unknown user. While this may sound similar, rejects send a 500 series email error to the senders server. Rejects do not send emails. As a result, the backscatter problem is stopped. Email Forwards Cause Backscatter Recently, I investigated a server that had become listed in backscatter.orgs RBL. This way surprising since every domain on the server was set to reject email to unknown users. I also verified that emails to users at the servers hostname, eg. nosuchuser@host.domain.com, also did not bounce. So I was surprised to see the IP in backscatter.org's RBL, and since they fail to provide any evidence for why the IP was listed, I had to dig further. Digging into emails sent to the postmaster, I found something curious. Someone had emailed an account which was being forwarded to another account. However, this other account was no longer valid. As a result, the forwarded email was bouncing. Since recipient checks do not pass through to forwarded emails, this opened the door for backscatter. This account had a high volume of spam which is likely what got it nailed in backscatter.org. Unfortunately, I don't see an easy way to resolve this other than remove the forward. The plesk server would have to do a sender verify call-out to prevent this from happening and sender call-outs are often just as bad as backscatter. So if you find you've been nailed for backscatter and cannot find the cause. You may want to look at forwards. I've not tested it but I suspect this behavior is consistent on other email platforms. TIP On plesk, I recommend creating a site or domain alias that matches the hostname of the server. You can then create a root email account with abuse, postmaster and other aliases. Collecting these emails can be useful for diagnostic purposes.

How To Remove Conficker Worm

2009-03-30T19:51:57Z

The infamous Conficker worm may be up to something on April 1st. Security experts are still not clear about the intention of this virus. A recent discovery of a fingerprint has allowed investigators to develop some tools to help with automated scanning. Popular tools like Nessus should be updated to assure you have this signature in place. I am still looking for the Nmap procedure referenced in TheRegister article but have not found it. Conficker is thought to have infected millions of computers. Since the infection it has been dormant. The only instructions are for it to call in on April 1st for more instructions. Though this may turn out to be another Y2K, I suggest running full virus scans on all of your systems. Conficker uses a variety of techniques to evade antivirus software, so if you do find an infected systems, remove it from the network immediately. Removing Conficker/Downadup/Kido For your desktop, be sure to get the latest version of your antivirus software and conduct a full virus scan of your system. For servers, if you are running Linux, then do not worry this is a Windows virus. If for some reason you don't trust or don't have desktop AV software, here are a few tools to help: Bitdefender has provided a stand alone removal tool conficker removal tool. F-Secure has as free online service that can help. At rackAID, we use AVG's anti-virus since they provide both Linux and Windows versions. Their network licensing fees and two-year subscriptions significantly lower costs over their competitors. Of these tools, I like Bitdefenders standalone tool as it is target specific and easy to use. Searchers Beware If you are searching for information on Conficker, be very careful. I have noticed a large number of spam links -- some containing other malware. If you want information, go to your AV vendor's web site.

10 Immutable Laws of Security Administration Revisited

2009-03-16T18:26:50Z

Over eight years ago, Scot Culp of Microsoft, published two white papers that get tossed around in security circles over and over. The 10 Immutable Laws of Security Administration and the 10 Immutable Laws of Security are often referenced in introductory security classes. Though these rules are dated, they are still relevant today. Just want to comment on a few of them and how we see them impacting our clients today.

10 Immutable Laws of Security Administration

Law #1: Nobody believes anything bad can happen to them, until it does
Law #2: Security only works if the secure way also happens to be the easy way
Law #3: If you don't keep up with security fixes, your network won't be yours for long
Law #4: It doesn't do much good to install security fixes on a computer that was never secured to begin with
Law #5: Eternal vigilance is the price of security
Law #6: There really is someone out there trying to guess your passwords
Law #7: The most secure network is a well-administered one
Law #8: The difficulty of defending a network is directly proportional to its complexity
Law #9: Security isn't about risk avoidance; it's about risk management
Law #10: Technology is not a panacea

Law #1 Nobody believes anything bad can happen to them, until it does This is probably the biggest stumbling block we encounter when working with small businesses. People like to think they are not targets, and to some extent, small businesses are not targets. The issue is that a significant amount of server compromises are not directed attacks but simply random scanning. If a bot scans your server and finds vulnerabilities, you quickly become a target. You don't have to hang a "Can't Hack This!" sign to solicit attention. If the random scanning turns up a juicy port or application, you become a target. So while you may not think it will happy to you, I can assure you that your server is being scanned frequently. Law #3: If you don't keep up with security fixes, your network won't be yours for long The key value of our linux server management offerings is the software update service. Beyond the monitoring and help desk support, our routine application of security patches keeps minor exploits from becoming major ones and reduces the chance of a critical security failure. Staying on top of security threats and plugging holes is vital. The number one issue we see is web applications. While we can do everything to keep the server secure, if you don't update your web applications, you can quickly become a victim. Web applications have quickly risen to the top of SAN's Top 20 threats and will likely remain there until easier update methods emerge. Law #6: There really is someone out there trying to guess your passwordsrackAID earns $1000's per year fixing security issues that were directly attributed to poor password security. Nearly every month, we encounter some system with a very poor password or using the same password in multiple contexts. You can use Winguide's Password Generator to make a strong password (8 characters with numbers, capitalization, and symbols). If you are worried about forgetting passwords, search for any number of password management tools. Law #8: The difficulty of defending a network is directly proportional to its complexity This is why when you want us to add some third party software we push back. Adding complexity should only be done when it is a business or technological necessity. As I pointed out with Red Hat Updates, keeping things stock is critical to easy server management. Law #10: Technology is not a panacea Too often people forget that there are other people out there trying to do back things to their network. Clever security technology can be cracked by clever hackers. Planning for security, implementing those plans, and knowing what to do when something does go wrong is key. This is one reason we push our CDP backup services. By retaining older backups, we can easily roll back a system to a pre-intrusion state. Our server backup services have save many people after their blog was hacked due to an outdated version. These are some of the key issues we see impacting our clients. I recommend you review the full list of 10 Immutable Laws of Security Administration and 10 Immutable Laws of Security. Though written years ago, they are still relevant today.

DSBL Defunct

2009-03-10T22:14:34Z

The DSBL real-time blacklist was shutdown almost a year ago. However, their nameservers continued to answer requests until yesterday. If you are using list.dsbl.org in your RBL settings, you will want to remove this as soon as possible. This may cause intermittent failures and RBL lookups timeout. On Plesk, go to Servers - Mail area and adjust the RBL lists your server is using.

Zoho: The Future of SMB IT Services?

2009-03-10T17:47:17Z

A few weeks ago, I stumbled onto Zoho. Zoho is a SaaS provider delivering many business targeted applications. Since we provide linux server management services, you would think we would just fire up our own software on any number of the servers we own. However, sometimes it is quicker just to outsource a function rather than deal with setting up software. Project Management We are currently overhauling our service offerings, launching a new marketing plan, and tuning up some items in the back office. When it was just a couple of us, those tasks were easy to manage, but with our growth, there are about a half dozen people involved with some working remotely. We then have about a dozen service partners that are helping us with marketing materials and development. To co-ordinate all of this, I started looking for project management solutions. I looked around and found several options, but I wanted something quickly. Getting going with Zoho was quick and free (for 1 project). That's a combination I could not resist. I am still getting used to the software, but thus far, the Zoho tools are working well. You can have multiple projects with the paid version, each project has key milestones, and then specific tasks. The system sends out email notices for meetings, task updates, and other changes to a project. As with any software, you have to get used to its quarks, but at little over $100 per year it is a bargain. Even if it does not serve our long term needs, the Zoho subscription model, low costs and easy sign up made the buying decision a trivial one. Hosted Services I think Zoho represents the future of "hosting". No longer can we sit back and provide a "hosting" plan with megabytes of space and dozens of email accounts. Businesses want solutions. When I talk to other small business owners, I rarely here them say, "I need about 500MB of disk space, 10GBs of bandwidth and 35 email accounts." Instead they ask for a place to share documents, a place to conduct virtual meetings, a place to have a shared calendar or address book. While Zoho may lake the popularity of Google Apps, they certain have a compelling suite of services. From document management to invoicing to remote desktop interaction, they have a wide range of services that will allow solution integrators to develop niche-specific solutions. Building Blocks I like Zoho because they provide the building blocks for integrated applications. This will allow tremendous flexibility for solution providers to deliver robust, customized applications to small business owners. I suspect I really like this approach as it is similar to the one we are adopting. rackAID taps many service vendors to create building blocks for a robust IT infrastructure. We integrate solutions from backup, security, and hosting vendors to provide a single integrated product. You no longer have to deal with a half dozen vendors when a problem arises; you can just contact us and we handle the vendor relationships. Zoho's model may grow to be similar. Integrated solution providers will abstract their clients from the gritty technical details. They will integrate solutions from Zoho, Microsoft, Google and others to give the client what they want. I know "cloud computing" is all of the rage now (and for the past year), but don't count SaaS out. The company that makes SaaS building blocks that are portable, easy to use, and easy to price, will certainly see a bright future.

How To Remove Your Email Server from Yahoo's Blacklist

2009-03-09T16:16:53Z

So you fire up your favorite mail client (mine's Thunderbird) only to find upset customers and bounced emails. Appears, Yahoo! has blacklisted you. You see tons of messages with an error:

421 4.7.0 [TS01] Messages from <1.2.3.4> temporarily deferred due to user complaints - <1.2.3.4> ; see http://postmaster.yahoo.com/421-ts01.html

So what now? Your server has been blacklisted by Yahoo!. The email queues are building up, clients are calling, and you cannot forward that latest NSFW video to your friends on Yahoo!. To get removed from Yahoo!'s blacklist, you need to follow some simple steps:

Find the cause
Fix the problem
Wait ...
Fill out the form
Wait ...

If you are a spammer (and you know if you are), then don't wast your time with these tips. They will not help you.
If your wearing a white hat happened to get nailed by the blacklist, read on. Find the Cause The first step is to find the cause. Email service providers are increasingly relying on sender reputation metrics to determine if email is spam or should be flagged for additional processing. Yahoo! works with ReturnPath for this feature. If you consistently trigger Yahoo!'s blacklist, you may find that you get permanently blocked or your email is routed to the junk folder. Yahoo! did not blacklist your server to spite you. Something triggered the listing. To prevent future listings, you must identify the trigger. If you get re-listed, your sender reputation will drop, and as a result, email will be even more difficult to delivery successfully. In my experience, if you run normally clean email operations and follow these steps, your Yahoo! email delivery problems will reside within 48 hours or less. Some key reasons for getting blacklisted:

Compromised Web Script
Compromised End-User Account
Legitimate Bulk Emailing
Forwarded Email

If you have a client forwarding large volumes of email to Yahoo! and the client flags these forwarded messages as spam, you could get blacklisted. The reason is it is impossible for the filtering to determine if the forwarding was intentional or a trick used by the spammers. As a result, spam filters often hold the last and first links in the delivery chain responsible. Historically, open relays have been prone to blacklisting, but recent versions of hosting control panels and email software often disables relaying by default. We encourage the use of SMTP AUTH on all servers to prevent unauthorized mail relay. Call the Postmaster All major ISPs, Yahoo! included, maintain very useful Postmaster help pages. These pages often detail the do's and don'ts for sending to their email servers. Yahoo!'s Postmaster help center provides a variety of topics. When you get a chance, I recommend you review their best practices area and assure your email policies are in check with what they recommend. Blacklist Removal Process Do not try to get removed from Yahoo!'s blacklist if you have not found the source of the problem. You don't want cycles of de-listing and re-listing to damage your sender reputation. If you have found the problem, then simple wait. In many cases, I find that the block is removed in under 48 hours. If you are still blocked after 48 hours (or impatient), you can then submit Yahoo!'s Mail Delivery Issues Form. Before submitting, be sure you are abiding by Yahoo!'s email policies. Also, verify that all of your DNS setting are correct. Reverse DNS (PTR), MX, A, and SPF records should all be checkek If you are using DomainKeys, be sure to test that those are working as they should. The key is not to raise any red flags during the review process. Make the reviewers job easy by fixing problems first. Though some fields on the delivery report are optional, it is best to provide as much information as possible. Especially, the "Enter additional information here:". You will want to detail in 2-3 sentences your remediation efforts. For example, if the problem was a compromised web script send them a note:

We have identified an insecure web application on our server that permitted unauthorized email relay through our system. We have removed this script.

Keep it short and technical. Yahoo! postmaster staff reviews 1000's of these requests, so being short and to the point is best. Bulk Senders If you are sending bulk email, you will receive a special type of bounce pointing you to Yahoo!'s error pages. Before starting the blacklist removal process, be sure to review Yahoo!'s bulk email sending policies. Once you are compliant, send in the Yahoo! Mail Bulk Sender Form to get your account exempted from the bulk sender triggers. You must maintain a clean list. Simply submitting the form does not let your email flow freely. If you are spamming, this will not help you. If you maintain clean, double opt-in lists, then your emails will get to their recipients. Quick Review When you get blacklisted by Yahoo! 1. Find the trigger 2. Fix the trigger 3. Submit a mail delivery report 4. Wait ... 5. Review best practices. In the next couple of weeks, I will be providing quick how-to's for getting out of the blacklists maintained by Google, MSN, Earthlink and Frontbridge. Also, if you send large volumes of email or are an email service provider, don't forget about email feedback loops. Yahoo! now provides a Compliant Feedback Loop based on DomainKeys.

Why Not Customize Red Hat? Security and Stability

2009-01-22T21:58:09Z

This week, Red Hat has rolled out RHEL 5.3,. We've been busy pushing it out to our servers as we like to keep them updated. Along with the 5.3 release, Mark Cox posted an interesting article on the risk profile of 5.2. He's examined the number of patches that have been submitted since the 5.2 release in May of 2008. For a default install, which many of our clients use, there were 45 advisories to address 127 vulnerabilities. This is 127 patches in 8 months. Seven advisories were rated critical, 21 were important, and the remaining 17 were moderate and low. Stock is Best Stats are great but there is a more important chart in this article which underscores why we discourage our clients from using software from third party sources. As you can see in the chart to the right. What you see here is the number of errata issued per month for each successive release. As Red Hat Releases mature, they typically require fewer updates. Fewer updates means fewer potential service impacts due to new patches. We also find that systems tend to become more secure and stable as the release progresses. When you use third-party or non-standard software packages (RPMS), you introduce a new management task, possibly increase your security risk profile, and increase the complexity of your operations. Do to these reasons, we often discourage using third party or alternate RPMS when possible. PHP Example We often get requests to have PHP upgraded to a non-standard version. For example, running PHP 5 on RHEL 4. While this is doable, doing so diminishes the benefits you see in the chart. When you introduce a non-standard software package, you are eliminating the benefits that Red Hat provides. Timely, tested patches are critical for operational stability and service continuity. Even for a subtle change, you now can no longer deploy Red Hat patches, but must check your 3rd party source for the updates. What if they do not build the package in a timely fashion? You then have to build it yourself (or get us to build it). This of course takes time which of course means increased costs. By sticking with the stock PHP packages, you save yourself these headaches. When to Customize? I often find that demanding the latest version is simply that -- a demand for the latest version. However, there are situations in which running a non-standard version may be required. For example, if you have a program that absolutely will not run on your current versions, you may find it better to run a non standard software package than to upgrade or migrate your server. There may be some functions provided by a third party package that is not included in Red Hat directly. These are both good reasons for using alternate software. What to Use? When you must use alternate software, I prefer to use sources that you can trust. Top on my list are CentOS Plus releases, EPEL, and Dag Wieers archive. I've used these resources many times. They have good support or user community behind them to keep the updates coming. TCO, Stability, and Security By using the stock packages, we can easily keep systems updated. As a result, you have better stability, increased security and lower total cost of operations. So when we strongly recommend you not switch to that latest version of package X, we do have broader goals in mind. While we are not fond of building custom RPMS, we can do so, but in most cases the alternate versions can be avoided.

How to Securely Managed Files with Secure FTP SCP WinSCP

2009-01-19T17:01:53Z

Still using FTP? If so, you may want to consider this: Every time your FTP client logs into your server, you are sending your user name and password over the internet unencrypted.. It is as if you stood at the ATM and yelled out your PIN number. If someone gained access to your ATM card, they could then get into your account. With FTP, your user name and password are not encrypted before they are sent to the server. So, if anyone is listening along the way, then your access credentials could be captured by a third party. Insecure WI-FI access points, networks with hacked servers, and malware could all lead to a compromised password. Enter Secure FTP I don't see these attacks often but they can be prevented by using Secure FTP (SFTP) or Secure Copy (SCP) methods. Most FTP clients now support Secure FTP. If your client does not, then you can grab a easy to use and free Secure FTP client WinSCP. When you use secure FTP methods, your username and password are encrypted before they are sent over the network. So if someone is listening, they get encrypted data and not your plain text information. On most Linux systems, Secure FTP is provided by the OpenSSH daemon, which can function as a shell client. Firewall Benefits Often, we receive support tickets because a client cannot connect via FTP. This is because FTP is a multi-port process. There is a port opened for the data and the commands. Firewall configurations can often block FTP access. Using SFTP, you can often avoid this problem since it only uses one port (TCP: 22). Using SFTP on Plesk On Plesk, you must set FTP user to have a shell in order to enable SFTP. You can do this under the hosting setup area. You can pick /bin/bash or /bin/bash (chrooted) to limit their access. If you are really concerned about them having a shell account, we can modify the shells to include a SFTP only shell. Advanced SFTP Usage If you manage many accounts, you may want to look into creating SSH keys for use with your SFTP client. SSH keys allow you to access sites without passwords. Instead you use your key file. A public key file is placed on the server and the private key file is kept on your system. Only when the public and private key files are in sync can you gain access. The benefit of this approach is that you can use a single SSH key on many different sites. You can then protect your key file with a passphrase. This allows you to use different passwords on each of your sites for enhanced security. This is how rackAID manages 100's of servers without worrying about password updates. We use SSH Keys to get into the systems. WinSCP includes a program called PuTTY Key Generator. You can find many tutorials by searching Google on how to use this. Once you create your key pair, you can upload your public key to the server and then begin using SSH keys instead of passwords.

Holiday Schedule 2008

2008-12-18T18:16:48Z

The 2008 holiday seasons is upon us. During the holidays, rackAID operates on special hours and terms so our hard working staff can get a bit of R&R. Non service impacting request may not be handled until our next business day. Christmas rackAID will close at 12:00 Noon Dec 24th and re-open at 9:00AM Dec. 29th. Case-based support requests submitted after 12:00 Noon Dec. 24th may not be handled until Monday, Dec. 29th. If you are subscribed to a server management package, your services will not be impacted. We ask that you please use the appropriate priority levels during the holidays. If something is urgent or critical, please use those levels. Staff will be working abbreviated schedules and on-call throughout the period. New Years rackAID will close at 12 noon Dec. 31st and re-open Jan 4th. Case based support requests submitted after 12 noon Dec 31st may not be handled until Jan 4th.

Update your Orbit/ServerCommand Accounts!

2008-12-16T23:15:58Z

Just a heads up to our management clients at thePlanet: ThePlanet has added an extra security feature to their Orbit account login. Users need to update their accounts to set a 'Verification Question' and Answer. Once these are set, you can log into your Orbit account as you normally would. You will not be prompted to choose a question and answer again. Please note, accounts cannot be accessed until the question and answer are set. If you use Server Command, you will also need to login and update your account. We rely on access to clients' Orbit accounts in case of outages and other emergencies. We've already had several cases this week where clients' accounts were not updated and our work to resolve issues was delayed. Please make sure you add your Verification Question and Answer to your accounts as soon as possible.