CloudFail.net

Rackspace Cloud FTP SAT1 | online

cloudfailbot — Wed, 08 Sep 2010 11:37:49 +0000

At approximately 6:21 AM CDT, Rackspace Cloud Engineers identified an issue with our FTP servers in our SAT1 data center. We are currently experiencing intermittent accessibility and some functionality may be degraded. Our engineers are working diligently to identify and resolve the issues.

update: 7:38am our engineers resolved the ftp issue.

Thank you for your patience and understanding during this time. If you have any further questions, please contact a member of our support team by visiting us in live-chat or by calling 1.877.934.0407 or INTL +1.210.581.0407.

Miami-b07 reboot

cloudfailbot — Wed, 08 Sep 2010 03:59:33 +0000

We had to reboot miami-b07 as it hit a Xen bug. It has been rebooted and is back online. All Webbies are running.

Please send us a ticket if you have any issues with your Webby on b07.

dev.twitter.com down for maintenance

cloudfailbot — Wed, 08 Sep 2010 00:04:00 +0000

We have taken dev.twitter.com down for maintenance. We are working to bring it back up.

Update at 8:36 PM: dev.twitter.com is back up.

SliceManager and API Maintenance

cloudfailbot — Tue, 07 Sep 2010 21:27:00 +0000

Slicehost Engineers will be performing maintenance on our infrastructure on Thursday, September 9th from 10:00PM – 11:00PM CDT (03:00-04:00 UTC – September 10th). During this maintenance window, the SliceManager and the API will not be accessible. Your slices will remain online and continue operating as expected. We will work to complete the maintenance as quickly as possible and apologize for any inconvenience.

If you have any questions or concerns please get in touch with our support team by visiting SliceChat or via email at support@slicehost.com. Thank you for your patience and understanding.

DATE: September 9th, 2010

TIME: 10:00PM-11:00PM CDT (03:00-04:00 UTC – September 10th)

IMPACT: SliceManager/API will be unavailable during the maintenance.

Google Mail

cloudfailbot — Tue, 07 Sep 2010 18:02:00 +0000

September 7, 2010 11:02:00 AM PDT

An incident report has been posted here.

August 25, 2010 10:27:00 PM PDT

The problem with Google Mail should be resolved. We apologize for the inconvenience and thank you for your patience and continued support. Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better.

August 25, 2010 6:41:00 PM PDT

We’re experiencing an issue affecting less than 2.5% of the Google Mail user base. The affected users are able to access Google Mail, but are seeing error messages and/or other unexpected behavior. We will provide an update by August 25, 2010 9:41:00 PM PDT detailing when we expect to resolve the problem. Please note that this resolution time is an estimate and may change.

Certain messages sent by affected users are being re-sent multiple times. We are working to implement a fix for this issue, after which point some of these messages may be re-sent one final time.

Packet Loss affecting some Linodes in London

cloudfailbot — Tue, 07 Sep 2010 17:43:03 +0000

1:46pm (EDT): Connectivity has returned to normal and we will continue to monitor at this time.

1:41pm (EDT): We are looking into a network issue in the London facility. There is no reason to issue reboots as Linodes are still up at this time.

Cloud Sites | DFW | MySQL Clusters | Maintenance Friday September 10th 01:30am – 05:30am CDT | Scheduled

cloudfailbot — Tue, 07 Sep 2010 16:34:34 +0000

Rackspace Cloud system engineers will be performing a series of hardware upgrades to MySQL servers in our DFW (Dallas Fort Worth) data center. This maintenance is scheduled for Friday September 10th, 2010 from 01:30am – 05:30am CDT. During this maintenance the MySQL servers listed below will have a period of 15-20 minutes in which they will be in a Read-Lock mode. While in Read-Lock INSERT, UPDATE, DELETE, ALTER queries will be ignored.

mysql50-01-master.wc1	mysql50-39-master.wc1
mysql50-02-master.wc1	mysql50-39-master.wc2
mysql50-05-master.wc1	mysql50-54-master.wc1
mysql50-06-master.wc1	mysql50-56-master.wc2
mysql50-12-master.wc2	mysql50-57-master.wc2
mysql50-27-master.wc1	mysql50-76-master.wc1
mysql50-31-master.wc1

We apologize for the inconvenience, but we need to complete this work in a timely manner. Please note that no other technologies will be effected by this maintenance. If you have any further questions, please contact a member of our support team by visiting us in live-chat or by calling 1.877.934.0407 or INTL +1.210.581.0407.

Emergency miami-b13 maintenance

cloudfailbot — Tue, 07 Sep 2010 14:40:51 +0000

We are currently investigating a problem in miami-b13. More updates to follow.

Update: We had to perform a reboot on miami-b13. The server is stable now.

Update 1:08PM The server is not responding once again, we are working to stabilize it and move the affected users on to other nodes asap.

Cloud Sites | Emergency Maintenance | mysql50-28.wc2.dfw1 | completed

cloudfailbot — Tue, 07 Sep 2010 03:25:57 +0000

At 11:30pm CDT mysql50-28.wc2.dfw1 in our Dallas datacenter (DFW) will undergo an emergency maintenance. The maintenance will be for approximately two hours, during this time customers and sites will experience connectivity issues to the database. We greatly apologize for the inconvenience, we will work as quickly as possible through the maintenance to prevent as much down time as possible. We will continue to update this post as we start and progress through the maintenance.

Update: 11:30 PM Maintenance has started.

Update: 12:45 AM Maintenance has now completed.

If you have any issues, please do not hesitate to contact a member of our support team via live-chat or at the following telephone numbers: 24-hour toll free 1.877.934.0407 and INTL +1.210.581.0407.

Cloud Servers | DFW and ORD | Control Panel and API Maintenance | September 9th 10:00pm – 11:00pm CDT | Scheduled

cloudfailbot — Tue, 07 Sep 2010 00:47:49 +0000

Rackspace Cloud system engineers will be performing maintenance on the Cloud Servers infrastructure which was originally scheduled for September 7th from 10:00pm to 11:00pm CDT but is now rescheduled for Thursday, September 9th from 10:00pm to 11:00pm CDT. During this maintenance window there will be no access to the Cloud Servers section of the control panel or API. All Cloud Server instances will remain online and be accessible.

If you have any questions or concerns in regards to this maintenance please do not hesitate to contact a member of our support team via live-chat or at the following telephone numbers: 24-hour toll free 1.877.934.0407 and INTL +1.210.581.0407.