[RESOLVED] Datacenter Maintenance
[RESOLVED June 21, 2009, 8:34 PM Pacific] All regions have been returned to service. The land store is open again.
[UPDATE June 21, 2009, 6:40 PM Pacific] We’re down to under one hundred regions waiting for reactivation. Hopefully this will be over very shortly.
[UPDATE June 21, 2009 6:22 PM Pacific] We’re still working to bring a few hundred regions online. We’ve found ourselves short on simulator hosts today due to a power issue in one of our colocation centers. As the colocation center gets that issue rectified, we’ll use the available hosts to return existing regions to service first, and then reopen the land store.
[UPDATE June 21, 2009 4:30pm Pacific] The Land Store will be offline while we work to restore the remaining regions.
[UPDATE June 21, 2009 2:25pm Pacific] Our technicians are working quickly to restore remaining regions. Please refrain from restarting regions as this could cause a longer downtime for that region during this maintenance issue.
[UPDATED June 21, 2009 1:43pm Pacific] Our technicians are continuing to work to restore service to all remaining regions.
[UPDATED June 21, 2009 12:56pm Pacific] Our technicians are continuing to work to restore service to all remaining regions.
[UPDATED June 21, 2009 11:35AM Pacific] Our technicians are continuing to work to restore service to all remaining regions.
[UPDATE June 21, 2009 10:42AM Pacific] We are continuing to work on restoring regions at this time.
[UPDATE June 21, 2009 9:38AM Pacific] Multiple regions are still offline at this time, we will continue to enable all remaining regions as soon as possible.
[UPDATE June 21, 2009 8:04AM Pacific] Our technicians are continuing to work to restore service to all remaining regions.
[UPDATE June 21, 2009 6:45AM Pacific] Our technicians are working to restore service to 70 remaining regions.
[UPDATE June 21, 2009 5:00AM Pacific] The number of regions that are offline is down to 250. Our technicians are working to restore service to these.
[UPDATE June 21, 2009 3:00AM Pacific] We’re working to restore service to approximately 450 regions.
[UPDATE 12:30AM PDT] Maintenance is ongoing, we are slowly bringing services back online. We had some secondary hardware failures, we hope to be back to full speed soon.
[UPDATE 22:10 PDT] The emergency maintenance has begun.
[UPDATE 21:50 PDT] The emergency maintenance has been moved forward, and will be starting at 22:10 (10:10pm). We will post here as further updates are available.
[June 20 18:37pm PDT] - At approximately 23:00 (11:00pm) PDT tonight, one of Second Life’s primary datacenters will be undergoing emergency maintenance. For an orderly maintenance cycle we will be bringing approximately 5,000 regions offline during the course of this maintenance starting at 22:30 (10:30pm) PDT. We will keep this blog updated with more information as it becomes available. Thank you.