Second Life Grid Status Reports

[RESOLVED – 18:37 PM PST] Logins have stabilized as well as all other issues. At this time everyone should be able to login again. Thank you for your patience.

[UPDATE – 18:26 PM PST] Continuing to have login problems. We are working to fix this as quickly as possible.

[UPDATE - 18:00 PM PST] Logging into Live Chat and Support Portal are also experiencing difficulties at this time. We will keep you posted on the status.

We have reports of sporadic unavailability of some services such as login not working, transactions failing, and land store down. We are aware of the problem and are working to bring these services back online as quickly as possible.

[Resolved 2:36 PM PDT] Services have been fully restored.

We are currently experiencing some difficulties with our database. This may result in difficulty teleporting, failure to rez objects and complete transactions, and inability to log in to the Support Portal. Until we solve this issue, we have restricted logins, as well as disabled the Land Store. We encourage you not to rez any valuable objects or make important transactions until this issue is resolved. As always we will keep you up to date here as the siatuation progresses, and we thank you for your patience.

All-Clear 9:56 p.m. PDT We have identified and corrected the routing issue.  It is all-clear to conduct transactions and issues stemming from this routing problem should be resolved.

Some residents are reporting problems with appearance and inventory. Operations is tracing potential routing issues. Please do not attempt valuable transactions until an all-clear is given.

[UPDATE 11:03 a.m. Pacific] Logins have been re-enabled.

We’ve temporarily closed logins to stabilize database performance. We’ll keep you posted on details as they emerge.

[RESOLVED 13:10 PDT]  The database issues are corrected and the all-clear has been sounded.  Please continue with your regularly scheduled Thursday.

[UPDATE 12:30 PDT] The Support Portal is currently down, and the Land store has been taken off-line for the duration. Please continue to monitor this post for updates.

Issues with the database are currently affecting inworld services, logins, and the website. Please refrain from attempting inworld transactions, rezzing valuable or no-copy items, and logging in until the all-clear is announced.

[ALL CLEAR 11:43 PM Pacific] Logins are open and the asset system is stabilized.  Please be patient logging in for the next few minutes.  If possible, wait for the first login to complete instead of canceling and retrying, so that you maintain your position in the login queue.

[UPDATE 10:58 PM Pacific] Logins are now Linden-only while Operations repairs the asset system. Please do not attempt valuable transactions until the ALL CLEAR is given.

[UPDATE 10:30 PM Pacific] Operations is continuing to assess the slowdown. They’re sufficiently engaged in the effort that getting more details from them at the moment would be a counterproductive distraction. We’ll ask them for more information when possible, and a follow up post when the slowdown is cleared.

There’s an issue with the asset servers which is interfering with a number of inworld services. It is being investigated, but until the issue is resolved, please refrain from making transactions or rezzing no-copy items.

[ALL CLEAR] The asset system is stabilized, and logins and registration are open again. Please be patient logging in for the next few minutes as the queues process a surge of requests. If possible, ride a single login attempt all the way through the wait in order to keep your place in the queue, rather than canceling and retrying.

[UPDATE 11:59 PST] Logins are Lindens only while Operations corrects the asset system slowdown.

[UPDATE 11:43 PST] Operations has isolated the trouble, but will need some time to mitigate it. We’ll be disabling registration for a short time. We’ll keep you posted.

There are reports of LindeX and land transactions failing, please do not attempt valuable transactions while we investigate and keep an eye on this blog for further updates. In-world L$ transaction may be affected as well, so please be wary of that as well.

[RESOLVED 07:10 AM PDT] These regions are now happily reunited with the rest of Second Life.

[6:45AM PDT] We currently have a networking issue that is affecting many regions. Anyone in these regions will be able to move and interact normally, except they will be essentially cut off from communicating with the rest of world until the problem is fixed. That means items will not rez, IM’s may not work, L$ transactions may fail, teleporting will not be possible and items will not be able to be saved to inventory.

We are working to fix the faulty component and should have things back up and running soon. We will post updates along with an estimate on how long this will take as soon as we can. -Chiyo

[FINAL UPDATE 4:27 p.m. Pacific] In world services and support portal access are fully restored. The Concierge and Grid teams are working with a slightly higher than normal count of regions which have entered restart, but that backlog should be cleared within the next few minutes.

[UPDATE 3:01 pm Pacific] Operations has spent the past hour swapping in some database resources. Their efforts have made a difference, but throughput could still stand some improvement. Work continues.

[1:53 pm Pacific] A sudden heavy load on the database is causing slow logins and inhibiting access to the support portal, and may be impacting in world transactions for some residents as well. We’ll keep you posted as we work to assess and correct the problem.

[UPDATE @ 1:26 PM PDT]  These issues have been resolved. Thank you for your patience.

The database is currently experiencing an excessive load, causing some inworld service failures. While this is being looked into, please avoid activities such as inworld transactions and rezzing of no-copy items.