Second Life Grid Status Reports

[RESOLVED 10:28AM PST] All services should now have returned to normal.

[UPDATE 10:13AM PST] Logins have been re-enabled and the maintenance is nearly complete.

[UPDATE 9:39AM PST] We have temporarily disabled logins while we work on solving this quickly.

Residents may experience teleport and asset failures while we perform some brief server maintenance at our Dallas facility. We do not expect this to last very long, but in the meantime we advise residents not to make any important transactions or rez any no copy objects. We will keep this post updated until the issue is resolved.

[RESOLVED 6:20PM Pacific ] The server is now up and running smoothly and affected residents should now be able to log in.

[UPDATE 5:00PM Pacific ] Server recovery is taking a little bit longer than anticipated, but we are still expecting a resolution soon.

[UPDATE 3:56PM Pacific ] Our technicians estimate that the inventory server will be back to normal within the next hour

[2:56 PM Pacific] A problem on one of our inventory servers means it is running more slowly than normal. Residents whose inventory is stored on this server may experience slow logins, delays or failed inventory transactions. Our technicians are still working on this issue.

[2:00 PM Pacific] We’re currently investigating reports from residents who’re getting stuck the logging in page when attempting to connect to Second Life.

[FINAL 10/8 4:58 pm Pacific] The system failure which has been spreading this annoying behavior to additional assets has been repaired.  We’ll be double-checking the support tickets we’ve received on this issue as quickly and thoroughly as possible.  If you discover another asset in your inventory which will rez in some regions but not others (allowing for rez privileges on the parcel where you try), please file a support ticket with at least the region name, and the object’s UUID if you can capture it from inventory. 

[UPDATE 10/8 00:40] Our technicians are continuing their work to correct this issue. If you are still encountering failures to rez, please file a support ticket with the time of the failure (in PDT, please), the region name and asset UUID that failed to rez so the information may be passed on to the developers.

[Update 3:36pm PDT] - Our technicians are continuing to work on the solution. We will keep you updated as we get more information on this matter.

[Update 2:36pm PDT] - We believe we have found the cause of this issue and are working to resolve it as quickly as possible. Please be aware that along with some people experiencing an issue with rezzing items, there may be some who cannot properly attach items either. We will continue to update this blog as information becomes available.

We are experiencing difficulties with the ability to rez items at this time. We will keep you posted as to when this problem is resolved.

[2:49 PM Pacific] Logins are functioning normally.

[1:20 PM Pacific]  We are currently looking into reports from residents who get error messages when trying to connect to Second Life.

[CLOSED 4:36 PM Pacific] 

Voice was down for approximately 30 minutes today.

***

[UPDATE 4:20PM Pacific] We’re checking with our vendor to get a current assessment on the partial voice chat outage.

***

Several calls have come in regarding voice chat outages in various regions.  We’re working with the vendor now to assess and repair the problem.

One of the twenty servers which provide asset management just recovered from a disruption lasting only a few minutes. It recovered while this post was being prepared.

If you are having residual problems with appearance or inventory, please clear your Second Life cache (Preferences->Network) and relog.

[CANCELLED] The Support Portal will remain online tonight. Please disregard the previous notice.

The Support Portal will be offline for approximately ten minutes tonight, begining at 23:10 PDT. Residents will not be able to submit Support Tickets or engage in Live Chat support during this time. Please monitor this post for updates.

[05.51 AM RESOLVED] Logins should be restored for all affected accounts. It may take some time for things to return to normal as the system catches up and everyone starts logging back in, but they are back online. -Chiyo

[05.16 AM UPDATE] The good news is we have found the problem with logins. The bad news it means logging off all accounts associated with one of our inventory databases while the fix is implemented. The accounts affected will not be able to log back in until they are complete.

There isn’t a way for you as a resident to know if you are one of those affected until the process begins. We apologize if this is abrupt or disconcerting for those involved, but ask your patience and understanding. Logins should be restored for everyone soon. We will update on the work progress and inform you when they are complete. -Chiyo

[04.55 AM ] We are seeing problems with many residents unable to log in. This may be related to the previous problems with packet loss and teleports, but we are not sure at this point. We are working to restore logins for everyone and will report our findings when we have a clear picture on the situation. -Chiyo