[RESOLVED - 7:55 AM Pacific] - Logins are now fully restored.
[6:59 AM Pacific] One of our main infrastructure servers has suffered a crash. This will result in various in world problems as well as failed logins. Our technicians are working on damage control and we hope to have things back to normal and logins fully restored shortly.
We are moving a rack of regions to new hardware in a short while. All affected regions will be given a 10 minute warning before the restart happens.
[Resolved - 3:20pm PDT] - The network issues have been resolved and all services have been restored. Please feel free to resume normal activities again.
Database issues are causing a number of inworld services interruptions including teleporting, transactions and rezzing failures. The technical team is investigating, but until the All-Clear is sounded please refrain from making purchases or rezzing valuable/no copy items. Please monitor this post for updates.
[RESOLVED 04:00] Access to the Tracker and the Wiki is now restored.
[UPDATED 00:14] The efforts to correct the login issue with the Public Issue Tracker and Second Life Wiki continue. We will update the this post again when the issue is resolved or the situation changes.
[UPDATED 21:33] Work continues to correct the login issue with the Public Issue Tracker and Second Life Wiki. Please monitor this post for further updates.
[8:02 PM - Pacific] We will keep you updated as more information becomes available.
[6:55 PM - Pacific] We are currently experiencing problems with our Second Life Wiki, and Public Issue Tracker, while still browsable, is not allowing logins at this time. We are currently looking into this problem, and will keep you updated.
[RESOLVED 16:25] The technical team has corrected the networking issues and has called All-Clear. Inworld services should be functioning normally, and transactions and rezzing may resume. If you continue to encounter teleport or other problems, please attempt clearing your cache and relogging.
[UPDATE 15:21] Work on the networking issues continues and inworld services are still being affected. A small number of additional regions may be briefly taken offline during this work, but downtime for these regions should be minimal. Reminder - please continue to refrain from making purchases or rezzing valuable/no copy items until the All-Clear is sounded.
[UPDATE 14:15] The database has recovered, however network issues continue to cause the inworld service interruptions noted earlier. Please continue to refrain from making purchases or rezzing valuable/no copy items until the All-Clear is sounded. The topic of this post has been changed to reflect the changed circumstances.
[UPDATE 14:12] As part of their work to repair the current issues, the technical team will shortly be taking at least 144 regions offline. The regions will remain down while work is being done, but will be brought back up as soon as possible.
Database issues are causing a number of inworld services interruptions including teleporting, transactions and rezzing failures. The technical team is investigating, but until the All-Clear is sounded please refrain from making purchases or rezzing valuable/no copy items. Please monitor this post for updates.
[RESOLVED 16:05] Ops has reported all issues have been returned to normal.
[UPDATE 15:50] Ops has given the all clear on all issues except logins. We are projecting this to be resolved in approximately 15 minutes. We will keep you posted. Thank you for your continued patience.
[UPDATE 14:55] Ops remains hard at work fixing the network problems. They have located the locus of traffic-overload, and are working to mitigate it. As before, please continue to refrain from making transactions or rezzing no-copy items until the All-Clear is sounded.
[UPDATE 13:44] Ops is continuing to work to fix the network problems. Please continue to refrain from making transactions or rezzing no-copy items until the All-Clear is sounded.
We are again experiencing some network issue which are affecting inworld functions, including teleporting, region crossing, transactions and rezzing. Additionally, logins may be affected and the Land Store is offline. Ops is investigating, but until the All-Clear is called please refrain from making transactions or rezzing valuable or no-copy items.
[CLOSED 4:43PM PDT] We’ve been monitoring in world services for stability for the past quarter hour or so, and we’re ready to announce an ALL CLEAR on TPs, border crossings, logins, and asset handling. We’ll continue to monitor throughout the evening.
[UPDATE 3:48PM PDT] We have our top people doing their best to attempt to track down the source of our network instability. We apologize that progress has been slow, but we are progressing.
[UPDATE 2:57PM PDT] We are still investigating the problems with teleporting and various other issues. Thank you.
We have reports of failed teleports, profiles and group info not loading, notecards and scripts not saving, avatars not loading, uploads not completing and failing L$ transactions. Our technicians are working on the problem and we ask that residents refrain from these activities as much as possible until we have a better picture of what is going on. We hope to have more information soon.
[RESOLVED 22:09] The networking problems have been dealt with, and Ops has called the All-Clear. All inworld services, including transactions and rezzing should be functioning normally. If you are still encountering issues, please try your clearing cache and relogging.
[UPDATED 21:41] The networking problem appears to be on the mend, however ops is continuing to monitor. All-Clear is not yet being called, so please still refrain from major purchases or rezzing.
[REOPENED 20:41] The networking issues have returned, and are once again causing problems. Affected residents may experience problems logging in, teleporting, failed inworld transactions, and failed rezzing of items. Until the All-Clear is sounded, please refrain from making transactions or rezzing valuable or no-copy items.
[Resolved 7:58 pm Pacific] Teleport problems have been cleared and all is well.
[Updated 7:38 pm Pacific] Teleport success is greatly improving, however, we are delaying an ALL CLEAR a bit longer.
[REOPENED 6:32 pm pacific] Some regions appear to have ongoing issues with teleportation. You can attempt to teleport via map, or you can relog in a different location if you find yourself stuck. We are working hard to get this fixed as soon as possible.
[RESOLVED 5:47 pm Pacific] We have restored all regions and everything should be functioning normally.
[UPDATE 4:57 pm Pacific] We have restored many regions that were offline as a result of this issue, the rest should be back up within the next 30 minutes. Teleports, avatar clouding, and asset-related issues should be clearing up as well.
[UPDATE 3:55] Teleports, avatar clouding, and asset non-performance are being stabilized. However, numerous regions are still inaccessible. We’re delaying an ALL CLEAR on assets a bit longer.
[UPDATE 2:54 pm Pacific] Unfortunately this issue is taking longer than expected to resolve. However, teleport success is improving, and we’re hoping to resolve this as soon as possible.
[UPDATE PENDING 1:54 pm Pacific] We’re about fifteen minutes way from having updated information.
[UPDATE 12:52 pm Pacific] We are still working on these issues, and we appreciate your patience.
[UPDATE 11:42 am Pacific] Logins are preforming normally. Network issues are still impacting teleport attempts and causing avatar clouding and difficulties crossing region borders. Two inventory servers are resynchronizing. Please avoid in world transactions with valuable (no-copy) assets until we give an ALL CLEAR. We’re escalating our network difficulties to our provider, and we’ll keep you posted.
[UPDATE 10:57 am Pacific] We are still working to bring the broken VPN back. If you experience loading issues after login, like your avatar appearing as the white cloud, please try to login to a different region.
Due to a problem with our network, Residents migth experience a slightly decreased teleport success rate at the moment, and logins will also be affected. We are aware of this and our Ops team is already working to fix it.
RESOLVED 10:55 am Pacific - The live chat function is back.
REOPENED 10:21 am Pacific - We see some more interruptions in our live chat portal. The issue has been escalated, and we hope to restore the full functionality of your support soon. Until this is fixed, you can always submit a ticket.
RESOLVED 2:55AM Pacific - Live Chat is now in service!
REPORTED 2:05AM Pacific - The Live Chat feature found in our Support Portal is temporarily unavailable. Our support vendor has been notified and is looking into this matter. In the meantime, residents may submit a ticket to contact Support.
[RESOLVED 7:05AM Pacific] The second vpn issue has been resolved and all regions should be once again accessible.
[UPDATE 6:52AM Pacific] We’ve configured our network to bypass the faulty vpn while we work on it. Those regions previously down should be accessible once more with only some residual tp and chat issues. Unfortunately another vpn has also gone down affecting 500 more regions and are working to bring them back up as soon as possible as well.
[UPDATE 6:22AM Pacific] Our technicians are confident they should have a temporary fix in place soon while they work on resolving the source of the problem.
[UPDATE 5:20AM Pacific] Things are taking a little longer than originally planned, this isn’t resolved just yet.
[UPDATE 4:36AM Pacific] We expect to have the problem fixed in the next 15-20 minutes.
Due to a vpn error 600 regions are currently offline or inaccessible. We are working to solve the problem and should have them back up shortly. We will update on our progress when we have a resolution or more information becomes available.
Unknown Feed