Second Life Grid Status Reports

[RESOLVED 20:08] The login problem experienced by some UK residents appears to have been resolved. Please contact support if you experience a return of this issue.

[3:49 PM Pacific] Our technicians are looking into the web-site login problems, and it appears to be working fine. However, there are still minor login problems from certain UK regions. We will keep you posted as more information becomes available.

For residents attempting to connect from the UK, you may experience slow connection times or the inability to log in to the website and/or in-world. We have received reports from AOL users; however, you may still experience these issues even if you are not using AOL.

Please watch this blog for further information as it becomes available.

[RESOLVED 10:15 am PDT] All problems are resolved, and our live chat is fully available again.

[UPDATE 09:50 am PDT] After talks to our vendor it appears that there is an ISP problem that affects our vendor’s network. They are working towards a solution with their ISP, and we will update you as soon as we have any further news.

[08:48 am PDT] We are getting reports that live chat requests time out, we are investigating the issue and will let you know as soon as we have any news.

[08:14 am PST] The VPN issue has been fixed, all systems are working as desired again.

[07:46 am PST] Due to a networking issue, you might see failing teleports, transactions going stale and logins not working. Our Ops team is aware of the issue and working to fix it.

Please hold off transactions and handling/rezzing of no-copy items until this issue is resolved

[Resolved 8:35 PM PDT] Teleports and Logins should now be behaving normally.

Due to a problem with our network, residents might experience a slightly decreased teleport success rate at the moment, and logins will also be affected. We are aware of this and our Ops team is already working to fix it.

[17:25PM Pacific] All vioce IM’s are back up now.  Thank you for your patience.

[12:20AM Pacific] The problem appears to be intermittent and is being looked into.  For the time being, we do not have any additional information.

[9:20PM Pacific] We have received scattered reports that voice IM is down within the Burning Life 2009 Estate.  Our vendor is working to resolve this issue.  Spatial (Local) and Group voice chat are both functional within the estate. We will update this post once the situation has been resolved.

[1650 PDT]  All concerns with logins have been corrected and all capabilities have been restored.  Thank you for your patience and understanding.

[2:36PM Pacific] Some logins are temporarily disabled, our technicians are working on the problem as quickly as possible.

[Resolved 4:00pm PDT] - Phone support has been fully restored.

[Update 3:00pm PDT] - Phone support is still unavailable. We are continuing to work on resolving this issue as quickly as possible. We will update this blog as information becomes available.

We have temporarily lost phone support via our service provider. We have taken steps to contact them and hope to have phone support back up as soon as possible. We will keep you posted and let you know as soon as phone support has been restored.

Linden Lab has released an optional update to the Second Life viewers today to address a potential security issue. Recently an audit identified a possible vulnerability. If a malicious user were able to obtain the IP address and port of a Resident’s viewer, then the malicious user could forge data packets to the Resident’s computer. This could be done in a way to cause the viewer to return enough information about its session to allow the attacker to initiate various server-side operations as if they were the Resident, including L$ transactions.

In the case of L$ transactions, this action would be visible to you: if this were to occur, the viewer would report the transaction after it occurred in the normal blue dialog box. Also, you are always able to inspect the transaction log to see recent transactions. This would allow you to notice and report these actions for violating the Second Life Terms of Service.

This type of malicious action would constitute a violation of the Terms of Service, and would be against the law in some locations. At this time we have no evidence that this vulnerability was ever exploited.

To eliminate this vulnerability, we have now updated the Second Life servers to transmit the messages over an encrypted channel (HTTPS). Now that the server upgrade is complete, we are releasing updated viewers that only accept these messages when transmitted over an encrypted channel. Once you have downloaded the update, if a malicious third party were to attempt to send messages over the old channel (UDP), they would be ignored.

Again, we have no indication to date that this security issue has ever been exploited or is being exploited currently. However, we strongly encourage Second Life Residents to update to the latest viewer with the security patches in place. The viewers are:

• Second Life Release Viewer 1.20.16 (this updates 1.20.15, released on July 24th)
• Second Life Release Candidate Viewer 1.21 RC3 (this updates RC2 and includes additional bug fixes as part of the usual release candidate cycle)

We have not provided a security update to older viewers (such as the 1.19 series) which are older than the official supported Second Life viewer. To take advantage of this security update, we do encourage Residents using version 1.19 to update if possible to version 1.20.16. However, this upgrade is not mandatory in order to continue to use Second Life. An old viewer will alert you that a new version is available.

The updated source code for these new 1.20 and 1.21 RC viewers is being made available via the usual open source channels.

For discussion about the issue, please visit the Second Life Forum: http://forums.secondlife.com/forumdisplay.php?f=350

[3:00 AM Pacific] Affected residents should now be able to connect to Second Life.  This issue is resolved!

[2:10 AM Pacific] An issue between two or more Internet Service Providers may prevent some residents from being able to login to Second Life.

Update 2008-09-25 08:10am: The second-half rolling restart is finished.

Update 2008-09-25 05:20am: The second-half rolling restart has begun.

Update 2008-09-24 08:50am: The first-half rolling restart is complete.

Update 2008-09-24 05:30am: The first-half rolling restart is beginning now. Even-numbered hosts will be updated to 1.24.6.97433.

Update 2008-09-23 09:07pm: Except for a handful of recalcitrant regions which we’re working on right now, the pilot rolling restart is complete.

Update 2008-09-23 08:09pm: The pilot rolling restart to ~300 hosts has begun.

Update 2008-09-22 10:14pm: We are delaying the 1.24.6 rolling restart by one day to the following schedule:

• Tuesday, Sep. 23, 8-10PM : A pilot group of ~3000 regions will receive 1.24.6
• Wednesday, Sep. 24, 5-9AM : Half of Second Life will receive 1.24.6
• Thursday, Sep. 25, 5-9AM : The rest of Second Life will receive 1.24.6

Update 2008-09-18 11:11am: The rolling restart to deploy 1.24.6 has been postponed until next week, and will follow the schedule:

• Monday, Sep. 22, 8-10PM : A pilot group of ~3000 regions will receive 1.24.6
• Tuesday, Sep. 23, 5-9AM : Half of Second Life will receive 1.24.6
• Wednesday, Sep. 24, 5-9AM : The rest of Second Life will receive 1.24.6

There will be a rolling restart this week to deploy server version 1.24.6 to Second Life. Release notes for what is changed may be found on this page on the Second Life Wiki. There are no new features in this release, only bug fixes. The version that will be deployed to Second Life is currently on the Preview Grid on regions running the Second Life Beta Server channel.

The rolling restart will follow this schedule:

• Wednesday, 8:30-10PM: a pilot group of ~3000 regions will receive 1.24.6.
• Thursday, 5-9AM: half of Second Life will receive 1.24.6.
• Friday, 5-9AM: the rest of Second Life will receive 1.24.6

Each region will go down for about 5-10 minutes once during the rolling restart. If your region stays down for more than 20 minutes, please contact support.

Discussion of this rolling restart can be found in this thread in the Second Life forums.