Archive for the ‘Service Status’ Category

[Completed] Rolling restarts starting at 11pm March 30

Monday, March 30th, 2009

[Completed 03:18am PST] The Rolling Restarts have been completed.

This evening, we will be continuing our project to improve both the performance and maintainability of our network, by moving many of the servers that host Second Life’s regions onto a new network.

This will require us to restart several hundred regions. Unfortunately, since every time a region restarts, it moves onto the first available hardware of the appropriate class, we cannot predict which exact regions will be affected. However, there will be in-world warnings prior to the restarts, as usual.

We will update with any further details when the rolling restart begins at approximately 11pm Pacific tonight.

[RESOLVED] Support Portal Temporarily Unavailable

Sunday, March 8th, 2009

[5:25PM Pacific]  Live Chat is restored to normal now.

[4:17PM PDT] Live Chat found at Second Life’s Support Portal is temporarily unavailable. This is being investigated and any updates will be posted to our Status Blog. Please note that phone support is still available.

[RESOLVED] German phone support temporarily unavailable

Friday, February 13th, 2009

[Resolved 6:00AM Pacific]  Service to our German phone support has been restored.

[Reported 5:30AM Pacific]  German phone support is temporarily unavailable.  We are investigating this and will post any updates as they become available.

[UPDATED] Network issues

Friday, January 30th, 2009

[UPDATED 9:58am PST] - While we have not had confirmation from our network providers, the issue appears to be resolved at this point. We will update this blog when we receive confirmation that this has been resolved.

[07:13AM] We are still working on a resolution to the ongoing issues with our network, we will continue to update you on the situation which we hope to resolve as soon as possible.

[04:13AM] The problems in-world being experienced by residents are related to difficulties at our network provider. We are liaising with them to resolve these as quickly as possible.

[03:50AM] RE-OPENED - We are still getting reports that some residents are still unable to teleport. We will update you as soon as the issue has been resolved.

[02:52AM] [RESOLVED] All issues have now been resolved.

[1:15AM] This is still being looked into. For now, there is nothing new to report.

[11:35PM] You may be unable to view L$ balances, teleport, search, cross regions or connect to Second Life. If you’re inworld, please refrain from rezzing or derezzing no-copy objects.

[11:20PM Pacific] We are aware of an issue with our network that may affect inworld services. Please keep an eye on this post for additional information as it becomes available.

[RESOLVED] Logins Disabled

Saturday, December 27th, 2008

[Resolved 5:02 PM PST] We have re-enabled logins and services should be returning to normal.

Logins to Second Life have been disabled while we work on fixing a database problem that is affecting teleports, transactions, and rezzing. We advise residents not to make any important transactions or rez any no copy objects until we resolve this issue. We will keep this post updated as the situation evolves.

[COMPLETE 09-25 08:10AM] Rolling Restart to deploy 1.24.6, Tue-Thu Sep. 23-25

Thursday, September 25th, 2008

Update 2008-09-25 08:10am: The second-half rolling restart is finished.

Update 2008-09-25 05:20am: The second-half rolling restart has begun.

Update 2008-09-24 08:50am: The first-half rolling restart is complete.

Update 2008-09-24 05:30am: The first-half rolling restart is beginning now. Even-numbered hosts will be updated to 1.24.6.97433.

Update 2008-09-23 09:07pm: Except for a handful of recalcitrant regions which we’re working on right now, the pilot rolling restart is complete.

Update 2008-09-23 08:09pm: The pilot rolling restart to ~300 hosts has begun.

Update 2008-09-22 10:14pm: We are delaying the 1.24.6 rolling restart by one day to the following schedule:

  • Tuesday, Sep. 23, 8-10PM : A pilot group of ~3000 regions will receive 1.24.6
  • Wednesday, Sep. 24, 5-9AM : Half of Second Life will receive 1.24.6
  • Thursday, Sep. 25, 5-9AM : The rest of Second Life will receive 1.24.6

Update 2008-09-18 11:11am: The rolling restart to deploy 1.24.6 has been postponed until next week, and will follow the schedule:

  • Monday, Sep. 22, 8-10PM : A pilot group of ~3000 regions will receive 1.24.6
  • Tuesday, Sep. 23, 5-9AM : Half of Second Life will receive 1.24.6
  • Wednesday, Sep. 24, 5-9AM : The rest of Second Life will receive 1.24.6

There will be a rolling restart this week to deploy server version 1.24.6 to Second Life. Release notes for what is changed may be found on this page on the Second Life Wiki. There are no new features in this release, only bug fixes. The version that will be deployed to Second Life is currently on the Preview Grid on regions running the Second Life Beta Server channel.

The rolling restart will follow this schedule:

  • Wednesday, 8:30-10PM: a pilot group of ~3000 regions will receive 1.24.6.
  • Thursday, 5-9AM: half of Second Life will receive 1.24.6.
  • Friday, 5-9AM: the rest of Second Life will receive 1.24.6

Each region will go down for about 5-10 minutes once during the rolling restart. If your region stays down for more than 20 minutes, please contact support.

Discussion of this rolling restart can be found in this thread in the Second Life forums.

[CLOSED] Upgrade to Land Store Taking Place 9am-11am PDT

Wednesday, September 17th, 2008

[Closed - 11:48pm PDT] - The upgrade to the Land Store has finished and is now live.

[UPDATE - 10:52 AM Pacific] - The upgrade to the land store will take a little longer than anticipated. It will continue to be offline beyond 11am. We will update here as soon as we have further information.

[UPDATE - 09:20 AM Pacific] - The upgrade to the Land Store is under way.

[12:08 AM Pacific] This is a reminder that our Land Store will be offline from 9am until 11am Pacific today, 17th September 2008. The store will be offline during this period. We will update you on progress with on this page once the deploy of the new Land Store begins.

See http://blog.secondlife.com/2008/09/15/minor-land-store-upgrade-on-wednesday/ for more information.

[RESOLVED] Temporary Database Issues

Wednesday, August 20th, 2008

[RESOLVED 5:07 PM PDT] The database has recovered and services should now be returning to normal.

[UPDATE 4:59 PM PDT] Logins have been re-enabled.

[UPDATE 4:05 PM PDT] We have temporarily disabled logins while this issue is investigated.

We are currently experiencing a problem with the database that may result in difficulties logging in, rezzing objects, completing transactions, and searching. We are aware of this issue and our ops team are working to resolve it as swiftly as possible. In the mean time, please avoid making important transactions or rezzing no-copy objects.

[RESOLVED] Logins and In-World Services are Intermittent for Some Residents

Wednesday, July 2nd, 2008

[RESOLVED – 18:37 PM PST] Logins have stabilized as well as all other issues. At this time everyone should be able to login again. Thank you for your patience.

[UPDATE – 18:26 PM PST] Continuing to have login problems. We are working to fix this as quickly as possible.

[UPDATE - 18:00 PM PST] Logging into Live Chat and Support Portal are also experiencing difficulties at this time. We will keep you posted on the status.

We have reports of sporadic unavailability of some services such as login not working, transactions failing, and land store down. We are aware of the problem and are working to bring these services back online as quickly as possible.

[CLOSED] Momentary disruption in voice chat in some regions

Friday, June 27th, 2008

[CLOSED 4:36 PM Pacific] 

Voice was down for approximately 30 minutes today.

The problem we observed was similar to the problem we encountered at the SL Birthday Party on Monday.  
Today and Monday, certain voice channels became overloaded and essentially caused other voice channel access to back up.
We worked quickly with our voice partner to increase capacity on the voice servers.  As capacity increased the voice systems caught back up, leading to a total outage of approximately 30 minutes.  The rest of the time has been spent monitoring prior to calling an all clear.

***

[UPDATE 4:20PM Pacific] We’re checking with our vendor to get a current assessment on the partial voice chat outage.

***

Several calls have come in regarding voice chat outages in various regions.  We’re working with the vendor now to assess and repair the problem.