Second Life Grid Status Reports

Update 2008-10-02 08:43am : the rolling restart is now complete.

Update 2008-10-02 05:18am : the second-half rolling restart has begun.

Update 2008-10-01 07:39am : the first-half rolling restart is complete.

Update 2008-10-01 05:08am : the first-half rolling restart has begun. This morning, odd numbered hosts are being restarted.

Update 2008-09-30 08:55pm : the pilot deploy to ~3000 regions is complete.

There will be a rolling restart to deploy server version 1.24.7 to Second Life this week. 1.24.7 is currently deployed to the “Second Life Beta Server” channel of the Preview Grid. The changes in this version may be found on the Second Life Beta Server Release Notes page on our public wiki.

Each region will be restarted but once, and should only remain down for 5-10 minutes. If your region stays down for more than 20 minutes, please contact support.

We will follow our usual three-stage deploy with this rolling restart. The current anticipated schedule for the deploy is:

• Tuesday, Sep 30, 8:00-10:00PM: A pilot group of ~3000 regions will be restarted.
• Wednesday, Oct 1, 5:00-9:00AM: Half of Second Life will be restarted.
• Thursday, Oct 2, 5:00-9:00AM: The rest of Second Life will be restarted.

Discussion of this rolling restart will be held in this thread on the Second Life forums.

Update 09/13 06:00AM : 1.24.5.96115 is everywhere on the grid now.

Update 09/13 05:05AM : We are going to revert the pilot group from 1.24.5.96378 to the more-stable 1.24.5.96115 that is on the rest of the grid, beginning shortly.

Update 09/12 08:47AM : The second-half rolling restart completed about 45 minutes ago.

Update 09/12 05:22AM : The second-half rolling restart is now beginning.

Update 09/11 10:15PM : the first-half rolling restart is complete.

Update 09/11 07:51PM : based on a slightly higher crash rate for 1.24.5.96378, we will be deploying the previous version (1.24.5.96115) as 1.24.5 to the entire grid. The first-half rolling restart is beginning now.

Update 09/11 01:00PM : the pilot group of ~3000 regions as been updated to 1.24.5.96378.

Update 09/11 05:13AM : our Brighton developers fixed a couple of more small bugs, the patches for which we want to get into the 1.24.5 update. We are going to put those two fixes through QA today, and redo the pilot roll. That puts us on this schedule:

• 09/11 before noon : redo the pilot roll to the same group of ~3000 regions
• 09/11 7-11pm : a rolling restart of half of the regions in Second Life
• 09/12 5-9am : a rolling restart of the rest of Second Life

Update 09/10 09:57PM : the pilot roll to ~3000 regions is complete.

We will be deploying server version 1.24.5 to Second Life this week. This version includes no new major features, but fixes bugs with the current 1.24.4 server that is on Second Life. More details of what’s been changed can be found on the Second Life Wiki’s release notes page for 1.24.

This rolling restart will follow our usual three-stage deploy:

• Wed, 09/10, 9pm : A pilot group of ~3000 regions will be restarted.
• Thu, 09/11, 5-9am : Half of Second Life will be restarted.
• Fri, 09/12, 5-9am : The remainder of Second Life will be restarted.

As with all rolling restarts, each region will receive warnings starting 5 minutes before they are restarted. No region should stay down more than 10 or 20 minutes; if your region stays down longer than 20 minutes, please contact support.

Further discussion of the 1.24.5 server deploy can be found in this thread in the Second Life forums.

Update 2008-09-03 10:00pm : The rolling restart is complete.

Update 2008-09-03 07:25pm : The second-half rolling restart is beginning now.

Update 2008-09-03 08:51am : The first-half rolling restart is complete.

Update 2008-09-03 05:37am : The first-half rolling restart is in progress. Odd-numbered hosts that do not yet have 1.24.4 are being restarted this morning.

Update 2008-09-02 09:37pm : The pilot roll is complete.

Update 2008-09-02 08:47pm : the pilot roll to 2859 regions has begun.

We will be performing a rolling restart to deploy server version 1.24.4 to Second Life. This server deploy does not contain any new features, but has fixes for some bugs found in the deploy of server version 1.24 that was completed last week.

Release notes for version 1.24 (and 1.24.4) can be found here on the Second Life Wiki.

The rolling restart will follow our usual three-stage deploy:

• Tuesday, 09/02, 08:00PM : A pilot group of ~3000 regions will receive 1.24.4 in a rolling restart.
• Wednesday, 09/03, 05:00-09:00AM : Half of the grid will receive 1.24.4 in a rolling restart.
• Wednesday, 09/03, 07:00-11:00PM : The remainder of Second Life will receive 1.24.4 in a rolling restart.

As is usual with rolling restarts, regions will be down typically for 5-10 minutes. If your regions stays down for more than 20 minutes, please contact support. There is no way to delay the restart of a region once the rolling restart has begun.

Discussion of this rolling restart will happen in the Second Life forums here.

For more information, see this post on the main blog.

The version to be deployed is currently on the “Second Life Beta Server” channel of the Preview Grid (Aditi).

(This roll has been superceded by the rolling restart to revert 1.23.0 to 1.22.4.)

After this morning’s rolling restart of half the grid, a problem was discovered with “floating text” over objects (SVC-2633). We have fixed that problem. We will be rolling out the fixed server version tonight (Friday night) at 8PM to all regions currently running server version 1.23.0. (That includes regions restarted last night and this morning.) The remaining regions (those currently on 1.22.4) will be restarted tomorrow morning for the deploy. For more information, see this post on the main blog.

The new server version which started rolling out today (currently deployed to half of the regions) has been found to contain a bug related to floating text, which results in text running together into one line.
Affected residents can reset scripts to make the text appear correctly, which will last until the region is restarted.

We believe we have identified the source of the bug, and will be testing a fixed version immediately.

Related Jira: https://jira.secondlife.com/browse/SVC-2633

Related blog: Rolling Restart Thu/Fri/Sat July 10-12 — Please Test Server Version 1.23 on the Preview Grid Now!

We are postponing the rolling restart again by a day. We may need to postpone it until next week; we will make that call tomorrow (Thursday). At the moment, the plan is to have the pilot roll Thursday evening, followed by half-grid rolls on Friday and Saturday morning. Please see the post on the main blog for more information about this rolling restart.

The rolling restart this week has slipped one day. We will do a pilot roll Wednesday evening, 07/09. Thursday morning between 5 and 9AM, we will deploy server version 1.23 to half of the regions on the grid. Friday morning between 5 and 9AM, we will deploy server version 1.23 to the rest of the grid. For more information, please see this post on the main blog.

For more information, please see this post on the public blog.

For more information, please see this post on our main blog.