[September 24 1:38PM Pacific] This issue has been resolved. Inworld deposits should now show on Xstreet balances without delay.
[Sep 23, 8:04PM Pacific] We are currently experiencing an issue with XStreetSL that may cause some residents’ in-world deposits to be delayed in appearing in their XStreet balances. Our engineers are working on resolving the issue.
Update 01-23 10:37am : The rolling restart is complete.
Update 01-23 07:05am : The second-half rolling restart has begun.
Update 01-22 09:21am : The first-half rolling restart is complete.
Update 01-22 07:08am : The first-half full rolling restart has begun. Regions on odd-numbered hosts are being restarted today. This includes the pilot regions, which are going to be updated to the latest and greatest 1.25.4.108489
Update 01-21 09:01am : the pilot rolling restart is complete
We will be doing a rolling restart to deploy Second Life Server version 1.25.4 from Wednesday through Friday, Jan 21-23. This version is currently on the “Second Life Beta Server” regions of the Preview Grid, and is a slightly modified version of the 1.25.3 server that is currently deployed to about 10% of the grid.
The schedule will be as follows:
As with all rolling restarts, each region will be restarted once as a result of the rolling restart. Most regions will restart over the course of several minutes. If your region stays down for more than 30 minutes, please contact support. Once the rolling restart has begun, there is no way to delay the restart for any given region.
Discussion of this rolling restart will occur in this forum thread.
[Resolved 11:51AM PST] Services should now be returning to normal.
[Update 11:45AM PST] Logins have been re-enabled.
Logins to Second Life have just been disabled, and residents who are in Second Life at present are advised not to rez any no-copy objects or make any important transactions, while we work on quickly resolving an issue with our database. We will keep this post updated until this problem is resolved.
[Resolved 5:02 PM PST] We have re-enabled logins and services should be returning to normal.
Logins to Second Life have been disabled while we work on fixing a database problem that is affecting teleports, transactions, and rezzing. We advise residents not to make any important transactions or rez any no copy objects until we resolve this issue. We will keep this post updated as the situation evolves.
[RESOLVED 10:28AM PST] All services should now have returned to normal.
[UPDATE 10:13AM PST] Logins have been re-enabled and the maintenance is nearly complete.
[UPDATE 9:39AM PST] We have temporarily disabled logins while we work on solving this quickly.
Residents may experience teleport and asset failures while we perform some brief server maintenance at our Dallas facility. We do not expect this to last very long, but in the meantime we advise residents not to make any important transactions or rez any no copy objects. We will keep this post updated until the issue is resolved.
Update 2008-08-29 12:42PM : The rolling restarted completed at about noon today.
Update 2008-08-29 07:11AM : The rolling restart to update the remainder of the grid to 1.24 began at 6:46 this morning.
Update 2008-08-28 07:29AM : The rolling restart to 1/4 of the grid is complete.
Update 2008-08-27 09:39PM : Server version 1.24.3 is now on the 1/10-grid “pilot” group of hosts.
Update 2008-08-27 03:54PM : Things are looking good. We’re going to deploy 1.24.3 to our pilot regions sometime this evening (probably after 7:30PM). If nothing alarming comes up from that pilot deploy, then tomorrow morning between 5 and 9AM, we will deploy 1.24.3 to 1/4 of the grid, and Friday morning between 5 and 9AM, we will deploy 1.24.3 to the remainder of the grid.
Update 2008-08-26 10:51PM : Our “pilot” group of ~3000 regions is now on 1.24.2, a bugfixed version. We anticipate one more roll to this pilot group (probably tomorrow) of 1.24.3, after which a roll to the rest of the grid will be announced.
Update 2008-08-25 05:11PM : We are still investigating some of the crash modes we’ve seen in 1.24.1, and will not be deploying 1.24 on any further regions tomorrow. On the morning of Tuesday, August 26, it is likely that all regions currently on 1.24.1 will receive a rolling restart to deploy a patched version (1.24.2). Updates will be posted here as they become available.
Update 2008-08-25 11:38AM : Approximately 1/10 of the grid is now running 1.24.1.
Update 2008-08-22 04:48PM: The ~500 regions that were on 1.24.0 are now on 1.24.1, and that version appears better. We are going to let them bake over the weekend, and on Monday will make sure that all still looks well. If it does, we will proceed with a rolling restart to deploy 1.24.1 to the rest of the grid according to the schedule:
Update 2008-08-21 10:58AM : About 520 regions in Second Life are currently running version 1.24.0. We’ve put the rest of the deploy on hold as we investigate some issues that have shown up with the pilot roll.
We will be doing a rolling restart from Wed., Aug 20, through Fri., Aug 22, to deploy server version 1.24 to Second Life. This rolling restart will proceed in three stages:
This server version has been extensively tested in our testing environments, so we do not anticipate problems. However, because the production Second Life environment is so much larger than any possible testing environment, it’s possible that problems we haven’t been able to find yet will arise. Based on the severity of those problems, we will halt or revert the roll after one of the first two stages if we deem that necessary.
Each region will receive warnings starting 5 minutes before it is to be restarted. Each region should not be down for more than 10 minutes or so; if your region stays down for more than 20 minutes during the rolling restart, please contact support.
Release notes describing what is new in this server version may be found here.
This release includes the introduction of the Mono scripting infrastructure. More about that may be found in this post by Babbage Linden on our main blog.
Discussion of this rolling restart is ongoing in this thread in the Second Life forums.
[RESOLVED - 09:35am PDT] The problem seems to be resolved but our team is continuing to look at the root cause.
[UPDATE - 09:30am PDT] Logins re-enabled.
We have temporarily disabled logins.
We are currently experiencing a problem with the database that may result in difficulties logging in, teleporting, rezzing objects, We’re aware of this issue and our ops team are working to resolve it as swiftly as possible. Meantime, please avoid making important transactions or rezzing no-copy objects.
[Resolved 7:49pm PDT] The payment system is back online and the land store and LindeX website have both been re-enabled.
[Update 6:35pm PDT] We have disabled the LindeX website until this issue is resolved. We apologize for any inconvenience this may cause.
Due to a hardware failure at one of our merchant provider’s data centers, land buys, L$ buys, and payment method updates are not working at this time. We have disabled the land store because of this issue. We are working with our merchant provider to resolve this as soon as possible, and will update this post as the situation progresses.
PayPal has notified us of their intention to perform routine maintenance Thursday, 7 August, between 11pm and midnight Pacific. During that hour, PayPal-backed LindeX transactions and other purchases will be declined.
[RESOLVED 08:54 am PDT] We have not gotten any further reports of ruthing or inventory not loading, we are officially declaring this problem dead. Thank you all for your continued patience on this prolonged hunt!
[UPDATE 08:16 am PDT] We do believe the problem has finally been defeated - yes, for real this time! - but we need your help to confirm this.
If you still are ruthed and have trouble accessing your inventory, and clearing your cache and relogging does not solve it, please let our suppor team know.
[REOPENED 07:12 am PDT] Continued reports are showing the problem has not been completely resolved and we are working on tracking down the problem.
[RESOLVED 06:10 am PDT] We’ve only received sporadic reports of continued problems. If the situation has not changed for you please contact support with the time and the specific symptoms in your region.
[UPDATED 04:35 am PDT] We believe the main problem to be fixed, but given the earlier recurrence we will leave this blog open for a bit longer and keep monitoring the situation closely.
If your avatar still won’t load fully and displays as ruthed or a white cloud, please try to clear your cache and relog, that should solve it.
[REOPENED 02:14 am PDT] Unfortunately, the all-clear was premature, and it turns out there are still network issues to be sorted. We will keep you posted.
[ALL CLEAR 1:35] Ops has called the All Clear on the networking issue affecting some residents and regions for the past few hours. If you are still experiencing avatar issues or teleporting problems, etc. please try clearing your cache and relogging.
[UPDATE 0:25]Ops is continuing to work to correct the networking issue affecting some residents. Please continue to monitor this post for updates.
[UPDATE 22:45]Ops is continuing to work to correct the networking issue affecting some residents. In addition to avatar rezzing and teleporting issues, we have received some reports of failing IMs for affected residents. Please monitor this post for updates as they occur.
[9:45PM Pacific] Our technicians are still working to resolve this issue, and this can take more than one hour of investigation. We will keep you posted as soon as we get further information on it.
[8:45PM Pacific] Many residents have reported that their clothing are not loading and they are seeing others as white clouds. There are also some other issues such as some regions down, inventory not loading and search not working. We are aware of these problems and are working to fix these issues as soon as possible.
[8:09PM Pacific] Please be aware that some residents are also experiencing issues with teleporting. We will update this blog as soon as we have more news.
[7:32PM Pacific] SL is experiencing some network difficulties. Some residents have reported problems with logging in. Please avoid conducting inworld transactions and purchasing lindens for the time being. We will update this space with details as soon as we get them.
Grab this RSS feed