I'm checking out the trial and was curious if any of you long time capsuleers could answer a question for me. When the servers go down like this, does it stop the clock on your training cue? Thanks.
"You'll never win an argument with an idiot because he is too stupid to recognize his own defeat." ~Anonymous
What makes up the live EVE Cluster and how it's all done is something of a mystery to many who have speculated what it's made of and how it's all connected together.
As you may know the Tranquility (TQ) cluster will be down for maintenance on Wednesday, June 23, 2010 from 0900 to 1500 UTC.
With a migration and a bit of a redesign on the way I thought it was about time to deliver the facts on what it is and will be.
Step one: A cozy new home
TQ has morphed and adjusted over the years as much as EVE Online has. It's gotten to the point were a couple of cabinets simply don't handle it anymore. So, this first step is to move TQ to a bigger place. We'll still be in the same datacenter and connecting to you from multiple networks to ensure the best performance, but this time with a lot more space and power and room to grow.
The new space is a whopping 79kW of power across 12 cabinets. With the larger space and added power, we can now aggregate TQ, Singularity, and the ancillary EVE Services (web, forums, account management, etc.) into a single location in the datacenter. This will provide better network connectivity, fewer intermediary devices and increased capacity.
As with any dense computer solution like our blade servers, heat is always a major concern. Sure, we get great management tools and reduced physical space requirements, but we still have to cool the servers. To do this we've moved from an ambient cooled system (basically the open room temperature is managed but not funneled direct to server intakes) to a completely self-contained, closed aisle cooling system. Cold air from the center of the aisle is force-fed into the cabinets reducing the loss or wasted cool air significantly and helping to focus cold air where it's needed most. This takes the industry standard "hot aisle/cold aisle" designs a step further without having to do anything crazy like running servers under nitrogen pools (although that is pretty cool).
Step two: Networking to 9000
Most of the traffic on the network in TQ is happening between the servers on the internal network. While the routers we use are quite powerful (Cisco 7600's with the RSP720 route processors), our internal switching needed a kick in the pants. With the move we are going to be adding about 800% capacity to our side to side network along with some really nice Cisco Distributed Forwarding Cards (DFC3) to the network blades themselves to help reduce the latency and reduce burden on the supervisor cards that run the switches.
Step three: Pics or it didn't happen
We are going to continue the information sharing about the infrastructure that makes EVE work on the next installment. Although not everyone gets excited about cabinets and a datacenter, there are a few that do. I personally keep them posted on my wall at home. This is meant to be the first of many installments as we continue to improve the infastructure that EVE runs on.
Step four: But, how does this help me get my ship back?
The increase in Layer 2 switching capacity, reducing in latency through Distributed Forwarding, and the extra cold hamsters will have an impact on the ability to reduce overall latency in EVE. It is not a single solution, but a good foundation where core infrastructure can be eliminated as a possible concern.
The next tech installment will have more details on Remapping EVE, the next level of Fleet Fighting and better prediction of hot spots for dedicated nodes.
TQ Tech Details: (Not the whole system, just what runs TQ)
Servers
64 x IBM HS21
2x Dual Core 3.33GHz CPU's
32GB of RAM Each
1x72GB HDD Each
2 x IBM X3850 M2's
2x Six Core 2.66GHz
128GB of RAM
4 x 146GB HDD
Cores
- 280 total Cores
- ~1 THz
RAM
- 2.3TB of Total RAM
Storage
- 4.8TB of Local Storage
- 2TB of SSD SAN
- 256GB of RAM SAN
Network
- Gigabit Ethernet
- 4Gb/s Fiber Channel
So what will June 23 look like?
Here's our current downtime schedule for when TQ will be offline:
0900: All EVE Services go offline. (Web, Forums, Test Servers, EVE Gate, TQ, basically everything hosted in London)
1200: EVE Online web, secure and Test Servers come back online. (all network services reestablished in London. Only TQ should still be down at this time)
Experienced pilots will be aware of these downtimes and adjust training queues to account for this, so instead of doing a skill due to finish teh day of patching they will put up a longer train time skill, just incase it goes BOOM and takes a bit longer to get servers up, you are not losing training time
Experienced pilots will be aware of these downtimes and adjust training queues to account for this, so instead of doing a skill due to finish teh day of patching they will put up a longer train time skill, just incase it goes BOOM and takes a bit longer to get servers up, you are not losing training time
Yep, I'm finally training Drone Interface V so im good lol.
Experienced pilots will be aware of these downtimes and adjust training queues to account for this, so instead of doing a skill due to finish teh day of patching they will put up a longer train time skill, just incase it goes BOOM and takes a bit longer to get servers up, you are not losing training time
EVE Online is currently unavailable as we move our server, Tranquility, to a new location. This also requires us to close down our website. The server will be down from 09:00 to 17:00 UTC, our website is expected to be up as early as 15:00 UTC though.
We thank you for your patience and look forward to seeing you join us in EVE Online!
Experienced pilots will be aware of these downtimes and adjust training queues to account for this, so instead of doing a skill due to finish teh day of patching they will put up a longer train time skill, just incase it goes BOOM and takes a bit longer to get servers up, you are not losing training time
Experienced pilots will be aware of these downtimes and adjust training queues to account for this, so instead of doing a skill due to finish teh day of patching they will put up a longer train time skill, just incase it goes BOOM and takes a bit longer to get servers up, you are not losing training time
25 days left on Minmatar BS 5...
I hate that skill.
Couple of years ago I forgot to update my clone and right after I decided to self destruct my pod (yeppers), I discovered that I had lost 20 days off of my Minny BS 5. I was a joy that day, let me tell you
"Mr. Rothstein, your people never will understand... the way it works out here. You're all just our guests. But you act like you're at home. Let me tell you something, partner. You ain't home. But that's where we're gonna send you if it harelips the governor." - Pat Webb
Experienced pilots will be aware of these downtimes and adjust training queues to account for this, so instead of doing a skill due to finish teh day of patching they will put up a longer train time skill, just incase it goes BOOM and takes a bit longer to get servers up, you are not losing training time
25 days left on Minmatar BS 5...
I hate that skill.
Couple of years ago I forgot to update my clone and right after I decided to self destruct my pod (yeppers), I discovered that I had lost 20 days off of my Minny BS 5. I was a joy that day, let me tell you
It's my 3rd BS 5.
The first is the worst... but Carrier 5 followed by Fighters 5. ow.
I always plan for 24 complete hours of downtime, whether they "say" it'll take shorter or not. That way, when it doesn't happen, you aren't disappointed. The one thing that kindof irks me about CCP, however, is that they always underproject the time it takes to complete patches or these types of maintenance issues. Honestly, they need to overproject and then have the server come up early rather than underproject and then frustrate half the playerbase.
The other thing that I *think* they don't have is a proper plan for these days. I mean, anyone who's ever moved an office knows that basically you show up, power everything down, load it into a U-Haul, get to the next location, and figure out how to reconnect everything on a sortof ad hoc basis. However, when you run a 23/7 MMO and you put out guidance to your customers, you need to: (a) make sure you are being fair and accurate; and (b) make sure you are being realistic. This means you need a plan, and you need to conduct rehearsals.
In the army, we have a process called the troop leading procedures. They go as follows: receive the mission; issue a warning order; make a tentative plan; conduct reconnaissance; complete the plan; issue an operations order; supervise and refine.
I don't work at CCP so I can't speak with authority as to how they do business. But I see the results and what it says to me is they run a somewhat disorganized and chaotic business. They need to have a discrete and definite plan for downtimes and it should be put out in writing. Timelines, etc. And timelines are not ever realistic unless you conduct a proper rehearsal and actually SEE how much time it is going to take. Seems that they tend to just pick a number out of thin air and say "that sounds good."
Anyway, this really isn't a rant because I was not planning on the server being up until tomorrow anyway, although I will admit I was *hoping* it would be done by now. Oh well.
but this is an exciting move.......one that might make things much more stable so ......i would suggest keeping stumm and waiting for much improvements.....if not ....come pvp my lame caldari ass hehehe
Its not EVE been down thats the problem, its the lack of communication between CCP and their paying customers. Since returning from work there has been no announcement anywhere that I can find, of when its likely to be back up. Web site is down , game log in screen says absolutely nothing.
Is there any notification anywhere of the the latest situation or are we just in the same old mushroom situation that they like us to be in?
Well around a hour ago there was a mesage that its up by 18:00 UTC (well it used to be 15:00 then updated to 17:00 and then 18:00) and yet its already 1 hour overtime.
And according to themessage .. website was supposed to be online 16:00 UTC
and im not annoyed by the fact that i cant play since iv never played.
I wanted t odownload the client and start the trial ... but now im having 2nd thoughts,
ah, so the same ol same ol lets think of a number, double it, add a bit and tell them that time scenario. I know if i did that in my line of work, id be looking for another job rather smartish but they seem to get away with it everytime.
Cheers for the reply
EDIT:- Nah dont let this stop you d/l a trial. Recently its been a hell of a lot more stable than it was when i first started around 12 months ago. Also this update could/should iron out a few of the problems that were left. IMO the only problem is that CCP have no idea of how to plan or announce a deadline.........games great, its CCPs customer suppotr skills that are lacking at times.
Well around a hour ago there was a mesage that its up by 18:00 UTC (well it used to be 15:00 then updated to 17:00 and then 18:00) and yet its already 1 hour overtime.
And according to themessage .. website was supposed to be online 16:00 UTC
and im not annoyed by the fact that i cant play since iv never played.
I wanted t odownload the client and start the trial ... but now im having 2nd thoughts,
It'll be up tomorrow. It always takes longer to get TQ up than they say it will.
Comments
Servers are down until 15.00 UTC due to tranquility being moved to new hardware and teh data core being upgraded
http://upload.wikimedia.org/wikipedia/en/a/ab/Norsefire-logo.png
yes they are updating the servers and if im not wrong it was due back up at 1500eve but dont count on it
http://www.worldtimeserver.com/current_time_in_UTC.aspx
its only 10.12 UTC, still another 5 hours till its due back up (assuming its 15.00 UTC as I thought it was 17.00 UTC)
http://upload.wikimedia.org/wikipedia/en/a/ab/Norsefire-logo.png
I stand corrected
(thansk to Eve Fansite)
The Tranquility cluster will be down for maintenance on Wednesday, June 23rd 2010 from 0900 to 1500 UTC.
http://upload.wikimedia.org/wikipedia/en/a/ab/Norsefire-logo.png
Server status can be seen here or here
I'm checking out the trial and was curious if any of you long time capsuleers could answer a question for me. When the servers go down like this, does it stop the clock on your training cue? Thanks.
"You'll never win an argument with an idiot because he is too stupid to recognize his own defeat." ~Anonymous
Server down time does NOT stop training time
From Scrapheap Challenge forums:
What makes up the live EVE Cluster and how it's all done is something of a mystery to many who have speculated what it's made of and how it's all connected together.
As you may know the Tranquility (TQ) cluster will be down for maintenance on Wednesday, June 23, 2010 from 0900 to 1500 UTC.
With a migration and a bit of a redesign on the way I thought it was about time to deliver the facts on what it is and will be.
Step one: A cozy new home
TQ has morphed and adjusted over the years as much as EVE Online has. It's gotten to the point were a couple of cabinets simply don't handle it anymore. So, this first step is to move TQ to a bigger place. We'll still be in the same datacenter and connecting to you from multiple networks to ensure the best performance, but this time with a lot more space and power and room to grow.
The new space is a whopping 79kW of power across 12 cabinets. With the larger space and added power, we can now aggregate TQ, Singularity, and the ancillary EVE Services (web, forums, account management, etc.) into a single location in the datacenter. This will provide better network connectivity, fewer intermediary devices and increased capacity.
As with any dense computer solution like our blade servers, heat is always a major concern. Sure, we get great management tools and reduced physical space requirements, but we still have to cool the servers. To do this we've moved from an ambient cooled system (basically the open room temperature is managed but not funneled direct to server intakes) to a completely self-contained, closed aisle cooling system. Cold air from the center of the aisle is force-fed into the cabinets reducing the loss or wasted cool air significantly and helping to focus cold air where it's needed most. This takes the industry standard "hot aisle/cold aisle" designs a step further without having to do anything crazy like running servers under nitrogen pools (although that is pretty cool).
Step two: Networking to 9000
Most of the traffic on the network in TQ is happening between the servers on the internal network. While the routers we use are quite powerful (Cisco 7600's with the RSP720 route processors), our internal switching needed a kick in the pants. With the move we are going to be adding about 800% capacity to our side to side network along with some really nice Cisco Distributed Forwarding Cards (DFC3) to the network blades themselves to help reduce the latency and reduce burden on the supervisor cards that run the switches.
Step three: Pics or it didn't happen
We are going to continue the information sharing about the infrastructure that makes EVE work on the next installment. Although not everyone gets excited about cabinets and a datacenter, there are a few that do. I personally keep them posted on my wall at home. This is meant to be the first of many installments as we continue to improve the infastructure that EVE runs on.
Step four: But, how does this help me get my ship back?
The increase in Layer 2 switching capacity, reducing in latency through Distributed Forwarding, and the extra cold hamsters will have an impact on the ability to reduce overall latency in EVE. It is not a single solution, but a good foundation where core infrastructure can be eliminated as a possible concern.
The next tech installment will have more details on Remapping EVE, the next level of Fleet Fighting and better prediction of hot spots for dedicated nodes.
TQ Tech Details: (Not the whole system, just what runs TQ)
Servers
64 x IBM HS21
2x Dual Core 3.33GHz CPU's
32GB of RAM Each
1x72GB HDD Each
2 x IBM X3850 M2's
2x Six Core 2.66GHz
128GB of RAM
4 x 146GB HDD
Cores
- 280 total Cores
- ~1 THz
RAM
- 2.3TB of Total RAM
Storage
- 4.8TB of Local Storage
- 2TB of SSD SAN
- 256GB of RAM SAN
Network
- Gigabit Ethernet
- 4Gb/s Fiber Channel
So what will June 23 look like?
Here's our current downtime schedule for when TQ will be offline:
0900: All EVE Services go offline. (Web, Forums, Test Servers, EVE Gate, TQ, basically everything hosted in London)
1200: EVE Online web, secure and Test Servers come back online. (all network services reestablished in London. Only TQ should still be down at this time)
1500: TQ back online
Experienced pilots will be aware of these downtimes and adjust training queues to account for this, so instead of doing a skill due to finish teh day of patching they will put up a longer train time skill, just incase it goes BOOM and takes a bit longer to get servers up, you are not losing training time
http://upload.wikimedia.org/wikipedia/en/a/ab/Norsefire-logo.png
Yep, I'm finally training Drone Interface V so im good lol.
Playing: Rift, LotRO
Waiting on: GW2, BP
25 days left on Minmatar BS 5...
Give me liberty or give me lasers
From eveonline.com:
EVE Online is currently unavailable as we move our server, Tranquility, to a new location. This also requires us to close down our website. The server will be down from 09:00 to 17:00 UTC, our website is expected to be up as early as 15:00 UTC though.
We thank you for your patience and look forward to seeing you join us in EVE Online!
Uggh I hate BS V skills.
Had to suffer through that with Gallente BS.
Needed it for my Neut Domi fit.
Playing: Rift, LotRO
Waiting on: GW2, BP
I hate that skill.
Couple of years ago I forgot to update my clone and right after I decided to self destruct my pod (yeppers), I discovered that I had lost 20 days off of my Minny BS 5. I was a joy that day, let me tell you
"Mr. Rothstein, your people never will understand... the way it works out here. You're all just our guests. But you act like you're at home. Let me tell you something, partner. You ain't home. But that's where we're gonna send you if it harelips the governor." - Pat Webb
It's my 3rd BS 5.
The first is the worst... but Carrier 5 followed by Fighters 5. ow.
Give me liberty or give me lasers
It's way past 1500 UTC, and they increased downtime to 1800. It's about 1630 or so now, and website is still down.
They can take all the time they want.
If this fixes most of the lag issues and brings Eve back to Pre-Dominion Stability im all for waiting a few more hours.
Playing: Rift, LotRO
Waiting on: GW2, BP
I always plan for 24 complete hours of downtime, whether they "say" it'll take shorter or not. That way, when it doesn't happen, you aren't disappointed. The one thing that kindof irks me about CCP, however, is that they always underproject the time it takes to complete patches or these types of maintenance issues. Honestly, they need to overproject and then have the server come up early rather than underproject and then frustrate half the playerbase.
The other thing that I *think* they don't have is a proper plan for these days. I mean, anyone who's ever moved an office knows that basically you show up, power everything down, load it into a U-Haul, get to the next location, and figure out how to reconnect everything on a sortof ad hoc basis. However, when you run a 23/7 MMO and you put out guidance to your customers, you need to: (a) make sure you are being fair and accurate; and (b) make sure you are being realistic. This means you need a plan, and you need to conduct rehearsals.
In the army, we have a process called the troop leading procedures. They go as follows: receive the mission; issue a warning order; make a tentative plan; conduct reconnaissance; complete the plan; issue an operations order; supervise and refine.
I don't work at CCP so I can't speak with authority as to how they do business. But I see the results and what it says to me is they run a somewhat disorganized and chaotic business. They need to have a discrete and definite plan for downtimes and it should be put out in writing. Timelines, etc. And timelines are not ever realistic unless you conduct a proper rehearsal and actually SEE how much time it is going to take. Seems that they tend to just pick a number out of thin air and say "that sounds good."
Anyway, this really isn't a rant because I was not planning on the server being up until tomorrow anyway, although I will admit I was *hoping* it would be done by now. Oh well.
s
of course its down
but this is an exciting move.......one that might make things much more stable so ......i would suggest keeping stumm and waiting for much improvements.....if not ....come pvp my lame caldari ass hehehe
I have shit to do anyway.
Twenty four hours of downtime could do some of us some good lol
Playing: Rift, LotRO
Waiting on: GW2, BP
Its not EVE been down thats the problem, its the lack of communication between CCP and their paying customers. Since returning from work there has been no announcement anywhere that I can find, of when its likely to be back up. Web site is down , game log in screen says absolutely nothing.
Is there any notification anywhere of the the latest situation or are we just in the same old mushroom situation that they like us to be in?
Well around a hour ago there was a mesage that its up by 18:00 UTC (well it used to be 15:00 then updated to 17:00 and then 18:00) and yet its already 1 hour overtime.
And according to themessage .. website was supposed to be online 16:00 UTC
and im not annoyed by the fact that i cant play since iv never played.
I wanted t odownload the client and start the trial ... but now im having 2nd thoughts,
ah, so the same ol same ol lets think of a number, double it, add a bit and tell them that time scenario. I know if i did that in my line of work, id be looking for another job rather smartish but they seem to get away with it everytime.
Cheers for the reply
EDIT:- Nah dont let this stop you d/l a trial. Recently its been a hell of a lot more stable than it was when i first started around 12 months ago. Also this update could/should iron out a few of the problems that were left. IMO the only problem is that CCP have no idea of how to plan or announce a deadline.........games great, its CCPs customer suppotr skills that are lacking at times.
It'll be up tomorrow. It always takes longer to get TQ up than they say it will.
Give me liberty or give me lasers
now says back up at 21:00.......pity it doesnt say which day lol
ah well, guess i ought to look outside the front door and see whats happening in the world.