PDA

View Full Version : ~~ MESSAGE FROM MORPH: The 6/17 Site Outage ~~


Morpheus
06-18-2009, 11:17 AM
Hello fellow HSG.com Members,

First, i apologize for this being the earliest that I could get to you. When the site came back up, i did not have access to a laptop to be able to communicate to you. I only had my phone, which only gave me limited access. This morning was the earliest that I could get back online to discuss.

Secondly, to what actually happened. Unfortunatlely, the site technical admins and I are completely at a loss. Certain tables/elements from the DB (database) started to disappear yesterday morning, and we cannot ascertain the reason.

It first happened yesterday morning, about 11am. I caught it the first time, and recreated the DB and the site was back up within about 30 minutes.

An hour or so later, it happened again but to a greater degree, more than i was able to jump in and recreate on my own in a timely fashion. So, i had called in some 3rd party, technical back-up to assist me restore the site. With all of us working on it, it still took a majority of the day.

Ok, to the actual community data. We take back-ups of all site data at least once a day, without question. But apparently, the data itself was starting to get somewhat corrupt over a week ago, but it wasn't to the point that it was bringing the site down. So, since we weren't aware of any issues, we continued to take backups as normal, not knowing that the backups we were taking were slightly corrupted themselves.

When we had to do the restore yesterday, and we attempted to restore from the previous days data, was when we saw the minor DB descrepancies/corruptions, and realized that restoring with THAT data would possibly cause more damage. So we had to back-track, one day at a time to find the last day with a CLEAN set of data. Which turned out to be a week or so ago, thus losing a weeks worth of topics, posts, messages, etc.

Trust me, that was the SUPER last resort. We tried every which way that we could to retain all of the information. I know there has been a lot going on.....posts, topics, event scores, hot topics, leaderboards, etc. You don' t know how hard we tried to keep everything intact, but it just was impossible to do.

In the next 5-7 days to come, drastic steps are going to be taken to stabalize the site. We will be moving the site to a new software platform. The one we are currently build on has just been too unstable for me to be 100% comfortable with it.

Anyways, thats where we are at now. I am holding off on doing any more updates or technically "touching" this site any further. I am setting my eyes on the transition to the new software as soon as possible. I will keep everyone WELL informed on the transition and letting you know what to expect and when to expect it. I'm thinking on trying to do it though within the next 5-7 days.

Although it will be somewhat of a change from what you are used to, I believe that you will completely embrace the new system once it is up and running! Several of the features include an INTEGRATED ladder system (no more logging into a different site for ladder), a WORKING arcade that will have its own Tournament system, an awards/trophy room....and those are just to name a few.

I am truly sorry for the data that was lost, not only the posts and topics in the public area, but the hard work that was going on behind the scenes in the staff areas. Let me know if there's anything i can do on my end to assist.

As always, looking foward to your feedback!

Morph.

pypelyr
06-18-2009, 12:57 PM
ya'll are working very hard that is so obvious! Thank you for this site and all your hard work! looking forward to whatever comes! ;D

the_smug_one
06-18-2009, 02:10 PM
Sounds like a frustrating situation Morph, but many thanks to you and your staff for all the hard work you continue to put into this site.

Optimus_Prime
06-18-2009, 02:21 PM
Morph, you and Babjusi did all you could do (and it was quite a lot from the sound of it). Thanks for letting us know what happened and what to expect in the coming days. Will be exciting to get a more stable, feature-rich site. Hope it doesn't hit your pocketbook too hard!

yasdad
06-18-2009, 03:55 PM
:o Ouch! That sounds like a major headache there :o Glad you managed to find a clean backup that was only a few days old :)

I guess that means I'll have to change my avatar again >:( j/k ;D

Looking forward to the new site :)

fried_blowney
06-18-2009, 04:10 PM
I think the biggest thing to come out of this is roughdawg lost 1000 posts :D

PizzaGuy8042
06-18-2009, 04:12 PM
All your hard work is always greatly appreciated. Thank you so much!

asg28
06-18-2009, 06:22 PM
Thanks for keeping us informed morpheus and thanks for all your hard work behind the scenes

jleviathan
06-18-2009, 08:47 PM
Must be a daunting task keeping up this site
thank you for your hard work