Server Melt Down!!!

[admin]

Just had a mojor melt down at the secure facillity. We managed to backup the most recent databases just before all hell broke loose. Well everything is backuped now and no posts should have been lost. Deeply sorry for the outage but we also managed a complete makeover over all the backend systems on this site including a software update that should make for quite a performance boost.

Again very sorry for any inconvience,
Will
eThink.net

[Maggers]

Hi, it looks like the time on the board has changed. Can you clarify for me what time zone we're working from?

[admin]

working on it...

[admin]

Just seeing what time this post is marked as...

Will

[admin]

Ok looks like every thing is A OK!!! Finally... what a day...

[dejo]

For example:

http://www.repairmanjack.com/forum/showthread.php?t=1050&page=16&pp=10

The last page with posts is page 8, not 16. (Pages 9 thru 16 will be empty)

P.S. Also seems earliest posts are missing. Post #1 is definitely not the original post, since it wasn't written by me...

[jimbow8]

For example:

http://www.repairmanjack.com/forum/showthread.php?t=1050&page=16&pp=10

The last page with posts is page 8, not 16. (Pages 9 thru 16 will be empty)
That's weird.

P.S. Also seems earliest posts are missing. Post #1 is definitely not the original post, since it wasn't written by me...
No, it was by me! DOH!! :(

[Maggers]

Rats, the first few pages of the "Haiku Anyone?" thread are missing also.

That would include my explanation of the thread and how to do haiku.

Too bad.

[admin]

Was this before or after the vbulletin upgrade?

[Maggers]

Was this before or after the vbulletin upgrade?

Hmm...I didn't check until today. So I can't honestly say.

[Keith the Elder]

Was this before or after the vbulletin upgrade?

Yeah, it was there before the upgrade

[Maggers]

Yeah, it was there before the upgrade

We began losing pieces of the haiku thread before the upgrade? I didn't know that.

At over 50 pages, is that particular thread too big now? Should be begin another? Please advise.

[Maggers]

I just tried looking at "Favorite movies scenes - 1960," and it looks like pages 2 and 3 are missing.

I guess this is happening on several multipage threads?

[Ken Valentine]

Just had a mojor melt down at the secure facillity. We managed to backup the most recent databases just before all hell broke loose. Well everything is backuped now and no posts should have been lost. Deeply sorry for the outage but we also managed a complete makeover over all the backend systems on this site including a software update that should make for quite a performance boost.

Again very sorry for any inconvience,
Will
eThink.net

When I click onto a couple of threads (favorite movie scenes-1980's, and Haiku) I get, "Invalid post specified. If you followed a valid link, please notify the web master."

Well, here's my notification. :D

Ken V.

[admin]

check it out now.. performed some thread maintenance not sure if it worked though..

Will

[KRW]

What happened to the post counts? I know mine is off, and I think a lot of the others are off to.


KRW

[Ken Valentine]

check it out now.. performed some thread maintenance not sure if it worked though..

Will

No change. I still can't access those two threads; "Haiku Anyone," and "Favorite scenes -- 1980's."

Ken V.

[Ken Valentine]

What happened to the post counts? I know mine is off, and I think a lot of the others are off to.


KRW

Jim and I have lost about a thousand posts each.

(Well maybe about 700 in my case.)

Ken V.

[Maggers]

Jim and I have lost about a thousand posts each.

(Well maybe about 700 in my case.)

Ken V.


I lost about 500, too! They disappeared sometime during the last few hours.

My average daily posting was 10.22 or thereabouts; now it's down to 6. something.

[jimbow8]

I was searching through posts by certain several individuals and noticed that there seem to be big gaps between late August and early December of 2004 where there are no messages. I wonder if the data from that time period got corrupted or was lost or something.

[Maggers]

I wrote a comparison of "Demongsong" and "Black Wind" that earned what I considered to be high praise from FPW. And now it's gone! It was posted in a thread of it's own, and then I copied it to the "compendium" thread. And it disappeared from both places. Fiddlesticks! I wish I'd copied it somewhere on my computer for safe keeping.

[Keith the Elder]

I wrote a comparison of "Demongsong" and "Black Wind" that earned what I considered to be high praise from FPW. And now it's gone! It was posted in a thread of it's own, and then I copied it to the "compendium" thread. And it disappeared from both places. Fiddlesticks! I wish I'd copied it somewhere on my computer for safe keeping.


Yes, I remember that, it was brilliant

[jimbow8]

Let me guess, it was posted between late August and early December of 2004. :confused:

[Maggers]

Let me guess, it was posted between late August and early December of 2004. :confused:

(*sniff sniff*) Yes, Jim, it was posted sometime between October 2004, when I joined the board, and December, 2004.

And thanks, Keith, for the kind words. It's really nice to get that sort of feedback.

[admin]

I have a copy of the data base before the upgrade I can post it on ethink for comparison and to copy from..

Let me know if this is what the forum wants.

Will
eThink.net

[Maggers]

I have a copy of the data base before the upgrade I can post it on ethink for comparison and to copy from..

Let me know if this is what the forum wants.

Will
eThink.net


I'm not sophisticated around this stuff. I don't know what ethink is. :confused: Would I be able to go there and find a copy of what I'd written and copy it for my files?

[Blake]

I see some of my own old posts are gone, too. It'd be nice to have those back.

I don't mean to criticize, but as an IT person I have to wonder: Shouldn't there be a better backup system in place?

Blake

[Maggers]

I see some of my own old posts are gone, too. It'd be nice to have those back.

I don't mean to criticize, but as an IT person I have to wonder: Shouldn't there be a better backup system in place?

Blake


It seems many wonderful, complicated and zany threads have disappeared. That's too bad, because it's fun every once in awhile to go back and read them. Or restart them, should something suddenly become relevant again.

There was a huge thread on "religious debate," I think, which I found hysterical and mind boggling. I think it's disappeared. It began before I arrived in October but I posted something on that thread in October or early November.

[admin]

I see some of my own old posts are gone, too. It'd be nice to have those back.

I don't mean to criticize, but as an IT person I have to wonder: Shouldn't there be a better backup system in place?

Blake

The back ups are done dailly at 3 am in the morning est, and are done on rotating media. The real problem is the upgrade from vbulletin 3.0 r4 to 3.0.7
which apparently corrupted alot of the posts. We were given no warning when the upgrade commensed and after people start posting on it, you have a choice; Do you delete all new posts and restore from an earlier file, or do you continue with the database the way it is and drop old corupted posts?

As for what i mean about posting the old forum version on ethink, Is that I put the vbulletin ver 3.0 r4 on http://www.ethink.net/rj (not there yet) so that people may view missing posts so that they can copy and past the info into the new forum.

Will
eThink.net

[Maggers]

As for what i mean about posting the old forum version on ethink, Is that I put the vbulletin ver 3.0 r4 on http://www.ethink.net/rj (not there yet) so that people may view missing posts so that they can copy and past the info into the new forum.

Will
eThink.net


Thanks, Will. You'll let us know when we can go to that link?

[Ken Valentine]

I have a copy of the data base before the upgrade I can post it on ethink for comparison and to copy from..

Let me know if this is what the forum wants.

Will
eThink.net

I'm still getting, "Invalid post specified." when I try to click onto a number of threads.

Ken V.

[Blake]

Shouldn't it be possible to merge the two databases using Impex? See here (http://www.vbulletin.com/forum/showthread.php?t=120125) and here (http://www.vbulletin.com/forum/showthread.php?t=116398), for example. As long as you've got good backups of the most recent copy of the database along with the last copy before the upgrade, the raw materials to merge them should be there.

Having a copy of the old database as an archive is okay, i.e., it's better than nothing, but it would be much better to just get all the old threads and posts back.

Blake

[Blake]

And I just noticed I'm down to 82 posts. I was over 150.

Blake

[jimbow8]

And I just noticed I'm down to 82 posts. I was over 150.

Blake
Yeah, both Ken and I were well over 2,000.

[Ken Valentine]

Yeah, both Ken and I were well over 2,000.

Yeah. I was up to 2,250-something, and you had at least 100 more than that.

Ken V.

[KRW]

Yeah. I was up to 2,250-something, and you had at least 100 more than that.

Ken V.

I figure about 500 off of mine. I finally broke 1000.

KRW

[Maggers]

I figure about 500 off of mine. I finally broke 1000.

KRW


Aww, man, you finally broke 1000 and lost 'em? That's just not right.

I was up to about 1600+ and went down to 1100+.

[Blake]

Aww, man, you finally broke 1000 and lost 'em? That's just not right.

I was up to about 1600+ and went down to 1100+.

Yeah, but I even dropped down to "Member" status from "Senior Member." :)

Seriously, I assume that means those posts are actually gone, which sucks. :(

Blake

[Maggers]

Yeah, but I even dropped down to "Member" status from "Senior Member." :)

Seriously, I assume that means those posts are actually gone, which sucks. :(

Blake


OK, now that's seriously stinky. To drop down to a lower member status!

What can be done, if anything? I know you had some suggestions, Blake. Can this board adopt those suggestions? I wonder.

[admin]

As i am not a Vbulletin expert by far, I am not sure exactly what can and can not be done. I will experment with those merging sugestions in next couple of days to see what the outcomes are. Hopefully they will work out. I can also provide coppies of the databases if it doesn't for people who would like to try as well. They are rather large though, the average sql dump is like 70+ megs.

Merging shouldn't be a problem, we do it all the time with other apps. However, things that should have caused no problems with Vbulletin (like upgrading) did! uhg... what a head ache.

Well thanks for the idea Blake, much appreciated,

Will

[Ken Valentine]

Aww, man, you finally broke 1000 and lost 'em? That's just not right.

I was up to about 1600+ and went down to 1100+.

'Nother thing . . .

Although they're still in our profiles, our avatars and signatures don't show up on our posts any more.

Ken V.

[Maggers]

'Nother thing . . .

Although they're still in our profiles, our avatars and signatures don't show up on our posts any more.

Ken V.


That's odd, Ken. I'm looking right at your cat and your signature quoting Heinlein. I can see everyone's avatars and signatures.

Hmm...

[Ken Valentine]

That's odd, Ken. I'm looking right at your cat and your signature quoting Heinlein. I can see everyone's avatars and signatures.

Hmm...

Well, we can add no avatars and signatures to my not being able to access some of the threads.

This ... is ... strange.

Ken V.

[Lisa]

Ken, you might want to check your options under User CP. I believe whether sigs and avatars show up can be controlled at the user level.

Lisa

[Ken Valentine]

Ken, you might want to check your options under User CP. I believe whether sigs and avatars show up can be controlled at the user level.

Lisa

Thanks Lisa.

I can now see avatars, signatures, and such, but still can't access some of the threads.

Ken V.

[Maggers]

Thanks Lisa.

I can now see avatars, signatures, and such, but still can't access some of the threads.

Ken V.

Just curious Ken, where are you clicking to view the threads that you can't get into?

[Ken Valentine]

Just curious Ken, where are you clicking to view the threads that you can't get into?

I've tried both the name of the thread, and the last person who posted. Neither one works.

Here's what I get:

vBulletin Message

Invalid post specified. If you followed a valid link, please notify the webmaster

Up above this, it says:

WARNING: Invalid argument supplied for each () in /includes/functions_threadmode.php on line 201

Ken V.

[Maggers]

I've tried both the name of the thread, and the last person who posted. Neither one works.

Here's what I get:

vBulletin Message

Invalid post specified. If you followed a valid link, please notify the webmaster

Up above this, it says:

WARNING: Invalid argument supplied for each () in /includes/functions_threadmode.php on line 201

Ken V.


Gee, that is peculiar. I hope the Admin folk can take care of you. I wonder if anyone else out there is having the same problem?

[Maggers]

I was checking the "Whose Online" part of the board, and I noticed the most extraordinary statistic.

At 11:59 PM on 3/20/05 we had the most people on line ever - 427!

Previously the most people online at once was something like 36 in September, 2004.

Now, I know this board is becoming more and more popular, but 427 people online at once????

[Blake]

As i am not a Vbulletin expert by far, I am not sure exactly what can and can not be done. I will experment with those merging sugestions in next couple of days to see what the outcomes are. Hopefully they will work out. I can also provide coppies of the databases if it doesn't for people who would like to try as well. They are rather large though, the average sql dump is like 70+ megs.

I would be willing to take a shot at it if you have trouble, but I don't have any of the vBulletin software myself, so that may not be a viable option. Good luck!

Blake

[Ken Valentine]

I was checking the "Whose Online" part of the board, and I noticed the most extraordinary statistic.

At 11:59 PM on 3/20/05 we had the most people on line ever - 427!

Previously the most people online at once was something like 36 in September, 2004.

Now, I know this board is becoming more and more popular, but 427 people online at once????

I saw that too . . . and I don't believe it either. :D

Ken V.

[admin]

Going to be experimenting with merging databases tonight after I get back from my poker game... If it works the board will go down tonight sometime around 1 am so that I can go live with the changes.. Wish me luck with both!

Will

[KRW]

Going to be experimenting with merging databases tonight after I get back from my poker game... If it works the board will go down tonight sometime around 1 am so that I can go live with the changes.. Wish me luck with both!

Will

Never bet on an inside straight! Hope that helps!


KRW

[jimbow8]

Going to be experimenting with merging databases tonight after I get back from my poker game... If it works the board will go down tonight sometime around 1 am so that I can go live with the changes.. Wish me luck with both!

Will
Please make sure you make a backup and that it isn't corrupt. That happened on my other board. We lost over 5 years of messages because the system backup said that it had been successful when it had not.

[admin]

Well I won the poker game.. However, I am still working on this database merge... I have all the old posts just wondering about some odd outcomes I have had in the merge... Still working on it... just wanted to give an update.

This will happen soon!

Sincerly,
Will

[admin]

OK... Lets give this a shot... I Think all post are in, however they are kind of spread out. The best way to find things is to use the search option. Moderators can merge threads they find duplicates of. Once they merge they will be listed by date. Most everyones post count should be closer actual now.

I didn't delete the previous board. It's still there, just not being accessed.

If worse comes to worse, I can revert back to 3/11/05 before all this junk happened.

Let me know what the board wants to do!

[Maggers]

The "Michigan Bill Legal?" thread has disappeared. I can't access it. What happened now?

[Maggers]

Before I vote, I need to know the implication of the choices.

Also, the smilie chart has no pictures just a red "x" next to the written description of the smiley.

This board is getting weirder and weirder.

[Maggers]

OK, we've got missing posts again. Looks like a lot of yesterday's posts have vanished.

If we go back in time and recreate the board as it was before the upgrade, what are the implications?

[admin]

Posts between 5 pm yesterday and 9 am this morning are not included in the merge.... Smilies can be fixed (are fixed)... and the implication of going back to 3/11/05 is loosing posts between then and now

will

[Maggers]

Posts between 5 pm yesterday and 9 am this morning are not included in the merge.... Smilies can be fixed (are fixed)... and the implication of going back to 3/11/05 is loosing posts between then and now

will

OK, I voted to just move on and not go backwards. But I sure hope we won't continue to lose posts and threads over time.

[Kenji]

"Honda commercial" thread has disappeared after #4. What happened? :confused:

[Kenji]

"Repairman Jack Message Board Statistics

Threads: 1,811, Posts: 25,504, Members: 379
Welcome to our newest member, bkwormonthenet"

He is not newest member....

[Blake]

That was weird. I just voted to leave it be and do clean-up, and upon voting, I was kicked into another thread! It seemed to be a thread with another poll going, but the poll shown at the top was the one for this thread. Weird, weird, weird. I kept a screenshot if anybody's interested.

I don't want to lose anymore posts, though, so the cleanup option seems best to me, if it's viable.

I would still like to see all those old posts, though, so maybe we'll have to go back to the archive idea.

Blake

[Blake]

I see my number of posts is higher now, but it's still about 50 less than it should be (i.e., roughly 1/3 of my posts must still be missing and/or the count is inaccurate).

Blake

[Lisa]

It did the same thing to me, Blake. :eek:

Lisa

[jimbow8]

I still short a few hundred posts, but oh well.

[KRW]

It did the same thing to me, Blake. :eek:

Lisa

Same here, but while I was there I voted for my favorite RJ novel. :D


KRW

[Ken Valentine]

Posts between 5 pm yesterday and 9 am this morning are not included in the merge.... Smilies can be fixed (are fixed)... and the implication of going back to 3/11/05 is loosing posts between then and now

willI just clicked onto the Haiku thread and got this:

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, will@ethink.net and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.


--------------------------------------------------------------------------------

Apache/2.0.51 (Fedora) Server at www.repairmanjack.com Port 80
*************************************************

I also can't access any of the "sticky" threads.

Ken V.

[admin]

I would like to know at what time you had a problem and exactly the method of what you did to recieve that error.. I can not seem to recreate it. Also try logging out and then accessing the threads in guest mode. I would like to see if could be anything to do with your account. Also try totally emptying your cache and history.


will

[Ken Valentine]

I would like to know at what time you had a problem and exactly the method of what you did to recieve that error.. I can not seem to recreate it. Also try logging out and then accessing the threads in guest mode. I would like to see if could be anything to do with your account. Also try totally emptying your cache and history.


will

I'm not sure what emptying cache and history means, but I did log out as you suggested, and tried it . . . with the same results. I did discover one thing; I CAN access those threads in Linear display mode, but cannot access them if I'm in Hybrid or Threaded mode.

The time was about 9:18 AM Pacific time, or 12:18 PM Eastern time.

Ken V.

[KRW]

I'm not sure what emptying cache and history means, but I did log out as you suggested, and tried it . . . with the same results. I did discover one thing; I CAN access those threads in Linear display mode, but cannot access them if I'm in Hybrid or Threaded mode.

The time was about 9:18 AM Pacific time, or 12:18 PM Eastern time.

Ken V.

Because I've never tried any thing but the Linear display, I've had no problems. I just tried the Hybrid and Threaded mode and they don't work for me either. :confused:


KRW

[Ken Valentine]

Because I've never tried any thing but the Linear display, I've had no problems. I just tried the Hybrid and Threaded mode and they don't work for me either. :confused:

I had used nothing but threaded mode. And on a few threads, it just stopped working.

Ken V.