Update on the status of importing missing topics

General Forums Blender Artists Updates
1

Sorry for the silence here. As you’re probably aware, a number of topics weren’t imported during the transition from vBulletin to Discourse. This was caused by a very specific formatting issue in combination with file attachments. We’ve been looking at fixing this, which has literally taken months.

The current status is that it doesn’t look promising and I’m seriously counting on the fact that we will not be able to import these missing topics anymore. There are two factors, the first one being the most critical:

  1. Our consultant gave us the green light to shut down the old server at some point. This server was a serious expense for the CG Cookie crew, so this was a priority. However, he later discovered he didn’t keep the attachments, and we’re now looking for a backup of the old system. If we don’t have this backup, we will not be able to import old topics. If we have an older backup we may still be able to import them partially.
  2. Technical complexity. I’m super anxious about re-importing things into the Discourse database, as this has proven to be a tricky process in the past. I need to be totally sure that we’re not going to cause any issues, and such issues may not be obvious or immediately visible. If we would discover such a problem after a few weeks or months, we’d be completely unable to fix it.

This project has been insanely stressful, and I’m very disappointed in this, but please keep in mind that such migrations are always hard. It seems we’re missing less than 0.05% of the imported topics, which given the complexity of the project is not a bad score.

Again, I should have communicated on this better.

2 Likes
2

So basically with a bunch of buzzword doublespeak you’re saying that you are not going to fix it, thus reneging on your original promise?

Kind of sounds like the type of crap a scammer, and fraudster would write. I suppose it’s time to start filing a fraud grievance with the crowdfunding site. Deeply disappointed in your underhanded, misleading and shady actions.

People specifically asked before donating if all the posts would be migrated and you unequivocally assured them they would, and you took the communities money. So either fix it, or give the money back.

3

Doublespeak? I’m trying to be totally honest and transparent here, and I’ve done my utmost best to do this migration - spending hundreds more hours than I expected. True, a small percentage of the topics have failed to migrate, but this doesn’t mean I failed 100% in the project. Hundreds of thousands of topics and attachments were succesfully migrated.

You calling me an underhanded fraudster is uncalled for, after everything I’ve done and tried to do for this community.

PS: I don’t think you’re a supporter of the campaign, are you?

4

Edit: insult

I was going to donate but the goal was reached so fast I didn’t have time. I have been a cg cookie citizen, paid for cg cookie add ons, and over the years spent many hundreds of hours helping people in the community.

One regular user here (BPR) had no computer and I donated $50 to help him get a new one.

Edit: insult. You lied and did not fulfill your promise.

5

I’m happy to have a decent conversation here, but your aggressive behaviour and personal attack is out of line. Behave, or I’ll suspend you for a month.

6

Before we go any further, I would like for you to answer a simple yes/no question for the community. Forgetting difficulty, do the old posts and images exist somewhere, like a tarball? Or is there literally no possible way to get them back?

Yes means they do exist somewhere
No means they are gone forever.

7

I’ve just now noticed that the “Opel GT by rogper” mega thread, that was on the Works in Progress, was one of the lost on the internet forever… kind of sad after so much work put on on it (4 years or so from start to finish)… but thats the way life works, some things we keep somethings we don’t :slight_smile:

8

I searched in google:
Are you sure this was in the WIP section? It is not this thread then I suppose:

Good car and render by the way! At least finished project thread remained :crazy_face:

9

Thank you, @bartv for the update.

I tried searching for some helpful information over at the Discourse forum, but I also noticed you and @pfaffman posting there. It looks like @pfaffman is a regular there as well.

Perhaps some Discourse dev can lend a hand, if one has not stepped forth or been reached out to already?

If the data is still somewhere, perhaps it could still be imported to a subform (Archived Posts) so as to not endanger all other posts?

Also, with a view to the present and to the future: could this forum be made Archive.org friendly?

Archive.org would have been a great fallback option for the old forum, but, for whatever reason (e.g. robots.txt robot exclusion protocol) Archive.org did not seem to capture much of the old forum.

I’m not sure how things are now, between the new Discourse BA forum and Archive.org, but I would encourage all those involved to see if it would be possible to make this new forum Archive.org compatible/friendly so as to avoid any future major topic / post / attachment loss.

Whatever was lost, in addition to some of the Weekend Challenge attachments and posts I listed earlier, might seem statistically insignificant as compared with the other posts which did make it through migration, but I know what stands to be lost (if it cannot be restored) represents literally thousands upon thousands of collective Blender user hours put into those creations.

While I’d hope that would be evident on its own, I say that in hopes of convincing those who can yet decide the fate of these lost posts that any extra effort would be worthwhile and very deeply appreciated.

I know this a complex undertaking, this forum, full of great responsibilities and unexpected challenges. And, for @bartv and those others who help this thing exist and grow I have great gratitude and respect.

We are a community, and we do share in the trust of this enterprise, between the admins and the members/artists.

Remaining open and honest about these things is such a good thing, even as strong emotions and perspectives may, understandably, might flare up. I think it’s just an imperfect indicator of how deeply everyone cares about this undertaking and wishes for it to be successful for everyone involved.

So, once again, I thank @bartv and encourage Bart and others to see if there just might be some way to pull this off, even if it’s to post these lost posts in another subform or static website.

And please look into Archive.org compatibility. What we have and continue to build upon is certainly worth preserving for current and future Blenderheads :slight_smile:

Thanks for hearing my concerns, and please know you all have, within and beyond our community, great gratitude and respect.

1 Like
10

Yeah! The finished thread remained.
The w.i.p. one explained the making of each one of the thounsands of parts that make the car :slight_smile: It was a good learning resource for everyone. It also recorded all the conversations with everyone that helped me and even my own learnings.

11

@bartv claims he is being transparent, yet he completely ghosts threads for weeks/months.

He says he welcomes a ‘decent conversation’ yet he won’t answer a simple yes/no question.

He promised absolutely that all of the old site would be ported over, now that has changed to ‘a high percentage’ and slaps himself on the back.

He brags about hundreds of hours of time, of course he receives money from the advertising.

12

Ouch. That’s one of the reasons why I avoid giving money to crowdfunding: Anyone can promise everything, but not everyone can deliver it.

The worst thing is that the mistake could’ve easily been avoided. Didn’t it occur to anyone to check if the number of posts and threads before the migration should be the same as after?

And wouldn’t it be possible to make a bot account here which would repost all missing threads? User and date might be wrong, but at least the content wouldn’t be lost.

It seems we’re missing less than 0.05% of the imported topics, which given the complexity of the project is not a bad score.

Considering that we are missing around 100% of the entry threads of the weekend challenge, that’s a very lame score. You handled this very poorly, specially considering that it was marked as “high priority”. And looking at what your consultant did to the data, it is clear that you didn’t learn from your mistakes.

But on the positive side, I think you just gave me some inspiration for this week’s WEC, themed “Dangerous situation”.

1 Like
13

Thanks for the update. Even though, it is a rather disappointing one. I’d have downloaded the entry threads/attachments myself if I’d even remotely considered this a possibility. Let’s hope some long (well, maybe not that long) forgotten backup file will magically show up somewhere and save the day (years).

Yes, I think it is worth noting that the distribution of that 0.05% has been quite unlucky. We are currently missing approximately the last two years of Weekend Challenge entry threads. The voting threads are there but without the attachments from the entry threads, checking those out doesn’t make that much sense anymore.
Even if we can’t get the entry threads back - reviving the voting thread attachments would be a great step forward.

That thought crossed my mind as well. :wink:

14

I addressed this in the original post:

we’re now looking for a backup of the old system

So I can’t give you a yes/no answer yet. I’m leaning towards ‘no’.

That’s a good idea, I’ll keep that in mind.

There’s an ongoing conversation about this on the Discourse forum - some users are working the the IA team to fix this.

Yes I fully agree with you, I should have done a better job at this. I relied too much on our consultant, and all I can say now is ‘mea culpa’ :frowning:

15

Hey bartv! If the images are the problem for “OpelGT by rogper” they do seem to exist:

OpelGTimagesStillExistButAreBlocked
OpelGTimagesStillExistButAreBlocked1876×1543 167 KB

The problem was that the previously free image server ImageShack decided to charge money at some point, so they seem to have block them until I pay or find 3 friends to have them for a month…
I think I may be able to find at least 3 friends yet :slight_smile:

Edit:
So, it is a bit worse than I tought :confused: 3 people need to sign imageShack using this link and all the images from Opel GT thread will be up for a month:

16

This sounds promising, can you elaborate a little? You need three people to sign in? What happens then?

17

On a site this big, I can see the difficulty in migrating everything. As a result, I wouldn’t try to press charges against Bart as that could lead to the wholesale deletion of BA (which in turn would hurt the Blender project and be far worse than a few missing threads).

By the way, CGSociety also did a migration to Discourse and the guys they hired arguably did an even worse job (all of their recent threads disappeared).

18

On actually porting the old threads, would it be possible to download them (in a formatted file format) to a hard-drive and then pop them into Discourse one at a time (using code to place them in the correct locations based on the time of the posts)?

19

Hey Photox,
This would only be required if there is a purpose for it and if the images are indeed the main problem.
I’m not sure also, I think people can just sign in for the free 30 days demo and then never use it again. If it proves to be relevant I’ll test it my self.

20

I didn’t see the ‘refer 3 friends and get 1 month free’ thing, ok that makes more sense now. So what BA used to host images on this site, stopped paying and now they are still there but you can only see them if you have a paid account? How do you search for them?