Standardized Blender Testing

General Forums Blender and CG Discussions
1

I’ll make this cut and dry: I suggest a standardized testing schema. details: There should be a database of listing a test, and the result of the test. something like below:
blender version| test | tester | result | notes

blender 2.70 |render|user_1 |crash | system specs and file used

there should be one for each little thing you do in blender and a sample file to download and test with.
A whole site for this could be made and users would log in and do checks and sign off on the results, and suggest a test and upload a testing file (preconfigured preferably to avoid long set up times.

all a user would do is get the file and do what the note says then write the result in the site. There would be a form for this which corresponds with the test.> and if a new bug is found, a user can make a user test for someone to repeat uploading the file with instructions.

As blender grows, many non-coders or anyone who wishes to help can do testing and find bugs in a systemic way. I’m glad the bug tracker exists to file after someone accidentally runs into a bug, but as blender grows, there seems to me that some bugs lurk there undiscovered until much later. Also, with the rapid bimonthly releases, this would help find bugs in a systemic way.

2

you are describing to me pretty much what the official bug tracker does :smiley:

3

Now that I think about it, I also have an idea that would make things a lot easier.

The idea is that Blender would more deeply integrate into the Blender bug tracker, so when you click on the menu entry, the release hashtag and detected system information would be automatically copied to the entry.

Even better, with this type of thing, it could even do an auto-attach of the .blend file if it is small enough, meaning that half the work is already done and you just need to name the entry and describe it.

The final enhancement would be that Blender simply goes into an inactive state when it crashes rather than quitting, all the while Blender would have a key-log that can also be auto-copied to the entry, this will make things a lot easier for users and developers alike (after work is done to implement it of course).

4

There’s already been something akin to this in place (a series of test .blends to test against prior to release). There’s a history of them here. The latest is test260.tgz, but it probably needs to be updated for the 2.70 series.