[BPY] Official project to improve scripts and scripting

Hi all,

With Blender itself getting better and more professional, scripts included with it must also be as well developed as possible.

To help with that, we started a project to review, discuss and improve scripts and scripting. There are script writers, enthusiasts and Blender / BPython devs involved.

But it’s basically open for anyone interested in participating, since there are many areas: discussing and defining goals, coding, ui, documentation, reviewing it all, testing in multiple platforms.

As a start, we’d like to work on a couple scripts to get more experience with this process:

  • Campbell (ideasman) Barton’s OBJ import and export;
  • Martin (theeth) Poirier’s Save UV Face Layout.

Plan, news, guidelines:

http://mediawiki.blender.org/index.php/ScriptDev

Our main goals:

  • Stable, well developed and integrated scripts.
  • Very good documentation (we’ll use Blender’s wiki, think “book quality”).
  • Having a fun and productive project to take care of scripts, discuss what is missing in Blender and work to get it done.

Those willing to participate or with suggestions or concerns are invited to reply to this announcement, as well as those already involved, so we all know who is aboard. The threads to discuss and review each of the above mentioned scripts should be created soon, maybe in one or two days.

Hoping this succeeds :),


Willian

Ian,

thanks for posting.

Scripters and users please have a look and help us out :slight_smile: We want the script process to be greatly improved, and we need your help to do so,

LetterRip

I am glad to see there is an effort in this direction, I look forward to the discusion thread

Many hands make light work. :slight_smile:

Just saying Hi. Look forward to some good results.

Brendon Murphy, tested script reference.

Thanks for posting IanWill, Ill definetly be involved,

After, I’ll have to review my own script and write up some more comprehensive docs probably… then others can review (if you want you can look at the script alredy)

Heya guys,

Just letting you all know of my involvement in this project.

Regarding the two ‘guinea pig’ scripts, great choice on those, since they are two of the longer standing scripts as well as both being well developed and maintained.

Disclaimer - I’ve looked at the 2.42a versions of these scripts. I guess ideasman has already updated his obj stuff to fit with these guidelines.

Usage
I’ve not personally used the obj exporter, but do use the import from time to time. I will have to do some more tests before posting any reliable data on either of these two functions.

theeth’s UV layout saver I use a hell of a lot though and have always found it extremely useful. In common usage, I have one major gripe with this script -

it is not immediately apparent what file format the exporter saves to and there is no option for what file type to save the output to, it seems to always be tga, which I have to open the script file itself to discover. Perhaps this could be indicated in the file browser used to set the output path/file.

Code
As per style guidelines we’ve thus far established at http://mediawiki.blender.org/index.php/ScriptDev/Guidelines nearly all bundled scripts need some work to bring them into line with these guidelines.

For the most part, this is code documentation. A quick glance at the obj exporter by cambo shows that it as documentation of functions but not docstrings. Picky?! yeh but if we really want to bring these bundled scripts into line quality wise, I think that’s necessary.

Further, the UV face layout saver, has basically no documentation in the script. The one docstring I found was in French along with a number of variables, and function parameters.

Data Safety
A big issue I have with the UV Face layout saver is that you are not warned if you opt to save the file out with the name of an existing file, thus you can easily save over existing data accidently. I’m unsure as to whether this is a Blender filebrowser limitation - which I kind of doubt given that it normally asks if you want to save over - or if the script is to blame. I note that the script actually adds the object name to the filename as a given option, which may also be the cause.

In fact obj exporter seems to also suffer this affliction, so it may very well be a Blender issue. If so, it definitely needs to be addressed.

[Added later] I see that the BpyMessages module has popups for existing files etc, perhaps the onus of testing for existing files needs to rest with the script and not the filebrowser.

Functionality
No major issues here, both scripts use the Blender.Draw API to interface with users, which keeps functionality in line with Blender.

Feedback / Efficiency
Both scripts can take a fair amount of time on larger models/image outputs. Neither seem to make use of the various user feedback mechanisms the API provides such as the wait cursor or the status bar.

I feel this definitely needs to be addressed. In my personal experience, this is especially the case with the UV layout saver, as it can take a very long time on larger models. This may also need to be looked at as an efficiency issue.

Error handling
I actually couldn’t cause an error in either script, so I assume they are either handled transparently, or robust enough to deal with any problems. Good work there.

File IO
Already covered this in data safety, but I see this part of the guidelines specifically mentions not overwriting files without permission.

Security
Not pertaining directly to these scripts, but all in general, I am fully against using eval() and exec(). In my opinion we should completly ban these two functions in the guidelines. The only place these functions are ever the only option, is for dynamic command input (which will obviously be an issue for the interactive command script from cambo). At the very very least, severe scrutiny should be given to any script which uses these two functions, one or the other.

BPython Header
Both scripts were great in this department, containing all relevant data.

Filenames
Currently the obj export/import uses the filenames ‘obj_export’ and ‘obj_import’, as per guidelines this needs to be changed to ‘export_obj’ and ‘import_obj’

The UV layout saver filename is kinda not really very good at the moment named ‘uv_exporter’. I actually had to open all the ‘uv_*’ files to find this one. Maybe we could make it more descriptive.

Documentation
Err…what documentation. I think basically every bundled script is severely lacking in this department. I may be wrong about these two, but a quick search turned up nothing specific for either.

As per IRC discussion, we really need to wikify all script docs.

I’ll look at both these scripts further over the coming days and see what else comes to light. Until then, feel free to respond to any of the points I’ve made.

Great initiative. I hope that, in time, this will help elevate the quality of Blender/Python scripts to greater standard.

Good thing one of my script was chosen as a first example, as I think it has a lot to gain from such a review.

It’s always TGA and that is very coupled in the script’s raster code. Eventually, it could be split off if the raster code is to be rewritten.

The TGA format is mentioned in the menu entry tooltip, but I guess it doesn’t hurt to specify it in the save dialog either.

The french part of the code was taken (with permission, IIRC) textually from one of JMS’ example.

Personnally, I feel that most of the code is self explicit, apart from the previous mentionned rasterizing code, which would serious need a rewrite/documentation. I do understand the need for docstrings though and I’ll add them soon.

Currently, it has to be handled in the callback function. It would probably be better if it was handled automaticly.

Ok with the wait cursor, that would be very simple and easy to add. The status progress bar though is a real hack (in the Blender code itself) and slows things down tremendously.

My script doesn’t check for editmode, that’s one way you might be able to get erronous result with. Fairly easy to check for though.

Something like: export_uv_layout.py perhaps?

They should be as self explanatory to the users as possible. If you need to look up documentation to use a script, then the UI failed, IMHO.

Martin

  • notes on the obj i/o - timmeh, thanks for your review.
  • obj expoert exports edges much faster
  • asks before overwriting

Improved docstrings for both, though Im not sure what needs to be documented exactly. the options have tooltips that explain each option fairly well.

Both should also use the wait sursor,
Theeths right, progress bar can slow things down a lot… not sure what to do for that one.

Hi,

Thanks all for the replies. Now that almost 300 have seen this post and you started bringing ideas and reviews, we should be ready to start two more threads:

[BPY] Review: OBJ i/o
and
[BPY] Review: Export UV Face Layout

and discuss the scripts there.

Today I updated the wiki doc template that Theeth started:

http://mediawiki.blender.org/index.php/ScriptDev/Template

Let’s review the template for layout, readability, completeness, etc. so docs can be written for the other i/o scripts, finally.

Note: the page looks dull w/o graphics, maybe we could use a default “banner” for all i/o scripts wiki docs.

We also need an example for another kind of script, with nice pics. Theeth’s Save UV Face Layout is the ideal candidate, since we’re reviewing it.

Update:

The wiki section for scripts has been created, where we will start the Scripts User Manual. The AC3D example was already added there. We’re still open for layout and content improvements, share your opinions.

http://mediawiki.blender.org/index.php/Scripts

Campbell: could you follow the template and the AC3D example and write the OBJ importer or exporter wiki doc, please?


Willian

Here’s a copy of a e-mail I sent ianwill

Hi Willian,
I have re-organized the Reference list by the catagories I feel the scripts belong in. This serves a few purposes.
First and foremost similar scripts are now grouped together making it easier to check out duplicate/similar scripts for same useful or merging.
Second, The list is better documented by me, making it easier to make the move to wiki.
Third, It’s now easier for noobs to understand.

I think it is important that we all work together so there is some sort of uniformity.

I’ll start moving\copying my list to wiki soon, also I can help a little with individual script feedback.

For one example “A.N.T. Landscape Creator” has some great functions. Even displaced Grass, Water Ripples, much more.
These functions can be acheived by tweaking settings in the script ui although they are listed as more mathamatical functions, meaning to create Grass ect properly becomes guesswork.
Many similar functions appear in other scripts, amalgamation
into a super script? I’ll leave these sort of issues to you.

Also some space handler scripts “mouse gestures” for example give a ‘script link related’ redraw error “can’t find gui” that appears to affect some other scripts accessing their own gui: thus returning the same error.?

“Mouse gestures” is anouther ui script that is great to use.
However I find that accidental rotation/scaling is an issue.
The 3d drag/move function is, once you use it, an essential
Modeling tool.

One script that I think has huge potential is the BPY Exporter
by Jean-Baptiste Perin. Being able to save models as a .py file has huge potential, could it save materials, particles, scenes too, maybe intergration with Blender Library, I can only imagine the results.

Anyway as I have stated to others my Reference to scripts
is a good foundation to build a more professional approach to
script usage.

I will gladly help support any Official Blender CVS/Repository/Wiki

Ideas gladly accepted. brendon

It seems the separate threads for obj i/o and export uv face layout were not created yet?

Anyway, I have some comments on the export uv face layout script. Python scripts can create images and read/write pixels through the API now, so I don’t see the need for writing a TGA file. The script should be consistent with the New image function in the same menu, by having a similar popup asking for the name and dimensions. Same goes for the baking scripts.

Seconded. The image can then be saved with the Image -> Save command in the format of one’s choosing.

Hi, has everone checked out reD_Fox’s Site and BpyMan script.
http://hhofministries.org/bpyc/scriptlinks.php
He really hasn’t had some support he needs.
The BpyMan script allows multiple users to browse and access scripts via the internet, from a repository/storage/authors site.
One Exellent Idea I have is that the Site and Script could be used as an initial testing ground for scripts. Scripts for testing could be uploaded to the site then everyone could have access to the scripts via BpyMan script.
This way many scripts could be uploaded and tested more quickly.
It could also allow specialists in certain areas to to test related scripts in their area. Surely this could help spread resources, scripts, people, over many tasks.
Then as results come in, documentation is done, bugs fixed ect…
The wiki could be updated properly and/or the script released/re-released.
I suppose I see the BpyConsortium as a somewhat underestimated and
undervalued resource.
Working together towards the same goal by different means that compliment and benifit each other.
What do you think?

About using new, then pixel R/W.

The baking scripts (scripts Iv been working on in the image menu) - make use of blenders rendering engine. That way I can avoid writing my own restarization code and write a mesh to be rendered. - In this case I thinnk its best not to try to redo it in Python.

It is possible that it could just render the image then leave the render window open and let the user save, but that could get confusing. Users might try apply the render result as a texface, or think that its saved and loose it on reopening or rendering again.

  • A popup “Now save your image” could solve that I suppose, but its just tricky to get it working right because when you remove the scene it removes the render.
  • EDIT - I may be wrong about removing teh scene removing the render from the Image window… maybe its possible. I still prefer rendering to a file directly.

You’re right about the risk of losing unsaved images. This is also a problem with the New image function though. How about automatically packing new images? Eliminating that “can’t paint packed images” error is on my todo list anyway (but it’s not a quick fix, will try to do it before next release).

brecht, this isnt a bad option.
at the moment its not supported though, you cant render to an image buffer, the only way to do it is render, save, load

I suppose you could render,save,load, pack, remove the image but seemes a bit crufty.

1 Like

Removed by author

:slight_smile: Hey All. Well thought out Progress is a slow thing. Just to let everyone here to know. There are now a couple of new pages in the scripts section at the wiki. Catalog. sub heading Scripts Catalog , This contains the list of script catagories and Template, the model for adding scripts to the catalog. section. Other pages will follow soon, then the scripts.:smiley:

Update: OK, If you have been following my progress, you will already know this. I have begun adding scripts to the wiki. Starting with scripts included in 2.42a. Progress is good, about 10+ scripts per day.
There are still some issues to iron out, mainly to do with sorting.
I will start a new thread to discuss these issues in a few days.
I hope once I start adding scripts not included in Blender that it helps you all tidy up the script content, merge functions and improve quality.
Don’t thank me, I’m not finished yet.

Could someone please post back on topic “improving scripts” as I have tired of seeing my name at the top of this thread.:o

Thanks. b

OK, I’ll jump in here with a few comments. First, I had a look at the UV exporter script, and wrote up some documentation:
http://mediawiki.blender.org/index.php/Scripts/Manual/UV/UVExport
Let me know what you think of it.

A couple of things I noticed while working with the script:

  1. It always adds “.tga” to the end, even if it is already there. I thought that was a little annoying.
  2. This is totally unrelated (even to the UV export script), but the registry and configuration editor do not store the ranges for numeric values. The config editor clamps everything to 0-10. This means that most scripts will not be configurable via the config script, not just the UV export script.

Then, to throw in my two cents regarding “discussing and defining goals,” I want to mention that I am in support of an “elite” approach, i.e. the scripts bundled with Blender should be the best of the best and be maintained by Blender developers and their respective authors with the same rigor and professionalism (or, as some may contend, the lack thereof) that characterizes the rest of the code base. And, if I may carry the concept further, I would propose that the scripts themselves be treated as a part of the Blender code base (with all of the pre-release bug testing/fixing, reluctance to abandon them, etc.) But maybe that’s going overboard.

However, I also want to add that I am very much in favor of the development of secondary distribution methods (e.g. an online repository or the compilation of unofficial bundles) from which scripts that are not bundled with Blender may be easily installed as well.

I think the guidelines you all have compiled are very good. They are a little on the technical, programmer side, though. I would also like to see the compilation of useability helps and guidelines relating more to script UIs, user interaction, and integration into Blender’s interface.

Levi