Results 1 to 12 of 12
  1. #1

    Blender python script to see changes in blend files (patch)

    I'd love to have a blender python script in order to see what has changed between two versions of a blend file (e.g. file.blend and file.blend1).

    The output could be as terse or as detailed as wanted and would indicate the elements and the settings that have changed. For changes too complex to output (e.g. all the vertices that have been added or have been displaced due to modelling), it is enough to give a summary (e.g. the object's modelling has changed with x new vertices...).
    From: http://overshoot.tv/ticket/3147
    I used to agree with Bao2, but he changed his signature...
    "Smoking is Cool" open source animation project and its discussion thread.



  2. #2
    Member
    Join Date
    Dec 2011
    Location
    Germany
    Posts
    3,991
    in what changes are you interested? just vertex counts, or also if they have been moved and the like? screen layout changes? object names?



  3. #3
    I'm interested in all changes.
    I'd like to see a summary of the changes, in order to see what aspects have changed.

    When .blend files are committed into a git repository,
    Code:
     git diff
    allows us to see whether the file in the working directory differs from the one that has already been committed. However, since we are dealing with binary files, we have no way of knowing the extent of the changes. Is it only the mesh that has changed? Or only the texture? Or both have changed? Or/and something else (animation, new objects, new scenes, etc.)

    Which settings have changed (their values)? For setting values, it should be fairly easy (?) to make a comparative output of the respective values in both files.

    Besides being better able to make a decision on whether to commit an updated .blend file in a repository, it also has more mundane, everyday uses: suppose I have a problem and send a .blend file to a friend who fixes a couple of things and tweaks various settings in order to improve a scene (e.g. lighting...), I could then have a complete summary of what my friend changed and learn from it.
    I used to agree with Bao2, but he changed his signature...
    "Smoking is Cool" open source animation project and its discussion thread.



  4. #4
    Member
    Join Date
    Dec 2011
    Location
    Germany
    Posts
    3,991
    "all settings" sounds tough, as there are thousands of RNA types, and there can be hundreds or millions of instances. And even a single deleted vertex could change other geometry orders, making it hard to detect what actually changed. You would have to iterate over all vertices and test, if they are still the same - by coordinate, as index is not reliable and there is nothing like a vertex id. I believe that is the main problem, that you can't identify objects in another blend.

    Let's say user 1 creates a lamp "Lamp" with energy 2.5, user 2 deletes it and adds a new lamp "Lamp" and sets energy to 2.5. It's not the same lamp anymore, but you can't detect that by looking at the datablock name nor by the property "energy". Or if user 2 just renamed "Lamp" to "Lightsource" - how to find out that it was renamed? One could search for lamp objects with the same property values. But if user 2 changed one of them, it would stop working. You could try to find a lamp which property values match the most, but at least 70%. But that seems arbitrary and not all properties are equally important / significant.

    The only reliable way of detecting all kind of changes would be to log while editing and use that history to see what changed.



  5. #5
    The goal being to provide a kind of human-readable 'changelog', I was aware from the beginning that the list of changes couldn't be exhaustive to their tiniest details. The list should be pruned to its higher-order/meta changes. Sometimes, it's useful to know that a lamp has changed but that no materials have been touched upon.

    The key world would be practical instead of exhaustive.

    An automatic log, as you mention, could be a way of achieving this, with the log being parsed to provide a readable and useful summary. But this approach appears to me, at first glance, to be more difficult to implement rather than simply parsing the object tree of each blend file.

    My stance is that some information (even if partial and heavily truncated) is better than no information (i.e. the situation today).

    Now, what would be the major hurdle to start scripting something like this in python?

    Would in be a python blender plugin or an external python script that takes two .blend files as arguments?
    I used to agree with Bao2, but he changed his signature...
    "Smoking is Cool" open source animation project and its discussion thread.



  6. #6
    Originally Posted by CoDEmanX View Post
    "all settings" sounds tough, as there are thousands of RNA types, and there can be hundreds or millions of instances.
    Yes, I was overly optimistic and naive when I said "all" settings. Let's simply grab what we can.

    Originally Posted by CoDEmanX View Post
    I believe that is the main problem, that you can't identify objects in another blend.

    Let's say user 1 creates a lamp "Lamp" with energy 2.5, user 2 deletes it and adds a new lamp "Lamp" and sets energy to 2.5. It's not the same lamp anymore, but you can't detect that by looking at the datablock name nor by the property "energy". Or if user 2 just renamed "Lamp" to "Lightsource" - how to find out that it was renamed? One could search for lamp objects with the same property values. But if user 2 changed one of them, it would stop working. You could try to find a lamp which property values match the most, but at least 70%. But that seems arbitrary and not all properties are equally important / significant.
    Yes, your example is a good one. You perfectly highlight some major hurdles. But if we can simply say that the lamp lineup are exactly the same or that they differ, this would already be more information than the command diff can provide: It can only say whether the binary .blend files differ or not.

    See my previous comment.
    I used to agree with Bao2, but he changed his signature...
    "Smoking is Cool" open source animation project and its discussion thread.



  7. #7
    Member
    Join Date
    Dec 2011
    Location
    Germany
    Posts
    3,991
    we could easily tell if the object counts changed, that would be something practical and useful i guess. we could furthermore try to find duplicates and linked duplicates, not sure if that is relevent information though.

    Here's a list of object properties:
    Code:
    <bpy_struct, PointerProperty("rna_type")>
    <bpy_struct, StringProperty("name")>
    <bpy_struct, IntProperty("users")>
    <bpy_struct, BoolProperty("use_fake_user")>
    <bpy_struct, BoolProperty("tag")>
    <bpy_struct, BoolProperty("is_updated")>
    <bpy_struct, BoolProperty("is_updated_data")>
    <bpy_struct, BoolProperty("is_library_indirect")>
    <bpy_struct, PointerProperty("library")>
    <bpy_struct, PointerProperty("data")>
    <bpy_struct, EnumProperty("type")>
    <bpy_struct, EnumProperty("mode")>
    <bpy_struct, BoolProperty("layers")>
    <bpy_struct, BoolProperty("layers_local_view")>
    <bpy_struct, BoolProperty("select")>
    <bpy_struct, FloatProperty("bound_box")>
    <bpy_struct, PointerProperty("parent")>
    <bpy_struct, EnumProperty("parent_type")>
    <bpy_struct, IntProperty("parent_vertices")>
    <bpy_struct, StringProperty("parent_bone")>
    <bpy_struct, EnumProperty("track_axis")>
    <bpy_struct, EnumProperty("up_axis")>
    <bpy_struct, PointerProperty("proxy")>
    <bpy_struct, PointerProperty("proxy_group")>
    <bpy_struct, CollectionProperty("material_slots")>
    <bpy_struct, PointerProperty("active_material")>
    <bpy_struct, IntProperty("active_material_index")>
    <bpy_struct, FloatProperty("location")>
    <bpy_struct, FloatProperty("rotation_quaternion")>
    <bpy_struct, FloatProperty("rotation_axis_angle")>
    <bpy_struct, FloatProperty("rotation_euler")>
    <bpy_struct, EnumProperty("rotation_mode")>
    <bpy_struct, FloatProperty("scale")>
    <bpy_struct, FloatProperty("dimensions")>
    <bpy_struct, FloatProperty("delta_location")>
    <bpy_struct, FloatProperty("delta_rotation_euler")>
    <bpy_struct, FloatProperty("delta_rotation_quaternion")>
    <bpy_struct, FloatProperty("delta_scale")>
    <bpy_struct, BoolProperty("lock_location")>
    <bpy_struct, BoolProperty("lock_rotation")>
    <bpy_struct, BoolProperty("lock_rotation_w")>
    <bpy_struct, BoolProperty("lock_rotations_4d")>
    <bpy_struct, BoolProperty("lock_scale")>
    <bpy_struct, FloatProperty("matrix_world")>
    <bpy_struct, FloatProperty("matrix_local")>
    <bpy_struct, FloatProperty("matrix_basis")>
    <bpy_struct, FloatProperty("matrix_parent_inverse")>
    <bpy_struct, CollectionProperty("modifiers")>
    <bpy_struct, CollectionProperty("constraints")>
    <bpy_struct, PointerProperty("game")>
    <bpy_struct, CollectionProperty("vertex_groups")>
    <bpy_struct, EnumProperty("empty_draw_type")>
    <bpy_struct, FloatProperty("empty_draw_size")>
    <bpy_struct, FloatProperty("empty_image_offset")>
    <bpy_struct, IntProperty("pass_index")>
    <bpy_struct, FloatProperty("color")>
    <bpy_struct, PointerProperty("field")>
    <bpy_struct, PointerProperty("collision")>
    <bpy_struct, PointerProperty("soft_body")>
    <bpy_struct, CollectionProperty("particle_systems")>
    <bpy_struct, PointerProperty("rigid_body")>
    <bpy_struct, PointerProperty("rigid_body_constraint")>
    <bpy_struct, BoolProperty("hide")>
    <bpy_struct, BoolProperty("hide_select")>
    <bpy_struct, BoolProperty("hide_render")>
    <bpy_struct, PointerProperty("animation_data")>
    <bpy_struct, PointerProperty("animation_visualization")>
    <bpy_struct, PointerProperty("motion_path")>
    <bpy_struct, BoolProperty("use_slow_parent")>
    <bpy_struct, FloatProperty("slow_parent_offset")>
    <bpy_struct, BoolProperty("use_extra_recalc_object")>
    <bpy_struct, BoolProperty("use_extra_recalc_data")>
    <bpy_struct, EnumProperty("dupli_type")>
    <bpy_struct, BoolProperty("use_dupli_frames_speed")>
    <bpy_struct, BoolProperty("use_dupli_vertices_rotation")>
    <bpy_struct, BoolProperty("use_dupli_faces_scale")>
    <bpy_struct, FloatProperty("dupli_faces_scale")>
    <bpy_struct, PointerProperty("dupli_group")>
    <bpy_struct, IntProperty("dupli_frames_start")>
    <bpy_struct, IntProperty("dupli_frames_end")>
    <bpy_struct, IntProperty("dupli_frames_on")>
    <bpy_struct, IntProperty("dupli_frames_off")>
    <bpy_struct, CollectionProperty("dupli_list")>
    <bpy_struct, BoolProperty("is_duplicator")>
    <bpy_struct, EnumProperty("draw_type")>
    <bpy_struct, BoolProperty("show_bounds")>
    <bpy_struct, EnumProperty("draw_bounds_type")>
    <bpy_struct, BoolProperty("show_name")>
    <bpy_struct, BoolProperty("show_axis")>
    <bpy_struct, BoolProperty("show_texture_space")>
    <bpy_struct, BoolProperty("show_wire")>
    <bpy_struct, BoolProperty("show_all_edges")>
    <bpy_struct, BoolProperty("show_transparent")>
    <bpy_struct, BoolProperty("show_x_ray")>
    <bpy_struct, PointerProperty("grease_pencil")>
    <bpy_struct, PointerProperty("pose_library")>
    <bpy_struct, PointerProperty("pose")>
    <bpy_struct, BoolProperty("show_only_shape_key")>
    <bpy_struct, BoolProperty("use_shape_key_edit_mode")>
    <bpy_struct, PointerProperty("active_shape_key")>
    <bpy_struct, IntProperty("active_shape_key_index")>
    <bpy_struct, BoolProperty("use_dynamic_topology_sculpting")>
    <bpy_struct, PointerProperty("cycles_visibility")>
    Not all of them are sensible, such as is_updated (to be used with a frame handler). But would you want to check all that are?

    The hurdle is pretty much that you need in-depth checks as simple == comparisons fail, and they would be pretty specific per type.



  8. #8
    Originally Posted by CoDEmanX View Post
    Not all of them are sensible, such as is_updated (to be used with a frame handler). But would you want to check all that are?
    Let's start with the obvious ones, those that would make sense and would be useful to the end user.

    Originally Posted by CoDEmanX
    The hurdle is pretty much that you need in-depth checks as simple == comparisons fail, and they would be pretty specific per type.
    That's the kind of thing I am not qualified enough to judge. I am a coder but I know very little python; I plan to learn, mostly to use with Blender. This project could be one way for me to get into it.

    If it iterate over the most important objects (meshes, lamps, etc.), how easy would it be to ascertain whether they are identical or not?

    Code:
    # blender -h:
    
    -P or --python <filename>
            Run the given Python script file
    
    --python-text <name>
            Run the given Python script text block
    
    --python-console 
            Run blender with an interactive console
    Assuming we have an external python script, how easy would be to look into a .blend file given by argument on the command line?

    We can start with a very simple script which takes two .blend files as arguments. The script would load the given files and look into them and count the mesh objects, and start with the most basic comparison that can easily be done (object count, their names...).

    This would be an open source script, so if we manage to create something at least a little bit useful, I'm sure eventually more python script gurus will chip in some more insights on how to best achieve what we are after.

    Then the script can evolve: more command line arguments can be added in order to better control the output (how terse or how thorough it is...) and thus better fit the end user's needs.
    I used to agree with Bao2, but he changed his signature...
    "Smoking is Cool" open source animation project and its discussion thread.



  9. #9
    Member
    Join Date
    Dec 2011
    Location
    Germany
    Posts
    3,991
    Ok, so you are mostly interested into bpy.types.Object instances. That sounds managable, but still a lot of work...

    in the meanwhile, you might have a look at this:
    https://github.com/CoDEmanX/blend_stats

    its not in-depth stats, but shows an example how to use command line



  10. #10
    Great! There is enough code examples there to get started.

    I have given a few use cases, already.

    And you? Do you think any such script would ever be useful to you? In what situations?
    I used to agree with Bao2, but he changed his signature...
    "Smoking is Cool" open source animation project and its discussion thread.



  11. #11
    Member
    Join Date
    Dec 2011
    Location
    Germany
    Posts
    3,991
    not sure... i gotta ask a client if such a script would be useful for them



  12. #12
    I did some research on how the blender files are saved. The blenloader part of the source code handles the file saving and reading. The function 'write_file_handle' calls all the relevant sub-functions, like 'write_scene' or 'write_meshs'. Currently there are 34 of those sub functions. However, it is not necessary to know all the background details to make a blend file human readable, cause AFAIK the file format already contains the relevant information to make it human readable and thereby easier for SVN/Git and similar to parse it. If you really want to see the source code for the blenloader the link is at the end of the post.

    Something more interesting than the source code I found was the following link describing the file format: htt p://www blender org/development/architecture/blender-file-format/ Sorry for the spaces: Am not allowed to post links according to welcoming private-message, because you need > 10 posts to post links.
    The file format is not the most interesting thing on that page. At the very end of the page is a link to a zip file on atmind.nl. This zip file contains a java application converting blend files to xml files and it also contains blenders suzy as an example in both .xml and .blend.
    When you unzip the example files you can see, that the xml file is almost 20 times as big as the blend file. Now the xml files created by this program could be a little optimized size-wise by making it more specific to blender and by also not saving values that are 00, cause in the example xml you can see a lot of faces and edges with mat_nr = 00, edcode = 00, crease = 00 etc., but I don't think it will get any better than 10 times as big when completely human readable.

    If you want the source code for the java application, as reference or maybe even as base, you should maybe consider asking the creator via atmind.nl, cause it currently is not included in the zip file. (But the .jar file is not obfuscated, so it can easily be reversed.)

    Hope some of this information was useful.

    For those who are interested into checking out the source code responsible for saving files:
    htt ps://svn blender org/svnroot/bf-blender/trunk/blender/source/blender/blenloader/intern/writefile.c



Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •