I’m curious if the Blender workflow has any process in place for measuring and testing a user’s experience within the program. As the user base grows, are there ways to measure the effectiveness of changes, or even use data to inform future changes? If so—great! One thing that I’ve seen is that what users say and what they do are both important and yet can be at odds.
Collecting verbal feedback to a screenshot or video can be helpful for general opinions but can also be misleading when basing large feature decisions. I’ve seen a lot of value in tests where you see what users do based on a provided goal. Insight from data might paint truly different pictures than verbal responses that are offered up from online forums or polls (though those are still helpful). There could be testing like:
- How many users truly use right vs left-click for select (twitter polls can only tell so much )
- Given a task of adding a new primitive, how many users are using Shift-A vs the Create > Add Primitive panel
- If proposing a new panel layout or visual UI change, how many users are able to find the new change on their own? Is it faster? Slower? Same? Are those complaining of slower workflow actually correct for everyone?
These are just crude examples, but knowing how the program is used on a large scale could offer some really useful insight when planning features. And it doesn’t mean you have to follow data—but when adding disrupting features, you can plan ahead by knowing how to ease users into a new workflow to follow a larger goal.
Even proposed ideas like Andrew Peel’s draw primitive or particle system painting proposals (thread here), where common tasks that users might often repeat over and over (maybe data and tests would confirm) could eventually be combined into convenience features without replacing original functionality. I think his proposals are really interesting and insights like these could be easier to identify with user testing.
I’m not sure of the Blender dev process right now, maybe they already do this. I’m just curious if anyone knows or if they are open to integrating this type of work into the process eventually. It can certainly slow down a development cycle so maybe a slow integration is the ideal, but with the right tasks and tests it could be pretty informative.
What do you think? Or does anyone know if this happens already?