Using Kitsu for solo production tracking

[thetony20]

*Note: This is going to be a very long post, sorry, but hey, no one can say I don’t provide background info… or a story to help people sleep!

With wanting to create my own animated ‘film’ for years (decades even), I’m trying to slowly put together and learn not just the tools but a process that should lead to some sort of organization and constant progress that keeps motivation up.

Clearly a single image is much easier to manage then a whole animation and as such the issue of ‘pipeline’ comes to mind. While there’s odd bits of information all over the place, so much of it seems to be either a closely guarded secret or its completely in-house written tools for large studios. As such, trying to determined what would work or is even needed for a solo production has required a lot of conceptual thinking (made harder by not even knowing what I so far don’t know).

So far I’ve already started to work on an overall folder/file/component structure and naming but was also wondering about overall production tracking that wouldn’t be too far over the top but flexible enough to help keep things… well on track.

I knew about the Blender Studio created Attract, which while ‘free and open source’, there seemed to be almost no documentation or easy means on how to setup your own install of it.

While Attract and others all have web sites for production tracking I didn’t want some month to month cost for something that I may not use much from one month to the next. And since it was just me, I really didn’t need anything ‘web based’ that others could access, all I needed was just some local, stand alone program.

Of course everything is web based now (which in most cases makes sense) and till recently I hadn’t looked much more into a production tracking solution. But then came the Blender Studio blob post: https://cloud.blender.org/blog/the-blender-studio-pipeline/

At that point I started to have a closer look at Kitsu (https://www.cg-wire.com/en/kitsu.html) and thinking that unlike Attract, even if the Blender Studio ultimately drop it, CG Wire are going to keep going and keep some level of development. Plus with it being open source, given enough install/setup documentation, I figured I could just do a local install myself. A little easier said then done, but more on that later.

I thought of a few possible ways to deploy it, not wanting to ‘clog up’ my main PC, which is Windows anyway and the whole setup only works under Linux. I do have a few older computers around, but between being either very old, maybe somewhat under-powered or just not wanting the electricity cost, etc or running an extra full desktop PC in order to have production tracking available in the background seemed a bit over the top for a single person.

I did think that a little Intel NUC (or other manufacture equivalent) could work pretty well. Get something like a 6 core CPU, install proxmox as the base hypervisor and then I could run 2-3 Linux VM’s for Kitsu and maybe some other future ‘services’. Problem is those things really aren’t cheap, not as a base unit in most cases and even less so when you add all the extras (like RAM and storage) that you actually need to make it all work.

Another option was a Raspberry PI, which funny enough will run it (more on that later as well), but even then, its still a couple hundred dollars all up, money which would be far better spent on future GPU update or more storage space. So in the end, I decided to go with what I had, without messing up any setup or install and if it didn’t work, I could just delete it all and the only thing I wasted was time. In other words, I used my main PC with Windows 10…

INSTALLING A LOCAL COPY OF KITSU

My solution was to use Hyper-V, which because I’ve always only ever installed the Pro versions of Windows, my Windows 10 Pro setup on my main PC was basically all I needed. Aside from some disk space, my thinking was that I could get a Linux VM install that could run more or less in the background using a minimal amount of RAM and a couple of virtual CPU cores. Outside of production work or just not needing Kitsu, I just wouldn’t start it up at all and if I was doing production work and found that I suddenly needed as much RAM and processing as possible (my system is a 5900X with 32GB of RAM), then I could just shutdown the VM.

So the first thing to do, was to actually have a working Linux VM, I went looking for system requirements for Kitsu and really couldn’t find anything and then tried to see what distro the Blender Studio used, thinking that they maybe have a local install of it (I don’t actually know if they do, possible they use the CG Wire one). From all I could tell, they use either Pop! OS or Debian, which seemed like valid options, but in the end I went with Ubuntu Server 20.04 LTS, turns out that was the right choice, since I’ve later found a reference on the CG Wire website that states it needs Ubuntu.

So I downloaded the ISO, created a Hyper-V VM and basically installed Ubuntu Server. For the most part it went fairly smooth, with looking up a couple of things on the web just to make sure settings were basically correct.

Once done I had a very minimal install of a command line only Linux OS for which I only planned to do one thing with. I then made sure it was patch updated, network was working and I had enough disk space for some more software. Being that I’ve not really used Linux for over 20 years or so now, much of what I wanted to do required looking up commands on the web.

I then did what ever server admin is likely going to say one should never do and that was install a GUI. Yes, I know, its a server, running a GUI is a waste of resources and just shouldn’t be needed. And they are right, and in the end it’s not actually really wasting resources, but I figured it would just be so much easier to do the initial Kitsu install and troubleshoot any possible issues if I had at least a basic visual file manager, settings and multiple terminal windows all running at once. As it turns out, it was easier.

The only question was which GUI to use, I didn’t want something that was totally ‘yuck’ but I didn’t need anything bloated with Office apps, etc as well, just a basic, lightweight GUI with the bare essentials. In the end I went with Xfce using the Lightdm window manager. Even then, it still used up another 1.5GB of disk space and while running pushes RAM usage to around 4GB, compared to 1GB without it.

However, I had a solution to that, once I looked it up on the web to find out how to do it, that being to stop the GUI loading up automatically when the OS boots up. This way, by default its still a low resource usage server OS and it only needs more RAM and processing power if I manually start up the GUI from the command line. Once I have things all running and humming along, I suspect that I’d barely use the GUI, in fact I won’t even connect to or login to the VM, all I’ll do is start (Power On) the VM in Hyper-V and then use Kitsu via the web browser in Windows.

Once that was all done and working, I then decided to do one more thing, which is really just a choice I made in order to give me a bit more control over where data, etc goes. On looking through the Kitsu install steps, I could see that the main Kitsu setup and its data (like the video previews, etc) are all installed in one location, that being under /opt on the linux drive. While software like the database and web server, etc are all installed on that same linux system drive and none of it works without the other, I actually wanted the main data for Kitsu to be on another drive (or in this case, another virtual Hyper-V drive, which is really just a big file on your host PC).

So, again using the web, I made another virtual drive in Hyper-V, added it to the Linux VM and then using a few Linux commands, partitioned, formatted and mounted it as /opt/data and then just used that path in place of /opt for all of the web instructions to install Kitsu. This way I split the data and avoid just having one big massive virtual hard drive file and more important, should the Kitsu data grow to a fair size, I can leave the main server drive file on say very fast NVMe storage, while putting all of the Kitsu (image/video, etc) data on to say a larger/cheaper SSD or even HDD.

So, now I was ready to install Kitsu. To do that, I followed two web guides, the main one at CG Wire (https://zou.cg-wire.com/) and another which runs through a full Raspberry Pi setup (http://vfxgeek.video/2020/09/10/raspberry-pi-vfx-task-management-server/).

Ignoring the Raspberry Pi specific stuff, the two are basically the same, no doubt the second mostly taken from the CW Wire site, but the reason I used both is that along the way each one may add additional comments or a more specific command to type in order to get the job done. It also provided a bit of a check and balance between both sites and the commands/text I was typing in the hope of getting it all correct.

And guess what, IT WORKED. Once done I could just boot up the Hyper-V Linux VM (which uses around 1.5GB of RAM with Kitsu, etc all running and usually stuff all CPU processing), and then connect to it via a web browser in Windows, which is of course super fast, since technically its all on the same local networking host.

Next step will be to start playing around with it a bit and decide how best to setup the production for a single person. So stay tuned or not, as this all maybe of little interest to 99.999% of Blender users, haha.

[thetony20]

So I’ve just started playing around a bit with Kitsu and adding the very first bits of info for my little project. Discovered a couple of things, some of it is VM related (at least with Hyper-V on Windows 10) and some is Kitsu, both based on just how it would normally be used and partly the ‘incomplete and/or old’ documentation.

First, from a VM point of view, in general it’s working pretty well, performance is fast, it fully boots up in around 20s and fairly light in resources, using under 2GB of RAM while running and with minimal data in the system, the whole thing uses under 8GB of disk space (but of course that will grow dependent on how much data is actually entered into Kitsu).

Due to the way Hyper-V works on Windows 10, the server always gets a different IP number via DHCP and it’s not really something one can hard set, at least not easily. While not ideal for a ‘server’, since it’s local and basically single user, it’s not really a big deal. I called the server ‘kitsu’, as such once the VM is started, I can just use ‘kitsu.mshome.net’ in a web browser and up it comes.

The next thing, which is something you have to deal with in the last stages of setup as well as on going use, is how Kitsu deals with user names for logging in. Basically it expects and pretty much only accepts an email address, as in ‘[email protected]’, now of course for the online version and even a local one deployed in a company that makes perfect sense, but for a single person at home, not so much.

The first time this will be am issues, is during initial install, when you need to manually create the Super Admin user for the system. Now normally that would be just ‘admin’, but it needs to be an email and I didn’t really feel like using my actual real email, so I settled on ‘[email protected]’, with kistu being the server name and .local fairly well recognized as being a non-routable reserved top level domain name.

This then brings us to the issue of what other users to create for the operation of Kitsu. Normally, you would of course have a team of people and create user logins for each (which then also allows you to assign people to various tasks and track all the work and progress, it is after all a production tracker…). But it’s just me, as such I could have used the default Admin user for everything, however, as someone that works in IT, always being logged in as the user that can totally destroy everything with one wrong click, is generally not a good idea. Mind you, I also didn’t want a layout, modeller, animation, etc login and have to keep logging in and out to do anything across the production.

As it turns out, Kitsu has a user level between the two, that being a supervisor, which can give you near full control over a production, while locking out the overall setup and configuration of the whole Kitsu server. So I thought the best solution would be to create a user per production (using the same email format, for example '[email protected]") as a supervisor and assign them to each task for that production. Then if I really need to make some overall system changes, I can always just login as the admin user.

To be continued…

[johantri]

Wow… What a cliffhanger ^o^ I want to know more about the sequel of this journey. Thanks for sharing this!

[thetony20]

Thanks, yeah I’ve been meaning to update this, I’ve just not been using it all that much so far, since I’ve been more focusing on pre-production and most of that has involved Krita.

I think Kitsu will more come into play once heavier Blender/shot production, etc comes into play, even so I can provide some updates so will try to do so in the next week.

[thetony20]

As promised, here’s the next chapter for anyone still following along and needing something late at night to help fall asleep.

Using the admin login, I created a new production (Ninja Sisters - working title for my little animation project) and set the basic parameters (see image below). While the options seemed simple enough a couple of them either aren’t as clear as one would think or explained at all. The ‘Resolution’ option isn’t the final render resolution of the project, as stated in the on-line docs, this relates to the preview video files that Kitsu will make on the server. At the time when I first created the production, there was no guidance as to the format of the aspect ratio, you could just type anything into a blank field and the on-line docs gave no information as to how the data should be entered.

It turns out that has changed with later versions. The new production page now has more base information and some fixed formatting for that data. This is also now shown in the updated documentation. However, if you go to edit an existing production, the old page, as per the image above is still the same.

With the basic production created I then had to consider the overall organisation of the system and how it would group or label the various assets and tasks for the project, or in other words how to use it as a production tracker, which is what Kitsu is. There are of course a number of already pre-defined names/tags within the system under the four main production sections, Departments, Task Types, Task Status and Asset Types.

My very first thought was to create/delete and align it all with the naming and workflow I had so far created in my Naming Conventions for Digital Production document (Naming Conventions and a Production Workflow). As luck would have, sanity quickly resumed and I realised that totally re-creating the wheel is stupid and that for the most part, what is already in Kitsu actually matched up pretty well.

I did however make a couple of changes/additions for what I wanted to keep track of in one place. First of, there is what I didn’t change at all and that’s the Task Status list. This covers things like Done or Work In Progress and even Waiting For Approval, which is a little pointless for me, either I approve it or I don’t, so what’s there to wait for, but it’s just easier to leave the default list in place and move on.

In the case of Asset Types the usual suspects are already there, like Characters, Environment and Props, however I wanted to also keep track of production files like script(s) or my Naming Conventions file or any other potential notes/files that are important to the project. So with that in mind I added a Document Asset Type.

Looking at the Departments list, which is where all later task will actually sit under, for example within the Modeling department you may have the task of modeling and shading a character or prop. This would likely more matter for productions at larger studios where you have many people and link those people to each Department within Kitsu, allowing for the monitoring and tracking of progress at a Department level. As it seems to be a basic part of Kitsu, even though I’m a department of one and part of them all, it was something I had to work with and from memory I added a Concept and Layout Department for later task linking.

It could be argued that any Layout tasks could have just been linked to the Animation Department or Concept work thrown in with Modeling. But then I did have a new Document asset type with a created Script task and it seemed much better to link all that to a Concept Department, given that it totally doesn’t belong to anything else.

Speaking of which, Script would be a new task and that was then added to the Task Types list, along with any other Department linking of any existing Tasks or new ones you want to create.

All of this was done using the main Super Admin login, as that’s the only one with Studio Manager role which has access to all these setup options within the Kitsu menu (the three stacked lines in the top left corner).

On then creating a new ‘supervisor’ login just for this production, I then started to add some basic early information which currently looks somewhat like this:

But more about all that later (I’ll try not to make it 4 months later, assuming that people are actually interested in any of this).

[johantri]

I’m having a problem when excuting this command.

DB_PASSWORD=yourdbpassword zou init-db
zou: command not found

Any suggestion of what could possibly go wrong? I follow the steps on those two websites.

[thetony20]

Looking over the two websites and the steps before that command, my guess is that you are trying to run the DB_PASSWORD command in a normal BASH terminal, rather then the ZOU virtual environment.

The Raspberry Pi site makes this a little clearer tho neither of them are a great set of instructions.

Looking at the R-Pi site and locating that command, scroll up till you see

We’ll create and operate in a virtual environment:

The last line in that block is the key part:

$ . zouenv/bin/activate

Which actually puts you within the Zou virtual environment and the next couple of commands are actually executed within that environment. However its the next line that changes that:

Now we’ll be creating a database. Let’s open a new terminal window, but keep the previous one for later – as it’s already working in Zou virtual environment.

In other words, and this is why I installed a basic window manager, as it lets you open up multiple terminal windows, the next 3 commands about creating the Postgres database ARE NOT executed within the Zou virtual environment.

BUT, you then hit the line just before the command you are having problems with:

You can close this terminal window, and we’ll create database tables, by typing:

$ DB_PASSWORD=yourdbpassword zou init_db

In other words we are now back to the previous terminal window, which is the one that is still running in the Zou virtual environment.

So, if you currently don’t have any terminal or the server running, then once logged and at a terminal windows, the following commands should get you back on track:

cd /opt/zou
. zouenv/bin/activate
DB_PASSWORD=yourdbpassword zou init_db

That should make the Zou folder current, activate the virtual environment and create the tables in the database.

This is where the R-Pi site is a bit better, in that it usually says when to open and close terminal windows, which a lot of the time is getting out of the virtual environment and back to just the normal Linux Server terminal, which is where a lot of the commands are executed.

Hope that helps.

[johantri]

I just notice there’s small update on kitsu documentation. It’s between underscore and dash
DB_PASSWORD=yourdbpassword zou init_db
DB_PASSWORD=yourdbpassword zou init-db

I finally solved the problem by following the video tutorial. It’s easier to understand for visual person like me https://www.youtube.com/watch?v=QtFzQMYZSDc

Now i just need to learn how to use Kitsu and and i still can’t log in using my self host. I assume that the Kitsu addon only works with CG-Wire host. Is this correct?

[thetony20]

@johantri

DB_PASSWORD=yourdbpassword zou init_db
DB_PASSWORD=yourdbpassword zou init-db

Hmm, so it is between the two sites, not exactly helpful.

To tell you the truth, I’ve yet to even install the Blender add-on for it, but I’m pretty sure it does work on self hosting, as I think the Blender Studio has it self-hosted and pointing to there local file storage.

May give it a go now, just to see what happens.

[thetony20]

Out of interest, how did you even get the add-on?

I know it’s here as part of the overall studio tools: https://developer.blender.org/diffusion/BSTS/repository/master/

But there is no way of downloading it, or does one need to be a subscriber to do that?

[thetony20]

So, to answer my own previous question I found a clone or pull of the studio tools on github: https://github.com/Warnakala/blender-studio-tools and this does allow you to download the whole lot.

It seems to have all of the same most recent commits so I guess its a mirror of the one at blender.org.

@johantri

Now i just need to learn how to use Kitsu and and i still can’t log in using my self host. I assume that the Kitsu addon only works with CG-Wire host. Is this correct?

Hence, as an update to this, I to got an error after filling in the details and pressing login, however, it did actually login. So a different error to what you had.

My guess is maybe you have the host name wrong or not specific enough to the Kitsu server. I just used exactly what I put in the web browser (that being kitsu.mshome.net which is running in Hyper-V on the same PC).

I then used the user name and password that I setup in Kitsu as the supervisor for my production. I know the login worked within the Blender preferences for Kitsu, as I had a Logout button and when selecting the Project folder I had the production name in the drop down list and it displays the general text description that I only entered in Kitsu, so its reading that from the server.

Also, on hitting N in Blender I have a side tab for Kitsu, showing I’m logged in and listing Context options for what I’ve currently got entered. Just at the moment I don’t have any Blender files loaded into Kitsu, so that’s sort of it, but still, it shows that a pure local install/setup works, give or take the odd error.

[johantri]

Hi, I use git clone to download the addon from blender studio website, and i finally managed to log in but with some error message on it. But i’m happy with my progress so far
Apparently i must put the IP address for localhost, which is 127.0.0.1 by default.

I’m wondering how do you setup your Root Project and your folder hierarchy to be recognized by kitsu? It seems that my error coming from that part.

I also happy i’m able to make my three devices to access Kitsu in my localhost, another milestone

Happy New Year 2022!!

[thetony20]

@johantri

i finally managed to log in but with some error message on it.

That’s the same list of errors I pretty much get and I have a theory as to why.

On login the addon is reading the project data from Kitsu (or at least the data that login has access to) and is trying to populate lists for the addon to display. The problem is most of the data (or I think in your case, none of it) currently exists in Kitsu so it’s basically trying to load null data and getting an error.

In my case if you look at the same Kitsu tab I do have a project listed under Context, due to the fact I’ve setup a production in Kitsu and am login with the Supervisor user for that production. If you look at the Kitsu screen in my previous post you can see I have a Characters and Document Assets defined (not with much in there but at least something) and when I click on the Assets under Browser in Blender I get those same 2 listed.

I however can’t select much more since I’ve not defined any Shots yet and I also don’t actually have any Blender files in Kitsu at this stage (everything has been 2D Concept art etc so far).

@johantri

I’m wondering how do you setup your Root Project and your folder hierarchy to be recognized by kitsu?

In the addon preferences, right under the login details you can select the root folder for a project and will then have a drop down list of productions that Kitsu user has access to that you can select to say which project root folder it is. This does of course assume that you have a production already started within Kitsu using the Kitsu web UI.

I think that will allow some files that are ‘used’ as part of the addon (like thumbnails and playblasts) to work within Blender.

As for the issue of using/maintaining a full file hierarchy for the project and how/where you store that, I think that is a much bigger issue and decision.

For example, I think you could say create an Asset in Kitsu (web) and attach/upload a Blender file. What this would mean is that a file from your local drive has been uploaded and is now stored in the Kitsu data folder that you set during initial configuration. In other words, you now have 2 copies of the same file in two locations.

You could then, in theory (not tested or tried this out but I think this is how it can work) load up Blender and via the Kitsu addon, access that Blender file and load it into Blender (basically the addon downloads the file from Kitsu data storage and opens it). You could then work on it and once done, push it back to Kitsu (this would be very much like using Kitsu via the Web UI to post a new ‘comment’ to that Asset and attach a new file) but letting the addon do the posting work.

The problem then is that the latest version of the asset (file) is in the Kitsu data folder and not within the ‘local’ root project folder. At least in theory, none of this I’ve yet to test out and decide exactly how I’m going to use/manage it all.

@johantri

I also happy i’m able to make my three devices to access Kitsu in my localhost, another milestone

Nice, can’t say I’ve actually tried or tested that since I’ve only used/needed it on the same PC I start the VM on. I should give it a go and see if my iPad can access it.

[johantri]

I try the free version on cg-wire and create my account and project there. There’s no error message when i log in. Could the problem be that i missed some steps while installing kitsu in localhost?

I still can’t make my asset to connect to the server when i click the refresh button though. Although i made the project folder hierarchy following Sprite Fright, for example /sprites/pro/lib/char/rex/rex.blend This is the same problem when in localhost. Probably the addon bugs? Unfortunately there are too little information about Blender-Kitsu online or people who interested to it. The documentation on https://developer.blender.org/diffusion/BSTS/browse/master/blender-kitsu/ doesn’t help either, with broken link to images

[thetony20]

@johantri

The documentation on https://developer.blender.org/diffusion/BSTS/browse/master/blender-kitsu/ doesn’t help either, with broken link to images

This has all the images linked and showing, which is also where I downloaded the tools from: https://github.com/Warnakala/blender-studio-tools/tree/master/blender-kitsu

@johantri

Could the problem be that i missed some steps while installing kitsu in localhost?

Then we both got the same steps wrong, as I get exactly the same error when I click Login.

However, I don’t get the error of Failed to find asset type Characters, but then I do have Characters assets defined in my Kitsu production with some tasks and user assigned.

Mind you, even after adding a .blend file to the Modeling task I still get a blank list for Select Asset or nothing found if I type in a character name I know is in Kitsu.

One interesting things I did just find tho, if I login to Kitsu with the Blender addon using the full admin (super user) Kitsu account, I don’t get any of those login errors. Still doesn’t give me anything listed for Select Asset tho.

Be nice to be able to ask some questions to those at the Blender Studio, not sure how to get their attention tho.

[thetony20]

OK, found out a couple more interesting things.

It turns out I don’t get all those Login errors, if and only if I use the initial Super Admin login within the Kitsu addon in Blender. Even if I set my other Kitsu login to the Studio Manager role (which is the same role that the Super Admin is and gives access to all the same settings), then I still get all those errors on clicking Login.

Found something else, even when using the Super Admin login, if I access the Kitsu tab, with the Blender files of a very early model stage of one of the Characters for my production open and click the little refresh icon under Context, I get the following error:

In other words, the addon is hard-coded for the specific folder structure that the Blender Studio uses or at least used on Sprite Fright.

In many ways I really only intended to use Kitsu as a production tracker within itself and not actually link Blender directly to it and hence not use the addon. It seems that unless you have a coder that can ‘adjust’ things to your ‘studio’ or setup, then its all very specific to working at the Blender Studio.

[comicsserg (Serg.)]

cant get kitsu to work , I can login but can’t pull or push in Blender
it always says In shot Tools “no active project”
do you have an idea how to enable it?

[thetony20]

Not sure I’m going to be of much help. I’ve really not spent any more time with Kitsu since those initial tests and playing around to see if it would work and be of any use.

There’s chance it may well have a lot to do with what I posted above, in how I think some parts are a bit hard-coded for the Blender Studio’s exact setup.

If I was to to take a guess/try to make it work, first thing would be the to check the Project Root Directory setting in the preferences for the Kitsu addon. That path and hence location of the project files needs to be, I think, somewhere on the file system that both Blender and Kitsu have direct and full read/write access too.

The second issue, and this is where I think things are hard coded, is that the whole folder structure has to be in a set order, otherwise you end up with the error I have above, about it expecting a ‘shots’ or ‘lib’ folder 3 levels up.

I mean its nice of the Blender Studio to release these sort of things, but I don’t think they are made with the average user in mind. It’s released for a studio (with coders/technical people) that can and will then partly re-code/adapt it to their production environment.

[joseph (J)]

Since Kitsu doesn’t seem to work at a user level, are there any good alternatives? Based on the documentation and screenshots, Kitsu looks perfect, is there another, better-behaved software out there?

[comicsserg (Serg.)]

Yea I saw they hardcoded stuff in it pretty sad as I was very excited to implement a nice pipeline. need to see how can I get it to work right now have my linux PC with install of zou en kitsu and a windows pc with blender, both in the same domain and network

What do you use to keep track of your production these days?