For complex cases (and especially with poly-paint like functionality becoming possible with PBVH), I think the number of layers allowed, at minimum, should be 32 (but if possible, it might even be necessary to have the maximum as high as 128).
As far as I know, it may not be that much work for Cycles and 2.8’s Eevee to support such a number (it may not be needed to have BI support it as it is slated to disappear anyway).
I vote for 0 = Black and 1 = White too.
Does this require compling or just download as zip and extract the files and use? I am not sure how to go about downloading the branch from the page? I am right clicking and clicking save target as but it seems to download as htm file? My apologies. I am not very familiar with the page.
Would like to contribute towards vertex paint development in Blender.
EDIT: Seems I need to build Blender from scratch in order to test the branch you are currently developing. The last guy who was working on this had a blender version people could download and test.
I am not sure a lot of people know how to compile Blender from source and stuff. If you guys want more testers, maybe a downloadable Blender branch like Nathan Vollmer? If this isn’t possible, it’s fine. No worries.
I have started work on vertex alpha channel. I have some doubts.
2)What file format you want to export in that will support vertex colors?
If anyone can suggest how the vertex alpha should there in vertex painting it will be very much helpful.
Blender has alpha feature in other parts, can you suggest any part where the expected result is similar?
I am not into the game development, so from the game development point of view how vertex alpha should work, do you have any suggestion or demos or anything that will make the task more clear.
Thank you.
From a Game dev perspective we do use vertex Alpha a lot.
Couple of scenarios, we use all 4 channels as vertex masks, an example would a terrain shader where vertex R = Mud, Vertex G = Rock, Vertex B is Grass and Vertex A is sand.
In our shader we read each channel separately and blend different parts of the shader (e.g. as simple as a lerp). In one project last year I used this http://polycount.com/discussion/136663/blender-vertex-alpha-hack
https://forum.unity3d.com/threads/vertex-rgba-blender-2-5x.254038/
The export_fbx (and the binary version attached in the thread) are modified to take the extra vcol layer and treat it as the Alpha.
As far as I know from Blender, really only need Vertex Alpha export to be added to DAE and FBX (perhaps Alembic?)
Another example is where are actually using vertex colours (we often do this with simpler art styles to save texture lookups) and again having an Alpha lets us control opacity in that fourth channel.
As for in the Blender Viewport…
Quite often I think I’d like to see it as transparency for the vertices. I think though, that there is a strong use for it as a mask in Cycles/Evee etc. and as such we would not want transparency, I think a tick box to treat Alpha as transparency would be a very good idea.
This video shows the use of Vertex Alpha in a material to modify procedural blending of moss and rock
As anadin points out, vertex colour is extremely useful in game and real-time applications because it enables what is essentially another 4 layers of data that can be treated simply as colour, or as masks, and is (usually) much less data than a new texture would be. Since alpha is supported in most (or the most popular) game engines, it would be awesome to get support in Blender too.
Thanks for these interesting ways in which vertex colors are used in games. Does this mean that it would be very useful to have paint modes that affect only one channel, and viz modes that show only one channel at a time?
For DAE format, I know there can be a lot of flexibility in how things are expressed there. Are there some important game engines that read DAE and vertex colors from it that we should make sure are catered to? Are there docs we could read about workflows taking vertex colors from modeling software into game engines?
I think both paint modes are important. In ue4, you can paint a particular selected channel and you can also chose to display the channel modes.
https://docs.unrealengine.com/latest/INT/Engine/UI/LevelEditor/Modes/MeshPaintMode/VertexColor/index.html
Hope that helps.
Adding an alpha channel to the vertex color will affect several functionalities in blender, but it would be beneficial for gameartists !
For the separate paint modes I’d say no, its basically just changing a color.
(A reworked Color picker would be more useful - not a topic for your GSOC ofc.)
For the viewer I’m not so sure. The artist could just setup a simple emission shader in the material editor to analyze the vertex color channels. (The material editors “Attribute”-Node should get an additional Alpha output of course.)
I think the effort for an artist to set any of this up, isnt high enough to justify putting your time into it. Also it keeps the UI decluttered.
About visualizing the alpha channel while painting:
Like anadin, I think I’d like it going straight to transparency.
Just for good measure:
Photoshop just lays a red mask over the parts where its not transparent.
Alpha values: 0.0 = redish; 1.0 = normal.
Either way a tickbox to disable viewing the vertex alpha would be handy.
I dont know if you can derive from the already existing vertex color functionality in dae, and just append the additional alpha channel in a similiar way ?
If it then works in Unity and Unreal, it surely will work in other modern engines too.
If not, lets work as a team here and blame them for not adapting to common (your) practises.
Seriously: If you somehow successfully feed it to the fbx & dae formats it would be instantly compatible I guess.
Low priority: Making as many formats compatible with this new vertex alpha would also be great. To maximize consistency.
Thanks so far, Howardt and Darshun !
Hi! It would be great to have the ability to paint rgb channels sepparately with black/white brush. For example we can paint bw masks in R and G channels, and use B channel for ambient occlusion.
Sorry for my English…
I have been wondering for a while if it is possible to have a mesh resolution paint brush for vertex painting kinda like dyno brush in Blender. Rather than subdividing your mesh to get the resolution you need to paint details why not a resolution brush based on brush size? Say you have a cube you need to add very small paint details in certain areas. Rather than subdivide the whole mesh to paint these in, just select a brush that subdivides the area based on the bush size and applies the selected color as you paint. Saves a lot of time and very convenient to use especially if you don’t need to sculpt the mesh or you are vertex painting on a flat surface say a wall for bake later to low poly meshes?
I happened to be doing some more vertex colouring work yesterday. Another brush idea that would be super handy would be a polyfill/uvfill brush.
I often use the face mode to select a number of faces then shif+k to set the colour of all those faces/verts to a specific UV color. I work on a lot of mobile games. 
This is a little clunky, even with circle paint. Press the Face Select mode button, a, a c, shift+drag, esc, shift+k repeat.
The having to hold Shift to select is another small impediment to the flow, so much so that we made a add-on internally to copy face signed materials to the vertex colour.
This would be something super useful for all painting modes btw.
One small (?) thing I’d like is for mesh edits to avoid polluting vertex colours. What I mean by this is when you modify the mesh by using tools such as loop insertion or vertex/edge slide, Blender (very usefully) blends existing vertex colours to create the colour for new or modified vertices. However, I think the blend function is not keeping the channels separate, so it’s possible that, maybe due to floating point rounding or whatever, you sometimes end up with a tiny amount of channel pollution, which can be annoying if you are using one channel to tell a shader to scroll uvs in reverse or something like that, as even a tiny amount of pollution in that case can completely mess up the result. If there is a good way to avoid this, that would be awesome.
Thanks for the great work! I was wondering if this vertex paint improvement will involve some improvements in performance, I found vertex paint a bit slow when its used in big models(lots of polygons) also when doing “paint/dirty vertex color”. Thanks
This was already adressed by nathan vollmers GSOC2016 vertex paint improvement, prior to this! 
To my knowledge though, nathans work isnt yet in stable builds (nor buildbot ?), but it will come eventually.
darshans vertex paint improvements are already building on nathans.
thanks that are great news !
Hello, I have made the changes in fbx exporter to handle the alpha channel,
Can someone help me for how to check whether it is working correctly or not,
or anyone ready to test it?
Any useful resource will also be helpful.
Thank you.
I use vertex color as a masking source to get sharp boundaries between materials when creating a note tree for cartoon shading.
However, there is one uncomfortable thing about it: it does not have the ability to do this when you want to select a mesh by color.
I think it would be great if you could choose faces by color like the material selection feature.
However, this selection option may include the concept of threshold because the colors on the faces can be painted as blending instead of some mono colors.
Thank you for your project.
oh wow! yes I will, I’ll have to build so will sync and build tonight
