The Heart Game

[ArtOfLight]

Next update is done! I went ahead and did new game builds as well found here: http://www.artoflightandmotion.com/downloads One of the biggest things this update is I finally finished the explanation of how to recreate the seasonal changes that are inside of the artwork/game! That can be found here: https://www.dropbox.com/sh/v03321o6yk41zx6/AABeZJFVSCkImy-XB50CLCYga?dl=0 and more permanently in the Free Resources section in the first post. So if anyone has been interested in knowing how all of that works, you can finally see it!

Another big thing on this update was getting the system worked out for having leaves on the ground that appear and disappear during the Fall. It took a while to create everything for it, as it often does when you’re doing something new, but the effect seems to work quite well in the end and adds a lot to the scene. There were also a few times that I wasn’t quite sure if it was actually going to work, but I kept having ideas for things and it does :D! Here are some images of how it looks:

One of the biggest problems I had with it was getting the meshes to flow with the terrain and along steeper hills. Previously, I had an idea of using a heightmap of the terrain that I made in Blender to then displace each vertex according to the heightmap, by first bringing all the vertices to point 0 on the Z axis (or Y in other programs). That way everything is flat. After that I subtracted from that point 0 to bring all the vertices down to the lowest point on the landscape. Then you can just multiply the Heightmap by a large amount, and add that to the previous calculations, which gives Displacement that lines up with the actual terrain.

This almost worked, but I was still having problems that I wasn’t really sure how to solve, as can be seen in this image of how the leaves look like stairs:

Initially I had thought that the problem was the texture resolution wasn’t high enough, which seemed reasonable because of how it looked pixelated, but I was already using an 8192 x 8192 texture size and couldn’t reasonably go any bigger than that with how much GPU memory that would take. While thinking about all of that, I all of a sudden realized that I could actually shrink the area that I was doing the Height map for. Originally, I was trying to do the whole map in the texture, as well as some area around the map, so that 1 pixel would equal one meter in the game world, but I realized I only needed a Heightmap for the playable area, which was only about 4000 meters. So shrinking the amount of area I was doing a Heightmap bake for increased the pixel resolution. To my dismay though, it still had that stair stepping in the same places, which meant it wasn’t a resolution problem. I should also mention that because I needed two displacement maps of the level to make things work for on top of the large rock features and under them, I was using the RGB Displacement Map compression settings in Unreal that allow for multiple color channels, and which I thought should work just fine for Displacement Maps like this. I was also always saving the images in 16 bit, rather than 8 bit.

So after that I thought it might be something with the actual way it was rendering in Blender, and when I looked at the texture, I noticed that it had that banding in the same place as what’s seen in unreal. It’s very faint and difficult to see, but if you move this image around a bit while looking at it, you can start to see the banding going on where it looks like there are flat surfaces going up the hill instead of a smooth gradient:

So after discovering this I started trying all kinds of different things in Blender to see if I could get rid of the banding, using things like dithering, which only added noise to the displacement in Unreal, but still had the banding clearly visible. I also tried blurring the texture a bit, which did help, but then all the details get lost and it still has banding, even though it’s less pronounced. So at this point I had tried pretty much everything I could think of but to no avail, the banding was still there. Then as a fluke, I was messing around with different things in Unreal, and I tried using the HDR compression settings and all of a sudden, everything magically worked :D!

As far as I can tell, the reason things weren’t working before is that the RGB Displacement compression setting takes the texture down to 8 bits instead of leaving it at 16 bits, while the HDR setting leaves it as 16 bits. The really weird thing though is that, with the HDR setting, it still has the banding going on before the texture gets compressed, and it’s only after you apply the compression that the banding goes away and it’s using the values that are actually in the texture instead of some kind of compressed texture, which seems quite opposite of what you would think it should be. It’s also quite interesting that the banding that can be seen in the texture in the image above isn’t actually affecting anything like it seems it should have been, even though it’s in the exact same spot as where the banding was in Unreal. So there are a lot of things with it that I don’t understand why it’s the way it is and it’s kind of a Duct Tape solution in the end, but at least it works

So just kind of an interesting story of compression in Unreal and if you want to have an RGB image for Displacement in what I think is also 16 bit, the HDR compression seems to be the only one that actually does this. It’s possible the displacement compression that says 8/16 bit would work as well, but I haven’t been able to get it show anything but a white image when I try it, whether or not I’m using an RGB image with three channels, or a Black and White image with only one channel. So I’m not sure how that one actually works.

There were a few other things that happened as well in this update. One was the Aspen bark texture I was working on a few updates ago is finished. Here it is in Blender and Unreal. I was going for kind of a damaged look where it seemed like some of the bark was starting to being ripped from something pushing out from within, like I’ve seen in some photos:

With that, I also had an idea for rearranging the Bark texture atlases so that ones that have more unique damage to them, like that one above and another one I had made, appear less often than a couple of the others that I’m going to be re-sculpting that won’t have any damage on them. This way the forest should feel less like things are repeating and damaged ones like above shouldn’t ever be right near each other.

Another thing that happened is after doing that test with all the trees last update, I started getting into a bit of performance profiling. After playing with things a bit, i think I found the main culprit for the strange changes in performance when the sun is at a low angle versus pointing straight down. Apparently to do the shadows, the light source has a depth pass that is basically the same as the camera’s depth pass, only it’s from the light source’s viewpoint. I think this is then used to help produce the shadows later on in the rendering pipeline. In order to do this though, the computer has to calculate where all the different model’s triangles are in the scene, but do it from the light’s perspective instead of the player’s. I think also if there is transparency on those triangles from the material setting, it also has to calculate triangles through other triangles that are seen because parts of the material are transparent. So when the sun is down low, there are a lot more triangles being processed because the light source is looking through lots of meshes all at once. Then the higher the sun gets in the sky, it’s looking through fewer and fewer meshes, so there aren’t as many triangles being calculated. Here are a few images of the tests I did. These first two are of the shadows at their highest quality settings and you can see how both the amount of triangles and the time it takes to do the shadow depth pass both skyrocket at sunrise versus midday, while everything else stays pretty much the same: (This is with shadows on their best settings)

Afternoon

Morning

The same thing is happening with Medium Quality Shadows, just not as big of a jump because there aren’t as many shadow fulstrums being used, or rather times the shadow gets calculated:

Afternoon

Morning

The interesting thing about this is that I tried putting the cull distance for the foliage, which is the distance from the player that the foliage stops being rendered, at a really small amount, so it was only the trees right next to the player that were still being rendered. In the end it didn’t actually have any effect on performance. So to me that means that it’s only the vegetation right next to the player that’s causing the really low performance, which is both a good thing and a bad thing. It’s a good thing because it means that having the whole map populated by trees isn’t going to be a big issue. It’s a bad thing though because it means that really all I can do to have better performance is to not do things super dense. I wasn’t planning on that anyway, but it does mean I’ll have to be careful of how dense the vegetation gets.

As part of that, I also added a bunch of options for the shadows in the options menu, so people will be able to decide what things they want on or off instead of it being decided for them by having only one slider. The smaller Aspen bushes also having LODs on them now, which helped more than I thought it would with performance. Since they were being culled if the camera was far enough away from them, I didn’t think they would need them, but I guess they still did. It actually helped with them staying more visible in the distance as well because I was able to add a slightly changed material for the ones in the distance that wasn’t as transparent, which made Unreal keep showing pixels from them longer, instead of just culling them because they weren’t contributing to the view enough.

A rather interesting thing that happened that made me go on kind of a rabbit hole adventure was that I had run into a blog post or something about the MineCraft developers having to create a system where, if there was an old save file from a previous game update, they had to make a way for the computer to look at any of that data in the save, then transfer that data to the way it was currently being used in the new game update. I wasn’t completely sure what they were talking about, but thought it was kind of neat, and moved on to something else. Then just like two weeks later, I started working on adding those options for the shadows and a few other things, and lo and behold, I realize I actually am going to need to figure out a way to take old save files and make them compatible with the new options I added, as well as make it so that the shadow quality setting I took away doesn’t run at start up and override the things I added when it gets loaded from an old save file. This all kind of led into changing the way a lot of the code worked for saving and loading, and I had to change a bunch of things to make things easier. I’ll have to see over the next updates if things work, but it seems like it should, and if it does I might write something about it.

So that was a neat thing that the blog article I ran into just a couple weeks before kind of gave me a place to start when I realized I would need to do that too.

I guess that’s everything for this update. Until the next one

[ArtOfLight]

Finally have another update ready! That’s the trouble with starting such long projects is that you usually start them when things aren’t too busy, but life isn’t always that way, so you have to kind of let things ebb and flow :).

For this update, the main thing I added was the grass. I added new Game Builds here if you would like to see it: http://www.artoflightandmotion.com/downloads , and here are some images of how it’s looking:

I spent a lot of time experimenting with it, trying to figure out what the best method might be for it in the end, working with the way the season’s function as well. In the end, I went with kind of a mix between small grass cards and individual blades of grass. One of the main difficulties with the grass is that, when you add seasons into the mix, the grass has to start disappearing in Winter, and if you’re using only grass cards for it, then you run into massive amounts of transparency as you might only have one blade of grass still visible on the card, but since that one blade is there, you can’t move the card out of view. So you end up with a very huge amount of calculations that have to be done each frame because there is so much transparency. Although going with the route of having one blade of grass per card, you can just move that card somewhere else in the level when that blade needs to disappear. The downside of course is that you also have more vertices that need to be rendered. So in the end I went with having between 1 and 4 blades of grass visible on each card, then just had that whole card disappear, even when there were 4 blades of grass on it when it came time for it to do so.

To talk a bit about performance, here are the results of some of the tests I did. I forgot to take screenshots of all of the original tests I did, but here is something I threw together real quick to show the end result of those tests. To start off, here are some of the meshes that I tried:

On the left, it starts out with 4 vertices on one card, then goes to 6, then 9, and lastly on the right, the final model I used for the blades of grass with 359 vertices. For the test, I did it in the early morning part of the game, when performance is the worst because the shadows are so long, and I also did it right when the transparency would be the worst, during the time that the grass goes from being dead to growing again in the Spring, and you have to make the dead grass transparent so that it can look like the grass is growing again. A few more things to note is that I did the tests while in the Unreal editor, so there is a little worse performance because the editor is still open. I also did it with all quality options turned on to their highest setting with 1920 x 1080 resolution. Lastly instead of painting new models onto the landscape each time, I would just replace the model in the foliage settings to make sure everything was in the same place and there were the same number of models in each test.

Now for the results, with the 4 vert card, I got around 53/54 - 69/70 FPS. With the 6 vert card, I got basically the same, maybe a half FPS less. Lastly the 9 vert card had about 51/52 - 68/69. Here is where things get quite interesting though. When I tested the grass blades with 359 vertices, the performance I get is 51/52 - 68/69, which is amazingly the same as the 9 vert card, with maybe just less than a half FPS worse.

So altogether the grass blades actually give about the same performance as the cards, even though there are so many more vertices. I believe the reason for this disparity can be seen here in the shader complexity view, which I believe shows something to do with the amount of instructions the GPU is doing in one of it’s cycles. I’m not sure exactly what those cycles are, but if there are more that need to be done than the max amount for a cycle, or 2,000 instructions, it has to do them in it’s next cycle, which means the frame can’t be rendered as fast because it has to wait for the next cycle. To show all of this, Unreal has a view you can look at that goes from 0 to 2,000 for each pixel. First, here is an image of what I was testing looked like in the normal game view:

Now here is how that looks in the Shader Complexity view with the single bladed grass. While there are some areas that start to get a bit white, meaning a very large amount of shader instructions are being done for that pixel each frame, it’s not too bad for how much is going on in the scene:

The grass cards however are really bad:

Keep in mind too that the white just shows up to 2,000 instructions being done, but the actual amount that has to be done can be quite higher in total. So with these cards, who knows how many instructions are having to be done. I believe this is where the disparity comes in, because even though there are less vertices to render, there are also far more instructions to do on each pixel.

Something to mention with this is that I didn’t have a texture for the grass cards when I did the testing because I would have had to change the final shader I made to do it. So they are fully transparent, but with how little the grass would actually be there at this stage, it wouldn’t effect performance that much, if any. So altogether, in this case, having cards instead of blades of grass is only a couple FPS better, but that’s only if you replace it by just one of those cards. In a more real world case, when using a 4 vert card, you would usually have at least two of those cards intersecting each other, per bunch of grass, to make it more 3D, which brings it to at least 8 verts per bunch. Many games now days also even go further and use the 6 or 9 vert cards instead, and putting several per bunch, which would bring the vertex count even higher, and would actually start to create even worse performance than the grass blades because of how much transparency there is, and the addition of more verts than I tested with. So it’s really quite a result to the test that having such a large amount of vertices could actually work better than cards with less vertices, simply because you don’t have such a huge amount of transparency.

As a note, since those tests were all done while the editor was open, it’s not a very good indicator of what the final performance is. To find that out, I did a build of the game and the final performance in the final built game seems to be above 60 FPS at 1920 x 1080, no matter where you are, what season you are in, or how low the sun is. So that’s definitely quite good. There is a really weird spot though when the sun is pointing in a certain direction, and you’re looking directly away from the sun that it all of a sudden goes down into the 40s. Somehow it seems to be connected with the Distance Field Far Shadows, and I believe the trees that are way out in the distance, but I really have no idea what is going on with it because their shadows shouldn’t even be being rendered when the player is looking away from them. I think I’m going to have to learn some more performance profiling to figure it out, using a program like RenderDoc.

One last thing to mention if anyone wants to replicate this is the 359 vert grass bunch takes up about 1 square meter of space in the game. So you have on average about 450 vertices per square meter, because they can overlap a little when painting them in Unreal. I should mention too that probably the only reason I can get such performance altogether is because of those earlier tests I’ve been doing with not using Unreal’s landscape system, but going with a static mesh landscape that I import from Blender. Without that, performance would probably be down into the high 40s at this point as the best.

Now that all of that is explained, there are some other things I’ve added to the game. One thing is I also came up with a way to have dead grass on the ground in meadows and places like that, where it would just look strange to have nothing there when there isn’t snow on the ground during the winter. What I was trying to replicate is this kind of idea, only for right now, with shorter grass: https://pixabay.com/photos/landscape-early-spring-2083440/

Here is another image of the basic idea I’ll be going towards as I get more grass types created, where it will be a bit like this image at first, but as the winter progresses, more and more dead grass will fall over from the snow, and eventually most of it will be lying on the ground like the above image: https://pixabay.com/photos/fog-foggy-nature-scenic-forest-1122507/

Here are some images of how that is looking in the game. It’s still not quite the way that I wish it looked, but at least the method for doing it has been all figured out. Since they are on the ground, I also went ahead and used cards for these:

One other pretty cool thing for the grass is I was able to figure out a way to have the grass blades go away over the course of the entire winter, instead of just before the year ends. To give a little explanation about why this is a really neat thing, the way I have been doing the seasons was to start off the shader in a state of being in winter at the beginning of the graph. So there are no leaves or anything like that. Then as the year progressed, I would gradually override this with the Spring leaves growing into full leaves, then override that with the Fall colors on the leaves, then override that with the leaves disappearing and the whole thing going back into winter. So it was a very linear sort of thing of just overriding each effect as the year would count up in days, but there was no way to have any effect overlap from one year to the next year, and for instance, the leaves on the ground had to all disappear by the end of the year so that it would match with the starting point of having no leaves at the beginning of the next year.

The problem with this whole thing though came with adding the grass because it doesn’t look right when you have just a barren wasteland without any grass during the winter, and all you have is a texture on the ground. So I had to add a way for the grass to basically halfway disappear by the end of the year, then in the next year, continue where the other year left off and finish disappearing right before Spring, when the grass would need to start growing again. My goodness, things definitely got a bit complicated there for a little while trying to do that :). Mainly because of how much the transparency and colors have to change throughout the year, and some season effects have to basically happen at both the beginning of the material shader, then also occur again at the end of the shader when winter comes. It worked out in the end though, so that’s good :). What’s great about it is that now there is much more continuity for the entire winter season, and I can also add the same thing to other things, like those leaves on the ground, which I think really do add a lot to the way the winter months look.

A couple more things that happened are I got some more Aspen bark sculpted. For these, I was doing a very generic sculpt that could be applied on a lot of the trees. This way the more unique sculpts feel more unique when you see them, and the main thing is that you don’t really notice that you’re looking at the same textures as much while walking around, unless you’re actively looking at the trees. I was also able to use some of the generic black “eye” areas I had sculpted from other bark sculpts, then add unique black “eye” areas around them. So that saved some time. Here is how those look. The second one is only partially finished, and the lines show the area that will actually be on the tree, as well as what parts would be visible when you can only see a quarter of the tree at a time when it’s applied on the tree:

The other thing was I finally found a solution to the way the branches on the trees would disappear during the winter as you got further away from them. This is something that I have been struggling with for quite some time, and the solution actually came as I was researching different ways of doing grass. In the end, it has to do with the Mip textures in Unreal. These are basically LODs for textures that make the texture resolution smaller as you get further away from it. I found out the problem was that the Mip textures were getting too blurry for branches to still be visible as you got further away from them. So I adjusted the Mip bias in the Material Editor for that texture so that it would use a more high res than normal texture, and it all works fine now. No more disappearing branches!

And I guess that should be everything for this one. Until the next one