Bugfixes and performance issues

This week is the last week that I can work on the project. The following week is for the graduation show preparations only. So the plan for this week is to only work on bugfixes and tweak the behavior and keep myself from adding cool new features or a whole new behavior goal.

Last Thursday I finished the peripheral and direct vision. So all data collected from those view cones is finally handled.

At Friday I started to do some more serious testing. I found a quite serious performance hit after I added several boxes to the level. When the agent or the player got near them the FPS would drop down to one! The problem was found in the navigation blocker property. When an actor got close, the boxes physics would exit sleep mode. A box would then start to send messages with it’s new transform. When the navigation blocker got this message it would update the mesh, even if the box hadn’t moved at all! It was a bit cumbersome to figure out what went wrong but luckily it was very easy to fix.

Today I think I fixed a bug where the agent would rotate back and forth when it discovered the player. I don’t really know the cause of the failure, but I saw that the rotation where updated very early in the code that handled the path-following of the agent. I changed it so that the rotation is only updated when the actual movement also is. It seem to have fixed the problem so far.

Peripheral vision

I’ve checked the project plan to see what features I had left to implement and found out that I’ve written the plan a lot less detailed than I remembered, the fact that you as a player should be able to hide beneath something is never mentioned. So according to the project plan, all required features of the project have been implemented!

So with time to spare I’ve started to improve the agent even more. The most obvious flaw for now is that the agent have a very narrow vision. So if the target is close to the agent and slightly to the side it’s never spotted. To combat this I plan to use the same system as in Thief: The Dark Project, where each agent has several view cones to emulate focus, peripheral vision and such.

One quick idea I got where to move the near clip plane of the agents camera. By increasing the distance, the near clip plane will increase in size, resulting in a wider area of vision close to the agent, perfect!

A view frustum. Each side is a clipping plane.

I got it working, but I didn’t quite think it through. With a wider plane, the plane will start to clip through walls. This results in that the agent can look behind walls when it turns around and the near plane spans across both sides of a wall.

So in the end I’ve done it exactly like Thief with several view cones. I got one for the peripheral vision and one for direct vision. Direct vision is used to cover the area close to the agent that falls outside of the view frustum. An entity is added to the view cone of highest priority it exists in so if an entity is inside both cones it will only be added to one. Entities inside the direct vision is added to the list of visible objects. What I’m going to do with entities inside the peripheral vision isn’t fully decided. For now the idea is to increase the probability at those locations causing the agent to gain a higher interest for those areas.

Oh, and I finally added the ability to reset the level. So now I doesn’t need to restart the application every time I’ve need to test something several times.

Looking beneath objects

The end of the week is here. The goal this week was to finish the logic system. I would say that is the case as these past few days I’ve only been implementing the behavior of the agent. The system works and can easily be extended with more behaviors and goals for the agent at a later stage.

So far the agent can:
Roam around to find the player, this uses the POM to steer where the agent should look.
Follow the player, as long as the player is visible for the agent, it will follow.
Search for the player, uses the last memory of the player and the POM to steer it’s search.
Look beneath objects, if the player goes somewhere the agent can follow, i.e. hides beneath something, the agent goes as close as possible and bends down to look at the player.

The last behavior isn’t fully functional. For now, the agent goes to the closest position and bends down. But what if the player hides in a tunnel, where only two sides is open and the closest position is next to a tunnel wall? The agent should realize this and try to look through the tunnel at one of it’s open ends. The plan for now is this:

Straight from my position, is it possible to see the target if I bend down?
Yes? The go straight ahead and bend down.
No? Where did the target enter? Go to that position and then bend down.

This will make it seem like the agent is aware of the fact that there’s specific openings to look through.


This past two days I’ve been hard at work on the logic for the agent.

As I mentioned earlier there was a need to redo the pathing of the agent. This was the first thing I finished. After that I threw together a quick goal-driven logic system to implement the easiest action, namely hunting the player. It might sound like a big thing, but when you think about it, it’s all about following a target if I see it. If I can’t see it, some other behavior can take over.

It was quite easy to get functional, and by implementing this basic system I had a better idea of how to design the whole creature system. It consist of three major parts, Sensory Systems, a Working memory and logic. The sensory system feeds the working memory with MemoryFacts about the world. For this project only the vision system is implemented, so raw data about what have been seen is used to create a VisualMemoryFact, it saves away the entity seen, the position it was observed at and the velocity it was traveling at. It’s easy to extend with new types of memory facts for say sounds or smells. When a memory fact is added to the working memory, it is compared to the facts already in the memory to detect if the new fact is just an updated version of an already existing memory. I’ve also added StaticFacts as I call them, they’re used for things the agent should always have knowledge about, such as the POM, the entity it’s hunting for and such.

The logic system then uses these memory facts to act upon. In a perfect world, no queries to new data, say the entities current position, should be made from the logic system. However in reality I have to do it at some occasions as the visual data is gathered on the render thread, so there’s a bit of delay between memory updates.

To get it all into Nebula I use a property to act as the “brain” and glue it all together. It contains the working memory for the agent and takes care of translating raw sensor data into specific memory facts.

Logic and refactoring

This week the goal is to finish the logic system for the agent. I had done some basic groundwork where the agent would use a FSM made up of lua scripts. However, due to some limitations on how Nebula handles the scripts (like that there’s no return value from a script function) I had to redo it in a different way.

As everything has to be done in code now, I also decided to change the whole pattern used as the choice to use a FSM where due to using scripts and the whole system would be easier to design in that way. So now I’m using Goal-driven behavior or Goal Oriented Action Planning (something I researched during the initial phase of the project). The gist of it is to use abstract goals. These goals can consist of smaller subgoals, that in turn can have subgoals. This chain keeps going until a small enough goal is encountered that can be performed as a concrete action, like GoToNextPathPoint or PlayAnimation. I got the basics up today, but with it I realized I now have to redo my path planning, because at the moment it’s pretty much impossible to get the logic to tell the agent to follow a new path due to the path planning being so well contained inside its own entity property.

Normalization issues

I fixed the problem with the probability growing beyond 1. As I sat down with a clear head this morning I divided up all parts of the calculation so I could inspect each intermediate result. The error lied with how the probability where divided to each nodes neighbor.

In the initial formula the diffusion constant where divided by a constant number multiplied with the sum of probability gained from neighbors. This constant number where the amount of connections to a node. The nodes in my map has a varying amount of neighbors so I just replaced this constant with the amount of connections this node currently have. What was a problem is that the formula described how to update the probability of a given node so it’s when I replace the constant with the current node connections that the problem occur. That whole part reflect the amount of probability gained form neighbors but the constant describes the probability given to neighbors. So to fix it, instead of getting the sum of a nodes neighbors probability and then multiplying, I let each neighbor do the whole calculation and give me the exact sum this specific node will give to a neighbor.

I then went on and tried to implement a velocity based spreading. It causes the probability to spread more in a given direction such as the last seen direction the target was moving in. I got it working but also here I had problems with the probability growing over time. In the end I just got tired of it so now after the new probability have been calculated it’s renormalized before culling of the visible nodes is performed.

I don’t know if this will come back and haunt me later but things seem to be working as expected for now.

Chrashes and assertions

Quite an eventful day today, but not in a good way. I got some progress in the early morning: I finished up the POM node culling and added confirmation of the target so the agent chases you around. There I found a problem that if the agent gets to close to you the most probable node will outside the view frustum due to it being so close to the ground. So the agent would get stuck in place. I got a solution for this, simply elevate all points to the height of the camera when I do the visibility culling and then use the actual position when it’s time to figure out if a point is beneath something.

Before I could start with that, Nebula decided is was time to assert. It asserted in a really strange place during the render loop. The strange thing is that I didn’t touch any code related to the rendering as I know it. As I tried to fix it I re-exported the game data and then, as I found out later, the skybox decided to stop working and crashing the application. So most of the afternoon was spent to get things to work again. I found out the cause of the crash, the skybox, and was able to fix that with some help. And the assertion disappeared as suddenly as it began so everything works now but I really don’t like it at all. As I don’t know what caused the assertion to fail or what got it to go succeed again I’m a bit anxious I’ll do something that will cause it to fail again.

3D Probability Map

These past days I’ve been hard at work with how the probabilistic occupancy map will be checked against the synthetic vision. As it turned out, my initial design where unusable.

My first thought where to render the map at different heights. This is so that the agent wont try to look beneath a box as the node at that level will be flagged as disabled as it is inside another object. The node on the next level is then considered the lowest one and the one used for logic.

The important thing to note is the word “render”. This where cumbersome to get working as I wanted to only get some objects, the nodes, to be rendered with a specific view. At first I tried to toggle the visibility of the objects so they would only be visible when the correct view where processed. This didn’t work, and I think I know why now. I was toggling the visibility of the objects but not the meshnodes the rendersystem used.

When this proved fruitless I my second thought where to put all these invisible entities in another stage, but the more I looked into it the more discourage I got as it would involve a third renderpass.

Then it hit me, I just needed to make another shader material that only the vision shader could/would render. With this I finally got something on the screen but to my dismay it was painfully slow even with instanced rendering, something I had hoped would make this a feasible solution.

So I looked back into the research papers and found the test-point method. Basically it translates a world point into NDC space and checks the depth of the point against the depth-buffer to determine if it is visible. This is fast enough to be used in real-time and it’s very easy to add my imaginary map planes to allow the agent to realize if the lowest node can’t be seen.

So all in all I spent two days to confirm that the culling present in the rendering pipeline wasn’t suitable to be used for visibility confirmation against such a dense grid and learned that I should read more carefully.

I see you…

Happy times! I got the world to render from the agents point of view.


What the player sees, the yellow triangle is the old 2D view cone for the agent.


What the agent sees in full 3D!

It was a bit of a headache to get working. I added a camera onto the agent and created a new view that would use the picking shader to render to an off-screen buffer. Through the debugging webinterface for Nebula applications I could see that the buffer where rendered to correctly but the whole screen kept flickering to black and back. After doing some debugging with Gustav, one of the developers of Nebula Trifid, he finally discovered the cause: The back-buffer where swapped after each view had been rendered. It’s fixed now so I can continue working without the danger of getting a seizure.

During the off-time when I couldn’t work on the agents vision, I finished the groundwork for the behavior state machine so it can now load scripts and change between them.

3D vision

Today I started implementing the actual vision system for the agent. As I mentioned earlier I will render the scene from the agents point of view and scan over a downsampled buffer to detect what have been seen.

Nebula have an easily extendable render system. There’s some parts needed to get a new add on working as the rendering occurs on another thread but it’s quite straightforward and there’s a lot of finished modules to look at and learn from. I was able to finish the skeleton for the module today so there’s a lot of time left to actually implement its functionality. And one thing I discovered is that there’s a shader available for rendering objects color-coded, so I don’t need to write my own to do that!

Though I didn’t get going with the vision add on until the afternoon as I wanted to get more information about the render system so I had to wait for answers from the people working on the engine. So during the morning I started to implement the logic system. The idea for now is to have a finite state machine where each state is a lua script so it’ll be easy to change and test during development, as scripts can be reloaded without the need to recompile the application. I didn’t fully finish the whole groundwork but I got the most basic components setup so it will be quick to finish when the time comes.