Archive for category Netstorm

Optimization of Draw Calls

I want to talk about what I think is a pretty cool trick for optimizing the number of draw calls in Netstorm 2 an RTS.  Dealing with draw calls in an RTS is more challenging than, say, an FPS because the player can create a large number of objects over the course of the game.  In an FPS, you know how many objects there will be (except maybe for missiles).  In an RTS, the point of the game is to dynamically create units for your army, and each one of these might be a draw call, which means it can get pretty expensive, pretty quickly.

The Bridges of RisingStorm County

There are a large number of bridge units built in a game of Netstorm

Netstorm has an additional challenge: one of the principle “units” a player will place is a bridge.  This is free (although limited to how many you can place every few seconds) and is used to maneuver.  It is in the player’s best interest to place bridges as fast as possible.  In an average game there are hundreds of bridge pieces being built.

Bridges of RisingStorm

The bridges have Tetris-like shapes

Each bridge piece is a Tetris-like shape that is made up of smaller atomic pieces, e.g., a straight, a corner and a tee.  Each of these under normal circumstances will be a separate draw call.  However, thanks to the Mesh Combiner script provided with Unity, these are very easily combined into the Tetris-like shapes.  However, there are still hundreds of Tetris-like shapes to deal with.

Small diversion:  A few years ago, a friend of mine and I worked on a game engine ourselves.  For better or worse, I pushed us into using threads aggressively.  One cool benefit was that since systems acted in independent threads, we could avoid the situation where the may game loop has to run as fast as possible to service any possible need.  Thus the CPU would only be used as much as it was needed, which for many situations was hardly at all.

In order to deal with these many independent systems we set up a publish/subscribe mechanism to decouple dependencies between the systems.  We also had several systems which would only run based on certain events happening, rather than running in a loop.  We created a central dispatch and scheduler which we called the Hub to manage the execution of these systems and dispatch messages to them.

Recently, I saw a way of bringing back some aspects of the Hub into Unity.  Specifically the dispatcher and the scheduler.  The dispatcher uses Unity’s SendMessage interface, but instead of needing to know which game object you want to send to, you just send the message to the dispatcher.  Any gameobject that wants to see that message simply needs to register for it.  One of the ways I use this inside Netstorm is to turn shadows on and off when the FPS are too low or safely high enough.  When the FPS drops below 15, a message is sent to the dispatcher saying “LowFPS” and the shadows disable.  Once the FPS goes back above 60 another message is sent and shadows are turned back on.  Because the FPS checker doesn’t know who is getting the message other effects could also register for the low and high FPS to decide when to turn off.  I could also use this for Level of Detail changes.

But, going back to the bridges, there’s an even cooler trick.  Running the mesh combiner is expensive and Unity’s dynamic batching is limited to 300 polygon object.  It would be nice to have something in between that we can choose to run as needed.  This is where the scheduler comes in.

It works as follows, when a bridge is placed it checks to see if it has been connected to another bridge or if it is independent.  If it’s independent it creates a parent object that contains a scheduled mesh combiner.  This mesh combiner will run 3 seconds after it has started.  However, if another bridge is placed connected it’ll child itself with that existing bridge’s parent and the scheduler will be pushed back by 3 seconds.  Once the player stops building a particule bridge path for 3 seconds the combiner will finally run and combine all the bridges.

Here’s what the code looks like:

public class MeshRecombiner : MonoBehaviour {
    public float recombineDelay = 10.0f;

    private bool dirty;
    void CheckRecombine () {
        if (dirty) {
            Recombine ();
            dirty = false;
        }
    }
    void MarkDirty () {
        foreach (Transform child in transform) {
            if (child.name == "Combined mesh") {
                child.parent = null;
                Destroy (child.gameObject);
            } else {
                child.gameObject.SendMessage ("MarkDirty", null,
                    SendMessageOptions.DontRequireReceiver);
            }
        }
        if (!dirty) {
            Scheduler.GetInstance().AddSchedule (recombineDelay, "CheckRecombine", false, gameObject);
            dirty = true;
        } else {
            Scheduler.GetInstance().AddSchedule (recombineDelay, "CheckRecombine", gameObject);
        }
    }

Notice a few things.  First off, there is a function called “Recombine” which is called by “CheckRecombine”.  “CheckRecombine” is what will be called by the scheduler.  “Recombine” is essentially the “Start” routine from the “MeshCombiner” class that comes default in Unity.  One change is that a child object “Combined mesh” is always created regardless of whether there’s only one material.  This is to simplify the “MarkDirty” code.  The “MarkDirty” function is called when a child is added or removed or modified in some way that requires recombining (e.g., a material has changed on the child).  This “MarkDirty” then disables all the current combinations sends message to all the children who should enable their individual renderers again.  It then sets up the scheduler to call it.

Here’s an example of a simple child which can be combined:

function Start () {
    transform.parent.gameObject.SendMessage ("MarkDirty");
    enabled = false;
}

function MarkDirty () {
    var renderers;
    renderers = GetComponentsInChildren (Renderer);
    for (var r in renderers) {
        r.enabled = true;
    }
}

The final piece to this code is of course the Dispatcher and Scheduler.  However, they are much to large to copy and paste here.  Instead, they exist on github here: https://github.com/tosos/Geewhiz under the project Geewhiz which is the name of the previous game engine I worked on.

Using this method in a situation like the screenshot shown above with all the bridges the draw calls can be taken from something like 700 in that image (with all effects added including shadows) and reduced to on the order of 200, which if it had been enabled at the time the screen shot was taken would have likely taken the frame rate from 3 to something more like 15, at least.  These days it is possibly to get much higher frame rates during multiplayer games of Netstorm.

Hopefully you will be able to get similar speed ups with dynamically created objects using a combination of these techniques.  Drop a line in the comments here if you are able to find some use with these.

1 Comment

Refactoring Part 2

Now I’m finally going to talk about specifics.

In the last post I mentioned attempting to refactor the build panel into both an in-game construction device and a level editor device.  It turns out one of the issues that had been bothering me for a while was in the bridge display.  Every time you place a bridge a new random one takes it’s place.  I put this in the build panel, because it was the obvious place to do so, but it bugged me that there was so much code in the build panel for just displaying icons and now it also had to be responsible for managing bridge replacement when all the other icons for other units and structures just stayed there (for the most part).

The level editor didn’t need this whole bridge mechanism, it just needed a panel that would show icons and allow you to drag them onto the field.  I finally had a reason to split out the bridge code.  It was beautiful.  At least mostly, I still haven’t entirely pulled out old assumptions.  I tore all the code that handled bridges into a separate file.  Because it was well separated I could remove a number of if checks and actually shrink the code somewhat because now I could assume that I was only dealing with bridges in that code.

I’ve been talking about assumptions a lot.  Back in school, it would have been called “invariants”.  Invariants are great, it means you *know* a certain thing is going to be true (or false) at a given time and so there’s no need to check.  Good coding practice will tell you to check anyway, and that’s true, but now you replace an “if this do this” to “assume this or throw an exception” and if your code never throws that exception during testing you did well.

So assumptions aren’t bad, they’ll make the code more efficient in the long run, but may also make it less general.  There’s a choice to make.  I could have written the level editor without reusing the build panel, but I was predicting when I made the choice that there would be a lot of repeated code and not much efficiency to be had.  As I said, it really was the same thing.  But in general, if there’s that choice to be made, I’ll usually pick the one with the smaller code, because smaller code looks like it was easier to do.

It turns out there are another couple bonuses which might be had from refactoring, but which I haven’t yet attained.  I’ll get to the bonuses in a few paragraphs (see that foreshadowing). The stumbling block I’m sitting on is that the bridge piece when placed notifies the build panel that it has been placed so that the build panel knows to randomly select a new piece of bridge to put in there.  Once I split out that bridge control code, the bridge piece was notifying the wrong system.

My assumption, for efficiency was to have the bridge piece just grab the build panel, because every draggable grabs the build panel.  In fact, it’s in the base class.  The build panel is supposed to know enough about all construction to darken the icon while any of its objects is being placed and not allow you to click a new one until you either place or cancel the last one.  So each object which is in the build panel needs to tell it when it’s placed so that the build panel knows it can relinquish the lock on the display.

I could use a publish subscribe model as I mentioned previously for the pathfinder, notifying any subscribers when the bridge is placed.  This would allow both the build panel and the bridge controller to get notifications and each do the appropriate thing with a good split. But this is a larger chunk of code that I’d have to write and in reality, it’s only this one case.  So having a full publish/subscribe is a bit over the top and a little to much architecting.  At least it feels that way…

I mentioned bonuses that I haven’t yet attained.  I haven’t attained them because currently the build panel still assumes there are icons for bridges at the top and still in the build panel.   The build panel then has to call into functions in the bridge controller to let it know when it gets notification from a bridge piece.  There’s a level of pointless indirection in that call.  The first bonus then would be that the level editor uses that space to place icons like “save/load” and “quit” as well as other editor specific icons like the player’s priest (his avatar in the game) which sets his starting location.  But because the build panel currently assumes bridges in those locations, the save/load button becomes “bridge slot 1″ (by a function call, SetBridgeIcon (1, “Save Icon”) normally used by the bridge controller when a new bridge is ready to replace the icon) and the priest icon is “bridge slot 6″.  Just arbitrary bridge locations instead of arbitrary icon locations.  And it looks silly.

The other bonus I would receive by fixing this is that there is a bug in Unity with the gui elements.  Because I’m changing the icons, on these buttons it redraws them.  For some reason, Unity’s gui doesn’t remove old draw elements, so as you place bridges, the draw calls goes steadily upwards.  Likely it’s drawing nothing, but a draw call is a call out to the graphics card and bridging is a very common thing so much that thousands of bridges will be placed during a game.  1000′s of draw calls will cripple the graphics performance.

By just establishing a graphical placeholder in the build panel and letting either the editor or the bridge controller actually draw in the icons they  need I can remove the sillyness in the editor of setting “Bridge Icons” and in the bridge controller I can optimize out the GUI bug.

So I still haven’t arrived at a solution.  I may just go ahead and write a publish/subscribe mechanism with as many assumptions as I can put in to keep the code as small as possible.  And maybe I’ll figure out a way to refactor it into something smaller later.

No Comments

Refactoring Part 1

I’ve started writing about this because it’s better than talking to myself and that’s how I work out problems in code.

For a little background, I started coding on RisingStorm,which is a remake of an old game from the 90′s called Netstorm,  back in late June (I actually started looking at it earlier, but then I had to travel to France for work, and who can work when you’re sitting in Paris?!  I mean come on!)  Since then it’s been coming together nicely:

Here’s a video someone put together play testing (he even used a cheat to give himself extra money): Rising Storm play test build The music is from Netstorm not from RisingStorm.

It’s funny, that was build 9.  Of course, that was build 9 after I started performing a build release cycle.  Before I would just make a few changes and upload them.  Occasionally I still do, but there are advantages to doing a cycle… if your testers pay attention to build numbers when submitting bug reports anyway :)  Perhaps I’ll talk more about that some other time.

This post is going to be about refactoring.  I just uploaded build 20 which contains some basic multiplayer and uses levels constructed in the in-game level editor I put in around build 17, I think.   Now I’m taking a moment (more like a day or two) to think through the code and do some reoganization.

There have been some elements I’ve been thinking about fixing, but not necessarily doing so.  The thing that’s been working really well for me and especially in a diverse, internet-only group is rapid deployment. When everyone can see and comment very early on it keeps progress smooth. This is a sort of agile formalism, but treating agile formally always strikes me as silly.  The idea is that you get something that works, but probably isn’t quite the solution you want, but it’s better to push it through and come back to it later when there’s a driver.  Despite what people think, coding very often isn’t engineering.  It’s more like crafting, like chipping away at a raw element (an idea) and shaping it.

I once heard that when Michelangelo was crafting his David out of a single block of marble, he did so by chipping away at the block.  He said the block gave him his sculpture that he was just pulling it out (sorry wikipedians, no reference, this is just hearsay).  And in the course of chipping away at the block he discovered a fault in the stone around where David’s hip was.  In order to work around the fault, Michelangelo gave David that bend he has in the upper body which starts at the hip.  In the same way, coding a raw formed idea can’t be forced, it has to be moulded.  Coding new ideas requires crafting them.  You can’t put the Requirements or Specification before the horse.  Horses eat hay. (RisingStorm is a remake, but making an RTS like Netstorm is new to me, so the ideas are new to me… otherwise it wouldn’t be nearly as interesting for me to do in my spare time :)

One of the first, simpler, cases of refactoring I did was in how the build system works.  In Netstorm and Risingstorm, there is a build panel where you can select either units or bridges to place.  These operate very differently after they’ve been placed, but placing them isn’t all that different with one exception.  After a bridge is placed it just appears there, but when a unit is placed this little sparkly thing needs to pathfind over to it and teleport it in.  Well, originally I started with the bridges (they are the most important element in the game) and just got something working with a simple function call.  But when I added in this little sparkly thing I decided to make use of Unity’s message passing interface first to notify on various states, like “OnDraggablePlaced” or “OnConstructionStarted” (draggable is the name of some element which can be placed from the build panel)  There’s also a message that gets sent by the sparkly thing when it arrives and a publish-subscribe mechanism that lets it send to only the objects that care about it arriving.  I’ll probably open source this sometime soon.  Look for the “RTS Source Code Components” at the top here.

One major refactoring I started recently was with the level editor.  I decided to try to reuse the code for the build panel and draggable stuff to allow the user to place objects in a level design similarly to how they would in game.

I discovered along the way that there were very many assumptions I had made about building that weren’t very abstracted.  They worked for in-game building, but there was quite a bit of spaghetti calling back and forth between objects.  Draggables talking back to the build panel in a way that seems to violate controller continuity.  I’m not sure that’s a thing, but it’s related to a real thing :)  I just can’t remember the real name.  The idea is that say when you construct something in a piece of code, that code should also be responsible for removing it.  It’s of course possible to code where you destroy an object into any place and why not?  It turns out if you decide to change something later trying to track down all the weird places you could have possible called that destruction can be a real pain.  Having the Draggables call back to the build panel isn’t necessarily wrong, but right now it’s just ugly.

Every great performer does something which is incredibly difficult, but it looks really easy when they do it.  I think the same should go for code.  It should be very simple to follow the logic.  It should even look so easy someone can look at it and go “Oh I could totally have done that in a weekend.” and yet the crafter behind it went through a painstaking thought process to carefully lay out the code and refactor the code so that it would lay so beautifully well.  Everyone can appreciate David, but not everyone can make a statue of similar caliber.  In the same way, everyone can code, it’s not really that hard.  But not everyone can make really good code.

It’s amazing how much refactoring like this can beautify your code.  It’s like proofreading text.  By the time I’m publishing this, I’ll have read through it a few different times.  When I’m writing through the first time there’s no way I can clearly see what I’ve written after this line until I write it.  I have an idea where I want it to go, but the actual word structure change and veer from my intention.  In fiction writing this is how you create those really great repeating themes or strong foreshadowing.  You may have had an idea on the first pass of how the story would go, but as you were writing it, it shifted slightly or formed in a way you didn’t really expect.  When you go back through the moon needed to be  full for the hero to look into the heroins eyes.  But it turns out that he’s a werewolf.  You can slip a little piece of text about the state of  moon earlier on and then it looks like you’re a really great planner.  “It looks so easy, I could totally write that book in a weekend.”

Enough of the diversions, I wanted to work on the actual problem.  But this post is getting long, so I’m going to think about the writing and save the problem solving for the next post.

Btw, have you ever noticed how the foreshadowing just isn’t as good in a serial like a tv series.  You get vague prophecies, but nothing perfectly clear (except maybe Joss Whedon.  I just don’t know how he does it, except that he doesn’t because nobody lets him finish it… sigh).  Nobody can predict the future, even though they may want to, so the TV series shifts from those original intentions and it may be hard to pull it back in line with consistency to the first season.  Oh, Lost with your islands and hatches, I miss your potential.

No Comments

Switch to our mobile site