Coroutines vs. Callbacks: Maintaining Sanity

Why

I’ve seen a lot of Unity3d projects over the years. The first thing I look for in every project is one critical piece: how does it handle asynchronous tasks. In Unity, there is a limitation that you can only talk to the engine on one thread or “train of thought.” Asynchronous tasks are how the project appears to do two things at the same time, even though they are actually only using one set of tracks. There are two very common reasons to need asynchronous tasks.

The first is for networking with a server. The Internet takes time, no matter how fast or strong the connection you have to it. And you can’t just make the entire game halt while you wait to talk to the server. Everything is on that one track – including drawing your game. If you wait for the network response, your game isn’t drawing anything at all. It just freezes.

The second reason to need asynchronous tasks is to load assets. If you’re loading a new level, it takes time to load all those neat objects and textures you spent so much time and effort into creating. Even for simple levels, most of the time it’s pretty fast. But these devices are doing a whole bunch of other things like checking your email. Sometimes it can take minutes!

Fortunately, most games don’t actually need this asynchronous ability. They can get by with freezing the screen for a moment and then moving on. Therefore typical Unity tutorials do nothing to teach about it. This is a real problem! Tutorials don’t offer much for best practices. This leaves many coders casting about for their own solutions, which often end up overcomplicated and wasteful. Worse is the fact that retrofitting the code is often very difficult.

Coroutines

Unity handles asynchronous tasks with a C# idiom called “coroutines”. They are a way to turn a regular function “inside out”, breaking it down into steps that will stop when the coder says so, allow it to do something else, and then come back later to where it was. The technology is part of the C# standard, which turns a method containing “yield return thing;” into an IEnumerable class that the caller uses for its side effects.

What Unity adds is StartCoroutine to most of the common classes to run those IEnumerable classes for you. This works fine in most cases. Until something goes wrong! Unfortunately, error handling isn’t included. If the coroutine method throws an exception, the most you get is an error in the log. It completely breaks anything that’s waiting on the coroutine, just jumping off into nowhere, with hardly any way to recover or even find out what happened!

Another place coroutines fall down is for returning values. Coroutines cannot have “out” or “ref” parameters. This means to get a value out of an asynchronous process, you have to pass in an object that will get the resulting value and then read from that.

Callbacks

Another standard C# idiom is the “callback.” These actually seem easier to use than coroutines, because you can create callback handlers just about anywhere. At the place you start an asynchronous call, you can immediately declare an “inner” method to handle the result of that call. All of it just happens for you, nice and clean.

Until you have to chain a bunch of tasks together, and properly deal with errors at any stage of the tasks. Suddenly you’re facing what I will refer to as “boomerang hell.” This is where there are so many callbacks and inner methods that the intent of the code is unreadable. Let’s start with a real world example. We need the game manager to “load next level.” This makes a server call to find out what the next level is. Back in the game manager, it asks the level manager to load the level. And then the game manager asks the game piece manager to load the player’s pieces based on the server response. Only then it ‘switches’ the game mode to the new level. Here’s what that would look like with callbacks.

public class GameManager
{
	 public Promise LoadNextLevel()
	 {
	 	var result = new Promise();

	 	var getNextLevelToken = _networkManager.GetNextLevelFor(_player);
	 	getNextLevelToken.OnSuccess += (getNextLevelResult) =>
	 	{
	 		// load the level
	 		var loadLevelToken = _levelManager.LoadLevel(getNextLevelResult.levelName);
	 		loadLevelToken.OnSuccess += (loadLevelResult) =>
	 		{
		 		// and start loading the pieces
		 		int numWaiting = 0;
		 		Exception lastError = null;

		 		// call when all pieces are done
		 		var loadPieceDone = ()=>
		 		{
		 			if (lastError == null)
	 				{
	 					// All finished without error.
	 					SetGameMode(getNextLevelResult.levelName);
	 					result.Succeed();
	 				} else {
	 					// error.
				 		// Note how we add information to the exception, creating nested Exceptions as a kind of higher-level stack trace
				 		result.Fail(new Exception("Trying to load next level "+getNextLevelResult.levelName,innerException:lastError));
	 				}
		 		};
		 		foreach (var name in getNextLevelResult.levelName)
		 		{
		 			var loadPieceToken = _pieceManager.LoadPiece(name, _player);
		 			numWaiting ++;
		 			loadPieceToken.OnSuccess += (loadPieceResult) =>
		 			{
		 				numWaiting --;
		 				if (numWaiting == 0)
			 				loadPieceDone();
		 			};
		 			loadPieceToken.OnFail += (error3) =>
		 			{
		 				numWaiting --;
		 				if (numWaiting == 0)
			 				loadPieceDone();		 				
		 			};
		 		}
	 		};

		 	// deal with errors appropriately
		 	loadLevelToken.OnFail += (error2) =>
		 	{
		 		result.Fail(new Exception("Trying to load next level "+getNextLevelResult.levelName,innerException:error2));
		 	};
	 	};

	 	// deal with errors appropriately
	 	getNextLevelToken.OnFail += (error1) =>
	 	{
	 		result.Fail(new Exception("Trying to load next level.",innerException:error1));
	 	};
	 	return result;
	 }
}

Notice how it is almost easy to read, until you add error handling. Then it gets long and difficult to read the flow of the method. And you will need error handling! You can’t just leave your players out in the cold with no indication that something happened.

A common response to this situation is “then make your methods do less.” At that point you’re making it even harder to work out the flow of what should happen.

Coroutines?

Let’s take another look at what that task would look like with coroutines. This time, we are going to use some custom wrappers that get around some of Unity’s limitations.


public class GameManager
{
	// caller will do self.Run(_gameManager.LoadNextLevel());
	public IEnumerator LoadNextLevel()
	{
		// make server call
		// This 'Run()' is a wrapper on StartCoroutine().
	 	var getNextLevelToken = Run(_networkManager.GetNextLevelFor(_player));
	 	yield return getNextLevelToken;
	 	if (getNextLevelToken.Failed)
	 	{
	 		// returning an exception automatically stops this coroutine.
	 		// Note how we add information to the exception, creating nested Exceptions as a kind of higher-level stack trace
	 		yield return new Exception("Trying to load next level.",innerException:getNextLevelToken.Error);	
	 	}

 		// load the level
 		var loadLevelToken = Run(_levelManager.LoadLevel(getNextLevelToken.levelName));
 		yield return loadLevelToken;
	 	if (getNextLevelToken.Failed)
	 		yield return new Exception("Trying to load next level "+getNextLevelToken.levelName,innerException:getNextLevelToken.Error);

	 	// start loading all game pieces.
	 	var pieceTokens = new List<Promise<GamePiece>>();
 		foreach (var name in getNextLevelToken.levelName)
 			pieceTokens.Add(Run(_pieceManager.LoadPiece(name, _player)));

 		// wait for all pieces to finish loading.
 		foreach (var token in pieceTokens)
 			yield return token;

 		// Any of them fail?
 		foreach (var token in pieceTokens)
 		{
 			if (token.Failed)
		 		yield return new Exception("Trying to load next level "+getNextLevelToken.levelName,innerException:token.Error);
 		}

		// All finished without error.
		SetGameMode(getNextLevelToken.levelName);
	 }
}

Isn’t that so much easier to read? You always know what steps are going to happen in what order. And as an added bonus, if something were to interrupt the flow like the game being force closed, or the caller being deleted for some other reason, this will automatically stop and clean up after itself.

What madness is this? How do we get such magic for ourselves? You can get the latest at GitHub.

In another post, we will dive in deep on the design and use of this magical wrapper.

Efficiency in Complex Mobile User Interfaces

In freemium games, presentation is everything. The game itself can almost seem secondary to all the screens these games need. Your typical game has a couple menu screens for adjusting options, a main Heads Up Display that stays (fairly) consistent during your game, and that’s about it.

But this article is for the games that have an inventory to manage, a store to buy stuff from at any time, ‘on sale’ offers, custom game modes. And this isn’t limited to freemium either! We will be discussing in terms of the Unity3d game engine, most of the underlying concepts apply to any game engine, including home grown.

I’ve had some experience with this, where I learned someone else’s system. And this, where I built my own. Two, in fact, since there was a major redesign just before release. Oh, and this too. I’ve been designing and implementing user interfaces for games on desktop for twenty years, and since the original iPhone.

Users are fickle. This is not a bad thing – it is purely a thought to keep in mind. You might have a great game with thousands of players but scratch your head why nobody is willing to pay money in it. What is often overlooked is how easy or hard it is for the player to pay you! The player does not have a mental tally in their head about whether they want to pay you. It is emotional, not logical. They don’t keep a list of bullet points in their head: “Is this game snappy? Does it act the way I expect it to? Is it leaving a small memory footprint so I’m not looking at a loading screen after I get off this phonecall?”

Players just have a feeling that this is a “good game”. Your job as a developer is to lower the “barriers” between players and the game. But I didn’t come here to tell you how to design a freemium UI. I came here to keep you from tearing your hair out after realizing what not to do.

In games, user interfaces are typically an afterthought. “Check out this great game!” Press “New Game”, and you’re off. The typical coding style behind most game interfaces reflect this. You start with a single “New Game” button. Then you add an options menu. Now add the player’s bought gold on every screen. Oh, and we want to be able to buy stuff with that gold. Wait, how do we get the gold? Another store area! Pretty soon, you’ve got this gigantic, complicated code that reflects a gigantic, complicated user interface. But there is a better way!

We can instead borrow from the kind of thought that has gone into decades of desktop application user interface design. And the neat thing is, it boils down pretty simply, you need only to keep a few rules in mind while writing your code.

Just the basics, Ma’am

The first thing to remember is to make separate chunks out of what you present to the user. If you’re showing a counter of how much gold the player has, you should be using the same visual language onscreen in every location. Otherwise the player will get lost, looking for it! Likewise, you should use the same code to display it in every location. The shift in code organization is subtle but important. Instead of a screen manager class saying “set this text to the player’s gold count”, you should instead have a consistent block of UI that knows how to set its own gold count when it is given a player record. Why does this matter? Because two weeks later when you decide you want to show the count in red, but only when the player is out of money, you need to change it only in one place – the code for that block of UI. Not all over your codebase.

Inconsistent Visual Language

Simple and Easy

Another often made mistake is to make each block of UI know how to find the ‘source of truth’, typically requesting values from a singleton. Because you’re always going to have just one player, right? Then the boss says “Social gaming is the big thing now. We want to see them on a leaderboard with their details.” Well guess what – you have to rework all the places that use an internal ‘source of truth’ to be able to use an external one. More time wasted and complications created.

Now to consider the next level up. We have these chunks of interface. In the desktop world, you can fairly easily classify each application’s display based on what will be shown in it. A typical application has only one “window” for each document, and then “dialog windows” for asking more detailed questions, when needed. If you consider your game as a “document,” then you will only have one “window.” This means that your settings screen is actually a “dialog window”. Like a desktop dialog, the settings ask more detailed questions, eventually to close and put you back on your main “document,” the game.

This seems simple enough on the surface. Until it’s not. What do you do when the boss says “we want to offer a sale after playing for five minutes.” Who knows where the player might be at that point. They could be in the main game. Or the leaderboard. Or the store. Then I have to save what screen I was on before I showed the sale, and return to it when I’m done. Oh, and then I have to tell that to the store screen where to go back to after it finishes the sale. Sure hope I made these all some consistent superclass so I don’t have to know how to reopen each and every screen. How do we make the boss happy, and not go crazy working out where to put the player after they “close” the sale display?

The Navigation Stack

It sounds complicated, but this is where desktop UIs have solved it for us. It’s called a navigation stack. Every new screen is a new entry on top of the stack. In the desktop world, when you close one dialog, it re-shows the last item that was hidden when you opened the new one. You don’t even have to think about what it was you hid, you just “go back.” A simple concept, easy to implement, and saves you loads of trouble. In fact, on Android devices there is a requirement that you must always support the hardware “back” button. If you have a navigation stack, you always know exactly what you’re going “back” to.

In this day and age, everyone wants everything “now.” It amuses me to remember back when I started that computers were advertising “no wait states”. That is, they will go so fast that you will never have to wait for your computer ever again. So many years later and we’re still waiting.

Unfortunately, most user interfaces still presume everything happens instantly. In reality, loading takes time. And even if it is truly instant, animations by definition take time. Those little flourishes which bring life to your game have a habit of moving pieces of your interface to unexpected places when you presume things are ready instantly. Take that navigation stack for example. You start loading your settings screen, and at that moment it’s time to display an offer for the “Newest stupendest game piece that gives you 50% extra damage!” But you don’t want to show players offers while on the settings screen, you don’t want to be that intrusive. So what actually happens is both screens load and try to show themselves at the same time! Worse, who do we go “back” to?

There is another easy solution to this. When making changes, like showing a new screen or closing the one you’re on – make those changes instant. But wait, I just said nothing is instant?! Make your navigation stack instant by having the stack entries be completely separate from the actual onscreen displays. This way you can ‘instantly’ show a new screen and make reasonable logic decisions, no matter how long it takes to actually load or animate that new screen for the player. (For those of you playing at home, this makes the stack entries behave like “models” of the MVC pattern.)

Here we are going to put some pieces together and get an unexpected but highly useful result. You wanted that consistent “player gold” display on most of the screens of your game, right? Well, we don’t need every screen to have a copy of that, do we? Just leave it up for the next screen. But then how do we know the next screen wants it or not? Guess what – the navigation stack entry for the new screen knows. The new navigation stack entry has already been added because it is instant; it knows that it has to start loading and animating a new screen, and it knows if it needs the player gold displayed. So while one is animating in, it can animate away the piece it doesn’t need. The whole thing looks slick and smooth to the player, like you meant it that way.

Same Dog, New Tricks

You’ve made it this far. How about one more trick up the sleeve? There are always special cases in user interfaces. An example: a dialog offers a reward for connecting the game to a social media provider. If the player accepts, you want to take them to your normal screen that actually does the connecting. Okay, fairly obvious. But we really want to drive the idea home by flashing the buttons that actually do the connection, and only when it comes from this reward dialog.

One implementation would have the buttons look on the navigation stack to see if that reward offer was on the stack, and flash the buttons when it sees that. Fair – it would work. But then the connect dialog would need changes to know the offer dialog exists, and then embed custom animations that have nothing to do with the core purpose of the dialog, complicating later improvements.

But what if… each navigation entry could accept a list of commands to run after the dialog loads? Now the offer dialog pushes the connect dialog on the stack, and the registers a command for when the dialog is done loading and animating into view. Once hit, the offer dialog finds the buttons in the UI by searching for a special tagged object, and adds the flashing animation to those tags. With that, the connect dialog doesn’t need to know anything more about the offer dialog other than a simple string to let designers know that the offer dialog does some custom stuff, and needs to be considered when making changes.

There is so much that can be done to make your life easier when building your games. Even in the simplest ones, adhering to these guidelines will save you untold headaches in the future.

Unity3d Asset Bundles – Unloading

This article is part of a series. If you’d like to skip ahead, you can go to the master post that links to them all.

This room is a mess. I’m constantly getting after my kids to clean their rooms. Why? Because two days later they complain they can’t find one of their favorite toys! It’s not my job to track their toys. But it is my job to track my game’s memory. We can’t leave stuff lying around in memory, or it’s just going to fill up and crash.

Memory management is of supreme importance in mobile game development. And of high importance in any other platform as well. Your game must run forever! You can’t just leave the textures from three levels ago in memory. You have to make sure things get unloaded from memory. But at the same time you don’t want to accidentally unload something you’re still using.

Obviously you will be tracking and deleting your loaded levels and GameObjects. But also remember that the asset bundles themselves have some memory overhead. In fact, on some platforms, you can only have a maximum number of bundles open at a time. You have to clear them out or the app simply crashes!

 

It’s been a year in the making of this single blog entry. Originally I wanted to discuss the unloading of asset bundles, and I will get to that. But in the course of working on my latest project I realized just how deep the rabbit hole goes.

It turns out that knowing when and how to unload a bundle is more art than science. It’s very specific to how you plan on using your assets. Even different assets within the same game have different needs. Here I’m going to describe what Unity allows, but there will need to be a bonus post on how to recognize what assets belong in what category.

The first rule to remember is that a bundle won’t unload anything that’s been loaded until the bundle is closed. I can’t stress this enough! It is not handled the same way the assets built into the player are. Assets loaded via Resources.Load() will go away when they aren’t referenced anymore and you do a Resources.UnloadUnusedAssets(). Bundles don’t do this! They will not unload until the bundle is closed, even if the assets aren’t used anymore. This makes closing bundles absolutely critical to managing memory effectively.

 

Toward this end, Unity offers two modes of closing a bundle: a) where the contained assets are left in memory and b) where the contained assets are removed from memory. This sounds so simple, but it actually creates significant potential complication.

A naive approach would always use the unloading version, so that you keep open a bundle as long as you’re using any of the assets inside it. However, there are limits to the number of bundles you can have open on some platforms. So what do we do? We unload as early as we can. But even that has a catch – an asset that was left in memory won’t be re-used if the same bundle is reopened later. It will instead make a duplicate copy. So be sure you’re really done with it before closing it!

So where do we go from here? For now, I leave it as an exercise to the reader to work out what bundles you need to keep around and which ones are safe to unload early. Remember to consider how assets shared between two bundles are also distinctly affected by whether the requesting bundles can be unloaded or not.

Automatic Texture Scaling Through Asset Bundles

This article is part of a series. If you’d like to skip ahead, you can go to the master post that links to them all.
Last time we upgraded a game to the Unity3d 5 asset bundle system, and dealt with handling shared data files. This time we will explore an unexpected use for asset bundles – automatic texture scaling for different classes of device.

On this project, the artists really outdid themselves – very high quality, high resolution artwork. Unfortunately, rather too high for most of the devices we wanted to play on! They had done a lot of work to balance each image size against its detail and how large it would be used in the game. Notice how we had to make the high detailed icon relatively large compared to the core game sprite, all because the icon had more detail.

Forced Balance

But there is a completely different way to approach the problem. A much simpler one, in fact. First you decide what device you consider the largest pixel size you want to support. In today’s market, this is typically a tablet device of some flavor. Art then designs everything at that consistent scale that looks good on that device. Now the art looks like the following image. Notice how they are all scaled based only on their visual importance in the game, with no wasted pixels. The car, a main focus of the game, is now large compared to the button icon that is only occasionally used.

Okay, but how do we support the rest of the phones and other devices that are smaller or less powerful? Trying to run those full size textures on just a different tablet is enough to drag the framerate down to unplayable. Why is that? It comes back to how mobile devices work. On a desktop machine, the video card typically has a dedicated memory space for textures. They are uploaded once (when things go right) and they never touch the main computer memory again. Every frame that is drawn uses that custom video memory. In mobile devices, there is no such dedicated memory. Every frame that is drawn pulls every texture through the same memory system the main processor is using. Only so much data can be pushed to the renderer in a given amount of time, and bigger textures means more data needed. Even if the framerate can be maintained, it takes precious battery power to transfer the data. Mipmaps can help reduce how much of the texture is used at a time, but you still need the entire texture somewhere in memory, taking up room that other applications use and need.

So back to the texture sizing. How do we handle those devices that don’t need the extra pixels? Simple – scale the textures. It’s a fairly simple thing to ask Unity to scale the textures for you, using the texture settings of the “maximum texture size”. And you can save those changes into asset bundles! Then the client once again can get the correctly sized textures by simply deciding which bundle directory to load. Easy peasy.

Things we learned while creating an automated system to do those bundle builds for us:

  • This kind of texture scaling and processing works just fine with both the pre-Unity 5 and Unity 5 asset bundle system.
  • Make sure you edit texture settings before any BuildPipeline.PushAssetDependencies. You can safely do so once you’ve balanced all the Push with Pops, however. Like if you are doing one build to generate multiple device asset sets.
  • Be sure to call AssetDatabase.StartAssetEditing/Stop only once before starting each build process. Unity does a bunch of extra processing even if you nest them. Wrap it in your own reference counting if you must.
  • It works pretty good to use the Default texture settings as the “master” value, setting the current platform’s custom settings to whatever you calculate for that bundle.
  • Remember that the default ‘max size’ of a texture is nothing like the actual size of the texture. There are ways of asking Unity for the true size of a texture to get the accurate default size.
  • Expect to scale the textures in powers of two, simplifies a lot of the task.

Next week we will wrap up our series by demystifying the unload process!

Unity3d Asset Bundle Data Sharing

This article is part of a series. If you’d like to skip ahead, you can go to the master post that links to them all.
Last week on our asset bundle journey we got started converting our old asset bundle system to the new Unity 5 one. It was actually really simple. But there is still a big problem that needs to be solved.

When you build bundles, you mark each asset that the game is going to ask for when you run the game. When you flag a single texture, it works just fine. But what about shared assets? For example a prefab holding a sprite that uses a texture atlas. Unity will gladly pick up the texture for the atlas and add it to the bundle for you. No problem there. But what happens when you add a different prefab in another bundle that wants the same atlas?

That’s when things get confusing. Will unity add the texture to both bundles? Yes. Uh-oh. Then you have wasted some memory on a duplicate texture. Worse, Unity seems to get confused by loading the same texture twice! No, Unity does not figure out that it is the same texture for you, loading it only once. Bad times. But I thought Unity handles dependencies for you in the new system? It does, but only considers each bundle separately. It does not figure out sharing data to other bundles.

But you can do something about it. All you have to do is mark the “shared” atlas as another asset bundle. In our example, the first prefab goes in bundle A, the second prefab in B. Even though the game does not ask directly for it, we set the atlas to be in new bundle C. Unity will then properly take over, making sure to load only one copy of the atlas, and only unload it after both prefab bundles are unloaded.

The more I think about it, the more it feels like there should be some automatic way of dealing with this resource sharing. Both in new projects and legacy ones that we are upgrading. However, this is something Unity does not provide. Why not? It’s easy enough to determine what assets are shared and used by what other assets. The catch is, what’s the most efficient way to assign which asset to which bundle? That will be different for every project.

That said, you can still write some tools that help in the process. For example, something that examines your bundle assignments and works out what should be shared. It could even go so far as to suggest what should share to what.

So there you go! That is how to deal with shared asset bundle data. Join us next week as we explore a useful ability that asset bundles make super simple: automatic texture scaling.