Dev Blog

Dev-Blog 223: Networking: Pack those bits!

06/08/2016

devblog_header03

Welcome back, followers of the fearsome!

This week is a follow up to Dev Blog 197, which was the first dev-blog on networking. I’ve been working on getting the networking system up and running for Viking Squad, and there are a ton of little peculiarities I could talk about, but I’ll start with a bit more of the basics, in particular about how to squeeze all the unused bits out of your precious network bandwidth.

After setting up the low level networking system described in Dev Blog 197, we are now able to send and receive messages from remote clients. These messages get bundled up into a network packet and sent, only to be broken into individual messages again when they reach the other side. Now, bandwidth is always a concern, so we want to try and minimize the packet sizes, which means minimizing the size of the message that get sent.

If you’re familiar with C#, you’re probably aware of the BinaryReader and BinaryWriter classes to save and load binary data. These are constructed with a stream, and then you can read and write different data types to these streams.

For example, say we have this hypothetical entity that we need to save to the network. (Note: this isn’t what our internal entity class looks like, this is just an example to illustrate what I am talking about). This is what the code would look like:

class Entity
{
	private UInt16 m_entityID;  // Unique ID to identify the entity by
	private Vector3 m_position; // 3D Position of the entity in the world
	private Vector3 m_velocity; // Velocity of the entity in the world
	private bool m_onGround;    // Boolean saving if this entity is on the ground or not.

	// .. More data here

	public void SaveToNetwork(BinaryWriter writer)
	{
		writer.Write(m_entityID);		// 16

		writer.Write(m_position.X);		// 32
		writer.Write(m_position.Y);		// 32
		writer.Write(m_position.Z);		// 32

		writer.Write(m_velocity.X);		// 32
		writer.Write(m_velocity.Y);		// 32
		writer.Write(m_velocity.Z);		// 32

		writer.Write(m_onGround);		// 8
							// 216 bits total = 27 bytes 
	}

	public void LoadFromNetwork(BinaryReader reader)
	{
		m_entityID = reader.ReadUInt16();
	
		m_position.X = reader.ReadSingle();
		m_position.Y = reader.ReadSingle();
		m_position.Z = reader.ReadSingle();

		m_velocity.X = reader.ReadSingle();
		m_velocity.Y = reader.ReadSingle();
		m_velocity.Z = reader.ReadSingle();

		m_onGround = reader.ReadBoolean();
	}
}

As you can see it needs 27 bytes to save the state to the network. The smallest unit of data that can be written using BinaryReader and BinaryWriter is a byte, which is actually quite large if you think about it. For example, when the SaveToNetwork function writes the m_onGround boolean value, it will write an entire byte (8 bits) to save the state of the boolean. That’s 7 wasted bits to write 1 bit of actual data!

Now, how about we take this one step further? How about saving all values using exactly the amount of bits we think it needs? And in our loading code, we use the same number of bits to read the value and store it in our variable. To make this easy to do, I implemented my own BitStreamReader and BitStreamWriter classes. These have the same functions to read and write data types, except they also require a number of bits, and in some cases a minimum and maximum value. Internally they pack data in bit by bit, making it possible to waste no bits when saving the entity state. If you’re interested in the internals of these streams, just click here to see what the C# code looks like.

To be able to save the maximum amount of space, we need to know what the limits are for each of our variables. The m_entityID is a 16 bit value, but we don’t see it ever going over 1024. In this example, we’ll assume that the X Coordinate is always between -150.0f and 150.0f, and the Y and Z coordinates are always between -6.0f and 6.0f. The velocity is always between -10.0 and 10.0.

Now how do we save this efficiently? Well, lets go through them one by one:

m_entityID: This value will never go over 1024, so we can save this unsigned integer by using only 10 bits.

For m_position, we’ll need to be able to save the floating point values using a specific amount of bits, while retaining a minimum precision. Say we determine we want a precision of about 0.02 units. To be able to save the X coordinate we would need to divide up (150.0 – (-150.0)) = 300.0 into 300.0 / 0.02 = 15000 parts. The nearest larger power of two would be 2^14 = 16384. So we’d need 14 bits to save the float and get a precision of 0.01831. For the Y and Z coordinates we do a similar calculation, and come to the conclusion that we can save using 9 bits, giving us a precision of 0.02343.

For the m_velocity we decide we don’t need as much precision, and we can handle 0.1 unit of precision. This allows us to save the X, Y and Z component using just 8 bits each.

The m_onGround just needs one bit to save, since it’s just a true or false value.

The code would look something like this:

class Entity
{
	private UInt16 m_entityID;  // Unique ID to identify the entity by
	private Vector3 m_position; // 3D Position of the entity in the world
	private Vector3 m_velocity; // Velocity of the entity in the world
	private bool m_onGround;    // Boolean saving if this entity is on the ground or not.

	// .. More data here

	public void SaveToNetwork(BitStreamWriter writer)
	{
		writer.Write(m_entityID, 10);                     // 10

		writer.Write(m_position.X, -150.0f, 150.0f, 14);  // 14
		writer.Write(m_position.Y, -6.0f, 6.0f, 9);       // 9
		writer.Write(m_position.Z, -6.0f, 6.0f, 9);       // 9

		writer.Write(m_velocity.X, -10.0f, 10.0f, 8);     // 8
		writer.Write(m_velocity.Y, -10.0f, 10.0f, 8);     // 8
		writer.Write(m_velocity.Z, -10.0f, 10.0f, 8);     // 8

		writer.Write(m_onGround);                         // 1
		                                                  // 67 bits total = 8.375 = 9 bytes 
	}

	public void LoadFromNetwork(BitStreamReader reader)
	{
		m_entityID = reader.ReadUInt16(10);
	
		m_position.X = reader.ReadSingle(-150.0f, 150.0f, 14);
		m_position.Y = reader.ReadSingle(-6.0f, 6.0f, 9);
		m_position.Z = reader.ReadSingle(-6.0f, 6.0f, 9);

		m_velocity.X = reader.ReadSingle(-10.0f, 10.0f, 8);
		m_velocity.Y = reader.ReadSingle(-10.0f, 10.0f, 8);
		m_velocity.Z = reader.ReadSingle(-10.0f, 10.0f, 8);

		m_onGround = reader.ReadBoolean();
	}
}

Now, what we ended up with is a routine that can save the entity data with the precision we needed, in just 9 bytes. That’s 1/3rd of the original implementation! Of course this is a hypothetical entity, but you can see here that there are huge space savings to be had if we know the constraints of our variables.

Alright, that’s it for this week, I need to get back to more network implementation. Writing this blog post made me realize how many little interesting things I did to make the networked game work better. It’s probably worth another future blog post or two!


As usual, today at 4pm there will be another Dev Stream with Jouste the Drawbarian. Don’t miss it!

VS_DevstreamBanner

 

-Nick

Twitter: Nick: @nickwaanders Jesse: @jouste Caley: @caleycharchuk SlickEntertainment: @SlickEntInc

Facebook: https://www.facebook.com/SlickEntertainmentInc

Twitch: http://www.twitch.tv/slickentertainmentinc

Posted by: Under: Slick Entertainment,Tech

Follow us!

titlebutton_twitter titlebutton_facebook titlebutton_youtube titlebutton_twitch titlebutton_spreadshirt

Join our mailing list!

/ /

Dev Blog

January 20 2017

I almost can’t believe it: Slick Entertainment is a decade old! In the last 10 years we’ve made a bunch of great games, and I am super proud of what we’ve achieved with our small team: 4 fun games, custom C++ engine on 6 different platforms, 3 games feature online multiplayer, all hand-drawn art for […]