Difference between revisions of "XML Network Message"

From Armagetron
m (typo)
(Non-standard Stuff written)
Line 61: Line 61:
  
 
== Non-standard Stuff ==
 
== Non-standard Stuff ==
 +
 +
There are some things about our protobuf message definitions that extend the standard.
 +
 +
=== Strings ===
 +
 +
Usually, when you define a string element of a message, it will get filtered by the receiver. Things filtered include
 +
* color codes
 +
* incomplete color codes
 +
* illegal whitespace
 +
* illegal characters
 +
depending on the configuration and current state. Sometimes, that is not desired. To turn off filtering, give your string
 +
element a name ending in _raw:
 +
 +
Message StringMessage
 +
{
 +
  optional string this_gets_filtered = 1;
 +
  optional string does_not_get_filtered_raw = 2;
 +
}
 +
 +
This is required for strings that don't represent strings in the game to be eventually rendered by clients, but rather internal strings like protocol IDs.
 +
 +
=== Legacy Stream Messages ===
 +
 +
Our old network system used simple unstructured stream messages. Compatibility with that is automatically ensured; protobuf messages are converted to stream messages by simply taking all elements appearing in the message definition before certain marker elements. Marker elements are those with an ID on the far side of the range reserved for internal protobuf use (>=20000).
 +
 +
In this message:
 +
message PlayerRemoved
 +
{
 +
  // the object ID of the player
 +
  optional uint32 player_id = 1;
 +
 +
  // legacy message end marker, extensions go after it
 +
  optional bool legacy_message_end_marker = 20000;
 +
 +
  optional string reason = 2;
 +
}
 +
only player_id will get written to and read from old stream messages, the reason will be omitted.
 +
 +
Some messages even have two markers; those are messages for object syncs, everything before the first marker is written in the creation section of the stream message, everything between the markers is written in the sync section.
 +
 +
The short version? If you see messages with legacy_end_markers, don't change the part before the last end marker, neither remove nor add elements. Only add new elements after the last end marker.
  
 
== Network Objects ==
 
== Network Objects ==

Revision as of 13:23, 13 February 2009


Sections: Installing the Game | Playing the Game | Competition Hub | Server Administration | Extending Armagetron Advanced | Development Docs


This wiki page has an associated Blueprint on launchpad.

XML-Like Network Messages using google::protobuf

Code and documentation of google protocol buffers can be found here. The basic workings are thus: you write a protocol specification .proto file where message types are defined. Each message consists of data elements that have a type, a name and a numeric ID. In your program, you access the elements of the message via regular functions named after the element name. Over the network, the numeric ID is used to keep elements apart. It is easily possible to add new elements to an existing message without breaking compatibility with older server/clients, they'll just ignore the unknown data.

This document focusses on the use of protobuf in arma. It is of interest to coders only.

Code Structure

Our .proto specification files reside in src/protobuf. There is a prototype file prototype.proto; you can use it as a template for new files. The .proto files are compiled to .pb.h and .pb.cc files by a batch file/Makefile. Typically, no header file should ever include a .pb.h file (exceptions would be templates that absolutely need to); instead, forward declare the used protobuf classes and include the .pb.h file from your .cpp file.

Minimal Example

The basic feature of protobuf messages is transmitting protobuf structures from client to server or vice versa. To use that, you need four things:

  • a message definition in a .proto file
  • a message handler responsible for receiving the message
  • a message descriptor gluing stuff together
  • code sending the message over the descriptor

First, a message definition in the file TestMessage.proto (see the google docs for details on what is allowed):

message TestMessage
{
   optional int32 test_data = 1;
}

Note that the message name is CamelCase and the_element_name_has_underscores. That's google's convention; it makes sure the generated code uses sensible conventions itself. Then, you need the handler function in your .cpp file:

#include "nProtoBuf.h"
#include "TestMessage.pb.h"
static void sx_TestMessageHandler( TestMessage const & message, nSenderInfo const & sender )
{
   // print data
   std::cout << mesage.test_data() << std::endl;
}

The first parameter is a const reference to the generated protobuf data type. The second parameter contains information about where the message came from; it wraps the message it came in and has member functions such as SenderID() giving the network ID of the sender.

The descriptor is just a static templated object:

static nProtoBufDescriptor< TestMessage > sx_testMessageDescriptor( 999, sx_TestMessageHandler );

The first parameter of the descriptor constructor is the message ID. No two IDs must ever be the same; if you pick a used one by accident, don't worry, the constructor will tell you so with a nice fatal error. Note the naming of handler and descriptor: s<prefix char of module>_<name of message>Handler/Descriptor, with the first character of the descriptor lowercased to indicate it's no function. Let's try to stick to that :)

For the sending code, there are several possibilities. The easiest is to use those methods of the descriptor that prepare a protobuf message for sending and return the protobuf to you to work with directly to fill. The message itself is not sent until the next network flush, which usually happens once per frame or if you trigger it manually. To just send a message to the server, use

TestMessage & content = sx_testMessageDescriptor.Broadcast();
content.set_test_data( 42 );

In client mode, Broadcast() sends the message to the server; in server mode, all connected clients receive it. There's two optional parameters: a boolean, telling whether the message should be protected against loss (true by default), and a nVersionFeature const & (must come first), restricting the message receivers to those supporting the given feature. To send a message to a specific peer, use the Send( int receiver ) function.

Sometimes, that direct sending is not quite what you want; sometimes you want to manipulate the message itself before it is sent. In those cases, use the more complicated procedure

nProtoBufMessage< TestMessage > * message = sx_testMessageDescriptor.CreateMessage();
TestMessage & content = message.AccessProtoBuf();
message.set_test_data( 42 );
// do whatever you need to do with *message.

Non-standard Stuff

There are some things about our protobuf message definitions that extend the standard.

Strings

Usually, when you define a string element of a message, it will get filtered by the receiver. Things filtered include

  • color codes
  • incomplete color codes
  • illegal whitespace
  • illegal characters

depending on the configuration and current state. Sometimes, that is not desired. To turn off filtering, give your string element a name ending in _raw:

Message StringMessage
{
  optional string this_gets_filtered = 1;
  optional string does_not_get_filtered_raw = 2;
}

This is required for strings that don't represent strings in the game to be eventually rendered by clients, but rather internal strings like protocol IDs.

Legacy Stream Messages

Our old network system used simple unstructured stream messages. Compatibility with that is automatically ensured; protobuf messages are converted to stream messages by simply taking all elements appearing in the message definition before certain marker elements. Marker elements are those with an ID on the far side of the range reserved for internal protobuf use (>=20000).

In this message:

message PlayerRemoved
{
  // the object ID of the player
  optional uint32 player_id = 1;

  // legacy message end marker, extensions go after it
  optional bool legacy_message_end_marker = 20000;

  optional string reason = 2;
}

only player_id will get written to and read from old stream messages, the reason will be omitted.

Some messages even have two markers; those are messages for object syncs, everything before the first marker is written in the creation section of the stream message, everything between the markers is written in the sync section.

The short version? If you see messages with legacy_end_markers, don't change the part before the last end marker, neither remove nor add elements. Only add new elements after the last end marker.

Network Objects

Control Messages

Protocol Specification