Adding documentation related to server project here.

4 files changed, 519 insertions, 0 deletions

diff --git a/docs/architecture.txt b/docs/architecture.txt
new file mode 100644
index 0000000..7cd54d5
--- /dev/null
+++ b/docs/architecture.txt
@@ -0,0 +1,222 @@
+-------------------
+SERVER ARCHITECTURE
+-------------------
+
+1. INTRODUCTION
+2. RO REVIEW
+3. EATHENA MODEL
+4. TMW SERVER
+5. TCP - UDP
+6. SECURITY
+7. DATABASE
+8. REGISTRATION
+9. SERVER CONTROL
+10. OPERATING SYSTEM
+11. ADVANCED DATA TRANSMISSION
+
+
+1. INTRODUCTION
+
+One of the most important thing in a online game is the architecture of the
+server system, which reflects in performances (lag and denial of service),
+scalability, fault tolerance. In this article we will examine the pre-existing
+model and we will evaluate a way to improve it and to add custom features.
+
+
+2. RO REVIEW
+
+Let's start by taking as reference the current server system used to play
+Ragnarok Online on the euRO server. (www.euro-ro.net)
+RO works by using 4 kinds of server:
+
+ - Login Server (L): takes care of verifying accounts with username-password
+   system, allows also encrypted login.
+
+ - Char Server (C): saves every player status (stats, items, equipment, skills
+   and so on.
+
+ - Map Server (M): the real game server, work as interconnection between
+   clients, manage chat, monster AI, damage calculations and everything you can
+   see in game.
+
+ - Inter Server (I): probably manages the messages between the other type of
+   servers.
+
+In euRO there are 1 login server, 1 char server, 1 inter server and 14 map
+servers.
+
+
+3. EATHENA MODEL
+
+The eAthena system mirrors the way used by official RO servers. eAthena
+implements 3 servers: login, char and map. It is allowed to have more than one
+map server at a time. Every server communicates with all the others.
+
+
+4. TMW SERVER
+
+The basic idea of TMW server architecture mainly is the same as the one used by
+eAthena. Since the login and char server don't have heavy traffic they could be
+melt togheter.
+
+        C       M
+          \   /
+        C - L - M
+          /   \
+        C       M
+        
+The login server manages new connections and stores all the informations about
+the player. The map server is pretty the same as the eAthena one. The login
+server manages also connections to a new map server when changing map. Having 
+only one server (L) to manage all the connections between clients and map
+servers is a bad point: if the login server crashes players won't be able to
+play anymore. In fact new connecting players won't be able to connect to login
+server, while map server will disconnect every player, since they can't save
+their infos.
+The solutions are:
+    
+    - implementing a distributed login server which can manage crashes and
+      redirect new connections to another login server. This way means a more
+      complex implementation and probably the need to other computers since we
+      want the login servers to be independent from each other crashes at all.
+      (Remember this is a free project so we should limit the number of
+      computers to act as servers)
+      
+    - RALS (Redundant Array of Login Servers): we can have two login servers,
+      only one of them is active at a time and they share the same database
+      where to store infos about players. If one of the map servers loose the
+      connection with the login server enables the other, while the crashed one
+      restarts. Even if it restarted is it now considered disabled. The new
+      active login server will send messages to every map server to inform them.
+      Every time a client connects check both of the login server and connect to
+      the active one. The bad points of this system are that will imply a lot of
+      data consistency checks: map servers should resend last data since it 
+      was lost when the login server crashed, the new login server should check
+      if some of the data was already stored before the crash, what if both of
+      them crash?
+      
+    - waiting it's the only solution! Let's design the login server as much
+      simple and stable as possible and create a smart restarting system.
+      This way we will have less frequent crashes and a very low restarting
+      time. Obiouvlsy this is the easiest and less expensive solution.    
+
+
+5. Network protocol: TCP
+
+RO is TCP based, mainly because you use the mouse to move the player. When you
+want to reach a point on the map, you click on it and the client send a packet
+to the server with destination coordinates. The server replies with an
+agreement if there's a path to that way. Using this way you don't need high
+speed nor a lot of packets, so TCP it's enough.
+
+With our custom server we want to achieve pixel movements, by that we mean that
+the player is not always positioned in the center of the tile, but will have
+fine coordinates.
+
+Asking the server if every destination coordinates is walkable means a lot of
+traffic from and to the server and probably will result in lag. Using UDP will
+probably help avoiding this problem, but having a reliable protocol such as TCP
+will make things easier and more stable. We just need to design an efficient
+prediction system (probably also a linear one will suffice).
+
+An idea could be using the system I used for a racing game where speed was
+fundamental. When you press a key to move, the client sends a packet with
+starting coordinates, direction and client time. When you release the key (or
+change direction) the client sends another packet with current coordinates and
+client time. According to the player speed and the difference of client times
+the server check if the coordinates sent by the client are right, if not reply
+with a packet with the correct position. The server also check if the time
+interval sent by the client is right: if not it means that the values have been
+hacked or the lag corrupted them. We can set a tolerance value: if the time
+interval exceeds the tolerance then the whole movement is discarded and the
+server send a packet to tell the client to place the player at starting coords.
+
+
+6. SECURITY
+
+To certificate authenticity of the player we can use a system based on digital
+signature and hash (encrypted login).
+
+Better security can be provided by encrypting payload of packets using RSA
+algorytm. When logging in the client generates both its public and private key
+and will send the public one to the server. The server acts the same: when it
+starts it creates both the key and replies to login with its public key.
+Using encryption will reduce client/server performances because this will need
+a lot more calculations.
+Furthermore if using digital signature will introduce a lot of overhead.
+So there's still the need to discuss if we need to use encryption not only in
+the login part.
+
+Solutions to keep the server working and avoid unfair players:
+
+ - DoS attack:
+        * Account activation.
+        * Limit number of accounts to 1 per email address.
+        
+ - Cheating/Botting:
+        * First of all just keep every calculation done by the server.
+        
+Also we need the possibility to warn/kick/ban players/accounts/ip addresses.
+
+
+7. DATABASE
+
+Player data should be stored using databases, probably MySQL. This way player
+infos could be easily accessed trough web and used to show stats or required
+infos on the website/forum.
+
+
+8. REGISTRATION
+
+Still to decide if we want to use a dialog (client registration) or to use a
+web based interface (web registration).
+Registration should ask for less details as possible. This way it will be less
+annoying. Required infos will be:
+    
+    - username
+    - password
+    - email (to limit account number and to be used to activate account)
+    
+More infos could be added later for security problem (such as activation codes
+and so on).
+In RO you also have to choose the sex of your player. It will be better to let
+the user choose the sex when creating the player: this way he can delete is male
+player and create a female one.
+
+
+9. SERVER CONTROL
+
+The server can be controlled in two ways:
+    
+    - In game control: server admins or GMs sending particular commands or a
+      trough a GUI (the way it is used in Ultima Online).
+    
+    - A graphical interface which connects to the server trough an open port.
+    
+The prefferred way is the first one.
+
+ 
+10. OPERATING SYSTEM
+
+We have two choices about this: the former is to follow as for the client the
+cross-compatibility philosophy. This means that the server will compile on every
+windows, unix, macos based system. The latter is to choose the best performance
+system (probably linux) and implement a unique os server.
+Just remember that the current game server run on linux.
+
+
+11. ADVANCED DATA TRANSMISSION
+
+Other ways to reduce bandwidth can be considered such as:
+    
+    - Using bitstreams instead of bytestreams: this way if you need to send a
+      boolean values only 1 bit will be required instead of 1 byte (compression
+      8:1), item types (4 different types) can be represented sending 2 bits
+      instead of 1 byte (compression 8:2), player coordinates can be represented
+      using 20 bits instead of 4 bytes (compression 24:20)
+      
+    - Compressing data following packet id could help reducing bandwidth usage
+      as well. RLE compression or more advanced statistical techniques can be
+      used. Compression can be disabled in very slow systems (If using
+      compression is declared to the server when the client connects to map
+      server.
\ No newline at end of file
diff --git a/docs/messages.txt b/docs/messages.txt
new file mode 100644
index 0000000..dddda23
--- /dev/null
+++ b/docs/messages.txt
@@ -0,0 +1,105 @@
+-------------------------
+MESSAGE PASSING MECHANISM
+-------------------------
+
+1. INTRODUCTION
+2. MESSAGE
+3. ROUTING
+4. SERIALIZING PRIMITIVES
+5. SERIALIZING CUSTOM DATA TYPES
+
+
+1. INTRODUCTION
+
+In server.txt we describe the datatypes involved at both client and server, and
+the related messages that are passed from client to server and back. In this
+file we will describe how the actual messages are composed and how they are
+routed from where they come from to where they are taken care of.
+
+ PLEASE NOTE: Everything in this file is open for discussion. This file is
+ meant to get ideas acros, find flaws in the reasoning and generally get
+ comments on the system before somebody sets out to implement it.
+
+
+2. MESSAGE
+
+The protocol is described in terms of messages. Each message has a certain
+type, a length, and a certain number of values in primitive data types:
+
+   { C msgId, S length, ... }
+
+The length stores the length of the remaining data. We need this so that the
+message can be cut off from the incoming stream of bytes and be encapsulated
+on its own to be further processed.
+
+The primitive data types are the same as those described in server.txt. The
+first thing that is done to the incoming message in to put it in an instance
+of the Message class. Below we outline the Message interface:
+
+  class Message
+  {
+    public:
+      Message(int length, void* data);
+      ~Message();
+
+      int getChar();
+      int getShort();
+      int getLong();
+      std::string getString();
+
+    private:
+      int length, readPointer;
+      void* data;
+  }
+
+
+3. ROUTING
+
+After the Message instance has been constructed, it is routed to the message
+handler that has registered itself to handle this message type. If no handler
+has been registered a warning should probably be issued. A message handler
+implements the following interface:
+
+  class MessageHandler
+  {
+    public:
+      void receiveMessage(int msgId, Message *msg) = 0;
+  }
+
+A mapping is made from message types to message handlers. One handler could
+be registered to handle a group of message types. This mapping can be stored
+in a std::map as follows:
+
+  std::map<int, MessageHandler*> messageHandlers;
+
+This will be a member of the class that processes the incoming byte stream,
+which will implement an interface to allow registration of message handlers,
+for example:
+
+  void registerHandler(int msgId, MessageHandler *handler);
+
+
+4. SERIALIZING PRIMITIVES
+
+Here we will describe for each primitive data type how it is serialized in
+bytes.
+
+  char:    1 byte  (direct copy)
+  short:   2 bytes (now need to keep in mind endian order)
+  long:    4 bytes (see short)
+  string:  2 bytes for length (short) + X bytes (characters)
+
+
+5. SERIALIZING CUSTOM DATA TYPES
+
+Custom data types will be serialized using a composition of the primitive data
+types, or other custom data types. They will need to implement the following
+interface:
+
+  class Serializable
+  {
+    public:
+      void serialize(Message *msg);
+      static Serializable *unserialize(Message *msg);
+  }
+
diff --git a/docs/server.txt b/docs/server.txt
new file mode 100644
index 0000000..e2d506a
--- /dev/null
+++ b/docs/server.txt
@@ -0,0 +1,172 @@
+-----------------------------
+THE MANA WORLD SERVER PROJECT
+-----------------------------
+
+1. INTRODUCTION
+2. MAP
+3. OBJECT
+4. BEING
+5. ITEM
+6. CHAT
+7. AUTHENTICATION
+
+
+1. INTRODUCTION
+
+First let me show you a screen shot of TMW. From left to right it shows a
+player, an enemy, a tree and an apple. In this document the player and enemy
+will go as beings, and the tree and apple will go as objects. Finally, the
+thing they're on is a map.
+
+ o                                        O
+ O : A Player        <> : A monster       | : A tree          o : An Apple.
+
+  -----------------   Fig. 1) screen shot of TMW showing three kind of objects
+  |               |
+  |   o        O  |      MAP
+  |   O <>     |o |      OBJECT
+  |               |      BEING
+  -----------------
+
+Each of these types has its own set of properties and things it can do in the
+game. A number of messages in the protocol can be grouped on these types. We'll
+go through each of them separately. Objects can be picked up and change into
+items, we'll mention those too.
+
+The effects of using objects or items, talking to beings and attacking enemies
+are all calculated server side. It is interesting to think about approaches
+that allow a scripting language to be used in these areas.
+
+In the messages described the following data types are being used:
+
+  A - char array (null terminated)
+  C - char (1 byte)
+  S - short (2 bytes)
+  L - long (4 bytes)
+
+
+2. MAP
+
+- Stored as XML file (.tmx)
+- Refers to tile set images and potentially to music file(s) and objects
+- Beings can change from one map to another (probably using warp and spawn
+  points)
+
+
+3. OBJECT
+
+- Most properties specified in XML file
+- Mostly static (at least doesn't move, but can change)
+- Has collision properties, which can change
+- Can be an item (allowing picking it up)
+- Can be animated and change animation (max 256 animations)
+- Can potentially be activated/used (door, chest, portal)
+
+ Server to client:
+
+  MSG_NEW_OBJECT     { L objectTypeId, L objectId, L x, L y }
+  MSG_REMOVE_OBJECT  { L objectId }
+  MSG_CHANGE_OBJECT  { L objectId, L x, L y, C currAnim, S flags }
+
+ Client to server:
+
+  MSG_PICKUP         { L beingId, L objectId }
+  MSG_USE_OBJECT     { L beingId, L objectId }
+
+
+4. BEING
+
+- Most properties specified in XML file.
+- Dynamic (can walk around)
+- Character animation, but could still show arbitrary animations.
+- Can equip stuff, which could change appearance (max 256 equipment slots)
+- Has inventory
+- Connects to questing system
+- Can fight other beings
+- Dispositions: friendly, neutral, enemy
+- Can be shop
+- Can be talked to, potentially to gain quests
+- Controlled either by player or AI, AI could be either server or client side.
+- Carries money
+- Can be associated with a client
+
+ Server to client:
+
+  MSG_NEW_BEING      { L beingTypeId, L beingId, L clientId, L x, L y }
+  MSG_REMOVE_BEING   { L beingId }
+  MSG_INVENTORY_UPD  { L beingId, L itemTypeId, L amount }
+  MSG_EQUIPMENT_UPD  { L beingId, L itemTypeId, C slot }
+  MSG_ATTACK         { L beingId, L targetId, L damage, C damType }
+  MSG_PATH           { L beingId, A path }
+
+ Client to server:
+
+  MSG_TARGET         { L beingId, L targetId }
+  MSG_WALK           { L beingId, L x, L y }
+  MSG_START_TRADE    { L beingId, L shopBeingId }
+  MSG_START_TALK     { L beingId, L talkBeingId }
+  MSG_REQ_TRADE      { L beingId, L playerBeingId }
+
+More messages are needed for the talking with NPCs and trading with shops and
+other players.
+
+
+5. ITEM
+
+- Properties specified in XML file
+- Beings can carry them around
+- Can be traded between beings
+- Can potentially be equipped (in a certain slot)
+- Can potentially be used
+
+ Client to server:
+
+  MSG_USE_ITEM       { L beingId, L itemTypeId }
+  MSG_EQUIP          { L beingId, L itemTypeId, C slot }
+
+
+6. CHAT
+
+There are several channels in the chat system: 
+
+ Area    -  To players around you             (default)
+ Region  -  To players on the same map        (default)
+ Global  -  To all players in the game        (default)
+ Team    -  To players in the same team       (when in team)
+ Guild   -  To players in the same guild      (when in guild)
+
+In addition to these there are also system messages, and announcements made
+by moderators / administrators.
+
+ Server to client:
+
+  MSG_CHAT           { L beingId, A name, A message, S channel }
+  MSG_SYSTEM         { A message }
+  MSG_ANNOUNCEMENT   { A message }
+
+ Client to server:
+
+  MSG_SAY            { L beingId, A message, S channel }
+  MSG_ANNOUNCE       { A message }
+
+
+7. AUTHENTICATION
+
+Of course before the client can send/receive any of this, he needs to login to
+the server. The idea is that the client will send login info to the server,
+and that the server either denies the request (giving a reason) or sends the
+client a client id. The client later uses the client id to determine which
+being(s) are to be controlled.
+
+ Server to client:
+
+  MSG_ACCEPT_GAME    { L clientId }     // Accepted into the game
+  MSG_ACCEPT_CREATE  { }                // Accepted into character creation
+  MSG_DENY           { A reason }
+
+ Client to server:
+
+  MSG_LOGIN          { A name, A password }
+  MSG_CHAR_CREATE    { ... }
+
+The character creation process will need to be thought out.
diff --git a/docs/serverdata.txt b/docs/serverdata.txt
new file mode 100644
index 0000000..161ef29
--- /dev/null
+++ b/docs/serverdata.txt
@@ -0,0 +1,20 @@
+As per Elven's request, I'm making this document to contain the information
+relevant to server development as far as my understanding goes. This file
+contains the following:
+ - Information that will be stored in Player Account
+
+
+
+1. Information stored in a Player Account
+  - Inventory - an array of 110 item IDs (the last 10 slots or so are current
+                equipment)
+  - Position - the place where the player is in worldspace (3 ints, x, y, map)
+  - Estate - stored as an array of Tiled maps, the same way any map is.
+  - Skills - an array of 100 Skill structures
+  - Quest Info - an array of X short integers, specifying the player's position
+                 in or accomplishment of the game quests. (X is the number of
+                 quests)
+  - Pet Status - pets are stored as an array of skills and some general
+                 information (age, type of monster, loyalty, equipment, etc.)
+                 You can have up to 3 pets at once.
+