Text Encodings

Evennia is a text-based game server. This makes it important to understand how
it actually deals with data in the form of text.
Text byte encodings describe how a string of text is actually stored in the
computer - that is, the particular sequence of bytes used to represent the
letters of your particular alphabet. A common encoding used in English-speaking
languages is the ASCII encoding. This describes the letters in the English
alphabet (Aa-Zz) as well as a bunch of special characters. For describing other
character sets (such as that of other languages with other letters than
English), sets with names such as Latin-1, ISO-8859-3 and ARMSCII-8
are used. There are hundreds of different byte encodings in use around the
world.
A string of letters in a byte encoding is represented with the bytes type.
In contrast to the byte encoding is the unicode representation. In Python
this is the str type. The unicode is an internationally agreed-upon table
describing essentially all available letters you could ever want to print.
Everything from English to Chinese alphabets and all in between. So what
Evennia (as well as Python and Django) does is to store everything in Unicode
internally, but then converts the data to one of the encodings whenever
outputting data to the user.
An easy memory aid is that bytes are what are sent over the network wire. At
all other times, str (unicode) is used. This means that we must convert
between the two at the points where we send/receive network data.
The problem is that when receiving a string of bytes over the network it’s
impossible for Evennia to guess which encoding was used - it’s just a bunch of
bytes! Evennia must know the encoding in order to convert back and from the
correct unicode representation.

How to customize encodings

As long as you stick to the standard ASCII character set (which means the
normal English characters, basically) you should not have to worry much
about this section.
If you want to build your game in another language however, or expect your
users to want to use special characters not in ASCII, you need to consider
which encodings you want to support.
As mentioned, there are many, many byte-encodings used around the world. It
should be clear at this point that Evennia can’t guess but has to assume or
somehow be told which encoding you want to use to communicate with the server.
Basically the encoding used by your client must be the same encoding used by
the server. This can be customized in two complementary ways.
  1. Point users to the default @encoding command or the @options command. This allows them to themselves set which encoding they (and their client of choice) uses. Whereas data will remain stored as unicode strings internally in Evennia, all data received from and sent to this particular player will be converted to the given format before transmitting.
  2. As a back-up, in case the user-set encoding translation is erroneous or fails in some other way, Evennia will fall back to trying with the names defined in the settings variable ENCODINGS. This is a list of encoding names Evennia will try, in order, before giving up and giving an encoding error message.
Note that having to try several different encodings every input/output adds
unneccesary overhead. Try to guess the most common encodings you players will
use and make sure these are tried first. The International UTF-8 encoding is
what Evennia assumes by default (and also what Python/Django use normally). See
the Wikipedia article here for more help.