iRis AG Post Mortem: What went wrong

[Johannes]

This is a collection of after-action reports of what went wrong on the project.

[Johannes]

Disaster strikes, but Python and Blender saved the day!

Part of the asset pipeline centers around how to take models that an artist would build and import them into the working program. Early on in pre-production a decision was made to try and use the .dae Data Asset Exchange format due to its backing by Kronos Group, the ones that actively manage OpenGL, OpenSL, etc.

As the import code was being developed, it was noticed that the artwork being produced, having been exported into DAE format, was loosing certain attributes through the conversion, as well as over complicating other attributes. The DAE format was an XML based format that seemed, at first glance, fairly modular and capable of being the middle ground between the artist and the programmer, but the lack of an appropriate C-based API (other than going through a straight XML library, such as libxml) made writing the importer much more a chore than it should have been. This, combined with the lack of support from the DAE exporter code in Blender, lead to a lot of head scratching.

Disaster started to strike as the development issues continued, and ultimately DAE proved to be a poor format for storing assets in the final program (as the name would imply, DAE is an exchange format, not a storage format, and this is even touched on in the technical documentation for DAE). In particular, one development group wrote about their adventures in Game Developer Magazine about how they used an XML format during production and had an automated XML->binary format conversion. When a newer XML file was detected than the binary, a conversion took place updating the binary. During release only the binary version would thus ship. They made mention that the difference between loading the XML and binary was a factor on the order of about 15:1. This sort of thing made an amazing amount of sense to us, and since DAE did not have a binary equivalent, the reasons were stacked and the decision was made to simply abandon DAE and go towards a more game specific format.

While looking at a variety of other formats, it was decided that there was too much emphasis in a particular area for a large number of formats, none of them tailored to what we needed. As such, we decided to go with our own special format, adapting the broken DAE reader code. While still risky and bumpy, it made sense then, and still does now.

A part of this custom format entailed writing a new exporter for Blender, and this was where Python served to save the day. Having had no major experience with such language, it was quickly found that Python is, in all honesty, an incredibly useful scripting language with a large following. The inane simplicity of using the language to accomplish complicated tasks allowed for a very short ordered export script to be manufactured that did exactly what we needed of it. Afterwords, we were simply in the state of mind "why didn't we just do this to begin with".

The importer code was still harsh and unforgiving (is about 5,000 lines long), but within a week's time art assets were being successfully imported into the program. As time went on, a few kinks in the Blender system were uncovered and fixed, and there still is a few floating issues, but the native ability to define assets, properties, etc. in these external XML scripts (which could be edited by hand very quickly and painlessly) allowed for a very important base to build a game off of.

It's interesting to note just how close the relationship between object definition and the XML storage of said object relate. It is really apparent, at least to us, that if one solves how to store the data one can solve how to define the data in the program. Really interesting tie together.

Much like Xcode's Interface Builder and Qt's GUI builder, the custom format that is being used can be made to benefit from the same basic premise, essentially (if given enough time and resources) giving us an ability to design a large amount of asset work in an external tool, similar to what Irrlicht's designers did with their viewer/editor tool. To be able to visualize and design the game systems in such a fashion simply leads to rapid prototyping.

In all, thanks to the power behind Python, the open initiatives in Blender, and just good 'ol hard work and not half-assing software, the asset pipeline was saved in the production of iRis AG. RPNi is now fully on board the Python language, and we wound up writting a variety of different tools in Python that aided immensely during production (for example, a PNG->PVRTC conversion script). High five to the developers working on Python.

[Johannes]

Export Compliance: > 64-bit symmetrical key encryption requires clearance from the Feds

When working with software that is destined to be distributed outside of the U.S., RPNi found out that we couldn't actually use the 128-bit encryption that we had set up to use. Thankfully, the algorithm we were using (a chaos theory based random number generator creating a one-time-pad, for those who know what that means) was trivial to rework as a 64-bit algorithm. Interesting piece of information for those working in such things. They don't teach these things at university.

The interesting part was more about how we managed to make this algorithm work just a tad better than what most programmers would expect from a one-time-pad xor encryption. The problem with the encryption was that using the same symmetrical key meant simply that the same one-time-pad was generated, which in effect made for a very easy to break encryption since a one-time-pad was being used more than once. The solution to this simply involved making a random public salt key that salted the private key and thus created a different symmetrical key to encrypt and decrypt with.

Thanks to these efforts, we have a fairly useful 64-bit symmetrical key encryption algorithm to help protect consumer data (e.g. unique device identifier) while staying out of way of federal mandated clearance orders.

While on this subject, I must repeat that, legally speaking, RPNi can not and will not guarantee the absolute privacy of data using this encryption mechanism. In particular, 64-bit keys can be cracked on modern super computers in a reasonable amount of time, not to mention to truly make this particular system work one would need to use RSA asymmetrical encryption for the establishment of a different private key on each communication, and let's not start to mention man-in-the-middle attacks and other related issues. SSL communications is really needed for the host of issues presented in truly securing online communications, but for a game we felt that there really isn't much point in going overboard on such things especially since some pieces of data get distributed back into the pulic in unencrypted form anyways (e.g. player initials, score, etc.). The security of the system isn't based on the key so much as it is based on verification of submitted data, not to mention a lot of the encryption was done in an effort to limit cheating more than anything else. It's just decent of us (and trivial for us) to extend it to help protect some kinds of data (e.g. unique device identifier), even though there are applications out there doing similar work that send this data in plain text most likely because it's easier and normally isn't that big a deal anyways (not like you can do much with a person's unique device identifier). If it were credit card numbers that were being transmitted (which isn't the case), naturally that would be a completely different story entirely. So with all things considered, it's still better than nothing, and pretty good for a small Indie company to stand by and say we can do.