I just pushed 1036 commits to the master branch. This switches it over to Doomsday 3.
Doomsday 3 in its current form is still quite unfinished and broken in many places. Especially the new renderer remains disabled by default, and cannot be enabled without recompiling. However, the basic things are running, and they’re running fast!
Even the classic renderer has gotten a boost. Although to be fair, it still has the same old bottlenecks as before. The difference is that the application event loop has been optimized, which raises the FPS ceiling even though the floor remains largely the same. On average, the game should be running more smoothly.
A new foundation
2020 has made me lose track of time a little bit, but I believe I’ve been working on the Doomsday 3 branch for a year and a half now. This was preceded by the rendering exploration in early 2018, on which the new renderer in Doomsday 3 is based. (The early commit history has been rebased a few times, so the commit dates are not accurate.)
The work actually traces back even further to 2017, when I got fed up with C++’s slow build times and started seriously considering whether plain old C could be sufficient instead. This led me to write the_Foundation. It is a C11 library for low-level functionality such as basic data structures, multithreading and stream I/O with API conventions designed for my personal tastes. I’ve grown quite fond of this little library over the past couple of years, and therefore I’m happy that it now takes on the burden of being a partial replacement for Qt in Doomsday 3.
the_Foundation weighs in at about 18 KLOC and compiles in seconds, as opposed to Qt that is about 1 MLOC and takes tens of minutes to compile. Of course, Doomsday doesn’t even need most of what Qt offers. It only ever was using the core functionality: data structures, threads, Unicode text, network sockets, and GUI widgets. Apart from the widgets (that Doomsday nowadays implements itself), the rest come from the_Foundation.
This transition was possible because Qt was never that deeply integrated into Doomsday. The most ingrained feature was Qt’s container classes, but it was relatively straightforward to replace all of them with compatible C++ classes based on C++ STL or the_Foundation.
I’ve been so enjoying the work on the_Foundation that I briefly entertained the notion of rewriting significant portions of Doomsday in C11, but that is simply too much work. What happened instead was that the Doomsday Core library, which was used for encapsulating Qt APIs, now uses the_Foundation instead. As a result, the rest of the codebase remains largely agnostic about what happens on the lower levels.
Slim and trim
One of the nice side effects of the change is that all text strings are now stored as UTF-8 in memory, as opposed to Qt’s choice of UTF-32. Doomsday mostly deals with ASCII/Latin text, so this means text strings take about a quarter of the memory they used to. This could be quite significant for you if you have a large add-on/mod library, because Doomsday keeps the file index in memory and that contains a lot of text.
To further simplify the internals, the dynamic plugin system has been removed. In the beginning (i.e., circa year 2000), there were intentions to allow “easily” creating new plugins, but in practice that never took off. Native code plugins are powerful but actually creating one is very challenging. Scripting is a much better solution to custom functionality and game behavior because it involves no complicated toolchains or learning C++ APIs.
Instead of dynamic plugins, Doomsday 3 has compile-time extensions only. This boiled down to converting the old dynamic plugins to fixed modules that can be included in or excluded from the build, so there is still some configurability and platform-specific variability. The great advantage of not doing dynamic plugins is that there is no runtime loading/unloading of libraries, which may involve complicated state initialization and cleanup.
SDL to the fore
SDL is an old-fashioned C library so it aligns nicely with the_Foundation. Funnily enough, Doomsday has been using SDL for many years but only for game controller support. However, SDL provides a lot more than that and has been particularly targeted for cross-platform games. This makes it a great fit for Doomsday’s video, audio and input functionality. Doomsday 3 now uses it for managing windows, display modes, and keyboard and mouse input.
SDL is good at handling raw input events, so the switch should take care of some Qt-specific keyboard input bugs where certain events would’ve been lost or mishandled. The recently released SDL 2.0.14 has plenty of game controller related improvements, so it’ll be nice to be able to take advantage of those, too.
Full plate for 2021
I intend to stick to the yearly release cycle, so the stable Doomsday 3.0 is targeting December 2021. There is certainly lots to do until then! After the build environments and other such practicalities have been dealt with, the main focus area will be the new renderer:
- Porting the 3D model rendering: keeping the forward rendering or switching to deferred, like the rest of the new renderer?
- All objects will be rendered as 3D objects, but defaulting to generated 2D billboards.
- I’m excited about the new renderer’s implementation as an indepedent library. You give it a
.packpackage containing all the resources, and there’s a runtime API for controlling object placement, plane movements, and other dynamic changes. The renderer could be run independently as a “dumb” viewer app, for example. - Somewhere down the line when the new renderer starts to be more full-featured, I’ll include both in the build and add an option to enable it at launch for testing.
The current challenge is to reconfigure the Autobuilder to support the new build setup. It will be particularly a problem on Windows. The price to pay for getting rid of Qt is that now I have to take on a bunch of low-level functionality. My plan is that initially — and possibly for the foreseeable future — Doomsday will be built with the help of MSYS2 on Windows. What does this mean in practice? If you’re just running Doomsday and playing games on it, this should be a non-issue; everything should work like before. If you look in the installation folder, you’ll see a different set of DLLs than before. However, if you have been brave enough to compile Doomsday on Windows, you’ll need to set up an MSYS2 environment to keep doing that. The greatest advantage of this change is that this allows using the same build procedure on all platforms. In fact, the Linux and Windows builds will be using the same compiler (GCC)! Fewer special cases reduces the maintenance burden.
A thousand commits is a big change to introduce at once and it will undoubtedly cause instability and a few new bugs. So this is a heads up — progress has its price.