Planning ahead for 2.1: multiplayer and OpenGL

The stable version 2.0 has been out for a bit over two weeks now. I hope you’ve been enjoying it! I have been busy working on bug fixes for 2.0.1, which is soon getting into shape for release.

Now that the goals for the 2.0 release have been met, it is time to look ahead to version 2.1. The first point release is planned to primarily feature improvements for multiplayer games. I won’t be exclusively working on that, though, so various minor improvements will also be included. Small things like a persistent console command history and options to start a game profile in a chosen map are already available in the unstable 2.1 builds.

When it comes to the release schedule, it remains futile to speculate about how much time I will have for Doomsday in the future, but my hope is that point releases in the 2.x series will have a roughly six month cadence. While there is no specific release date set for 2.1, I am uncomfortable with the idea of more than 12 months between stable releases. If that much time passes I am willing to postpone the remaining work to a later point release and do a smaller stable release instead. I prefer that there is a steady and predictable flow of stable releases, much like the situation is with the unstable builds.

In any case, if you run unstable builds you will of course get all the latest improvements as soon as they are implemented.

Multiplayer

2.1’s primary objective in terms of user-facing features is multiplayer improvements. This means both bugfixes and new features. I envision the following MP-related improvements in concrete terms (list still subject to change, in no particular order):

  • Bug fixes. Multiplayer gameplay is suffering from a variety of small glitches, particularly in Hexen. The goal here is to get rid of the more significant ones.
  • Movement smoothing for enemies. Single-player games use movement smoothing for everything, so MP games should be no different.
  • New chat UI. Modern games have some established conventions regarding in-game chat, so Doomsday should take inspiration from them. The chat should use Doomsday’s native UI widgets so the font isn’t blurry or too large, text can have colors and rich formatting, there is nice scrolling, text entry is more versatile, and there is autocompletion for usernames and commands. One particular direction this feature could take is integration with the console command prompt, so one can still issue console commands using IRC-like slash prefixes while primarily still chatting.
  • New scoreboard UI. A better scoreboard would be a nice thing to have for competitive game types. There should also be an option for a small, always-on scoreboard HUD. I will likely end up with a toggle between completely hidden, small HUD, and full-size scoreboard modes.
  • More GUI options for configuring game via Shell GUI. Most of the gameplay cvars should be configurable via the GUI. The 2.0 Shell already offers a couple of options (such as game type and current map), but there are more options that would be useful to have conveniently available. My strategy with regard to the Shell remains unchanged, though: for hosting games, the Shell should be the tool you use. Duplicating its features in the client GUI would be more or less redundant and wasted effort. However, I can look into integrating the Shell better with the client GUI, so it is quick and easy to get a local server started, for instance, by launching the Shell via the client.
  • GUI for configuring map cycle. Map cycling is one important MP feature, especially for longer-running servers. By default, a co-op server will not restart the episode once it ends; map cycling should for instance allow configuring entire episodes to be played through and then restarted.
  • Remote packages. The key feature here is loading PWADs from the server.

OpenGL

The internal objective for 2.1 is to complete or at least take a major step toward migrating the rest of the OpenGL code to version 3.3, making it also OpenGL ES 2 compatible. GLES is a requirement for Android and iOS, and also Raspberry Pi (a good low-powered testing/development platform). As a nice side-effect, this transition will also future-proof much of the code for subsequent migrations (particularly thinking about APIs like Vulkan, or some other low-level graphics API of the future). The crucial advantage that this transition will provide is support for GLSL shaders when rendering world surfaces. This truly opens up the potential for renderer improvements.

However, in the scope of 2.1, it must be noted that this will be an internal change, meaning the renderer features available to you remain roughly the same as in 2.0. Some older GPUs may no longer be supported, although it is also possible that some OpenGL drivers work better with OpenGL 3 instead of the old compatibility profile. It may also be necessary to drop some older graphics settings that are no longer relevant, if it isn’t feasible to reimplement them with new GL code.

Finally, please remember that plans may change so none of this is particularly set in stone. Stay tuned for updates. 🙂

Website refresh

In preparation of the 2.0 release of Doomsday I have redesigned the dengine.net website from the ground up. The new design went live today.

While the old design has been perfectly adequate during the past years, I felt that a bit of freshness would do the site good. I also took the opportunity to tie all the parts of the site together and reconsider the overall organization.

There is a new navigation bar visible on all pages, even in the Autobuilder and Bug Tracker that have previously been detached from the rest of the site. This should make it easier to move around and find what you’re looking for. In the bottom of most pages, you can find the latest news and blog posts, recent builds, and currently running multiplayer servers — these were previously only shown on the dengine.net front page.

Following the migration of the forums, the Doomsday Engine Wiki has been replaced with the more focused and structured Manual. It uses DokuWiki, which makes it considerably less complex and lighter to run than MediaWiki. The old wiki will remain accessible at its old address (as read-only).

I quite enjoyed the chance to do some web development. The new implementation aims to be as light-weight as possible: there are no frameworks or toolkits in use beyond basic PHP and MySQL. Shared elements like the top bar are easily inserted into other pages via PHP includes. File caching is used in many places to reduce database access and other repeated processing. I’ve also been paying attention to making the CSS responsive and mobile-friendly.

Let me know if you find anything that seems broken. I will undoubtedly continue doing minor tweaks to the site over the coming weeks.

Memory savings and metadata cache

Build 2237 introduces a number of changes aimed at making Doomsday launch faster and use less memory overall. It also fixes a couple of threading issues that were found with the help of Thread Sanitizer.

As part of preparing the stable 2.0 release, I’ve been looking at performance and memory optimizations.

Trimming memory use

I examined memory use in Instruments on macOS and identified a number of places for savings:

  • There is now a purpose-built class for keeping track of observer relationships between objects. There are hundreds of thousands of these being used in the engine, so using a class that is more optimal for the task saves both memory and improves performance.
  • Some objects had a lot of observers (20,000+) but worst of all, this was completely unnecessary because the observers were never being notified. These cases have now been removed.
  • Reduced memory used by Doomsday Script bindings for file system access by removing several unnecessary variables and replacing them with callable functions. This saves memory because functions can be stored in the class objects rather than in the objects themselves.
  • The file system caches uncompressed ZIP file contents in memory to avoid repeatedly decompressing them when the ZIP is being accessed. However, these cached contents were never released. Now ZIP caches are freed from memory whenever you return back to the Home screen.

After all these changes, Doomsday’s memory use on my system was roughly halved (from 830 MB to 460 MB). In practice, though, this heavily depends on how many resource packs and other data files you have available.

Caching metadata

When Doomsday is starting up it indexes and quickly analyzes all the resource packs, add-ons and other data files that you may have available. This may take a while depending on the number of these files and how fast your hard drive is. Starting with build 2237, Doomsday is now able to cache this information for future use, so after the analysis has been done it doesn’t have to repeated until the files change or new ones are added. Continue reading Memory savings and metadata cache

Improved UI for multiplayer

For the past two weeks I’ve been focusing on finishing the Home UI for 2.0. I’ve also been doing plenty of small improvements and bug fixes.

The final element that has been missing from the Home is the Multiplayer tab. That is now no longer the case: build 2227 has a new UI for viewing server information, and a couple of important bug fixes for multiplayer servers.

Server info dialog

In the Multiplayer tab you can see more information about a server by right-clicking it or by clicking the […] button in the server description. This pops up the new dialog that can be seen in the screenshot above.

A direct connection to the server is automatically attempted so that you’ll see the server’s current status and to verify that the server is reachable. The status information includes the current map outline, just like in the Shell.

In addition to status information, the dialog has a “Join Game” button, a list of the packages loaded on the server (the client must load the exact same ones when it connects), and you can choose additional packages to load locally. These local packages are game-specific rather than server-specific and are remembered in the client configuration (part of persist.pack). The “Local Packages” button is disabled by default. It can be enabled in Network Settings.

Note that if you load additional local packages in multiplayer games, you must ensure that they don’t contain gameplay-affecting changes. For example, loading a different map is out of the question, as are changes to mobj states. Doomsday 2 packages with 3D model assets are safe to use locally in multiplayer games.

I now consider this UI, along with the rest of the Home, feature-complete for 2.0. The long-term thinking behind the server info dialog is to evolve it toward a mini-Shell interface so that you could issue quick console commands to servers that you control. That will not be part of the 2.0 release, though.

Continue reading Improved UI for multiplayer

Improved UI for packages

For the past week I’ve been finishing up with the Home UI for the 2.0 release. The Packages tab has gained a lot of useful functionality.

Package popup

The Packages tab helps you browse and manage your PWADs, add-ons, resource packs and other data files (all of which are called “packages” in Doomsday 2). Build 2208 introduces a new UI for viewing information about packages and performing quick actions on them. Like the game tabs, I consider the Packages tab now feature-complete for 2.0.

The new package info popup has the following features:

  • The popup can be quickly opened by right-clicking a package in any Packages list.
  • There is a convenient “Play in…” button for quickly loading the package in an existing game profile, without making any permanent changes to profiles. This works great for quickly trying out PWADs, for example.
  • The “Add to…” button lets you add the package to any existing game profile.
  • The “Show File” button opens the folder containing the package in Windows Explorer / macOS Finder / etc. (This may not be very helpful if all your data files are lumped in a single folder.)
  • The “Options” button allows selecting from the optional .box contents. It was relocated here from the old context menu that no longer exists.

A little bit of content analysis is done on the package:

  • Doomsday tries to guess which game the package is for, looking through any auxiliary readme files, the used WAD map format, Snowberry metadata, names of parent folder(s), etc.
  • In the case of WAD files, the game title picture is shown in the popup as the package icon.
  • If the package contains an “icon.jpg” (or “icon.png”) file, it is used as the package icon image. Currently these images are only displayed in the popup.

Continue reading Improved UI for packages

Holiday update

As usual December has been a busy time, but a little bit of progress has been happening with Doomsday, too.

Saving 3D model state. I’ve put the most effort into ensuring that the state of 3D model animations gets saved and restored when writing and reading savegames. This has no bearing on gameplay as such, but visually it is pretty important that the objects get rendered the same way before and after loading a save. In practice, this was quite challenging because the new model renderer supports advanced features like multiple animation sequences, scripted rendering passes, and shader variables. All this state information gets included in the savegame package as a new file, so it can also be gracefully ignored in case any compatibility issues or other problems arise when dealing with old save files.

Multiplayer resource check. When joining a multiplayer game, the client checks that the same packages are loaded as on the server. This includes all WADs, DEHs, and PK3s. Any resource mismatches could lead to crashes or other problems during MP games.

Mac menu. I did a small tweak for the Mac UI: Doomsday now has a native menu bar for switching between the available games.

Fate. Finally, I’d like to call attention to a cool Hexen mod by fate that I recently enjoyed playing through. If like me you are a fan of Hexen, this is a nice way to spend a few hours during your holidays: fate_v0.1.pk3. Please leave your feedback and/or comments to the author on the forums.

Merry Christmas and happy holidays everyone! *<:)

UDMF experiment, Shell game options, installer facelift

Last week I went on a brief modding tangent and added a plugin that loads UDMF maps.

UDMF is an attractive choice for map editing due to its flexibility and potential to be widely compatible with map editors and source ports. However, it is basically only a syntax for describing map data; the real trick is interpreting the contents and supporting all the required gameplay logic, line and sector specials, etc. Doomsday at the moment supports only vanilla maps with a couple of Boom features. The new UDMF plugin also doesn’t yet transfer all the data to Doomsday, for example most line flags are ignored.

When it comes to extending Doomsday’s support for modding, I view full Boom compatibility as the first milestone to reach. Modding is on the roadmap for the future 2.4 release, so in the near future the UDMF importer plugin remains an experimental feature.

I also did many small improvements in preparation for the stable 2.0 release. Continue reading UDMF experiment, Shell game options, installer facelift

Version 2 multiplayer servers

Yesterday I merged a work branch where I’ve revised the way information about multiplayer servers is stored and distributed. The changes from “mp-serverinfo” are available in build 2144.

There are some important changes to discuss. Doomsday 2 servers are now incompatible and separate from Doomsday 1.x servers. Doomsday 2 uses a new status information format, new master server web API and new, compressed LAN broadcast messages. Doomsday 1.x clients will therefore not detect Doomsday 2 servers, and cannot query information about them (and vice versa). Continue reading Version 2 multiplayer servers

Tightening all the screws

For the past week I have been exclusively focusing on optimizations.

Work in the 2.0 unstable builds has been progressing without much attention having been paid to performance profiling. However, it has become clear that even in very basic maps (like Doom E1M1) there were unacceptable FPS dips. I decided to get to the bottom of what was going on. Continue reading Tightening all the screws