Planning ahead for 2.1: multiplayer and OpenGL

The stable version 2.0 has been out for a bit over two weeks now. I hope you’ve been enjoying it! I have been busy working on bug fixes for 2.0.1, which is soon getting into shape for release.

Now that the goals for the 2.0 release have been met, it is time to look ahead to version 2.1. The first point release is planned to primarily feature improvements for multiplayer games. I won’t be exclusively working on that, though, so various minor improvements will also be included. Small things like a persistent console command history and options to start a game profile in a chosen map are already available in the unstable 2.1 builds.

When it comes to the release schedule, it remains futile to speculate about how much time I will have for Doomsday in the future, but my hope is that point releases in the 2.x series will have a roughly six month cadence. While there is no specific release date set for 2.1, I am uncomfortable with the idea of more than 12 months between stable releases. If that much time passes I am willing to postpone the remaining work to a later point release and do a smaller stable release instead. I prefer that there is a steady and predictable flow of stable releases, much like the situation is with the unstable builds.

In any case, if you run unstable builds you will of course get all the latest improvements as soon as they are implemented.

Multiplayer

2.1’s primary objective in terms of user-facing features is multiplayer improvements. This means both bugfixes and new features. I envision the following MP-related improvements in concrete terms (list still subject to change, in no particular order):

  • Bug fixes. Multiplayer gameplay is suffering from a variety of small glitches, particularly in Hexen. The goal here is to get rid of the more significant ones.
  • Movement smoothing for enemies. Single-player games use movement smoothing for everything, so MP games should be no different.
  • New chat UI. Modern games have some established conventions regarding in-game chat, so Doomsday should take inspiration from them. The chat should use Doomsday’s native UI widgets so the font isn’t blurry or too large, text can have colors and rich formatting, there is nice scrolling, text entry is more versatile, and there is autocompletion for usernames and commands. One particular direction this feature could take is integration with the console command prompt, so one can still issue console commands using IRC-like slash prefixes while primarily still chatting.
  • New scoreboard UI. A better scoreboard would be a nice thing to have for competitive game types. There should also be an option for a small, always-on scoreboard HUD. I will likely end up with a toggle between completely hidden, small HUD, and full-size scoreboard modes.
  • More GUI options for configuring game via Shell GUI. Most of the gameplay cvars should be configurable via the GUI. The 2.0 Shell already offers a couple of options (such as game type and current map), but there are more options that would be useful to have conveniently available. My strategy with regard to the Shell remains unchanged, though: for hosting games, the Shell should be the tool you use. Duplicating its features in the client GUI would be more or less redundant and wasted effort. However, I can look into integrating the Shell better with the client GUI, so it is quick and easy to get a local server started, for instance, by launching the Shell via the client.
  • GUI for configuring map cycle. Map cycling is one important MP feature, especially for longer-running servers. By default, a co-op server will not restart the episode once it ends; map cycling should for instance allow configuring entire episodes to be played through and then restarted.
  • Remote packages. The key feature here is loading PWADs from the server.

OpenGL

The internal objective for 2.1 is to complete or at least take a major step toward migrating the rest of the OpenGL code to version 3.3, making it also OpenGL ES 2 compatible. GLES is a requirement for Android and iOS, and also Raspberry Pi (a good low-powered testing/development platform). As a nice side-effect, this transition will also future-proof much of the code for subsequent migrations (particularly thinking about APIs like Vulkan, or some other low-level graphics API of the future). The crucial advantage that this transition will provide is support for GLSL shaders when rendering world surfaces. This truly opens up the potential for renderer improvements.

However, in the scope of 2.1, it must be noted that this will be an internal change, meaning the renderer features available to you remain roughly the same as in 2.0. Some older GPUs may no longer be supported, although it is also possible that some OpenGL drivers work better with OpenGL 3 instead of the old compatibility profile. It may also be necessary to drop some older graphics settings that are no longer relevant, if it isn’t feasible to reimplement them with new GL code.

Finally, please remember that plans may change so none of this is particularly set in stone. Stay tuned for updates. 🙂

Doomsday 2.0 released

The first stable build of Doomsday 2.0 is now available.

The highlighted features of 2.0 are:

  • Home Screen replaces the frontend app. A launcher is no longer needed; games and add-ons can be managed using Doomsday itself.
  • Built-in support for different data file and package formats. Used packages are tracked in saved games and multiplayer games.
  • The new 3D model renderer that has been in development since the 1.15 release is now mature enough for use. It supports FBX and MD5 models, skeletal animation, GLSL shaders, and scripting.
  • This is the first 64-bit stable release for Windows.

See the Manual for the complete release notes.

Thanks to everyone who reported bugs in the release candidates! A number of nasty bugs were found and fixed. There is still a few less serious known issues that have been scheduled for next month’s 2.0.1 update.

Work on the 2.x series will now continue along the lines planned in the roadmap, with the first focus area being multiplayer improvements. Under the hood, there is still lots to do with updating remaining old OpenGL rendering code so we can take full advantage of shaders in future releases.

Website refresh

In preparation of the 2.0 release of Doomsday I have redesigned the dengine.net website from the ground up. The new design went live today.

While the old design has been perfectly adequate during the past years, I felt that a bit of freshness would do the site good. I also took the opportunity to tie all the parts of the site together and reconsider the overall organization.

There is a new navigation bar visible on all pages, even in the Autobuilder and Bug Tracker that have previously been detached from the rest of the site. This should make it easier to move around and find what you’re looking for. In the bottom of most pages, you can find the latest news and blog posts, recent builds, and currently running multiplayer servers — these were previously only shown on the dengine.net front page.

Following the migration of the forums, the Doomsday Engine Wiki has been replaced with the more focused and structured Manual. It uses DokuWiki, which makes it considerably less complex and lighter to run than MediaWiki. The old wiki will remain accessible at its old address (as read-only).

I quite enjoyed the chance to do some web development. The new implementation aims to be as light-weight as possible: there are no frameworks or toolkits in use beyond basic PHP and MySQL. Shared elements like the top bar are easily inserted into other pages via PHP includes. File caching is used in many places to reduce database access and other repeated processing. I’ve also been paying attention to making the CSS responsive and mobile-friendly.

Let me know if you find anything that seems broken. I will undoubtedly continue doing minor tweaks to the site over the coming weeks.

Doomsday 2.0 RC3

Update: New build 2272 available with updated IWAD recognition rules.

The third and possibly final release candidate of version 2.0 is now available (build 2272).

There is a whole bunch of fixes:

  • IWAD files are now recognized with stricter rules to avoid files from being misidentified as known IWADs. If you have been seeing problems dealing with WAD files in the Packages browser, try DE → Clear Cache and restart.
  • Under specific circumstances, opening the game profile package selection dialog would crash.
  • Hang when trying to spawn damage particles triggered by a very large amount of damage.
  • Light from the Torch powerup in Heretic and Hexen was rendered incorrectly.
  • 3D model animation problem where a non-animating mobj would be repeatedly restarting its animation sequences.
  • Several UI glitches of varying severity. For example, the Update Available notification was triggering a fatal error, and there were occasional random color changes happening when UI widgets were being drawn.

Another notable change is that the folders where Doomsday looks for IWADs and other data files are now configured in the same Data Files dialog. I’ve added a new quick-access button to the bottom of the Home’s Packages list for opening this dialog.

Full change log

Doomsday 2.0 RC2

The second release candidate is available: build 2257.

This build has a number of bugfixes and minor improvements, particularly when it comes to loading and unloading data files. Some of the fixes affect metadata generated for data files, so if you’ve been running RC1, I recommend clearing Doomsday’s resource cache. This will also ensure that all add-ons are detected again, if some files were previously missing.

When one loads a savegame, Doomsday will attempt to load or unload resources depending on what was used in the save. However, this was quite buggy and could easily lead to a crash. There was also a mixup that caused Doomsday to try load and unload the same files twice. These issues are resolved in RC2.

I’ve made a couple of usability tweaks in Home. It should now be clearer when a profile is unplayable because the game IWAD is missing, or if the problem is that your selected additional packages are unavailable.

RC1 notes

It’s time to Talk

Things have been a little stagnant here on dengine.net for a while, so I’ve made some improvements.

The discussion forums have been migrated to a new platform. phpBB was getting really old and clunky so I made the decision to replace it with something more sleek and modern. You can find the announcement on Talk.

Another nice improvement is that the autobuilder now has a proper database stored here on dengine.net. This means that the download pages can show you the latest files instantly, and the build feed actually reflects the currently available data. The feed itself is dynamically generated from the database instead of being just a static file. Plus I’ve given the build report pages a facelift so they’re nicer to look at. Finally, all downloads are hosted primarily here on dengine.net, but they are also mirrored on SourceForge as before. The mirror download buttons can be found on the build report pages.

Doomsday 2.0 RC1

Build 2250 is the first Release Candidate for 2.0.

The full release notes will be available later, but the short version is that 2.0 introduces two big changes in how Doomsday works:

  • The Snowberry front-end application has been replaced with a new Home screen within Doomsday itself.
  • Doomsday has gained more understanding of various data file formats, resource packs, and add-ons so that it can load them without assistance from a front-end app. This Packages system has wide effects inside the engine, influencing game session configuration, savegames, multiplayer games, and the UI.

The updated Readme covers most of the new stuff, so that is a good place to start if you havent been using any 2.0 unstable builds recently.

There are certainly still bugs left to find and fix, so please report any problems you encounter with the RC build(s). The best place to do so is the Tracker.

In practical terms, development of the 2.0 release has been switched to the stable branch and is now in feature freeze mode. The master branch will roll over to 2.1 for new unstable builds. Now that the stable branch is using CMake as well, I’ve resumed the first-day-of-the-month stable builds.

Memory savings and metadata cache

Build 2237 introduces a number of changes aimed at making Doomsday launch faster and use less memory overall. It also fixes a couple of threading issues that were found with the help of Thread Sanitizer.

As part of preparing the stable 2.0 release, I’ve been looking at performance and memory optimizations.

Trimming memory use

I examined memory use in Instruments on macOS and identified a number of places for savings:

  • There is now a purpose-built class for keeping track of observer relationships between objects. There are hundreds of thousands of these being used in the engine, so using a class that is more optimal for the task saves both memory and improves performance.
  • Some objects had a lot of observers (20,000+) but worst of all, this was completely unnecessary because the observers were never being notified. These cases have now been removed.
  • Reduced memory used by Doomsday Script bindings for file system access by removing several unnecessary variables and replacing them with callable functions. This saves memory because functions can be stored in the class objects rather than in the objects themselves.
  • The file system caches uncompressed ZIP file contents in memory to avoid repeatedly decompressing them when the ZIP is being accessed. However, these cached contents were never released. Now ZIP caches are freed from memory whenever you return back to the Home screen.

After all these changes, Doomsday’s memory use on my system was roughly halved (from 830 MB to 460 MB). In practice, though, this heavily depends on how many resource packs and other data files you have available.

Caching metadata

When Doomsday is starting up it indexes and quickly analyzes all the resource packs, add-ons and other data files that you may have available. This may take a while depending on the number of these files and how fast your hard drive is. Starting with build 2237, Doomsday is now able to cache this information for future use, so after the analysis has been done it doesn’t have to repeated until the files change or new ones are added. Continue reading Memory savings and metadata cache

Improved UI for multiplayer

For the past two weeks I’ve been focusing on finishing the Home UI for 2.0. I’ve also been doing plenty of small improvements and bug fixes.

The final element that has been missing from the Home is the Multiplayer tab. That is now no longer the case: build 2227 has a new UI for viewing server information, and a couple of important bug fixes for multiplayer servers.

Server info dialog

In the Multiplayer tab you can see more information about a server by right-clicking it or by clicking the […] button in the server description. This pops up the new dialog that can be seen in the screenshot above.

A direct connection to the server is automatically attempted so that you’ll see the server’s current status and to verify that the server is reachable. The status information includes the current map outline, just like in the Shell.

In addition to status information, the dialog has a “Join Game” button, a list of the packages loaded on the server (the client must load the exact same ones when it connects), and you can choose additional packages to load locally. These local packages are game-specific rather than server-specific and are remembered in the client configuration (part of persist.pack). The “Local Packages” button is disabled by default. It can be enabled in Network Settings.

Note that if you load additional local packages in multiplayer games, you must ensure that they don’t contain gameplay-affecting changes. For example, loading a different map is out of the question, as are changes to mobj states. Doomsday 2 packages with 3D model assets are safe to use locally in multiplayer games.

I now consider this UI, along with the rest of the Home, feature-complete for 2.0. The long-term thinking behind the server info dialog is to evolve it toward a mini-Shell interface so that you could issue quick console commands to servers that you control. That will not be part of the 2.0 release, though.

Continue reading Improved UI for multiplayer

Improved UI for packages

For the past week I’ve been finishing up with the Home UI for the 2.0 release. The Packages tab has gained a lot of useful functionality.

Package popup

The Packages tab helps you browse and manage your PWADs, add-ons, resource packs and other data files (all of which are called “packages” in Doomsday 2). Build 2208 introduces a new UI for viewing information about packages and performing quick actions on them. Like the game tabs, I consider the Packages tab now feature-complete for 2.0.

The new package info popup has the following features:

  • The popup can be quickly opened by right-clicking a package in any Packages list.
  • There is a convenient “Play in…” button for quickly loading the package in an existing game profile, without making any permanent changes to profiles. This works great for quickly trying out PWADs, for example.
  • The “Add to…” button lets you add the package to any existing game profile.
  • The “Show File” button opens the folder containing the package in Windows Explorer / macOS Finder / etc. (This may not be very helpful if all your data files are lumped in a single folder.)
  • The “Options” button allows selecting from the optional .box contents. It was relocated here from the old context menu that no longer exists.

A little bit of content analysis is done on the package:

  • Doomsday tries to guess which game the package is for, looking through any auxiliary readme files, the used WAD map format, Snowberry metadata, names of parent folder(s), etc.
  • In the case of WAD files, the game title picture is shown in the popup as the package icon.
  • If the package contains an “icon.jpg” (or “icon.png”) file, it is used as the package icon image. Currently these images are only displayed in the popup.

Continue reading Improved UI for packages