Memory savings and metadata cache

Build 2237 introduces a number of changes aimed at making Doomsday launch faster and use less memory overall. It also fixes a couple of threading issues that were found with the help of Thread Sanitizer.

As part of preparing the stable 2.0 release, I’ve been looking at performance and memory optimizations.

Trimming memory use

I examined memory use in Instruments on macOS and identified a number of places for savings:

  • There is now a purpose-built class for keeping track of observer relationships between objects. There are hundreds of thousands of these being used in the engine, so using a class that is more optimal for the task saves both memory and improves performance.
  • Some objects had a lot of observers (20,000+) but worst of all, this was completely unnecessary because the observers were never being notified. These cases have now been removed.
  • Reduced memory used by Doomsday Script bindings for file system access by removing several unnecessary variables and replacing them with callable functions. This saves memory because functions can be stored in the class objects rather than in the objects themselves.
  • The file system caches uncompressed ZIP file contents in memory to avoid repeatedly decompressing them when the ZIP is being accessed. However, these cached contents were never released. Now ZIP caches are freed from memory whenever you return back to the Home screen.

After all these changes, Doomsday’s memory use on my system was roughly halved (from 830 MB to 460 MB). In practice, though, this heavily depends on how many resource packs and other data files you have available.

Caching metadata

When Doomsday is starting up it indexes and quickly analyzes all the resource packs, add-ons and other data files that you may have available. This may take a while depending on the number of these files and how fast your hard drive is. Starting with build 2237, Doomsday is now able to cache this information for future use, so after the analysis has been done it doesn’t have to repeated until the files change or new ones are added. Continue reading Memory savings and metadata cache

Game profiles, autobuilder upgrade

Progress has been somewhat slow recently. The central new component I’ve been working on is game profiles. They represent the combination of a game and a list of packages to be loaded with that game — much like a Snowberry profile. A key feature of the Home UI is to let you create and manipulate game profiles.

You may recall that a few years ago we added renderer appearance profiles to manage the various renderer settings. The implementation of these was tied to the old console variable system, but there was a good portion of code that was reusable also for game profiles. I refactored the appearance profiles to extract the generic parts that deal with saving and loading profiles to/from an Info document, and generalized the concept of a profile. That allowed me to rather easily implement game profiles. In practice, the profiles will be written to a file called “configs/game.dei” under the runtime folder.

Now that Ubuntu 16.04 is around the corner, the autobuilder server is due for a system upgrade. My plan is to get rid of the 32-bit Linux builders (Launchpad should take care of that), and only set up builders for 64-bit Ubuntu and Fedora. The Windows builder will be using Windows 10 and the upcoming Qt 5.6 with MSVC2015. I should be finally able to configure a 64-bit build also for Windows (fingers crossed!).