To kick off the multiplayer improvements for version 2.1, I’ve started with adding access to data files over the network. For example, if a server is running a custom PWAD that you don’t have, the client will automatically download a copy before joining the game.
That pretty much sums it up for the user-facing portion of this feature. You will see a popup displaying download progress, and there is an option to cancel.
Much of this work was recently merged to the master branch (in the form of 83 commits) and is included now in unstable builds. Note that this kind of a larger influx of changes usually leads to new glitches also being introduced… I will be improving the code in the coming days/weeks.
Internally, this is part of a larger feature called remote package repositories. The basic idea is that there is a collection of packages stored on a web server, and Doomsday is able to download required packages when necessary. There are two kinds of repositories:
- A native package repository is provided by a running multiplayer server. In practice, all data files and packages loaded in the game session are offered as a package repository to clients. However, this excludes vanilla IWADs and Doomsday’s own core packages: the former because of copyright, and the latter to avoid version conflicts.
- A web-hosted package repository is a collection of data files and packages on a web server where the file tree is publicly accessible via HTTP. I will talk more about this in a later blog post.
Before going further into the implementation, I would like to mention one nice addition to the Doomsday core library. Network operations can take a few seconds to finish, which makes it important to have a convenient API for asynchronous operations — the goal is to never freeze the UI. For this kind of tasks I added a new de::async utility that executes a callback in a background thread, and when that is finished, executes another callback in the main thread. This allows performing tasks like event processing, UI, and graphics in the right thread. At some point in the future I expect this will allow getting rid of the old “Busy Mode” background thread.
de::async has been very helpful with implementing the communication between the client and a file repository. The protocol works roughly as follows:
- When first connecting to a repository (e.g., the multiplayer host), a list of the available files and/or packages is downloaded. The client can then quickly determine if required packages are available remotely.
- A client may query file metadata such as names, sizes, and modification timestamps. In Doomsday’s virtual file system, this information is used for creating symbolic links that represent the remote files. The links can be manipulated like regular local files, with the exception that the contents need to be downloaded before reading.
- Downloaded file contents are cached under <runtime>/cache/remote/ so that in the future the same file doesn’t have to be downloaded again. By default, downloaded files are tagged as “hidden” and “cached” so that they are hidden in the UI. (Note that entering “hidden” in a package search field will show only the hidden packages.)
These changes enable several UI and user experience improvements in the future. For instance, there can now be an option for installing a remote package locally. Also, the Shell can be enhanced with a UI for easily selecting PWADs and other mods to use in multiplayer games, and the clients don’t have to worry about setting up the same files manually.