This Week In Veloren 44

Small blog this week! Advent of Code has started, and I've become strapped on writing time. Others also have a lot going on, and so there aren't any major writeups. However, I have compiled some of my ramblings into a new TWiV section.

- AngelOnFira, TWiV Editor

Contributor Work

Thanks to this week's contributors, @AngelOnFira, @xMAC94x, @Pfau, @Adam, and @Treeco!

This week, @Pfau got the first animated UI element going! @Zesterer and @imbris helped out with the implementation. @Sharp is continuing to work on sediment transport in the erosion system. @AngelOnFira has worked on more GitHub Actions CI testing and compression of assets.

@Silentium also prepared a flow chart to guide the topics covered in design meetings.

Since I don't have much content to curate about this week, it's time to rant about stuff I've been exploring. As meta lead, I deal with a lot of experimentation of stuff that you don't see in the game. From this, I've written some incomplete blog posts. Taking inspiration from His Dark Materials, here is the first edition of the "Lantern Slides"

Lantern Slides: Mac Build Predicament

Building for Linux is easy. Veloren developers work across many different flavors of it, so we have a good breadth of installation instructions. Windows is also pretty standard, and its drivers are quite resilient. Mac, on the other hand, is not so friendly.

With our deployment setup, we want to be able to build executables from an Ubuntu runner. Although this is Linux, it can build for Windows with a specific Rust target installed. Mac also has a target that can be installed on Linux, however, it also needs a specific compiler.

The compiler that can be used to build Linux and Windows executables is quite standard. Mac has a special SDK that has to be used to build the compiler. I'm still learning why this is the situation, but it adds to the process required to build a Mac executable. This is much easier done when building from a Mac, however that is not great for us. The intended solution is as follows:

1. Dump the Mac SDK from XCode
2. Package the SDK in a specific format
3. Build the compiler with OSXCross

First off, I don't want to install XCode just to get the SDK. Luckily, there is a GitHub repo that has the SDKs prepared. I then made my own repo that packaged them as they weren't already. Next up, I started working with OSXCross. Unfortunately this is where I'm stuck.

OSXCross requires a lot of manual tinkering to get it working. The latest example of it working with a Rust project is a few years old. The documentation can only get you so far, and there is a lot of digging that has to be done. Hopefully, this can be ironed out before the 0.5 release, as I've promised Mac builds for quite a while.

If I get it working, I would like to build a prototype Docker image that can be used to target Mac from Linux for Rust. This will hopefully be useful for the Rust CI community, as currently, I'm not aware of any other option than building from a Mac device. With GitHub actions just releasing with Mac runners, hopefully, there will also be other solutions parallel to this one.

Lantern Slides: Unfinished Rust CI Blog

Since Rust is a compiled language, it uses many techniques to help lower compile time. Most of these include storing files in the target/ folder of the project.

Looking at Veloren as a Case Study

Veloren is an open source game being developed with Rust. One of the key reasons we use GitLab for Veloren is the integrated CI.

We have several GitLab runners that are each used for all of the CI tasks that Veloren requires. Each runner can compile, test, lint, verify, and build executables.

The Docker image used on the runners contains:

• All of the dependencies for builds
• A specific version of Rust that is forced in dev as well
• Targets for cross-platform builds
• Rustup components for tools like Rustfmt and Clippy
• The cloned project for easier checkout during builds
• A target folder populated from cargo run

This means that the Docker image is quite large, around 6gb. This is unfortunate, however, it greatly helps CI times. During CI, some of the benefits from this are:

• The entire ~150mb project doesn't need to be re-cloned each time
• The specific Rust version doesn't need to be installed
• Many of the dependency crates won't need to be rebuilt, let alone redownloaded
• Many of the source files for Veloren are already compiled. Therefore, since many merges to master don't change all the files, they only have to compile the files they changed.

The runners have a cache that stores the target/ folder after each build. It uses a Docker anonymous volume for this. Here is what the target folder looks like after a week of a runner's work:

11G target/debug
1.3G target/release
2.5G target/x86_64-pc-windows-gnu

The size difference makes sense since the initial compile and all tests are run with the Linux target. This stores artifacts in target/debug. The target/release and target/x86_64-pc-windows-gnu only hold the final executables for their respective platforms.

However, the size of the folders is quite a problem. Since this runner has only been running for a week, a 15gb target folder is quite large. A runner that is new and contains only a single build of each type takes up the following space:

6.1G target/debug
1.5G target/release
2.4G target/x86_64-pc-windows-gnu

Note, the reason that target/release is bigger is most likely that the runner didn't get asked to run that specific stage.

So, in the week that the runner was doing jobs, it accumulated around 5gb in just the target/debug folder. In fact, some of our runners have accumulated 100gb+ of target artifacts. This has caused us to sometimes run out of space on the runners, which we've then had to manually restart. Not ideal.

Let's examine some other differences between the target folders of these runners. Let's call the runner that has run for a while old runner, and the new one new runner.

First, here is some great information about what the target folder contains. Let's take a look at what the target folder of new runner contains:

drwxr-xr-x 8 root root 4096 Jul 21 20:29 .
drwxr-xr-x 5 root root 4096 Jul 21 19:43 ..
drwxr-xr-x 153 root root 12288 Jul 21 20:29 build
-rw-r--r-- 1 root root 0 Jul 9 12:05 .cargo-lock
drwxr-xr-x 2 root root 131072 Jul 21 20:33 deps
drwxr-xr-x 2 root root 4096 Jul 21 20:29 examples
drwxr-xr-x 836 root root 45056 Jul 21 20:30 .fingerprint
drwxr-xr-x 41 root root 4096 Jul 21 20:33 incremental
drwxr-xr-x 2 root root 4096 Jul 9 12:05 native
-rwxr-xr-x 2 root root 4487824 Jul 21 20:29 veloren_chat_cli-94fcc09b2c5cad1e
-rw-r--r-- 1 root root 2942 Jul 21 20:29 veloren_chat_cli-94fcc09b2c5cad1e.d
-rwxr-xr-x 2 root root 4487808 Jul 21 20:29 veloren_client-d0c1376a22680bbe
-rw-r--r-- 1 root root 2896 Jul 21 20:29 veloren_client-d0c1376a22680bbe.d
-rwxr-xr-x 2 root root 11732848 Jul 21 20:29 veloren_common-0b3ae1d1ad5c1cb1
-rw-r--r-- 1 root root 2812 Jul 21 20:29 veloren_common-0b3ae1d1ad5c1cb1.d
-rwxr-xr-x 2 root root 82117288 Jul 21 20:20 veloren-server-cli
-rw-r--r-- 1 root root 3758 Jul 21 20:19 veloren-server-cli.d
-rwxr-xr-x 2 root root 4487840 Jul 21 20:29 veloren_server_cli-d029ad657c74936b
-rw-r--r-- 1 root root 3775 Jul 21 20:29 veloren_server_cli-d029ad657c74936b.d
-rwxr-xr-x 2 root root 5311464 Jul 21 20:29 veloren_server-fc3a2f6366701d99
-rw-r--r-- 1 root root 3725 Jul 21 20:29 veloren_server-fc3a2f6366701d99.d
-rwxr-xr-x 2 root root 227318048 Jul 21 19:34 veloren-voxygen
-rw-r--r-- 1 root root 9061 Jul 21 19:29 veloren-voxygen.d
-rwxr-xr-x 2 root root 23485928 Jul 21 20:29 veloren_voxygen-fb154c7c142b5881
-rw-r--r-- 1 root root 9078 Jul 21 20:29 veloren_voxygen-fb154c7c142b5881.d
-rwxr-xr-x 2 root root 4563864 Jul 21 20:29 veloren_world-1f1dbc7fde8cc98d
-rw-r--r-- 1 root root 3466 Jul 21 20:29 veloren_world-1f1dbc7fde8cc98d.d

Here is where I stopped writing. Maybe someday I'll finish these ideas and write a remediation blog alongside this analytical post.