This Week In Veloren 140

This week, we hear about some bugs that are in the game right now. We also explore some upcoming changes to the economic system.

- AngelOnFira, TWiV Editor

Contributor Work

Thanks to this week's contributors, @zesterer, @imbris, @Sam, @ubruntu, @a1phyr, @Yurimomo, @flo, @sylvain_m, and @Bafon!

@zesterer got shrubs merged, so at last most existing water issues have been fixed and rivers should generally look better. Also, waterfalls!

@ubruntu made a small change so that the interact key can be used on campfires. Right now it makes you sit at a campfire which activates the campfire healing aura. This opens up the possibility that campfires open a crafting menu in the future.

@imbris created a credits screen so attribution for various art assets (especially the ones that require it) can be added. They also refactored resizing a bit making it no longer produce weird behavior or Vulkan validation errors when rapidly resizing.

Network speed issues by @AngelOnFira

Of recent, we've been seeing a lot more people mention they have issues with downloading, connecting, or playing the game due to network issues. This week, I've been working on examining the cause of these issues more.

First off, even for fast internet connections, people haven't been able to download updates on Airshipper at more than around 100Mbps. This issue could come from many places, including the server network, the server filesystem, the datacenter networking, the Airshipper client, or something more obscure.

I examined some code on both the Airshipper client and server to track down where the key points of the download are happening. I didn't find anything that stood out right away, such as an artificial bandwidth limit hardcoded or anything.

I then came up with some tests to rule out areas where the issue might be caused. I first launched a dozen or so Airshipper instances, and got each of them to update at once. You can see the network usage in the image below. Note; something is wrong with the metrics, since our server is capped at 1Gbps.

However, with this graph, we can see that we are able to transmit far over the 100Mbps rate that clients are receiving. This was backed up by the fact that all of my Airshipper clients were able to download the updates in about the same amount of time as if it was just a single client downloading.

This increased the probability of the problem being on Airshipper's client, however, I quickly ruled this out by trying to manually download the updates through my web browser. I was limited to the same speed, and Firefox can certainly download higher speeds.

I then simulated the Airshipper server on my local machine. It uses Rocket under the hood, so @imbris gave me a hand looking around to see if it could cause any issues. Once I ran the tests, I did see that it allowed me to download files locally at greater-than-gigabit speeds. At this point, it seems that the client code is fine, the server code is fine, and the problem has to be in the network or the server's filesystem.

I spend some time playing with how Docker access files from its filesystem. I'm not sure about this, but I think that if a volume is mounted into Docker, it is accessed differently than if it was built into the image. I tried running some tests on the server to see how it was able to access data from within the container vs on the filesystem.

These tests aren't very accurate, but it does show us that it can write files inside and outside the Docker container at about the same speed. Next, generated a garbage file that was a gigabyte in size inside the server, and tried downloading it through HTTP, as the client would. The download was on the same host, and it was able to download at local speeds.

After all of this, it seems the problem is around how fast a single stream can download data from Hetzner. I ended my investigations for the night at this point, and sent off an email to Hetzner to see if they could help. They didn't know of any immediate issues, and suggested running some diagnostics from their end.

Going forward, there are still a few more tests I want to run. One is getting a generic file server to try and download a file from. I want to prove that we can't download any file from the server at more than 100Mbps. From there, I'll probably dive further into networking on the VM itself. Hopefully, we can see improvements to download speeds in the future.