This Week In Veloren 57

This week we hear from @xMAC94x about his work on the new networking protocol. We also have another contributor spotlight, this time with @Songtronix!

- AngelOnFira, TWiV Editor

Contributor Work

Thanks to this week's contributors, @Lorage, @xMAC94x, @AngelOnFira, @Artem, @imbris, and @Shandley!

@Kalculate is working through the NPC spawn code to fix spawn camping issues. Some of the core developers are working on figuring out a suitable modding language to handle animations. This is being headed up by @Zesterer and @imbris. @xMAC94x worked on automating Docker builds for the game server and Torvus. He is also working on networking systems and benchmarking them. @untouchedhappiness is working on Japanese translations.

New Networking Protocol by @xMAC94x

@xMAC94x has been working on the networking systems for quite some time. Here, he discusses a new communication protocol that he is prototyping.

The new networking protocol will add some layers above a simple TCP stream. Right now, I'm focusing on testing this protocol with benchmarks. The most basic TCP benchmark would contain two programs. One, the server, is running and listening on a local port, while the other, the client, connects to it. After the connection is made the client sends a ton of bytes. The server reads the bytes and promptly discards them, as their contents don't matter for the benchmark. You can then measure how much bytes the clients could send in a minute and then calculate the bandwidth throughput.

For TCP networking basically two factors are important:

1. The number of bytes per second
2. The number of messages per second. You can either send fewer big or a lot of small messages. Big messages are faster.

Simple TCP Networking has some disadvantages though, the major two being:

1. It cannot make use of the faster but unreliable UDP. UDP is great for sending data quickly, but it does not provide any guarantees that it will make it to the destination properly.
2. When a big message like a Chunk comes in, all other messages have to wait. This leads to high ping or latency.

That's where Veloren's new network protocol will fit in. It allows an abstraction over multiple Protocols at once: TCP, UDP, and direct mpsc channels. The Networking Layer knows which Ports belong to the same participant (client or server) and can send messages most efficiently. Also, big messages are split up into smaller parts which can be sent with a respective priority.

The stack is quite complicated and for the first time during development, I got it in a working state for a minimal example. That means, only 1 client each, only 1 protocol each (TCP) and only a single stream of messages. Before investing time in continuing to add more protocols or more clients I wanted to test the performance. If my way should prove to be inefficient, I can adjust it directly. That's why a benchmark at an early stage sometimes is a reasonable choice.

So what does the benchmark do? It's a program that can be either client or server (depending on the command line arguments). A, the server, will be launched and listen in a sperate thread for messages. B will connect to it and will start sending messages in a separate thread. The first result was; it didn't crash. This means we have an integrated pentest (or how I like to call it, Yuri-Test ;)

Every message is a simple ID, which increases, plus 1000 bytes of zeros (just to give it some work). These messages will be encoded and decoded twice, once when sending them. This allows the framework to handle messages like a large collection of bytes. These bytes will then be sent to the networking layer where routing and decisions are done. When the networking layer decides the TCP layer is ready to send, it takes a small part of the messages and brings it to the TCP layer. Here, it gets encoded a second time.

The message is sent to the other side. On the other end, the stack is applied backwards. It will get decoded, routed to the client and decoded again. In the end we get around 100.000 messages per second and about 800 Mbit/s. For a single client, both results are more than Veloren will ever need. But they are far from perfect compared to the big players in networking. So there are 2 things to do in the future:

1. Increase those numbers, to more like 500.000 messages/s and multiple Gbit/s
2. Focus on multiple clients, sending and receiving as this will be similar to the workload of a real Veloren server

Contributor Spotlight: Songtronix

Hey everyone, it's @Songtronix! I'm a 20-year-old guy living in Germany. I strive to learn more about what interests me: code, UI/UX/Graphic Design, and music/video production). I started working on Veloren in mid-February 2019. I found it while exploring the Rust community on Reddit/the web. Veloren was linked, and so I checked it out.

I'm the developer behind Airshipper, the Veloren launcher! It looked frustrating to explain for the 10th time how to download the latest version of Veloren. As such I've tasked myself to ease these kinds of experiences all around Veloren. I've made a download page which has the latest and greatest version of Veloren. I've made the launcher which currently does its job of automating the updating experience.

Written in one of the best languages, Rust, Veloren as an open-source project has opened me up to a lot of opportunities to improve myself. I've learned to work together with people who have different skill sets than me. I've learned to compromise but also bring in my own views and challenge myself to grow beyond my skills.

I want to make Veloren a first-class experience, not necessarily in-game but all around it. I want to improve the book, the documentation, and the onboarding process to make your first contribution that much easier ^^