This Week In Veloren 100

51 minute read28 December 2020

Authored by AngelOnFira

This week we have the 100th edition of This Week in Veloren! We have writeups from many contributors, making this the longest edition yet.

- AngelOnFira, TWiV Editor

Contributor Work

Thanks to this week's contributors, @XVar, @Acrimon, @nwildner, @Pfau, @SarraKitty, @Slipped, @Pfau, @Snowram, @Entropy, @zesterer, and @Frinksy!

This week we have a unique blog post. It's our 100th edition! For this blog, we had to make it the biggest one yet. We have writeups from 20 of Veloren's developers, artists, composers, and other contributors. Because of this, here is a table of contents with all of the topics. Enjoy!

Preface by @AngelOnFira

I joined the Veloren project in January 2019. I was intending on making a very similar project myself, and in doing research, I stumbled across Veloren. I knew some Rust at the time; enough to solve the kinds of problems you'd find on HackerRank or Advent of Code. So I looked for other areas that I could contribute, and found TWiV. There was only a single post that had been made at the time, and it dated to a few months prior.

So I started writing the blog. At first, it there was just an art and programming section. Around the 12th edition, we saw the first writeup. More structure started being added to the blog, and more content was coming in from contributors. We saw concept art, ideas in prototyping phases, new systems being described, and much more.

We've seen many great benefits from the blog: people have found out about the project, aspiring developers have seen something they could contribute to, and it's created discussion around many elements of the project. Further, it's been great seeing what others have been working on in a curated format.

So here's to the progress that has been made over the last 100 weeks, and here's to another 100 🍻

A Veloren Short Story by @zesterer

You start your journey in a town named Mosswall. The town is small, but it has most of the essentials: a fletcher, a carpenter, a small open-air market, and a tavern. You realise that you have a few pieces of gold about your person so you decide to visit the market. Since you appeared to start in a small square, you do so via a narrow cobbled street, lined with houses. Townsfolk walk past and greet you as you go.

Once you reach the market, you find several merchants flogging their wares. You walk over to the nearest: she's selling weapons, telling you that her swords came from a blacksmith in Cliffheim, a dwarf settlement to the north. Curious you pull out your map: sure enough, you can see Cliffheim nestled amongst the foothills of a mountain several days walk to the north. You like the look of this sword: it is sturdy and simple, with a leather grip. You feel that it'll serve you well.

Satisfied, you pay the merchant, and she thanks you for your business. Curious about what you might need next for your adventure to truly begin, you enter the tavern on the opposite corner of the marketplace. It's dark and quiet, much as you'd expect from such a small town. The only light comes from a fireplace set into the wall and sunlight coming in through the window.

There are a few travellers at the bar (as implied by their heavy backpacks and light travel armour). You walk over to one and ask them where they're visiting from. They say they've come from the west, a lakeside village known as Feastloch. The town had recently been having trouble with some of the wildlife: trolls from the northern mountains had expressed interest in the village's plentiful supply of fish and had invaded the town. Most of the inhabitants had been forced to flee. Those that remain are asking for adventurers to help them fend off the trolls.

You bid the traveller farewell and they thank you for your company. This seems to you like an opportunity to make a name for yourself as good as any, so, after visiting the market for some food supplies, you make your way down the road that heads out of town. Cobble gives way to gravel, and gravel gives way to dirt: eventually, you find yourself wandering through an oak forest.

There is birdsong and flowers, and a slight breeze in the air. The path continues for an hour's walk until you come to a fork in the road. A sign near the fork reads 'Feastloch, south-west. Cliffheim, north'. Encouraged by your progress, You set off to the west. The terrain begins to change. Broad oak trees give way to sparse pine, and the grass dulls.

The air chills and the wind picks up as you climb to the crest of a hill. Staring down into the mist, you see the mirror-like surface of a lake, scattering the early evening sun across the valley. Smoke rises from the chimneys of a dozen houses on the lake's northern shores. This must be Feastloch.

Some distance ahead you spot a figure walking the path. As the figure approaches, he yells a greeting, and asks you where you're headed. You tell him that you're travelling to Feastloch. He shakes his head and tells you that the trolls have taken the village for their own. An unprepared traveller like yourself would surely struggle to take them on. Undeterred, you tell him that you intend to free the village from the trolls. He protests, but wishes you well and hands you a flask of liquid, telling you that it'll heal you if you get hurt.

You reach the outskirts of the village in time to hear a bellowing roar from the direction of the village hall. Carefully, you make your way toward the noise, taking care to remain hidden behind trees and marsh reeds. The bellowing continues, and you catch sight of an enormous troll attempting to break through the wall of the village hall.

Thinking quickly, you dig through your bag. Amongst other items, you find a flash bomb: just what you need. The troll roars once more and you throw it toward the village hall. It's well-aimed: it hits the ground next to the troll, exploding with bright light and flames. The huge beast jumps in surprise and runs away into the tree line behind the town, dropping its huge club as it does so. You have scared away the troll, and obtained a heft club in the process!

UI Progress by @Pfau

As the only developer mostly working on UI design and UI frontend for Veloren I found myself contributing to two different branches lately.

For the inventory upgrade branch, we needed to rethink and resort some of the bag elements, add a completely new system to expand the inventory view, and fit in new slots for bag items. Additionally, this involved planning out the game design of this exciting new feature and organizing some test runs. And this was just my (small) part in this. Everything going on "below the surface" took weeks for @XVar to fully flesh out until we could even think about UI.

Another very interesting new feature I was involved in was skill trees. These first required a more or less fully automated way to generate a layout. Then, the developer (@Sam) writing the underlying systems just has to place simple button widgets onto the pre-generated layout and make them trigger events on pressing them. One can basically put in some values at the top of the file and the code below creates just the right amount of background elements and places them based on a certain ruleset. Working on this was really cool as it felt like creating a 2D level generator.

All of this is especially amazing to me because I started out with just knowing a bit of CSS and LUA when I joined the Veloren Team in 2018. My last CS class dated all the way back to around 2008 when I started to learn graphic design. My origins are actually in the World of Warcraft UI modding community, which you may or may not recognize in some of the Veloren UI elements 😉

~Pfau

Glider Physics by @Slipped

I'm working on an animation branch by @lboklin that adds realistic glider physics to the game. While the master setup only allows for turning with the glider, this adds aerodynamics that will simulate the movement down to details like air density and surface area of the aerofoil to calculate drag/lift and track pitch, yaw, and roll. You can pull back on the glider to enter a slow glide or lower yourself to the ground. Push forward on the glider to go into a fast dive. Banking properly allows for a ton of fun movement, even in tight spaces. The player is also now a lot more responsive to the movement, getting pinned to the glider at high speeds and hanging freely when coasting.

@LunarEclipse's Veloren Journey

I joined the project soon after @zesterer created the Discord server. I offered to help set it up and that's how I got involved. At the time I had very little programming experience, so I was open to learning any tools others would choose to develop the game with. After Rust was chosen as the language we would use, I started reading the official Rust book. It wasn't until the engine rewrite when I started working on the actual codebase. By that time I had read about half of the book, and with a lot of help from @zesterer, I took on the task of writing the networking code.

The API was really simple - the server would have a PostOffice, which would receive connections represented as PostBox. The client would call PostBox::new(address) which would connect to the server, and again yield a PostBox instance. After that, utilizing the connection was as simple as calling .send(data) and .receive() on the PostBox. Unfortunately, while it passed the unit tests I wrote, it didn't actually work. I couldn't find the issue, so @zesterer took over and managed to make it work. The overall architecture didn't change for quite a long time though, so I was a bit proud of at least getting that right. I learned a lot along the way as well, especially about multithreading and sharing data between threads.

For some time after that, I contributed more minor things, or helped with things unrelated to the actual code. Even at times when I didn't contribute much to the actual game I've always been around on the discord server. I've dipped my toes into the world of voxel art - I actually created one model that is still in game

the wheat model. At some point I learned how to create packages for Arch Linux, and have been maintaining a few for Veloren ever since (veloren, veloren-bin, and veloren-git).

My biggest contribution was probably adding controller support. I initially started working on it around 0.3, but lost motivation for a while, and ended up starting from scratch sometime before 0.5. While it took me some time, I actually pulled through that time, and controller support was included in the 0.6 release. It required me to learn about multiple in-game systems like settings, the input system, and the code consuming the input events (that was especially the case for analog inputs like joysticks, since before the game only supported key press/release events with the exception for mouse movement).

I used the GilRs library to handle the low-level implementation details, and after familiarizing myself with the examples I started plugging it into Veloren. There are 3 input types recognized by Veloren a controller can provide: (check out the code here):

Button - binary input - like with a keyboard a button can either be pressed or not pressed, but nothing in-between
AnalogButton - analog input which can be represented by a number between 0.0 and 1.0 - usually those would be for example triggers.
Axis - analog input which can be represented by a number between -1.0 and 1.0 - an example of that would be one axis of a joystick.

For the purpose of configuring key bindings, each of them is represented by an enum containing two variants:

Simple(GilX) where GilX is an enum provided by the library representing all Buttons/Axes it recognizes
EventCode(u32) is used for inputs to which GilRs assigns the Unknown variant of the aforementioned GilX enum

Due to the way the raw events from GilRs are converted to the Veloren-specific types, for example Button::Simple(GilButton::Unknown) will never match any button press, as all events corresponding to unknown buttons are represented by the Button::EventCode variant. It's similar for the AnalogButton and Axis enums.

This can be used to our advantage, as instead of using additional Option<T> wrappers in the key-mapping code, we can use the Simple(GilX::Unknown) variants to signify not binding an action to any key. While this might seem confusing, I think it's a pretty neat solution, and I made sure to place comments informing about this in the relevant places of the codebase.

Let's move on to the actions the inputs can correspond to. As you might expect, there are 3 types of actions depending on the input type. But in addition to that, they are divided into menu actions and game actions. The names are self-explanatory, the actions a controller can be bound to in menus are completely different than in-game actions.

Now let's talk about settings. As you can see, there are key-maps for each input type, one for the menu inputs and one for the game inputs. In addition to that, you can configure dead zones per-input, invert some axes, adjust the in-game camera pan sensitivity and sensitivity of the menu mouse emulation (more on that later).

Those were some of the more interesting things about the controller support system, if you're interested in learning more about it, you can look at the merge request. Most of the magic happens in controller.rs and window.rs. Some of the code has changed by now, but most of what I'm talking about here stayed the same.

Now let's talk about some things related to controller support that I didn't implement. First off, a graphical configuration interface. Implementing it requires knowledge about the graphical parts of Veloren, which I haven't touched to this day. The other thing was proper menu navigation - with a controller the most natural way is to have one element be highlighted, and move the selection with a joystick. That would require a lot of work though, as it would probably require major changes to the way the GUI is handled, so I settled for mouse emulation, which while not optimal, was a much simpler solution to implement.

Since implementing the above, the biggest thing I've worked on was planning a screenshot and trailer competition which was supposed to take place around the 0.8 release, but it turned out to require much more time than I had anticipated to properly prepare, so I have put that project on hold for now. I might take another shot at it in the future. No promises for now though ;)

To finish up, I've learned a ton while working on the project and met lots of amazing people. I would like to thank everyone involved in making Veloren what it is today - a promising game, and an amazing community. I look forward to seeing what the future brings us!

Modeling with @Gemu

When I first found Veloren, I was a bit anxious to join and hadn't even considered being able to contribute. After I joined the Discord, I noticed the art section channels, which sparked something that made me want to try after all. After I posted my first model attempt, the nice reception kept me motivated to make more. By now, I have made many models that are present in the game through the help of others. The continuous support I've received kept me going, and I plan to contribute just as much next year as well.

The biggest thing I've worked on was probably the creature model rework. I proposed the remodelling of the animal models at the time, which I took upon myself. Lately, I haven't been able to contribute much, due to being busy. I made a few voxel models of item drops, like crafting materials such as wood, ores and gems. When the new model-skeleton for smaller bipeds was introduced however, I got a surge of inspiration and made a bunch of models for demihuman creatures alongside others. As all dungeons were generally occupied by humanoid NPCs for now, I'm definitely looking forward to the diversity this will add to the content.

Modeling with @Snowram

Heya! It has been a year since I have discovered Veloren and its awesome community. This year has been crazy and I'm proud to have taken part in this adventure.

I started to contribute to Veloren by creating some NPC models. Learning how to make them has been a really good learning experience as it was the first time I played with voxel modeling. After delving into the process to add these new species into the game, I discovered how to make skeletons and animations for them. At first, it was a time-consuming process but we achieved to make the workflow easier and easier to allow for complex animations. Now I am working with the art team to make voxels come to life and integrate them into the ecosystem.

There are now almost 100 different species and we are only starting since new skeletons are often getting added. I have also started to learn some Rust basics to be more involved in the development process. So far I added some simple features like a weapon speed stat or allowing items to have buffs. For the future, I'd like to progress further and finally overcome the steep learning curve. The year to come will be very exciting and I'm eager to see what comes next!

Work by @XVar

I'm XVar - Technical Architect for a UK-based software company by day, Veloren developer by night (and often weekends...). I first came across Veloren on Reddit, it seemed like a pretty neat project and I'd been looking for a more substantial project to improve my Rust skills with, so I joined the Discord and more or less jumped straight in. At that point my Rust experience was fairly limited, comprising of a mostly finished Gameboy emulator and not much else.

Bringing Clippy Back

My first MR came about when I ran cargo clippy and was faced with hundreds of warnings, rendering Clippy completely useless within the Veloren project due to the noise of the vast number of existing warnings. Cue me adding hundreds of Clippy suppressions to get to a baseline of 0 warnings and creating a tracking issue that to this day still needs completing. Although many of those initial suppressions still exist, following this MR we added a zero-tolerance policy for Clippy warnings to our CI pipeline so Clippy continues to be a valuable tool (especially to developers new to Rust) within the project.

Persistence Overhaul

Following this MR I made numerous bug fixes and minor changes before coming across an issue that I considered myself fairly well qualified to deal with - Veloren's database persistence. At this point, Veloren had only had persistence for character data for a few months - a vast improvement over the previous complete lack of it but a fairly brittle implementation. With item data stored in the database as serialized JSON including a copy of the actual item definition itself this led to serious issues whenever item definitions were changed leaving "legacy" items in existing inventories and loadouts with only newly created items having the new properties from the asset files.

The solution to this was to rework the database schema to only store the ID of item definitions rather than copies of the entire item, as described in this RFC. As well as removing the issue of "legacy" items requiring complex JSON migrations whenever item definitions were changed I also introduced the concept of "every item can be a container" - more on that later. The persistence MR was a bit of a beast, with development ongoing between August 2nd and September 18th with 77 commits total including an 800 line database migration to extract all of the existing JSON blobs into the new database schema. @Sharp's input and collaboration on this MR proved to be invaluable particularly with regards to ideas on how to make the database schema even more robust.

"Has the server crashed?"

Anyone who spent more than a few days in #general on Discord prior to December 2020 will have been familiar with the frequent questioning from players of why the server just went down following the merge of an MR and the automatic server restart. That's to be expected when you're faced with an obscure networking error out of the blue, after all. Seeing this happen so often prompted me to investigate how we could give players warning of an upcoming restart, and it turned out our Docker infrastructure included all the required components already due to our use of Watchtower for pulling docker image updates.

Watchtower by default sends SIGTERM to the watched container (in this case Veloren-server-cli) - which results in the server instantly going down. However, it also supports sending custom signals to processes rather than straight up killing them. I made use of this by adding functionality to Veloren-server-cli which starts a two minute timer upon receiving a SIGUSR1 signal. For these two minutes global chat messages are sent to all connected players informing of them of the pending restart, before the server is finally shut down after the two minutes expire.

Unfortunately, this feature didn't work as intended for a further two months as I kind of forgot about it after it went in and there was a bug in our Dockerfile that prevented the SIGUSR1 signal from actually reaching the Veloren server process. Once this fix went in though, the calls of "has the server crashed?" were instantly silenced.

Inventory / Loadouts

After completing the persistence MR I vowed to avoid tackling such a large task for a while, so naturally, a month later I began work on a complete refactor of inventory/loadout handling.

For as long as I've been part of the project, players have had a fixed 36-slot inventory with no way to upgrade the size of their inventory. Numerous discussions were had (and aborted prototypes were made) before we settled on implementing inventory upgrades with 4 new bag slots in the loadout.

Not only will this allow players to carry many more items than today, but it also introduces the concept of items acting as containers that hold other items. Every item that's in a slot other than the basic 18 slots of the the inventory itself will actually be "inside" the equipped bag item that provides that slot.

This functionality won't have much of an impact when it's first merged, but it'll allow for some interesting possibilities in the future. For example, if weight is added to items combined with overburdening you might want to quickly drop a specific bag of stuff on the ground in order to run faster and escape imminent death.

The requirement to have 4 loadout slots that could all hold the same kind of item (bags) necessitated refactoring to remove the previous design restriction of a 1:1 mapping between ItemKind and EquipSlot. This not only allowed us to implement the 4 bag slots, but after this MR is merged you'll notice that you can equip a ring on both hands rather than the previous single ring slot.

This change hasn't merged yet, but it should be merged in early January 2021 😁 Pfau's blog post has more info on the UI changes related to this change too.

Closing

While 2020 has been a pretty crap year by all accounts, I've had a blast working on Veloren and seeing it evolve over the past six months. I still don't consider myself a Rust expert by any means but working with it consistently has not only massively increased my productivity in the language, but also I frequently find myself thinking about problems from a different angle when working in other languages too due to Rust's disallowance of data races and such.

Seeing all the work ongoing with world-gen, the transition to wgpu and iced, the huge improvements to combat and the beginning of a plugin framework all make me excited to see what's next for Veloren and I think the game has a bright future in 2021 and beyond.

Combat Improvements by @James

Hi there! My name is James and I have been contributing to Veloren since August, where @Sharp was kind enough to let me volunteer my hardware to fix an NVIDIA shader bug. I have mostly worked on combat and AI code. For anybody who would like to contribute but is a little timid about it due to little experience or anything else, I encourage you to jump on in! The devs are very friendly and have helped me a lot.

My first major contribution was adding additional skills to the hammer, axe, and bow. @Sam (combat lead) graciously wrote a guide for adding skills. With help from many of the other devs I managed to add two new character states to allow for charged melee attacks and ranged projectile bursts. This was my first time working on a project larger than a few files and was quite overwhelming initially.

Since then, I have had fun programming humanoid NPCs to attack with all of their newfound skills. We currently use a state machine to determine NPC behavior. The only NPC programmed to use multiple attacks when I added these skills was the stone golem, which uses a timer and proximity to target to determine attack. I began by implementing human attacks in this manner as well. When NPCs were first added to Veloren, they were designed to take the same inputs as player characters (as far as the code is concerned).

Minimizing the difference between players and NPCs is part of our attempt to heighten immersion in our simulated game world. Using an arbitrary timer works against this purpose and can make NPC attacks too regular and predictable. Since the humanoid NPCs use the same attacks as players with the same energy costs, I transitioned the conditions for firing the different attack skills to energy cost with a little bit of RNG to simulate stress-induced, split-second decision making.

Currently, humanoid NPC attacks tend to be a little over-aggressive and in some cases play better than most players with the same weapon. I intend to give NPCs varying skill and adeptness with weapons to make combat against AI feel more like PVP. Before working on more considerations with the AI code, I plan to transition the state machine to a behavior tree for better control in the new year (assuming no one else does it before I get to it).

I also got to work with @Slipped on some animal attacks. Currently, testing AI code to get it just right can be time-consuming with having to recompile after every change to see how it plays. I hope to modularize the code better with the switch to a behavior tree and perhaps move some tactic components into RON files.

After working with combat for a while, I ventured into the land of audio. I play several instruments IRL but had no experience with audio software or coding. The first step was to allow for biome specific music. The biome of each chunk is stored in the terrain metadata. Querying that information was fairly simple. I then worked on some rather convoluted biome transition code before we decided to let the songs play to their end.

Biomes, as defined in the code, are simply labels applied to chunks that meet certain temperature, altitude, humidity, etc. conditions. I tweaked these values a little to match the world gen. At the time, the soundscape of Veloren was being discussed. It was quite empty. Luckily for me, @zesterer had implemented a "blocks of interest" system that selects certain blocks from the loaded terrain to emit particles. These are used for falling leaves, beehives, fireflies, and fireplaces.

I repurposed the system to emit sound effects from blocks. The sound effects can be emitted at different times of day allowing for leaf blocks to emit bird calls during the day and owl hoots at night. @Daforlynx worked on and provided me with a wind loop. I created a non-spatial ambient sound manager and set it up to play the wind sound at varying volumes depending on the altitude and tree density. In the future, this will likely be tied to weather simulation.

As more people play Veloren, we hear all kinds of feedback. Many people asked for a passive health regen. Most of the core devs do not like the idea of a passive health regen for many reasons. To help the player situation out, I made campfires regen health when a character sits nearby. To accomplish this, I developed an aura system to apply buffs to nearby entities. While relatively bare-bones right now, this system will allow for things like poisonous gas vents, enchantments, and more.

Most recently, I have been working on a poise system to complement Veloren combat. In many old school RPGs, combat is little more than firing off different attacks while the character remains stationary. In Veloren, we hope to develop a more dynamic combat system that rewards player skill with regards to movement and weapon skill use. The addition of i-frames for melee attacks to the roll/dodge was a major first step. The poise system will govern stuns and take-downs. With this system weapons have a poise damage stat in addition to regular power. This makes some weapons better for stunning while having lower DPS allowing for more play styles.

Armor also will have a poise damage reduction stat. The exact balancing details are not sorted out yet, but the general idea is as follows. Character entities have a poise component similar to health. Different species have different base poise values. When poise is lowered below a predetermined fraction of the total (tentatively 50%) the entity enters a stunned state, interrupting their attack and affecting movement. The stunned entity's poise immediately resets to 100%. The further below the initial stun level, the larger the effects on the stunned entity.

I am still experimenting with knockback as well. This system motivates the attacker to save up heavy attacks for when the defender is most vulnerable and provides further impetus to avoid being hit. With the addition of @Sam's skill trees I believe we will be well on our way to a fun, competitive combat system.

Starting with Audio by @Badbbad

I found Veloren at a point when I had a lot of motivation to write music for open-world games but I couldn't find an outlet to fulfill it. Finding Veloren community amazed me, how so many talented and hard-working people can team up for a passion project so organized and well-directed. I was immediately hooked by the concept, downloaded the game, and played. 5 hours later I already was done writing the track "Between the Fairies" and sent it to #audio. And today I'm one of the team leads of the audio team.

Veloren really helped me learn to enjoy the process and cherish the progress step by step. It also showed me that people with passion and talent are a huge force, that can build an entire game only by volunteers! (that's crazy)

Currently, I'm making and implementing sound effects for a free-to-play game called "Tadpole Tales". You can check out some more of my work here. It's more than exciting to be a part of this community, and I'm looking forward for our future!

The walls of Gronk Keep must be fascinating

Network Analysis by @xMAC94x

Follow-up Performance Analysis from devblog-93

When I wrote this article it ------------------------- ------------------------- Language Rust ------------------------- voxygen/src/hud/slots.rs voxygen/src/hud/spell.rs voxygen/src/hud/minimap.rs voxygen/src/hud/social.rs voxygen/src/hud/esc_menu.rs voxygen/src/hud/overitem.rs voxygen/src/hud/overhead.rs voxygen/src/hud/group.rs voxygen/src/hud/map.rs voxygen/src/hud/util.rs voxygen/src/hud/chat.rs voxygen/src/hud/popup.rs voxygen/src/hud/item_imgs.rs voxygen/src/hud/skillbar.rs voxygen/src/hud/bag.rs voxygen/src/hud/crafting.rs voxygen/src/hud/buffs.rs voxygen/src/hud/img_ids.rs voxygen/src/hud/hotbar.rs voxygen/src/hud/buttons.rs voxygen/src/hud/settings_window.rs voxygen/src/hud/mod.rs ------------------------- Total -------------------------

Some of had the working title "how to make sure that the next release party can handle 100 players". During the investigation, I dived deep into the Veloren source code and found multiple regions that have difficulty scaling. The server only needs a single bad scaling component which is enough to slow down everything. I was even happier to actually see 100 people on our latest party. In its current state, none of the devs could dream of it scaling so well. Anyway, I owe you some physics and networking details.

How we Calculate Physics

Physics in this context describes handling positions, velocities, and hitboxes. We break it up into 2 parts: first, we do entity <> entity collisions. then we do terrain collisions and apply the movement.

Entity <> Entity Collisions

For this, we iterate over all entities. Technically, only those with position and velocity except for mounted entities and projectiles. For each of these entities, we check if it collides with another entity. We skip entities checking against themselves, other mountings, projectiles, beams, and shockwaves. Therefore we draw a cylinder hitbox around the entity. If we detect a collision we adjust the velocity so that our entity moves away from the collision.

Entity <> Terrain Collisions

For this, we iterate over all entities (technically only those with position and velocity except for mounted entities). We then apply movement parameters (gravity, friction, water physics) to the entity. Then we try to apply the resulting velocity in small steps, which doing collision checks with the terrain. Therefore we draw a rectangular hitbox around the entity.

I used tracy (see the previous devblog) to analyse the bottleneck of our physics coding. The more players are spread around the server, the more entities are loaded. The entity <> entity code scales very badly, as it needs to check each entity with each entity. For 1000 entities we have 1.000.000 checks. During load the server often has 3000 entities loaded, resulting in over 9.000.000 checks. In order to speed this process up we do a quick-check where we approximately some parameters, so quickly skip far apart entities, though this check is far from optimal. Additionally, we parallelize this calculation to profit from multiple cores.

This is our main bottleneck for < 30 players, it will draw resources and burn CPU time (which costs us server host money to keep the server tick time at 30 TPS).

Overview of new Network Backend

Our old code had many bottlenecks that got fixed within the last 2 months.

We used to actively wait for new players to connect, actually creating a "laggy" tick, that can take up to half a second. We only do this for the auth token check currently, but plan to even put this in a background task.
We send a big message on a single channel for all kinds of communication which got compressed always. We started to split this up in multiple channels, so we can process PING messages independently of CHUNK messages or GAMESTATE messages. Also we can apply compression algorithms more carefully and save some CPU time.
We no longer need to serialize the same message multiple times if we want to send it to multiple participants. This massively saves CPU time on the server.

In the future, we will apply separation into multiple channels even more carefully and optimize our ECS format, to reduce bandwidth usage by 75% profiting users with slow internet connections.

I would like to take this opportunity on this special 100th blog to thank all contributors, artists, players, developers, musicians, and translators for their time and dedication spend on Veloren. There are 1000 different faces, and each of you is doing your unique but essential part for this project!

The State of Graphics and UI by @Imbris

We have two major transitions in the crates (Rust libraries) being used for rendering and UI respectively. For rendering, we are switching from gfx to wgpu and for UI we are switching from conrod to iced. One of the shared motivations for these switches is to move to crates that are being more actively developed.

The old gfx crate is pretty much inactive since the GFX team moved on to developing the much lower level gfx-hal and wgpu which is closer to the old gfx crate's level of abstraction. While conrod is more mature than iced and is still maintained to an extent, updates are infrequent and there doesn't appear to be a significant driving force to fix the current missing features and rough edges.

Wgpu transition

Currently, we just use the gfx OpenGL backend. wgpu automatically supports multiple backends including more modern backends such as Vulkan, dx12, and metal. Hopefully, this should allow us to eke out more performance. The wgpu rewrite is currently fairly complete, mainly with shadow rendering needing to be fixed as well as interpolation of LoD texture sampling. There are, however, a few other issues to consider:

drawing into shadow cubemap textures
OpenGL support
Shaderc/SPIR-V

Shadow Cubemaps

We use cubemaps to render point light shadows. Previously, we could render to all the faces using a single draw call via a geometry shader that duplicated vertices and selected a separate face of the cube for each duplicate. However, wgpu doesn't provide support for geometry shaders so this isn't possible. To get around this we have to have a separate render pass for each face where we render all the terrain into that face. This works but potentially has an impact on performance.

OpenGL Support

Currently, we target OpenGL 3.3 with the goal that people on old machines should still be able to play Veloren. Unfortunately, OpenGL support from wgpu is very limited and only exists on Linux/MacOS due to complications with surface creation and I don't know what the minimum required OpenGL version would be and/or how much overhead there is to emulate the modern API of wgpu for older OpenGL versions. One thing that will help us is that there is also support for dx11 which should cover many of our Windows users on older machines without dx12.

Shaderc usage

wgpu consumes shaders compiled into SPIR-V. To get the SPIR-V shaders from our GLSL shaders we have to use shaderc which is a non-rust dependency that introduces additional friction to getting compilation of Veloren set up. A potential workaround for this that many projects use is to precompile the shaders to spirv and track those in git along with the GLSL source files. This will make shader hot-reloading more complicated as we can currently just change the shaders and save them and the game will recompile them automatically. A potential middle-ground would be to include a new feature in voxygen that enables the use of shaderc/hot-reloading which is off by default that graphics developers can enable as needed. We could even have it enabled in the distributed version of the game if there are no complications for compiling in the CI with shaderc.

Iced transition

As I think I wrote in a previous post, we have fully transitioned the main menu and character selection screen to using iced. All that remains is to covert the HUD however this contains much more UI code than both the menus combined.

> tokei -f voxygen/src/hud/ --------------------------------------------------------- --------------------------------------------------------- Files        Lines         Code     Comments       Blanks 22        14609        12811          994          804 ---------------------------------------------------------- 185          160            1           24 114           86            9           19 465          387           39           39 602          549           33           20 182          162            6           14 144          108           22           14 603          519           53           31 838          764           51           23 820          724           67           29 183          154            3           26 568          482           45           41 243          212           12           19 192          166           13           13 1020          905           94           21 930          848           57           25 514          465           30           19 515          466           23           26 402          309           40           53 118          102            4           12 408          388            8           12 2832         2486          167          179 2731         2369          217          145 ---------------------------------------------------------- 22        14609        12811          994          804 ---------------------------------------------------------- this code is reusable but most of it is conrod specific UI code. In order to break this down into surmountable bytes, we will need to have a way to incrementally transition the hud code. The first step is implementing a custom iced widget that can manage a list of windows to allow easily laying out a HUD like UI. This can then be drawn alongside the conrod hud. Then we can reimplement each major portion of the hud as one of these windows (e.g. the map screen, the chat, the skill bar, etc).One other thing we will need to keep an eye on is the performance of text widgets. Unlike iced, conrod requires an ID for every widget. It is nice to not have manage IDs when writing iced UIs; however, in our conrod rendering backend, IDs are used for caching text rendering information for efficient lookup. conrod itself also keeps some state associated with each text widget which allows it to identify when the text is changed or when previous computations can just be reused.
There is a proposal for iced retain information about the widget tree between frames which could be helpful for this issue. We could also create a custom widget that grants IDs for text either managed by the user like in conrod or automatically derived (e.g. using #[track_caller) or perhaps we will find some other way to efficiently cache and retrieve this state.
Fontdue
On the topic of text there is another crate that I have been considering transitioning to at some point. fontdue is the fastest text renderer. Currently, we use glyph_brush which currently uses the ab_glyph rasterizer crate (a rewrite from the developer of rusttype which glyph_brush used to use). While fontdue is much faster than other rasterizers, glyph_brush provides a glyph and layout cache which we would need to find or implement to be able to use fontdue.
Skill Trees by @Sam
For the past month and a half, I have been working on skill trees, and they are finally beginning to near completion. Skill trees will completely replace the current leveling system where you get a few points of max health every level. In my branch, I will be adding a general combat skill tree, as well as a skill tree for each of the six weapon kinds. Each skill tree will have its own experience bar and skill points (which can be considered somewhat equivalent to levels).
Skills will naturally be purchased with skill points, and the amount of experience required to get a skill point will always be constant and will not scale with the number of skill points earned. Currently, skill trees for each of the six weapons are done, support for prerequisite skills is done, leveled skills are done, and database persistence for skills is done. The general combat skill tree, UI for skills, functionality for refunding skills, balance testing the skill trees, and creating database migration templates still need to be done before skill trees can be merged.
Procedural Trees by @CCGauche
I wanted to add procedurally generated trees to Veloren since the current ones have to be created manually. I’ve started iterating on tree designs. The main problem while developing was Veloren's compile times. It took about 2 minutes for each change, plus 1 minute to get in-game. To solve this, I designed a wrapper for Rust and Java Minecraft plugin and used a custom Minecraft server to lower this time to around 3 seconds. This helped speed up prototyping a lot. The first ones were T shaped as you can see:
I decided to work on a leaf growth algorithm and obtained a very symmetric trees. This first algorithm was simulating realistic leaf growth with variables like sun exposure, position, and precipitation:
Then @zesterer came to help and together we designed a new branch generation algorithm! After lots of work, iterations, and optimizations we were able to generate very unique, good looking, and fun to climb trees! The leaf algorithm has been updated to match performance constraints. We removed all the original variables and instead added a health parameter that is different for each branch and based on the soil and sun exposure. I also revised the branch generation algorithm to use vector projections that reduced block updates and managed to give a 5000% improvement.
There is still a lot of optimizations to do so the game can generate trees at chunk load. @zesterer and I are working on interpolation to reduce the number of computations for leaves gen. You can’t see it on the graph but on the 5th iteration, we are at 7ms for branches and 130ms for leaves.
Special thanks:
@zesterer | Help for branch algorithm and optimization ideas
@Sharp, @zesterer | Code reviewing
@Pfau | For the general tree design
@Pfau, @Gemu, @Treeco, @Slipped, @James, @Sarra_Kitty, @Scott, @XVar, and @Snowram | For the tree design corrections
New Website Design by @Songtronix
We can't hide the fact that our website isn't the prettiest one. It works perfectly fine but also misses a lot of links, information, and content. So I've been prototyping a new website you can check out here or just take a look here:
It is pretty bare-bones right now, but there will be things added like a feature section (combat, vision, general gameplay aspects and what makes Veloren special) and a lot more. Suggestions are welcome! Furthermore it's fully responsive too. And as a side note, the green download buttons are not Linux exclusive don't worry. They adjust based on your OS automatically.
The layout is pretty good although I'm still tinkering around to maybe match the feeling of my website more as I like the background and look of it. Let me know what you think!
Now onto other work which I haven't mentioned that much in the blogs:
A while ago I wrote a Discord bot that allows you to control the Veloren server it manages (which I'm able to pay for with the help of my patrons - thank you!). You can adjust/view all settings including downloading the database, switch/update a branch of your choosing as long as it's on the main repository and view the logs live! As it seems it greatly improved the speed of development or rather testing of new features so it serves its purpose well.
As it's the 100th edition of the blog a lot of time has passed! When I started on Veloren, I was a Rust beginner and I hadn't worked on anything open-source related at all. Now I'm the UX working group lead, I've revamped the book, redesign the website, created a test server with a Discord bot, wrote Airshipper, and I've learned so much during that time and continue to do so!
Why Veloren is Special (to me) by @Christof
Watching this year’s news covering more and more Rust I decided to take a look into this new language, to augment two decades of professional C++ experience. So I started following the weekly Rust newsletter, which brought me to "This Month in Rust Game Dev" where Veloren showed up with beautiful screenshots. After reading some chapters of the Rust book, I then looked for an open-source project to gain practical experience and remembered this promising project.
The sheer size of Veloren’s codebase was intimidating at first, so I was more than happy to get a shorter tour around the concepts by reading the few hundred lines of teloren source code. This let me quickly become familiar with the tooling and enabled me to add some small patches to teloren. From there on I got more confident to not break anything when modifying Veloren’s code. Of course, I initially stayed more with databases which I already had gathered experience in beforehand.
Up to now, I have seen a significant portion of the code and mostly feel at home with it. To me the codebase is clean and most times easy to understand, with the exception of the conrod UI. Dwelling into the code taught me a lot of Rust idioms, of course most in-depth when modifying the code and from the valuable feedback on merge requests. Also, I found that Rust indeed makes it easy to change unknown code; once it compiles the most basic problems are solved. From my experience, a C++ project with that many contributions by non-core developers would suffer from significant quality issues.
Most of the complex code is yet less visible to players (procedural world generation, erosion, economic simulations) but form a good basis for future in game features. And I fully agree that Veloren is one of the most beautiful open source games around.
The community has always felt welcoming to me and I still haven’t figured out how the main developers manage to read and even answer the huge number of messages in the Discord server. Just judging from the number of participants online (at times more than 2000), I see that Veloren is a very active project. I can tell that contributing artwork, e.g. voxel models, is a low bar and a very satisfying experience. The project team and players are really persistent at celebrating beginners' contributions and thus I have high hopes that the next year in Veloren will be as bright as what I have seen from this one.
Getting into RTSim by @Ubruntu
Congrats to the Veloren community on the big 💯
I started playing Veloren about 4 months ago, around the 0.7 launch. It already feels like a completely different game today than it did when I started my first character. With some big branches in active development, the game will feel completely different again soon. I really like that Veloren is under constant development and there’s so much active discussion around it.
I’m happy to see Veloren getting more popular, and also getting more and more talented contributors. As someone who contributes code, I like that Rust was the chosen language for the project. Rust is my favorite programming language and I find Veloren to be a very approachable large Rust codebase. Rust has amazing tooling and offers the compile-time checks so that new changes I make likely won’t introduce catastrophic bugs that cause players to set the game aside and wait until a patch is released.
Speaking of contributing code, I’ve recently been experimenting with the real time simulation (RTSim) code. It started with the idea that the game has travelers, and the game has birds that can fly, so why not turn some of the travelers into birds and make them fly? I posted some screenshots of birds in the air and got some positive feedback so I kept working at making this a feature in the game. It also sparked some discussion around the RTSim implementation in general and how we can move it forward as a core part of Veloren.
The 0.8 release was a lot of fun. It was exciting to see over 100 players online, and to form a dungeon party with about 15 of them to raid one of the highest difficulty dungeons. Afterwards there were some discussions around issues that only MMORPGs usually deal with. If some people consider Veloren an MMO now then that’s great news to me, because I personally love the dungeon party experience of MMOs.
Happy 100th blog post Veloren. I’m excited to see what’s to come in 2021.
Finding Veloren by @Yusdacra
I found Veloren while I was searching for projects that use Rust about 1.5 years ago. I wanted to see if the language was mature enough for me to pick it up, as I wanted something that could potentially replace C# and C++ for my use cases. I was pretty surprised to see such a video game project existed, and it was build on Rust, which was the perfect candidate for me to see how the language was practical. The community was very welcoming and I got up to speed to do my first contributions, most of which were low hanging fruits fit for a beginner to the codebase.
As time passed, I moved onto my own projects, but I'm still maintaining the Nix expressions and Turkish translation. Compared to 1.5 years ago, I now feel pretty confident with my Rust code, it was not too hard to get into, but also provided enough challenges, while also being fun to write. And it's all thanks to Veloren and it's community!
New Developer Experience by @Entropy
Honestly, I started working on Veloren with literally no programming knowledge whatsoever, so when someone said "this is an easy fix but we’re all too busy!" I took the opportunity to just... go for it I guess. It’s been definitely a new experience, but going from "I looked at the code and it might as well be Latin" to "I actually kinda understand what’s going on here" has been real satisfying and encouraging.
Looking to the Graphical Future by @Sharp
Recently I've been working on the wgpu branch. My most recent change was switching to reversed depth buffers which lets us theoretically avoid all clipping while extending views far enough to see for hundreds of kilometers with better quality than we have now.
The biggest system I've worked on is still initial worldgen (erosion). The project is far from finished, but being able to use realistic simulation techniques to produce rivers, mountains, hills, and valleys that actually feel real and don't just feel like noise has been incredibly rewarding. However, I've also touched a lot of the graphics stack, in particular in helping add level of detail (LoD) and shadows to the world, which really helps make its scale look visually breathtaking.
I didn't know much of anything about graphics when I started working on Veloren, but doing more with it has given me a real appreciation for the math behind it as well as how shaders work and how to write code that's efficient for the GPU.
I'm really looking forward to Veloren continuing to become more immersive in many different ways. For worldgen to improve further, especially in terms of large scale structure (i.e. mountains no longer randomly popping up everywhere using tectonic and glacier simulations, much larger worlds), performance improvements (many are now available to us after switching to wgpu, including potentially thousands of dynamic lights with deferred rendering), and more. I will especially be happy when upcoming work simulating economics, NPC history, and human structures start to become better integrated into the game, together with graphical improvements to things like LoD that reveal more of the world.
Combat System Design by @Silentium
Veloren’s combat system is a combination of dedicated programmers and many hours worth of game design meetings. It’s not finished, and it won’t be for some time, but it’s a fantastic start. The Game Design group has seen many people come and go, but the current team is very capable. The core mechanics for our combat prototype is looking extremely promising, and I’m so excited to see the rest of our vision come to life. I have so many people to thank for contributing to the design and development work that has gone into the current Veloren combat system.
Cheers, -Silentium
Watching the full moon. See you next week!