Classified: Connectivity

Whether you're jumping into Arcade or getting your squad together for a Ranked session, the one thing that all players need is a reliable online experience. In this blog, we'll shed some light on connectivity in 2023, look at what we've achieved, analyze any unplanned downtime, and take a peek at what the Connectivity team is working on.

A YEAR IN REVIEW

Firstly, let's take a closer look at some of the changes brought to the game over the past year that had directly impacted Connectivity, so that we can deep dive into some of the solutions put in place to improve the overall player experience.

UNPLANNED MAJOR DOWNTIME  

Downtime is likely to be the biggest concern for most players when it comes to connectivity. Downtime of a game's online services can have a wide array of causes, many of which are not linked to the game's servers themselves. Issues related to new features included as part of an update or the process and logistics behind patch deployment are just a couple of examples of triggers that could result in an online services outage. While our Connectivity team works to ensure our servers are ready for players at all times, we have multiple teams who are making improvements on their side too, with an aim to minimize these extra trigger sources as much as possible.

Below you will see the total unplanned major downtime for each month this year so far. We can see that while the most recent months of September and October have been a relative success, there are three main spikes throughout the year prior to this. Let’s inspect those spikes, why they occurred and how our teams responded.

JANUARY

January saw 15 hours and 54 minutes of unplanned downtime with a number of unique causes being identified, part of which was related to the release of Crossplay. The Connectivity team worked through both December and January bringing stability to the game and establishing areas where revisions or improvements were required. Each instance of downtime was also independently investigated by our teams, with steps taken on how to avoid or mitigate the effects of similar issues in the future. Changes made as a result of the investigations have since helped to reduce impacts from similar events.

APRIL

In April, we had 7 hours of unplanned outages across three separate incidents all again having a unique cause. In all cases, the Connectivity team was able to detect the point of failure which was causing players to be unable to connect, and was then able to make immediate or planned changes to our systems and procedures to remedy the issues going forward.

AUGUST

Y8S3 Operation Heavy Mettle was released on August 29^th, and with this we encountered a major outage with overall downtime totaling 17 hours and 49 minutes. In essence, this outage was caused by a specific group of conditions arising, which led to our services being unable to handle the high amount of players attempting to connect at season launch.

To alleviate the issue, the team disabled systems that would deliver Rank information to players, while still allowing Rank calculations to take place in the background. This allowed players to connect, matchmake, and complete meaningful Ranked matches while the team continued their work to get the game services fully operational once more. The team was then able to reinstate the delivery of Rank information to the game once conditions were stable.

In response to this situation, teams were able to make both specific and systemic changes to help ensure that the same outcome would not be repeated under similar circumstances.

DDOS (Game server)

When talking about game server DDOS, we're looking at DDOS that directly affects players as well as a group of matches hosted on the targeted game server. The team have previously implemented a feature to reduce the number of matches affected by these attacks, resulting in less players being ejected from matches. 

Although the team has made progress in this area this year, we know that players still experience some issues with this. We see your feedback; we want to share that our work continues in this area and that the team will be making further changes in the future to help improve this situation.

SERVER REGIONS & PING TESTING

When a player boots up Siege, it checks to find the server which will provide the lowest ping based on the player's current location. This functionality has allowed us to run live tests using new data centers, while gathering performance data. These tests are helping us to understand how activating new locations can not only improve connectivity for certain communities, but also impact the experience of all the other players.

During 2023, after such tests had been completed, the Connectivity team made two new server regions available (one for Korea and another one from Europe Central). We've found that these new locations have been a boost for many players, enabling them to play with decreased pings leading to an improved experience overall.

SERVER REGIONS - WHAT'S NEXT?

We've got nothing final to announce regarding additional new server locations as of right now, though we can confirm that it's an area the Connectivity team is putting a lot of resources into. In the coming seasons, the team will be running new internal tests, and we'll share more updates on this as soon as results become available.

In addition to testing new server locations, the Connectivity team is also working on updating and evolving its tooling. This will allow the team to pull more detailed information regarding their usage and ultimately facilitate decision-making when it comes to deciding which regions could benefit from additional resources the most.

CROSSPLAY

Crossplay was implemented back in December and has been one of the biggest online changes to come to Siege. The feature has been a great success, though it's important to highlight just how this affected our online services in the weeks following release. During this time, we did experience issues leading to downtime which ultimately stemmed from the Crossplay feature. These issues were due to the challenges of deploying such a game-changing feature, while ensuring that all of our systems were ready to provide a stable experience with this additional load. Crucially however, the team was able to ensure that the feature remained live, while bringing stability to the game's online services, providing a highly awaited change that many players had been asking for.

TECHNOLOGY REPLACEMENT

Over the last few seasons, our teams have been working iteratively to replace older technology which is integral to our online services. Some of this technology has been in place since Siege's early development, and as the game continues to evolve, there is a need to ensure that we still have state-of-the-art systems in place. Once complete, our team and players alike will be looking to gain multiple advantages, including decreased maintenance downtime. 

COLLABORATION

At Ubisoft we have a wide range of titles, each with their own Programming, Art, and of course, Connectivity teams. In the last year, our Connectivity team has found new ways to collaborate with such teams from other titles, including For Honor and Roller Champions. Since the new efforts were established, the data being shared has enabled us to mitigate online issues that might have otherwise affected players and has resulted in a significant positive impact overall.

LOOKING FORWARD

The team has made multiple learnings over past seasons and applied a number of changes while working on others which will be implemented in the future. While some of these changes only represent remedies for a fraction of causes, there is no doubt that the steps taken will be beneficial moving forward. 

To sum-up, here's a look at some of the areas the Connectivity team is currently working on:

• Putting strategies in place to minimize unplanned downtime.
• Reducing the impact of game server DDOS.
• Tightening collaboration with other titles including For Honor and Roller Champions.
• Testing for possible new data centers.
• Removal and replacement of old technologies used throughout the game's life.
• And much more behind the scenes.

This is just a brief overview of what the Siege Connectivity team has been up to, we hope that this article provides useful insight into some of the topics that are being looked at. We want to add that this is only a snippet of what this team is working on, and we'll share more on this topic in the future.

We thank you for providing feedback on your online experience as it's extremely helpful to the team and we couldn't make improvements without you!