Getting Rid of Lagggg, Part 3By Victor Zuylen on 15/Jul/2014 17:41:13
In part 3 of our ongoing series, Lead Game Tech Jorrit Rouwe walks us through the results of Guerrilla’s efforts to reduce lag in Killzone Shadow Fall – what proved effective, what didn’t, and what warrants further investigation.
Since my last post, we’ve been focusing on reducing the lag even further. We’ve been doing this by exploring various new server regions and seeing the effect. There are two important questions that need to be answered when adding a new server region:
- 1. Does it give a significantly lower ping for players?
- 2. Are there enough players nearby to consistently offer a reasonable amount of full games?
Because data on the internet does not follow a straight line from but is routed through internet highways (the ‘backbone’) and because there are many companies involved and each company has to negotiate contracts with the other companies, it is very hard to predict the path that data is going to take. Quite often the data takes an incredible detour, so what may seem close on the map is worlds apart on the internet.
These 2 points together make it really hard to choose the right location. Here is what we tried.
The first location we tried out (from May 28th through June 5th) was Istanbul. We selected that location because we saw that players in the Middle East have a high ping to both Singapore and London. We had a hard time finding a reliable hosting partner and in the end settled for Istanbul. We hoped that Istanbul would serve both the Middle East and South East Europe. The results were a bit different than we expected…
Figure 1. Per country the amount of players that are closest to a particular server (= the server with lowest ping). This data was collected when the Istanbul server was live.
Our second experiment started on July 9th. We installed a server in Dallas to try to lower ping to Central USA and Mexico.
Figure 2. Closest server per state. Data collected after July 9th.
In Figure 2 you can see the results of the Dallas server. You can see that players from Central USA mostly connect to this new server. We can also measure that the ping for these players has dropped by 30-40 ms on average. This is a very good result and we’ll keep the servers around provided there are enough players on them!
The third experiment was a server in Milan which also started on July 9th and ended on July 16th. In Figure 3 you can see the results of this experiment. It turns out that quite a large portion of Europe is closer to Milan than to London in terms of ping. The difference in ping between those locations, however, is quite low (only 5-10 ms). During the test week we had 3 mass disconnects (many players receiving a network error at the same time) and we measured that the amount of laggy kills (a kill taking more than 0.5 second to register) in Milan was twice as much as any of our other server regions. So we decided that the quality of the data center was not good enough and took down the servers.
Figure 3. Which players are closest to our Milan servers.
As the amount of players from South East Asia dropped, it became increasingly difficult to provide full games for them. Because of this we had to take the hard decision to turn off the servers in Singapore. Although this does increase lag for players in that region, it will allow them to find more full games.
Figure 4. The players from South East Asia are mostly playing on the Tokyo servers now.
The future: Frankfurt
While doing the experiments in Istanbul and Milan we found out that quite a lot of data for our players is being routed through Frankfurt. So we’re going to set up an experiment in Frankfurt in the near future.
Have you noticed improvements or reductions in connection quality in Killzone Shadow Fall multiplayer lately? Let us know in the comments below!
Getting Rid of Lagggg, Part 3By Victor Zuylen
Getting Rid of Network ErrorsBy Jorrit Rouwe
Sound Design Behind The ScenesBy Victor Zuylen
Shadow Fall Music Production BTSBy Victor Zuylen
VSA Special ForcesBy Bart Suichies
Stun DroneBy Bart Suichies
Tutorial VideoBy Victor Zuylen
Helghast SecurityBy Dan Calvert
Support ClassBy Victor Zuylen
Buddy DroneBy Dan Calvert