Official

Cloud infrastructure rollout

Hi all,
We are currently in the process of rolling out a new cloud-based infrastructure for races, much like the one that runs the websites. 

The aim of these changes is to improve the latency / ping and reliability of the service overall under high load, for things like boost, real-time strategy etc. This has been a priority task ever since the update and our main focus outside of developing the game. We heard your feedback about the viewer being slow to respond and made some improvements up to this point, but this should be a more permanent solution.

Please group all feedback about issues connecting to race viewers here with race IDs and any information you can provide.

I have already encountered one problem where I could connect on WebGL / desktop but not Android to a race, and am investigating that now. If any issues are persistent or hard to resolve, we will simply revert to using the older system until it is resolved. Both are currently running in parallel, so some races will be on the old system and some on the new cloud system.

UPDATE 16:21 GMT: The issue I was experiencing in Android has been resolved by updating some firewall rules in the system. We are now not aware of any further issues and will await reports.

UPDATE 17:00 GMT: An incident involving a few races between 15:00 and 16:00 has been logged whereby a some races ran in parallel in the cloud and on the old system. Results were posted from the "wrong race" and did not resemble what users saw in the viewer. The good news: these cases can be easily resolved by re-parsing the results from the "right" server, which we can do manually. A thread of reports for this issue can be found here: https://igpmanager.com/forum-thread_1/61790

All races in the service are now running exclusively in the new cloud infrastructure. We are prepared to revert to the old system at any moment if issues occur.

We've identified a configuration issue with the cloud servers which will be corrected now. This impacted the race simulation and was the cause of reports of 350kph top speeds etc.

In the meantime we've reverted to the old system while we cycle the configuration for the cloud servers to ensure it shares identical settings to the old system. I've traced the root cause to a configuration setting that server management transferred from a development server when it should have been from a production server. It impacted settings for the race simulatoin slightly. Apologies for any confusion this may have caused, the simulations should be as normal from this point onwards with 370kph top speeds.

We're back in the cloud infrastructure again, with the updates.

The old system has been rotated out of use again.

We've just tripled our cloud server capacity in preparation for a marketing push in the weeks ahead. Lots of new servers in the service, and we hope latency and pings will be noticeably improved from tonight.

The old system has been retired from use.

UPDATE 00:08 GMT: Some servers were found to not be pulling the correct weather data earlier in the day. We believe this to be resolved now and have restored all cloud servers to the service and will continue monitoring their performance.