Solved

[DEV] Races not finishing

There have been several races today which ran but did not finish or post results.

Because this issue only really started yesterday, I'm applying an attempted fix based on my best hypothesis. I believe that it is caused when cars retire in the race from running out of fuel, potentially only if they retire in the pit lane itself, not out on track. Changes were made to car behaviour in the pit lane yesterday, which is why I suspect this is the cause. This exact combination may cause a specific kind of exception where the car is never retired and the race never finishes. I'm currently working on a patch for this which should be live by the end of the night, but it is not definitively the cause. The update tonight will also restore something like the old behaviour where cars can roll around the track at reduced speed when out of fuel.

If you were in a race that hasn't ended and have any data on it, let me know if it matches this pattern or not. What makes this tricky is because the races never finish, there is no data to analyse, essentially, I can't see what happened in the races! Or, the only ways to gather data will be too slow to get meaningful insights. That's why I have to operate on an educated guess for now.

The issue definitely seems related to pit stops and refuelling. I am looking into a more thorough patch now.

I managed to join one race where the issue was present and found a retired car in the pit lane.

I've moved all races to two new servers that I've applied an attempted fix for this on. It looks very much related to the pit lane, and as I mentioned today's sim change did modify something there. Fingers crossed this puts an end to these issues and we can focus properly on the update again.

Early signs look very promising from the changes applied to the sims now. Massive increase in stability across the servers running the service now. The races rnuning from 22:00 all have the fix.

I'll keep you updated on how it goes.

More servers are joining the service with the fix now. I've restored 3/4 of our server capacity. The service for the last few days has been running on a single server for most of the time, which was the source of all the reports of lag. This should also be significantly improved now.

If you could see what causes these problems, you would laugh. The last one was a commented out line that basically said "leavePits" buried in thousands of lines of code.

I'm working on one remaining issue: ALL reports of races not working are now on Spain (100% in the last few hours). I've tested it and confirmed the track is the problem. I'll try to get a new version rolled out to the servers ASAP. Otherwise, the service now seems stable on the simulator side.

We're back at full server capacity. All servers are live with the fixes. Now I just need to figure out what's wrong with Spain.

UPDATE: Spain has been fixed. The update is live with immediate effect.

After this, I might regenerate all tracks just to be sure, because I addressed a few issues with track generation.

Also, more code on pit stops was reverted after another report of something weird in the pit lane. Hopefully these fixes combined address the remaining simulation issues and I can refocus on everything else tomorrow.