News

By: Jarno Kokko May 09, 2007

European LOTRO login woes continue

For the second week running, The Lord of the Rings Online European servers managed by Codemasters have been rejecting logons. Problems seem to be concentrated to European peak times; sometimes you immediately get a cryptic error message, sometimes you are told you do not have a valid subscription(!) and rest of the time the game just hangs at "Logging in...". With some luck, you might be able to get in after waiting for up to 30 minutes, while most of the time you error out sooner or later. All this points to a capacity issue.

Codemasters issued a statement on the problem yesterday.

The problem is with our Billing system and specifically the integration code between LOTRO and this system which is unable to process fast enough at high volumes and then starts back loading requests resulting in a huge numbers of retries.

A number of changes to the systems have been made to try and help with this, and additional servers are already in place, but the problem is not one with server hardware currently. We have already doubled the number of login and integration servers in recent days.

The main issue occurs when over-whelming demand and load hits the login servers (usually at peak times) and creates a bottleneck in the authentication process. The knock on effect to this is the system becomes very slow and to all intents and purposes to you, the players, it looks like it is completely offline.

When this happens, people *are* able to login, albeit very slowly. As more of you refresh and try to login, each new request you send through actually compounds the situation making it worse, hence why we asked that you wait some 20 minutes between each re-attempt.

It actually works out faster for you to get in-game in the long run if you try to login fewer times.

It’s important to emphasise that this issue is a code performance issue associated with the integration between the game and billing system and not a hardware issue. Unfortunately adding new servers (which we have already done in great volume), will not fix this particular problem. Therefore we are currently working to resolve the code issue that is affecting us.

All of these systems were intensively tested during our BETA stress tests with thousands of players simultaneously accessing the service, however we did not see people on the scale that we are seeing them today. The immense popularity of the game has brought numbers to the game that cause these issues to now manifest themselves and affect us.

It is also important to point out that the previous issues experienced last week have been completely resolved and the issues experienced recently are new and as described above to do with the code integration between the game and billing system and not a hardware issue.

(emphasis ours)

Based on the statement, it would appear that during last week's similar issues, their billing and authentication servers died under the load. Once they got that fixed - by tossing in more hardware - the increasing demand then uncovered a programming bottleneck that can't be solved by just adding more hardware. Maybe they should have tested their systems with a bit more than "thousands" of players?

On the upside, it can't be that bad in the long run if your game is getting so popular that it uncovers whole new issues with your "intensively tested" backend systems. Hopefully they can fix the issues before the issues fix themselves - the playerbase is not very happy when it's impossible to, you know, get into the game at all during peak hours almost every night.

Codemasters is working on a plan to compensate the players in some way for these service disruptions, and has promised more details on that next week. Hot question on that subject is, how do you compensate someone who has paid a single fee for a lifetime subscription? You can't exactly add free days to that, so it'll be interesting to see how Codemasters solves this completely new puzzle as LOTRO is the first major online game that offers this subscription model as an option.

It should be noted that the problems probably do not involve actual LOTRO game code as Turbine's US servers have been running perfectly fine during all this, and the game has been just as popular on both sides of the big pond.

Please head over to our discussion board to talk about this news (registration not required).