FM 2009 - was it tested thoroughly enough?

[Wakers]

if it was as simple as that then presumably SI wouldn't need a structured testing process to find it. which can only mean your sayoing they noticed it but ignored it.

exactly - ive heard from testers who've said theyre infuriated that big problems are reported but not fixed!

[GillsMan]

He [GillsMan] didn't define any glaring bugs, he made a generalisation.

Knowing how they test won't help you, it's as effective as that method can be and isn't going to change in procedural terms, because the only perfect way is to code lock months before release and test and fix for months/years from then till release, can't happen/won't happen.

This is true - but also deliberate. I also mentioned in my OP the frankly billions of variables on user machines and, while I didn't explicitly state it, I'd hope it was obvious that I was referring to the difference between testing a game like FM and testing a software for release to a company which have a broadly similar computer spectrum across their users.

I hadn't realised the testers only tested parts of the game, but I can understand the reasons why. That was kind of the point of this thread - to get more detailed information about the testing process. As a part time tester myself, I was interested in the process - hope that makes sense!

[kccircle]

What do the game testers do?

They are the ones at fault for the issues on this demo.

Granted SI make the game but they put it in the hands of people (both in and out of house) to test it prior to release.

Did no one spot the injuries issue?

It has spoilt my demo for sure!

[Abs_UK]

It could be possible that SI made a last min inclusion of something which has caused the surface of the Injury Bug...

A single change in a code could triger some bugs..

Rest assured SI do test their games..

[kccircle]

It could be possible that SI made a last min inclusion of something which has caused the surface of the Injury Bug...

A single change in a code could triger some bugs..

Rest assured SI do test their games..

Do you become that complacent and change something so close to release that could potentially alienate your audience?

[Abs_UK]

Maybe a last min fix of some sort??

I dunno i am just trying to find an excuse for SOMEONE missing that obvious bug..

[wardog]

Do you become that complacent and change something so close to release that could potentially alienate your audience?

I was upset at this bug too.

However the change of code may have been causing a far more serious bug. I cant afford 09 anyway(at the moment) so wont play too much of the demo but at the end of the day i will play FM09 and i shall assure you the bug will be fixed by some point in time

[redlion1986]

Do you become that complacent and change something so close to release that could potentially alienate your audience?

Changes are made continuously up to a game going gold and then beyond in the form of patches. And the complexity of games these days mean a small change on some unrelated bit of code could cause an adverse effect in another part of the code. Suffice it to say that they wouldn't purposefully make a bug appear nor would they change some bit of code for a trivial reason.

[T4RG4]

QA teams almost always find a ton of bugs, they dont have the authority to force the development team to fix anything. That is up to the SI devs/management and SEGA.

[kccircle]

Changes are made continuously up to a game going gold and then beyond in the form of patches. And the complexity of games these days mean a small change on some unrelated bit of code could cause an adverse effect in another part of the code. Suffice it to say that they wouldn't purposefully make a bug appear nor would they change some bit of code for a trivial reason.

But they released a demo which has gone gold of a game that is not ready

[mije1983]

I wouldn't be suprised if someone got into serious trouble for missing this. But I hardly think SI did it on purpose. Wouldn't exactly be the best business plan!

But your final line brings up a good point. It's the demo. I would much rather it happened now so we can have a patch for release day, when the game will be fully playable, than it only surface when the game is released. Although obviously it would be better not to happen at all

[Abs_UK]

But they released a demo which has gone gold of a game that is not ready

The game is not unplayable.. there is no show stopper bug in my opinion, but SI have announced a patch will be ready on lunch day.

[kccircle]

QA teams almost always find a ton of bugs, they dont have the authority to force the development team to fix anything. That is up to the SI devs/management and SEGA.

SEGA?

No chance.

[kccircle]

The game is not unplayable.. there is no show stopper bug in my opinion, but SI have announced a patch will be ready on lunch day.

I find the demo unplayable. I have no interest in playing it. Never been the case before and having played since CM2 it worries me

[Just-Wool]

It's the same old roundabout every year.

Tralalalalala lalalalalala.

[kccircle]

It's the same old roundabout every year.

Tralalalalala lalalalalala.

Expand on that?

[Abs_UK]

Actually if i remember correctly i think that CM4 was even worse then this...

[Just-Wool]

It's the same old boring argument someone comes out with about SI testers not doing a good job if the game isn't perfect on release, it gets tiresome.

They test it and test it and test it. They can't test it on the same hardware as everyone has, nor can they find everything.

Any problems that happen now will be fixed by patches, the same as they were last year, and the year before that, the year before that too. In fact, every year possible it has been patched and refined to clear as many bugs and errors as possible. Even CM4 was pretty much playable after the second patch. Maybe they come quickly, maybe in February. Hardly worth worrying about though.

[Gregs_Gooners]

I'm in November in my game now and I dont have any major injury prolems, yes at the beginning of the game I had about 8 players out but it seems to have smoothed out now

[kccircle]

It's the same old boring argument someone comes out with about SI testers not doing a good job if the game isn't perfect on release, it gets tiresome.

They test it and test it and test it. They can't test it on the same hardware as everyone has, nor can they find everything.

Thats just something you have read.

If I had had the game for half an hour I would have spotted the injury issue.

It's quite poor to spout the rubbish you posted. Mine might not be original but yours is blatantly post increasing!

[T4RG4]

SEGA?

No chance.

I dont understand - are you saying they are not interested in it or are you claiming they are too rubbish to carry out part of their role as publisher? Maybe you work within SEGA QA?

[GillsMan]

Quite a few interesting posts in here:

http://community.sigames.com/showthread.php?t=56930

Should answer many of the questions - especially the posts from Kris which are quite insightful.

[Zvonimir Boban]

Thats just something you have read.

If I had had the game for half an hour I would have spotted the injury issue.

It's quite poor to spout the rubbish you posted. Mine might not be original but yours is blatantly post increasing!

agreed 100%

[Just-Wool]

Thats just something you have read.

If I had had the game for half an hour I would have spotted the injury issue.

It's quite poor to spout the rubbish you posted. Mine might not be original but yours is blatantly post increasing!

Haha, mate, I have been here for 6 years, and only made 2,100 posts, do I really look the sort who just posts to increase his post count?

At the end of the day, the injury problem was there last year too, it got fixed after the first patch.

I have not had an injury problem on my game, neither has someone else who has posted on this very same thread.

Could it be possible that the testers just didn't have the "problem" you claim to have? Or could it be possible the testers saw some injuries and thought that they weren't a problem.

Considering you started a thread to complain about people not testing the game properly, despite this being the same as previous years I find it a bit rich you accuse me of upping my post count!

[woody2goody]

The game is not unplayable.. there is no show stopper bug in my opinion, but SI have announced a patch will be ready on lunch day.

Mmm... lunch... Sorry mate I couldn't resist

Seriously though I think it's ok and the injuries aren't too bad in my opinion.

[matthiele]

last years bugs were very bad and did make the game hard to play at the start. however to suggest that one bug (injuries) is a gamestopper is just silly. get over it. it will be fixed on release day and although annoying does not stop me enjoying the game. maybe it stops some people but i think those people need to take a few deep breaths and relax. fm09 is a good game. ignore the injury bug and you might end up liking it.

[charlo116]

Haha, mate, I have been here for 6 years, and only made 2,100 posts, do I really look the sort who just posts to increase his post count?

At the end of the day, the injury problem was there last year too, it got fixed after the first patch.

I have not had an injury problem on my game, neither has someone else who has posted on this very same thread.

Could it be possible that the testers just didn't have the "problem" you claim to have? Or could it be possible the testers saw some injuries and thought that they weren't a problem.

Considering you started a thread to complain about people not testing the game properly, despite this being the same as previous years I find it a bit rich you accuse me of upping my post count!

You tell him!

If you all spent less time complaining and more time relaxing you might actually live longer.

[PompeY2J]

Perhaps the poor guys at SI had to finalise the demo before they wanted to?

And seriously, no game, or program, EVER has been released bug-free. As it is, this is only a demo, this gives SI a chance to fix the problem, so they can have a patch ready when the game is released. Be patient.

[PaulC]

Regarding the injuries - the main problem is in pre-season.......players are probably starting the pre-season on slightly too low condition and that combined with low match fitness ( which affects stamina ) sees a lot of players getting injured in pre-season and early season.

The answer is to manage your players condition more carefully especially in pre-season. Once the patch comes around things will be a little less harsh in this respect and things should balance out better.

What we wont do is simply cut injuries down so teams can perform unrealistically well with small squads.

[Nomis07]

Does PaulC get a million dollars now?

[playmaker]

I had a feeling halfway through pre-season that you couldn't do what you did in FM08, which was to pick 11 players, keep playing them and only make tactical subs. Once I realised that, the only injury I got was where I hadn't rotated the strikers.

I still think forward runs drains the condition slightly too much. There's no way I can substitute the number of players my assistant suggests.

[Super Bladesman]

Does PaulC get a million dollars now?

One would assume so, I'm sure the OP can find his address to forward on the funds

[Serpico]

Does PaulC get a million dollars now?

Nah, he'd not notice that piffling amount. Send it to me.

[Kriss]

Kriss - an entire first team squad out injured isn't obvious until the public get their hands on it?

I'm sorry but you're just blindly sticking up for SI here and ignoring the facts.

It didn't happen to any testers that I know, of course that would be partly because they know how to start a game and would have taken precautions some players wouldn't do.

Thus would have suffered it less, and as I said earlier testers always whinge about injuries:D

[PaulC]

Regarding the forward runs and condition.....condition is pretty much adjusted by distance moved and the speed its moved at.....so to change this would entail changing the way forward runs work.

[Nomis07]

I want at least 10% for drawing it to the OP's attention, regardless of who gets it.

[Kriss]

I was more referring to beta testers who I presume have complete builds rather than in-house testers but I hope integration of the entire system is carried out extensively. I am aware of differences between FM and the software I test and our system is much larger in scale with less staff.

They certainly don't have complete builds, that's far too big a risk even if it were techically possible.

It's not my place to go any deeper on the actual procdeures, I hope some of you get the chance to test sometime.

It's very hard to convince people when you can't divulge all the facts.

I do believe testing can be improved even considering the necessary limitations.

I do know SI/Sega will always try to improve testing and problem finding.

There are however huge limitations involving time, security and less tangible things.

Keep pointing out the shortcomings by all means, they do listen and will pick up on any good ideas/valid criticisms.

Don't rant or be insulting, do you listen to people who do that?

[Kriss]

its gonna be fun when full game is out on 14th people play longer than 6months what else will thwy find.

Everybody else in this thread is discussing the shortcomings of testing in a rational and balanced way.

If you can't contribute more than stupid one liners please go find a whinge thread which matches your mental capacity (and typing ablity)

[Rb]

Having had a brief stint at testing the FM series in the past (Back in the fateful closed invite beta in 06 I think it was) I would say that its very hard to recreate what all of you thousands of users can do. Which is really get into the nitty gritty of every aspect with all sorts of different machines and find the problems.

The problem is that because SI have been let down in the past, they probably have a bit of trust issues with the beta system now. Hopefully FML has helped that. I would certainly like to see a proper beta invite for the FM series in the future. And I really think it would aid the game, more so than the current closed beta would.

[homerjnick]

As I said in another post why not have your in-house testers and then have an invite beta like they did before but make each beta have some unique way of identifying it and get in-house testers/forum member testers to register with IP/Name/Address etc so that if any code was leaked then it is traceable.

Having 1000 people at home sitting playing the beta would be far better than any in-house testing and thus you are also securing your code with perhaps beta testers signing an agreement saying if they leak it then prison it is!

[Rb]

As I said in another post why not have your in-house testers and then have an invite beta like they did before but make each beta have some unique way of identifying it and get in-house testers/forum member testers to register with IP/Name/Address etc so that if any code was leaked then it is traceable.

Having 1000 people at home sitting playing the beta would be far better than any in-house testing and thus you are also securing your code with perhaps beta testers signing an agreement saying if they leak it then prison it is!

They do to an extent currently. But its a closed beta so its only trusted invitee's only.

When they did the beta I was on, users had to message in why they think they'd be of help in the beta, then got invited. Unfortunately someone leaked the code which was a massive betrayal of trust. Hence why it doesn't happen any longer.

[homerjnick]

They do to an extent currently. But its a closed beta so its only trusted invitee's only.

When they did the beta I was on, users had to message in why they think they'd be of help in the beta, then got invited. Unfortunately someone leaked the code which was a massive betrayal of trust. Hence why it doesn't happen any longer.

I know as I was on the closed beta when they did it and we found a lot of bugs which really helped.

I know some trusted members of the forum are invited but not to the scale that they had their invite beta demo last time and I think it would be well worth if they did it again but secured the code like I mentioned earlier....

Kriss says that each in-house tester does not play the full game but only modules, well when we were doing that beta demo 2 years back (or whenever it was!) over 1000 people had the "complete" beta demo game and not just modules of it and so therefore was very effective in finding bugs.

I appreciate that SI cannot afford to have their code leaked like what happened last time (I was raging about that) but if they just spent a little time securing it a bit more or made sure that everyone was accountable then you have SI staff, in-house testers testing modules of FM BUT ALSO over 1000 guys at home playing a beta of it and posting all the bugs.

[Kriss]

Kriss says that each in-house tester does not play the full game but only modules, well when we were doing that beta demo 2 years back (or whenever it was!) over 1000 people had the "complete" beta demo game and not just modules of it and so therefore was very effective in finding bugs.

I appreciate that SI cannot afford to have their code leaked like what happened last time (I was raging about that) but if they just spent a little time securing it a bit more or made sure that everyone was accountable then you have SI staff, in-house testers testing modules of FM BUT ALSO over 1000 guys at home playing a beta of it and posting all the bugs.

It was never the complete demo, it can't be because that doesn't exist until very late in the day.

I qualify that by saying it might have been the complete demo if it hadn't turned up any problems but that was never going to be the case:D

Security is the issue as you point out, but it has to be security that prevents code leak, catching the culprits after the code leak is "stable door etc."

That's the problem which haunts a product like this, imagine the value of a playable version of FM in the hands of a pirate before release date, and imagine the damage to SI.

[StewieGriffin]

From what I have read, some people are experiencing "bugs" and some people aren't. I for one have not had an injury issue as I continually rotate my players during pre-season. Are those that are experiencing this problem doing so with relatively small squads which they can't easily rotate to keep players fresh.

IMO the injury issue is not a bug but caused by the players fitness/energy levels dropping to fast and as the players tire they are more prone to injuries. This will not be seen where managers can rotate and replace players due to having larger squads.

The demo will currently be played by hundreds of thousands of players and the fact that not everyone is seeing the same issues, should therefore indicate to those doubting the testing process that the testers maybe just didn't have the problem and certainly will not have had the various setups that are currently playing the demo.

If any of these issues were ones which everyone had I would agree that they should have seen them, but we now have more variables due to the increased number of players and therefore some issues will now be found that previously were missed.

If you think SI are bad for releasing a game which has a few issues, non of which are game stoppers, for me anyway, then just have a look at Microsoft and Vista. This is a system that has major issues and people buying new computers have very little choice but to have Vista installed. At least we can choose to not buy FM09 after testing the demo.

[homerjnick]

It was never the complete demo, it can't be because that doesn't exist until very late in the day.

I qualify that by saying it might have been the complete demo if it hadn't turned up any problems but that was never going to be the case:D

Security is the issue as you point out, but it has to be security that prevents code leak, catching the culprits after the code leak is "stable door etc."

That's the problem which haunts a product like this, imagine the value of a playable version of FM in the hands of a pirate before release date, and imagine the damage to SI.

Your right Kriss it wasn't the complete demo but it was not modular like you say in-house testers do at the moment so the beta back then was a "more" complete version of the game that was playable.

FM has always been pirated, as every game is, and I guess we are both right in that there is no way to make it secure when you are inviting the public to test.

I'm sure if it could be made secure then SI would take the offer up as it was a real boon to find bugs and report them prior to the demo being released.

[homerjnick]

From what I have read, some people are experiencing "bugs" and some people aren't. I for one have not had an injury issue as I continually rotate my players during pre-season. Are those that are experiencing this problem doing so with relatively small squads which they can't easily rotate to keep players fresh.

IMO the injury issue is not a bug but caused by the players fitness/energy levels dropping to fast and as the players tire they are more prone to injuries. This will not be seen where managers can rotate and replace players due to having larger squads.

The demo will currently be played by hundreds of thousands of players and the fact that not everyone is seeing the same issues, should therefore indicate to those doubting the testing process that the testers maybe just didn't have the problem and certainly will not have had the various setups that are currently playing the demo.

If any of these issues were ones which everyone had I would agree that they should have seen them, but we now have more variables due to the increased number of players and therefore some issues will now be found that previously were missed.

If you think SI are bad for releasing a game which has a few issues, non of which are game stoppers, for me anyway, then just have a look at Microsoft and Vista. This is a system that has major issues and people buying new computers have very little choice but to have Vista installed. At least we can choose to not buy FM09 after testing the demo.

I have had Vista from MS at least a year before they released it as part of their Technet team and we have never had any major issues with it, by far the most stable MS OS there is and that is true even when it was beta.

[matthiele]

i have been rotating my squad and still getting injuries. however i have been able to get it down to a more manageable level by taking my training down a bit. still get those long term injuries though which is the main problem in my opinion.

[thisboyshouts]

i suspect the injury issue was not picked up in testing because it is not per se, a bug. it's not the game functioning incorrectly , or crashing, or anything like that, it's just an imbalance.

i suspect SI would use an out-of-house games testing agency to perform stress tests. the staff conducting the tests would not necessarily be as familiar with the world of football as the average member of this forum, so would probably not pick up on level of injuries. even if they did, they probably wouldn't consider it their position to challenge it.

that's why the only way to get an idea of the balance of the game and match engine is to unleash it on the mass public and make tweaks based on their feedback. it takes time but they do get it right eventually.

[Be¢kham]

I can report that SEGA has a dedicated FM 2009 testing team, that I was, very very briefly, a part of.

In all honesty I was there for a day, mainly because of the long journey from where I live to Chiswick (having to set off at 5.30am, Monday to Friday), and the fact that I wasn't willing to do it for the low wage on offer.

Anyway, enough about me. The testing team is large to say the least and the day is split into three shifts (7-3, 3-11, 11-7...or something along those lines and even includes weekend shifts) and there was around 20-30 people testing FM 09 when I was there (a good month or so ago on the 7-3 shift).

My point is that plenty of testing is actively going on, I assume at SI and also at SEGA and bug testing is clearly a high priority. So I really can't see how these obvious injury and text bugs managed to get through the testing stage (maybe the guys at SEGA were working on a different build to the one finally released as the demo?). No doubt the guys at SEGA would have picked them up (especially the injury bug) and they are receiving some unfair criticism.

[andysafc]

I think that if the demo was merely a demo then it would be acceptable. But it is exactly the same as what we will get on Nov 14th makes it unacceptable. No one should need a patch in order to run the game. Other software companies release a demo which is work in progress and not yet finished. SI release a demo that is finished until it shows up loads of bugs so the answer seems clearly - no it was not tested enough.