FM2011 vs. real life - goals scored empirical comparison

I posted these over at The Dugout forum, and I am not the kind of guy that goes around posting same stuff on various forums for selfpresentational reasons, I post this here as I really want to bring it to the attention of the people at the SI.

So, I was playing FM2011, and was wondering on the side: How does FM fare in comparison to real life in terms of goals scored? After a hands-off game with the top five European leagues, I compared the data with the ones available here: http://www.football-data.co.uk/englandm.php. I present you the results in the following table. The RL data are from the 2009/2010 season, compared to the FM 2010/2011 season. I present total goals per match, as well as for home and away teams separately (The Bundesliga data is missing as the German table from the mentioned site is f*** up for some reason):

Now, at first sight everything seems okayish, but further analysis reveals an astonishing difference. Mean goals per match is 2.627 for real life, compared to 2.571 in FM. It appears that there is a slight underestimation of total goals in FM compared to real life. The difference isn't large, so this is a minor issue. The problem is in the pattern of the goals. The bias for home team compared to the away team is much smaller. In real life home teams score an average of 1.557 compared to FM's 1.378, and away teams 1.069 compared to FM's 1.193 goals. So the bias in favor of the home teams in real life is 1.465 (meaning that home teams score almost 1.5 as many goals as the away teams), while in FM it's only 1.155. In other words, for every 100 goals scored by the away teams in real life, home teams will score 146, while in FM they will score only 115. This may not seem like it, but this is a HUGE difference.

One can criticize this data and say that it's only one season in four leagues, not a very large sample - that is true, but keep in mind that those are 1520 individual matches (one season in four countries with 20 clubs), which most certainly IS a large sample. This could be just a fluke (even though I highly doubt it), but I just wanted the SI people to see this and maybe address the issue at some point.

Thank you for your attention.

Although you've given alot of details your sample is too small for us to reach a definitive answer that the game doesn't give realistic numbers of goals a season. To add more clout I'd repeat this at least 5 times using a first season base and then come back and show us how its translated.

Not got anything to add, but good work

Yeah, ru-run that first FM season a couple more times and see if the data's consistent.

I don't agree, it's a sample of 1520 matches. In statistics that would be considered huge. Large samples are of the 200 size or similar.

what about the previos season irl. these stats change every year, and as said above, if you re-run your first FM season you should find that these figures will change.

although the stats are different from FM to real life, there doesn't seem to be anything that is glaringly different and wouldn't be overly concerned. barring the comparisons in domestic bias between FM and irl the differences are very small and there is always the chance they will even out over the years

Interesting (I'm an economist and spend must of my time calibrating eco. models) so just a few comment that hope can help you further:

- Did you calculated the Standar Deviaton (SD) for the home & away games? if so check which league is giving you more noise (higher SD) IRL and in FM.

- Try checkin correlations between Real Life and FM, if those correlations are high then maybe FM is producing the right tendencies and the differences (in level) you're observing can be attributed to the small sample.

I'll access to the data you're using and try to provide better tips to see if FM biasing as you suggest (I suspect home & away goals may have a lot of answers in this low FM Domestic Bias)

Interesting (I'm an economist and spend must of my time calibrating eco. models) so just a few comment that hope can help you further:

- Did you calculated the Standar Deviaton (SD) for the home & away games? if so check which league is giving you more noise (higher SD) IRL and in FM.

- Try checkin correlations between Real Life and FM, if those correlations are high then maybe FM is producing the right tendencies and the differences (in level) you're observing can be attributed to the small sample.

I'll access to the data you're using and try to provide better tips to see if FM biasing as you suggest (I suspect home & away goals may have a lot of answers in this low FM Domestic Bias)

Unfortunately it is impossible to extract data from FM, I had to write down all the individual goals scored and add them up, I guess i could calculate the SDs, but at this large sample sizes the differences observed are bound to be statistically significant.

So, even if this is a fluke, i beg the people at the SI to crunch their own numbers, as I can't extract data for a bigger comparison, it would just take too much time.

what about the previos season irl. these stats change every year, and as said above, if you re-run your first FM season you should find that these figures will change.

although the stats are different from FM to real life, there doesn't seem to be anything that is glaringly different and wouldn't be overly concerned. barring the comparisons in domestic bias between FM and irl the differences are very small and there is always the chance they will even out over the years

Even so, this is a sample size of 1520 real matches, at that level the numbers will be pretty stable.

Even so, this is a sample size of 1520 real matches, at that level the numbers will be pretty stable.

not arguing with the size of the survey or dismissing the efforts you have put into your research, but i'm sure you will agree that one season is different from another, and that if you looked through the records from the previous 5 years (IRL) you will find that these figures will differ

I think it's a great start. Sample size is definitely large enough but I'd still re-run the FM one another 2 times and take that average. You never know.

Also, I'd run this exact same test 20 years out when the game has pretty much no real players in it to see how close to real life sticks.

As someone mentioned, I'd also look at the numbers for those leagues the last few years just to make sure that it's a ratio that remains fairly constant through time. I wouldn't be surprised if teams were having mroe trouble winning at home than they did 10 years ago (although that is not necessarily relevant for this).

not arguing with the size of the survey or dismissing the efforts you have put into your research, but i'm sure you will agree that one season is different from another, and that if you looked through the records from the previous 5 years (IRL) you will find that these figures will differ

Agreed, they will differ - but not much. A 5% or 10% difference at most. I will check it out.

I think it's a great start. Sample size is definitely large enough but I'd still re-run the FM one another 2 times and take that average. You never know.

Also, I'd run this exact same test 20 years out when the game has pretty much no real players in it to see how close to real life sticks.

As someone mentioned, I'd also look at the numbers for those leagues the last few years just to make sure that it's a ratio that remains fairly constant through time. I wouldn't be surprised if teams were having mroe trouble winning at home than they did 10 years ago (although that is not necessarily relevant for this).

I would definitely do this, if there was any way to extract the data from Fm without too much trouble. And believe me, I tried (even tried screenshoting the tables and then using an OCR software to turn it into numbers - it didn't work). That's why I'm imploring the SI people who have the resources to do it, actually do it.

You can extract league tables can't you?

You can extract league tables can't you?

I was never able to do it. Have you? If there is a way, please share!

Not in front of FM.... print to file or something?

Not in front of FM.... print to file or something?

Don't see it...

I guess i could calculate the SDs, but at this large sample sizes the differences observed are bound to be statistically significant.

I suggested this because your sample is only for one season and 4 leagues (even if one season consist of hundreds of games, the relevant variable is the average seasonal number of Home/Away goals) so calculating SD's can tell you if the data from FM is inside the probable numbers or if FM data is unrealistics according to IRL data.

Go to league table

FM button

Print Screen

Text file

This is good stuff. Are these matches being simulated in full detail for all the leagues? That would be important information.

If so, I am wondering if this is an issue of tactics?

What about home records vs. away records? This would seem to indicate that sides perform better on the road in general than in real life.

The Difference is most likely due to the effects of fans in real life. It is impossible to put a number on the exact benefit given by the home fans however SI imo have done a decent job.

I think two more sets of data would be useful, especially given that in one league, the home teams scored more than they should.

Also, I'd argue that last season in the English Premiership was possibly anomalous. Chelsea racked up several large scores at home, at Spurs got one too. These will have driven the average up significantly, I'd imagine. That is probably part of the problem, massive thrashings between teams of similar quality are quite rare on recent FMs, so there are no 9-1s to drive up the average.

Go to league table

FM button

Print Screen

Text file

Great! this is very helpful. I will compute a three year long survey on the topic (that will be over 3500 matches), but it will probably take some time, tomorrow evening, I guess.

This is good stuff. Are these matches being simulated in full detail for all the leagues? That would be important information.

If so, I am wondering if this is an issue of tactics?

What about home records vs. away records? This would seem to indicate that sides perform better on the road in general than in real life.

Full detail, naturally.

No problem - very rare to see a thread on this topic that doesn't include the line "because I'ze gonE 3 gamez witot scoringz it most be rooined"!