Evidence Based FM's testing seems to show that Match Prep sessions increase the likelihood of losing the next match compared to no training at all. Is this a bug or something else?

[phnompenhandy]

@weasel

Yeh, with my tactical familiarity full, I sub a Community Outreach for Review, and Match Practice for Preview. I'm in pre-season now - I'll try that double-recovery thing for the day after a match and occasionally in midweek when we have lots of double-General days - see if injuries are reduced.

Cheers.

[ImDaWeasel]

35 minutes ago, phnompenhandy said:

@weasel

Yeh, with my tactical familiarity full, I sub a Community Outreach for Review, and Match Practice for Preview. I'm in pre-season now - I'll try that double-recovery thing for the day after a match and occasionally in midweek when we have lots of double-General days - see if injuries are reduced.

Cheers.

I'd be wary of using a match practice the day before a match, it will affect a players overall condition right before a match. I only use it during the week if there is no match during the week. I've always kept the match preview. The double recovery is only on 2 match weeks.

Edited June 29, 2022 by ImDaWeasel

[Daniel Evensen]

11 minutes ago, ImDaWeasel said:

I'd be wary of using a match practice the day before a match, it will affect a players overall condition right before a match.

If you watch Max's video carefully, you will see that it has no impact on the condition of players.  He shows this pretty clearly.

[ImDaWeasel]

47 minutes ago, Daniel Evensen said:

If you watch Max's video carefully, you will see that it has no impact on the condition of players.  He shows this pretty clearly.

Yup, I tried that route only to have players with 90% or less condition on match day vs 95%+ by not having it the day before. Just my own experience. Great during the week, not before game day imo.

I'm not saying Max's trials is now the bible for training methods but he did show alot of good evidence that I've now adopted into my own training schedules as I've said before.

Edited June 29, 2022 by ImDaWeasel

[Andrew Marines]

It's interesting to see players having opinions about how things work but never tried to see if they actually work that way. Max here tried a scientific approach(that could have some flaws, we don't know how the game is actually coded) and here we should try with the same method.

[werogatda]

Hey all, Max here. Apologies I am late to the conversation as I'm not usually active here. Appreciate all the comments and feedbacks. My responses to some of the issues raised:

1. If the main benefits of the Match Prep sessions are long-term and accumulative in nature (eg. boosting tactical familiarity / player attributes), then of course a single session conducted the day before a match won't produce any difference in terms of the win rate. But that's not why we all like to include a Match Prep session into our training regimen before an important match, is it? Match Prep sessions are presented to us gamers as having a temporary boosting effect for an upcoming match. This is suggested by:
a) the in-game description for the sessions mention "Upcoming Match..."
b) when we leave the training schedule to the AssMan, he will always include a Match Prep session the day before a match
c) Before an FA Cup final or a Champions league final, the AssMan will use one of the default training schedules "Big-Match Preparation" which includes multiple types of Match Preparation sessions.
d) When you conduct a Match Prep session, the match preview screen (just before starting a match) will say something like "for this match, special focus has been placed on Teamwork / Att Movement / Def Shape..." again reinforcing the notion that the Match Prep sessions have had a temporary boost just on that match.

This is why it is important the investigate whether a single Match Prep session has a one off, temporary boosting effect when conducted just before a match. Multiple Match Prep sessions conducted repeatedly over a season will boost a range of player attributes and this will of course raise the win rate of your team in the long run. But that was not the focus of my experiments.

2. What I said in the video (around 3:30) regarding Tactical Familiarity is slightly misleading. TF was maxed for both teams of the experiment (Team A/B) because it was important to make both teams identical in strength. It is possible that the Match Prep sessions boost TF of a team and raise the win rate of the team through a heightened TF. The effect that Match Prep sessions may or may not have on TF was NOT reflected in my results because TF was maxed out for both teams in the first place. However, again the focus of my investigation not the accumulative benefits of the boost in TF but the existence of any one-off boost, possibly hidden under the bonnet, in the performance of a team. It made sense to equalise the TF of both teams in my experiments so that any one-off boosting effect of the Match Prep sessions, if any, could be isolated from the effects of TF. 

3. Both teams of the experiment (Team A/B) used a generic 4-4-2 tactic that I made myself and use for my FM experiments. This tactic uses generic roles (eg. Central Midfielder / Central Defender) and contains no fancy tactical instructions. It is a tactic designed to be as vanilla and generic as possible which makes it suitable for scientific FM experiments. This tactic is visible very briefly in the video (3:16). But at the end of the day the exact nature of the tactic isn't crucial - what is important is that both teams (Team A/B) use the exact same tactic during the experiments, and that the same tactic is used for the different samples (as I change the variables). The tactic being used will no doubt affect the performance of players on the pitch  - but as long as that effect is held identical throughout the experiment, it doesn't matter.

4. I admit that 15 in all attributes was too high. I will change to 12-13 in my future experiments. But even with superstar players on the field, if the Match Prep sessions had a temporary boosting effect, it still should have created a difference in the win rate. Surely, Cristiano Ronaldo who studies and analyses a match the night before would perform better than just slagging off & playing FIFA the night before the match! (or.. not?)



 

[Siven]

Set piece training video is out, and very interesting.

Really enjoying the videos.

[Dbuk1]

It's nothing new to players that have played fm for years and years. The intention of fm is to make you feel like a footy manager.....if it does then enjoy your time but don't look under the hood, you will be very disappointed. Ignorance is bliss.

Edited July 11, 2022 by Dbuk1

[Daniel Evensen]

28 minutes ago, Dbuk1 said:

It's nothing new to players that have played fm for years and years. The intention of fm is to make you feel like a footy manager.....if it does then enjoy your time but don't look under the hood, you will be very disappointed. Ignorance is bliss.

Not sure why you're being so dismissive of this. 

I doubt you'll find many long-time FM veterans who tell you that set piece training does very little to impact your success in an individual match, for example.  In fact, most people I've seen who are putting together their own training sessions (including on the same Discord group that Max posts in) continue to use these training routines almost out of habit.

Don't you think there is at least some value in looking under the hood and seeing how things actually work?  Certainly it deserves a better response than "nothing new to see here, we knew this all along."  Max's research really flies in the face of all the conventional wisdom about training that is out there.

[dasnooz]

On 02/07/2022 at 03:36, werogatda said:

4. I admit that 15 in all attributes was too high. I will change to 12-13 in my future experiments. But even with superstar players on the field, if the Match Prep sessions had a temporary boosting effect, it still should have created a difference in the win rate. Surely, Cristiano Ronaldo who studies and analyses a match the night before would perform better than just slagging off & playing FIFA the night before the match! (or.. not?)

If the match prep sessions are a temporary boost to player attributes, then maybe it is capped? E.g. with a Def. Shape session, you can boost the Positionning attribute of all players to max 15? Something like

boosted_attribute = min(original_attribute + boost, 15)

It would also makes sense to have diminishing returns as player attributes are high.

That would explain the results you obtained. Would be a shame if match preps sessions are useless - having to choose between short-term success and long-term development is great for gameplay

Edited July 12, 2022 by dasnooz

[el_manayer]

I am afraid that, although the experiment is very interesting and involves a lot of work, the results can not be conclusive. First, there is the discussion about if match preparation, even if it "boosts" your chances of winning the match, it only does it when you have done this repeatedly in time, which would be the logical thing imo. It shouldn't make a significance difference for a brand new team to do something once and then be better at a level enough to have a significance result on a 200 sample. And then, more about this, the sample size is probably far from being large enough. The sample size is 200 and your "standard" result (victories) is around 75, meaning that is only going to show some results if the "boost" effect is of about 11%, and this would be only in the 1-sigma level assuming a Poisson process, which, even if proven, would also only be conclusive on a 1-sigma level, which is usually not enough. I'll explain this a bit for those who do not often work with statistic (and please correct me if I am wrong, which I might be):

- A 1-sigma Poisson confidence level means that, if you would repeat the experiment infinitive times, ~68% of the results would lie between this uncertainty. This is easily calculated assuming a Poisson process (flip a coin, for example), since it is just the squared root of the value. Thus, in our example, assuming the standard "unmodified" result of victories without training is 75, the 1-sigma level is +-8.6. This means, if we repeat the experiment (1 experiment = 200 matches) infinitely, ~68% of our results would be between ~84 and 66. Still, if one of these experiments would give us, for example, 90 victories, it would not mean much, since that even can happen "randomly" in about 30% of cases. In the results we do not even see the 1-sigma deviation at all.

- Assuming again the expected value of 75 victories, with +-8.6 in the 1-sigma level, we see that 8.6/75 is approximately 11%, meaning that only if the boost modify the probability of victories by that or more, we should see a deviation from the 1-sigma confidence (again, only 68% confidence). A larger sample would make the results much more sensitive, for example if we would have a sample of 2000 expecting 750 victories, then we would be 1-sigma sensitive to a boost of sqrt(750)/750 ~ 3.6% (I am not saying that you or any other should do something similar with a 2000 samples... I understand the work involved in doing it with 200 to start with, so in my opinion this is definitely not worthy unless you have a really quick way to do each iteration).

- Training would be broken if a single session of match preparation or similar stuff would boost the chances of winning by more than a few percent, I do not know which number would be appropriate (1% maybe?) but definitely much smaller than 11%. If you assume a number here, you can calculate how large your test sample need to be so you can include or exclude that tested hypothesis.

- As I said before, I would not conclude anything from a study that shows some deviation from a Poissonian process in the 1-sigma level, since there is a ~34% probability that this would happen... you usually want more certainty.

Edited July 17, 2022 by el_manayer

[AndreasAust]

one thing cannot be contested after several of those experiments (each with 200-300 matches under lab conditions):

the training sessions clearly do not train the attributes they claim to train AT ALL (for example one of the earlier videos showed that the "general" sessions train everything but the ones in their description).

"attacking corners" trains a number of attributes but of all things it does not train "corners" at all.
And as I have said earlier, I can confirm to this after several seasons of using that session regularly.

And along with that, many other things clearly do not do the things claimed by the ingame descriptions.

So, coming from kinda being a 'Sports Interactive' fan, I am now pretty disappointed. I have put countless of hours into tinkering with my training plans, in order to exactly train the attributes I wanted and having the "upcoming match bonuses" active at all times etc.    and nobody can tell me that this is just a bug. Of course SI must have been aware of that. Especially after all this time that has passed and a new FM game coming out each year. Also, sometimes there is a clear pattern, like all the set-piece-sessions training the exact same bunch of attributes - that of course have nothing to do with the ingame description.

So my own imagination was fooling me:  every time my boys scored a goal from corners, I thought to myself it must have come from my upcoming match bonus "attacking corners" ;-)

(edited since I was being too harsh and personal)

Edited July 17, 2022 by AndreasAust

[XaW]

@AndreasAust - Thinly veiled insults to the developer aside, you posts includes a whole lot of noise with very little substance. If you have examples of errors, bugs, or other types of issues with the game, they need to be reported in the bug tracker. Please make sure to follow the guidelines in order for the report to be useful.

Also, quit the digs please. Read up on the house rules, especially this one:

[Feron]

1 hour ago, XaW said:

@AndreasAust - Thinly veiled insults to the developer aside, you posts includes a whole lot of noise with very little substance. If you have examples of errors, bugs, or other types of issues with the game, they need to be reported in the bug tracker. Please make sure to follow the guidelines in order for the report to be useful.

Also, quit the digs please. Read up on the house rules, especially this one:

"a lot of noise with very little substance"
since I clearly am referring to someone else who has literally put in hundreds of hours doing these experiments under the most fair and "level" conditions possible, trying to be as objective as possible,  I wouldn't say that there is little substance.

But I do apologise for my tone. It just is so hard for me to believe that one can program this game and don't become aware of such a major thing. We are not talking about having one attribute swapped here and there by accident (THAT indeed would be a bug).   And like I said, I've paid for the game and put in hundreds of hours tinkering with training plans - reading and learning all these descriptions and creating and saving training plans for every occasion - which I now know was a complete waste of my time. So maybe you can understand the way it makes me feel a little bit.

 

Edited July 17, 2022 by Feron

[XaW]

On 17/07/2022 at 20:05, Feron said:

"a lot of noise with very little substance"
since I clearly am referring to someone else who has literally put in hundreds of hours doing these experiments under the most fair and "level" conditions possible, trying to be as objective as possible,  I wouldn't say that there is little substance.

If you look at the 2nd post in this thread, SI have thanked for the video and told that SI will look into it. Based on that, what more does your post bring into this discussion with sentences like "We are paying customers and it feels bad and strange being treated like that. Also, who knows what other things don't work as intended?  And by intended I mean the in game descriptions, who knows what SI's true intentions are."? 

While I appreciate you editing it after my response, you see why I question you in this regard? The user who made the video has also posted more info in this thread, and these things are always appreciated when they are reported. Either because SI can explain the reasoning, or find out that something is not working as intended and log it as a bug. This has happened on countless occasions here, but things must be done in a civil matter. If you have a legit point, it can be easily lost when the messenger mixes it with insults and/or accusing the recipient of malice. Please try to take this with you in any further discussion about bugs or mechanics in the game, as we mods don't want to step into a good discussion, but we will if it's done in a non-constructive matter.

[herne79]

13 hours ago, Feron said:

which I now know was a complete waste of my time

 

15 hours ago, AndreasAust said:

one thing cannot be contested after several of those experiments (each with 200-300 matches under lab conditions):

the training sessions clearly do not train the attributes they claim to train AT ALL

Conclusions should never be drawn with this type of experiment until SI have checked and verified the data.  There may well be an issue but until we have that sort of feedback from SI (could one of the Mods please follow up with @Neil Brock after his initial post for an update?) all we can say for certain is that it’s an area of interest that needs clarification.

In my experience SI have always been very open to going through such data and honest with their responses, so rather than jumping to conclusions give SI a chance to check this out.  Of course if it turns out there actually is an issue then give ‘em hell (but don’t get personal ).

[Feron]

On 17/06/2022 at 00:59, DarJ said:

... it doesn’t say you’ll score more goals or win more games.

Training doesn’t directly help you win games, it’s solely to improve the attributes of your players.

....
You want them (specifically attacking movement, defensive shape and match practice) because I presume everyone have given their players specific roles to train in, and those are the only schedules that target the specific attributes of the roles you

 

I agree that this misconception exists where people think that (for example) "chance conversion" helps them score more goals in the next game which of course is not true at all - it only trains the attributes long term.

However, the "upcoming match bonuses" definitely should give you a short term boost aside from training the attributes in the long run - this you can already tell by the fact that these bonuses last for exactly 14 days (for all games during that time) and then disappear. So how can we be talking about attributes here. Apart from the fact that the text "upcoming match" would be really misleading and this cannot/shouldn't be the intention of a developer.

And while boosts don't guarantee you win more games, this definitely is the whole purpose of course and should make a statistical difference in the win rate - if you looked at enough games - which Max did when he each simulated 200-300 matches. What Max should have done though is having like 7-8 "upcoming match bonuses" active at the same time (include some of the set piece category) and then see what difference that makes.

you said they were the only ones that target the specific attributes - yes they claim to train them - but they are not the only ones of course (things like ground defence/aerial defense for attackers and chance creation/conversion for defenders, ALL of the goalkeeping stuff for both defense/attack target the roles aswell). 

 

The point that really bugs me though is that several videos of Max have shown now that the training sessions simply don't train the attributes they claim to do in their description - like not at all. as if there was no connection between description and reality.

 

 

I feel like this is such a major thing and completely made me lose trust in all the descriptions.

SI said they were gonna look into this but more than a month has passed now and it cannot be too hard to find such a huge (alleged) deviation. So aside from the fact that this shouldn't happen at all (for a programmer it must be hard to not notice this), this game is published every year with the game code not changing too much, and like I said more than one more month has passed now in which SI was notified.

So I so much hope SI will publish a statement soon and either fix the game or say that Max's findings are not true.

Edited July 21, 2022 by Feron

[glengarry224]

On 18/07/2022 at 00:27, herne79 said:

 

Conclusions should never be drawn with this type of experiment until SI have checked and verified the data.  There may well be an issue but until we have that sort of feedback from SI (could one of the Mods please follow up with @Neil Brock after his initial post for an update?) all we can say for certain is that it’s an area of interest that needs clarification.

In my experience SI have always been very open to going through such data and honest with their responses, so rather than jumping to conclusions give SI a chance to check this out.  Of course if it turns out there actually is an issue then give ‘em hell (but don’t get personal ).

First, thanks to @werogatda for doing all this work.  I second what @herne79 wrote and ask again:  does SI have any update or response to this as it is now more than five weeks since @Neil Brockposted that they are looking into it?

I don't have the statistical expertise - and in the video, Max sagely acknowledges that he does not either - but these videos do raise serious questions.

[glengarry224]

@werogatda: have you looked at the data of how, for example, 'Attacking Movement' training affects 'goals scored' or 'passing stats' as opposed to game outcomes?  Just brainstorming, I suppose it's possible that if you train 'Attacking Movement' and not 'Defensive Shape', then your team might become more attacking and less defensive?

Not trying to put extra work on you, just wondering.  Thanks again.

 

[Neil Brock]

Just to be clear on this one, it is something we've looked into internally and found  the way the test was run means the results are basically just a result of variance in the match engine rather than a good measure of the impacts of training. 

That's not trying to dismiss the test at all as it has sparked some interesting and insightful conversation. But that's what we found after investigating. 

 

[Feron]

5 hours ago, Neil Brock said:

Just to be clear on this one, it is something we've looked into internally and found  the way the test was run means the results are basically just a result of variance in the match engine rather than a good measure of the impacts of training. 

That's not trying to dismiss the test at all as it has sparked some interesting and insightful conversation. But that's what we found after investigating. 

 

ok thanks,

but what about the fact that training sessions don't train the attributes they claim to? How can that be a "variance in the match engine" ?

Those set-piece things all seem to train a set number of the same attributes rather than the ones of the individual description (let alone the one attribute they should train, like "attacking corners" doesn't train corners...)

And in the other video from Max the sessions of the "general" training section seem to train anything but what's said in the description.

I personally have been training "attacking corners" regularly for several seasons now and never has it helped with the corner attribute.

 

And if you say the "variation in the match engine" leads to the upcoming match bonuses not yielding an effect in like 300 games, shouldn't you tweak something then? I mean of course they shouldn't make you win impossible matches but isn't the reason you'd go for these bonuses to have some kind of benefit from them?  And in football 'benefit' usually means the score.

[Dbuk1]

I wonder if SI can provide a test and results that show anything to the contrary??

[glengarry224]

18 hours ago, Neil Brock said:

Just to be clear on this one, it is something we've looked into internally and found  the way the test was run means the results are basically just a result of variance in the match engine rather than a good measure of the impacts of training. 

That's not trying to dismiss the test at all as it has sparked some interesting and insightful conversation. But that's what we found after investigating. 

 

@Neil Brock but if that's true, then you are supporting @werogatda's claim that the tested match preparation sessions have de minimis (and perhaps even opposite) effects on match outcomes, because he ran each tested group 200 times.

Basically, you are indicating that the effects of variance are so great as to render this type of match preparation insignificant.  Now, irl, some would similarly argue that much match preparation is a waste of time and has little effect on results.

note:  I personally believe that variance plays a much, much greater role in football and in most areas of life than we are willing to accept and that all of us seek a level of control and narrative which turn out to be fictitious, if not comforting nevertheless.

[el_manayer]

2 hours ago, glengarry224 said:

Basically, you are indicating that the effects of variance are so great as to render this type of match preparation insignificant. 

That's exactly what I was trying to explain above. Statistically, the experiment sample is not large enough unless the effects of the training are huge.

[Nacaw]

1 hour ago, el_manayer said:

That's exactly what I was trying to explain above. Statistically, the experiment sample is not large enough unless the effects of the training are huge.

You explained it very well - but to people not familiar with statistical analysis it will look like the rantings of a madman 

 

 

[Feron]

I can only repeat myself, 
please put the upcoming match boosts aside for a second:   why does nobody seem to be interested in the fact that the training sessions don't train the attributes they have in their description? 

Just have a look at Max's video about the general sessions, but also in the set piece video you can see that these sessions do anything but what they are supposed to train - in many cases they even leave out the main attribute completely (for example 'attacking corners' doesnt train corners).

I am still making training plans but it all feels so meaningless now, why am I even reading and learning the attributes of each session which is such a big effort, knowing that most probably the game doesn't even register what I am doing.

Edited July 23, 2022 by Feron

[glengarry224]

11 hours ago, el_manayer said:

That's exactly what I was trying to explain above. Statistically, the experiment sample is not large enough unless the effects of the training are huge.

Right, sorry, was not trying to steal your thunder (and I upvoted both of your posts!) but was instead trying to point out that Neil's response, which on the surface appears to explain away the problem as "variance", instead appears to concede the problem.  Perhaps that was not his intent and his response was loosely worded but that's what it looks like. 

Of course, variance plays a large role in FM and irl, but that does not mean that the stated training effects actually work or that someone could not construct an experiment to test whether they do.  I am not opining that the experiment was properly constructed, or that it was not properly constructed, but I know enough about probability and variance, and I've dealt with enough experts, to know that Neil's explanation is not an explanation but appears to be a concession.

How large would the experiment sample need to be?  And what do you think the test results indicate, if anything?

Obviously, if he'd tested 4 matches with each training, we'd all realize that the results were not meaningful, but this has the appearance of a large enough sample size, but I admit my ignorance.

[werogatda]

On 22/07/2022 at 21:28, Neil Brock said:

Just to be clear on this one, it is something we've looked into internally and found  the way the test was run means the results are basically just a result of variance in the match engine rather than a good measure of the impacts of training. 

That's not trying to dismiss the test at all as it has sparked some interesting and insightful conversation. But that's what we found after investigating. 

 

Challenge accepted.

[phnompenhandy]

On 23/07/2022 at 00:09, Feron said:

I personally have been training "attacking corners" regularly for several seasons now and never has it helped with the corner attribute.

 

Just on this specific point - to be fair, "Attacking corners" should involve the players in the box, not the corner-taker. That would more likely be 'set piece delivery'.

 

Having said that, Max has just dropped his latest YouTube where he confirms that NO training schedule trains Corners, Free Kicks, Penalties or Long Throws.

Edited July 24, 2022 by phnompenhandy

[Feron]

11 hours ago, phnompenhandy said:

"Attacking corners" should involve the players in the box, not the corner-taker. 

 

I have no idea how you got to that opinion. First, logic already tells us that if a couple players kick hundreds of corners, why would it only train those on the other end?

second, the ingame description clearly says:
secondary focus: set piece takers. attributes: CORNERS, technique

And there's the issue that I have again:  The descriptions are wrong

And that kills my fun, my immersion, my trust in the game etc. and even gives me thoughts like:  what else doesn't do what it says in the texts?

And still no statement from SI

In case you think I am overreacting:   as mentioned earlier, I have burried hundreds of hours of my personal time into making training schedules and learning what each session does. It just feels bad. And since I am still playing this game, I am just hoping to get some kind of reaction from SI that does NOT talk about "variations in the match engine", ignoring the elephant in the room.

Edited July 24, 2022 by Feron

[enigmatic]

14 hours ago, werogatda said:

Challenge accepted.

You could certainly considerably improve the test by not eliminating the main variable the sessions are supposed to improve in the short term (tactical familiarity)....

[Neil Brock]

Just to be clear, the best way to raise specific issues is via our Bug Tracker here - https://community.sigames.com/bugtracker/football-manager-2022-bugs-forum/

Always a risk that if an issue is raised within a thread on a different topic won't be picked up. The set pieces training has been flagged to our QA team to investigate further. 

As an initial reaction, the way training is designed in game is so that it's designed to work realistically. If you set your entire team to only train say set pieces for weeks on end, the overall progression of players ability would be affected negatively, meaning attributes wouldn't rise to any great degree and most likely see a general drop off. The way attribute weighting works may show jumps in some areas which will be counter-balanced by drops elsewhere (overall CA should as said, remain consistent or drop, albeit in part dependent on a number of other factors - coaching/training facilities etc). Training is meant to use an overall balance in schedules to work over time, not be set in isolation. 

Looking at this video, this kind of control test where all the other factors that go into progression (player form, happiness etc) aren't considered, there's a strong chance that it won't really give the most accurate results. But again as said, this is just from an initial quick look and our team will, as with everything that gets flagged on the Bug Tracker, check it out. 

[Feron]

8 hours ago, Neil Brock said:

Just to be clear, the best way to raise specific issues is via our Bug Tracker here - https://community.sigames.com/bugtracker/football-manager-2022-bugs-forum/

Always a risk that if an issue is raised within a thread on a different topic won't be picked up. The set pieces training has been flagged to our QA team to investigate further. 

As an initial reaction, the way training is designed in game is so that it's designed to work realistically. If you set your entire team to only train say set pieces for weeks on end, the overall progression of players ability would be affected negatively, meaning attributes wouldn't rise to any great degree and most likely see a general drop off. The way attribute weighting works may show jumps in some areas which will be counter-balanced by drops elsewhere (overall CA should as said, remain consistent or drop, albeit in part dependent on a number of other factors - coaching/training facilities etc). Training is meant to use an overall balance in schedules to work over time, not be set in isolation. 

Looking at this video, this kind of control test where all the other factors that go into progression (player form, happiness etc) aren't considered, there's a strong chance that it won't really give the most accurate results. But again as said, this is just from an initial quick look and our team will, as with everything that gets flagged on the Bug Tracker, check it out. 

thanks Neil, then we'll wait for the result of your investigation. Would be great if you could also include the sessions of the "general" training section as it was very noticeable in Max's other video that they didn't train the attributes that were advertised. Since they are a little more versatile, what you said about monotonous training should be less of an issue. 

I personally have trained the set piece sessions and other things each only once every 14 days, in order to have their bonus active at all times. As mentioned for at least 5-6 seasons and have never seen the corner attribute go up (with players who had a lot of unused PA-points left and perfect training conditions).

Edited July 25, 2022 by Feron

[glengarry224]

On 25/07/2022 at 02:45, Neil Brock said:

Just to be clear, the best way to raise specific issues is via our Bug Tracker here - https://community.sigames.com/bugtracker/football-manager-2022-bugs-forum/

Always a risk that if an issue is raised within a thread on a different topic won't be picked up. The set pieces training has been flagged to our QA team to investigate further. 

As an initial reaction, the way training is designed in game is so that it's designed to work realistically. If you set your entire team to only train say set pieces for weeks on end, the overall progression of players ability would be affected negatively, meaning attributes wouldn't rise to any great degree and most likely see a general drop off. The way attribute weighting works may show jumps in some areas which will be counter-balanced by drops elsewhere (overall CA should as said, remain consistent or drop, albeit in part dependent on a number of other factors - coaching/training facilities etc). Training is meant to use an overall balance in schedules to work over time, not be set in isolation. 

Looking at this video, this kind of control test where all the other factors that go into progression (player form, happiness etc) aren't considered, there's a strong chance that it won't really give the most accurate results. But again as said, this is just from an initial quick look and our team will, as with everything that gets flagged on the Bug Tracker, check it out. 

That makes sense, and I've used the bug tracker several times.  However, I think that very few people read the bug tracker posts and if you want to raise an issue with the community, it needs to be raised here.

[Neil Brock]

9 hours ago, glengarry224 said:

That makes sense, and I've used the bug tracker several times.  However, I think that very few people read the bug tracker posts and if you want to raise an issue with the community, it needs to be raised here.

It sounds like it's dependent on what you're looking to do. If you want something to be checked by us and potentially improved or fixed if a problem is discovered then it really needs to be raised via the Bug Tracker. 

If you want to raise something for discussion with the community then that's fine to raise it within GD and have people comment. It's just understanding that whilst we try to be across as much as possible, there's a far greater chance we'll pick up anything raised within the Tracker itself. 

[riv3th3ad]

Without even mentioning the inadequate sample size, the trouble with attempting to isolate just match prep sessions or just set piece sessions is that it ignores so many factors that are supposedly linked together.

It's a mistake to run test with both teams at 100% tactical familiarity.

It's a mistake to ignore whether players are or are not on PRD and/or additional focus training. 

It has been widely discussed that the match bumps do not stack, so running the set piece test with a bunch of attacking corner or attacking free kick sessions, according to what we supposedly know, shouldn't result in massive increases in goals from corners/FKs. What is worth investigating is whether this many sessions has bumped the attributes claimed by the card.

On top of the tactical familiarity flaw, the test does not account for morale, training happiness over the course of a season. 

A better test of match prep and set piece sessions would run for multiple seasons with Team A and Team B set for identical training, excepting the match prep/set piece sessions, and equal, but lower than 100%, tactical familiarity at the beginning of each season. After watching several of the videos, I'm not prepared to completely throw the baby out with the bath water regarding match prep or set piece sessions. 

What is significantly more concerning is the testing that suggests the training cards do not actually boost the attributes they claim to boost. I believe there may be something to this, as, anecdotally, I constantly see instances of targeted attributes barely increasing across a season or two in players that are in the 18-23 range. If I have wingbacks set to WB-At PRD with additional focus training set to crossing and I routinely run attacking overlap and play from the back sessions, it's absolutely absurd that I shouldn't see noticeable improvements in crossing, flair, passing, vision in at least some of those players and in most they barely move. 

[Feron]

6 hours ago, riv3th3ad said:

.....

What is significantly more concerning is the testing that suggests the training cards do not actually boost the attributes they claim to boost. I believe there may be something to this, as, anecdotally, I constantly see instances of targeted attributes barely increasing across a season or two in players that are in the 18-23 range. If I have wingbacks set to WB-At PRD with additional focus training set to crossing and I routinely run attacking overlap and play from the back sessions, it's absolutely absurd that I shouldn't see noticeable improvements in crossing, flair, passing, vision in at least some of those players and in most they barely move. 

This. There apparently is someting off with the training sessions. They don't train the attributes in their description and this is not a minor thing.

But just as you described about the match boosts, I'd never use my own annecdotal impressions of my games to say that training is not working as intended, however here we have the evidence from the videos which - in this case - cannot be brushed off by match engine or morale, happiness, cohesion etc. PLUS the annectodal personal impressions like the corner attribute never improving for like 10 years of using the session regularly which just suits the findings in Max's video.  

Edited July 31, 2022 by Feron

[herne79]

7 hours ago, riv3th3ad said:

What is significantly more concerning is the testing that suggests the training cards do not actually boost the attributes they claim to boost. I believe there may be something to this, as, anecdotally, I constantly see instances of targeted attributes barely increasing across a season or two in players that are in the 18-23 range. If I have wingbacks set to WB-At PRD with additional focus training set to crossing and I routinely run attacking overlap and play from the back sessions, it's absolutely absurd that I shouldn't see noticeable improvements in crossing, flair, passing, vision in at least some of those players and in most they barely move. 

 

1 hour ago, Feron said:

This. There apparently is someting off with the training sessions. They don't train the attributes in their description and this is not a minor thing.

But just as you described about the match boosts, I'd never use my own annecdotal impressions of my games to say that training is not working as intended, however here we have the evidence from the videos which - in this case - cannot be brushed off by match engine or morale, happiness, cohesion etc. PLUS the annectodal personal impressions like the corner attribute never improving for like 10 years of using the session regularly which just suits the findings in Max's video.  

Why is it “absurd”?  Just because we want our players to do something doesn’t mean they will.  The training module has never been tell a player to train X and they do it, it’s way more complex than that.  As a (rhetorical) counter point - why are you, as the manager, constantly asking players to do something which they clearly aren’t capable of doing?  To quote Neil above, how is that “realistic”, bearing in mind that’s how training is designed?  Isn’t a manager who spends 10 seasons (or just 2 seasons without change) asking players to improve crossing without success absurd?

Re-read Neil’s post:

On 25/07/2022 at 10:45, Neil Brock said:

As an initial reaction, the way training is designed in game is so that it's designed to work realistically. If you set your entire team to only train say set pieces for weeks on end, the overall progression of players ability would be affected negatively, meaning attributes wouldn't rise to any great degree and most likely see a general drop off. The way attribute weighting works may show jumps in some areas which will be counter-balanced by drops elsewhere (overall CA should as said, remain consistent or drop, albeit in part dependent on a number of other factors - coaching/training facilities etc). Training is meant to use an overall balance in schedules to work over time, not be set in isolation. 

I agree it’s frustrating and could be made clearer in game, but if we’re doing things in an unrealistic manner or “without an overall balance in schedules to work over time, not be set in isolation” we can’t just blame the game.

Therefore doesn’t the “evidence from the videos” actually support SI’s training module design rather than finding a flaw?  Doesn’t it demonstrate that if we do something outside of the realistic design intent we get unrealistic results, which is exactly as expected?  So going right back to the thread title I’d suggest that no it’s not a bug, it’s working as intended and the conclusions being drawn are incorrect as the testers have done so without a full understanding of the game’s design using unrealistic inputs.

I’d be far more interested in results of evidence based testing which, over multiple seasons and multiple saves, uses realistic and balanced training programs which are balanced with the rest of the running of the club(s).

[Daveincid]

vor 3 Minuten schrieb herne79:

 

Why is it “absurd”?  Just because we want our players to do something doesn’t mean they will.  The training module has never been tell a player to train X and they do it, it’s way more complex than that.  As a (rhetorical) counter point - why are you, as the manager, constantly asking players to do something which they clearly aren’t capable of doing?  To quote Neil above, how is that “realistic”, bearing in mind that’s how training is designed?  Isn’t a manager who spends 10 seasons (or just 2 seasons without change) asking players to improve crossing without success absurd?

Re-read Neil’s post:

I agree it’s frustrating and could be made clearer in game, but if we’re doing things in an unrealistic manner or “without an overall balance in schedules to work over time, not be set in isolation” we can’t just blame the game.

Therefore doesn’t the “evidence from the videos” actually support SI’s training module design rather than finding a flaw?  Doesn’t it demonstrate that if we do something outside of the realistic design intent we get unrealistic results, which is exactly as expected?  So going right back to the thread title I’d suggest that no it’s not a bug, it’s working as intended and the conclusions being drawn are incorrect as the testers have done so without a full understanding of the game’s design using unrealistic inputs.

I’d be far more interested in results of evidence based testing which, over multiple seasons and multiple saves, uses realistic and balanced training programs which are balanced with the rest of the running of the club(s).

This

[riv3th3ad]

42 minutes ago, herne79 said:

Why is it “absurd”?  Just because we want our players to do something doesn’t mean they will.  The training module has never been tell a player to train X and they do it, it’s way more complex than that.  As a (rhetorical) counter point - why are you, as the manager, constantly asking players to do something which they clearly aren’t capable of doing?  To quote Neil above, how is that “realistic”, bearing in mind that’s how training is designed?  Isn’t a manager who spends 10 seasons (or just 2 seasons without change) asking players to improve crossing without success absurd?

Yes. It's absurd. It's not realistic that multiple professional players at a club with world class facilities and some of the best coaches in the Premier League can't improve crossing by half or 1 point.

You are 100% putting words in my mouth. I've not said I was asking a bunch of players to do a training for 10 years or even 2 seasons on a constant basis. I was simply using WB-At and crossing as an example because the tactic I'm currently running uses that role so I have 4 senior players as well as players in the U teams on that PRD and regularly cycling through crossing as an additional focus, and utilizing att overlap and play from the back sessions--certainly the young players ought to be able to improve crossing from 9 to 10 or 11 if they're training it (and getting great training ratings, I may add). I'm not asking for their crossing to go from 11 to 19 in 3 months. Crossing is not the only attribute I've seen behave this way. 

The point is that it's frustrating to know that session X may be having literally zero impact upon the attributes it lists. 

 

[herne79]

34 minutes ago, riv3th3ad said:

The point is that it's frustrating to know that session X may be having literally zero impact upon the attributes it lists. 

I agree it is frustrating.  In one of my saves I had Moukoko not develop at all in 3 seasons at my club - not a single attribute - despite what I’d consider a well balanced training program and a healthy amount of match time.  I ended up selling him.  Yet in a different save I turned him into a world class striker.

The training module is far more complex than we sometimes appreciate.  Some may (with reason) call it random or unrealistic.  Perhaps there is room to make a tweak to the system so that targeted attribute development does bear a greater resemblance to what we ask our players to do.  I don’t know, but if that were to happen it’d need to come from SI’s own close links and reviews with real life clubs and how they develop actual players.  

So when we talk of “unrealistic” attribute development is it really unrealistic or just our own view?  SI work closely with real clubs and their development module is based on those case studies.  Personally I have zero experience to say with any confidence what is or isn’t realistic, but I know SI do.  I imagine there’s room for improvement but that’s nothing more than my own supposition and knowing that SI are always looking at ways to improve the game in a realistic manner.

[Feron]

2 hours ago, Daveincid said:

This

again, this is why we said we are NOT basing anything on our personal experience. annectodal experience is just an addition to the findings in Max's videos where he literally has looked at hundreds of matches/players/training hours trying to make the playing field as level as possible with the help of the editor. 

are you really saying that all that means nothing? just because we might WANT our players to improve in a certain attribute?  sorry but this is insulting to us.  here is some of the findings again:

• all purple defending sessions train the exact same range of attributes
• all green attacking sessions train the exact same range of attributes 
• "att. corners" doesn't train the corner attribute - in fact most set piece sessions train the exact same range of attributes instead of those in the description
• all lilac technical sessions train the exact same range of attributes (only ball destribution doesn't really help with anything)
• both light green tactical sessions train the exact same range of attributes
• the extra curricular sessions don't boost the one attribute they claim to:  Teamwork

 How about you guys at bug fixing look at it without prejudice and THEN tell us it all is a gigantic fuss about nothing and we all just want our players to improve with blinders on? thank you

 

 

[Daveincid]

vor 7 Minuten schrieb Feron:

again, this is why we said we are NOT basing anything on our personal experience. annectodal experience is just an addition to the findings in Max's videos where he literally has looked at hundreds of matches/players/training hours trying to make the playing field as level as possible with the help of the editor. 

are you really saying that all that means nothing? just because we might WANT our players to improve in a certain attribute?  sorry but this is insulting to us.  here is some of the findings again:

• all purple defending sessions train the exact same range of attributes
• all green attacking sessions train the exact same range of attributes 
• "att. corners" doesn't train the corner attribute - in fact most set piece sessions train the exact same range of attributes instead of those in the description
• all lilac technical sessions train the exact same range of attributes (only ball destribution doesn't really help with anything)
• both light green tactical sessions train the exact same range of attributes
• the extra curricular sessions don't boost the one attribute they claim to:  Teamwork

 How about you guys at bug fixing look at it without prejudice and THEN tell us it all is a gigantic fuss about nothing and we all just want our players to improve with blinders on? thank you

 

 

I only agree with herne's view about this topic. That's all.

[riv3th3ad]

7 hours ago, herne79 said:

I agree it is frustrating.  In one of my saves I had Moukoko not develop at all in 3 seasons at my club - not a single attribute - despite what I’d consider a well balanced training program and a healthy amount of match time.  I ended up selling him.  Yet in a different save I turned him into a world class striker.

The training module is far more complex than we sometimes appreciate.  Some may (with reason) call it random or unrealistic.  Perhaps there is room to make a tweak to the system so that targeted attribute development does bear a greater resemblance to what we ask our players to do.  I don’t know, but if that were to happen it’d need to come from SI’s own close links and reviews with real life clubs and how they develop actual players.  

So when we talk of “unrealistic” attribute development is it really unrealistic or just our own view?  SI work closely with real clubs and their development module is based on those case studies.  Personally I have zero experience to say with any confidence what is or isn’t realistic, but I know SI do.  I imagine there’s room for improvement but that’s nothing more than my own supposition and knowing that SI are always looking at ways to improve the game in a realistic manner.

I understand that players in FM and in real life sometimes do not perform well in matches or do not improve despite training well.

The main takeaway is that if a training session says it works on attributes A, B, C but a test suggests that session in fact has zero impact on A, B, C and instead impacts attributes X, Y, and Z--that is a serious, serious problem that needs to be looked into by SI. 

Edited July 31, 2022 by riv3th3ad
typos

[glengarry224]

On 30/07/2022 at 17:39, riv3th3ad said:

It's a mistake to run test with both teams at 100% tactical familiarity.

Is it?  [serious question]  Counter argument:  we want to test the effect of the 1-match boosts and not test how well each 'match preparation' session improves tactical familiarity, therefore we maximize tactical familiarity to take that out of the equation, otherwise the results of the test cannot separate out how much the results were influenced by TF vs. match-boost?

[glengarry224]

On 31/07/2022 at 00:53, herne79 said:

Therefore doesn’t the “evidence from the videos” actually support SI’s training module design rather than finding a flaw?

I don't think so.  Neil's post does point out why the test might've been poorly constructed to test what it set out to test, but it was not constructed to test the overall module design, therefore it indicates nothing about that.

As far as we all see different results with the same players/training, I agree and anecdotes are useful only to come up with ideas for objective tests.  Though different results is variance and why it is important to properly construct a test, including with adequate sample sizes and statistical power, so that the effects of variance can be reduced (hopefully to insignificance).  It's the same with any test:  I might have a different overall clinical response to a particular new drug if it was possible to test it on me over and over again under the same circumstances (easier to retest in FM than in real life), and you and I would have different responses to that drug.

Ideally, one would have a null hypothesis, for example:  "training 'chance creation' does not train the attributes which the card indicates" and then properly construct a test to try to disprove that hypothesis.

[riv3th3ad]

19 hours ago, glengarry224 said:

Is it?  [serious question]  Counter argument:  we want to test the effect of the 1-match boosts and not test how well each 'match preparation' session improves tactical familiarity, therefore we maximize tactical familiarity to take that out of the equation, otherwise the results of the test cannot separate out how much the results were influenced by TF vs. match-boost?

You literally cannot isolate the match boost because there will be a difference in fatigue vs. the team that didn't do the training.

[glengarry224]

11 hours ago, riv3th3ad said:

You literally cannot isolate the match boost because there will be a difference in fatigue vs. the team that didn't do the training.

Then in the control group, is it better to schedule a session with similar fatigue effects but without the match boost?  Seems like that would introduce other factors.  Maybe compare 'defensive positioning' with 'attacking movement' and measure attacking stats vs defensive stats?  I really don't know.

And, regardless, is it desirable to delete tactical familiarity from the results, by maximizing it for both the control group and the test group?

[Xavier Lukhas]

@werogatda has uploaded a new video to his channel. Spoiler alert: there's no proposed training schedule yet as he wants to run further tests.
 

 

[Xavier Lukhas]

This time around, we do have the schedule.