Suggestion: offer alpha testing roles to some forum members

[pauly15]

There are a number of people who've done great analysis on the current ME on the forums. @Mensell76, @Amarante, @Armistice are a few from the top of my head. 

Several serious issues passed by the alpha testing phase which were picked up and clearly defined by people like this within hours of release. Once the game is largely defined, these issues must be harder to fix. 

The attacking movement, central play, through balls, crossing issues were glaring problems that made it into the release ME, and show some holes in your alpha testing process. Having people who have shown they are able to analyse in depth and pinpoint issues can only be a good thing. If they want to help, which it seems they do, bring them on board for FM20.

[FrazT]

There already exists a private beta testing team which members are invited by SI to participate in.  These testers identified many areas of the ME that required attention and just because it wasn't fully fixed for the release, does not mean that it wasn't identified.  ME fixes are notorious for causing consequential issues and do require considerable testing before implementation.  I am sure that the SI staff will be aware of the users who have been helpful this year.

[pauly15]

58 minutes ago, FrazT said:

There already exists a private beta testing team which members are invited by SI to participate in.  These testers identified many areas of the ME that required attention and just because it wasn't fully fixed for the release, does not mean that it wasn't identified.  ME fixes are notorious for causing consequential issues and do require considerable testing before implementation.  I am sure that the SI staff will be aware of the users who have been helpful this year.

Hi FrazT, 

My point is that once we hit beta, large parts of the ME are fairly set. Making big changes tricky due to the knock ons as you say. It's like playing Jenga 20 rounds in;)

I struggle to believe that the crossing, attacking movement and central play problems were "identified but not fully fixed", as after several patches there's been no improvement. The attempted improvement in one of the early patches tweaked passing directness overall and made things much worse. 

I can't really comment from the outside, but to me it looks like these issues had simply not been noted. The idea that they were noted and were being worked on doesn't fit to me. 

If that is the case, the alpha testing process can definitely be improved and my suggestion is a free way to get (more) good people in during the alpha phase.

Fans have been clambering for more ME focus from the developers for years, this is a cheap way to do that using the skills of dedicated people who know the game and football too.

[FrazT]

As you have admitted, you have no knowledge of how the testing procedures, both in-house or in the private beta work or the coding issues that arise during a testing process, so it is wrong to suggest that issues with the ME were not noted, because they were.

There is never any doubt that testing procedures and the ME can and will be improved and I am sure that SI will note your suggestions.

[pauly15]

31 minutes ago, FrazT said:

As you have admitted, you have no knowledge of how the testing procedures, both in-house or in the private beta work or the coding issues that arise during a testing process, so it is wrong to suggest that issues with the ME were not noted, because they were.

Whether the issues were known and being worked on or not is something developers will know and review. If they weren't, I think certain people have shown they have something good to add and hope SI consider using them as a resource. 

Out of interest, do you know they were for a fact?

Cheers

[Amarante]

A think what @pauly15 is alluding to is that and this might come out wrong, the alpha testers i believe are testing functionality but not necessarily realistic behavior, patterns, and tactical movement. 

What we are doing is testing out not functionality and that it's stable but tactical problems patterns in aim for a more realistic ME.  Granted we don't know the testing process but it still baffles as to how some of the things spotted in beta/open beta was able to go live. As a comp science man my self. i believe the alpha testers have a huge amount of work so they were not able to spot or maybe(pure speculation) not tactical astute with the game to realize that this pattern and behavior is wrong. 

 

I do think open-beta is the closest we will come so while we will divide our saves between enjoying the game and fixing the game. If SI sends us oranges instead of apples we'll send it right back as i doubt we will be able to join the bunch that say hey these are oranges not apples. 

[FrazT]

The issues mentioned by @pauly15 and many more were discussed in the private beta test forums.  These testers test the whole game and are not structured to test for just specific issues, so many problems are and will be found, in all areas of the game.  Obviously, the more users that test the game, the more issues large and small will be found, so that in itself is of value.  Regardless of numbers, however, there will always be some issues found after full release that have not been found before- some because of their obscurity and some because they have been difficult to reproduce or tune the code after fixing.  The testers can only highlight an issue but after that do not know how easy or complex to fix they are and particularly with the ME, the consequences of making fine adjustments

[Mensell76]

11 hours ago, FrazT said:

The issues mentioned by @pauly15 and many more were discussed in the private beta test forums.

If that is true how come no one has told us that yet. @pauly15 and many others like myself have been working hard showing SI several issues which apparently have been under the attention already? Or were these brought under the attention by us and hadn' t been spotted by alpha or private beta testers before?

[RTHerringbone]

Here's the deal: Alpha testers test the whole game, but there are "specialists" who focus more on the Match Engine than anything else. There is a mixture of people therein and I'd say a large proportion of them have at least of 15+ years of experience with the game. There are testers from all over the world, so there's a good mix of personal playing preferences, and people with very strong opinions on the various phases of play.

That testing team saw things that you won't even have seen, as issues they flagged were documented and fixed for release. Everything raised in the Public Bugs Forum has been raised internally. You guys are all very good, observant players, but please don't imagine that you represent elite-level analysts beyond the resources that SI already have at their disposal - that's actually quite insulting.

The reality is that, as FrazT has already stated, some stuff is just hard to fix.

18 hours ago, pauly15 said:

I struggle to believe that the crossing, attacking movement and central play problems were "identified but not fully fixed", as after several patches there's been no improvement. The attempted improvement in one of the early patches tweaked passing directness overall and made things much worse. 

The bold bit - this is exactly the point. All these things were picked up, but things like attacking movement and central play are not inconsequential. They're big old elements to sensitively refine, so it's to be expected that areas like these take longer to tune.

You guys do a great job and I can understand your frustrations, but please understand that the game is complicated and logging issues isn't a guarantee that a fix can be rapidly coded, tested and released. Similarly, SI might not reply to threads as often as you'd hope, but they do read everything and are working as hard as they can.

Keep it up 

[Mensell76]

57 minutes ago, RTHerringbone said:

Here's the deal: Alpha testers test the whole game, but there are "specialists" who focus more on the Match Engine than anything else. There is a mixture of people therein and I'd say a large proportion of them have at least of 15+ years of experience with the game. There are testers from all over the world, so there's a good mix of personal playing preferences, and people with very strong opinions on the various phases of play.

That testing team saw things that you won't even have seen, as issues they flagged were documented and fixed for release. Everything raised in the Public Bugs Forum has been raised internally. You guys are all very good, observant players, but please don't imagine that you represent elite-level analysts beyond the resources that SI already have at their disposal - that's actually quite insulting.

The reality is that, as FrazT has already stated, some stuff is just hard to fix.

The bold bit - this is exactly the point. All these things were picked up, but things like attacking movement and central play are not inconsequential. They're big old elements to sensitively refine, so it's to be expected that areas like these take longer to tune.

You guys do a great job and I can understand your frustrations, but please understand that the game is complicated and logging issues isn't a guarantee that a fix can be rapidly coded, tested and released. Similarly, SI might not reply to threads as often as you'd hope, but they do read everything and are working as hard as they can.

Keep it up 

First of all, I am not even about to compare myself with an SI alpha tester let's make that clear. I have the utmost respect for the work being done on an ME that is complex beyond reason. And to add to that, I am not looking to be an alpha tester or join the private beta group. I am not looking for a hug or a compliment either so let's get that out of the way. I am only offering my help because a) I would like more return on investment  on a currently extremely poor FM iteration and b) SI more or less wanted our help by setting up the public beta which sort of implies that their resources currently aren't covering all area's as wanted.

So now to get into your reaction. It has quite a bit of irony in it, in the sense that if all what you say is true SI is indeed acting like an elitist group. All issues raised here were already raised internally you say and SI has all the resources they need. Yet they ask for our help on a public beta but fail to find it noteworthy that they were already aware of some complex ME issues such as attacking movement and center play. And that basically with the resources, experience, personal playing preferences and strong opinions they already have inhouse, SI was not specifcally looking for an extra set of hands on these issues anymore. 

It would actually quite frankly be insulting to ask paying customers to help out, then to never really dedicate the time to insightfully reply to specific hot threads on the bugs forums and then, when someone dares to suggest lending a free hand for future releases, to reply that SI was of course already aware of such issues being the elite level analysts they are. At the end of the day, it is just only our spare time SI would be toying with hey, nothing that hurts them.

Luckily it is you behind these words and not an SI official so I am not about to hold SI accountable for this post. So by all means keep it up right back at you!

[pauly15]

1 hour ago, RTHerringbone said:

Here's the deal: Alpha testers test the whole game, but there are "specialists" who focus more on the Match Engine than anything else. There is a mixture of people therein and I'd say a large proportion of them have at least of 15+ years of experience with the game. There are testers from all over the world, so there's a good mix of personal playing preferences, and people with very strong opinions on the various phases of play.

That testing team saw things that you won't even have seen, as issues they flagged were documented and fixed for release. Everything raised in the Public Bugs Forum has been raised internally. You guys are all very good, observant players, but please don't imagine that you represent elite-level analysts beyond the resources that SI already have at their disposal - that's actually quite insulting.

The reality is that, as FrazT has already stated, some stuff is just hard to fix.

The bold bit - this is exactly the point. All these things were picked up, but things like attacking movement and central play are not inconsequential. They're big old elements to sensitively refine, so it's to be expected that areas like these take longer to tune.

You guys do a great job and I can understand your frustrations, but please understand that the game is complicated and logging issues isn't a guarantee that a fix can be rapidly coded, tested and released. Similarly, SI might not reply to threads as often as you'd hope, but they do read everything and are working as hard as they can.

Keep it up 

Sorry, I had to laugh at this.

I don't doubt that they're very good at what they do, but I see two alternatives:

1. Issues from the original beta like:

Absurd overabundance of wide play.

Stationary strikers.

And less than 1/100 passes from the AMC going forward centrally.

Were either known, being worked on and released anyway...

Or,

2. Weren't seen, or the severity wasn't realised. 

Judging by the first patch which attempted to improve through balls amongst other things, and which negatively affected passing length all over the pitch... I'd say they were surprised by the above and or its severity. 

For that reason, I would say from the outset that their process for alpha testing is not as good as it could be. 

[RTHerringbone]

I didn't say that SI have the resources they need, I was responding to comments herein which insinuated that the issues flagged in this thread hadn't already been observed and reported.

A public Beta allows SI to get more examples of instances that need attention. That's presumably why they've rolled out a public Beta for the first time because more .pkms equals more code to refer to and more chances to understand what the underlying problems are. They absolutely wanted extra sets of hands on these issues.

[pauly15]

To be clear, I'm talking about improving processes to better the original beta release. Once that is released, being specifically tweaked to integrate a bunch of new features, it must be much harder to make bigger changes to the game. 

[Mensell76]

@RTherringbone. If I am correct Rashidi has always been part of the private beta testing group right? Or is he even an alpha tester?

[grff]

SI would've been aware of issues that needed fixing during the alpha stage. However, a private beta helps them to obtain save game/pkm files to be able to look at the coding in multiple examples.

Many of the public beta threads were raising things that had already been raised in the private beta (I was a tester) but that doesn't mean to say that fixes weren't already in the process of being rolled out.

[pauly15]

20 minutes ago, grff said:

SI would've been aware of issues that needed fixing during the alpha stage. However, a private beta helps them to obtain save game/pkm files to be able to look at the coding in multiple examples.

Many of the public beta threads were raising things that had already been raised in the private beta (I was a tester) but that doesn't mean to say that fixes weren't already in the process of being rolled out.

Ok, well, can we improve on the fact that 1 month after full release, and 6 weeks after beta release, the game is still in an (undeniably) extremely poor state where the above problems are concerned? 

Is the best process to release a game with the above problems because they need PKMs from a wide audience? I sincerely doubt it. I think this is why a number of people wait for the winter update before buying. They can do better than that. I am merely suggesting one possible and free way to improve things. 

Also, perhaps my terminology is off. By alpha what I mean is just 'well before the release of the beta'. I don't mean some central part of the coding or anything. 

What I'm sure of, is that releasing a game that is (for me and many others) not enjoyable until January because of some really basic play issues.... can be improved upon. If what you're saying is true and the issues and their severity were known, then it becomes a question of ethics. If you're releasing a product with known serious problems that requires beta work for over 6+ weeks, you should label it as such. 

[Mensell76]

Regarding the lack of attacking movement by strikers the official answer by SI on November 9th, after receiving quite a few examples was:

- we shall be taking a closer look at attacking movement

On the lack of forward through balls by an AMC (or mc) Rashidi stated last tuesday:

- If you think its an issue you need to post your pkm, cos at the moment by AMC is blistering the middle of the park with passes. It could be your combination of roles and duties that SI need to take a look at.

Of course when we asked him to show us pkm's and examples of solid central play, all went quiet. While it could actually prove a great learning case for us. And the same goes for SI, if we are clearly wrong about something, point us in the right direction.

Both answers give no indication whatsoever that SI was already very much aware of these big issues. If they were, these statements are actually misinforming us and is precisely what I have been adamant about in recent weeks: If SI is aware of huge issues, be transparant about it, ask for more specific situations, give more insightful information as to where things stand. Again: communicate with us.

I know that SI does communicate with its private beta testers as I have also been one of them in the past. Opening a public beta forum i.m.o actually requires even more a quid pro quo situation since you are now basically asking your investors to help out as well.

 

• 
   

 

 

[RTHerringbone]

27 minutes ago, Mensell76 said:

@RTherringbone. If I am correct Rashidi has always been part of the private beta testing group right? Or is he even an alpha tester?

Correct, but there are several other posters (you can see them all in the credits actually) whose names may be less familiar. It's actually many of the names you may not necessarily know who are often the most passionate and dogged testers.

[Mensell76]

1 minute ago, RTHerringbone said:

Correct, but there are several other posters (you can see them all in the credits actually) whose names may be less familiar. It's actually many of the names you may not necessarily know who are often the most passionate and dogged testers.

I know and their work is much appreciated. But just to put things in perspective, Rashidi started adding examples to my thread on lack of off the ball movement by attackers on november 10th stating there is a "current issue", and on the AMC lack of forward through balls thread he stated that his AMC is blistering through the center of the park.

Not trying to pin you into a corner, but it sincerely doesn't look like SI or Rashidi had already been aware of these issues. And yet again if they were, why would Rashidi start adding examples on my thread while he must have done a dozen of times in the private beta and surely has a direct line with SI/DEVS/Elite analysts that we do not have. 

[grff]

39 minutes ago, pauly15 said:

Ok, well, can we improve on the fact that 1 month after full release, and 6 weeks after beta release, the game is still in an (undeniably) extremely poor state where the above problems are concerned? 

Is the best process to release a game with the above problems because they need PKMs from a wide audience? I sincerely doubt it. I think this is why a number of people wait for the winter update before buying. They can do better than that. I am merely suggesting one possible and free way to improve things. 

Also, perhaps my terminology is off. By alpha what I mean is just 'well before the release of the beta'. I don't mean some central part of the coding or anything. 

What I'm sure of, is that releasing a game that is (for me and many others) not enjoyable until January because of some really basic play issues.... can be improved upon. If what you're saying is true and the issues and their severity were known, then it becomes a question of ethics. If you're releasing a product with known serious problems that requires beta work for over 6+ weeks, you should label it as such. 

Severe issues would've been fixed in the alpha stage. A beta is then released to iron out bugs in the same way early access games are released.

With how complex the ME coding is, changing certain aspects would have a knock on affect so things like the AMC may not have been an issue in the early stages until something else was adjusted in the ME.

Also further builds to the ME will be in the works, even if not released yet.

[RTHerringbone]

Just now, Mensell76 said:

Not trying to pin you into a corner, but it sincerely doesn't look like SI or Rashidi had already been aware of these issues. And yet again if they were, why would Rashidi start adding examples on my thread while he must have done a dozen of times in the private beta and surely has a direct line with SI/DEVS/Elite analysts that we do not have. 

Rashidi has been extremely close to the development cycle in the past, but that history doesn't automatically mean that SI take his opinion as gospel and rule out any changes if he / Cleon / anyone else "of that calibre" (my words) say in a GD thread that something isn't an issue. In my experience, nobody is more constructively critical of the ME than the SI team themselves.

The issues of central play and attacking movement are just there - everyone can see them. They have been logged in many places and for a period of time, and the best minds at SI will be busily trying to balance things for the better.

[Mensell76]

7 minutes ago, RTHerringbone said:

Rashidi has been extremely close to the development cycle in the past, but that history doesn't automatically mean that SI take his opinion as gospel and rule out any changes if he / Cleon / anyone else "of that calibre" (my words) say in a GD thread that something isn't an issue. In my experience, nobody is more constructively critical of the ME than the SI team themselves.

The issues of central play and attacking movement are just there - everyone can see them. They have been logged in many places and for a period of time, and the best minds at SI will be busily trying to balance things for the better.

Okay fair enough, appreciate your straight answer.

[pauly15]

13 minutes ago, grff said:

Severe issues would've been fixed in the alpha stage. A beta is then released to iron out bugs in the same way early access games are released.

With how complex the ME coding is, changing certain aspects would have a knock on affect so things like the AMC may not have been an issue in the early stages until something else was adjusted in the ME.

Also further builds to the ME will be in the works, even if not released yet.

I would class the issues in the original beta as serious. They're not game breaking like that one release ~5 years ago where offsides where ridiculous, but they're enough for a number of people to pause playing. 

Despite that, I understand what you're saying. Any change is going to have knock on effects etc.

None of that changes that routinely releasing a game that is badly flawed for months after release is not a process that you should cease trying to improve. 

[themadsheep2001]

41 minutes ago, RTHerringbone said:

Rashidi has been extremely close to the development cycle in the past, but that history doesn't automatically mean that SI take his opinion as gospel and rule out any changes if he / Cleon / anyone else "of that calibre" (my words) say in a GD thread that something isn't an issue. In my experience, nobody is more constructively critical of the ME than the SI team themselves.

The issues of central play and attacking movement are just there - everyone can see them. They have been logged in many places and for a period of time, and the best minds at SI will be busily trying to balance things for the better.

Also should note: Those on the private beta can't actually mention what they have seen, because the private alpha/beta is subject to non disclosure agreements. So the reason you don't see mention is because no one is allowed to talk about it without permission. 

So get finish the final point about the request: it already exists and adding the names above would have done nothing to change outcome.

[pauly15]

9 minutes ago, themadsheep2001 said:

 the final point about the request: it already exists and adding the names above would have done nothing to change outcome.

If that is the case it becomes a question of ethics. If you need a longer beta period to produce a game without serious issues, you should have one, and label it as such. 

I can't imagine what you're saying is true. If it were, then basically the above people have done a month+ of detailed analysis when all they needed to do was be told about issues and that PKMs and examples were required. In that reality, SI would have done nothing to communicate that anything other than PKMs for these known issues were effectively a waste of the player's time.

[themadsheep2001]

6 minutes ago, pauly15 said:

If that is the case it becomes a question of ethics. If you need a longer beta period to produce a game without serious issues, you should have one, and label it as such. 

I can't imagine what you're saying is true. If it were, then basically the above people have done a month of detailed analysis when all they needed to do was be told about issues and that PKMs and examples were required. 

It doesn't matter if you can't imagine what I'm saying is true. It is true. 

And again wrong, because more evidence is always better. Because you might see different issues after updates, you might be seeing the same issues after updates.

I mean this is how beta testing works. Its constant. The above names have been doing it for a month. Those on the beta have been doing for several months more, and churning out a lot more threads in a private beta

[Mensell76]

10 minutes ago, themadsheep2001 said:

you might be seeing the same issues after updates.

I mean this is how beta testing works. Its constant. The above names have been doing it for a month. Those on the beta ahve been doing for several months more, and churning out a lot more threads in a public beta

At least the private beta testers are/were being informed by SI.

So there we are again. SI knowing of certain huge issues and not once levelling with us while we invested so much time, but even misinforming us and leading us on.

Appalled by this disrespectul uncustomer friendly attitude. Done with it as off now.

[pauly15]

13 minutes ago, themadsheep2001 said:

It doesn't matter if you can't imagine what I'm saying is true. It is true. 

And again wrong, because more evidence is always better. Because you might see different issues after updates, you might be seeing the same issues after updates.

I mean this is how beta testing works. Its constant. The above names have been doing it for a month. Those on the beta ahve been doing for several months more, and churning out a lot more threads in a public beta

The reason I can't imagine it's true is that.... I prefer to believe these are problems of process rather than problems of integrity. I get that the game is a business and it can't be released with an issue/bug log, but to suggest that they say nothing and allow people to do hundreds of hours of analysis unearthing known issues... 

[themadsheep2001]

6 minutes ago, pauly15 said:

The reason I can't imagine it's true is that.... I prefer to believe these are problems of process rather than problems of integrity. I get that the game is a business and it can't be released with an issue/bug log, but to suggest that they say nothing and allow people to do hundreds of hours of analysis on known issues... 

Known issues doesn't mean the work stops, or that new information isn't valuable. When SI don't need more information, they put that out on the relevant thread. There is rarely a time when more information isn't needed. Even now, more examples help refine future updates. It's not a black and white tick box and never has been. Which is why testing is endless, and often thankless, and why people are handpicked for this. I always say it, but I'm not convinced many people really get the true nature of beta testing

[pauly15]

Just now, themadsheep2001 said:

Known issues doesn't mean the work stops, or that new information isn't valuable. When SI don't need more information, they put that out on the relevant thread. There is rarely a time when more information isn't needed. Even now, more examples help refine future updates. It's not a black and white tick box and never has been. Which is why testing is endless, and often thankless, and why people are handpicked for this.

The ethical thing to do would be: 

In the beta and public beta threads, have a log of known issues that require PKMs and examples. 

This would obviate the need for people to write explanations and analysis of issues that SI are already aware of the existence of. It would make it clear that the time spent doing this detailed analysis, rather than just providing PKMs of known issues, was/is a waste of that person's time...

[themadsheep2001]

Just now, pauly15 said:

The ethical thing to do would be: 

In the beta and public beta threads, have a log of known issues that require PKMs and examples. 

This would obviate the need for people to write explanations and analysis of issues that SI are already aware of the existence of. It would make it clear that the time spent doing this detailed analysis, rather than just providing PKMs of known issues, was/is a waste of that person's time...

That's releasing privileged information and is unlikely to happen. That's why there is a private beta.

Do the public beta because you want to, or because you enjoy it, and you want to help. But that doesn't mean they are going to release privileged information. If that doesn't work for you, that's also absolute fine too.

[pauly15]

1 minute ago, themadsheep2001 said:

That's releasing privileged information and is unlikely to happen. That's why there is a private beta.

Do the public beta because you want to, or because you enjoy it, and you want to help. But that doesn't mean they are going to release privileged information. If that doesn't work for you, that's also absolute fine too.

A bizarrely uncooperative approach that looks at those trying to help as enemies from whom information must be hidden.

[themadsheep2001]

2 minutes ago, pauly15 said:

A bizarrely uncooperative approach that looks at those trying to help as enemies from whom information must be hidden.

Its not at all actually

This is now no longer a features request since the feature exists and we're going in circles. 

I'll tag @Neil Brock in if he can offer more information, as beyond this no else who is on the private beta can go into more detail without breaking NDAs, and anyone hasn't actually experienced the private beta isn't in the best position to leave feedback either