Product Chat / TGC, please create "Performance testing suite!"

Author
Message
yrkoon
20
Years of Service
User Offline
Joined: 14th Jan 2004
Location:
Posted: 18th Apr 2017 11:33

The current case of user "Imhotep" shows a basic problem.

Every other week, we have someone like him ranting in fury about GG,
and we have kind moderators and other, experienced users bending over backwards
to soothe and help him, but the discussion often - if not always -escalates, nevertheless.

I think this has to do with the fact, that especially new users think they bought a solution (to the problem: "I want to show my friend a playable game, next weeek at the latest, that I made myself"), but they actually are getting a tool instead, in the first place.

Now, they would probably put up with a framerate of around 25-30 fps, but not with 13 fps, understandably, because they can't risk to show their friends a thing like that.
And, in a situation like this, the readiness to cooperate drops sharply on the user side, especially, if he is told to provide this data and then that and then the other, consecutively, because he - like his friends - is used to high-polished, AAA games performing well.

Psychologically, THIS is NOT the point in time to tell him that he needs to "optimize".

With only little exaggeration,
from the user's viewpoint, it's like he's coming to a hospital with signs of appendicitis, only to be told :
"The little room over there is for cases like yours,
you'll find a stretcher and two bags,
one with detergents and desinfectants,
and the other with scalpell and things.
Lay down in the proper position.
We'll tell you over the loudspeaker how to proceed further.
But excute cuts carefully, for your own sake."

No wonder he's crying out loud.

I think a better way to help were if we had a defined "performance suit", i.e. a collection of levels with well-defined conditions,
either delivered with GG or / and downloadable from an official location, say (hope, the names speak for themselves):
"1. The bare plain with one house, low visual settings ",
"2. The bare plain with one house, high visual settings",

"3. The empty trench, low visual settings ",
"4. The empty trench, high visual settings", "

"5. The hole in the ground with some objects, low visual settings",
"6. The hole in the ground with some objects, high visual settings",

"7. The forest with animated trees, low visual settings",
"8. The forest with animated trees, high visual settings",

"9. The oasis in the valley, low visual settings",
"10. The oasis in the valley, high visual settings",

(I'm convinced, the real GG performance experts here can think of more and/or provide better examples)

which should run "automatically", make the Player go along well defined paths and record data (such as framerate, tris etc.) as needed.
The user should neither be required nor be able at all to change the visual settings, in order to get consistent, comparable data

Outside of that, a reporting program should run which checks

a) on what drive and what location in the filesystem does GG live?
b) on what device type (HDD/SDD) is that drive located ?
c) what exact driver version and video device is GG talking to ?
d) how much memory is available ?
e) what is the CPU type used ?
f) What user type (admin or simple) is the user running GG
g) ...

This all should generate - at ONE command of the user - one or (at the most) two report files which the user then shall post to the forums
(or mail 'em to support, if he goes to support).

First off all, this is ONE action the user has to perform, not many.
Second: the measured data are independent of arbitrary positions and orientations of the player
Third: it will make computers in use - and the results received - comparable.

Over time, "we" will accumulate a collection of machine configurations, which will help giving more precise advice
(instead of, saying "Maybe, it's your antivirus software, often this kicks in, try another of your choice".
Users in distress just don't want to read the latter, because it has an inkling of "mate, it's all your fault" - to the user.
At least it will show the user where he stands with his gear, in comparison.
but if they see that comparable computers run significantly better - they might be more inclined to listen to advice to investigate deeper, if necessary)


Yes, there is some effort to go into this, but it is largely a one time effort
(perhaps to be adapted once a year), and I'm expecting it will save a lot of effort and nerves on both the enraged user's and the helpers from the forum's side.

And it can be used by TGC interenally before releasing a new version, because they'll see in advance what might come up out of userland - and point
out specific weaknesses of GG.

Just my two cents
Y(Maybe, Imhotep specifically can post a reply, whether he, too, thinks this would be helpful.


Lives of great men all remind us we may make our lives sublime
PM
arfur9
12
Years of Service
User Offline
Joined: 23rd Jan 2012
Location: Cold op norf UK
Posted: 18th Apr 2017 12:18
I think a lot of the problem is supposed to be an entry level game engine, and the minimum and recommended spec reflects that, but the reality is you need a lot better GPU than stated.
A lot of the problem I Lee and the testers have way better computers than the average Joes, there's no demo so people will get pee'd off if their computer should be fine in theory
lotgd
3D Media Maker
14
Years of Service
User Offline
Joined: 2nd Apr 2010
Location: italy
Posted: 18th Apr 2017 12:48
I think many problems could be solved using direct x 11 or 12 in the engine.

My Pc Specs : Win10pro64bit /Msi z97 / i74770k / gtx1070 / 16gb ram / SSD 850Evo
LeeBamber
TGC Lead Developer
24
Years of Service
User Offline
Joined: 21st Jan 2000
Location: England
Posted: 18th Apr 2017 14:50
@yrkoon : I was thinking along similar lines, that when GameGuru promises 'easy game making for everyone', it can easily be interpreted as "your game will be finished in five minutes and run as fast as you expect it to". Microsoft used to do a 'performance profiler' which would measure your PC and give you a rating compared to other data they gathered, to give you an idea how up to date your system was. I was thinking something along the lines of a 'performance profiler' that runs when GameGuru first launches, tests the system CPU, graphics, storage and memory, then reports a score based on other users, and a recommendation, which could even be to seek a refund if the system is below minimum specifications. Of course, this does not derail my plans to improve core performance, but it might help some users avoid a bad experience while the product is still being optimized.
PC SPECS: Windows 8.1 Pro 64-bit, Intel Core i7-5930K (PASSMARK:13645), NVIDIA Geforce GTX 980 GPU (PASSMARK:9762) , 32GB RAM

smallg
Community Leader
18
Years of Service
User Offline
Joined: 8th Dec 2005
Location:
Posted: 18th Apr 2017 15:26
Nice idea, perhaps it could also auto generate the settings based on the results - setting low, medium or high etc
Also a way to toggle super flat terrain for the editor (in the editor) would be a good option, it's quite common to use GG to build in door levels and if you already have low fps then the terrain is a huge drain (sure you can hide it or fix it for the end game but its not fun trying to actually create your level with a low fps in the first place).
lua guide for GG
https://steamcommunity.com/sharedfiles/filedetails/?id=398177770
windows 10
i5 @4ghz, 8gb ram, AMD R9 200 series , directx 11
yrkoon
20
Years of Service
User Offline
Joined: 14th Jan 2004
Location:
Posted: 19th Apr 2017 15:11
@Lee hmmm.... I wonder whether a performance profiler not employing real scenes/level could do a good job.

It can deliver basic data, yes, as I suggested
[quoting myself]
Outside of that, a reporting program should run which checks

a) on what drive and what location in the filesystem does GG live?
b) on what device type (HDD/SDD) is that drive located ?
c) what exact driver version and video device is GG talking to ?
d) how much memory is available ?
e) what is the CPU type used ?
f) What user type (admin or simple) is the user running GG
g) ...

[end quoting myself]

I was NOT pushing in the direction that a new user shall really get a "good" game within 5 minutes or five days or ANYTHING remotely near that time span.
But I also think they don't really expect an AAA game out of GG, but something that can at least be presented to their friends, possibly pampered WITH a comment from them that it at least "runs fairly, not barely, but cannot be expected to match a polished AAA game. " Getting them ito such a state is crucial for them WILLING to be listening to optimizing advice.

It's the same with the flak usually received by poor wretched newbies from experienced developers when they innocently DARE to use the term "MMORG" ... they DON'T really want to server 1000s of users, but may be a dozen at the most, just their entourage, nevertheless, they usually get CRUSHED by such devs.


Now, I rather have a feeling that those who are trying to help often make educated guesses from their experience,
but if one tries to walk in the new user's shoes, he's bombarded with requests for details (all somehow necessary, no doubt) he might not even understand when he's calm, let alone enraged. If he can runs a test suit, he'll see that he is actually spared to do this that and the other, and the figures appearing might

But most of those details can be determined by his machine, and if they are, he "just" has to post one report to show them all. AND if he can inspect the reports of others, he might see himself that his hardware is possibly weak - maybe, only in places.

If he watches those levels being run, he may get a feeling how things are, like , say, a "Wood level" with 30 static trees in one corner and then another with 30 trees with moving branches, the corners separated visually from one another .
Of course, much of this is in the tutorial videos, but again, it takes the user some effort he might not be willing to take in his current state.

AND: the levels could well be designed by the community here, not by poor Lee with another big task on his desk

As we long time users all know, a few things set or not set can make all the difference, and sometimes, even turning the player's back can make differences.
Lives of great men all remind us we may make our lives sublime
PM

Login to post a reply

Server time is: 2024-11-24 14:11:51
Your offset time is: 2024-11-24 14:11:51