Product Chat / Memory problems

Author
Message
Teabone
Forum Support
17
Years of Service
User Offline
Joined: 8th Jun 2006
Location: Earth
Posted: 13th Dec 2017 23:59 Edited at: 14th Dec 2017 05:56
Okay so if for example I made a game that had 3 levels and packed and only just working for me on my video card, and someone had a lower model GPU... is it possible they will get an error when reaching that 3rd level?
Twitter - Teabone3 | Youtube - Teabone3 | Twitch - Teabone3 | TGC Assets - Store Link | Patreon - Teabone3

i7 -2600 CPU @ 3.40GHz - Windows 7 - 8GB RAM - Nivida GeForce GTX 960
Earthling45
7
Years of Service
User Offline
Joined: 3rd Sep 2016
Location: Zuid Holland Nederland
Posted: 14th Dec 2017 03:45
With a lower gpu model, the performance will of course be lower than on your system but that doesn't mean errors when reaching a thirth level.

The errors occur when the memory limit is reached.
That does not have to be the case with a three level game.
I've filled three maps with some stuff and reached 2300 mb when the thirth level was reached and loaded.
If both of you have 4 gb of video memory, there is no problem, even if one card has only 2 gb it is not so dramatic that errors will occur, but there will be a performance hit.
In the view tests i've posted you see that when the limit of the video memory is reached, the rest is loaded into the system memory, and if there is enough room, there should be no question of errors, just a hit on performance.

When you reach the limit of your system memory, the same thing happens, the excess data then is loaded into the pagefile of the harddisk, this is very noticable because the system will be very slow and errors starts to occur.

I hope i've explained this properly, if not please correct me
Earthling45
7
Years of Service
User Offline
Joined: 3rd Sep 2016
Location: Zuid Holland Nederland
Posted: 14th Dec 2017 04:06
Quote: "Look up TechpowerUp GPU-Z and download. you have CPUID open in picture above this is mainly for CPUs not GPUs."


Thanks, i've found it and going to test with it tonight
For the most part i'm well clear of 4gb which is on my card, but with the chess script it is exceeded and the village level did exceed it, but i will run a test with the village map again using GPU-Z.

I'll restart the system, i've not really bothered about doing that because after closing the game file the video memory is cleared.
AmenMoses
GameGuru Master
8
Years of Service
User Offline
Joined: 20th Feb 2016
Location: Portsmouth, England
Posted: 14th Dec 2017 09:14 Edited at: 14th Dec 2017 09:22
This is really interesting, pics of GPU-Z output with my asteroids demo vs one with garbochess.

As you say. virtually empty map but vram full!

(note that I restarted GG between tests and garbochess isn't actually playing yet, have it on player vs player mode)

(ooh, just noticed I've got another badge! It's Master AmenMoses now. )
Been there, done that, got all the T-Shirts!

Attachments

Login to view attachments
PM
Earthling45
7
Years of Service
User Offline
Joined: 3rd Sep 2016
Location: Zuid Holland Nederland
Posted: 14th Dec 2017 15:08
Quote: "If you have a Nvidia Card GPU-Z won't show you the Dynamic Memory only Dedicated Memory. "


It's best for me to stick to HWM64 because it also shows the dynamic memory.

This is from the chessmap after a fresh start.

Will test the village map again DX11with pbr=0 and pbr=1 and DX9 after a fresh restart.

Attachments

Login to view attachments
Earthling45
7
Years of Service
User Offline
Joined: 3rd Sep 2016
Location: Zuid Holland Nederland
Posted: 14th Dec 2017 19:18
Quote: "(ooh, just noticed I've got another badge! It's Master AmenMoses now. )"


Well earned, Master.

MK83
GameGuru TGC Backer
17
Years of Service
User Offline
Joined: 10th Jun 2006
Location: Greeneville, TN USA
Posted: 14th Dec 2017 21:38
Quote: "(ooh, just noticed I've got another badge! It's Master AmenMoses now. )"
Awesome
AMD Phenom x4 9850 2.70 Ghz , 6 Gb ram, 2GB EVGA Geforce GTX 750, Win 10 x64 Passmark score: 3091



PM
Jerry Tremble
GameGuru TGC Backer
11
Years of Service
User Offline
Joined: 5th Nov 2012
Location: Sonoran Desert
Posted: 14th Dec 2017 22:11
Quote: "(ooh, just noticed I've got another badge! It's Master AmenMoses now. )"


Nice job, Mister Master!
Desktop: i7 4770@3.4Ghz (passmark 9809), 12GB RAM, Win 10/64, GeForce GTX 1080 (passmark 12006), 1TB SSD, 1TB HDD; Laptop: i7 4800MQ@2.7Ghz, 16GB RAM, Win 10/64, GeForce GTX870M , 1TB SSD.
PM
Earthling45
7
Years of Service
User Offline
Joined: 3rd Sep 2016
Location: Zuid Holland Nederland
Posted: 15th Dec 2017 00:47
i've removed the light maps from the village map and made three new files to test.
What stands out is of course the loading time which is hugely improved in the DX11 public preview.
Even better if pbroverride=1 like it is default, when it is set to 0, it adds another 10 percent of load.
the framerate is somewhat lower in DX11, i've measured this after placing the player character in front of the gate.
DX9 17 fps
DX11 pbroverride=0 13 fps
DX11 pbroverride=1 15 fps (default setting)
Between each test the system was restarted.

My graphics card is a GTX 960 with 4gb.

DX9


DX11pbr=0


DX11pbr=1 (default)


My plan now is to make a stand alone file with ten levels and run the same test while going from level 1 to 10 and also keeping an eye on the memory usage of the editor itself.
AmenMoses
GameGuru Master
8
Years of Service
User Offline
Joined: 20th Feb 2016
Location: Portsmouth, England
Posted: 15th Dec 2017 11:25 Edited at: 15th Dec 2017 11:28
Ok, so my normal reaction to being baffled by something is to get the manuals out. Attached is a script that will show the memory usage of Lua in kilobytes updated every second. (attach it to an always active entity, the memory value will be displayed 'on' the entity itself so put it where you can see it in game)

Using this script with garbochess running on level 6 CPU vs CPU showed about 1Mb of memory in use, which is sort of what I would have expected.

So now I'm even more baffled, GPU-Z says Gbs in use Lua reports a mere Mb. So what is gobbling up the vram?

Maybe Lee has some clue because it has me stumped.

For comparison an empty map shows around 1Gb vram usage and about 100Kb for Lua.

The (Dynamic) bar on GPU-Z actually shows a steady 50Mb and the GPU Load is barely registering, I also noticed that even starting a new level doesn't reduce the (Dedicated) value, so I assume once GG has grabbed vram it doesn't free it until you quit completely (or is that the GPUs job to figure out?).

Still can't figure out why chess set and pieces, with all entities active but no scripts assigned give ~2Gb (Dedicated) reading whereas attach scripts and it jumps to >3Gb.
Been there, done that, got all the T-Shirts!

Attachments

Login to view attachments
PM
Earthling45
7
Years of Service
User Offline
Joined: 3rd Sep 2016
Location: Zuid Holland Nederland
Posted: 16th Dec 2017 01:26 Edited at: 16th Dec 2017 12:43
The tutorial maps provided me a nice base for a multilevel stand alone file which contains 10 levels.
I've attached a radar script and some soldiers in a few levels.
Each level has a teleporter so it is clear where the winzone is and on the first level there are three bunkers placed.
All in all, not much more than was already in place on the maps and there are no light maps.
While advancing through the levels in the stand alone file, the memory is filling up with each level that is loaded, when i reached level ten, the memory was 98 percent full.

Edit: it seems that i reach the limit at level 6.
Will do another test to be sure.

Attachments

Login to view attachments
Bugsy
15
Years of Service
User Offline
Joined: 24th Nov 2008
Location: Savannah
Posted: 16th Dec 2017 03:56
this is the most necessary thread ever. here's my 10 cents

Quote: "Consoles and some PC game engines handle things differently, by allocating a large chunk of the heap and then manage this heap directly for all game data and resources. It is then easy to say this heap is for 'level one', use it to the max, then discard the whole heap ready for 'level two'. Unfortunately, to implement this would require taking over quite a large chunk of aforementioned systems with some kind of custom memory manager and I would not even know where to start on a mission of that magnitude. "


Perhaps before new stock media and graphical improvements like "PBR" and whatnot is included, this should have all attention devoted to it. As it is: it's one of the only things stopping a competent studio from creating a game that advertises GameGuru to other competent studios and creates a more financially "liquid" audience of actual gamedevelopers using GG. Take a look at the prices of assets on average in the unity store or UDK marketplace and think about what a lucrative market you could attract by fixing what is a core fundamental flaw that needed to be looked at since day 1.

Quote: " There is a hope that if there are some small memory leaks, and some items are not being deallocated when they could be, then this would go a long way to ensuring levels can load without significant fragmentation worries."


absolutely, but I think focusing on the core memory management, even if that means outsourcing some of the work you yourself cannot do would put gameguru into the "viable solution" category for a lot more knowledgable aspiring game devs. These are people who are going to bring all sorts of experience and assets with them to our community.

Quote: "the engine has never really had an overhaul that specifically focused on the memory management of the engine and chunks of the code derived from the even older DBP technology that pre-dates even FPS Creator, so there is plenty of scope for memory management improvements. A quick hack could be to restart the application for each level loaded, thus clearing the fragmentation issue, but this would add 5-10 seconds onto your level loading time, and is a bit crude, but it's another solution that might work for the short-term while plans are put in place to solve the larger issue of game engine memory management."


that sounds like a fantastic course of action. Load times in GG are very long already so this would be a great solution, especially if some time was spent to create a seamless experience, say via a dummy program that will keep the loadingscreen up while the app reboots! Once load times (another serious GG issue) are looked at (hopefully in the future once core functionality is stable), if this was still even necessary it would be a small load time price to pay for the ability to create more levels.

Quote: "From feedback so far, there is a camp which argues that GameGuru does not yet produce games of sufficient quality to warrant efforts to improve the standalone game side of things (including multi-level games), and much prefer energy spent on performance, AI and general core features."


One could argue performance should be looked at first, given that you wouldn't need to create a lot of levels if you could create longer ones, but I notice that most people's maps around here don't nearly span the full expanse GG gives you, and if it does, it is usually very underdetailed, or running at a very slow frame rate. Even The Big Escape's 5 minute enjoyable linear experience proves that a perfectly servicable FPS game experience can be made, which was (i had assumed) the plan all along.

In creating short, linear experiences, gameguru can handle decently detailed scenery on a pretty good scale at a pretty decent FPS (improving on FPSC in all the basic necessities, more enemies, bigger maps, terrain, shaders, etc) and as such I have noticed that's what most developers that get the best results tend to do; they make bigger-scale FPSC games.

in one SHORT level, gameguru can create some truly visually splendid awesome experiences, but one short level is not enough to make an impression these days, especially with how short gameguru's performance constraints require that level to be.

That being said, 20 short levels could make that profound impression that continues to sell gameguru (and thus help fund its development) for years to come, but as it is, that's made very difficult due to these specific constraints stopping larger projects, especially ones with a high level of graphical fidelity. (which coincidentally are the ones that will probably be the most successful) Therefore I think memory management/multi level capability should be priority number one, with frame rate coming right behind and load times for dessert
yrkoon
20
Years of Service
User Offline
Joined: 14th Jan 2004
Location:
Posted: 16th Dec 2017 07:31
Quote: "
Quote: "the engine has never really had an overhaul that specifically focused on the memory management of the engine and chunks of the code derived from the even older DBP technology that pre-dates even FPS Creator, so there is plenty of scope for memory management improvements. A quick hack could be to restart the application for each level loaded, thus clearing the fragmentation issue, but this would add 5-10 seconds onto your level loading time, and is a bit crude, but it's another solution that might work for the short-term while plans are put in place to solve the larger issue of game engine memory management."

that sounds like a fantastic course of action. Load times in GG are very long already so this would be a great solution"


Or would it ? I never played with this, but would that manoeuvre not discard the global vars that are supposed to carry values (e.g. items, weapons the player acquired) between levels ?
Lives of great men all remind us we may make our lives sublime
Core i7 4710 QM GTX 980 MQ 16GB RAM Win7
PM
Solar
7
Years of Service
User Offline
Joined: 11th Apr 2017
Location: South Africa
Posted: 16th Dec 2017 08:45 Edited at: 16th Dec 2017 08:49
I truly hope that the memory problem in GameGuru gets attention and the problem is resolved. If I cannot make multiple levels with it, then I will be forced to give up my hopes of ever making a game.

And that would be heartbreaking for me, because I love GameGuru and it would mean that all the time I have invested so far in my game is wasted.
PM
Earthling45
7
Years of Service
User Offline
Joined: 3rd Sep 2016
Location: Zuid Holland Nederland
Posted: 16th Dec 2017 15:39
Multilevel stand alone file which i've used to test.

In the case of a graphicscard with only 2gb, the limit would be reached with level 3, when level 7 is loaded, i've reached the memory limit of 4 gb on my graphicscard.
From there on until level 10, almost 1 gb is added to the system memory.

Attachments

Login to view attachments
Bugsy
15
Years of Service
User Offline
Joined: 24th Nov 2008
Location: Savannah
Posted: 16th Dec 2017 16:47
Quote: "Or would it ? I never played with this, but would that manoeuvre not discard the global vars that are supposed to carry values (e.g. items, weapons the player acquired) between levels ? "


I'm not sure, that definitely sounds like it would cause issue but that's why I included this:

Quote: "especially if some time was spent to create a seamless experience, say via a dummy program that will keep the loadingscreen up while the app reboots!"


perhaps this dummy program could keep track of gameguru scripts as well, i'm assuming it would have to to know when loading of a level has begun.
Wolf
Forum Support
16
Years of Service
User Offline
Joined: 8th Nov 2007
Location: Luxemburg
Posted: 17th Dec 2017 02:16
@Teabone:
Quote: "Okay so if for example I made a game that had 3 levels and packed and only just working for me on my video card, and someone had a lower model GPU... is it possible they will get an error when reaching that 3rd level?"


That is precisely what happened with my release. People with a weaker GPU will eventually load half levels.
@all
Quote: "A quick hack could be to restart the application for each level loaded, thus clearing the fragmentation issue, but this would add 5-10 seconds onto your level loading time, and is a bit crude"


Crude indeed! GG games also tend to crash when you launch them only to work perfectly fine when you launch them again. Having the app shut down and reboot in the background will likely end in a crash fest.

Quote: " there is a camp which argues that GameGuru does not yet produce games of sufficient quality to warrant efforts to improve the standalone game side of things"


This camp will never fade because I feel that some only look at what they can achieve and ignore what other users are able to do. Others just made up their mind and thats that. It should be obvious by now that some quality hobbyist games have been made using GG. Quality by most indie dev standarts.

Quote: "in one SHORT level, gameguru can create some truly visually splendid awesome experiences, but one short level is not enough to make an impression these days, especially with how short gameguru's performance constraints require that level to be.
"


Yes. Constantly interrupting the flow of your game with a long loading screen is frustrating and would deter a lot of players.

Quote: " but as it is, that's made very difficult due to these specific constraints stopping larger projects, especially ones with a high level of graphical fidelity"


Yes. And if not high level of graphical fidelity at least a certain level of detail and scale of map.



-Wolf
Bolt Action Gaming
GameGuru Tool Maker
10
Years of Service
User Offline
Joined: 24th Oct 2013
Location: Harrisburg, PA (USA)
Posted: 17th Dec 2017 03:49
I don't feel that constantly chasing graphical fidelity for the sake of satisfying some steam reviews is the right answer here, Lee. PBR and DX11 are good moves mostly but at this point I think we can safely say that the engine is done chasing low-hanging graphical fruit with bloom, ssao, and now PBR all integrated into the engine.

I think that given the core issues of the engine remain and it really destroys any ability to make and produce a game in a fashion that would actually survive real distribution.

At this point there are several major, outstanding hurdles that need overcome but far and away stability of the product and memory management as part of that umbrella are priority one in my book. Second would be the AI, third would be quality of life issues such as having a meta-tag searchable item list, or grouping objects into their own stored structure.

All of that aside though truly we need a workable solution for developing games of any significant length that are stable and functional. At this point I'd drop every item on the voting board for just a non stop year long blitz on memory leaks, management, and stability fixes. I realize that probably puts me in a smaller camp than most, but I'm ok with that. I've literally never even tried to push a product to completion simply because of the crashing issue(s) and stability problems. Once those are knocked out, I will happily finish the 3-4 outstanding works I have hanging out there.


yrkoon
20
Years of Service
User Offline
Joined: 14th Jan 2004
Location:
Posted: 17th Dec 2017 10:00
Quote: "

Quote: "especially if some time was spent to create a seamless experience, say via a dummy program that will keep the loadingscreen up while the app reboots!"

perhaps this dummy program could keep track of gameguru scripts as well, i'm assuming it would have to to know when loading of a level has begun."

Although I'm inclined to say that this additional functionality could not be read from your initial sentence, I'm not going to argue with you over THAT.

But I'm quite sure that it is not a good idea to introduce yet another instance into this "game" involving itself into the management of a - let's put it mildly : - dishevelled memory heap. This would most likely push the shallow no-man's-land zone between "working" and "crashing" a little bit more outward, but, since there is no way for the game developers to predict - let alone control - when the border will be crossed, it simply is not a reliable solution. Considerable amount of work would go into it which I think, - given the limited resources at TGC - would be spent more efficiently following different schemes.

Of course, for a few people, the "clandestinely restart the engine" solution WILL work, but in cases like this, those for whom it won't work are those that count. Now, one could say : "Well you never know until you try", but, unfortunately, to try means to spend the effort, on a tweedy workaround which itself contributes exactly nothing to solve the underlying structural problem.

Just my 2 cents.



Lives of great men all remind us we may make our lives sublime
Core i7 4710 QM GTX 980 MQ 16GB RAM Win7
PM
AmenMoses
GameGuru Master
8
Years of Service
User Offline
Joined: 20th Feb 2016
Location: Portsmouth, England
Posted: 17th Dec 2017 11:58
The Lua environment and physics engine should not be using GPU RAM anyway so in theory you should at the end of each level be able to flush the GPU RAM prior to loading the next, from the loading times I had assumed that was happening anyway!

Been there, done that, got all the T-Shirts!
PM
smallg
Community Leader
18
Years of Service
User Offline
Joined: 8th Dec 2005
Location:
Posted: 17th Dec 2017 15:32
Lua loads from a file between levels and by default nothing saves except stuff like weapons and health anyway (there's a method to save custom variables between levels in my thread) so it wouldn't make any difference there with a little bit of coding.

This wouldn't really be that hard to test as you can always use a third party program to load the .exes separately, you're not forced to use the multilevel standalone system.
lua guide for GG
https://steamcommunity.com/sharedfiles/filedetails/?id=398177770
windows 10
i5 @4ghz, 8gb ram, AMD R9 200 series , directx 11
Teabone
Forum Support
17
Years of Service
User Offline
Joined: 8th Jun 2006
Location: Earth
Posted: 18th Dec 2017 19:24 Edited at: 18th Dec 2017 19:31
yeah I definitely need to retain certain variables between levels. For the types of games I'm interested in making (and am making).

I'm sure games like Space Losers would benefit from this.
Twitter - Teabone3 | Youtube - Teabone3 | Twitch - Teabone3 | TGC Assets - Store Link | Patreon - Teabone3

i7 -2600 CPU @ 3.40GHz - Windows 7 - 8GB RAM - Nivida GeForce GTX 960
Nomad Soul
GameGuru Tool Maker
17
Years of Service
User Offline
Joined: 9th Jan 2007
Location: United Kingdom
Posted: 22nd Jan 2018 21:51
Yeah so I guess there's a couple of things to consider here.

If its easier to make GG 64bit than implement a custom memory management system that should probably be the direction. I don't really get the support concern as Steam's December 2017 report shows 68.8% of users have Win7 64 and 24.7% Win10 64 with 32bit systems only 1% or less.

If the memory management system were the preferred option it would appear to make sense contracting this out which should not be an issue now GG is using C++ except for the financial implications of course.

It would be interesting if anyone has a solution to reload the application for each level but could this be done smoothly. Would this require a change to GG source code or can it be done with a custom loader application etc.

Also if GG has been made large address aware it should be able to use more than 2GB.
Teabone
Forum Support
17
Years of Service
User Offline
Joined: 8th Jun 2006
Location: Earth
Posted: 27th Jan 2018 23:30
Whatever happens I hope we can still carry over global variable values between levels.
Twitter - Teabone3 | Youtube - Teabone3 | Twitch - Teabone3 | TGC Assets - Store Link | Patreon - Teabone3

i7 -2600 CPU @ 3.40GHz - Windows 7 - 8GB RAM - Nivida GeForce GTX 960
Cylo
Game Guru Backer
13
Years of Service
User Offline
Joined: 2nd Jun 2010
Location:
Posted: 28th Jan 2018 13:51
Someone mentioned Voting Board.
Well, this is a bug. A huge one.
We are talking about a fix, not a feature.
Voting Board is for Features, not for fixes, so it has to be fixed regardless of votes.
PM
Duchenkuke
GameGuru VBOTB Developer
7
Years of Service
User Offline
Joined: 7th Jun 2016
Location: Germany
Posted: 28th Jan 2018 15:19
Quote: "Voting Board is for Features, not for fixes, so it has to be fixed regardless of votes."


exactly. Though I dont see that anytime soon.
Windows 10 64bit - Intel Core i7-2600 CPU @ 3.40GHz, 8,0 GB Ram, Geforce GTX 1050ti
https://sites.google.com/view/dkproductions




AmenMoses
GameGuru Master
8
Years of Service
User Offline
Joined: 20th Feb 2016
Location: Portsmouth, England
Posted: 28th Jan 2018 16:44
The great thing now is that instead of whingeing about it you can just sign up to GitHub and fix it yourself.

I've been pestering Lee for certain Lua commands I need, took me about 30 minutes to add them myself.
Been there, done that, got all the T-Shirts!
PM
synchromesh
Forum Support
10
Years of Service
User Offline
Joined: 24th Jan 2014
Location:
Posted: 28th Jan 2018 17:13
Quote: "I've been pestering Lee for certain Lua commands I need, took me about 30 minutes to add them myself."

It would be awesome if you are uploading your changes to Lee for core approval ?

Quote: "Voting Board is for Features, not for fixes, so it has to be fixed regardless of votes."

This would count as a bug and one for Lee to look into ..
The only person ever to get all his work done by "Friday" was Robinson Crusoe..
PM
AmenMoses
GameGuru Master
8
Years of Service
User Offline
Joined: 20th Feb 2016
Location: Portsmouth, England
Posted: 28th Jan 2018 17:29
I'm waiting on Lee, apparently he has weekends off!

He is moving/renaming the GitHub page ore something like that, and I'm not an official 'collaborator' yet if I'm understanding it correctly.

I need to create a branch (or twig or bush maybe) and make a 'pull' request, ooer missus.
Been there, done that, got all the T-Shirts!
PM
synchromesh
Forum Support
10
Years of Service
User Offline
Joined: 24th Jan 2014
Location:
Posted: 29th Jan 2018 01:23
Lee is sorting the Gateway for verified GG owners only .. Once it up an running he will let us know
The only person ever to get all his work done by "Friday" was Robinson Crusoe..
PM

Login to post a reply

Server time is: 2024-04-20 14:25:26
Your offset time is: 2024-04-20 14:25:26