Throughout all development endeavours on GG, from ancient Reloaded versions to todays 1.3 GameGuru, we have always stumbled across one particular bug sooner or later.

Short description:


At some point, when a level reaches a significant size, terrain complexity and entity count, freshyl added complex models will appear with wrong / broken Lighting on Entity Shader settings "high".

Known Background Information (leads to repro steps, too):

It happens both with self-made, stock and store assets, the more complex a model is, the likelier. We used "House 76" from GG Buildign Pack VOl. 2 for demonstration.
You need a "demanding" level with sophisticated terrain, hundred(s) of different props / structures resulting in 3000+ entities before this starts happening.
Setting the SAME model onto a blank, new level will result in no bug at all
We had the bug now in 4 completely different levels / maps with different terrains, Sky, entities e.t.c.
The bug only appears on Entity Shader setting "High", setting to "Medium" will result in "normal" rendering of the texture
De-Installing, Re-Installing or verifying has no effect whatsever on this issue
The issue is neither OS nor Hardware related, we can reproduce it on AMD, INTEL, RADEOn, GEFORCE, Windows 7, 8 and 8.1 systems
No setup.ini flag modification has been found to have any effect on this issue
On rare occasions, a map with a building or entity affected will appear correct during a testrun, however, saving as standalone, saving, adding entities and saving will result in a return of the issue

Attached Pictures:

building_allright: correctly rendered House 76 within Level "Fathers Island Main"
building_broken: broken rendered House 76 within Level "Fathers Island Main"

Grateful for any hint or insight.

Addendum:

Commenting out the normal mapping in the basic entity shader code fixes the problem, but that´s not an option as the textures are flat afterwards.

No that is not a good option.

How does inverting the normal map blue channel make it work?

Let me acquire this building and play with the textures.

You won´t be able to reproduce without a huge, diverse, demanding level

I am setting one up. I know that Lee upped the number of entities to 20,000 instead of 10,000.

I have a level with that many objects in it. Testing right now.
This is all real time lighting right?

Yeah, it is. I already had a former GTA shader coder looking over it, and he believes that here:



The values fed into the shader for WorldTangent and WorldBinorm are somehow corrupted.

OK thanks. Let me see if I can get it to do it on my machine.

Then I will have to make a video for Lee and then point him at this thread.

Thanks for posting.

There are a few other users that make larger levels, might have them check this out also.

Thank you very much.

I can also provide a compiled game, of course.

And didn´t help, unfortunately.

To invert the normal blue channel in GIMP
You go to the channel dialog and click on the blue channel.
Go to the colors dialog and then choose invert.
Export the image back out.

Now I just imported this House 76 into 3ds max.
Looking at your posted image I can see a pattern of stack UV islands.
Importing to 3ds max and applying the UV unwrapper to it. This is a true scenario. As with most stock and DLC packs.

My question is now directed at your custom media. Are those models using stacked UV also?

There was another thread last week where there was a lot of discussion on light mapping and real time lighting with objects having stacked UV's were not rendering proper, and one wiith out stacked UV's were rendering proper.

Let me find that will I set this map off. 3192 objects, characters, trees, heavy scripting music and to flying dragons always acting in the sky and see what happens. 95% of the 1300 trees in my level have physics turned off. ( so this may nit be conclusive. Hoping the RPG scripts and the 45 characters spread out thru the map will be enough of a demand to see what you are seeing.

My next step will be to take the house 76 model and just flatten out all the UV's to non stacked and export it out as a new xfile and then set them side by side and see if that may help.

The shaders do have a long way to go, and it does seem if your push the upper end of 1700 entities, that more things tend to go wrong.

This will take a little bit to start test game and then make the changes and do it again with the control object. I will report back my findings.

Edit:
That post I was referring to was here:
https://forum.game-guru.com/thread/214914

But it has to do with light mapping getting similar results. So this is were I will base my test from starting with real time lighting. You may have just solved this mystery with the shader being the main culprit.

First test report.

Must not have stressed it out enough yet. Will add more Entities in the level.



My current map:



Frame rate is very slow, and my poor little 660 is always in full power mode. with 3191986 polys in the scene real time lighting, it is actually running 3 fps more then the last time I ran it wit 1100 less trees. so there has been some improvement at the cost of how much memory you may have available.

I have decals and collectable objects also in this map.

Not sure, I leave that to the pro's.

Will thru more stuff at it.

Oh my! Please don´t kill your poor brave 660 for my game!

Well, you have a rather simple terrain and many copied / similar objects inside, and no matter the pure amount of entities, the complexity of the information thrown at the GPU will raise with the different models and textures.

I can spray the world with 2000 of the houses and 5000 trees and will get no bug at all. If I have 180 different entities with some (trees, trunks, foliage, rocks) of them sprayed around madly, then it blows in my face.

I have now added more of the Valuable Assets Buildings, all have the same issue. But I didn´t blame it on the artist, so I added more buildings from the death valley pack. And those were then light on the wrong side or partially lit on the wrong side, resulting in a very funny look.

Fun fact: If I add the medieval standard stock church afterwards, this building alone of all the new ones will appear normally.

Screens attached.

Still one of the best quality processers Intel have ever made, I have one in my backup machine and sometimes it actually works better than my i5

I will be very sad when my Q6600 processor dies. Great work horse for 3d applications.

@Ertlov: I will start some random spray tests on many different objects and see what I come up with.

My 3D lead believes that it might be still an error in the models, not in the 3D models geometry itself, but in the UV Layout. He will try to fix that now for a test run.
However, if he is right, MANY store items would need an overhaul. Hundreds of them.

nope, wasnt conclusive so far.

Ok, thanks for reporting back.

Hi Ertlov, how have you been. Yes I also had this same problem with a city map I'm working on. One building (sorry cant recall which one) was doing the same as your first screen shots. At first I thought it was the model, however tested it by itself and no issues. I wasn't sure if it was the draw position/location on map or because of a conflicting model near by, or just overload as you said which caused the issue.
The map has a lot of custom and many models in close proxy of each other, though this building was a background prop and wasn't a key model so I discarded it and went with another option.
Either way, I hope you find a solution and looking forward to reading more on this.

Hoi Defy!

Yeah, I am still alive.

And I came further: Now I can reproduce the bug with almost EVERY SINGLE ENTITY I add to the already created level. Even post boxes or the stock assets church.

Hey hey Yeah good to hear, and hope the project is going well.

Wow, ok so there is a certain limiter as to the amount of assets? or are we talking load on the engine when in the proxy of none flat terrain or complex mesh grouping in an area?

Entity limit in the last build was raised to 20,000 objects instead of 10,000.

Yeah, but that doesn´t mean you can pump 10000 - 20000 Diffuse, N and S maps into the GPU RAM

I can now confirm that EVERY NEW entity I add has this bug, it is lit on the sides that shouldnt be lit and dark on the sides where exposed to the sun.

Even if I delete some previous assets from the project, any new ones will have the bug. Very frustrating.

//bump

email received.

EDIT:
I see what you mean completely.

Yeah, it´s easier to provide something playable than to explain

Hundreds of models will need fixing, wait!...hundreds of models don't need fixing....of all the models this one works fine....wait!...now it doesn't....lol.

This isn't a problem with uv's, it isn't a problem with the models. It is a problem with the lighting and memory and there is something wrong with the lightmapper too. Fixing one model by giving uv islands their own space won't fix the engine. You hit it on the head with the amount of models and textures being thrown in to a level. It has been encouraged from the outset, even skybox textures were encouraged to be 2048 x 2048, this increased quantity doesn't always mean increased quality and certainly isn't good for a game engine. I see a lot of excessive texture sizes for even small models using 2048 when a 512 would work just fine. Smaller texture sizes and good shaders are required, already using five terrain textures with _n and _s and I cant wait till they get to the sixteen on the voting board....lol.... again 2048 where they should be 1024 at most or optimally 512. I have no idea where someone got the idea that these huge textures were ok for game design, I will use large myself but only where it is really needed.

Have you tried reducing the scale of these? not in the setup.ini, but the textures themselves? You might get a bit of headroom by reducing the normal and spec maps which don't need to be 4096 or 2048 same as the diffuse. Those 10000-20000 textures increase to 30000-60000 when you include the _N and _s that go with them. This is a ridiculous amount of memory to expect a GPU to handle. I cant be certain and won't open my mouth on this subject again but my two cents is that I won't be nit picking my way through models trying to fix something that works fine in any other engine, the only difference between GG and others is this completely irrational idea that textures need to be huge to make things look 'good as'.

You are one of the few people I would read everything you post hoping to catch some insight or something helpful from. But I see you floundering around here just as others do trying to find a fix for something which should have been foreseen and prevented from the outset. Of course there may be some models needing fixed, when you got thousands in the store there has to be some issues. But they shouldn't work one minute then stop working the next, this is something needing looked at properly by the Dev's and not something latched onto as the 'solution' when a single model is 'fixed' by a single user doing something they claim is 'industry standard' where it is clearly not.

As I said I am finished with these lighting issues as I am happy to leave it to the professional nit pickers and it either gets fixed or it doesn't but I won't waste any more time on it. What I will be doing from now on is reducing texture sizes and using the shaders for detail (as it should be) and if the store rejects my media then I will simply sell over on the Unity store where they have a clue and a GAME engine that actually works. properly.

This will need to be a shader and lighting fix for sure. No way every model is going to bypass this issue.

I fully agree on that.



This wasn´t actually my idea, but this was the verdict of an experiencedID4 and Unreal developer after reviewing a bunch of the models. However, I ruled that out pretty soon afterwards, didn´t I ?



This is actually the next step, but it would be far easier if we hadn´t that ridiculus decryption of the exported assets. I could simply downscale the textures in a standalone and check how it behaves.
However, the entire level has now 640 MBs of dds files when exported, I imagine I won´t hit any GPU limit when playing on cards with 2 or 4 GB VRAM - and yet it happens.



yes, I think so, too. But without proper debugging tools I have to ask you, lee, et.c.

Edit:

Ripped 180 MB of texture size from the level - no effect.

I wasn't referring to yourself or your Artists when I made comments about uv layout etc this was another thread, however I am tired of this being a sweeping statement when someone finds something wrong with a few models that it means 'hundreds' need a fix. Like I said I am just going to sit this one out and if myself or others have been doing it wrong all this time then I will fix it but I aint going on any witch hunts with my media till I am told by a Dev that standards aren't being met for GG

I am even getting the feeling I don't want to release any more media till we get something in concrete from TGC regarding this as I have no desire to be going over hundreds of old models and 'fixing' them so they work in this engine after the fact. If your lead Artist found anything wrong with my own models I would be happy to take a look.

You used to be able to turn on off encryption for Classic but it looks like that option isn't available in this case.

As said, I did it now the hard way, wothout any positive effects.

And it´s pretty obvious that the models can´t be the issue when they work fine in all other engines or even in GG when being in an empty level.

Also, we had created these very creative desert buildings for Antinomy, which had the same bug in GG, and they were good enough to survive the switch to Unreal 4, where they work perfectly - without any changes to model or UV layout.

So yes, it is an engine bug, and a very nasty one indeed.

Edit:

And from what I see in the latest Antinomy build, some assets bought from you also made it to Unreal

Out of curiosity, did your level look any less detailed and 'rubbish' with the texture reduction? I am not referring to larger scale models which may require the larger resolutions but overall

Edit*
Anything I gave you which was WIP doesn't count

nope, your crates, honestly bought!!

The ship sank on its way to the Unreal shores, unfortunately...



Slightly, but hardly recognized.

Not surprising it was held together with string and duct tape....I really should finish it up properly.

Check your emails