Product Chat / [SOLVED] PBR Texturing Packing to Save Memory?

Author
Message
Stab in the Dark software
GameGuru TGC Backer
21
Years of Service
User Offline
Joined: 12th Dec 2002
Location: Upstate New York USA
Posted: 8th Sep 2018 00:30
Is it possible to use packed textures in GG?
If so what is the protocol? For example would it be the AO in the red channel, the Gloss in the Green channel and
the metalness in the blue channel?

The coffee is lovely dark and deep,and I have code to write before I sleep.

The author of this post has marked a post as an answer.

Go to answer
Pirate Myke
Forum Support
13
Years of Service
User Offline
Joined: 31st May 2010
Location: El Dorado, California
Posted: 8th Sep 2018 00:51
Good question. From what I understand the shaders are not written that way.

Will we get that in the future?

Will it reduce memory?

Maybe we can have or quarter the metalness, gloss, illumination textures, as they are just grey scale.

I will do some test with texture size differences. Then compare side by side for visual quality.

I will post my results.
Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 2400 Mhz, 4 Core(s), 4 Logical Processor(s), 8gb RAM, Nvidia gtx660, Windows 7 Pro 64bit, Screen resolution 1680 x 1050.

Pirate Myke
Forum Support
13
Years of Service
User Offline
Joined: 31st May 2010
Location: El Dorado, California
Posted: 8th Sep 2018 07:46
OK so the lamp post are 2K textures on the right.
On the left is the greyscale maps reduced to 1K



In this shot the one on the left is 4K textures.
On the right is the greyscale maps reduced to 1K


I can not tell the difference.
Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 2400 Mhz, 4 Core(s), 4 Logical Processor(s), 8gb RAM, Nvidia gtx660, Windows 7 Pro 64bit, Screen resolution 1680 x 1050.

Belidos
3D Media Maker
8
Years of Service
User Offline
Joined: 23rd Nov 2015
Playing: The Game
Posted: 8th Sep 2018 09:24 Edited at: 8th Sep 2018 09:25
Graphixs already tested this, ifvi remember right he found that you can drop most pbr generated textures (metels plastics etc) down to 512 before loss, and testures made pbr to 1024 before degredation. Obviously more detail and more uv islands will also effect the size needed.
[img]belidos3d.swgrp.co.uk/wp-content/uploads/2016/03/blogo.png[/img]
Primary Desktop:
i7 7700,k NV1070 8GB, 16GB 3200mhz memory, 1x 2TB Hybrid, Win10.

Secondary Desktop:
i5 4760k, NV960 2GB, 16GB 2333mhz memory, 1x 2TB Hybrid, Win10.

Laptop:
i3, Intel 4000 series graphics, 6GB memory, 1x 500gb HDD, Win8.1.
GraPhiX
Forum Support
19
Years of Service
User Offline
Joined: 15th Feb 2005
Playing:
Posted: 8th Sep 2018 10:40
Hi yes I did some basic test too https://forum.game-guru.com/thread/220015#msg2605112
Welcome to the real world!
Main PC - Windows 10 Pro x64 - Core i7-7700K @4.2GHz - 32GB DDR4 RAM - GeForce GTX 1060-6G 6GB - 1TB NVe SSD
Test PC - Windows 10 Pro x64 - G4400 @3.3GHz - 16GB DDR3 RAM - GeForce GTX 950 2GB - 500GB SSD
Laptop - Helios 300 Predator - i7 7700HQ - 32GB - Nvidia GTX1060 6GB - 525GB M2 - 500 SSD - 17.3" IPS LED Panel - Windows 10 Pro x64
Various Tutorials by me
Pirate Myke
Forum Support
13
Years of Service
User Offline
Joined: 31st May 2010
Location: El Dorado, California
Posted: 8th Sep 2018 12:29
Yes, but the last one is a 4k Color, normal, ao texture with a 1k metalness, gloss texture. saving about 15mb of texture memory.
Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 2400 Mhz, 4 Core(s), 4 Logical Processor(s), 8gb RAM, Nvidia gtx660, Windows 7 Pro 64bit, Screen resolution 1680 x 1050.

GraPhiX
Forum Support
19
Years of Service
User Offline
Joined: 15th Feb 2005
Playing:
Posted: 8th Sep 2018 13:22 Edited at: 8th Sep 2018 13:23
yes I had 128kb textures with 1024 normal and it looked ok

In my honest opinion I think artists should release Models/Textures with the highest texture size possible (I still prefer 2k) and as DDS with a zip file containing the .PNG's this will allow the level designer to resize textures according to his needs, as an artist I will not know what level my asset will be used in so I need to give the level designer the option of being able to reduce textures accordingly.


PNG maps are ok for GameGuru but can be larger than DDS, the GPU and GG better utilises DDS textures.

Textures can always be scaled down but cannot be scaled up.

DDS textures cannot be resized without losing quality so hence packing PNG's with the assets its is very easy to reduce a 2k map to 1k
this is approx. sizes of my maps:

4096x4096 = 8Mb DDS
2048x2048 = 2Mb DDS
1024x1024 = 512Kb DDS
512x512 = 128Kb DDS
256x256 = 32.1Kb DDS
128x128 = 8.12Kb DDS

as you can see using DXT1 compression a 1k map uses only 25% of a 2k map with no visual difference (as far as my models are concerned)
DXT5 for transparency adds about 25%
Welcome to the real world!
Main PC - Windows 10 Pro x64 - Core i7-7700K @4.2GHz - 32GB DDR4 RAM - GeForce GTX 1060-6G 6GB - 1TB NVe SSD
Test PC - Windows 10 Pro x64 - G4400 @3.3GHz - 16GB DDR3 RAM - GeForce GTX 950 2GB - 500GB SSD
Laptop - Helios 300 Predator - i7 7700HQ - 32GB - Nvidia GTX1060 6GB - 525GB M2 - 500 SSD - 17.3" IPS LED Panel - Windows 10 Pro x64
Various Tutorials by me
Stab in the Dark software
GameGuru TGC Backer
21
Years of Service
User Offline
Joined: 12th Dec 2002
Location: Upstate New York USA
Posted: 8th Sep 2018 14:02 Edited at: 8th Sep 2018 14:04
I think the terrain already uses a packed texture so maybe it should be the same for
the entities. I think packing would be better than reducing the textures.
I think the normal map texture should not use compression or was it mip maps.
The coffee is lovely dark and deep,and I have code to write before I sleep.
Pirate Myke
Forum Support
13
Years of Service
User Offline
Joined: 31st May 2010
Location: El Dorado, California
Posted: 8th Sep 2018 14:53
Hard to say as most of the dlc that has been released, have no set standard yet.
Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 2400 Mhz, 4 Core(s), 4 Logical Processor(s), 8gb RAM, Nvidia gtx660, Windows 7 Pro 64bit, Screen resolution 1680 x 1050.

Stab in the Dark software
GameGuru TGC Backer
21
Years of Service
User Offline
Joined: 12th Dec 2002
Location: Upstate New York USA
Posted: 8th Sep 2018 16:05
After some research this is what I believe should be used to save memory.
Red channel = Metalness
Green channel = Gloss/roughness ( green gives better resolution for gloss)
Blue channel = AO
Alpha channel = Emmissive

Then you just have your color and normal map as seperate textures.
Also it should be DXT5 commpression for the normal map so as to not loose
resolution.

This should be considered as there has bee a lot of discussion lately on memory comsumption.
I think Preben already does this in GG loader.

The coffee is lovely dark and deep,and I have code to write before I sleep.
Bugsy
15
Years of Service
User Offline
Joined: 24th Nov 2008
Location: Savannah
Posted: 8th Sep 2018 17:32
preben said that automatic texture baking is a planned feature. This will definitely help save memory.

also, from my normalmap making experience, the RGB channels are needed for the normal map. Blue channel works for AO since it's the toplit channel, but red and green need to have the light coming from X and Y to get the normal bump effect correct if I understand correctly...





if there's more i have to learn here, someone please drop the knowledge on me!

I think we should go back to the DNS naming convention, but use all the specular map channels for PBR-specific textures (since only alpha channel works as specular as it is on that texture)

this would also make updating DNS textures to be PBR easier, with less fpe changes necessary.





Stab in the Dark software
GameGuru TGC Backer
21
Years of Service
User Offline
Joined: 12th Dec 2002
Location: Upstate New York USA
Posted: 8th Sep 2018 18:35 Edited at: 12th Sep 2018 16:07
Bugsy I was not suggesting using the normal map for packing.
I meant we would have a color texture, a normal map texture and
a third texture that would contain the Metalness, gloss, AO and emissive.

As far a automatic texture packing we really do not need that as most modeling/ texturing
programs can pack the textures. I use 3D Coat which can do it on export.
Maybe this could be a FPE setting if your textures are packed.

I use this method in my UE4 projects and it saves a lot of memory.
The coffee is lovely dark and deep,and I have code to write before I sleep.
Bugsy
15
Years of Service
User Offline
Joined: 24th Nov 2008
Location: Savannah
Posted: 8th Sep 2018 18:50
ah, in that case we are in agreement then.




Stab in the Dark software
GameGuru TGC Backer
21
Years of Service
User Offline
Joined: 12th Dec 2002
Location: Upstate New York USA
Posted: 13th Sep 2018 17:42
I am bumping this to get Lee or Prebens attention.
The coffee is lovely dark and deep,and I have code to write before I sleep.
cybernescence
GameGuru Master
11
Years of Service
User Offline
Joined: 28th Jan 2013
Playing: Cogwheel Chronicles
Posted: 13th Sep 2018 19:11
It would be a trivial settings.fx and apbr core type parameter change that could then ensure that packed textures would be used, however a convention would have to be agreed on.

Personally I think the suggestion by Preben ticks a lot of boxes in that the pbr textures would stay separate but would not all be loaded at game run time (only the relevant channels from the separate textures).

Cheers.
Preben
19
Years of Service
User Offline
Joined: 30th Jun 2004
Location:
Posted: 14th Sep 2018 10:39 Edited at: 14th Sep 2018 10:48
My plan with this was to:

1. Make a new set of "faster" shaders that use the baked textures.
2. If you use a existing PBR object , the new baked textures will be created for you , IF the timestamp of the _ao,_gloss,_metalness,_normal and/or (_illumination,_emissive),_detail is newer then the baked textures. ( so you can update the old images, and it will bake again ).
3. If a baked texture exist it will be loaded and the new faster shaders will be selected ( and sure use way less mem. ).
4. Only the baked textures would go into the standalone.

Stab: This would allow you to directly use the new shaders in the .fpe and bake your own texture sets.
cybernescence: Your old media will be baked for you.

About what goes into what is not really planned yet , but i might go selfish here and follow the GG Loader setup, would make things easier for me in the end
So:
_D is always left alone.
_N can have a emissive texture baked into the alpha , if a illum,emissive exist. Done it this way as there are few objects with emissive so you normally do not have to bake the _N ( saves alot of processing time ).
_S always have R=(_height,_detail,_spec, or other less used formats), G=_metalness, B=_gloss, A=_ao

Bugsy: Yes i use the DNS formats in GG Loader , as i can have the same loading,setup if it use PBR or the old DNS. As you can see the _S can still have the _spec in the red channel so old DNS textures can be used in the same setup as the new PBR baked format with no changes.

EDIT: the baked naming would NOT be DNS , as normally you will leave the _color,_normal alone, and only make one new baked image, Just so you dont think we are going back to the old naming

But hey not sure if its time to do this baking , or other problems have higher pri. so many things on the list , just check github
Subscribe and checkout great GameGuru/AGK video's here: Videos click here
Latest GameGuru Loader news: News click here
Get GameGuru Loader PBR version here: Steam click here
best regards Preben Eriksen,
Bugsy
15
Years of Service
User Offline
Joined: 24th Nov 2008
Location: Savannah
Posted: 14th Sep 2018 17:21
this sounds like it would be user friendly and a good improvement to memory access, perhaps resulting in us one day successfully being able to build bigger games, and unifying our pbr rendering system to work with our old media!

go preben
DVader
20
Years of Service
User Offline
Joined: 28th Jan 2004
Location:
Posted: 16th Sep 2018 23:23
Not to rock the boat or anything but packed textures are just that, packed. They still need to be unpacked into memory at some point, where they will take just as much memory as before. There's no magical fix for this unless GG is deliberately lowering certain texture sizes. In which case it would be far better for the artist to make those textures at the size they want. Compression only really affects the download size, not the memory required to use them.

I would not be massively impressed if GG just shrunk certain maps on it's own bat personally. Definitely an easy way to help performance but if I had gone to the trouble of making the higher res images in the first place I would like to use them. Then I would have to choose between quality and performance myself. A way to get GG to do this when you want to is another matter of course. It would save a lot of image editing time.
SPECS: Ryzen 1700 CPU. Nvidia 970GTX. 16 Gig Memory. Win 10.
Preben
19
Years of Service
User Offline
Joined: 30th Jun 2004
Location:
Posted: 17th Sep 2018 08:37
This post has been marked by the post author as the answer.
DVader:
Sorry thats not how it works, currently say a AO image use a RGBA image that are sent to the GPU , when you use a baked image the AO "only" use the R channel (8 bit) all the other textures are assigned to the other channels , so memory wise you can bake 4 textures into 1 using the same GPU mem.

There is no compression going on when baking , so you dont loss any quality, and the baked image do not have to be unpacked, the new shaders will use the baked image directly.

But yes you also get "Faster loading" / "Less memory used CPU and GPU" / "Faster render speed".

Its a must for all game engines, and one feature that "only" GameGuru is missing , all the other already do this

Subscribe and checkout great GameGuru/AGK video's here: Videos click here
Latest GameGuru Loader news: News click here
Get GameGuru Loader PBR version here: Steam click here
best regards Preben Eriksen,
DVader
20
Years of Service
User Offline
Joined: 28th Jan 2004
Location:
Posted: 22nd Sep 2018 19:27
Ah okay, I stand corrected.
SPECS: Ryzen 1700 CPU. Nvidia 970GTX. 16 Gig Memory. Win 10.

Login to post a reply

Server time is: 2024-05-04 19:31:33
Your offset time is: 2024-05-04 19:31:33