H15 Baking
26499 75 8
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
-
- mdavies
- Member
- 110 posts
- Joined: May 2015
- Offline
@MagnusL3D: The script would be called with the full paths to all files generated by the render. This would allow you to do your own processing of the images (building texture atlases, color space conversions, renaming). I could provide you with an example script that you could modify, if that helps.
We could also add a format string, which would work like the ‘Output Picture’ parameter. You refer to the various parts of the filename with an expression:
B - Base image name
P - Plane name
T - Udim tile number
E - Extension
%B_%P.%T.tga
…which would generate:
image_Nt.1001.tga
This might be the most flexible as you wouldn't need to choose the output format: it would be picked up from the extension. You would not need a script in this case.
We could also add a format string, which would work like the ‘Output Picture’ parameter. You refer to the various parts of the filename with an expression:
B - Base image name
P - Plane name
T - Udim tile number
E - Extension
%B_%P.%T.tga
…which would generate:
image_Nt.1001.tga
This might be the most flexible as you wouldn't need to choose the output format: it would be picked up from the extension. You would not need a script in this case.
-
- mark
- Staff
- 2592 posts
- Joined: July 2005
- Offline
MagnusL3D
btw, could there be a option to write out the lo-res object as a .obj as it bakes out the textures to disc? i am thinking of setting it up to generate several tombstones and that would be handy. Getting the lo-res written out with same naming standard besides the textures.
Could you set the low res geometry to point to a file SOP that has the File Mode set to save the geometry when it cooks?
-
- mdavies
- Member
- 110 posts
- Joined: May 2015
- Offline
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
@MDavies, here is a simplified example what I would like to be able to do:
Diffuse = <path><hipname>_<$F>.tga
Normal = <path><hipname>_<$F>_local.tga -invert Cg
Specular = <path><hipname>_<$F>_s.tga
Roughness = <path><hipname>_<$F>_pm.tga -invert Cd
Object = <path><hipname>_<$F>.obj
This is what I was planning to set up in COP's (except the object part) but could I do that with the script then I would most certainly use that because I wouldn't have to render out a .RAT and then put that through COP's
@Mark, that is what I was thinking of doing, but if I can set it up like MDavies suggest I would feels like it would be a more clean solution.
/M
Diffuse = <path><hipname>_<$F>.tga
Normal = <path><hipname>_<$F>_local.tga -invert Cg
Specular = <path><hipname>_<$F>_s.tga
Roughness = <path><hipname>_<$F>_pm.tga -invert Cd
Object = <path><hipname>_<$F>.obj
This is what I was planning to set up in COP's (except the object part) but could I do that with the script then I would most certainly use that because I wouldn't have to render out a .RAT and then put that through COP's
@Mark, that is what I was thinking of doing, but if I can set it up like MDavies suggest I would feels like it would be a more clean solution.
/M
-
- mdavies
- Member
- 110 posts
- Joined: May 2015
- Offline
@MagnusL3D: Thinking some more about the formatting of the output images, it might be difficult to second-guess what users need. Adding too many controls to the UI may also be more confusing than helpful. If you're Ok with the script approach I think this might be best. This way you're free to do any post-processing (renaming, conversion) you like.
The output geometry would likely be a parameter just after the ‘Output Picture 1’ path, under the ‘UV Render Objects’ list. The default path will be empty, meaning no geometry will be output.
In terms of flipping the normal map channels, this is something that is already possible with the VOP but has not been exposed in the Bake Shader; I can add this. When you say “-invert Cd” for roughness, what do you mean? Inverting the color?
There is one small UI change coming: the image planes that are related to the Bake Shader have been relocated to that tab to make things clearer (ie. they are only relevant for that shader). I'll look into adding the script and let you know once there is something to test with. Thanks for all the suggestions
The output geometry would likely be a parameter just after the ‘Output Picture 1’ path, under the ‘UV Render Objects’ list. The default path will be empty, meaning no geometry will be output.
In terms of flipping the normal map channels, this is something that is already possible with the VOP but has not been exposed in the Bake Shader; I can add this. When you say “-invert Cd” for roughness, what do you mean? Inverting the color?
There is one small UI change coming: the image planes that are related to the Bake Shader have been relocated to that tab to make things clearer (ie. they are only relevant for that shader). I'll look into adding the script and let you know once there is something to test with. Thanks for all the suggestions
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
@MDavies, ah thats nice, I always have to look for where the image planes are located when doing a new scene
Yes the script sounds like it will be the most flexible way of solving it, looking forward to test it to see if I can get out what I want
regarding invert -Cd for roughness, this is usually just a grayscale image anyway and the more white the more rough, but here for some reason they changed the map into “smothness” maps, so more white means more smooth.
And I think the same is true if you are using the PBR spec/gloss game shaders that have become popular as of late.
Sounds great if the flipping normal map channels are exposed !
Yes the script sounds like it will be the most flexible way of solving it, looking forward to test it to see if I can get out what I want
regarding invert -Cd for roughness, this is usually just a grayscale image anyway and the more white the more rough, but here for some reason they changed the map into “smothness” maps, so more white means more smooth.
And I think the same is true if you are using the PBR spec/gloss game shaders that have become popular as of late.
Sounds great if the flipping normal map channels are exposed !
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
Some thoughts on the whole object exporting thing…since .obj files from Houdini dont get any .mtl files created the automatic material creation in Unreal doesnt work, and even if Houdini exported a .mtl file, those textures where just baked so how would it know to generate the .mtl file with those.
And similar issues with FBX plus that I guess it wont be accessable as export format since it is not available in the File SOP export now ? but rather sits in its own File->Export place in the menus.
And similar issues with FBX plus that I guess it wont be accessable as export format since it is not available in the File SOP export now ? but rather sits in its own File->Export place in the menus.
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
BTW, im rather new to Unreal so not sure the automatic material creation ever works for .obj or if it is a FBX only feature.
EDIT: Opened a .obj generated by Houdini in modo, assigned materials and saved. so now I have a .mtl file and automatic material is created when I import that so I guess that works. except that the .mtl file does not include the assigned normalmap from Modo, if that is a modo issue or if .mtl format does not support normalmaps I do not know.
Anyway..my dream for this is to generate 100 of models in one go and then it would be awesome if the exported content from Houdini was setup in such a way that the automatic material creation in Unreal worked.
EDIT 2: aaand if it is FBX or OBJ that works in the end doesnt matter as long as it works
EDIT: Opened a .obj generated by Houdini in modo, assigned materials and saved. so now I have a .mtl file and automatic material is created when I import that so I guess that works. except that the .mtl file does not include the assigned normalmap from Modo, if that is a modo issue or if .mtl format does not support normalmaps I do not know.
Anyway..my dream for this is to generate 100 of models in one go and then it would be awesome if the exported content from Houdini was setup in such a way that the automatic material creation in Unreal worked.
EDIT 2: aaand if it is FBX or OBJ that works in the end doesnt matter as long as it works
-
- mdavies
- Member
- 110 posts
- Joined: May 2015
- Offline
@MagnusL3D: In terms of the asset preparation step this is likely something that we would defer to our Games team as they are familiar with the requirements of various engines (eg. Unreal, etc). This would likely be implemented as a dedicated post-render script that could be shipped with Houdini. You would select “Unreal” formatting and the asset could be packaged as required.
I've almost completed the work with the filter script, which is just an extension to our Python filter. There is a new function ‘filterOutputFiles()’ which is passed a list of files output by the ROP. You are free to process these as needed (renaming, conversion, publishing). This is the most generic way to present this, and won't require us hardening specifics of your pipeline into Houdini. Once this is in place we will be able to give you some examples of how best to format for Unreal.
I've almost completed the work with the filter script, which is just an extension to our Python filter. There is a new function ‘filterOutputFiles()’ which is passed a list of files output by the ROP. You are free to process these as needed (renaming, conversion, publishing). This is the most generic way to present this, and won't require us hardening specifics of your pipeline into Houdini. Once this is in place we will be able to give you some examples of how best to format for Unreal.
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
@Mdavies, your a rockstar with all these additions. when will all these things be available ? im checking the dailybuild logs in anticipation every day
Sounds great to get the game team onboard, not sure if I would call it so much “asset preperation” as saving out FBX files and or OBJ files “properly”, but in the end, all good anyway
Very much looking forward to look at the python script!
Sounds great to get the game team onboard, not sure if I would call it so much “asset preperation” as saving out FBX files and or OBJ files “properly”, but in the end, all good anyway
Very much looking forward to look at the python script!
-
- mdavies
- Member
- 110 posts
- Joined: May 2015
- Offline
@MagnusL3D: The script is part of the Python filtering in Mantra, which is really handy. I've added a new function that gets called after the render completes, which passes a list of generated files. For example:
[['/obj/lowres', <- Asset name
[
, <- File, Type
<- File, Type
<- File, Type
]
]]
…where ‘Type’ identifies the file type:
0 - Image
1 - Texture
2 - Geometry
I've attached an example filter that you can modify to suit your purposes. You specify the filter to Mantra using the ‘-P’ flag:
mantra -P filter.py
I'll get back to you about the Games group's input on this.
[['/obj/lowres', <- Asset name
[
, <- File, Type
<- File, Type
<- File, Type
]
]]
…where ‘Type’ identifies the file type:
0 - Image
1 - Texture
2 - Geometry
I've attached an example filter that you can modify to suit your purposes. You specify the filter to Mantra using the ‘-P’ flag:
mantra -P filter.py
I'll get back to you about the Games group's input on this.
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
-
- mdavies
- Member
- 110 posts
- Joined: May 2015
- Offline
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
And here is my new “dirty” way to get some automatic things going..
First step is obviously to bake out .obj file and textures.
something similar to:
object_01.obj
object_02_diffuse.tga
object_02_specular.tga
object_02_roughness.tga
object_02_normal.tga
then since Houdini .OBJ saver does not write a .MTL nore assign a link to the none existing .MTL inside the .OBJ, I have written a python script that goes through the directory, inserts the link into the .OBJ and creates the .MTL based on some assumptions regarding file naming etc.
Onto the next problem. The UnReal .obj file importer can automate material creation, but it seems to only support the diffuse (map_Kd) in the .MTL file format. And the Autodesk FBX converter seems to suffer from the same limitation.
However if I open the .OBJ files in Modo with my generated .MTL files from the previous python script. all textures are assigned correctly.
So I wrote a small python script in Modo which imports the .OBJ files and saves them out as .FBX files. which then get the automatic material creation in Unreal to work.
Attached image shows the first itteration of it. since then added some more features like fixing scaling and rotation in Modo for the mesh.
First step is obviously to bake out .obj file and textures.
something similar to:
object_01.obj
object_02_diffuse.tga
object_02_specular.tga
object_02_roughness.tga
object_02_normal.tga
then since Houdini .OBJ saver does not write a .MTL nore assign a link to the none existing .MTL inside the .OBJ, I have written a python script that goes through the directory, inserts the link into the .OBJ and creates the .MTL based on some assumptions regarding file naming etc.
Onto the next problem. The UnReal .obj file importer can automate material creation, but it seems to only support the diffuse (map_Kd) in the .MTL file format. And the Autodesk FBX converter seems to suffer from the same limitation.
However if I open the .OBJ files in Modo with my generated .MTL files from the previous python script. all textures are assigned correctly.
So I wrote a small python script in Modo which imports the .OBJ files and saves them out as .FBX files. which then get the automatic material creation in Unreal to work.
Attached image shows the first itteration of it. since then added some more features like fixing scaling and rotation in Modo for the mesh.
-
- mdavies
- Member
- 110 posts
- Joined: May 2015
- Offline
@MagnusL3D: Wow, now you're jumping through some serious hoops (on fire?)! Hopefully we can get you going with the combination of script post-processing and geometry output.
I've added the object output (see attached), which uses the same formatting as the ‘Output Picture’ parameter. All formats supported by the File SOP are available. This way you can process in batch, like so:
UV Object 1: /obj/lowres*
High Res Object 1: /obj/highres*
Output Picture 1: $HIP/*.rat
Output UV Object 1: $HIP/*.obj
Could you send an example .mtl file? It would be great to understand your setup to see if it applies to others. There could be some standard post-processing scripts we could add (eg. write out HDAs, import baked shaders into Houdini, etc.).
I've added the object output (see attached), which uses the same formatting as the ‘Output Picture’ parameter. All formats supported by the File SOP are available. This way you can process in batch, like so:
UV Object 1: /obj/lowres*
High Res Object 1: /obj/highres*
Output Picture 1: $HIP/*.rat
Output UV Object 1: $HIP/*.obj
Could you send an example .mtl file? It would be great to understand your setup to see if it applies to others. There could be some standard post-processing scripts we could add (eg. write out HDAs, import baked shaders into Houdini, etc.).
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
@MDavies, nice with the object output. gotta get me a new build soon with all these featuers.
Yes alot of hoops jumping, that way I can get 3 out of the 4 maps automated, unfortunatley it seems that the .mtl file for .obj files do not include a channel for roughness, so even with this method I can only get diffuse/specular and normalmap over. but it is better than nothing
I've attached my modified .obj and .mtl file.
in the .obj file I had to insert the follow at the right place:
mtllib test88.mtl
usemtl test88_material
and then create the .mtl file with this
newmtl test88_material
map_Kd C
TEMP/H15/Bake_Test/test/test88_diffuse.jpg
map_Ks CTEMP/H15/Bake_Test/test/test88_specular.jpg
map_Bump CTEMP/H15/Bake_Test/test/test88_normal.jpg
But yeah, can't get the roughness to work with this. I guess the best option would be if you could export a .FBX from the lores obj, with a standard material with the assumed textures setup in the right channelsl from the baking.
Anyway .mtl/.obj file attached.
Yes alot of hoops jumping, that way I can get 3 out of the 4 maps automated, unfortunatley it seems that the .mtl file for .obj files do not include a channel for roughness, so even with this method I can only get diffuse/specular and normalmap over. but it is better than nothing
I've attached my modified .obj and .mtl file.
in the .obj file I had to insert the follow at the right place:
mtllib test88.mtl
usemtl test88_material
and then create the .mtl file with this
newmtl test88_material
map_Kd C
TEMP/H15/Bake_Test/test/test88_diffuse.jpgmap_Ks C
TEMP/H15/Bake_Test/test/test88_specular.jpgmap_Bump C
TEMP/H15/Bake_Test/test/test88_normal.jpgBut yeah, can't get the roughness to work with this. I guess the best option would be if you could export a .FBX from the lores obj, with a standard material with the assumed textures setup in the right channelsl from the baking.
Anyway .mtl/.obj file attached.
-
- MagnusL3D
- Member
- 1104 posts
- Joined: Aug. 2008
- Offline
And in the spirit of sharing idea's I came up with this.
Since my procedurally generated crosses are of different size I put down a meassure SOP, promoted the primitive “area” attribute with a Attribute promote SOP set to “sum”, and then put this crude little thing in the Bake ROP resolution:
{
r_size = detail(“../../obj/Setup/Lo_Res/”, “area”, 0);
out = if ( r_size > 9, 2048, 1024);
out = if ( r_size > 28, 4096, out);
return out;
}
So now depending on the size of the generated object it will switch between baking at a 1024, 2048 and 4096 texture.
Speaking of which, I did some test renderd of my crosses that later will be baked that you can see here:
https://www.dropbox.com/s/czfsvd7v3kxurhw/CrossGen5.jpg?dl=0 [dropbox.com]
Since my procedurally generated crosses are of different size I put down a meassure SOP, promoted the primitive “area” attribute with a Attribute promote SOP set to “sum”, and then put this crude little thing in the Bake ROP resolution:
{
r_size = detail(“../../obj/Setup/Lo_Res/”, “area”, 0);
out = if ( r_size > 9, 2048, 1024);
out = if ( r_size > 28, 4096, out);
return out;
}
So now depending on the size of the generated object it will switch between baking at a 1024, 2048 and 4096 texture.
Speaking of which, I did some test renderd of my crosses that later will be baked that you can see here:
https://www.dropbox.com/s/czfsvd7v3kxurhw/CrossGen5.jpg?dl=0 [dropbox.com]
-
- Quick Links