Afanasy 3.2.1 version supports TOPs:
https://cgru.readthedocs.io/en/latest/software/houdini.html#afanasy-top-scheduler [cgru.readthedocs.io]
Found 17 posts.
Search results Show results as topic list.
PDG/TOPs » Afanasy TOP Scheduler
- TimurH
- 17 posts
- Offline
PDG/TOPs » Has anybody implemented the "Static Cook Hybrid Approach"?
- TimurH
- 17 posts
- Offline
May be can just skip work items that has a parent Batch work item.
And make a single task for each unique batch work item.
And make a single task for each unique batch work item.
PDG/TOPs » Has anybody implemented the "Static Cook Hybrid Approach"?
- TimurH
- 17 posts
- Offline
May some working method is to get all work items at first :
And later, we should "parse" tasks batches (if any) and replace tasks that are in the same batch by one batch task.
Or there is some "magic" method that I missed?
dependencies, dependents, ready = scheduler.dependencyGraph(True)
And later, we should "parse" tasks batches (if any) and replace tasks that are in the same batch by one batch task.
Or there is some "magic" method that I missed?
PDG/TOPs » Has anybody implemented the "Static Cook Hybrid Approach"?
- TimurH
- 17 posts
- Offline
Hi, everyone!
Has anybody implemented the "Static Cook Hybrid Approach"?
https://www.sidefx.com/docs/houdini/tops/custom_scheduler.html#static-cook-hybrid-approach [www.sidefx.com]
Maybe there are already some examples in Houdini Python scripts?
Maybe some Tractor or Deadline can do this?
There is some example in documentation:
This way we get all work items.
But later on onSchedule(self, work_item) event, we can get batches.
So we can just construct all tasks in suspended state, then just trigger them to ready state on onSchedule event, like documentation says.
Is there a way to get the same tasks (work items), as they will be on an onSchedule event?
Has anybody implemented the "Static Cook Hybrid Approach"?
https://www.sidefx.com/docs/houdini/tops/custom_scheduler.html#static-cook-hybrid-approach [www.sidefx.com]
Maybe there are already some examples in Houdini Python scripts?
Maybe some Tractor or Deadline can do this?
There is some example in documentation:
dependencies, dependents, ready = scheduler.dependencyGraph(True)
But later on onSchedule(self, work_item) event, we can get batches.
So we can just construct all tasks in suspended state, then just trigger them to ready state on onSchedule event, like documentation says.
Is there a way to get the same tasks (work items), as they will be on an onSchedule event?
PDG/TOPs » PDG Afanasy Scheduler
- TimurH
- 17 posts
- Offline
Ah, I forgot, I also modified this:
We should not modify item_name, as it later looks for json data files, according this name.
- block.setEnv('PDG_ITEM_NAME', str('workitem_{}'.format(item_name))) + block.setEnv('PDG_ITEM_NAME', str(item_name))
Edited by TimurH - May 28, 2021 03:21:03
PDG/TOPs » PDG Afanasy Scheduler
- TimurH
- 17 posts
- Offline
alexwheezy
2) If the render manager does not support requests via xmlrpclib, how can I properly embed the pdgcmd.py file? Because now I just use a dirty hack in the form of its patch with a fixed file.
Hi, Alex!
You do not need to do it at all!
That "workItemStartCook" callback in pdgcmd.py needed to notify Houdini PDG graph that the item task launched a command. It is already launched by render manager and there is no need to connect the render manager here. So it is a "Houdini internal theme", not Afanasy related.
And your implementation tries to connect to Afanasy server address to notify PDG. Of course, Afanasy server can not recognize this message. And PDG thinks that tasks are not started, as it is waiting for that xmlrpc.
I just commented this two lines in your code (AfanasyScheduler class):
def workItemResultServerAddr(self): return self['address'].evaluateString()
We should not override this function.
Thank You!!!
Edited by TimurH - May 28, 2021 03:20:08
3rd Party » Free & Open Source Render Farm Manager
- TimurH
- 17 posts
- Offline
Version 3.0.0 Released.
Pools and Tickets
You can set root pool NUKE:20 pool tickets to limit Nuke licenses on the entire farm.
You can set some pool MEM:64 host tickets to limit RAM.
And set corresponding tickets to your job blocks.
So you will limil 20 NUKE tasks on the entire farm.
And each host can run 1 task with 64 MEM tickets, 2 tasks with 32, or 1*32 and 2*16 on each host.
Documentation moved on ReadTheDocs
Changes log:
https://cgru.readthedocs.io/en/latest/changes_log/changes_log.html#v3-0-0 [cgru.readthedocs.io]
Step-by-step Houdini distribute simulation guide:
https://cgru.readthedocs.io/en/latest/software/houdini.html#distributed-simulations [cgru.readthedocs.io]
Pools and Tickets
You can set root pool NUKE:20 pool tickets to limit Nuke licenses on the entire farm.
You can set some pool MEM:64 host tickets to limit RAM.
And set corresponding tickets to your job blocks.
So you will limil 20 NUKE tasks on the entire farm.
And each host can run 1 task with 64 MEM tickets, 2 tasks with 32, or 1*32 and 2*16 on each host.
Documentation moved on ReadTheDocs
Changes log:
https://cgru.readthedocs.io/en/latest/changes_log/changes_log.html#v3-0-0 [cgru.readthedocs.io]
Step-by-step Houdini distribute simulation guide:
https://cgru.readthedocs.io/en/latest/software/houdini.html#distributed-simulations [cgru.readthedocs.io]
Edited by TimurH - Sept. 13, 2020 14:26:49
Technical Discussion » Distribute Pyro Simulation Issue
- TimurH
- 17 posts
- Offline
Hi!
- I created a sphere.
- Created simulation via Wispy Smoke shelf tool.
- Applied Distribute Container shelf tool.
- Removed resize_container node.
- Set slices divisions 1x2x1
- Started htracker manually and specified its address and port.
- Opened 2 Houdini application instances specifying different slices.
- Pressed play button on both applications.
- Smoke from source (bottom) slice can't reach second (top) slice, it looks like it collides with a first slice border (ceiling).
But it works after a small network adjusting.
I connected distribute_pyro node to the solver Velocity Update input.
Originally it was merged with the source.
Houdini FX Version 17.5.460
- I created a sphere.
- Created simulation via Wispy Smoke shelf tool.
- Applied Distribute Container shelf tool.
- Removed resize_container node.
- Set slices divisions 1x2x1
- Started htracker manually and specified its address and port.
- Opened 2 Houdini application instances specifying different slices.
- Pressed play button on both applications.
- Smoke from source (bottom) slice can't reach second (top) slice, it looks like it collides with a first slice border (ceiling).
But it works after a small network adjusting.
I connected distribute_pyro node to the solver Velocity Update input.
Originally it was merged with the source.
Houdini FX Version 17.5.460
Edited by TimurH - Sept. 3, 2020 01:15:44
Technical Discussion » Houdini tiled rendering problems with Image Planes
- TimurH
- 17 posts
- Offline
Hi.
The next Afanasy release will use Houdini native itilestitch tool. Bug will gone (tested).
https://cgru.readthedocs.io/en/latest/software/houdini.html#tile-render [cgru.readthedocs.io]
The next Afanasy release will use Houdini native itilestitch tool. Bug will gone (tested).
https://cgru.readthedocs.io/en/latest/software/houdini.html#tile-render [cgru.readthedocs.io]
Edited by TimurH - Sept. 2, 2020 05:05:34
3rd Party » Free & Open Source Render Farm Manager
- TimurH
- 17 posts
- Offline
http://cgru.info/changes_log#release200 [cgru.info]
Major cghanges in latest releases (since last post here):
- GitHub project started https://github.com/CGRU/cgru [github.com].
- New site http://cgru.info/ [cgru.info].
- Afanasy server is available for MS Windows OS.
- JSON protocol. Python API is the same, but now it just helps to constuct a valid JSON objects and send them.
- Web GUI http://cgru.info/afanasy/gui#web [cgru.info].
Online demo http://black.cinelab.ru:51000/ [black.cinelab.ru].
- All config files are based on JSON too.
- Automatic Wake-On-Lan.
- Automatic Nimby and Free depends on CPU percentage for some time.
- CGRU Keeper to launch all it's tools and any other configured.
http://cgru.info/keeper [cgru.info]
- Python 3 full support (Keeper and Movie Maker PyQt/PiSide, Afanasy job API, task services and parsers on a render client side).
- Task output parser can return activity parameter, GUIs will show and update it along with percentage (for example when stereo it can show left or right eye is processing).
- Thumbnails: Task output parser can recognize output image(s) and generate thumbnails with a configurable command.
http://cgru.info/afanasy/gui#watch_jobs [cgru.info]
- Afanasy server events: server can generate an event (for example job error or done) and push a task in a system job with all uts data. Client has a special service for it, it can constuct a command from event data (it also includes job user data) and launch this command. For you can configure afanasy to send emails on job done and error.
- RULES: Another big project started in CGRU and it will be the main its part. It is a project tracker. Online demo http://cgru.info/rules/#/CG_PROJECT/SCENES [cgru.info].
It has Web GUI and keeps all data on a file server along with projects/scenes/shots/sequences on a file server. There is no need to fill in RULES with projects/shots. Just describe where projects are located and what the structure is, rules how to find scenes, shots, sources and output. So it name comes out hee: describe rules - navigate file server. The project in its beginig but it already can:
- Navigate projects, scenes, shots on a file server. This navigation will update thumbnails, show latest result sequences and movies. Helps artists to find sources.
- Define tasks, assign artists on shots, place comments. Mark shots with any defined tags (key, cleanup, full3d, done, error, any custom). Specify percentage, see average percentage for scene/project.
- Artists receive new (comments) for shots they assigned and any other they interested in and have subscribed.
- Create Dailies: RULES always knows what is a project, scene, shot, company, logged user and so on. Simple clicking on sequence in output it sends a job in Afanasy. It annotate images (project, shot, version, artist, comments and so on), encode a movie and place it in a special location. Later RULES shows dailies (all or latest) for each shot.
- Convert movies to other format, codec, fps, explode to images sequence. All can be done parallel via Afanasy.
- View, annotate, comment jpeg sequences via a special web player.
http://cgru.info/rules/player.html#/CG_PROJECT/SCENES/PRO/PRO_0210//RESULT/JPG/v001 [cgru.info]
- Process any custom operations (commands) with files and folders via Afanasy. ( in Cinelab it also copying huge files and generates md5 )
Globally RULES is a web engine. It navgates a special folders stucture, collencts hierarchical data. Data can contain rules where to find a special assets, how to show them, change them. There is a 4 ready assets designed specially for CG VFX: Project, Scenes, Scene, Shot. But you can do anything on RULES web engine.
Here is a forum asset for RULES, you can ask questions there.
http://cgru.info/rules/#/Ask_Questions_Here/AFANASY [cgru.info]
In our company we have a special RULES servers that just copy huge files, generate md5.
Major cghanges in latest releases (since last post here):
- GitHub project started https://github.com/CGRU/cgru [github.com].
- New site http://cgru.info/ [cgru.info].
- Afanasy server is available for MS Windows OS.
- JSON protocol. Python API is the same, but now it just helps to constuct a valid JSON objects and send them.
- Web GUI http://cgru.info/afanasy/gui#web [cgru.info].
Online demo http://black.cinelab.ru:51000/ [black.cinelab.ru].
- All config files are based on JSON too.
- Automatic Wake-On-Lan.
- Automatic Nimby and Free depends on CPU percentage for some time.
- CGRU Keeper to launch all it's tools and any other configured.
http://cgru.info/keeper [cgru.info]
- Python 3 full support (Keeper and Movie Maker PyQt/PiSide, Afanasy job API, task services and parsers on a render client side).
- Task output parser can return activity parameter, GUIs will show and update it along with percentage (for example when stereo it can show left or right eye is processing).
- Thumbnails: Task output parser can recognize output image(s) and generate thumbnails with a configurable command.
http://cgru.info/afanasy/gui#watch_jobs [cgru.info]
- Afanasy server events: server can generate an event (for example job error or done) and push a task in a system job with all uts data. Client has a special service for it, it can constuct a command from event data (it also includes job user data) and launch this command. For you can configure afanasy to send emails on job done and error.
- RULES: Another big project started in CGRU and it will be the main its part. It is a project tracker. Online demo http://cgru.info/rules/#/CG_PROJECT/SCENES [cgru.info].
It has Web GUI and keeps all data on a file server along with projects/scenes/shots/sequences on a file server. There is no need to fill in RULES with projects/shots. Just describe where projects are located and what the structure is, rules how to find scenes, shots, sources and output. So it name comes out hee: describe rules - navigate file server. The project in its beginig but it already can:
- Navigate projects, scenes, shots on a file server. This navigation will update thumbnails, show latest result sequences and movies. Helps artists to find sources.
- Define tasks, assign artists on shots, place comments. Mark shots with any defined tags (key, cleanup, full3d, done, error, any custom). Specify percentage, see average percentage for scene/project.
- Artists receive new (comments) for shots they assigned and any other they interested in and have subscribed.
- Create Dailies: RULES always knows what is a project, scene, shot, company, logged user and so on. Simple clicking on sequence in output it sends a job in Afanasy. It annotate images (project, shot, version, artist, comments and so on), encode a movie and place it in a special location. Later RULES shows dailies (all or latest) for each shot.
- Convert movies to other format, codec, fps, explode to images sequence. All can be done parallel via Afanasy.
- View, annotate, comment jpeg sequences via a special web player.
http://cgru.info/rules/player.html#/CG_PROJECT/SCENES/PRO/PRO_0210//RESULT/JPG/v001 [cgru.info]
- Process any custom operations (commands) with files and folders via Afanasy. ( in Cinelab it also copying huge files and generates md5 )
Globally RULES is a web engine. It navgates a special folders stucture, collencts hierarchical data. Data can contain rules where to find a special assets, how to show them, change them. There is a 4 ready assets designed specially for CG VFX: Project, Scenes, Scene, Shot. But you can do anything on RULES web engine.
Here is a forum asset for RULES, you can ask questions there.
http://cgru.info/rules/#/Ask_Questions_Here/AFANASY [cgru.info]
In our company we have a special RULES servers that just copy huge files, generate md5.
3rd Party » Free & Open Source Render Farm Manager
- TimurH
- 17 posts
- Offline
Installation on Linux managed through native distribution packages. On Windows there is no installer yet. But the next version will have be an utility to setup it and on Linux and on Windows hosts, and to integrate it with other software. Now all software integration should do system administrator.
About 10 companies use it in production about 2 years. Since version 1.0.0 this project is not for developers only, or some “beta”. But it's still in active development, and i hope it always will be so.
In the next version some “about” page will be created, with the list of companies and projects.
I send you my contacts to discuss errors.
About 10 companies use it in production about 2 years. Since version 1.0.0 this project is not for developers only, or some “beta”. But it's still in active development, and i hope it always will be so.
In the next version some “about” page will be created, with the list of companies and projects.
I send you my contacts to discuss errors.
3rd Party » Free & Open Source Render Farm Manager
- TimurH
- 17 posts
- Offline
Version 1.5.0 released. [cgru.sourceforge.net]
It has major enhancements for houdini:
Sub-task Dependence:
Tasks can depend on other task(s) sub-progress. They can run when some task finished partially. It is useful to render simulations. As simulations usually performed by a single task on a whole frame range. Now render tasks can run when the entire simulation is not finished.
Houdini Submission Improvements: [cgru.sourceforge.net]
You can connect several Afanasy and other ROP nodes together to describe a complex job with dependences between ROPs.
This job consists of a simulation with sub-task dependence. Two caches waiting the simulation, but cat run independently from each other. Mantra tile render which produces three blocks which wait all the cache. Two blocks for preview which can run independently but wait tile render tasks. One to convert EXR files to JPEGs and one to generate a preview movie form EXRs.
It has major enhancements for houdini:
Sub-task Dependence:
Tasks can depend on other task(s) sub-progress. They can run when some task finished partially. It is useful to render simulations. As simulations usually performed by a single task on a whole frame range. Now render tasks can run when the entire simulation is not finished.
Houdini Submission Improvements: [cgru.sourceforge.net]
You can connect several Afanasy and other ROP nodes together to describe a complex job with dependences between ROPs.
This job consists of a simulation with sub-task dependence. Two caches waiting the simulation, but cat run independently from each other. Mantra tile render which produces three blocks which wait all the cache. Two blocks for preview which can run independently but wait tile render tasks. One to convert EXR files to JPEGs and one to generate a preview movie form EXRs.
3rd Party » Free & Open Source Render Farm Manager
- TimurH
- 17 posts
- Offline
General Features:
https://cgru.info/ [cgru.info]
Houdini Integration:
https://cgru.readthedocs.io/en/latest/software/houdini.html [cgru.readthedocs.io]
https://cgru.info/ [cgru.info]
Houdini Integration:
https://cgru.readthedocs.io/en/latest/software/houdini.html [cgru.readthedocs.io]
Edited by TimurH - Sept. 2, 2020 13:15:17
Technical Discussion » FBX import in File Op.
- TimurH
- 17 posts
- Offline
http://sourceforge.net/projects/cgru/ [sourceforge.net]
Not all tool are documented, yet.
But if you download a release, you can find in cgru/bin folder
“fbx2clip” - FBX to HOUDINI clip command line converter
Not all tool are documented, yet.
But if you download a release, you can find in cgru/bin folder
“fbx2clip” - FBX to HOUDINI clip command line converter
Technical Discussion » BUG: mplay + xorg + "fit window to image" = X hang
- TimurH
- 17 posts
- Offline
I still have problems with build 8.1.683
On resolution 1024@768 (or less) mplay hungs, untill I begin to resize it`s window, when I press “f” key.
On 1280@1024 X hungs immediately (no need to press “f”).
(this is my monitor resolution at home)
(All images were opened from disk)
On resolution 1024@768 (or less) mplay hungs, untill I begin to resize it`s window, when I press “f” key.
On 1280@1024 X hungs immediately (no need to press “f”).
(this is my monitor resolution at home)
(All images were opened from disk)
Technical Discussion » BUG: mplay + xorg + "fit window to image" = X hang
- TimurH
- 17 posts
- Offline
Technical Discussion » BUG: mplay + xorg + "fit window to image" = X hang
- TimurH
- 17 posts
- Offline
I have the same problem.
Athlon XP
SUSE 10.1
kernel 2.6.16.13
X Window System Version 6.9.0
NVIDIA GeForce FX 5600 driver 8756
Houdini 8.1.655
I open a photo from disk (1024*768 8bit JPG) with mplay, resizing mplay window and pressing “f” key.
X hangs after that.
If I open smaller image (640x480) mplay hangs for a minute after “f” key press.
Athlon XP
SUSE 10.1
kernel 2.6.16.13
X Window System Version 6.9.0
NVIDIA GeForce FX 5600 driver 8756
Houdini 8.1.655
I open a photo from disk (1024*768 8bit JPG) with mplay, resizing mplay window and pressing “f” key.
X hangs after that.
If I open smaller image (640x480) mplay hangs for a minute after “f” key press.
-
- Quick Links