Free & Open Source Render Farm Manager

   28865   19   6
User Avatar
Member
17 posts
Joined: Jan. 2006
Offline
General Features:
https://cgru.info/ [cgru.info]

Houdini Integration:
https://cgru.readthedocs.io/en/latest/software/houdini.html [cgru.readthedocs.io]
Edited by TimurH - Sept. 2, 2020 13:15:17
User Avatar
Member
17 posts
Joined: Jan. 2006
Offline
Version 1.5.0 released. [cgru.sourceforge.net]

It has major enhancements for houdini:

Sub-task Dependence:
Tasks can depend on other task(s) sub-progress. They can run when some task finished partially. It is useful to render simulations. As simulations usually performed by a single task on a whole frame range. Now render tasks can run when the entire simulation is not finished.

Houdini Submission Improvements: [cgru.sourceforge.net]
You can connect several Afanasy and other ROP nodes together to describe a complex job with dependences between ROPs.





This job consists of a simulation with sub-task dependence. Two caches waiting the simulation, but cat run independently from each other. Mantra tile render which produces three blocks which wait all the cache. Two blocks for preview which can run independently but wait tile render tasks. One to convert EXR files to JPEGs and one to generate a preview movie form EXRs.
User Avatar
Member
1 posts
Joined: Oct. 2011
Offline
Afanasy looks nice and seems to have had a lot of hard work go in to it.

However, there seems to be much that can be done to document and streamline the installation process. In searching the web it appears you have this squarely directed at Houdini since little else pops up in results.

I have noticed a number of errors trying to use 1.50 version as it exist. Is this still in active development or was this more of a school/business project?

-V
User Avatar
Member
17 posts
Joined: Jan. 2006
Offline
Installation on Linux managed through native distribution packages. On Windows there is no installer yet. But the next version will have be an utility to setup it and on Linux and on Windows hosts, and to integrate it with other software. Now all software integration should do system administrator.

About 10 companies use it in production about 2 years. Since version 1.0.0 this project is not for developers only, or some “beta”. But it's still in active development, and i hope it always will be so.

In the next version some “about” page will be created, with the list of companies and projects.

I send you my contacts to discuss errors.
User Avatar
Member
1615 posts
Joined: March 2009
Online
About 10 companies use it in production about 2 years.

11, actually :wink:
Martin Winkler
money man at Alarmstart Germany
User Avatar
Member
17 posts
Joined: Jan. 2006
Offline
http://cgru.info/changes_log#release200 [cgru.info]



Major cghanges in latest releases (since last post here):

- GitHub project started https://github.com/CGRU/cgru [github.com].

- New site http://cgru.info/ [cgru.info].

- Afanasy server is available for MS Windows OS.

- JSON protocol. Python API is the same, but now it just helps to constuct a valid JSON objects and send them.

- Web GUI http://cgru.info/afanasy/gui#web [cgru.info].
Online demo http://black.cinelab.ru:51000/ [black.cinelab.ru].

- All config files are based on JSON too.

- Automatic Wake-On-Lan.

- Automatic Nimby and Free depends on CPU percentage for some time.

- CGRU Keeper to launch all it's tools and any other configured.
http://cgru.info/keeper [cgru.info]

- Python 3 full support (Keeper and Movie Maker PyQt/PiSide, Afanasy job API, task services and parsers on a render client side).

- Task output parser can return activity parameter, GUIs will show and update it along with percentage (for example when stereo it can show left or right eye is processing).

- Thumbnails: Task output parser can recognize output image(s) and generate thumbnails with a configurable command.
http://cgru.info/afanasy/gui#watch_jobs [cgru.info]

- Afanasy server events: server can generate an event (for example job error or done) and push a task in a system job with all uts data. Client has a special service for it, it can constuct a command from event data (it also includes job user data) and launch this command. For you can configure afanasy to send emails on job done and error.

- RULES: Another big project started in CGRU and it will be the main its part. It is a project tracker. Online demo http://cgru.info/rules/#/CG_PROJECT/SCENES [cgru.info].

It has Web GUI and keeps all data on a file server along with projects/scenes/shots/sequences on a file server. There is no need to fill in RULES with projects/shots. Just describe where projects are located and what the structure is, rules how to find scenes, shots, sources and output. So it name comes out hee: describe rules - navigate file server. The project in its beginig but it already can:
- Navigate projects, scenes, shots on a file server. This navigation will update thumbnails, show latest result sequences and movies. Helps artists to find sources.
- Define tasks, assign artists on shots, place comments. Mark shots with any defined tags (key, cleanup, full3d, done, error, any custom). Specify percentage, see average percentage for scene/project.
- Artists receive new (comments) for shots they assigned and any other they interested in and have subscribed.
- Create Dailies: RULES always knows what is a project, scene, shot, company, logged user and so on. Simple clicking on sequence in output it sends a job in Afanasy. It annotate images (project, shot, version, artist, comments and so on), encode a movie and place it in a special location. Later RULES shows dailies (all or latest) for each shot.
- Convert movies to other format, codec, fps, explode to images sequence. All can be done parallel via Afanasy.
- View, annotate, comment jpeg sequences via a special web player.
http://cgru.info/rules/player.html#/CG_PROJECT/SCENES/PRO/PRO_0210//RESULT/JPG/v001 [cgru.info]
- Process any custom operations (commands) with files and folders via Afanasy. ( in Cinelab it also copying huge files and generates md5 )

Globally RULES is a web engine. It navgates a special folders stucture, collencts hierarchical data. Data can contain rules where to find a special assets, how to show them, change them. There is a 4 ready assets designed specially for CG VFX: Project, Scenes, Scene, Shot. But you can do anything on RULES web engine.
Here is a forum asset for RULES, you can ask questions there.
http://cgru.info/rules/#/Ask_Questions_Here/AFANASY [cgru.info]
In our company we have a special RULES servers that just copy huge files, generate md5.
User Avatar
Member
1615 posts
Joined: March 2009
Online
Great to see that 2.0 made it out to the public!
Martin Winkler
money man at Alarmstart Germany
User Avatar
Member
267 posts
Joined: July 2010
Offline
This looks like just what i need
Director @ Valkyrie Beowulf
www.vwulf.com
https://linktr.ee/neilrognvaldrscholes [linktr.ee]
User Avatar
Member
279 posts
Joined: Dec. 2009
Offline
@TimurH

Why is there not an ArchLinux installation ?

Thanks
User Avatar
Member
1615 posts
Joined: March 2009
Online
fsimerey: you can just use the generic edition to make your PKGBUILD, or even use it without a package, just “raw”.
Martin Winkler
money man at Alarmstart Germany
User Avatar
Member
279 posts
Joined: Dec. 2009
Offline
protozoan
fsimerey: you can just use the generic edition to make your PKGBUILD, or even use it without a package, just “raw”.
Thanks protozan and it's what i'm trying to do but get some errors.
Result from Afcmd:
AFERROR: Can't get file:
/opt/cgru/.htdigest
No such file or directory
Digest not loaded, authentication is disabled.
Googled it without any success.
I posted a question on Cgru forum, waiting for reply.

But just a question, in installation documentation, i not saw if i need (or how) install the database server.

Thanks
User Avatar
Member
279 posts
Joined: Dec. 2009
Offline
I would like to thank TimurH for help on CGRU forum.
Now, GCRU works fine now on my Archlinux.
User Avatar
Member
614 posts
Joined: Aug. 2008
Offline
WOW!!! so many features.
is it easy to install and configure?
User Avatar
Member
1615 posts
Joined: March 2009
Online
It's a bit like Houdini.. once you're over the initial hump, it's easy
Martin Winkler
money man at Alarmstart Germany
User Avatar
Member
1 posts
Joined: Oct. 2014
Offline
Does anyone has experience with the work of farm management software, http://www.agrivi.com/features/#plantations [agrivi.com] screened for achieving better management of agriculture and achieving better business results in the farm business?
User Avatar
Member
1391 posts
Joined: Dec. 2010
Offline
Great , Thanks
https://www.youtube.com/c/sadjadrabiee [www.youtube.com]
Rabiee.Sadjad@Gmail.Com
User Avatar
Member
96 posts
Joined: May 2008
Offline
Looks nice, I think I'll give it a go when I find some free time.

At some point someone should give the color palette and ui design a bit of thought, though (just my humble opinion)
User Avatar
Member
17 posts
Joined: Jan. 2006
Offline
Version 3.0.0 Released.

Pools and Tickets

You can set root pool NUKE:20 pool tickets to limit Nuke licenses on the entire farm.
You can set some pool MEM:64 host tickets to limit RAM.
And set corresponding tickets to your job blocks.
So you will limil 20 NUKE tasks on the entire farm.
And each host can run 1 task with 64 MEM tickets, 2 tasks with 32, or 1*32 and 2*16 on each host.

Documentation moved on ReadTheDocs

Changes log:
https://cgru.readthedocs.io/en/latest/changes_log/changes_log.html#v3-0-0 [cgru.readthedocs.io]

Step-by-step Houdini distribute simulation guide:
https://cgru.readthedocs.io/en/latest/software/houdini.html#distributed-simulations [cgru.readthedocs.io]
Edited by TimurH - Sept. 13, 2020 14:26:49
User Avatar
Member
2 posts
Joined: July 2020
Offline
Thank you TimurH

CGRU is a very nice Render Farm Manager. I use it for all my jobs.
User Avatar
Member
1529 posts
Joined: July 2005
Offline
Thanks Timur.

Been running Afanasy for a few years now. It's solid product.

Best,

G
  • Quick Links