Martin Matzeder

sanostol

About Me

Expertise
Technical Director
Location
Germany

Connect

Recent Forum Posts

PDG Graphs on Farm Oct. 22, 2019, 6:46 a.m.

that would help a lot

PDG Graphs on Farm Oct. 15, 2019, 4:15 a.m.

Hi Manuel,

yes we are using the muster scheduler. the vvertex developer is very supporting, very cool and we are in contact with him.

Hi Chris,

I get Your point (more or less), with a typical static job description You loose a lot of functionality. Still trying to get the broader concepts when submitting stuff to farms. I think I have to install at some point hqueue to be able to compare. The downside I see right now with a full pdg submission to farm is the lack of overview. Jobs pop into existence depending on previous jobs, what makes it hard to keep track if You have multiple jobs with dependencies. it is hard to keep track. Right now if a job in the chain fails, I'm not to sure if I can restart this specific job/junk and everything is just fine.

I specifically have have tasks in mind, that are low maintenance, proven and of high count. Generic stuff, creature fx, stuff like that. they are less dynamic in job creation. asset imports, collision preparation, presim, sim and postsim passes, flip book and ffmpeg. maybe a bit wedging. they are not difficult, but most of the time You have to deal with huge amounts and just want to throw it on to the farm, and see result clips before You continue to push it into the next department.

It would be easier with the top interface embedded into the scheduler.
What position does Pilot have here?

Thanks for Your insight
Martin

PDG Graphs on Farm Oct. 14, 2019, 10:38 a.m.

Hi Manuel, thank You.
I'm a bit confused as we now connect to the farm renders with a musterscheduler, not sure where a additional scheduler fits here. But I see, there are “submit graph as job buttons” in hqueue and deadline as well. Does that mean that the complete graph is processed as a job on farm, just as if You open the scene and do a Cook Output Node?

I was looking for a even more static way. More like all static tasks are evaluated with all dependencies, and then all nodes that create files are submitted as “indipendent” job to the dispatcher, but with correct dependencies set on job or chunk level.
I tried to resolve the dependencies with the api, but I could not easily get over nodes like wait for all, as they seem to have no staticWorkItems jsut staticWrapers, that made me wonder is there maybe already a defined way in the api, to resolve all dependencies.

About the expanding jobs stuff, I think this is possible. right now when I start a job with several top nodes, the first batches of tasks get launched on the assigned computers, and no other job exists for muster at this moment. as soon as one batch is finished a dependent batch pops into existence on muster. the only info it has is the, top node, task name and framerange.
looks like thisL: –batch -p “filename” -n “\obj\geo1\fileCache1\render” -i “ropfetch40” -fs 1 -fe 1 -fi 1
muster seems to just connect to the farm. all the dispatching happens in pdg. hope that helps

Martin