HDK replace it as other method which I don't know or I don't need to call this function in HDK?

Hello,

I'm not sure what you are trying to do exactly but you shouldn't need to call Py_Finalize in your HDK plugin. If you just want to clean up Pythonic things on shutdown then you can register an exit callback with UT_Exit::addExitCallback:
http://www.sidefx.com/docs/hdk16.0/class_u_t___exit.html [sidefx.com]


Cheers,
Rob

rvinluan
Hello,

I'm not sure what you are trying to do exactly but you shouldn't need to call Py_Finalize in your HDK plugin. If you just want to clean up Pythonic things on shutdown then you can register an exit callback with UT_Exit::addExitCallback:
http://www.sidefx.com/docs/hdk16.0/class_u_t___exit.html [sidefx.com]


Cheers,
Rob

Thanks for your prompt! So does it mean I don't need to call PY_Py_InitializeEx in HDK as well ?

And I have ran into another problem about PY_IntrepreterAutoLock now. Can I get any help about this function ? I have browsed the messages in that header file but still feel a little bit confuse. And the wrong way to use it cause my Houdini crash with a segmentation fault.

That's correct. You do not need to call Py_Initialize since Houdini does the Python initialization for you on startup.

Check out $HFS/toolkit/samples/HOM/ObjNode_setSelectable.C. There's an example of using PY_InterpreterAutoLock in the HOMextendLibrary() function. The idea is to create a PY_InterpreterAutoLock object right before you intend to make Python calls. Typically you create the object on the stack so that it is created when execution enters the code block's scope and is destroyed when execution exits the scope.

Cheers,
Rob

Thank you for reminding. This example is really make sense.

Actually I have use HOMextendLibrary long times ago but totally unaware of GIL problems. But this time when I try to call external python module in HDK context, it cause a big trouble.

rvinluan
That's correct. You do not need to call Py_Initialize since Houdini does the Python initialization for you on startup.

Check out $HFS/toolkit/samples/HOM/ObjNode_setSelectable.C. There's an example of using PY_InterpreterAutoLock in the HOMextendLibrary() function. The idea is to create a PY_InterpreterAutoLock object right before you intend to make Python calls. Typically you create the object on the stack so that it is created when execution enters the code block's scope and is destroyed when execution exits the scope.

Cheers,
Rob

Hi, Rob. I have followed the way to use PY_InterpreterAutoLock from ObjNode_setSelectable, but it looks can't work well in my context.

I try to call external python module in my custom FS_ReaderHelper, the pseudo-codes are like this:

pyHelper(UT_String source, UT_String &target)
{
PY_InterpreterAutoLock interpreter_auto_lock;
…make Python API calls…
}

MY_Helper::createStream(const char *source, const UT_Options *)
{
…call pyHelper to convert source…
}

void installFSHelpers()
{
new MY_Helper();
}

Then Houdini will crash during starting with a segmentation fault from PY_PYModule_GetDict in my pyHelper function, the traceback information figure out things happen in libpthread. Obviously about GIL.

This is confusing me. Any suggestions would be helpful!

The pseudo-code looks ok to me.

Would you be able to send me your source code? I can give it a try here and see if I can reproduce the crash.

rvinluan
The pseudo-code looks ok to me.

Would you be able to send me your source code? I can give it a try here and see if I can reproduce the crash.

Thanks so much. The attachment is my source code.

The whole version is in our company's computer, I simply rewrite this to reproduce the crash. But interestingly, I found it will crash Houdini/Hython/Hbatch in Linux but only crash Houdini in Windows.

I was able to reproduce the crash on startup. It's a tricky situation that took me a bit of time to figure out.

The problem is that FS_HelperConverter::pyExecutionis called extremely early on Houdini startup, well before the Python library has loaded and initialized.

The best solution that I could come up with was to force loading the Python library and then check if Python is initialized. If it is initialized then run Python code as usual, otherwise, do nothing since Python is not ready yet. Don't even try to initialize Python. Just let Houdini initialize Python later on during the regular startup sequence since Houdini needs to do other important things for the initialization.

Anyway, in pyExecutionI added these lines of code at the very top of the function body:

I hope this helps.

Cheers,
Rob

Years ago I ran into the same issue trying to have a FS helper that would do various Python related calls. I think we got around the issue (at the suggestion from someone at SESI) by checking whether OPgetDirector() returned a non-null value. Basically our helper would only work if Houdini was in a far enough along state that we'd know Python was initialized properly. If OPgetDirector() didn't return valid we'd just say we couldn't handle things, etc.

Oh, one other thing I forgot to mention is that `sys.path` may not be configured yet with Houdini paths when pyExecutionruns. When I was testing, I had to set the PYTHONPATH environment variable to point to the folder containing myHelper.pybefore launching Houdini.

Without setting PYTHONPATH I found that myHelper.py could not be found and imported so pyExecutionwould crash still.

graham
Years ago I ran into the same issue trying to have a FS helper that would do various Python related calls. I think we got around the issue (at the suggestion from someone at SESI) by checking whether OPgetDirector() returned a non-null value. Basically our helper would only work if Houdini was in a far enough along state that we'd know Python was initialized properly. If OPgetDirector() didn't return valid we'd just say we couldn't handle things, etc.

Ah, that's good way to check too. It is a more indirect check for Python initialization but a good one especially if the helper needs to access Houdini things.

Also, if you check for OPgetDirector()instead then I believe that you don't need to set the PYTHONPATH environment variable beforehand like I mentioned above.

rvinluan
Oh, one other thing I forgot to mention is that `sys.path` may not be configured yet with Houdini paths when pyExecutionruns. When I was testing, I had to set the PYTHONPATH environment variable to point to the folder containing myHelper.pybefore launching Houdini.

Without setting PYTHONPATH I found that myHelper.py could not be found and imported so pyExecutionwould crash still.

It works! There are only two questions which aren't really serious.

First, it looks like HDK doesn't need PY_Py_DECREF to release the memory of PY_PyObject, this function will crash Houdini.

And second, when I try to let createStream & splitIndexFileSectionPath & combineIndexFileSectionPath call Python at same time, Houdini will crash when it starts (but Hython & Hbatch are well). I'm not sure whether my code leads to this or not.

Thanks a lot, I'll keep work on this and share new findings here.

YKZDY
First, it looks like HDK doesn't need PY_Py_DECREF to release the memory of PY_PyObject, this function will crash Houdini.

I guess that's possible if something else is implicitly calling PY_Py_DECREF for you. I don't think it's Houdini/HDK though.

YKZDY
And second, when I try to let createStream & splitIndexFileSectionPath & combineIndexFileSectionPath call Python at same time, Houdini will crash when it starts (but Hython & Hbatch are well). I'm not sure whether my code leads to this or not.

Hmmm. Maybe you need to create the interpreter auto lock in each of those functions?

Cheers,
Rob

rvinluan
Hmmm. Maybe you need to create the interpreter auto lock in each of those functions?

Cheers,
Rob

No, there's only one function contains CPythonAPI, but there are lots of methods (over ten) from FS_InfoHelper & FS_ReadHelper need to call this function at same time. So finally I make a static local variable in that function then use it to avoid repeatedly python call.

So everything work well now, thanks for your help !