How to use scipy with Houdini 17 on windows?

   989   13   4
User Avatar
Member
11 posts
Joined: Jan. 2017
Offline
Hello!

before Houdini 17 this link showed how to use scipy with Houdini even for windows http://wordpress.discretization.de/houdini/home/advanced-2/installing-and-using-scipy-in-houdini/ [wordpress.discretization.de]

However, this trick does not work anymore. I really need it for my work.

Overwriting too many files using the anaconda distribution gets Houdini stuck at startup.

Copying only the scipy and/or numpy packages into C:\Program Files\Side Effects Software\Houdini 17.0.352\python27\lib\site-packages results in errors at the node.
Error
Python error: Traceback (most recent call last):
File “”, line 3, in
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.352\python27\lib\site-packages\scipy\__init__.py”, line 119, in
from scipy._lib._ccallback import LowLevelCallable
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.352\python27\lib\site-packages\scipy\_lib\_ccallback.py”, line 1, in
from . import _ccallback_c
ImportError: DLL load failed: The specified module could not be found.

When overwriting the C:\Program Files\Side Effects Software\Houdini 17.0.352\python27\dlls with the anaconda distribution, _ctypes.pyd and _socket.pyd seem to cause errors at startup.

I hope that someone here has more knowledge on setting up python libraries and can help out.

Has anyone managed to use scipy in Houdini and can please share how to do that here?

I will update the above website with that information.

thanks!
User Avatar
Member
127 posts
Joined: Jan. 2013
Offline
also ask
User Avatar
Member
358 posts
Joined: Nov. 2008
Offline
That error looks like you will probably have to build scipy with Visual Studio 17 (C++14)
Houdini 17.0 has already numpy 1.12.1
Edited by pezetko - Oct. 18, 2018 14:18:18
User Avatar
Member
11 posts
Joined: Jan. 2017
Offline
Yes you might be right and I wish I could try it out.

According to this site https://docs.scipy.org/doc/scipy/reference/building/windows.html [docs.scipy.org] I need to go through several steps in order to try it. I have not worked with Visual Studio yet. Has anyone tried this successfully?
User Avatar
Member
698 posts
Joined:
Offline
I've spent a few weeks on this and asked more clever people for guidance. After setting up mingwpy and going through various compilation guides I gave up and told Sidefx to drop the rfe I've posted for including it to Houdini as I understood the scale of issue.
B.Henriksson, DICE
User Avatar
Member
11 posts
Joined: Jan. 2017
Offline
kahuna031
I gave up

Me too. Scipy on Houdini 17 also did not work on my opensuse but I did not experiment enough why.

I just made the rfe too.
User Avatar
Member
698 posts
Joined:
Offline
What are you after from scipy? I'm thinking a.more realistic expectation would be for sidefx to support specific features rather than the whole package.
B.Henriksson, DICE
User Avatar
Member
11 posts
Joined: Jan. 2017
Offline
kahuna031
What are you after from scipy? I'm thinking a.more realistic expectation would be for sidefx to support specific features rather than the whole package.

We do mathematics and computer graphics research and need the linear algebra packages because of its sparse matrix solvers. Python is the goto language for many algorithms we need and it would be great to stick to it.(examples [page.math.tu-berlin.de])
Edited by MarcelPP - Nov. 12, 2018 12:33:33
User Avatar
Member
698 posts
Joined:
Offline
I was also after the sparse solvers but ended up modifying a git project that was wrapping libigl that in turn is a Eigen wrapper where you got sparse solvers:
https://github.com/bhenriksson031/EdgyEggs [github.com]
Edited by kahuna031 - Nov. 12, 2018 12:40:00
B.Henriksson, DICE
User Avatar
Member
89 posts
Joined: Jan. 2008
Offline
Any news on this?
It's actually of major importance for us as well.
User Avatar
Member
11 posts
Joined: Jan. 2017
Offline
No sorry still no news. We just keep using 16.5 and hope for change.
User Avatar
Member
11 posts
Joined: Jan. 2017
Offline
I am trying this again with Houdini 17.5

When installing a version of scipy, I copy over the folders from anaconda2

scipy
scipy-1.1.0-py2.7.egg-info

into C:\Program Files\Side Effects Software\Houdini 17.5.173\python27\lib

then get this error when trying to import scipy in a python node e.g. from scipy.sparse import csr_matrix

Python error: Traceback (most recent call last):
File “”, line 1, in
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.173\python27\lib\site-packages\scipy\__init__.py”, line 77, in
from . import _distributor_init
ImportError: cannot import name _distributor_init

according to this source: https://stackoverflow.com/questions/48423782/importerror-cannot-import-name-distributor-init [stackoverflow.com]
I can comment out the import _distributor_init and in the __init__.py so after that I get this message:

Error
Python error: Traceback (most recent call last):
File “”, line 1, in
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.173\python27\lib\site-packages\scipy\__init__.py”, line 119, in
from scipy._lib._ccallback import LowLevelCallable
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.173\python27\lib\site-packages\scipy\_lib\_ccallback.py”, line 1, in
from . import _ccallback_c
ImportError: DLL load failed: The specified module could not be found.

I think this error goes back to the _ctypes.pyd in C:\Program Files\Side Effects Software\Houdini 17.5.173\python27\dlls
placing the_ctypes.pyd into that folder makes Houdini not start up anymore.

What now? Can anyone help out? Especially if you have windows, please write us if anything worked for you.

thanks in advance.
User Avatar
Member
89 posts
Joined: Jan. 2008
Offline
As far as I know the python distribution that comes with Houdini on Windows was compiled with Visual Studio 2017. You can read this of from the first line in the python shell

Python 2.7.15 (default, Jan 13 2019, 12:40:45) on win32


So whatever you want to add to Houdini on Windows needs to be compiled with the same version ( MSC v.1916 ). Unfortunately, I wasn't able to find any scipy binaries compiled with this particular compiler for python 2.7. So I guess if you don't want to compile it yourself, and sesi cannot do that for you, it might challenging to get it to work.


Depending on your usecase there might be certain workarounds possible. But all of them are not ideal.
Please let me know if you find any solution working for you.
User Avatar
Member
698 posts
Joined:
Offline
The same compilers is simply not available on windows and it would be a massive job for sidefx supporting a workaround. Reading the history around the project behind the old win-scipy has convinced me of this.

As I mentioned before my suggestion would be to find the specific feature you want from scipy and implement this. Think that would also be a more reasonable rfe for sidefx.
B.Henriksson, DICE
  • Quick Links