In these times a product has to appeal to a larger audience for a more secure future. It doesn't have to sacrifice any of the current paradigms which made Houdini the powerhouse for TDs that it is, just add and improve in certain areas.
Keeping it in this esoteric state does not bode well with popularity, which, whether we like it or not, does have a lot to say in this economy as far as a product's survival goes.
Well, an audience above a certain size can have an unfortunate diluting effect (I'm just stating facts, not promoting elitism, just saying to avoid any misunderstandings ) I'm certainly not talking about Softimage (especially ICE) users. On the contrary: I wish there were more procedural-approach 3d apps out there (with SI, at least there was two, now with the EOL it will be only Houdini.)
Also, probably what you feel like “esoteric” is the SOP/modeling part, which is mostly quite old (see my comment above about the 10+yrs tools). Although Houdini can have a certain “old-school” feel to it, most of the relevant internal parts are quite modern, and it is a robust app (I do VFX, so I mostly have experience specific to that, but I'm pumping through it an insane amount of geometry data on a daily basis). It's a workhorse.
A few unrelated examples:
- you can open a geometry details view of a million-poly object or particle and it pops up instantly, and you won't experience any slowdowns
- the native bgeo geometry format is also the geometry archive format for mantra, the renderer (so your saved objects are automatically renderer stand-ins)
- the shading language can be used (apart from shaders) to modify (or build) geometry, simulation behaviours, animation channels, etc.
- the mantra renderer is about as capable as Arnold.
If I had my own shop, I'd definitely go for a mostly Houdini-based pipeline (except modeling and perhaps animation).
Anyway – my guess is that any area you feel “esoteric” is where there wasn't enough emphasis for development over the years (it also can be a cycle, e.g. “why develop animation tools if no one animates / no one animated because the tools are not good enough”).
Also, major features that are implemented are generally well maintained and cared for (they implement a feature in one version, then usually you can expect vast improvements in the next big release – not like in Maya where they just go for the feature sheet, and once it “has the feature” they just don't care any more).