Area Smoothing in Distort

   402   4   0
User Avatar
Member
122 posts
Joined: March 2025
Offline
I really like the Distort tool, but I really dislike the jagged pixels (aliasing artifacts) that appear under heavy compression. Even though OpenCL has the imagesamplerect() method specifically designed for these cases, for some reason, it isn't utilized in Distort. Right now, I have to manually do an extra pass using a UV map and smoothing. Attached is a screenshot comparing native distort versus smoothed. I think it would be great to have built-in smoothing directly in Distort without all these extra operations. Maybe there is already a way to achieve this that isn't obvious to me?
There is also another frustrating nuance with borders during distortion. Currently, the "Constant" boundary mode fills areas outside the UV square with zero. This works okay for brightness and color when these areas are distorted into the visible zone. However, for UV maps, 0 represents the center. This means during sampling, we end up filling these areas with the color of the center pixel instead of actual zero. As a workaround, I have to distort the alpha channel simultaneously and multiply by it.
Honestly, I keep thinking how much easier it would be to have a customizable constant value. If we could set it to -2, for example, the UV sample would return "black," and a single pass would solve all our problems. Furthermore, introducing a negative default value for brightness and backgrounds during rasterization would make it incredibly easy to clip unwanted areas while keeping zero functional.
Edited by Gaalvk - Oct. 9, 2026 08:26:02

Attachments:
Screenshot 2026-10-09 145825.jpg (251.1 KB)

User Avatar
Staff
217 posts
Joined: Sept. 2017
Offline
The crunchiness is likely due to the default Filter being Bilinear. It should look better if you change that to something higher-quality, like Box or B-spline.

That's true with zero being a bad constant border value for UV maps, but any constant value will probably have issues of one sort or another... Your idea to distort the "premultiplied" UV values is a pretty clever workaround. Does Clamp border work any better in these cases as a border condition?
User Avatar
Member
9738 posts
Joined: July 2007
Offline
omarz
The crunchiness is likely due to the default Filter being Bilinear. It should look better if you change that to something higher-quality, like Box or B-spline.
in my experience this doesn't help much for such areas as the samples within neighbouring pixels in those cases come from source pixels quite far apart, so source filtering doesn't matter as much
this would need to be antialiasing post filtering, using more samples per pixel or something which COPs are infamously lacking for now compared to COP2
so I usually upscale-distort-downscale

EDIT: here is a simple comparison, crop of distorted 2k texture, which obviously gets temporarily to 8k for 4x upscale
Edited by tamte - Oct. 9, 2026 13:46:22

Attachments:
cop_distort_sampling_22.0.465.gif (134.4 KB)

Tomas Slancik
CG Supervisor
Framestore, NY
User Avatar
Member
122 posts
Joined: March 2025
Offline
Yes, exactly: two adjacent pixels in the deformed texture are far apart on the original—tens or hundreds of pixels away—so jaggedness is inevitable, and standard filtering won't help. A standard resample is quite heavy and only samples four pixels. However, I can easily look up the original X/Y coordinates of the neighbors via the deformed UV map, calculate the rectangle effectively hidden within the current pixel, and compute the average color from the source. I capped the sampling area at 64 pixels and added adjustable blending and thresholds for each axis. There is virtually no speed difference compared to a standard distort operation.
I assumed that the "distort" node—while generating the distortion—already has access to this neighbor data. This would allow it to sample a calculated rectangle based on neighbor coordinates rather than just a single pixel from the source UV, producing a smooth result immediately without requiring the user to perform these operations externally. Then again, I might be mistaken, and a second pass might be necessary regardless.
I don't think full mipmapping is required; simply setting reasonable limits on the averaging area should suffice.
Regarding UV map clamping: I simultaneously distorted a white layer set to "constant" border mode and multiplied it by the sampling result—effectively using it as an alpha channel—which solved all the edge issues. My specific use cases always stayed within a single square.
User Avatar
Member
9738 posts
Joined: July 2007
Offline
Gaalvk
. However, I can easily look up the original X/Y coordinates of the neighbors via the deformed UV map, calculate the rectangle effectively hidden within the current pixel, and compute the average color from the source.
While this sounds like an useful approximation it may be lacking some precision
Id hope for some generic adaptive AA sampling for COP nodes which would compute and filter N real samples per pixel based on some metrics like in this case maybe based on distortion map derivatives or something
Edited by tamte - Oct. 9, 2026 14:44:48
Tomas Slancik
CG Supervisor
Framestore, NY
  • Quick Links