Welcome, Guest
Username Password: Remember me

TOPIC: Documentation for developing Lightworks effects (Post # 1 contains summary links)

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179699

  • schrauber
  • OFFLINE
  • Platinum Boarder
  • Posts: 4454
  • 2 years, 2 months ago
Many Thanks.

jwrl wrote:
However what happens with border addressing is a problem that I had seen raised in other threads. The GPU is apparently interpolating the border position, resulting in the effect that you see in the left hand panel. The border bites into the legal frame area.

That's probably a good thing to do from the point of view of smoothing out frame movement, but it means that we are left with a flickering black line at the edge of frame as the effects calculate discreet positions...


I tried to combine the advantages of mirror and border :
Edit:
This attachment is hidden for guests. Please log in or register to see it.




Pass 1: Mirror Sampler (reflected 10% 1% beyond the texture edges).
Pass 2: Border-Sampler (Border interpolates instead of black now with the mirroring.)

Pass 3:
RGB values from the mirrored Pass1 texture
From Pass 2, only the alpha value is used to mix the RGB values with the gray background.
Alpha after mixing I set to 1.0.

The result is very dependent on the scaling settings with my GPU.
At low minimization I have very quiet and well interpolated edges even with slow keyframing.
This attachment is hidden for guests. Please log in or register to see it.


With strong minimization I have no improvement.
The GPU load is about 50% higher than in your variants.
Mainly automatically translated
--------------------------------------------
Software: Lightworks 2020.1; || Windows 10, 64 Bit
Hardware: Intel i5-4440 (3,1 GHz); || shared RAM: 8 GB; || Intel HD Graphics 4600 (can use max. 2 GB of shared RAM)
Last Edit: 2 years, 2 months ago by schrauber. Reason: Effect replaced

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179702

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
I've never been terribly happy with border addressing, because it does different things in Cg and D3D. From memory in one it has alpha set to 1, and in the other set to 0. I'm not in front of my edit system at the moment, so I can't check - in any case my Windows rig is the only one working. My Linux box is undergoing a rebuild.

At the moment I'm exploring the option of using alpha interpolation at the edge of frame to see if that helps, but I'm not sure how much additional load it will impose. It certainly makes the logic more complicated.
Last Edit: 2 years, 2 months ago by jwrl.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179704

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
As I said, I'm not in front of my edit system at the moment so I can't test this, but this should perform an alpha interpolation if we're inside a pixel at the border edge. It might be possible to simplify it somewhat, but I'll have to wait until tonight my time (about five hours) before I can test it. First, here are the declarations.

float _OutputWidth;
float _OutputHeight;

#define EMPTY 0.0.xxxx
#define BGND  float2(0.5, 1.0).xxxy

#define FRAME float2(_OutputWidth, _OutputHeight)

The revised function should look like this. I haven't commented it, so it's a bit "magic code" at the moment.

float4 fn_tex2D_3 (sampler Vsample, float2 uv)
{
   float2 xy = abs (uv - 0.5.xx);

   if ((xy.x > 0.5) || (xy.y > 0.5)) return EMPTY;

   float2 edges = 0.5.xx - (1.0.xx / FRAME);
   float2 alpha = max (0.0.xx, 1.0.xx - (max (0.0.xx, xy - edges) * FRAME));

   float4 retval = tex2D (Vsample, uv);

   retval.a = min (retval.a, min (alpha.x, alpha.y));

   return retval;
}

To see the result of that we need to modify our ps_main(). That will now need to be something like this.

float4 ps_main_3 (float2 uv : TEXCOORD1) : COLOR
{
   float scale = 1.0 / max (Amount * 2.0, 0.00000001);
   float2 xy = ((uv - 0.5.xx) * scale) + 0.5.xx;

   xy += float2 (0.5 - CentreX, CentreY - 0.5) * scale;

   float4 retval = fn_tex2D_3 (s_Input, xy);

   return lerp (BGND, retval, retval.a);
}

The appropriate technique will then need to be added and a change made to SetTechnique to call it.

If you're still awake, schrauber, I'd appreciate a test. I think that it should work, but what I don't know is how much damage it would do to the performance. I'll test your version when I try this code out tonight.
Last Edit: 2 years, 2 months ago by jwrl.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179705

  • schrauber
  • OFFLINE
  • Platinum Boarder
  • Posts: 4454
  • 2 years, 2 months ago
Thank you, very interesting.
I'll take a closer look at the details later.

Addendum to my effects file:
I replaced my file because I accidentally used "clamp" for uv instead of xy.
Now the GPU load at scale 5% is much lower.
Probably my test still has problems when images are used with transparency, because these internal alpha values are minimized 2 times.
Presumably, a simple monochrome texture from the sampler source would have been sufficient for the border pass.
Mainly automatically translated
--------------------------------------------
Software: Lightworks 2020.1; || Windows 10, 64 Bit
Hardware: Intel i5-4440 (3,1 GHz); || shared RAM: 8 GB; || Intel HD Graphics 4600 (can use max. 2 GB of shared RAM)
Last Edit: 2 years, 2 months ago by schrauber.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179706

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
Because I haven't commented the function I'll describe what I believe will happen. (Well, I actually did comment it at first, but I pulled the comments out because I didn't feel that they were at all clear!)

  1. We first centre the address around the midpoint of the frame and make it an absolute value, so that 0.5 corresponds to the edges of the frame. This will make later maths simpler.
  2. We then check to see if either component of xy falls completely outside the frame. We return transparent black if it does.
  3. The next calculation subtracts a pixel from the frame boundary to set the inner margin, inside which we can use any pixel we find without modification.
  4. The alpha calculation determines x and y values of alpha, and if a pixel falls outside the edge value, interpolates a value between 0 and 1 for that address. 1 is closest to the safe (inner) boundary, and 0 is at the outer boundary.
  5. The pixel we need is recovered, and its alpha value is modified so that the minimum value of its alpha, alpha.x and alpha.y is used. This adds transparency at the frame boundary.
  6. The result is then returned.

And I hope that it works! I can't see why it won't, but you never know.
Last Edit: 2 years, 2 months ago by jwrl.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179709

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
Thinking about it some more, I've simplified the maths somewhat. I realised that there really is no need at all to calculate the edge width. Instead of doing this:

   float2 edges = 0.5.xx - (1.0.xx / FRAME);
   float2 alpha = max (0.0.xx, 1.0.xx - (max (0.0.xx, xy - edges) * FRAME));

- it would be possible to do this:

   float2 alpha = min (1.0.xx, (0.5.xx - xy) * FRAME);

This would work because once we get more than a pixel inside the legal edges the alpha value will become very much more than 1, so it will be limited by min() to 1. We get exactly the same result as before, with very much less messing around. And that would have to be a good thing.
Last Edit: 2 years, 2 months ago by jwrl.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179710

  • khaver
  • Pro User
  • OFFLINE
  • Platinum Boarder
  • Posts: 3510
  • 2 years, 2 months ago
jwrl, is there any difference in the border anomaly using TEXCOORD0 as opposed to TEXCOORD1?
Work Comp: Retired!
Home Comp: Newer! Bigger! Faster!
Send me a tip: paypal.me/4khaver

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179714

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
No, because from memory, it's a difference in the state of the alpha channel. In one it's turned on, in the other it's turned off. It's the difference between transparent and opaque.

I've just had my first chance to test the code that I posted in #179704 and my two subsequent posts. Just tested superficially at this stage, but it looks like it's working. The next thing is to see if it can also be readily adapted for cropping. The advantage that you get is there is no aliasing visible. When everything's square this isn't an issue, but once you start to rotate the image aliasing becomes quite a problem.
Last Edit: 2 years, 2 months ago by jwrl.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179717

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
OK, I've set up the same tests that I did previously. Here are the results, laid out the same way as the last group were. I've also included the originals that I posted for comparison.


This image is hidden for guests. Please log in or register to see it.

This image is hidden for guests. Please log in or register to see it.


The thing that interests me is how little difference there now is between the addressing modes. There's also zero impact that I can detect on the GPU too. There may be an alpha difference between border on Windows and border on Linux/Mac though.

[EDIT] I absolutely swear, hand on heart, that those three images are genuinely from three different versions of the code. I've included them here so that you can prove it for yourselves. I've done a hasty bit of code cleanup on 2 and 3, but in terms of active code they're the same as the versions I've used.
Attachments:
  • Attachment This attachment is hidden for guests. Please log in or register to see it.
Last Edit: 2 years, 2 months ago by jwrl.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179720

  • schrauber
  • OFFLINE
  • Platinum Boarder
  • Posts: 4454
  • 2 years, 2 months ago
I tested your Mirror interpolated effect.
Very good result ("Scale(x2)" >20%)
Low GPU load.
Many Thanks!

I'd be glad to see this code documented on GitHub soon.
Previously, I still need to create a developer repository

I am currently publishing the content from my post 1 (short overview) of these threads as individual Markdown files on Github.
Because the structure is still under construction, it is still temporarily on FX-Schrauber.

Example of a macro documentation.

In this example, emphasis was placed on describing the parameters in detail. Thus, without understanding the code, the developer can easily insert it and use it according to the documented parameter and environment requirements.

Similar could be created for functions and slightly modified also for directly inserted code.
Mainly automatically translated
--------------------------------------------
Software: Lightworks 2020.1; || Windows 10, 64 Bit
Hardware: Intel i5-4440 (3,1 GHz); || shared RAM: 8 GB; || Intel HD Graphics 4600 (can use max. 2 GB of shared RAM)
Last Edit: 2 years, 2 months ago by schrauber.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179753

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
khaver wrote:
jwrl, is there any difference in the border anomaly using TEXCOORD0 as opposed to TEXCOORD1?

It just occurred to me that what you might have been referring to is whether the line appears at all. I haven't tested it (yet), but I would be very surprised if there was a significant difference. I suspect that the scale and position values at which it showed up may differ, but you should still see it.

The reason that I believe that to be so is that the error sometimes appears on the right, sometimes on the left, sometimes in combination with top or bottom, sometimes not. If it was caused by a half texel shift it would be on the same two edges each time. It isn't. I went to a lot of trouble to set up defaults that deliberately created the fault on the edges that I wanted.

Quite frankly, my experience with the half texel shift is that it has been successfully handled by Lightworks and there is no need to worry about it as long as you follow the rules. When I ran the recent multi-pass tests by schrauber it proved just that.
Last Edit: 2 years, 2 months ago by jwrl.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179754

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
schrauber wrote:
I tested your Mirror interpolated effect.
Very good result ("Scale(x2)" >20%)
Low GPU load.
Many Thanks!

Glad it worked for you, and thank you for the feedback. Next weekend I'll be rebuilding (from the ground up) my Linux box, so I should soon be able to check cross platform again. My previous Linux setups have been built from scrap odds and ends I've had lying around, so they've never had a long service life. Hopefully with new parts it will be different.

Personally, as I think I've said elsewhere, I don't favour the use of macros, and won't use them in my code. They can lead to compiled code inflation, which in this context is not a good idea. There is also a tendency to regard them as being the actual value returned rather than a macro that returns a value. That again can add overhead to the code when a macro is used instead of a variable, leading to inefficiencies. And using them without understanding exactly what it is that they're doing internally can build bugs into code that can be extremely hard to track down and fix.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179757

  • schrauber
  • OFFLINE
  • Platinum Boarder
  • Posts: 4454
  • 2 years, 2 months ago
My interest is in general a structured code library that contains frequently or helpful small code.
These do not have to be macros.
Mainly automatically translated
--------------------------------------------
Software: Lightworks 2020.1; || Windows 10, 64 Bit
Hardware: Intel i5-4440 (3,1 GHz); || shared RAM: 8 GB; || Intel HD Graphics 4600 (can use max. 2 GB of shared RAM)

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179762

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 12887
  • 2 years, 2 months ago
I think that a structured code library is a good idea.

Continuing my testing: I cobbled together a really crappy 3D DVE effect with the third and fourth algorithms. That's the last "bad" algorithm, and the "good" one. Here are the results on a rotated image, using default addressing, i.e., "sampler s_Input = sampler_state { Texture = <Inp>; };". I did it using an undefined address mode because it was the worst case scenario that I could think of.


This image is hidden for guests. Please log in or register to see it.


Looks hopeful, no? I'm not saying that the problem is solved and I won't say that until I can test the code on a Linux system, but I'm cautiously optimistic. If it works cross-platform I can see a lot of my past code getting this fix.

One word of caution: it probably wouldn't be a good idea to use this inside a loop. For example, I wouldn't use it with a blur effect. The extra overhead isn't much, but it all adds up if you do it sixty times in a row! Actually two words of caution: it's also a good idea to stick with explicitly defining samplers, too!!!

[EDIT] One thing that occurs to me is that this should very neatly bypass the Clamp/ClampToEdge issue. It should also be fully backwards compatible - it doesn't depend on anything that wasn't already present in the first multi-platform version of Lightworks.
Last Edit: 2 years, 2 months ago by jwrl.

Re: Documentation for developing Lightworks effects 2 years, 2 months ago #179819

  • schrauber
  • OFFLINE
  • Platinum Boarder
  • Posts: 4454
  • 2 years, 2 months ago
I have yet seen a small side effect.
If the original scaling and original position are set, then the edges are interpolated with the gray background.
This image is hidden for guests. Please log in or register to see it.

(1 pixel wide gray border)

For comparison purposes; Effect disabled:
This image is hidden for guests. Please log in or register to see it.
Mainly automatically translated
--------------------------------------------
Software: Lightworks 2020.1; || Windows 10, 64 Bit
Hardware: Intel i5-4440 (3,1 GHz); || shared RAM: 8 GB; || Intel HD Graphics 4600 (can use max. 2 GB of shared RAM)
Last Edit: 2 years, 2 months ago by schrauber.
Time to create page: 0.43 seconds
Scroll To Top