Welcome, Guest
Username Password: Remember me
  • Page:
  • 1
  • 2

TOPIC: Improved Background Export [feature]

Improved Background Export [feature] 3 months, 1 week ago #191348

  • hugly
  • OFFLINE
  • Platinum Boarder
  • Posts: 18064
  • 3 months, 1 week ago
Lightworks stops export in the background as soon playback is started in the foreground, unfortunately without respecting system load. Stopping background processes makes surely sense as soon as editing activities reach the limits of the computer used, but always? On my new computer CPU load looks like this.

This image is hidden for guests. Please log in or register to see it.

There's plenty of headroom. The lower values indicate periods of fast playback back/forth in preview with MP4 1080p30, no effects (export in background is not running), the higher values indicate playback stopped (export is running in background). Some headroom is good to have, but over 50 percent of CPU usage in total?. I made some additional tests with HQX sources exported to Cineform, the curves don't look much different, a bit flatter maybe.

I'd suggest implementing either an automatic which allows simultaneous editing and background export and balances load of the processes or, maybe easier to implement, some settings adjustable by the user to enable/disable background export while playing and to assign a certain number of cores to foreground/background as soon as export in background is active. An alternative could be a separate program just to run scheduled export tasks in the background.

A second point: It looks to me as if Multi Core CPUs aren't well supported. A member of this forum reported very similar things (30% CPU usage on export) with a 2x12-core dual Xeon quite a while ago, with V12.6.

x264 encoder of Handbrake V1.2.0 creates the load shown below with the same source and destination format and with Legacy High settings. There's still plenty of headroom but it doubles CPU usage which might indicate that there aren't significant bottlenecks on the interconnecting buses of my system.

This image is hidden for guests. Please log in or register to see it.
It's better to travel well than to arrive...

Re: Improved Background Export [feature] 3 months, 1 week ago #191354

  • schrauber
  • OFFLINE
  • Platinum Boarder
  • Posts: 2692
  • 3 months, 1 week ago
Note also the GPU load.
In special cases maybe the free GPU memory, and the data transfer load of the drives?
Mainly automatically translated
--------------------------------------------
Windows 10, 64 Bit
Intel i5-4440 (3,1 GHz) ; Intel HD Graphics 4600

Re: Improved Background Export [feature] 3 months, 1 week ago #191355

  • hugly
  • OFFLINE
  • Platinum Boarder
  • Posts: 18064
  • 3 months, 1 week ago
As far as I know, Handbrake, when using the x264 encoder, scales on CPU only and only if necessary. As far as I understand, Lightworks uses the GPU not only for effects and display, but also for scaling. I suspect that Lightworks pumps all frames when exporting over the effects pipeline (into Video RAM over PCI and back) necessary or not. However, I'm not sure if this helps understanding the low CPU usage (and comparably low throughput) on multi-core CPUs.
It's better to travel well than to arrive...

Re: Improved Background Export [feature] 3 months, 1 week ago #191356

  • hugly
  • OFFLINE
  • Platinum Boarder
  • Posts: 18064
  • 3 months, 1 week ago
.. And it explains by no means the need to always stop background export on playback in preview.
It's better to travel well than to arrive...

Re: Improved Background Export [feature] 3 months, 1 week ago #191359

  • schrauber
  • OFFLINE
  • Platinum Boarder
  • Posts: 2692
  • 3 months, 1 week ago
I wanted to recall other cases with complex effects and low-power GPU.
When exporting I sometimes have 100% GPU load.
If in this case the export speed is not reduced, then the playback would stutter.

CPU processes can be prioritized. Does this work with the GPU?
Mainly automatically translated
--------------------------------------------
Windows 10, 64 Bit
Intel i5-4440 (3,1 GHz) ; Intel HD Graphics 4600

Re: Improved Background Export [feature] 3 months, 1 week ago #191360

  • hugly
  • OFFLINE
  • Platinum Boarder
  • Posts: 18064
  • 3 months, 1 week ago
No idea.

May I point out that this feature request is about enabling export in the background while video is playing in the foreground.

The second issue, poor support of multicore CPUs with 8 phsical cores and above is an issue known for years, because reported here and there from time to time. I think most pros are satisfied with knowing that they have a Xeon with 24 physical cores, such an expensive machine must simply perform well.
It's better to travel well than to arrive...
Last Edit: 3 months, 1 week ago by hugly.

Re: Improved Background Export [feature] 3 months, 1 week ago #191365

  • schrauber
  • OFFLINE
  • Platinum Boarder
  • Posts: 2692
  • 3 months, 1 week ago
hugly wrote:
...May I point out that this feature request is about enabling export in the background while video is playing in the foreground.
Yes, that's what I've answered, thinking of all the factors involved in implementing that goal.
Mainly automatically translated
--------------------------------------------
Windows 10, 64 Bit
Intel i5-4440 (3,1 GHz) ; Intel HD Graphics 4600

Re: Improved Background Export [feature] 3 months, 1 week ago #191371

  • RWAV
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 5874
  • 3 months, 1 week ago
Export is a critical phase of the process. When playing out in a video format via the play-out tool LW was/is configured such that the active Playout tool is highly prioritised in resource allocation in favour of other activity.

That really is the only paradigm which makes practical sense.

Most user's probably don't pass the final result to a mandatory QC checker whether human or electronic; it may even be that most users don't even look at their final render themselves - simply post it on YouTube or wherever.

Sharing limited computer resources, even if they are high capacity resources, to do wherever one pleases within LW, not knowing what demands will be placed on resources by 'editing', while simultaneously processing the final version of a work, unable to predict whether multiple high demand exporting/editing sections will collide; all of this is unlikely to be a risk-free enterprise.

Given that exporting single or a list of items can be an unattended process seems that running that risk may be easily avoided.
BETA System
Microsoft Windows 7 Professional 64BIT
HP Z800 Workstation

Re: Improved Background Export [feature] 3 months, 1 week ago #191378

  • hugly
  • OFFLINE
  • Platinum Boarder
  • Posts: 18064
  • 3 months, 1 week ago
schrauber wrote:
Yes, that's what I've answered, thinking of all the factors involved in implementing that goal.

I see.

Not long ago, with an earlier version, Lightworks exported two jobs from the export cue simultaneously. That has been dropped, because it was slower than doing it one by one which was a good decision, from my point of view. It is of course possible to run a single GPU in a multi-treading/tasking environment, same as it is with a single physical CPU. That's essential for all existing major operating systems, they wouldn't work as we know it, otherwise. How performance is affected by multi threading is another discussion and pretty complex, but looking at the chart I've posted initially, it seems I have enough unused resources waiting for a job. The logic behind it, switch one off if the other runs, is the most simple approach, but not very clever.
It's better to travel well than to arrive...
Last Edit: 3 months, 1 week ago by hugly.

Re: Improved Background Export [feature] 3 months, 1 week ago #191379

  • RWAV
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 5874
  • 3 months, 1 week ago
Another possibility which might impact on some users?
- sign off on the 'x' Hrs/Mins of game-play video edit
- replete with multiple V tracks and VFX
- and DVE positioned commentator
- set to export in LW
- minimise LW while it renders the export

Never waste an opportunity
- return to GPU intensive game-play
- and GPU intensive hi-rez screen capture

Much later
- post about very slow LW export.
BETA System
Microsoft Windows 7 Professional 64BIT
HP Z800 Workstation

Re: Improved Background Export [feature] 3 months, 1 week ago #191381

  • hugly
  • OFFLINE
  • Platinum Boarder
  • Posts: 18064
  • 3 months, 1 week ago
When exporting h.264 1080p24 to Cineform 1080p24 without effects, overall CPU load is at appr. 50 percent, GPU load at appr. 40 percent. The difference to 1080p24 Youtube export is marginal.

In fact the 1920x Threadripper isn't much faster than my old 2600 Quadcore with a 1050 TI on export with Lightworks, although all sorts of benchmarks I have run so far, show it close to the top of the list next to Xeons and Opterons.

Schrauber,

What do you think, where is the bottleneck?
It's better to travel well than to arrive...

Re: Improved Background Export [feature] 3 months, 1 week ago #191384

  • hugly
  • OFFLINE
  • Platinum Boarder
  • Posts: 18064
  • 3 months, 1 week ago
RWAV wrote:
Another possibility which might impact on some users?
- sign off on the 'x' Hrs/Mins of game-play video edit
- replete with multiple V tracks and VFX
- and DVE positioned commentator
- set to export in LW
- minimise LW while it renders the export

Never waste an opportunity
- return to GPU intensive game-play
- and GPU intensive hi-rez screen capture

Much later
- post about very slow LW export.

I'm still thinking about alternatives to fill the gap. Transcoding with Handbrake and ffmpeg works well simultaneously. The downside, I haven't enough material to convert, but I can do it twice or dance the Rumba, instead of editing,
It's better to travel well than to arrive...

Re: Improved Background Export [feature] 3 months, 1 week ago #191385

  • RWAV
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 5874
  • 3 months, 1 week ago
I'm still thinking about alternatives to fill the gap
Anything at all but stay off that computer.
Read a book, listen to as many versions as one can find of Glen Gould playing the Goldberg Variations, ponder upon the state of the universe.
BETA System
Microsoft Windows 7 Professional 64BIT
HP Z800 Workstation

Re: Improved Background Export [feature] 3 months, 1 week ago #191386

  • jwrl
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 10727
  • 3 months, 1 week ago
RWAV wrote:
listen to as many versions as one can find of Glen Gould playing the Goldberg Variations

No, I prefer Arthur Rubenstein playing the Chopin Nocturnes or perhaps Beethoven's Moonlight - although I'm not in love with the way that he handles the second movement. But at least he doesn't mutter while he's playing!

Re: Improved Background Export [feature] 3 months, 1 week ago #191387

  • RWAV
  • Moderator
    Pro User
  • OFFLINE
  • Moderator
  • Posts: 5874
  • 3 months, 1 week ago
That he was such an idiosyncratic aka flawed character as well as a genius piano player simply draws me to Gould.
BETA System
Microsoft Windows 7 Professional 64BIT
HP Z800 Workstation
  • Page:
  • 1
  • 2
Time to create page: 0.33 seconds
Scroll To Top