Displaying 2 results from an estimated 2 matches for "done_chunk".
2024 Mar 25
1
Wish: a way to track progress of parallel operations
...ble to give wrapped
cluster objects to unchanged code, but requires knowing the precise
number of chunks that the workload will be split into.
Could it be feasible to add an optional .progress argument after the
ellipsis to parLapply() and its friends? We can require it to be a
function accepting (done_chunk, total_chunks, ...). If not a new
argument, what other interfaces could be used to get accurate progress
information from staticClusterApply and dynamicClusterApply?
I understand that the default parLapply() behaviour is not very
amenable to progress tracking, but when running clusterMap(.scheduli...
2024 Mar 25
3
Wish: a way to track progress of parallel operations
...ter objects to unchanged code, but requires knowing the precise
> number of chunks that the workload will be split into.
>
> Could it be feasible to add an optional .progress argument after the
> ellipsis to parLapply() and its friends? We can require it to be a
> function accepting (done_chunk, total_chunks, ...). If not a new
> argument, what other interfaces could be used to get accurate progress
> information from staticClusterApply and dynamicClusterApply?
>
> I understand that the default parLapply() behaviour is not very
> amenable to progress tracking, but when runn...