For HPC users: Replace crew_controller_local() with crew_controller_slurm() and define your job submission template. The API remains identical.
But the real magic happens when you pair crew with targets . In a _targets.R file, changing the controller is a one-line edit: the crew pkg
That’s it. The controller sits in your main R session. You push tasks to it, and it distributes them to persistent, resilient R sessions running in the background. # Non-blocking push controller$push( name = "long_compute", command = slow_function(data) ) Collect results later result <- controller$pop() In a _targets
Because workers auto-restart after a memory threshold or crash, that file that causes a segmentation fault only kills its worker. The other seven keep humming along, and a new worker spins up to retry the bad file. crew is not for every use case. If you are doing interactive, exploratory work where you need to inspect every object in the global environment immediately, stick with lapply or furrr . If you are doing interactive
But crew (which stands for oordinated R esource E xecution W orker) isn't just another entry in the parallel-processing catalog. Created by William Landau, the author of the targets package, crew is a fundamental rethink of how R should talk to background jobs.