mirai 1.3.1

Updates

• Cleanup of packages only detaches them from the search path and does not attempt to unload them, as it is not always safe to do so. Fixes daemon crashes using packages such as data.table (thanks @D3SL, #166).
• serialization() deprecated in mirai 1.2.0 is now removed.

mirai 1.3.0

New Features

• daemons(dispatcher = "thread") implements threaded dispatcher (experimental), a faster and more efficient alternative to running dispatcher in a separate process.
• mirai_map() adds [.progress_cli] as an alternative progress indicator, using the cli package to show % complete and ETA.
• daemons() gains argument 'force' to control whether further calls reset previous settings for the same compute profile.
• daemon() gains argument 'asyncdial' to allow control of connection behaviour independently of what happens when the daemon exits.

Behavioural Changes

• For daemons():
  • Argument 'dispatcher' now takes the character options 'process', 'thread' and 'none'. Previous values of TRUE/FALSE continue to be accepted (thanks @hadley #157).
  • Return value is now always an integer - either the number of daemons set if using dispatcher, or the number of daemons launched locally (zero if using a remote launcher).
  • Invalid type of ... arguments are now dropped instead of throwing an error. This allows ... containing unused arguments to be more easily passed from other functions.
• For mirai_map():
  • Now only performs multiple map over the rows of matrices and dataframes (thanks @andrewGhazi, #147).
  • Combining collection options is now easier, in the fashion of: x[.stop, .progress].
  • Collection options now work even if mirai is not on the search path e.g. mirai::mirai_map(1:4, Sys.sleep)[.progress].
• dispatcher() drops argument 'asyncdial' as it is rarely useful to set this here.
• everywhere() now errors if the specified compute profile is not yet set up, rather than fail silently.
• launch_local() and launch_remote() now strictly require daemons to be set, and will error otherwise.
• serial_config() now validates the arguments provided and returns them as a list. This means any saved configurations from previous package versions must be re-generated.

Updates

• Fixes daemons() to correctly handle a vector of URLs passed to 'url' again.
• Fixes flatmap with mirai_map()[.flat] assigning a variable 'typ' to the calling environment.
• Performance enhancements for mirai(), mirai_map() and the promises method.
• Requires nanonext >= 1.3.0.
• The package has a shiny new hex logo.

mirai 1.2.0

• everywhere() adds argument '.serial' to accept serialization configurations created by serial_config(). These allow normally non-exportable reference objects such as Arrow Tables or torch tensors to be used seamlessly across parallel processes without additional marshalling steps. Configurations apply on a per compute profile basis.
• serialization() is now deprecated in favour of the above usage of everywhere(), and will be removed in a future version.
• mirai_map() enhanced to perform multiple map over 2D lists/vectors, allowing advanced patterns such as mapping over the rows of a dataframe or matrix.
• 'mirai_map' [] method gains the option [.flat] to collect and flatten results, avoiding coercion.
• Collecting a 'mirai_map' no longer spuriously introduces empty names where none were present originally.
• Faster local daemons(dispatcher = FALSE) and make_cluster() by using asynchronous launches (thanks @mtmorgan #123).
• Local dispatcher daemons now synchronize with host, the same as non-dispatcher daemons (prevents use before all have connected).
• Fixes rare cases of everywhere() not reaching all daemons when using dispatcher.
• More efficient dispatcher startup by only loading the base package, in addition to not reading startup configurations (thanks @krlmlr).
• Removes hard dependency on stats and utils base packages.
• Requires nanonext >= 1.2.0.

mirai 1.1.1

• serialization() function signature and return value slightly modified for clarity. Successful registration / cancellation messages are no longer printed to the console.
• dispatcher() argument 'retry' now defaults to FALSE for consistency with non-dispatcher behaviour.
• remote_config() gains argument 'quote' to control whether or not to quote the daemon launch command, and now works with Slurm (thanks @michaelmayer2 #119).
• Ephemeral daemons now exit as soon as permissible, eliminating the 2s linger period.
• Requires nanonext >= 1.1.1.

mirai 1.1.0

• Adds mirai_map() for asynchronous parallel/distributed map using mirai, with promises integration. Allows recovery from partial failure or else early stopping, together with optional progress reporting.
  • x[] collects the results of a mirai_map x, waiting for all asynchronous operations to complete.
  • x[.progress] collects the results whilst showing a text progress bar.
  • x[.stop] collects the results applying early-stopping, which stops at the first error, and aborts remaining in-progress operations.
• Adds the 'mirai' method x[] as a more efficient equivalent of the interruptible call_mirai_(x)$data.
• Adds collect_mirai() as a more efficient equivalent of the non-interruptible call_mirai(x)$data.
• unresolved(), call_mirai(), collect_mirai() and stop_mirai() now accept a list of 'mirai' such as that returned by mirai_map().
• Improved mirai print method indicates whether a mirai has resolved.
• Calling daemons() with new settings when the compute profile is already set now implicitly resets daemons before applying the new settings instead of silently doing nothing.
• Argument 'resilience' retired at daemons() as automatic re-tries are no longer performed for non-dispatcher daemons.
• New argument 'retry' at dispatcher() governs whether to auto-retry in the dispatcher case.
• Fixes promises method for potential crashes when launching improbably short-lived mirai.
• Fixes bug that could cause a hang or crash when launching additional non-dispatcher daemons.
• Requires nanonext >= 1.1.0.

mirai 1.0.0

• Implements completely event-driven (non-polling) promises (thanks @jcheng5 for prototyping).
  • This is an innovation which allows higher responsiveness and massive scalability for 'mirai' promises.
• Behavioural changes to mirai() and everywhere():
  • (breaking change) no longer permits an unnamed list to be supplied to '.args'.
  • allows an environment e.g. environment() to be supplied to '.args' or as the only element of '...'.
  • allows evaluation of a symbol in the 'mirai' environment, e.g. mirai(x, x = 1).
• ssh_config() improvements:
  • new argument 'host' allows specifying the localhost URL and port to create a standalone configuration object.
  • order of arguments 'tunnel' and 'timeout' reversed.
• stop_mirai() now resolves to an 'errorValue' 20 (operation canceled) in the case the asynchronous task was still ongoing (thanks @jcheng5 #110).
• Rejected promises now show the complete error code and message in the case of an 'errorValue'.
• A 'miraiError' reverts to not including a trailing line break (as prior to mirai 0.13.2).
• Non-dispatcher local daemons now synchronize with host in all cases (prevents use before all have connected).
• [ method for 'miraiCluster' no longer produces a 'miraiCluster' object (thanks @HenrikBengtsson #83).
• Faster startup time as the parallel package is now only loaded when first used.
• Requires nanonext >= 1.0.0.

mirai 0.13.2

• mirai() and everywhere() behaviour changed such that '...' args are now assigned to the global environment of the daemon process.
• Adds with() method for mirai daemons, allowing for example: with(daemons(4), {expr}), where the daemons last for the duration of 'expr'.
• Adds register_cluster() for registering 'miraiCluster' as a parallel Cluster type (requires R >= 4.4).
• Adds is.promising() method for 'mirai' for the promises package.
• A 'miraiError' now includes the full call stack, which may be accessed at $stack.trace, and includes the trailing line break for consistency with 'as.character.error()'.
• mirai promises now preserve deep stacks when a 'miraiError' occurs within a Shiny app (thanks @jcheng5 #104).
• Simplified registration for 'parallel' and 'promises' methods (thanks @jcheng5 #103).
• Fixes to promises error handling and Shiny vignette (thanks @jcheng5 #98 #99).
• Requires R >= 3.6.

mirai 0.13.1

• Fixes regression in mirai 0.12.1, which introduced the potential for unintentional low level errors to emerge when querying dispatcher (thanks @dsweber2 for reporting in downstream {targets}).

mirai 0.13.0

• serialization adds arguments 'class' and 'vec' for custom serialisation of all reference object types.
• Requires nanonext >= 0.13.3.

CRAN release 0.12.1

• Dispatcher initial sync timeout widened to 10s to allow for launching large numbers of daemons.
• Default for ssh_config() argument 'timeout' widened to 10 (seconds).
• Fixes daemons() specifying 'output = FALSE' registering as TRUE instead.
• Fixes use of everywhere() specifying '.args' as an unnamed list or '.expr' as a language object.
• Ensures compatibility with nanonext >= 0.13.0.
• Internal performance enhancements.