mlflow.projects module provides an API for running MLflow projects locally or remotely.
run(uri, entry_point='main', version=None, parameters=None, experiment_name=None, experiment_id=None, backend=None, backend_config=None, use_conda=True, storage_dir=None, synchronous=True, run_id=None)
Run an MLflow project. The project can be local or stored at a Git URI.
You can run the project locally or remotely on a Databricks.
For information on using this method in chained workflows, see Building Multistep Workflows.
ExecutionException – If a run launched in blocking mode is unsuccessful.
- uri – URI of project to run. A local filesystem path or a Git repository URI (e.g. https://github.com/mlflow/mlflow-example) pointing to a project directory containing an MLproject file.
- entry_point – Entry point to run within the project. If no entry point with the specified
name is found, runs the project file
entry_pointas a script, using “python” to run
.pyfiles and the default shell (specified by environment variable
$SHELL) to run
- version – For Git-based projects, either a commit hash or a branch name.
- experiment_name – Name of experiment under which to launch the run.
- experiment_id – ID of experiment under which to launch the run.
- backend – Execution backend for the run: “local” or “databricks”. If running against
Databricks, will run against a Databricks workspace determined as follows: if
a Databricks tracking URI of the form
databricks://profilehas been set (e.g. by setting the MLFLOW_TRACKING_URI environment variable), will run against the workspace specified by <profile>. Otherwise, runs against the workspace specified by the default Databricks CLI profile.
- backend_config – A dictionary, or a path to a JSON file (must end in ‘.json’), which will be passed as config to the backend. For the Databricks backend, this should be a cluster spec: see Databricks Cluster Specs for Jobs for more information.
- use_conda – If True (the default), create a new Conda environment for the run and install project dependencies within that environment. Otherwise, run the project in the current environment without installing any project dependencies.
- storage_dir – Used only if
backendis “local”. MLflow downloads artifacts from distributed URIs passed to parameters of type
pathto subdirectories of
- synchronous – Whether to block while waiting for a run to complete. Defaults to True.
Note that if
synchronousis False and
backendis “local”, this method will return, but the current process will block when exiting until the local run completes. If the current process is interrupted, any asynchronous runs launched via this method will be terminated.
- run_id – Note: this argument is used internally by the MLflow project APIs and should not be specified. If specified, the run ID will be used instead of creating a new run.
mlflow.projects.SubmittedRunexposing information (e.g. run ID) about the launched run.
Wrapper around an MLflow project run (e.g. a subprocess running an entry point command or a Databricks job run) and exposing methods for waiting on and cancelling the run. This class defines the interface that the MLflow project runner uses to manage the lifecycle of runs launched in different environments (e.g. runs launched locally or on Databricks).
SubmittedRunis not thread-safe. That is, concurrent calls to wait() / cancel() from multiple threads may inadvertently kill resources (e.g. local processes) unrelated to the run.
SubmittedRunmust expose a
run_idmember containing the run’s MLflow run ID.
Cancel the run (interrupts the command subprocess, cancels the Databricks run, etc) and waits for it to terminate. The MLflow run status may not be set correctly upon run cancellation.