Self-Hosting MLflow

The most vendor-neutral MLOps/LLMOps platform in the world.

MLflow is fully open-source. Thousands of users and organizations run their own MLflow instances to meet their specific needs. Being open-source and trusted by the popular cloud providers, MLflow is the best choice for teams/organizations that worry about vendor lock-in.

The Quickest Path: Run mlflow Command

The easiest way to start MLflow server is to run the mlflow CLI command in your terminal. This is suitable for personal use or small teams.

First, install MLflow with:

pip install mlflow

Then, start the server with:

mlflow server --backend-store-uri sqlite:///mlflow.db --port 5000

This will start the server and UI at http://localhost:5000. You can connect the client to the server by setting the tracking URI:

import mlflow

mlflow.set_tracking_uri("http://localhost:5000")

# Start tracking!
# Open http://localhost:5000 in your browser to view the UI.

Now, you are ready to start your experiment!

• Tracing QuickStart
• LLM Evaluation Quickstart
• Prompt Management Quickstart
• Model Training Quickstart

tip

The --backend-store-uri option is not mandatory, but highly recommended for better performance and reliability. Check out Backend Store.

Other Deployment Options

Docker Compose

The MLflow repository includes a ready-to-run Compose project under docker-compose/ that provisions MLflow, PostgreSQL, and MinIO.

git clone https://github.com/mlflow/mlflow.git
cd docker-compose
cp .env.dev.example .env
docker compose up -d

Read the instructions here for more details and configuration options for the docker compose bundle.

Kubernetes

To deploy on Kubernetes, use the MLflow Helm chart provided by Bitnami or Community Helm Charts.

Cloud Services

If you are looking for production-scale deployments without maintenance costs, MLflow is also available as managed services from popular cloud providers.

• Databricks
• AWS Sagemaker
• Azure Machine Learning
• Nebius
• GCP (GKE)

Architecture

MLflow, at a high level, consists of the following components:

1. Tracking Server: The lightweight FastAPI server that serves the MLflow UI and API.
2. Backend Store: The Backend Store is relational database (or file system) that stores the metadata of the experiments, runs, traces, etc.
3. Artifact Store: The Artifact Store is responsible for storing the large artifacts such as model weights, images, etc.

Each component is designed to be pluggable, so you can customize it to meet your needs. For example, you can start with a single host mode with SQLite backend and local file system for storing artifacts. To scale up, you can switch backend store to PostgreSQL cluster and point artifact store to cloud storage such as S3, GCS, or Azure Blob Storage.

To learn more about the architecture and available backend options, see Architecture.

Access Control & Security

MLflow support username/password login via basic HTTP authentication, SSO (Single Sign-On), and custom authentication plugins.

MLflow also provides built-in network protection middleware to protect your tracking server from network exposure.

Try Managed MLflow

Need highly secure MLflow server? Check out Databricks Managed MLflow to get fully managed MLflow servers with unified governance and security.

FAQs

See Troubleshooting & FAQs for more information.