License & Deployment Mix: 16 tools – 10 OSS, 4 SaaS. (OSS and SaaS counts can overlap when an open-source tool also offers a vendor-hosted edition.)
What Is AI / ML Platform Software?
AI / ML platform software provides the infrastructure and tooling needed to develop, train, deploy, and manage machine learning models and artificial intelligence workloads. These platforms span the full ML lifecycle – from data preparation and experiment tracking to model serving and production monitoring.
Unlike general-purpose compute platforms, AI / ML platforms include domain-specific capabilities that generic infrastructure cannot provide:
- Model training – distributed training across GPUs and TPUs with automatic resource scheduling, hyperparameter tuning, and checkpoint management
- Inference serving – low-latency model serving with autoscaling, batching, GPU multiplexing, and OpenAI-compatible API endpoints
- Experiment tracking – versioned tracking of parameters, metrics, artifacts, and code for reproducibility across training runs
- Model registry – centralized catalog of trained models with versioning, stage transitions (staging, production, archived), and lineage
- Feature stores – managed repositories of engineered features for consistent training and serving data pipelines
- MLOps pipelines – automated workflows for data processing, training, evaluation, deployment, and monitoring (CI/CD for ML)
- GPU management – scheduling, allocation, and monitoring of GPU resources across teams and workloads
- LLM serving – specialized inference engines optimized for large language models with KV-cache management, continuous batching, and quantization
- Vector databases – storage and retrieval of high-dimensional embeddings for RAG (retrieval- augmented generation) and semantic search
- Data labeling – annotation interfaces for images, text, audio, and video with team collaboration and quality control
The information on these pages was researched by a combination of human review and large language models. To suggest an addition or correction, please contact us. Prepared by Rhodium Systems Inc., author of the ResorsIT platform — a unified IT operations management platform for IT teams and MSPs that integrates a curated suite of open-source, commercial, and SaaS applications into a single system with shared identity, single sign-on, access control, and a common audit trail. Use this catalogue only as a starting point for your own research, and review any tool carefully against your own requirements before relying on it. Catalogue data version 2026.197.
Comparison
Feature Matrix – Open Source / Self-Hosted
| Feature | Ollama | vLLM | MLflow | Kubeflow |
|---|---|---|---|---|
| LLM serving | Yes | Yes | No | Yes* |
| Training | No | No | Tracking | Yes |
| Experiment tracking | No | No | Yes | Yes |
| Model registry | No | No | Yes | Yes |
| Pipeline orchestration | No | No | No | Yes |
| Data labeling | No | No | No | No |
| GPU management | Auto | Manual | N/A | K8s |
| Multi-model | Yes | LoRA | N/A | Yes |
| Prometheus metrics | No | Yes | No | Yes |
* = via KServe component
| Feature | Ray | Label Studio | W&B | LocalAI |
|---|---|---|---|---|
| LLM serving | Yes* | No | No | Yes |
| Training | Yes | No | Tracking | No |
| Experiment tracking | No | No | Yes | No |
| Model registry | No | No | Yes | No |
| Pipeline orchestration | Yes | No | No | No |
| Data labeling | No | Yes | No | No |
| GPU management | Yes | N/A | N/A | Auto |
| Multi-model | Yes | N/A | N/A | Yes |
| Prometheus metrics | Yes | No | No | Yes |
* = via Ray Serve (often with vLLM backend)
| Feature | LangChain | text-gen-webui |
|---|---|---|
| LLM serving | Via LangServe | Yes |
| Training | No | LoRA |
| Experiment tracking | No | No |
| Model registry | No | No |
| Pipeline orchestration | Chains | No |
| Data labeling | No | No |
| GPU management | N/A | Auto |
| Multi-model | Yes* | Yes |
| Prometheus metrics | No | No |
* = connects to multiple LLM providers/backends
Feature Matrix – Commercial / Cloud
| Feature | OpenAI | SageMaker | Vertex AI |
|---|---|---|---|
| LLM serving | Yes | Yes | Yes |
| Training | Fine-tune | Yes | Yes |
| Experiment tracking | No | Yes | Yes |
| Model registry | No | Yes | Yes |
| Pipeline orchestration | No | Yes | Yes |
| Data labeling | No | Ground Truth | Yes |
| Feature store | No | Yes | Yes |
| AutoML | No | Yes | Yes |
| Foundation models | GPT-4o, o3 | JumpStart | Gemini |
| Feature | Azure ML | Hugging Face | Replicate |
|---|---|---|---|
| LLM serving | Yes | Yes (TGI) | Yes |
| Training | Yes | Yes (libs) | Fine-tune |
| Experiment tracking | Yes | No | No |
| Model registry | Yes | Hub | Marketplace |
| Pipeline orchestration | Yes | No | No |
| Data labeling | No | No | No |
| Feature store | No | No | No |
| AutoML | Yes | AutoTrain | No |
| Foundation models | Azure OpenAI | Open models | Community |
Deployment Comparison
| Tool | Type | Self-Hosted | Cloud | GPU Required |
|---|---|---|---|---|
| Ollama | OSS | Yes | No | Optional |
| vLLM | OSS | Yes | No | Yes |
| MLflow | OSS | Yes | Managed* | No |
| Kubeflow | OSS | Yes (K8s) | Managed* | For workloads |
| Ray | OSS | Yes | Managed* | For workloads |
| Label Studio | OSS/Comm | Yes | SaaS | No |
| W&B | Commercial | Enterprise | SaaS | No |
| LocalAI | OSS | Yes | No | Optional |
| LangChain | OSS | Library | N/A | No |
| text-gen-webui | OSS | Yes | No | Recommended |
| OpenAI | Commercial | No | SaaS | N/A |
| AWS SageMaker | Commercial | No | AWS | For workloads |
| Vertex AI | Commercial | No | GCP | For workloads |
| Azure ML | Commercial | No** | Azure | For workloads |
| Hugging Face | Hybrid | Libraries | SaaS | For inference |
| Replicate | Commercial | No | SaaS | N/A |
* = managed versions available from cloud providers or vendors ** = Azure Arc for hybrid, but requires Azure connectivity
Licensing Comparison
| Tool | License | Type | Cost |
|---|---|---|---|
| Ollama | MIT | Open source | Free |
| vLLM | Apache 2.0 | Open source | Free |
| MLflow | Apache 2.0 | Open source | Free |
| Kubeflow | Apache 2.0 | Open source | Free |
| Ray | Apache 2.0 | Open source | Free |
| Label Studio | Apache 2.0 | OSS / Comm | Free / Enterprise |
| W&B | Proprietary* | Commercial | Free tier / $50+/user |
| LocalAI | MIT | Open source | Free |
| LangChain | MIT | Open source | Free |
| text-gen-webui | AGPL-3.0 | Open source | Free |
| OpenAI | Proprietary | Commercial | Pay-per-token |
| SageMaker | Proprietary | Commercial | Pay-per-use |
| Vertex AI | Proprietary | Commercial | Pay-per-use |
| Azure ML | Proprietary | Commercial | Pay-per-use |
| Hugging Face | Apache 2.0* | Hybrid | Free / Pro / Enterprise |
| Replicate | Proprietary* | Commercial | Pay-per-second |
* W&B client library is Apache 2.0; server is proprietary * Hugging Face libraries are Apache 2.0; Hub platform is proprietary * Replicate’s Cog framework is Apache 2.0; platform is proprietary
SSO / OIDC Comparison
OIDC is the preferred SSO protocol.
| Tool | OIDC | SAML | Authentik Notes |
|---|---|---|---|
| Ollama | No | No | Reverse proxy |
| vLLM | No | No | Reverse proxy |
| MLflow | No* | No | Reverse proxy |
| Kubeflow | Yes | Yes | Via Dex broker |
| Ray | No | No | Reverse proxy |
| Label Studio | Ent | Ent | Enterprise direct; OSS proxy |
| W&B | Yes | Ent | Team+ tiers |
| LocalAI | No | No | Reverse proxy |
| LangChain | N/A | N/A | FastAPI middleware |
| text-gen-webui | No | No | Reverse proxy |
| OpenAI | N/A | Ent | API key only |
| SageMaker | Yes | Yes | Via AWS IAM IdC |
| Vertex AI | Yes | Yes | Via GCP federation |
| Azure ML | Yes | Yes | Via Entra ID |
| Hugging Face | Ent | Ent | Enterprise Hub |
| Replicate | No | No | No SSO support |
* = Databricks managed MLflow supports SSO Ent = Enterprise/paid tier only
Best SSO support: Kubeflow (native OIDC via Dex), cloud platforms (AWS, GCP, Azure via identity federation), and W&B (Team+ tiers).
Self-hosted OSS tools: most self-hosted tools (Ollama, vLLM, MLflow, LocalAI, Ray) lack built-in SSO. Use a reverse proxy (Traefik, Caddy) with Authentik OIDC for access control.
LLM Serving Comparison
| Tool | Throughput | Latency | Ease of Use | Multi-GPU |
|---|---|---|---|---|
| Ollama | Low | Good | Easiest | No |
| vLLM | Highest | Best | Moderate | Yes (TP) |
| LocalAI | Low | Good | Moderate | Limited |
| text-gen-webui | Low | Good | Moderate | No |
| Hugging Face TGI | High | Good | Moderate | Yes (TP) |
| Ray Serve + vLLM | Highest | Best | Complex | Yes (TP) |
Best for production: vLLM (or Ray Serve + vLLM for autoscaling) Best for development: Ollama (simplicity) Best for multi-modal: LocalAI (text + image
- audio API)
Monitoring Integration
| Tool | Prometheus | Native Metrics | Monitoring Fit |
|---|---|---|---|
| Ollama | No | No | Process/GPU only |
| vLLM | Yes | Comprehensive | Excellent |
| MLflow | No | No | PostgreSQL metrics |
| Kubeflow | Yes | Via Istio | Excellent |
| Ray | Yes | Dashboard | Good |
| Label Studio | No | No | PostgreSQL metrics |
| W&B | No | Built-in* | Self-hosted only |
| LocalAI | Yes | Basic | Good |
| LangChain | No | No | App-level only |
| text-gen-webui | No | No | Process/GPU only |
| OpenAI | No | Usage API | Custom metrics |
| SageMaker | No | CloudWatch | Via CW plugin |
| Vertex AI | No | Cloud Monitoring | Via GCP plugin |
| Azure ML | No | Azure Monitor | Via Azure plugin |
| Hugging Face TGI | Yes | Comprehensive | Excellent |
| Replicate | No | Usage API | Custom metrics |
* = W&B tracks system metrics per training run internally, not via standard Prometheus
Best monitoring integration: vLLM, Kubeflow, Ray, and Hugging Face TGI expose Prometheus metrics that Telegraf can scrape directly.
Tools
16 tools.
AWS SageMaker
Amazon SageMaker is AWS’s fully managed ML platform covering the complete machine learning lifecycle: data preparation, model training, experiment tracking, model deployment, and monitoring.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: OIDC, SAML
Azure Machine Learning
Azure Machine Learning (Azure ML) is Microsoft’s cloud ML platform. It provides end-to-end ML lifecycle management including data preparation, automated ML, model training, deployment, and monitoring.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: OIDC, SAML
Google Vertex AI
Google Vertex AI is Google Cloud’s unified ML platform. It combines AutoML (no-code model training), custom model training, model serving, and the Gemini foundation model family into a single platform.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: OIDC, SAML
Hugging Face
Hugging Face is the central hub of the open-source ML ecosystem. Its model Hub hosts 500,000+ models and 100,000+ datasets, making it the “GitHub of machine learning.” The transformers Python library is the most widely used interface for…
License: Apache-2.0 (OSS) · Kind: web · Deploy: saas · SSO: none
Kubeflow
Kubeflow is a Kubernetes-native platform for developing, orchestrating, deploying, and running scalable and portable ML workloads.
License: Apache-2.0 (OSS) · Kind: web · Deploy: k8s · SSO: OIDC
Label Studio
Label Studio is the leading open-source data labeling and annotation platform. It supports labeling for images, audio, text, time series, video, and multi-modal data types.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: SAML
LangChain / LangServe
LangChain is the most widely used framework for building applications powered by large language models.
License: MIT (OSS) · Kind: web · Deploy: saas, docker, k8s · SSO: none
LocalAI
LocalAI is a self-hosted, OpenAI-compatible API server that acts as a drop-in replacement for the OpenAI API.
License: MIT (OSS) · Kind: web · Deploy: native · SSO: none
MLflow
MLflow is the de facto standard open-source platform for managing the end-to-end machine learning lifecycle.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Ollama
Ollama is a lightweight tool for running large language models locally. It wraps llama.cpp in a user-friendly CLI and REST API, making it trivial to download and run open-weight models like Llama, Mistral, Gemma, Phi, and Qwen on consumer h…
License: MIT (OSS) · Kind: web · Deploy: native · SSO: none
OpenAI API Platform
OpenAI’s API Platform is the dominant commercial LLM API service. It provides access to the GPT family of models (GPT-4o, GPT-4.1, GPT-4o mini, o1, o3), DALL-E for image generation, Whisper for speech-to-text, and TTS for text-to-speech thr…
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: SAML
Ray / Ray Serve
Ray is a general-purpose distributed computing framework that has become a dominant platform for scaling ML workloads. Developed at UC Berkeley (RISELab), Ray provides a simple API for distributing Python code across clusters of machines.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Replicate
Replicate is a cloud platform for running ML models via API. It provides a model marketplace with thousands of community-contributed models that can be called with a single API request.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
text-generation-webui (oobabooga)
text-generation-webui (commonly called “oobabooga” after its creator) is a Gradio-based web interface for running large language models locally.
License: AGPL-3.0-only (OSS) · Kind: web · Deploy: native · SSO: none
vLLM
vLLM is a high-performance inference engine for large language models. Developed at UC Berkeley, it introduced PagedAttention – a memory management technique that dramatically improves GPU memory utilization and throughput compared to naiv…
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Weights & Biases (W&B)
Weights & Biases (W&B) is a commercial ML experiment tracking and MLOps platform. It provides best-in-class experiment tracking, model registry, dataset versioning, and collaborative dashboards.
License: Proprietary (proprietary) · Kind: web · Deploy: native, saas · SSO: OIDC, SAML
