What Is AI / ML Platform Software?
Mix (16 tools)
- License: 10 open source · 6 proprietary
- Hosting: 8 self-hosted · 6 SaaS · 2 both
AI / ML platform software provides the infrastructure and tooling needed to develop, train, deploy, and manage machine learning models and artificial intelligence workloads. These platforms span the full ML lifecycle – from data preparation and experiment tracking to model serving and production monitoring.
Unlike general-purpose compute platforms, AI / ML platforms include domain-specific capabilities that generic infrastructure cannot provide:
- Model training – distributed training across GPUs and TPUs with automatic resource scheduling, hyperparameter tuning, and checkpoint management
- Inference serving – low-latency model serving with autoscaling, batching, GPU multiplexing, and OpenAI-compatible API endpoints
- Experiment tracking – versioned tracking of parameters, metrics, artifacts, and code for reproducibility across training runs
- Model registry – centralized catalog of trained models with versioning, stage transitions (staging, production, archived), and lineage
- Feature stores – managed repositories of engineered features for consistent training and serving data pipelines
- MLOps pipelines – automated workflows for data processing, training, evaluation, deployment, and monitoring (CI/CD for ML)
- GPU management – scheduling, allocation, and monitoring of GPU resources across teams and workloads
- LLM serving – specialized inference engines optimized for large language models with KV-cache management, continuous batching, and quantization
- Vector databases – storage and retrieval of high-dimensional embeddings for RAG (retrieval- augmented generation) and semantic search
- Data labeling – annotation interfaces for images, text, audio, and video with team collaboration and quality control
About this data & disclaimer
The information on these pages was researched by a combination of human review and large language models. This catalogue serves as a reference for the resources, applications, devices, deployment payloads, and other components that the ResorsIT platform integrates, deploys, and manages. To suggest an addition or correction, please contact us. Prepared by Rhodium Systems Inc., author of the ResorsIT platform — a unified IT operations management platform for IT teams and MSPs that integrates a curated suite of open-source, commercial, and SaaS applications into a single system with shared identity, single sign-on, access control, and a common audit trail. This information is compiled on a best-effort basis: it may be incomplete, contain errors, and become out of date as tools, vendors, licences, and features change. Use this catalogue only as a starting point for your own research, and independently verify and review any tool carefully against your own requirements before relying on it.
Comparison
Feature Matrix – Open Source / Self-Hosted
| Feature | Ollama | vLLM | MLflow | Kubeflow |
|---|---|---|---|---|
| LLM serving | Yes | Yes | No | Yes* |
| Training | No | No | Tracking | Yes |
| Experiment tracking | No | No | Yes | Yes |
| Model registry | No | No | Yes | Yes |
| Pipeline orchestration | No | No | No | Yes |
| Data labeling | No | No | No | No |
| GPU management | Auto | Manual | N/A | K8s |
| Multi-model | Yes | LoRA | N/A | Yes |
| Prometheus metrics | No | Yes | No | Yes |
* = via KServe component
| Feature | Ray | Label Studio | W&B | LocalAI |
|---|---|---|---|---|
| LLM serving | Yes* | No | No | Yes |
| Training | Yes | No | Tracking | No |
| Experiment tracking | No | No | Yes | No |
| Model registry | No | No | Yes | No |
| Pipeline orchestration | Yes | No | No | No |
| Data labeling | No | Yes | No | No |
| GPU management | Yes | N/A | N/A | Auto |
| Multi-model | Yes | N/A | N/A | Yes |
| Prometheus metrics | Yes | No | No | Yes |
* = via Ray Serve (often with vLLM backend)
| Feature | LangChain | text-gen-webui |
|---|---|---|
| LLM serving | Via LangServe | Yes |
| Training | No | LoRA |
| Experiment tracking | No | No |
| Model registry | No | No |
| Pipeline orchestration | Chains | No |
| Data labeling | No | No |
| GPU management | N/A | Auto |
| Multi-model | Yes* | Yes |
| Prometheus metrics | No | No |
* = connects to multiple LLM providers/backends
Feature Matrix – Commercial / Cloud
| Feature | OpenAI | SageMaker | Vertex AI |
|---|---|---|---|
| LLM serving | Yes | Yes | Yes |
| Training | Fine-tune | Yes | Yes |
| Experiment tracking | No | Yes | Yes |
| Model registry | No | Yes | Yes |
| Pipeline orchestration | No | Yes | Yes |
| Data labeling | No | Ground Truth | Yes |
| Feature store | No | Yes | Yes |
| AutoML | No | Yes | Yes |
| Foundation models | GPT-4o, o3 | JumpStart | Gemini |
| Feature | Azure ML | Hugging Face | Replicate |
|---|---|---|---|
| LLM serving | Yes | Yes (TGI) | Yes |
| Training | Yes | Yes (libs) | Fine-tune |
| Experiment tracking | Yes | No | No |
| Model registry | Yes | Hub | Marketplace |
| Pipeline orchestration | Yes | No | No |
| Data labeling | No | No | No |
| Feature store | No | No | No |
| AutoML | Yes | AutoTrain | No |
| Foundation models | Azure OpenAI | Open models | Community |
Deployment Comparison
| Tool | Type | Self-Hosted | Cloud | GPU Required |
|---|---|---|---|---|
| Ollama | OSS | Yes | No | Optional |
| vLLM | OSS | Yes | No | Yes |
| MLflow | OSS | Yes | Managed* | No |
| Kubeflow | OSS | Yes (K8s) | Managed* | For workloads |
| Ray | OSS | Yes | Managed* | For workloads |
| Label Studio | OSS/Comm | Yes | SaaS | No |
| W&B | Commercial | Enterprise | SaaS | No |
| LocalAI | OSS | Yes | No | Optional |
| LangChain | OSS | Library | N/A | No |
| text-gen-webui | OSS | Yes | No | Recommended |
| OpenAI | Commercial | No | SaaS | N/A |
| AWS SageMaker | Commercial | No | AWS | For workloads |
| Vertex AI | Commercial | No | GCP | For workloads |
| Azure ML | Commercial | No** | Azure | For workloads |
| Hugging Face | Hybrid | Libraries | SaaS | For inference |
| Replicate | Commercial | No | SaaS | N/A |
* = managed versions available from cloud providers or vendors ** = Azure Arc for hybrid, but requires Azure connectivity
Licensing Comparison
| Tool | License | Type | Cost |
|---|---|---|---|
| Ollama | MIT | Open source | Free |
| vLLM | Apache 2.0 | Open source | Free |
| MLflow | Apache 2.0 | Open source | Free |
| Kubeflow | Apache 2.0 | Open source | Free |
| Ray | Apache 2.0 | Open source | Free |
| Label Studio | Apache 2.0 | OSS / Comm | Free / Enterprise |
| W&B | Proprietary* | Commercial | Free tier / $50+/user |
| LocalAI | MIT | Open source | Free |
| LangChain | MIT | Open source | Free |
| text-gen-webui | AGPL-3.0 | Open source | Free |
| OpenAI | Proprietary | Commercial | Pay-per-token |
| SageMaker | Proprietary | Commercial | Pay-per-use |
| Vertex AI | Proprietary | Commercial | Pay-per-use |
| Azure ML | Proprietary | Commercial | Pay-per-use |
| Hugging Face | Apache 2.0* | Hybrid | Free / Pro / Enterprise |
| Replicate | Proprietary* | Commercial | Pay-per-second |
* W&B client library is Apache 2.0; server is proprietary * Hugging Face libraries are Apache 2.0; Hub platform is proprietary * Replicate’s Cog framework is Apache 2.0; platform is proprietary
SSO / OIDC Comparison
OIDC is the preferred SSO protocol.
| Tool | OIDC | SAML | Authentik Notes |
|---|---|---|---|
| Ollama | No | No | Reverse proxy |
| vLLM | No | No | Reverse proxy |
| MLflow | No* | No | Reverse proxy |
| Kubeflow | Yes | Yes | Via Dex broker |
| Ray | No | No | Reverse proxy |
| Label Studio | Ent | Ent | Enterprise direct; OSS proxy |
| W&B | Yes | Ent | Team+ tiers |
| LocalAI | No | No | Reverse proxy |
| LangChain | N/A | N/A | FastAPI middleware |
| text-gen-webui | No | No | Reverse proxy |
| OpenAI | N/A | Ent | API key only |
| SageMaker | Yes | Yes | Via AWS IAM IdC |
| Vertex AI | Yes | Yes | Via GCP federation |
| Azure ML | Yes | Yes | Via Entra ID |
| Hugging Face | Ent | Ent | Enterprise Hub |
| Replicate | No | No | No SSO support |
* = Databricks managed MLflow supports SSO Ent = Enterprise/paid tier only
Best SSO support: Kubeflow (native OIDC via Dex), cloud platforms (AWS, GCP, Azure via identity federation), and W&B (Team+ tiers).
Self-hosted OSS tools: most self-hosted tools (Ollama, vLLM, MLflow, LocalAI, Ray) lack built-in SSO. Use a reverse proxy (Traefik, Caddy) with Authentik OIDC for access control.
LLM Serving Comparison
| Tool | Throughput | Latency | Ease of Use | Multi-GPU |
|---|---|---|---|---|
| Ollama | Low | Good | Easiest | No |
| vLLM | Highest | Best | Moderate | Yes (TP) |
| LocalAI | Low | Good | Moderate | Limited |
| text-gen-webui | Low | Good | Moderate | No |
| Hugging Face TGI | High | Good | Moderate | Yes (TP) |
| Ray Serve + vLLM | Highest | Best | Complex | Yes (TP) |
Best for production: vLLM (or Ray Serve + vLLM for autoscaling) Best for development: Ollama (simplicity) Best for multi-modal: LocalAI (text + image
- audio API)
Monitoring Integration
| Tool | Prometheus | Native Metrics | Monitoring Fit |
|---|---|---|---|
| Ollama | No | No | Process/GPU only |
| vLLM | Yes | Comprehensive | Excellent |
| MLflow | No | No | PostgreSQL metrics |
| Kubeflow | Yes | Via Istio | Excellent |
| Ray | Yes | Dashboard | Good |
| Label Studio | No | No | PostgreSQL metrics |
| W&B | No | Built-in* | Self-hosted only |
| LocalAI | Yes | Basic | Good |
| LangChain | No | No | App-level only |
| text-gen-webui | No | No | Process/GPU only |
| OpenAI | No | Usage API | Custom metrics |
| SageMaker | No | CloudWatch | Via CW plugin |
| Vertex AI | No | Cloud Monitoring | Via GCP plugin |
| Azure ML | No | Azure Monitor | Via Azure plugin |
| Hugging Face TGI | Yes | Comprehensive | Excellent |
| Replicate | No | Usage API | Custom metrics |
* = W&B tracks system metrics per training run internally, not via standard Prometheus
Best monitoring integration: vLLM, Kubeflow, Ray, and Hugging Face TGI expose Prometheus metrics that Telegraf can scrape directly.
Tools
16 tools.
AWS SageMaker
Amazon SageMaker is AWS’s fully managed ML platform covering the complete machine learning lifecycle: data preparation, model training, experiment tracking, model deployment, and monitoring.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: OIDC, SAML
ResorsIT usage: Web App with SSO
Access & Identity: Single sign-on · Directory / LDAP
Azure Machine Learning
Azure Machine Learning (Azure ML) is Microsoft’s cloud ML platform. It provides end-to-end ML lifecycle management including data preparation, automated ML, model training, deployment, and monitoring.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: OIDC, SAML
ResorsIT usage: Web App with SSO
Access & Identity: Single sign-on · Directory / LDAP
Google Vertex AI
Google Vertex AI is Google Cloud’s unified ML platform. It combines AutoML (no-code model training), custom model training, model serving, and the Gemini foundation model family into a single platform.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: OIDC, SAML
ResorsIT usage: Web App with SSO
Access & Identity: Single sign-on · Directory / LDAP
Hugging Face
Hugging Face is the central hub of the open-source ML ecosystem. Its model Hub hosts 500,000+ models and 100,000+ datasets, making it the “GitHub of machine learning.” The transformers Python library is the most widely used interface for…
License: Apache-2.0 (OSS) · Kind: web · Deploy: saas · SSO: OIDC, SAML
ResorsIT usage: Web App without SSO
Access & Identity: Single sign-on
Kubeflow
Kubeflow is a Kubernetes-native platform for developing, orchestrating, deploying, and running scalable and portable ML workloads.
License: Apache-2.0 (OSS) · Kind: web · Deploy: k8s · SSO: OIDC, SAML
ResorsIT usage: Web App with SSO
Access & Identity: Single sign-on · Directory / LDAP
Label Studio
Label Studio is the leading open-source data labeling and annotation platform. It supports labeling for images, audio, text, time series, video, and multi-modal data types.
License: Apache-2.0 (OSS) · Kind: web · Deploy: direct · SSO: OIDC, SAML
ResorsIT usage: Web App with SSO
Access & Identity: Single sign-on · Directory / LDAP
LangChain / LangServe
LangChain is the most widely used framework for building applications powered by large language models.
License: MIT (OSS) · Kind: web · Deploy: saas, docker, k8s · SSO: none
ResorsIT usage: Web App without SSO
Access & Identity: Forward-auth · Vaulted credential
LocalAI
LocalAI is a self-hosted, OpenAI-compatible API server that acts as a drop-in replacement for the OpenAI API.
License: MIT (OSS) · Kind: web · Deploy: direct · SSO: none
ResorsIT usage: Web App without SSO
Access & Identity: Forward-auth · Vaulted credential
MLflow
MLflow is the de facto standard open-source platform for managing the end-to-end machine learning lifecycle.
License: Apache-2.0 (OSS) · Kind: web · Deploy: direct · SSO: none
ResorsIT usage: Web App without SSO
Access & Identity: Forward-auth · Vaulted credential
Ollama
Ollama is a lightweight tool for running large language models locally. It wraps llama.cpp in a user-friendly CLI and REST API, making it trivial to download and run open-weight models like Llama, Mistral, Gemma, Phi, and Qwen on consumer h…
License: MIT (OSS) · Kind: web · Deploy: direct · SSO: none
ResorsIT usage: Web App without SSO
Access & Identity: Forward-auth · Vaulted credential
OpenAI API Platform
OpenAI’s API Platform is the dominant commercial LLM API service. It provides access to the GPT family of models (GPT-4o, GPT-4.1, GPT-4o mini, o1, o3), DALL-E for image generation, Whisper for speech-to-text, and TTS for text-to-speech thr…
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: SAML
ResorsIT usage: Web App with SSO
Access & Identity: Single sign-on
Ray / Ray Serve
Ray is a general-purpose distributed computing framework that has become a dominant platform for scaling ML workloads. Developed at UC Berkeley (RISELab), Ray provides a simple API for distributing Python code across clusters of machines.
License: Apache-2.0 (OSS) · Kind: web · Deploy: direct · SSO: none
ResorsIT usage: Web App without SSO
Access & Identity: Forward-auth · Vaulted credential
Replicate
Replicate is a cloud platform for running ML models via API. It provides a model marketplace with thousands of community-contributed models that can be called with a single API request.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
ResorsIT usage: Web App without SSO
Access & Identity: Vaulted credential
text-generation-webui (oobabooga)
text-generation-webui (commonly called “oobabooga” after its creator) is a Gradio-based web interface for running large language models locally.
License: AGPL-3.0-only (OSS) · Kind: web · Deploy: direct · SSO: none
ResorsIT usage: Web App without SSO
Access & Identity: Forward-auth · Vaulted credential
vLLM
vLLM is a high-performance inference engine for large language models. Developed at UC Berkeley, it introduced PagedAttention – a memory management technique that dramatically improves GPU memory utilization and throughput compared to naiv…
License: Apache-2.0 (OSS) · Kind: web · Deploy: direct · SSO: none
ResorsIT usage: Web App without SSO
Access & Identity: Forward-auth · Vaulted credential
Weights & Biases (W&B)
Weights & Biases (W&B) is a commercial ML experiment tracking and MLOps platform. It provides best-in-class experiment tracking, model registry, dataset versioning, and collaborative dashboards.
License: Proprietary (proprietary) · Kind: web · Deploy: direct, saas · SSO: OIDC, SAML
ResorsIT usage: Web App with SSO
Access & Identity: Single sign-on
