How to Run the Same OpenStack Tests We Used to Validate Private Cloud Core v4

When we released Private Cloud Core v4, we ran OpenStack’s own integration test suite, Tempest, against it. Every test that ran passed, with 19 tests excluded and the expected skips for services we don’t enable. That’s a useful number, but you shouldn’t have to take our word for it. The run guide, the configuration, and the full exclusion list are public on GitHub, so you can run the same tests against your own cloud and check the result yourself.

Here’s the problem this solves. You’re evaluating a hosted OpenStack platform, or you’ve just had one deployed. The provider says it works. Your team needs to know that compute, block storage, networking, images, identity, and object storage all behave the way the OpenStack APIs say they should, before you start moving production workloads onto it.

Clicking around the dashboard won’t tell you that. Launching a few test VMs won’t either. What you want is a broad, repeatable check against the public APIs, run by your own team, with results you can keep.

That’s what Tempest is for. Below is how it works, how to run our configuration, and how to read what comes back.

What Tempest Actually Tests

Tempest is the integration test suite maintained by the OpenStack project. The Tempest documentation describes it as a battery of API validation tests, scenario tests, and other checks meant for validating a working OpenStack deployment.

A few design choices make it a good fit for checking a provider’s cloud:

• It only uses public APIs. Tempest tests talk to OpenStack through the same interfaces your applications and automation will use, not through private or implementation-specific hooks.
• It runs against any OpenStack cloud. The same suite is meant to work on a single-node test install or a large production cloud.
• It covers real scenarios, not just endpoints. Beyond checking that an API call returns the right response, scenario tests do things like boot a server, attach networking, and connect to it.

Tempest also drives real load. It creates projects, users, servers, volumes, networks, and images, then cleans up after itself. That’s worth keeping in mind for where and when you run it, which we’ll cover below.

What We Ran Against Private Cloud Core v4

Private Cloud Core v4 runs OpenStack 2026.1 on Ubuntu 24.04, with Ceph v19 providing block and object storage. We ran Tempest against that stack before release. Every test included in the run passed. We excluded 19 tests, and Tempest skipped tests for features that aren’t enabled in our configuration, which is normal for any deployment that doesn’t run every OpenStack service.

The full procedure is in our Tempest run guide on GitHub. It includes the exact tempest.conf settings and the exclusion file with a written reason for every excluded test. If you’d rather not run anything, the exclusion list alone is worth reading, because it tells you how the platform is configured in a level of detail most provider documentation doesn’t.

Running Tempest Against Your Own Cloud

The run guide walks through every step. Here’s the shape of it, with the decisions that matter called out.

Step 1: Check the Prerequisites

You’ll need:

• Python 3.10 or newer on the machine running the tests
• Admin credentials for the target cloud
• A small cloud image, such as Cirros, uploaded to Glance
• At least two flavors available in Nova
• An external network that provides floating IPs

On OpenMetal, you have full root and admin access to your Private Cloud Core, so the admin credentials are yours to use.

Before you run anything from a machine outside the cloud, add that machine’s network as a trusted network in OpenMetal Central. In v4, the host firewall denies inbound traffic on each node’s external interface by default, with only rate-limited SSH open, and trusted networks you add in Central are how you open broader access. [VERIFY BEFORE PUBLISHING: confirm with Jamie whether OpenStack API endpoints and instance floating IPs on a v4 cloud are reachable from outside before a trusted network is added, and update this paragraph to match.]

Step 2: Install Tempest and the OpenStack Client

The guide installs Tempest into a Python virtual environment, then installs python-openstackclient in the same environment. You create a clouds.yaml file with your cloud’s Keystone URL and admin credentials, and confirm it works by requesting a token:

openstack --os-cloud my-cloud token issue

From there, you list images, flavors, and external networks to collect the IDs Tempest needs, and upload a Cirros image if you don’t have one.

Step 3: Create a Workspace and Configure It

tempest init creates a workspace with a skeleton tempest.conf. You then fill it in with your cloud’s values. A few settings in our configuration are worth understanding rather than copying blindly:

• [service_available] tells Tempest which services to test. Our configuration tests Cinder, Glance, Neutron, Nova, and Swift (the object storage API), and leaves Horizon out of the run.
• [validation] is set to connect to test servers over floating IPs with key-based SSH. This is what lets scenario tests confirm a server is actually reachable, not just reported as active.
• [volume] storage_protocol = ceph reflects that block storage on Private Cloud Core is backed by Ceph.
• [scenario] target_dir = /var/tmp changes a default that only works with Cirros. The guide explains why: the default /tmp location doesn’t persist across a reboot on other images.
• use_dynamic_credentials = true lets Tempest create its own test projects and users, so tests don’t collide with each other or with your real projects.

Once the file is filled in, tempest verify-config probes the live API and reconciles the extension and feature flags in your configuration with what the cloud actually exposes.

Step 4: Snapshot the Cloud’s State

Before running tests, the guide has you record the cloud’s current state:

tempest cleanup --config-file ~/.config/tempest/my-cloud/etc/tempest.conf \
  --init-saved-state

This writes a saved_state.json file. Later, the cleanup tool uses it to remove only what Tempest created and leave your existing resources alone. Don’t skip this step.

Step 5: Run the Tests

Start with a smoke run as a quick confidence check:

tempest run --workspace my-cloud --concurrency 2 --smoke

Then run the full suite with the exclusion list:

tempest run --workspace my-cloud --concurrency 2 \
  --exclude-list ~/.config/tempest/my-cloud/exclude.txt

Concurrency controls how many tests run in parallel. Each worker needs its own credentials, and with dynamic credentials the real limit is usually how much capacity your cloud has free for test servers and volumes.

Step 6: Read the Results

Results are stored in the workspace’s .stestr directory. stestr last --subunit | subunit-stats gives you a summary. stestr last gives you the detailed report. You can save the raw results with stestr last --subunit > results.subunit and keep them alongside your acceptance records.

Step 7: Clean Up

tempest cleanup removes what Tempest created, using the saved state from Step 4. Run it with --dry-run first and review the dry_run.json file it produces before deleting anything.

What the Exclusion List Tells You

Nineteen excluded tests can sound like 19 problems. They aren’t, and the exclusion file explains each one. They fall into a few groups.

Object Storage Runs on Ceph RGW, Not OpenStack Swift

Private Cloud Core provides the Swift object storage API through Ceph’s RADOS Gateway (RGW). Several Swift tests check behavior that RGW handles differently:

• RGW returns a 404 rather than Swift’s 401 for unauthenticated writes and deletes.
• RGW doesn’t enforce Swift’s arbitrary caps on container metadata count and length.
• Anonymous downloads and static website hosting are RGW options we don’t enable by default. Static web hosting requires knowing the serving domains at configuration time.

If you rely on any of these Swift-specific behaviors, this is exactly the kind of thing you want to find before migration, not after.

Image Handling Reflects Deliberate Choices

• Image sharing is admin-only. Tests that share images as a normal project user are excluded because our policy intentionally forbids it.
• There’s one writable image store. Glance advertises a read-only HTTP store alongside the Ceph-backed RBD store, so Tempest thinks multi-store tests apply. Only RBD accepts image data, so multi-store import and copy tests don’t apply.

Block Storage Runs Active/Active

Cinder’s volume service runs active/active across the cluster. One retype test asserts that a volume’s host attribute stays the same after a retype with no migration. With active/active, a different cluster member can handle the retype, so the host attribute can change even though no data moved. The assertion doesn’t hold for this design.

Test Image Limitations

One networking scenario test checks that subnet DNS updates reach a guest by asking the guest’s DHCP client to renew its lease. The Cirros image doesn’t run its DHCP client in the way the test expects, so the test can’t trigger the renewal. The guide notes that Neutron does push the DNS update correctly.

One Group Waits on an Upstream Fix

The guide is also direct about one group that isn’t a configuration choice. Three tests that create an image from a paused, stopped, or suspended server are excluded because of an upstream Glance issue in multi-store, high-availability setups. The guide labels it a real bug pending an upstream fix, documents a workaround, and doesn’t pretend otherwise.

That candor is the point of publishing the list. You can see exactly what was excluded, why, and whether any of it affects your workloads.

Where and When to Run It

Tempest is a validation tool, so run it when the answer matters:

• During a proof of concept, before you commit to a platform. Run it in your first week and keep the results.
• After initial deployment, before production workloads move in, as part of your acceptance checklist.
• After significant changes, such as adding capacity or changing configuration, to confirm nothing regressed.

Because Tempest creates real resources and generates real load, avoid running it during peak production hours. Leave enough free capacity for its test servers and volumes, take the state snapshot first, and clean up afterward.

What Else to Check in a New Private Cloud Core v4

Tempest covers the OpenStack APIs. A few other things are worth confirming in your first session with a new v4 cloud:

• Your trusted networks. Add the networks your team and automation will connect from in OpenMetal Central.
• Your dashboard. Horizon is the default dashboard on v4. Skyline, the newer OpenStack dashboard, is available as an opt-in from the Cloud Settings page in Central.
• Your networking backend. New v4 clouds use OVN for Neutron by default. If you need full-featured load balancers with TLS termination or Layer 7 routing, that’s a decision to make before deployment.
• Your expansion path. In v4, the same Ansible playbook suite handles both the initial deployment and adding nodes later, so growing the cloud follows the same automated process that built it.

Who This Is For

A good fit if you:

• Are evaluating hosted OpenStack providers and want evidence you can generate yourself
• Need an acceptance test before moving production workloads onto a new cloud
• Run regulated or customer-facing workloads where you have to document that the platform was checked
• Are migrating from VMware or public cloud and need to confirm API behavior your tooling depends on
• Want a regression check you can repeat after changes

Probably not the right fit if you:

• Need application-level testing; Tempest checks OpenStack, not your software
• Want performance benchmarks; Tempest validates behavior, not throughput or latency
• Can’t spare capacity for test servers and volumes during the run

The Short Version

A provider’s test results are a claim. Running the same tests on your own cloud turns that claim into evidence, and a published exclusion list with a reason for every line tells you more about a platform than any feature page.

If you want to run Tempest against a Private Cloud Core v4 before you commit, apply for a proof of concept. You can also size a cloud with our private cloud pricing calculator or talk with our team about your evaluation plan.

Frequently Asked Questions

What is OpenStack Tempest?

Tempest is the official integration test suite for OpenStack. It validates a deployment through the public OpenStack APIs, with API tests, scenario tests that boot and connect to real servers, and other checks. It’s designed to run against any OpenStack cloud.

Did Private Cloud Core v4 pass Tempest?

We ran Tempest against OpenStack 2026.1 on Private Cloud Core v4. Every test that ran passed, with 19 tests excluded and the usual skips for features not enabled in our configuration. The run guide and the reason for each exclusion are public on GitHub.

Can I run Tempest on my own OpenMetal private cloud?

Yes. You have admin access to your Private Cloud Core, and our public run guide includes the configuration and exclusion list we used. Add the network you’ll test from as a trusted network in OpenMetal Central first.

Why were 19 tests excluded?

Most exclusions reflect configuration choices: object storage runs on Ceph RGW instead of OpenStack Swift, image sharing is admin-only, there’s a single writable image store, and Cinder runs active/active. One test is limited by the Cirros test image, and one group of three is excluded pending an upstream Glance fix. The exclusion file documents each one.

Will Tempest affect my existing workloads?

Tempest creates and deletes real resources and generates load, so run it outside peak hours with enough free capacity. Take a state snapshot with tempest cleanup --init-saved-state before running, so the cleanup step removes only what Tempest created.

Does Tempest measure performance?

No. Tempest checks that OpenStack behaves correctly through its APIs. For throughput or latency, use dedicated benchmarking tools against your actual workloads.