Troubleshooting
Use the public address for visitor problems and the CMS address for editing or
login problems. Commands below use Website example in websites-example and
the default operator Deployment in viking-cms; replace those values with your
own. Run kubectl config current-context first to confirm the cluster. Do not
paste Secret values, cookies, Bearer tokens or raw import credentials into logs
or issue reports.
Gather evidence before changing anything
kubectl get website example -n websites-example -o yaml
kubectl describe website example -n websites-example
kubectl get deploy,pod,svc,ingress,pvc -n websites-example
kubectl get events -n websites-example --sort-by=.lastTimestamp
kubectl logs -n viking-cms deployment/viking-cms-controller --since=15m
kubectl logs -n websites-example deployment/example-cms -c cms --since=15m
kubectl logs -n websites-example deployment/example-frontend --since=15m
If no CMS/frontend Deployment exists yet, its log command will fail; inspect the
Website conditions and controller first. For a restarting container, select its
pod with kubectl get pods and use kubectl logs POD -n NAMESPACE -c CONTAINER --previous.
Include the site identifier, affected URL/path, time, build version, status/release
number and steps to reproduce when reporting an issue.
Interpret Website conditions
observedGeneration should catch up with metadata.generation after a spec
change. Inspect condition type, status, reason and message rather than relying
on the top-level Ready column alone. Conditions are observations, not an audit
of every public request.
| Condition/reason or symptom | Check | Next step |
|---|---|---|
MissingSecret / InvalidSecret |
Auth reference and key names in the Website namespace | Correct the Secret; its data fingerprint triggers CMS recreation |
IpseityUnavailable |
Provider Deployment labels, selected namespace/Ingress and HTTPS /oidc route | Resolve ambiguity/availability using the Ipseity guide |
IpseityProvisioningFailed |
Retained client ownership, key material, namespace realm and home administrator | Review the exact condition message; do not overwrite membership grants blindly |
HostnameConflict / invalid hostname |
Effective aliases and all competing Website/Ingress hosts | Give CMS/public routing distinct hostnames; fix duplicate aliases |
StorageReady=False / PVC Pending |
kubectl describe pvc and storage provisioner events |
Select a real RWX-capable class, quota/capacity and permissions |
FrontendReady=False, no release |
Fresh site with no /www/current |
Sign into CMS, create /, save and publish before waiting for frontend readiness |
Published=False / NoRelease after publication |
CMS status, current pointer and status-updater logs | Check status Role/token/API connectivity; status synchronization is asynchronous |
| Deployment unavailable | Pod scheduling, image pulls, init containers and probes | Inspect kubectl describe pod and the specific failing container |
See Website status for the field definitions and condition meanings.
CMS is unavailable or will not sign in
A missing/invalid auth Secret causes the operator to block CMS routing and scale its Deployment to zero. A syntactically valid Secret can still fail CMS startup validation or OIDC discovery; the protected listener then returns 503 and admin readiness remains 503. Check the CMS logs without exposing credential contents.
For sign-in loops or callback failures, check all of the following:
- Browser URL, Website CMS hostname, optional
cmsHostnamepin and the IdP's registered callback are the same external host, with/oauth2/callback. - External CMS traffic uses HTTPS with a valid certificate. Secure cookies will
not work over plain HTTP;
ingress.tls: falseneeds another TLS terminator. - CMS pods can resolve and reach IdP discovery, JWKS and token endpoints, and trust their certificates. Scratch images have no curl/shell; use cluster debugging tools or a separate diagnostic pod under your platform procedure.
- Provider issuer/audience/scopes and role claims match the Secret. 401 means invalid/missing identity; 403 usually means roles, CSRF or origin, not a reason to disable authentication.
- Exactly one CMS pod handles the site. Login state and CSRF sessions are in-memory; multiple writers or a restart during login can invalidate flows.
Changing auth Secret data triggers a Recreate rollout. If local configuration
files change outside the operator, restart the CMS to reload them. Changing the
session key invalidates old sessions. Pending logins and large cached reference
sessions also need a new login after process restart.
If an editor session expires, Carry on working permits local changes while writes stay blocked. Log in again checkpoints the tab's draft and signs in in the same window. Recover with the same account; never reload simply to dismiss an expiry dialog containing unsaved work. Storage/checkpoint failure should be resolved while the tab remains open.
Storage permission or init-container failure
Inspect kubectl logs POD -n websites-example -c volume-permissions and pod events.
The init container recursively chowns both mounts to UID/GID 65532 using root
plus CHOWN capability. Storage exports must permit that operation. A driver
ignoring fsGroup or refusing chown/root access needs storage-side work; setting
the app container to root is not the documented solution. Check free space,
quota, read-only mounts and file/directory sync support when saving or publishing
fails. Do not delete PVCs to reset permissions; they contain the site's source.
Public site missing, stale or returning errors
A new site without an active release is deliberately unready. A published site
without a page at / can be ready but return 404 for its homepage. A hidden menu
page is still accessible; deleting a draft only removes it publicly after the
next successful publication. URL changes do not create redirects.
Check generated Ingress hosts/TLS/backends, DNS resolution, Service endpoints, frontend pods and the active release. Forward directly to a frontend pod to separate application behaviour from ingress/cache behaviour:
kubectl port-forward -n websites-example deployment/example-frontend 18080:8080 18081:8081
In another terminal:
curl -i http://127.0.0.1:18081/readyz
curl -i http://127.0.0.1:18080/
curl -i -X POST http://127.0.0.1:18080/
curl -I https://example.com/
The GET should serve a published homepage; POST should return 405. Both should
identify Viking Frontend <version> in Server. The application version is
not the content release ID. Stop port-forward with Ctrl+C when finished.
If the external response identifies a proxy/library, compare it with this direct response before changing application code. Configure the reverse proxy/ingress to preserve the upstream Server header. If only external content is stale, review browser/proxy/CDN caching; the default frontend root-resolution interval is one second. Check asset URLs and ensure the asset existed when that release was published. Multipart byte ranges return 501; the frontend supports single ranges only. Non-GET/HEAD methods on the public site are rejected.
Publish, rollback and import problems
Check the saved page URLs and reserved namespaces, selected theme/layouts, installed renderer packages, broken raw-object references and free storage. The publisher renders every saved page, so a bad page elsewhere can fail a publication. Renderer execution is limited; repair content or the package when it traps/times out. A late activation/storage-history failure can occur after the current pointer moved: read CMS status and public content before retrying.
No Roll back button means no usable historical release may be retained.
retainReleases: 0 removes history after a 30-second grace period, and a release
can expire while confirmation is open. Rollback does not restore draft source.
Use backup/restore for source recovery.
A WordPress import can complete partially; inspect created/skipped/failed paths. Retry skips existing destinations. Public-image copying refuses private/local networks and has stricter limits than manual media upload. Use WordPress limits to split work.
New Website fields disappeared
An old installed CRD can prune unknown fields before the controller sees them.
Inspect the stored Website with kubectl get website ... -o yaml, not just the
local input. Update the installed CRD, including the separate Orbit copy where
used, then reapply original manifests to restore values already pruned. An image
upgrade alone cannot recover lost fields. See upgrades.
