Commit Graph

43 Commits

Author SHA1 Message Date
Stephen Schuresko
3a0ec17c7b Treat node-local storage as temporary data only
StorageClasses that keep data on one node's disk (hostpath, local-path,
OpenEBS local PV, Longhorn with 1 replica and strict-local) are now
marked node-local:

- init never records one as platform storage, even with --allow-non-ha.
- doctor lists PVCs on node-local classes that running pods mount. One
  annotated zero-scale.io/data=temporary is a note; one whose workload
  runs on 2+ nodes (a database cluster with a local volume per member)
  is accepted; any other is a failure, and pins its Ingress backend.
- doctor warns when a node-local class is a default StorageClass and
  prints the annotate command that removes the default.
- App volumes take a :tmp mode: scratch space on the gateway pod's own
  disk instead of the shared RWX volume.

doctor also now reports Ingress backends whose Service does not exist,
host+path routed by several Ingresses to different Services, Knative
DomainMappings with no Ingress rule, and judges recorded platform
choices instead of asking for them again.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 07:24:02 -04:00
Claude
2161fac218 doctor: report what a single node failure takes down
List every Ingress backend whose running pods sit on one node (or have
none), Services whose externalIPs claim a node's own address (which
steals that node's ingress traffic, as kourier does with .134 today),
and more than one default StorageClass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016uKdmp7tLySXvQXALz2x9V
2026-10-27 07:17:21 -04:00
Claude
e489275c7f Fix the gateway outside 'default', single-replica PDBs and -n on status/logs
The gateway runs as a systemd service, so it never saw the pod's
POD_NAMESPACE or IDLE_TIMEOUT: a gateway in qa read app ConfigMaps from
default and started apps without volumes, database env or their idle
timeout. The unit now passes those variables on, and the gateway falls
back to PID 1's environment and the service account namespace. The
effective default idle timeout stays 10s.

A one-replica gateway gets maxUnavailable: 1 instead of a PDB that blocks
node drains. status, logs, list-apps and delete-app accept -n/--namespace
(logs keeps a numeric -n as the line count). --expand is documented as
writing YAML alongside a real apply.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016uKdmp7tLySXvQXALz2x9V
2026-10-27 07:08:56 -04:00
Claude
084c87064f Let an environment run its own Postgres cluster
install postgres --env <name> creates the CNPG cluster inside the
environment's namespace (without re-applying the namespace, which would
drop its labels) and records it in that env's zero-scale-env ConfigMap
instead of the platform ConfigMap. The default env quota now leaves room
for a two-instance cluster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016uKdmp7tLySXvQXALz2x9V
2026-10-27 07:08:56 -04:00
claude
e33abed59b Add QA and feature-branch environments
env create/delete/gc/list manage a namespace per environment, labelled
zero-scale.io/env, with a ResourceQuota, its own gateway and an optional
wildcard route. A zero-scale-env ConfigMap records the env and may override
the platform's database clusters. Hosts use ${ENV_SUFFIX}; databases in a
non-primary env are named db_<env>_<app>_<hash> and carry an owner comment,
so env delete drops only what the env owns. oauth-setup --auth-host serves
every app's login from one callback host. deploy/ manifests no longer
hardcode a namespace; deploy/kustomization.yaml sets it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016uKdmp7tLySXvQXALz2x9V
2026-10-27 06:44:55 -04:00
claude
030ba83733 Per-app persistent volumes on the shared HA RWX volume
Apps declare volumes: name:/path[:ro] (mounts: still accepted). The gateway
binds /srv/zero-scale-volumes/<app>/<name> into the unit, running it as a
fixed zs-<app> dynamic user so ownership is stable. gateway install creates
zero-scale-volumes-pvc; delete-app --purge --delete-volumes removes the data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016uKdmp7tLySXvQXALz2x9V
2026-10-27 06:44:55 -04:00
claude
fdfd615fd9 Add gateway install/status, ingress host routing and delete-app --purge
- gateway install/upgrade renders an HA gateway (2 replicas, RollingUpdate,
  PDB, hostname spread, downward-API namespace, RWX /opt/apps on the platform
  class); a Helm-managed gateway is recorded and left alone
- gateway status shows which apps run on which replica
- ingress add-host/remove-host/list: one Ingress per host or wildcard, TLS
  from a Secret or a cert-manager ClusterIssuer (wildcards require DNS-01),
  upstream-hash-by $host pinning; hosts stored in the platform ConfigMap
- apply skips the per-app Ingress when routed hosts already cover the app
  in the gateway's namespace and no OAuth is used; per-app Ingresses also pin
  by host
- status, logs and binary updates reach every gateway replica
- delete-app --purge [--drop-database]; Postgres drops only databases whose
  owner comment matches; apply records database_engine for later cleanup
2026-10-27 06:44:55 -04:00
claude
88638b60e5 Add install longhorn and install postgres
- Plans are printed first and applied only with --yes; Helm when available,
  otherwise the pinned release manifests (Longhorn 1.9.1, CNPG 1.26.0 /
  chart 0.24.0, all overridable)
- Replica counts follow schedulable nodes (max 3) with an HA warning below 3
- Longhorn: longhorn-ha (RWO) and longhorn-ha-rwx StorageClasses with Retain,
  optional default-class flag, optional backup target with daily snapshot and
  weekly backup RecurringJobs; existing installs only get the classes
- Postgres: CNPG operator (skipped when present) and a Cluster with pod
  anti-affinity, superuser access for provisioning, and a Ready wait
- Both record their result in the zero-scale-platform ConfigMap
2026-10-27 06:44:55 -04:00
claude
49f4a11bcc Add doctor and init; provision databases from the platform ConfigMap
- internal/platform: detect and grade storage classes (Longhorn, Ceph, Mayastor,
  LINSTOR, Portworx, GKE/Azure/AWS, NFS, local), Postgres operators (CNPG,
  Zalando, Crunchy, Percona, StackGres), MySQL operators (InnoDB Cluster, PXC,
  MariaDB, legacy shared-mysql), ingress classes, gateways and Redis
- Choose picks HA candidates (default class wins, ambiguity is an error with
  the flag to resolve it) and stores them as flat zero-scale-platform keys
- zero-scale-ctl doctor (read-only) and init (diff, confirm or --yes)
- DB provisioning goes through per-operator adapters; SQL travels on stdin,
  the MySQL admin password is read from a Secret and sent on stdin, and
  identifiers and passwords are validated
- Postgres endpoint now names the namespace the cluster is in (was the app's)
- Redis service, ingress class and config-file DB hosts come from the platform
- Legacy fallback with a warning when no platform ConfigMap exists
2026-10-27 06:44:55 -04:00
claude
cd30ee48ba Share app naming between CLI and gateway; map every hostname to one app
- internal/appname: one naming rule (first label, lowercased, [a-z0-9] only)
- internal/registry: host -> app cache built from zero-scale-app ConfigMaps
  (hosts key), with direct and legacy-name fallbacks and a 10s TTL
- gateway: units, binaries and config keyed by the app's canonical host, so
  secondary hostnames share one process and one config
- reaper reads app config from the registry instead of the API every tick
- zero-scale-ctl writes a hosts key and uses the shared naming rule
2026-10-27 06:44:55 -04:00
Stephen Schuresko
f16ba7ff49 Add turnkey deployment manifests, RBAC, and binary pre-flight validation 2026-10-21 07:23:45 -04:00
Stephen Schuresko
1f7d6b7da2 docs: document shared OAuth2/OIDC prerequisites and add shared-redis pre-check warning 2026-10-20 18:33:31 -04:00
Stephen Schuresko
82307ce625 feat: make deployment cluster-neutral, add operator detection with docs links, and clean up hardcoded domains 2026-10-20 18:29:52 -04:00
Stephen Schuresko
1387848297 feat: add shared OAuth2 setup for ingress-nginx
- Introduced GatewayConfig struct to hold OAuth2 configuration in AppConfig.
- Implemented OAuthSetupName method to resolve shared OAuth2 profile name.
- Enhanced LoadConfig function to parse gateway configuration from the config file.
- Added oauth-setup command to create or update shared ingress-nginx OAuth2 setup profiles.
- Implemented ReconcileOAuthSetup function to provision/update OAuth2 credentials and resources.
- Created OAuth2 proxy deployment and service for shared OAuth2 setup.
- Added Ingress configuration for OAuth2 callback routing.
- Updated DeployBinary function to check and upload binaries based on SHA256 hash.
- Enhanced secret management in ProvisionDataLayer to support cross-service references.
- Updated ingress creation to support OAuth2 authentication.
2026-10-19 13:48:09 -04:00
Stephen Schuresko
0c5b1eba04 feat: Add support for managed secrets in application configuration and provisioning 2026-10-12 16:04:18 -04:00
Stephen Schuresko
9a3835b925 feat: Add idle timeout configuration and enhance scaling logic for applications 2026-10-12 12:44:49 -04:00
Stephen Schuresko
303196f6ea feat: Add support for auxiliary config files and enhance deployment process with variable expansion 2026-10-12 10:28:55 -04:00
Stephen Schuresko
c57ed49839 feat: Implement auto-generation of MySQL root password and update shared-mysql-admin secret example 2026-10-11 23:55:09 -04:00
Stephen Schuresko
7422ea2521 feat: Refactor collectZeroScaleApps to use a dynamic map for application discovery and enhance parseIngressSites for improved categorization 2026-10-11 23:49:27 -04:00
Stephen Schuresko
4cc8733d1b feat: Add Zero-Scale Platform Management CLI with database provisioning and Kubernetes integration
- Implemented `database.go` for MySQL and PostgreSQL provisioning, including password generation and secret management.
- Created `k8s.go` for Kubernetes command execution, resource checks, and secret/configmap management.
- Developed `main.go` to handle CLI commands for applying configurations, checking status, logging, and managing applications.
- Added `storage.go` for binary deployment to the gateway pod and symlink configuration for application entrypoints.
- Enhanced CLI with commands for application management, including listing, deleting, and uploading binaries.
2026-10-11 23:31:54 -04:00
Stephen Schuresko
7659c711ac feat: enhance zero-scale dashboard with resource display and update functionality
- Added RAM and CPU resource display in the zero-scale dashboard for active applications.
- Implemented dynamic updates for resource display based on application state.

feat: expand zero-scale control script with new commands and options

- Introduced `apply`, `upload-binary`, and `provision-db` commands to the zero-scale control script.
- Added options for specifying configuration files, binary uploads, and database provisioning parameters.

feat: implement temporal JWKS aggregator service

- Created a new service to aggregate JWKS from Gitea and Kubernetes API.
- Implemented caching and refresh logic for JWKS keys.

feat: deploy shared MySQL and Redis services

- Added Kubernetes manifests for shared MySQL and Redis services, including StatefulSet and ConfigMap definitions.
- Configured persistent storage and resource limits for MySQL and Redis deployments.

feat: add temporal OAuth secret example and values configuration

- Created example secret manifest for OAuth credentials required by Temporal.
- Added a values file for configuring Temporal server with JWKS integration and resource limits.

feat: implement Kubernetes token provider for Temporal client

- Developed a K8sTokenHeadersProvider to supply Kubernetes service account tokens to Temporal.
- Added tests for token retrieval and environment variable fallback.
2026-10-11 17:43:21 -04:00
Stephen Schuresko
a8b4bedfbd Remove Applications & Services, Storage & Data, and Media & Social sections from web and api 2026-09-25 09:33:24 -04:00
Stephen Schuresko
8e177b0092 Modularize zero-scale dashboard into go and static subfolders with pure-Go asset minifier 2026-09-25 09:07:21 -04:00
Stephen Schuresko
b27b40bf5a fix: remove initial loading placeholders from grid containers to prevent them occupying card slots 2026-09-24 11:40:56 -04:00
Stephen Schuresko
e66345f8d2 feat: implement glitch-free asynchronous background updates fetching from https://kstat.fairfaxmedia.net/api/status 2026-09-24 11:34:53 -04:00
Stephen Schuresko
4f69be5a96 feat: display total zero-scale footprint for all nodes and clarify CloudNativePG net savings vs actual consumption 2026-09-24 11:28:24 -04:00
Stephen Schuresko
c26aac5e0b feat: measure and display zero-scale RAM separately per node alongside top 24/7 memory consumers 2026-09-24 11:23:02 -04:00
Stephen Schuresko
b6601f5ac9 feat(dashboard): showcase architecture efficiency and memory savings from zero-scale and shared operators 2026-09-24 11:10:01 -04:00
Stephen Schuresko
14f550c21b feat(dashboard): add zero-scale explanation banner and trigger status refresh on Open App & tab switch 2026-09-24 10:59:27 -04:00
Stephen Schuresko
d44144e66e feat(reaper): enforce 10s scale-to-zero across all running app units with self-healing recovery 2026-09-24 10:57:17 -04:00
Stephen Schuresko
429abfcd06 fix(dashboard): accurately detect active transient systemd services via /run/systemd/transient 2026-09-24 10:42:58 -04:00
Stephen Schuresko
0de82c19c4 feat(dashboard): show Longhorn and CloudNativePG operators with replica counts 2026-09-24 10:29:00 -04:00
Stephen Schuresko
1118b8e39e Center hostname on its own line in zero-scale service cards 2026-09-24 10:09:22 -04:00
Stephen Schuresko
b59091630c Do not mask container engine in unauthenticated telemetry 2026-09-24 09:49:46 -04:00
Stephen Schuresko
4c4abb3c3d Show OS and runtime on row 1, kernel and kubelet on row 2 of node cards 2026-09-24 09:47:37 -04:00
Stephen Schuresko
d5d35db8da Implement ingress rate-limiting, in-memory caching, and RBAC-protected kernel telemetry 2026-09-24 09:45:38 -04:00
Stephen Schuresko
3c1ef8bc7d Auto-port-forward registry on push and enhance dashboard noscript fallback 2026-09-24 09:31:26 -04:00
Stephen Schuresko
6202ed7628 Add automated build-and-deploy Makefile target and update dashboard metadata 2026-09-24 08:17:53 -04:00
Stephen Schuresko
cc29f00e1e Add 2nd ingress and zero-scale host mapping for kstat.fairfaxmedia.net 2026-09-24 08:09:13 -04:00
Stephen Schuresko
3c15747fe5 Rename dashboard to Zero-Scale Platform Dashboard in cmd/zero-scale-dashboard 2026-09-24 07:45:48 -04:00
Stephen Schuresko
1321e65be1 Add kubernetes-status dashboard in cmd/status-dashboard with nodes, cores, RAM, uptime, and cluster site directory 2026-09-24 05:15:14 -04:00
Stephen Schuresko
5b4416f7b8 Refactor zero-scale gateway into modular Go microservice packages 2026-09-16 10:13:43 -04:00
Stephen Schuresko
e5e0dc2b7c Organize zero-scale-platform project with standard layout, go.mod, Makefile, deployment manifests, CLI, and docs 2026-09-13 21:23:51 -04:00