push-to-remote Guide

Overview

push-to-remote is the other half of the post-simulation handoff, alongside update-from-remote. Run it on the simulation server (e.g. an HPC login node) right after a batch of analyses finishes there, to send the results to the machine that actually generates and deploys the Hugo site (the “analysis server”, e.g. bb-server1).

It exists because the two machines are not symmetrically reachable: the simulation server typically has outgoing network access only, and the analysis server has no route back to it at all. Any transfer must be initiated from the simulation server, pushing out — never initiated from the analysis server trying to pull in. push-to-remote wraps manage-analyses sync ... --put, which is exactly that: an rsync, run from here, pushing local files out.

Basic usage

# Dry run first (shows what would transfer, moves nothing):
push-to-remote kb@bb-server1:/data/OceanICU/oceanicu_3d/analyses/ ./analyses

# Actually push:
push-to-remote kb@bb-server1:/data/OceanICU/oceanicu_3d/analyses/ ./analyses --apply

# With a specific SSH key:
push-to-remote kb@bb-server1:/data/OceanICU/oceanicu_3d/analyses/ ./analyses --apply --ssh ~/.ssh/id_rsa

REMOTE is the destination path on the analysis server. LOCAL_ANALYSES is this machine’s own local analyses directory (the one your analysis scripts just wrote into).

Only files matching the canonical analyses/ layout are transferred — plots/tables under areas/*/validations/*/..., the BC scenarios tree, simulation_list.db, statistics.db, staging/**. Old-layout files at the wrong directory depth are excluded automatically (see manage-analyses).

This step only moves files into place. It does not generate any web pages and does not touch the live site.

The full handoff

On the simulation server (e.g. scylla, after a run finishes):
  push-to-remote kb@bb-server1:/data/OceanICU/oceanicu_3d/analyses/ ./analyses --apply

On the analysis server (e.g. bb-server1) -- --no-sync because the push
already moved the files; a plain sync here would try to pull FROM the
simulation server, which the analysis server usually can't reach:
  update-from-remote kb@simserver:/path/to/its/analyses/ /data/OceanICU/oceanicu_3d/analyses \
      --no-sync --apply --deploy

update-from-remote’s own REMOTE argument is still required (no defaults, by design — see its guide) even with --no-sync, but is unused in that mode; any placeholder value works.

Alternative: regenerate + deploy directly from the simulation server

You don’t have to log into the analysis server separately to finish the job. The calling project’s own regenerate_hugo.py and this repo’s deploy_ghpages.py both auto-relay over SSH to the real analysis/relay host (see each project’s own regen_hosts.yaml / deploy_hosts.yaml) — whichever machine they’re run from — with no entry needed for the calling machine itself in either YAML file. Relaying only forwards the command; it never needs to look up the caller’s own host config, so a new machine (the simulation server) needs no new config to do this.

So right after the push above, from the same simulation-server session:

cd ~/source/repos/OceanICU/oceanicu_3d   # the CODE repo checkout (not analyses)
./regenerate_hugo.py --apply             # relays to bb-server1, regenerates
cd ~/source/repos/ocean-post
./deploy_ghpages.py --apply              # relays to bb-server1, builds + pushes gh-pages

Both commands actually execute on the analysis server (that’s what the relay does) using its own real paths — this is equivalent to update-from-remote --no-sync --apply --deploy, just two separate calls instead of one, and without needing update-from-remote’s own merge/scan steps. Pick whichever’s more convenient; they end up running the identical underlying code either way.

Prerequisite, not yet verified from this side: this needs a checkout of the oceanicu_3d code repo (not just an analyses tree) on the simulation server, since regenerate_hugo.py has to run locally first before it can relay. If that checkout doesn’t exist yet there, add one (git clone git@github.com:BoldingBruggeman/oceanicu_3d.git) — no other setup needed; regen_hosts.yaml/deploy_hosts.yaml need no new entry for this machine.

Two bugs fixed 2026-10-07

update-from-remote --deploy used to silently do nothing (the underlying deploy_ghpages.py call omitted --apply, so it just printed its own help and exited 0). And --hugo-dir’s default pointed at the Hugo site directory under $HOME, which ocean-reporting --hugo then also used as the content-output root — wrong on bb-server1, where real generated content/static live under /data/OceanICU/oceanicu_3d instead (to avoid filling up $HOME; see regen_hosts.yaml’s own comment). Both fixed in commit 7cecf42; update-from-remote now does the right thing with no extra flags needed, exactly as shown above.