Compute Environment Setup

Prepare user-owned installation and removal

Produce a bounded, copyable installation plan for the user or host administrator. When the host is configured for Slurm, explain whether the plan must be run in an interactive allocation or submitted with provider-approved #SBATCH directives. Do not run the bootstrap through callCommand or submitJob: package installation, image pulls, caches, and activation files outlive the Open-Science process and have no application-owned receipt or uninstall lifecycle.

Name every path the plan will create, its expected storage/network impact, and a matching idempotent removal command. Preserve shared modules, package caches, base Conda installations, and images unless the user explicitly identifies them as exclusively theirs. Never use recursive deletion on a path derived only from an environment name; give the user the exact canonical path to verify first.

The user-run plan should create the environment before installing its activation file. It should write the activation file atomically: create a temporary file, set mode 600, and rename it to <name>.sh only after the environment succeeds. A conda activation file can be as small as:

source "$HOME/miniforge3/etc/profile.d/conda.sh" || return $?
conda activate protein-gpu || return $?
export HF_HOME="${SCRATCH:-$HOME/.cache}/huggingface"
export OMP_NUM_THREADS="${SLURM_CPUS_PER_TASK:-1}"

Guard every required setup command with || return $? so a missing module, activation failure, or invalid export stops before the workload. Open-Science also treats any non-zero result from sourcing the activation file as a job failure. Do not append repeatedly or put activation in .bashrc; the named file makes job behavior deterministic without changing the user's interactive shell.

End the plan with an explicit removal procedure for the exact activation file and exclusively user-owned environment prefix. Removal must be safe to repeat and must not scan for similarly named resources. If ownership or sharing is unclear, remove only the activation file after the user verifies its contents and leave the environment/cache/image for the administrator.

Validate where jobs run

Validate the exact activation file, the imports or executables the task needs, and a small output witness. An import alone is insufficient for compiled or GPU software.

For direct SSH, run the witness with callCommand:

const witness = await compute.callCommand(
  '. "$HOME/.open-science/environments/protein-gpu.sh" && python -c "import sys; print(sys.executable)"',
  'Validate protein-gpu environment',
  { loginShell: true, timeoutSeconds: 120 }
)

For Slurm, run the witness through the same job path users will use. Put the provider-known #SBATCH directives first, select the new logical environment, and request a small text output:

const job = await compute.submitJob(
  'Validate protein-gpu on one Slurm node',
  '#SBATCH --partition=<provider-known-partition>\n#SBATCH --time=00:05:00\npython -c "import sys; print(sys.executable)" > environment-witness.txt',
  {
    environment: 'protein-gpu',
    outputs: ['environment-witness.txt'],
    timeoutSeconds: 600
  }
)
await new Promise((resolve) => setTimeout(resolve, 2000))
return compute.attachJob(job.job_id).result()

The immediate result read is a single non-blocking failure check. End the cell afterward; Open Science polls and harvests the job in the background and starts the analysis turn when it finishes. Do not poll.

When validation fails, diagnose the layer identified by the error: environment definition, activation, shared filesystem visibility, scheduler request, binary compatibility, or cache population. Prepare revised user-run commands; do not mutate the durable environment, add a readiness flag, or bypass the named activation file.

Record reusable host facts

After a successful witness, append a concise host-scoped note with the environment name, activation path, environment system, shared paths, scheduler requirements, and validation date. Keep the project definition in the project and reference its path rather than copying it into the note.

Record only facts established by host documentation or an explicit check. A successful witness proves that the environment was visible on that allocation; it does not by itself prove that home directories or software paths are shared across every compute node. Describe filesystem scope as unknown or limited to the observed allocation unless stronger evidence establishes it. Likewise, do not infer sudo, package-manager, network, quota, or administrator permissions from a missing tool or one failed install command. Separate known host facts, this witness's observations, and assumptions that still need confirmation in both the project reproduction notes and host knowledge.

await host.compute.details(providerId, {
  mode: 'append',
  text: '\n### Environment: protein-gpu\nActivation: ~/.open-science/environments/protein-gpu.sh\nDefinition: environment.yml in the project\nValidated: <date>, direct or Slurm witness succeeded\n'
})

If the requested environment already exists and the exact witness passes, leave it unchanged and record only genuinely new host knowledge.