Environments

A computational environment describes the necessary conditions for code to run properly. Ensuring that every stage in your pipeline is run within a defined environment is a great way to improve reproducibility. Typically these will be created inside the global system environment, where the system environment has an environment manager installed. Because the system environment itself is somewhat of a "primary artifact" (cf. reproducibility), i.e., it is manually manipulated by the user, we want to minimize its uniqueness or changes to it, since there's a risk we may not properly document its state. This means we want to limit foundational dependencies to very common tools like Git, common shells like Bash, and environment managers.

There are many different environment management tools out there to choose from, and Calkit attempts to provide a similar interface for all of them. Calkit also attempts to enforce their usage in such a way that all important information about the environment is captured locally in the project in so-called "lock files." This way, the project can be moved to other machines without needing to worry about manually installing packages.

Calkit provides a means for defining or declaring environments inside a project's calkit.yaml file. There is also a command line utility calkit xenv for executing a command in one of these, which ensures that the environment matches its specification before execution.

Environment types and definitions

Calkit supports the following environment types:

• Docker
• Conda
• venv (included in the Python standard library)
• uv (both venv and project-based)
• Pixi
• renv
• Julia
• MATLAB
• Nix (flake-based)
• SLURM
• system (the machine itself, local or reached over SSH)

Environment definitions live in the project's calkit.yaml file in the environments section. Most environments will have a path property pointing to a file that lists the necessary dependencies--the "spec." For example, a Python virtual environment or "venv" can be defined as a simple list of dependencies in a requirements.txt file, which might look like:

pandas>=2
polars==0.17.1
matplotlib

Automatic detection

For common environment types, Calkit will register a new environment upon its first use with calkit xenv. For example, if you run:

calkit xenv python scripts/run.py

Calkit will attempt to find an environment spec, create the environment, save it in calkit.yaml, export a lock file, and run the command in that environment. If there are multiple env specs, e.g., requirements.txt and environment.yml, you can provide the path, e.g.,

calkit xenv -p environment.yml python scripts/run.py

and Calkit will use that one to create the environment and save the path in calkit.yaml.

Checking, syncing, and executing

An environment can be checked that it matches its specification with:

calkit check env --name {env-name}

This will produce a "lock file" (inside the project's .calkit/env-locks directory if the environment manager doesn't export lock files by default), which uniquely identifies the actual environment that was created to help diagnose reproducibility issues down the road.

A command can be executed in an environment with:

calkit xenv --name {env-name} -- {command}

Before the command is executed, Calkit will check that the environment matches its specification, and if it needs to be updated, that will be done before execution.

All project environments can be checked at once with:

calkit check envs

Inspecting environment paths

To see where an environment's specification and lock file live, use:

calkit describe env --name {env-name}

Adding --json prints the same information as machine-readable JSON, which is handy for other tools that need to locate these files:

calkit describe env --name {env-name} --json

The output is a single line, shown formatted here for readability (pipe it through something like jq to format it yourself):

{
  "kind": "uv-venv",
  "spec_path": "requirements.txt",
  "lock_path": ".calkit/env-locks/my-env/linux-64.txt",
  "prefix": null,
  "python": "3.13"
}

All keys are always present, so a null value means the field doesn't apply to that kind of environment. To describe every environment at once, keyed by name, use calkit describe envs [--json].

Choosing an environment type

So which type of environment should you use? The short answer is: any. Any environment is better than none, where none means installing dependencies in the global host machine environment. If you want the long answer, keep reading.

Docker is probably the most reproducible out of any environment type, since a Docker image includes information about the operating system. If it's convenient, e.g., if an image already contains all the necessary dependencies, go with a Docker environment. However, in some cases Docker may be a bit heavier than necessary.

If you're running Python code, a uv-venv environment is a good default choice. uv is very easy to install and very fast.

If you have non-Python dependencies that depend on complex compiled binaries (as scientific and engineering oriented tooling often does) and a uv-venv can't be built on your machine, A Conda environment is a good choice. However, Pixi has access to the same packages and is a bit faster. It's sort of like uv for Conda packages, and is similarly very easy to install.

If you're working on a machine for which you don't have control to install dependencies, or working as part of a team, a plain old Python venv could be the best option.

Again, try not to get too hung up on the decision of which environment type to use. Try one and see how it goes. Calkit should make the experience similar for all types.

Examples

Creating any type of environment from the Calkit CLI follows a similar pattern starting with calkit new. You can view the help output with calkit new --help and filter it down to environment-related commands with calkit new --help | grep env.

Docker

A new Docker environment can be added to the project with calkit new docker-env. A Docker environment can use an existing image, e.g., from Docker Hub, or it can create a new image, e.g., from a Dockerfile stored in the project repo.

Let's say you want to add an OpenFOAM environment to your project. This can be achieved with something like:

calkit new docker-env --image microfluidica/openfoam:2412 --name foam

Then you can run a command in that environment with:

calkit xenv -n foam -- icoFoam -help

You can similarly jump into an interactive bash terminal with:

calkit xenv -n foam bash

Some Docker images are CLI tools with an image entrypoint already defined (for example minlag/mermaid-cli). In that case, use command_mode: entrypoint so Calkit passes your command arguments directly to the container entrypoint instead of running shell -c ....

# In calkit.yaml
environments:
  mermaid:
    kind: docker
    image: minlag/mermaid-cli
    wdir: /data
    command_mode: entrypoint

Then execute with:

calkit xenv -n mermaid -- \
  -i figures/my-mermaid-diagram.mmd \
  -o figures/my-mermaid-diagram.pdf

But what if there isn't an image out there that has everything you need already installed into it? In this case, you can define and build a new derived image in the project by using the --from parameter, optionally adding predefined "layers" to the image with --add-layer. This will produce a Dockerfile defining the image, and when that environment is run with calkit xenv, that image will be built and a lock file produced.

For example, running:

calkit new docker-env \
    --from microfluidica/openfoam:2412 \
    --name foam2 \
    --add-layer miniforge

will create a Dockerfile in the project and add the environment named foam2 to the calkit.yaml file. Calling calkit xenv -n foam2 bash will cause the image to be built and a lock file Dockerfile-lock.json to be created. Note that the Dockerfile path can be controlled with the --path option.

You can go in and modify the Dockerfile, e.g., to add more installation commands, and another call to calkit xenv -n foam2 will kick off a rebuild automatically, since the lock file will no longer match the Dockerfile.

If you're copying local files into the Docker image, you can declare these dependencies in the environment definition so the content of those will be tracked as well:

# In calkit.yaml
environments:
  foam2:
    kind: docker
    image: foam2
    deps:
      - src/mySolver.C

This highlights Calkit's declarative design philosophy. Simply declare the environment and use it in a pipeline stage and Calkit will ensure it is built and up to date. There is no need to think about building images as a separate step.

Caching images in a registry

An image built from a Dockerfile can take a long time to build, and once it's gone from the local Docker image store, rebuilding it is the only way to get it back. Worse, a rebuild isn't guaranteed to produce the same image, since the packages the Dockerfile installs move on over time.

Setting registry on the environment lets calkit push publish the image, so that Calkit can pull it back by digest whenever it's missing rather than rebuilding it:

# In calkit.yaml
environments:
  foam2:
    kind: docker
    path: Dockerfile
    image: foam2
    registry: ghcr.io

image can be left out when there's a path to build from, in which case the image is named after the project and the environment, e.g., someone/some-project.foam2, matching how the environment's Jupyter kernel is named. An environment defined purely by someone else's image has to name it.

That name comes from owner and name in calkit.yaml, or from the Git remote if they're not set. A project with neither says nothing about what it's called, and Calkit asks for image, or for owner and name, rather than naming the image after the directory the project happens to sit in, which would rename the image whenever the directory moved.

ghcr.io on its own resolves to the project's namespace in the GitHub Container Registry, e.g., ghcr.io/someone/some-project, naming the registry rather than leaving it to be worked out, but any registry prefix works, and leaving registry unset or null keeps images local. The lock file records the image's digest, so what gets pulled back is exactly what was built, and Calkit checks the image's layers against the lock after pulling.

calkit push sends the images of every environment with a registry set, alongside Git and DVC, skipping any the registry already has. An environment built before its registry was configured is pushed as-is, without being rebuilt first. A project with no image to send never reaches a registry at all: an environment with no registry, or one whose image was never built on this machine, is settled locally rather than with a round-trip and a request for credentials. Writing digests into lock files is a check's job, not a push's, so a push leaves the lock files alone. Pass --no-docker to skip images, or name what to send: calkit push docker publishes the images and nothing else, which is handy mid-work when the code isn't ready to go out with them. Checking an environment builds or pulls whatever the project needs to run, and leaves publishing to calkit push. The digest goes into the lock file as soon as the image is built, since a manifest is content-addressed: the digest an image is built with is the one it has once it's pushed, so pushing it leaves the lock alone.

To rebuild or repull an image and write fresh lock files, e.g., after an image's tag has been moved out from under the digest in the lock, run:

calkit update docker-env --name foam2 --lock

Locking multiple platforms

A Docker environment is locked per architecture, in .calkit/env-locks/{name}. Calkit reads the image's manifest from the registry and writes a lock file for each platform it provides, not just the one it's running on, so moving a project between an arm64 laptop and an amd64 server doesn't invalidate every stage in the environment.

Images from a registry are multi-platform already. An image built from a Dockerfile is built for one platform by default; to build and lock more than one, declare them:

# In calkit.yaml
environments:
  foam2:
    kind: docker
    path: Dockerfile
    image: foam2
    registry: ghcr.io
    build_platforms:
      - linux/amd64
      - linux/arm64

Building for multiple platforms requires a registry, since a multi-platform image can only be kept in one.

How images are fetched

The lock file for a Docker environment records the exact image its stages ran in: its layers, and the digests it can be pulled back by. Whenever an environment is checked, e.g., as part of calkit run, Calkit works from that record rather than assuming a rebuild will do, since building the same Dockerfile again produces a different image once the packages it installs have moved on.

If the image named in calkit.yaml is already in the local Docker image store and its layers match the lock, there's nothing to do. Otherwise Calkit tries, in order:

1. Pull the digest in the lock, from the environment's registry if it has one, or from wherever the image came from originally. This is the usual case for an image deleted by a docker system prune, or for a collaborator who's never built it. The layers of whatever comes back are checked against the lock, so a tag that's been moved out from under the digest can't quietly substitute a different image.
2. Fetch it from a project release. Releases record the images they archived (see Archiving Docker images), so if no registry can serve the image, Calkit looks through the project's releases for one whose layers match the lock, downloads it from Zenodo, loads it into Docker, and checks its layers again. This is what keeps an old version of a project reproducible after a registry has dropped the image, or after the account that published it is gone.
3. Build it, for an environment with a Dockerfile, or pull it by tag for one named after an existing image. Only at this point does the environment get an image that isn't the one the lock describes, and the lock is rewritten to record what was actually built.

The effect is that deleting an image locally costs a download rather than a rebuild, and doesn't invalidate any stage that used it, since the lock file doesn't change.

A downloaded release image is cached under .calkit/local/container-images so it isn't fetched twice. Fetching from a release needs the release to have been published, since that's where the file is downloaded from.

uv

uv can create both project and venv virtual environments. Project environments are defined by a pyproject.toml file, while venv environments are defined by a requirements.txt file.

To create a new uv project environment, inside a project directory run something like:

calkit new uv-env -n my-env "polars>=1.0" matplotlib

By default, this will create a pyproject.toml file in .calkit/envs/my-env/pyproject.toml, but the path can be controlled with the --path option.

To create a new uv venv, simply replace uv-env with uv-venv in the above command and a requirements.txt file will be created instead.

calkit xenv -n my-env python -c "import matplotlib, print(matplotlib.__version__)"

If you were to run something like:

calkit xenv -n my-env python -c "import pandas, print(pandas.__version__)"

it would fail, since pandas is not present in the spec file (pyproject.toml or requirements.txt). However, if you add it in there, calling the above command again will succeed because Calkit automatically checks or syncs the environment before execution.

venv

A venv environment, which uses Python's built-in venv module, can be used nearly identically to the uv example above. Simply replace uv-venv with venv in the calkit new call.

Conda

As you might expect, Conda environments again work nearly identically to uv-venv and venv environments.

You can create a new Conda environment with something like:

calkit new conda-env -n my-conda-env numpy matplotlib --pip pandas

Note that in this case, we specified one package, pandas, to be installed from the Python Package Index (PyPI) with pip using the --pip option.

The new Conda environment spec will be written to environment.yml by default, which can be controlled with the --path option.

A prefix for the environment can be specified to keep all packages under the project directory, e.g., by adding --prefix .conda-envs/my-conda-env. If this option is omitted, the environment will become part of Conda's system-wide collection of environments with a name like {project_name}-{env_name}, where the project name is added to avoid conflicts.

Similar to other environment types, any time a command is executed with calkit xenv, this environment will be checked and created or updated as necessary.

Calling:

calkit xenv -n my-conda-env -- which python

will create it. If you add any dependencies to environment.yml, calling that same command will cause the environment to be rebuilt before execution, and an updated environment-lock.yml file will be created. Again this highlights Calkit's declarative design philosophy. Declare the environment and what command should be executed inside, and Calkit will handle the rest.

Julia

Julia environments have paths that point to a Project.toml file. Creating a new Julia environment is similar to creating a Python environment:

calkit new julia-env \
    --name my-julia-env \
    --path ./envs/my-julia-env/Project.toml \
    --julia 1.11 \
    WaterLily \
    Makie

With Julia environments, it's possible to execute a command:

calkit xenv -n my-julia-env -- -e "println(\"hello world\");"

or a script:

calkit xenv -n my-julia-env -- my_julia_script.jl arg1 arg2

Running Julia this was will ensure the global environment is ignored, meaning you can be sure if it's successful on your machine, it will be successful on others.

SLURM

SLURM is a job scheduler commonly used for high performance computing (HPC). A SLURM environment can be defined in calkit.yaml as follows:

environments:
  my-hpc-cluster:
    kind: slurm
    host: hpc.myinstitute.org

See the HPC guide for how to use SLURM (and PBS) environments in pipeline stages.

System

A system environment is the machine as it is, with nothing built, installed, or isolated by Calkit. It's an escape hatch for software Calkit doesn't manage, e.g., a site-wide module system or a hand-built toolchain.

The simplest form is the machine you're on:

environments:
  local:
    kind: system
    lock:
      - os
      - python-version

Nothing is pinned by default, since opting out of isolation is the whole point of this kind. The lock property is how a project says which properties of the machine its results actually depend on. Locked properties are written to the environment's lock file, which stages depend on, so moving to a machine where one of them differs reruns the stage rather than silently reusing a cached result.

The properties that can be locked are:

Run calkit describe system to see what these are on the machine you're on.

The built-in _system environment is shorthand for this kind on localhost with nothing locked.

Requirements

A system environment can also declare requirements---what has to be true of that machine before stages run on it:

environments:
  cluster:
    kind: system
    host: hpc.example.edu
    requirements:
      - kind: cpu-count
        min: 16
      - julia>=1.10
    lock:
      - cpu-count
      - julia-version

requirements and lock answer different questions, which is why a property can appear in both. A requirement is a precondition: it's checked before anything runs, and one that isn't met stops the run and says what was found and what was needed. A lock is a cache input: nothing is checked, but the property's observed value is recorded, and a stage reruns when it changes. So the example above means "refuse to run on fewer than 16 CPUs," and separately "rerun everything if the number of CPUs isn't what it was last time."

If you don't care what a property is but do care when it changes, lock it and leave it out of requirements. A requirement that constrains nothing is rejected, since it asserts nothing.

Requirements are checked on the machine the environment names. For a host that isn't this one, that means an app is looked for on that host's PATH, a variable is read from the shell a login gets there, and a setup requirement's check_command runs there. Nothing is offered as a fix in that case---installing something on another machine belongs to whoever administers it---so Calkit reports what was missing and where. Checking a requirement about the machine itself, like cpu-count, needs Calkit installed on that host, since that's what reports its properties.

The project's own top-level requirements describe the host you're driving from, which is the _system environment. They're checked on every calkit run, wherever stages end up running, because that machine still has to be able to drive the pipeline.

Running on another machine

A system environment's host names the machine the work belongs on. SSH is how a machine is reached, not a kind of environment in its own right, so there's no separate ssh kind: if host names the machine you're on, the stage runs right there, and otherwise Calkit connects over ssh and copies files with scp. This is useful, e.g., for offloading work to a cluster login node or a cloud VM with a more powerful GPU.

It is assumed that dependencies on the other machine are managed separately, unless you pair it with an inner environment (see below).

environments:
  cluster:
    kind: system
    host: "10.225.22.25"

The host is the only thing you have to declare. Everything else has a sensible default:

• user is left to SSH, which resolves it from ~/.ssh/config or falls back to your current account. Repeating it here would only be a second place for it to be wrong.
• wdir---the project's workspace on that machine, a clone of the project and the directory stages run in---defaults to ~/.calkit/workspaces/<hub>/<owner>/<name>. A relative path is taken from the connecting user's home directory. It's qualified by hub and owner because a host is shared: two projects named example-ssh from different owners are different projects. It's hidden because Calkit checks the workspace out with --force, so it must not look like somewhere you'd keep your own work.
• ssh_key is left to SSH and its agent. If a particular host needs a particular key, that belongs in ~/.ssh/config, which already answers "which key for which host" and is where people look for it. The field is for cases where that isn't available, such as CI dropping a key at a known path.

So the fuller form, if you do need to be explicit, is:

environments:
  cluster:
    kind: system
    host: "10.225.22.25"
    user: my-user-name
    wdir: /home/my-user-name/calkit/example
    ssh_key: ~/.ssh/id_ed25519

Getting set up

Check that the host is actually reachable before running anything:

calkit check env -n cluster

calkit run does the same check for every environment in the pipeline, so you don't have to remember to. Either way it connects without allowing a password prompt, so an unauthorized key is reported now rather than hanging halfway through a pipeline.

In a terminal, that check walks you through whatever is missing, asking before each step:

• Any environment variable the definition refers to, such as a host written as ${CK_SSH_HOST}, is prompted for and saved to .env, so it's only asked once. .env is added to .gitignore if it isn't already.
• If you have no SSH key, it offers to create one (ed25519, no passphrase, so stages can run unattended).
• If this machine isn't authorized on the host yet, it offers to run ssh-copy-id, then re-checks rather than assuming it worked.
• If Calkit isn't installed on the host, it offers to install it there. That's needed to activate an inner environment or to read the machine's properties for a lock.

Nothing happens without you agreeing to it, since creating a key and authorizing a machine both change things outside the project.

Without a terminal---in CI, say---none of this is attempted, because there's nobody to answer. It fails instead with the exact commands to run, e.g.:

ssh-copy-id -i ~/.ssh/id_ed25519 my-user-name@10.225.22.25
ssh my-user-name@10.225.22.25 'curl -LsSf install.calkit.org | sh'

To execute a command in this environment, we can add a stage like this to our pipeline in calkit.yaml:

pipeline:
  stages:
    run-simulation:
      kind: shell-script
      environment: cluster
      script_path: script.sh
      outputs:
        - results

How the workspace is kept in sync

Notice that nothing above says which files to copy back and forth, and nothing in the compiled pipeline does either. Calkit works it out, because a list of paths written down anywhere is a list that can fall behind the pipeline---at which point the stage quietly runs against stale inputs, which is the failure you'd least want here.

Before the command runs, Calkit captures your working tree---including edits you haven't committed---as a Git snapshot, pushes it straight to the workspace, and checks it out there detached. No branch is created on either side, so several people (or several clones) can share one workspace without their branch names colliding, and cleaning up afterwards is a single reserved namespace rather than a set of names someone has to recognize. Data that DVC tracks is ignored by Git, so it can't ride along in the snapshot. It travels through the workspace's own DVC cache instead, which Calkit addresses as a DVC remote: only the objects the workspace is missing cross the wire, deduplicated by content.

Afterwards, the workspace is asked what the run produced---anything it reports as changed or newly appeared since the snapshot it was given---and that is what comes back. dvc.lock is deliberately never carried back: your DVC writes its own from what it hashes locally.

The workspace is reused between runs, which is what keeps environments, the DVC cache, and the Git history warm---a fresh one would rebuild all three every time. Because it's a single checkout at a single commit, a run holds a lock on it, and a second run that wants the same workspace is told who has it rather than checking out over them. The lock sits beside the workspace, not inside it, and is released once a run's outputs have been collected; if the run is interrupted while the remote job is still going, the lock stays, because the workspace really is still busy.

One consequence worth knowing: if the project changes locally while a stage is running elsewhere, Calkit refuses to collect the results rather than recording them. DVC hashes a stage's dependencies from your local files once the command returns, so recording a result in that situation would write a dvc.lock pairing inputs that were never used with outputs they never produced---and unlike a stale stage, which simply reruns, a lock file like that goes on looking up to date indefinitely.

lock works the same way here as it does locally, except that the properties recorded are the host's---what a stage's results depend on is the machine it actually ran on. Calkit reads them from that machine when the environment is checked, which means Calkit has to be installed there. It already is if you pair the environment with a runtime (see below), since that's what activates the inner environment on the far end.

Pairing with a runtime

Because a system environment says where a stage runs rather than what it runs in, it can wrap another environment the same way a SLURM environment can, using the composite <outer>:<inner> syntax:

pipeline:
  stages:
    simulate:
      kind: python-script
      environment: cluster:py
      script_path: simulate.py

Calkit dispatches to cluster first, then activates the py environment once there, so the workspace on that machine needs both Calkit and the project.

MATLAB

Adding a MATLAB environment to a project will cause Calkit to automatically generate a Docker image based on its version and products attributes. A MATLAB_LICENSE_SERVER environmental variable must be set so the container can properly contact a license server. This can be done with:

calkit set-env-var MATLAB_LICENSE_SERVER <XXXX@some.server.edu>

Note that environmental variables set this way will be ignored by Git, and so will need to be set on each new machine on which the project is to be run.

A MATLAB environment (and a pipeline stage that uses it) looks like:

# In calkit.yaml
environments:
  my-matlab-2024b:
    kind: matlab
    version: R2024b
    products:
      - Simulink
      - Global_Optimization_Toolbox
      - Parallel_Computing_Toolbox

pipeline:
  stages:
    my-matlab-script:
      kind: matlab-script
      script_path: scripts/run_sim.m
      environment: my-matlab-2024b
      inputs:
        - config/my-sim-config.json
      outputs:
        - results/sim-results.h5

Pixi

Pixi environments typically have the path pixi.toml:

environments:
  my-pixi:
    kind: pixi
    path: pixi.toml

Nix

Calkit supports Nix environments via flakes. Reproducibility comes from flake.lock, which pins every input (including nixpkgs) to an exact revision. Calkit tracks flake.lock as a DVC dependency, so pipeline stages re-run when the environment changes.

Create one with:

calkit new nix-env --name my-nix-env python3 R uv

In calkit.yaml:

environments:
  my-nix-env:
    kind: nix
    path: flake.nix

Projects can contain multiple Nix envs. The first one lands at the repo root (flake.nix); subsequent ones get nested under .calkit/envs/{name}/flake.nix so each env has its own independent flake.lock. You can override the path with --path if you want a different layout.

To enter a specific dev shell from the flake (instead of the default), set shell:

environments:
  my-nix-env:
    kind: nix
    path: envs/my-nix-env/flake.nix
    shell: r-shell

Run a command in a Nix environment:

calkit xenv -n my-nix-env -- python --version

Add more packages to an existing Nix env (this edits the flake's packages = with pkgs; [ ... ] list, refreshes flake.lock, and commits):

calkit update env --name my-nix-env --add-package R --add-package polars

On Linux and macOS, install Nix with calkit install nix — this runs the Determinate Systems installer, which enables flakes by default. Nix is not supported natively on Windows; run Calkit inside WSL2 and install Nix there.

Calkit itself is also available as a flake — see Nix in the installation guide to add the Calkit CLI to your own Nix environments via inputs.calkit.url = "github:calkit/calkit".

R

R environments can be managed with Conda, Pixi, or renv (recommended). The env spec path for an renv environment is typically a DESCRIPTION file.

environments:
  r:
    kind: renv
    path: DESCRIPTION

The imports inside DESCRIPTION are used to create and sync the environment:

Package: CalkitProject
Version: 0.0.1
Title: Auto-generated R environment
Imports: tidyverse

System environments

A system environment runs things on the machine as it is, with nothing built, installed, or isolated. It's the escape hatch for software Calkit doesn't manage, e.g., a site-wide module system or a hand-built toolchain.

Nothing about the machine is pinned by default, since opting out of isolation is the point of this kind. lock is how a project says which properties its results actually depend on:

environments:
  cluster:
    kind: system
    host: gpu-node-1.example.edu
    lock:
      - os
      - julia-version

Locked properties are written to the environment's lock file, which stages depend on, so running on a machine where one of them differs invalidates the cached result instead of silently reusing it. Locking a property the machine can't supply, e.g., a tool that isn't installed, is an error rather than a recorded null---a stage that claims to be pinned to something it isn't is worse than one that pins nothing.

Environment kind reference

Environment definitions belong in the environments section of calkit.yaml.

conda

Model class: CondaEnvironment

docker

Model class: DockerEnvironment

julia

Model class: JuliaEnvironment

matlab

Model class: MatlabEnvironment

nix

Model class: NixEnvironment

pbs

Model class: PBSEnvironment

pixi

Model class: PixiEnvironment

renv

Model class: REnvironment

slurm

Model class: SlurmEnvironment

system

Model class: SystemEnvironment

The machine as it is, with nothing built, installed, or isolated.

An escape hatch for software Calkit doesn't manage, e.g., a site-wide module system or a hand-built toolchain. Nothing is pinned by default, since opting out of isolation is the whole point of this kind, so lock is how a project says which properties of the machine its results actually depend on.

Locked properties are written to the environment's lock file, which stages depend on, so moving to a machine where one of them differs invalidates the cached result rather than silently reusing it.

requirements is the other half, and answers a different question. It says what must be true of this machine -- apps that must be installed, variables that must be set, at least this many CPUs -- and is checked before anything runs, on the machine the environment names. A requirement that fails stops the run and says how to fix it; a locked property that changes silently invalidates a cached result. One gates, the other pins, so a property that matters both ways is written in both places.

host names the machine. SSH is how a machine is reached, not a kind of environment, so there is no separate ssh kind: a system env whose host isn't this machine is reached over SSH, and one whose host is this machine runs here, the same way a SLURM env does. The built-in _system environment is shorthand for this kind on localhost with nothing locked.

machine_id says which machine, where host only says what it answers to. Names are renamed, resolve differently from different networks, and are reused; a project that means one particular machine can name it here instead and have that survive all of it. It replaces the name in deciding whether this is that machine, and is checked again on the far end when it isn't -- so a host that has come to point at a different box is reported rather than run on. host is still what reaches it, so both are worth declaring for a machine that isn't this one. Run calkit describe system on a machine to read its ID.

Declaring one says where to run, which is a separate question from whether results depend on the machine: moving a project to a new one and updating this need not invalidate everything computed on the old one. Whether it does is left to lock, where machine-id is available for projects whose results really are machine-specific.

wdir is the project's workspace on that host -- the directory the stage runs in. It defaults to ~/.calkit/workspaces/<hub>/<owner>/<name>, so a project that just names a host lands somewhere predictable rather than having to spell out a path that is the same on every machine anyway. Qualified by hub and owner because a host is shared, and hidden because transfers check out with --force: a path that looks like the user's own checkout is one whose edits would be silently destroyed.

What moves in and out of that workspace is deliberately not declared here. An environment doesn't know which files a stage reads, so a list kept alongside it can fall behind the pipeline and quietly run against stale inputs; the paths are taken from the stage instead.

uv

Model class: UvEnvironment

uv-venv

Model class: UvVenvEnvironment

venv

Model class: VenvEnvironment