api

import "github.com/umbralcalc/stochadex/pkg/api"

Package api is the configuration tier: it turns one YAML document into a running simulation. Everything the engine can do — the forward model, how its partitions are wired, how many seeded members to run, where observations come from, where results go, and the inference, aggregation or optimisation layered on top — is expressible here as data, so a whole run becomes a single artifact that can be versioned, diffed and executed by a prebuilt binary with no Go toolchain present.

How a component is named

Every position in a config that holds a framework component is a data spec — a mapping selecting a registered name, e.g. iteration: {type: wiener_process} or timestep_function: {type: constant, stepsize: 1.0}, resolved at load time by this package’s registries. There is no Go-expression spelling: the whole document is data, so LoadApiRunConfigFromYaml resolves it and RunWithParsedArgs runs it in-process with no Go toolchain. A component given as a scalar Go string is rejected at load.

A partition’s bespoke maths is data too: an expressions: entry (ExpressionConfig, inlining general.ExpressionIteration) states the per-step update as expressions. The registries are for the framework’s own catalogue; the expressions DSL is for a model’s arithmetic. Genuinely novel algorithmic iterations that are neither in the catalogue nor expressible in the DSL belong in a downstream repo that embeds the engine as a Go library (Settings + Implementations), not in a config.

The config surface

main:     partitions and the simulation block (output condition and function,
          termination condition, timestep function) — see RunConfig.
embedded: named sub-runs, each a whole RunConfig (EmbeddedRunConfig). A main-run
          partition whose name matches one is replaced by an embedded simulation
          iteration wired to it, which is how a simulation nests inside a partition.
run:      execution mode — batch or ensemble, with seeds and concurrency (RunModeConfig).
data:     a StateTimeStorage, produced either by a sub-simulation or by a pre-recorded
          source (DataSource: csv, json_log, postgres, plus registered ones).
macros:   each entry expands one pkg/macros constructor into a set of partitions over
          that storage, or runs live with no data: block at all.

Where the registries live

registry.go          data-only iterations — a name maps to a constructed type, params
                     carry the rest.
registry_compose.go  composable iterations, whose interface-typed fields (kernel,
                     likelihood, jump distribution, prior, nested iteration, named
                     function) are themselves specs, resolved recursively. The
                     "expression" builder here makes the whole expressions DSL usable
                     as an inline iteration spec, so maths can appear anywhere an
                     iteration is expected — inside a macro's window, or an embedded run.
macros*.go           the macro tier: decodes typed spec structs straight from YAML and
                     calls the matching pkg/macros constructor. One file per family
                     (aggregation, inference, smc, optimisation, stats, data, mcts).
registry_environment.go
                     agents.Environment implementations for the mcts_self_play macro.
                     Unlike the registries above this one is empty by default and filled
                     by downstream modules calling RegisterEnvironment: decision rules
                     are not part of the framework catalogue and have no data spelling,
                     so the engine passes an env: spec through to its registered builder
                     without interpreting it. The tictactoe fixture is the exception the
                     engine ships, so the path stays covered end-to-end in CI.

Staying honest, and staying lean

Two drift tests guard the iteration registry (registry_test.go and registry_coverage_test.go): every registered name must construct the type it claims, and a go/ast scan requires every Iterate-implementing type in the candidate packages to be either registered or listed in excludedIterations with a reason. A newly-added iteration therefore fails CI until it is classified, which is what stops the registry silently lagging the framework.

Imports drive go.mod, so components with heavy dependencies are not named here directly. RegisterDataSource (and simulator.RegisterComponent for sinks) lets a package layered above this one contribute a source or output spelling without the engine depending on it; the Arrow, S3 and DuckDB spellings are registered this way by cmd/stochadex. An unknown key reports the spellings the running binary actually has.

Pre\-flight

CheckForDeadlock runs before any batch or ensemble simulation. Within-step wiring (params_from_upstream) that forms a dependency cycle would otherwise surface as an opaque runtime “all goroutines are asleep” with no indication of which partitions are at fault; the check names them and says how to break the cycle. It runs no simulation. See pkg/graph.

Scope

Inference as forward simulation — a posterior stepped as a partition — is in scope, which is why posterior_estimation and the other inference macros live here. Inference against real data is the data: resource, which a downstream repo supplies. The decision layer stays in Go on purpose: an agents.Environment is arbitrary game rules, not representable as data.

Index

Variables

BuildVersion and BuildFeatures describe the executable orchestrating a run. They default to the base engine’s view — “dev”, no optional features — and are overwritten by a CLI’s main package before RunWithParsedArgs is called. cmd/stochadex sets them from its own -ldflags version stamp and its compiled-in feature list, so a provenance line reports exactly the binary that ran rather than a generic default.

var (
    BuildVersion  = "dev"
    BuildFeatures []string
)

BuildRevision is the git commit a build was stamped with via -ldflags, for builds that cannot read it from the embedded Go build info. The OCI image is exactly this case: its build context excludes .git (see .dockerignore), so the toolchain has no VCS to read and the release workflow passes the commit in explicitly instead. When set it wins over the embedded build info; when empty the build info is used, which is how the binary releases (built with .git present) report their revision.

var BuildRevision = ""

func CheckForDeadlock

func CheckForDeadlock(generator *simulator.ConfigGenerator) error

CheckForDeadlock reports whether the generator’s within-step wiring (params_from_upstream) contains a dependency cycle that would deadlock the channel-based execution strategies (the default and persistent-worker strategies), returning a descriptive error naming the partitions in each cycle. It runs no simulation. Without this pre-flight check such a cycle surfaces only as an opaque runtime “all goroutines are asleep - deadlock!” with no indication of which partitions are at fault. See pkg/graph.

func LoadFormat

func LoadFormat(format string, fields map[string]interface{}) (*simulator.StateTimeStorage, error)

LoadFormat loads storage for a single named source format from raw fields. It exists so a transport registered above this package — the S3 source, which fetches an object and then needs it parsed — can reuse the local loaders verbatim instead of re-implementing their field handling (a CSV’s time_column/state_columns/skip_header, and so on).

The fields are round-tripped through YAML into the same typed structs the config path uses, so a transported source validates exactly like a local one.

func LogRunProvenance

func LogRunProvenance(w io.Writer)

LogRunProvenance writes a single machine-parseable provenance line to w at the start of a run. It is deliberately sent to stderr, never stdout, so it never corrupts a data output (StdoutOutputFunction writes result rows to stdout); in a containerised run — the image’s whole reason for being — the job log that captures stderr is the durable record a result is reproduced against. The line is space-separated key=value pairs so it survives log aggregation and greps cleanly:

stochadex-run version=v0.7.0 os=linux arch=arm64 revision=d44bcd8… features=arrow,cblas,duckdb,postgres,s3 image=sha256:abc…

revision and dirty come from the Go build info the toolchain embeds automatically (buildvcs, on by default); image= is present only when STOCHADEX_IMAGE_DIGEST is set. A result is only reproducible if you know which build produced it, and this line is that binding for the CLI/image surface.

func RegisterDataSource

func RegisterDataSource(name string, build func(fields map[string]interface{}) (*simulator.StateTimeStorage, error))

RegisterDataSource adds a data: source spelling that this package cannot depend on directly. It mirrors simulator.RegisterComponent: the engine stays lean, and the distributed CLI contributes the sources whose dependencies it alone carries.

func RegisterEnvironment

func RegisterEnvironment(typeName string, build EnvironmentBuilder)

RegisterEnvironment adds an environment spelling usable as the mcts_self_play macro’s `env:` type. Call it from an init function in the module that owns the environment; the engine only sees it if the binary links that module, so a downstream ships its own CLI (as cmd/stochadex does for the ONNX partition).

Panics on a duplicate name, so two modules cannot silently claim one spelling.

func RegisterIteration

func RegisterIteration(typeName string, build func(simulator.ComponentSpec) (simulator.Iteration, error))

RegisterIteration registers an iteration builder for a {type: …} spelling that lives downstream of this package. Call it from an init(); it panics on a duplicate so two packages cannot silently claim one name. The builder receives the whole ComponentSpec (Type plus Fields) and is responsible for strict field validation, just like the core builders.

func RegisteredEnvironments

func RegisteredEnvironments() []string

RegisteredEnvironments returns the registered spellings in sorted order, for error messages and for a downstream to check what its binary has linked.

func ResolveEnvironment

func ResolveEnvironment(spec simulator.ComponentSpec, settings macros.MCTSSearchSettings) ([]*simulator.PartitionConfig, error)

ResolveEnvironment dispatches an `env:` spec to its registered builder.

func ResolveIteration

func ResolveIteration(spec simulator.ComponentSpec) (simulator.Iteration, error)

ResolveIteration builds a simulator.Iteration from a data-spec ComponentSpec.

func Run

func Run(config *ApiRunConfig, socket *SocketConfig)

Run executes the configured simulation under the mode named by the config’s run: block. The default (empty or “batch”) preserves pre-run:-tier behaviour: serve a websocket when a socket config is active, otherwise run once to completion offline. “ensemble” runs one seeded member per seed concurrently.

func RunEnsembleToStorage

func RunEnsembleToStorage(config *ApiRunConfig) ([]simulator.EnsembleRun, error)

RunEnsembleToStorage runs the config’s ensemble (run: {mode: ensemble}) and returns each member’s recorded storage, index-aligned to run.seeds. It is the programmatic form of Run for ensemble configs — Run prints every member to stdout and exits, which suits a CLI but is unusable from a caller that wants the storages (to compute a statistic across the ensemble) or the error. Like RunMacros for the macros: tier, it lets the seeds and member count live in the config’s run: block rather than being passed in Go.

It runs the same deadlock pre-flight as Run and returns the error rather than exiting. The remaining constraints are inherited from the ensemble mechanism, which rebuilds each member by re-loading the source file for fresh, non-shared iteration instances:

An empty run.seeds is rejected.

func RunMacros

func RunMacros(config *ApiRunConfig) (*simulator.StateTimeStorage, error)

RunMacros expands and runs a config’s macros: tier and returns the resulting storage. It is the programmatic form of Run for macro configs: Run prints and exits, which suits a CLI and makes it unusable from a caller that wants the output or the error — a downstream driving a registered environment, say.

func RunWithParsedArgs

func RunWithParsedArgs(args ParsedArgs)

RunWithParsedArgs runs the configured simulation. The whole config is data (data-spec partitions and simulation, or expressions:, plus optional data:/macros: tiers), so it is resolved and run in-process with no Go toolchain.

func StepAndServeWebsocket

func StepAndServeWebsocket(generator *simulator.ConfigGenerator, stepDelay time.Duration, handle string, address string)

StepAndServeWebsocket steps a simulation and streams state updates over a websocket using simulator.WebsocketOutputFunction.

Usage hints:

type ApiRunConfig

ApiRunConfig is the concrete, YAML-loadable configuration for an API run: a main RunConfig, optional embedded runs, and an optional run-mode selector.

type ApiRunConfig struct {
    Main     RunConfig           `yaml:"main"`
    Embedded []EmbeddedRunConfig `yaml:"embedded,omitempty"`
    Run      RunModeConfig       `yaml:"run,omitempty"`
    // Data is the optional data: tier — a sub-simulation run to produce storage
    // for the macros: tier to analyse.
    Data *DataConfig `yaml:"data,omitempty"`
    // Macros is the optional macros: tier — partition-set-producing analysis
    // functions expanded against Data's storage.
    Macros []MacroConfig `yaml:"macros,omitempty"`
    // contains filtered or unexported fields
}

func LoadApiRunConfigFromYaml

func LoadApiRunConfigFromYaml(path string) *ApiRunConfig

LoadApiRunConfigFromYaml loads simulation configuration from a YAML file.

The whole config is data: partition iterations are {type: …} specs or expressions:, and every simulation: component is a {type: …} spec. It resolves and runs in-process — no code generation, no Go toolchain.

Parameters:

Returns:

YAML File Format:

main:
  partitions:
    - name: "process1"
      iteration: {type: wiener_process}
      params:
        variances: [0.1, 0.2]
      init_state_values: [0.0, 0.0]
      state_history_depth: 10
      seed: 42
  simulation:
    output_condition: {type: every_step}
    output_function: {type: stdout}
    termination_condition: {type: number_of_steps, max_steps: 1000}
    timestep_function: {type: constant, stepsize: 0.01}
    init_time_value: 0.0
embedded:
  - name: "sub_simulation"
    partitions: [...]
    simulation: [...]

Error Handling:

func (*ApiRunConfig) GetConfigGenerator

func (a *ApiRunConfig) GetConfigGenerator() *simulator.ConfigGenerator

GetConfigGenerator returns a ConfigGenerator for the main run. Any partition whose name matches an embedded run is replaced by an embedded simulation iteration wired to that embedded run.

type DataConfig

DataConfig is the data: tier: it produces the StateTimeStorage that macros analyse, either by running a sub-simulation (Partitions run for Steps) or by loading a file (Source). Sub-simulation partitions carry data-spec iterations (or expressions), like any other run.

type DataConfig struct {
    // Source loads storage from a file. When set, the sub-simulation fields below
    // are ignored.
    Source      *DataSource                 `yaml:"source,omitempty"`
    Partitions  []simulator.PartitionConfig `yaml:"partitions,omitempty"`
    Expressions []ExpressionConfig          `yaml:"expressions,omitempty"`
    Steps       int                         `yaml:"steps,omitempty"`
    Timestep    float64                     `yaml:"timestep,omitempty"`
    InitTime    float64                     `yaml:"init_time,omitempty"`
}

type DataSource

DataSource is the data: tier’s optional pre-recorded source: instead of running a sub-simulation, storage is loaded from a file or a database. Exactly one field is set.

type DataSource struct {
    Csv      *csvSource      `yaml:"csv,omitempty"`
    JsonLog  *jsonLogSource  `yaml:"json_log,omitempty"`
    Postgres *postgresSource `yaml:"postgres,omitempty"`
    // Extra captures any source key not named above, so a package layered on top of
    // api can contribute one through RegisterDataSource without this struct (and
    // therefore the engine's go.mod) having to know about its dependencies. The Arrow
    // source is registered this way by the distributed CLI, because arrow-go lives in
    // a separate opt-in module.
    Extra map[string]map[string]interface{} `yaml:",inline"`
}

type EmbeddedRunConfig

EmbeddedRunConfig names and embeds an additional RunConfig that can be wired into a partition in the main run.

type EmbeddedRunConfig struct {
    Name string    `yaml:"name"`
    Run  RunConfig `yaml:",inline"`
}

type EnvironmentBuilder

EnvironmentBuilder constructs the partitions of an MCTS self-play stack from a downstream environment’s data spec plus the search settings the config stated. Implementations fill the typed half of a macros.MCTSSelfPlaySpec, call macros.ApplyMCTSSearchSettings with the settings, and return macros.NewMCTSSelfPlayPartitions of the result.

type EnvironmentBuilder func(
    spec simulator.ComponentSpec,
    settings macros.MCTSSearchSettings,
) ([]*simulator.PartitionConfig, error)

type ExpressionConfig

ExpressionConfig binds a declarative expression specification to a partition by name, so that a partition’s whole update can be written as data in the config file.

An expression specification is just data: it is loaded straight from the YAML and evaluated at run time, so a config using only expressions needs no compilation at all. This is what lets a simulation be specified by something that does not write Go.

A partition named here may omit its iteration field, exactly as a partition backed by an embedded run may. The specification is inlined, so its keys are those of general.ExpressionIteration:

expressions:
  - partition: battery
    fields:
      - {name: soc}
      - {name: actual_dispatch}
    bindings:
      - {name: dispatch, expr: "clamp(dispatch_mw, -power_rating_mw, power_rating_mw)"}
    outputs: ["clamp(soc + dispatch * dt, 0, energy_capacity_mwh)", "dispatch"]
type ExpressionConfig struct {
    Partition                   string `yaml:"partition"`
    general.ExpressionIteration `yaml:",inline"`
}

type MacroConfig

MacroConfig is one entry in the macros: tier. It decodes its `type` and then the whole entry into that type’s typed spec, so field values keep their YAML types (see the package note on the `y`->true coercion).

type MacroConfig struct {
    Type string
    Spec macroSpec
}

func (*MacroConfig) UnmarshalYAML

func (m *MacroConfig) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML reads the macro type, then decodes the entry into the matching typed spec. The type is read via a map (which accepts any keys) so this does not trip the strict dead-key check; the typed spec decode that follows is what validates the remaining keys.

type ParsedArgs

ParsedArgs bundles CLI-derived inputs for running the API: the YAML config path and an optional socket config path.

type ParsedArgs struct {
    ConfigFile string
    SocketFile string
}

func ArgParse

func ArgParse() ParsedArgs

ArgParse parses CLI flags into a ParsedArgs.

type RunConfig

RunConfig represents a complete simulation run configuration with partitions and simulation settings.

This struct combines partition configurations with simulation control parameters to define a complete simulation run. It serves as the primary configuration structure for YAML-based simulation setup.

Fields:

YAML Structure:

partitions:
  - name: "process1"
    iteration: {type: wiener_process}
    params:
      variances: [0.1, 0.2]
    init_state_values: [0.0, 0.0]
  - name: "process2"
    iteration: {type: poisson_process}
    params:
      rates: [0.5, 1.0]
    init_state_values: [0.0, 0.0]

Related Types:

type RunConfig struct {
    Partitions []simulator.PartitionConfig `yaml:"partitions"`
    // Expressions declaratively supply the iteration for the partitions they name.
    Expressions []ExpressionConfig `yaml:"expressions,omitempty"`
    // SimulationStrings holds the simulation block as loaded (component fields are
    // {type: ...} data specs). It is resolved into Simulation at load time.
    SimulationStrings simulator.SimulationConfigStrings `yaml:"simulation"`
    // Simulation is the resolved simulation config used to build the generator.
    Simulation simulator.SimulationConfig `yaml:"-"`
}

func (*RunConfig) GetConfigGenerator

func (r *RunConfig) GetConfigGenerator() *simulator.ConfigGenerator

GetConfigGenerator constructs a ConfigGenerator preloaded with the run’s SimulationConfig and Partitions, and gives any partition named by an Expressions entry a declarative ExpressionIteration built from that entry.

type RunModeConfig

RunModeConfig selects what a run *does* with the assembled simulation — the one thing that is not a partition and that the partition tiers cannot express.

Modes:

type RunModeConfig struct {
    Mode string `yaml:"mode,omitempty"`
    // Seeds are the per-member global seeds for ensemble mode (one member each).
    Seeds []uint64 `yaml:"seeds,omitempty"`
    // Concurrency bounds how many ensemble members run at once; <= 0 defaults to
    // GOMAXPROCS.
    Concurrency int `yaml:"concurrency,omitempty"`
}

type SocketConfig

SocketConfig configures an optional real-time websocket used to stream simulation updates.

type SocketConfig struct {
    Address          string `yaml:"address"`
    Handle           string `yaml:"handle"`
    MillisecondDelay uint64 `yaml:"millisecond_delay"`
}

func LoadSocketConfigFromYaml

func LoadSocketConfigFromYaml(path string) *SocketConfig

LoadSocketConfigFromYaml loads a SocketConfig from YAML. If the path is empty, it returns a zero-valued config and logs that sockets are disabled.

func (*SocketConfig) Active

func (s *SocketConfig) Active() bool

Active reports whether the websocket server should be started.

Generated by gomarkdoc