analysis
import "github.com/umbralcalc/stochadex/pkg/analysis"Package analysis is the data layer around a simulation: getting time series into a *simulator.StateTimeStorage, addressing series inside one, and rendering them.
It deliberately does not build simulation topologies. The Applied* specs that expand into multi-partition inference, aggregation and optimisation topologies live in pkg/macros, which imports this package for its vocabulary. The dependency runs one way:
pkg/api → pkg/macros → pkg/analysis → pkg/simulatorThe vocabulary
DataRef is the shared currency: a partition name plus optional value indices and time range, resolvable against a storage. Both the plotting helpers here and every windowed construction in pkg/macros are expressed in terms of it, which is what lets a config name a series once and use it for either. GroupedStateTimeStorage layers a grouping over a storage so aggregations can be taken per accepted value group.
Getting data in and out
- csv.go, logs.go — load a storage from a CSV file or JSON log entries.
- postgres.go — read a storage from PostgreSQL, write one back, and PostgresDbOutputFunction to stream a live run into a table.
- partitions.go — build a storage by running partitions, or append partitions to an existing one (this is what the `data:` tier of a YAML config resolves to).
- dataframe.go — convert a partition to and from a gota DataFrame.
Rendering
plot.go produces go-echarts line and scatter charts, from either a storage (via DataRef) or a DataFrame. ColourGenerator cycles a palette across series.
Index
- func AddPartitionsToStateTimeStorage(storage *simulator.StateTimeStorage, partitions []*simulator.PartitionConfig, windowSizeByPartition map[string]int) *simulator.StateTimeStorage
- func GetDataFrameFromPartition(storage *simulator.StateTimeStorage, partitionName string) dataframe.DataFrame
- func NewLinePlotFromDataFrame(df *dataframe.DataFrame, xAxis string, yAxis string, groupBy …string) *charts.Line
- func NewLinePlotFromPartition(storage *simulator.StateTimeStorage, xRef DataRef, yRefs []DataRef, fillYRefs []FillLineRef) *charts.Line
- func NewScatterPlotFromDataFrame(df *dataframe.DataFrame, xAxis string, yAxis string, groupBy …string) *charts.Scatter
- func NewScatterPlotFromPartition(storage *simulator.StateTimeStorage, xRef DataRef, yRefs []DataRef) *charts.Scatter
- func NewStateTimeStorageFromCsv(filePath string, timeColumn int, stateColumnsByPartition map[string][]int, skipHeaderRow bool) (*simulator.StateTimeStorage, error)
- func NewStateTimeStorageFromJsonLogEntries(filename string) (*simulator.StateTimeStorage, error)
- func NewStateTimeStorageFromPartitions(partitions []*simulator.PartitionConfig, termination simulator.TerminationCondition, timestep simulator.TimestepFunction, initTime float64) *simulator.StateTimeStorage
- func NewStateTimeStorageFromPostgresDb(db *PostgresDb, partitionNames []string, startTime float64, endTime float64) (*simulator.StateTimeStorage, error)
- func SetPartitionFromDataFrame(storage *simulator.StateTimeStorage, partitionName string, df dataframe.DataFrame, overwriteTime bool)
- func WriteStateTimeStorageToPostgresDb(db *PostgresDb, storage *simulator.StateTimeStorage)
- type AppliedGrouping
- type ColourGenerator
- type DataPlotting
- type DataRef
- func (d *DataRef) GetFromStorage(storage *simulator.StateTimeStorage) [][]float64
- func (d *DataRef) GetSeriesNames(storage *simulator.StateTimeStorage) []string
- func (d *DataRef) GetTimeIndexFromStorage(storage *simulator.StateTimeStorage, timeIndex int) []float64
- func (d *DataRef) GetValueIndices(storage *simulator.StateTimeStorage) []int
- type FillLineRef
- type
GroupedStateTimeStorage
- func NewGroupedStateTimeStorage(applied AppliedGrouping, storage *simulator.StateTimeStorage) *GroupedStateTimeStorage
- func (g *GroupedStateTimeStorage) GetAcceptedValueGroupLabels() []string
- func (g *GroupedStateTimeStorage) GetAcceptedValueGroups(tupIndex int) []float64
- func (g *GroupedStateTimeStorage) GetAcceptedValueGroupsLength() int
- func (g *GroupedStateTimeStorage) GetGroupTupleLength() int
- func (g *GroupedStateTimeStorage) GetGroupingPartition(tupIndex int) string
- func (g *GroupedStateTimeStorage) GetGroupingValueIndices(tupIndex int) []float64
- func (g *GroupedStateTimeStorage) GetPrecision() int
- type IndexRange
- type PostgresDb
- func NewPostgresDb(db *sql.DB, tableName string) *PostgresDb
- func (p *PostgresDb) OpenTableConnection() error
- func (p *PostgresDb) ReadStateInRange(partitionName string, startTime float64, endTime float64) (*sql.Rows, error)
- func (p *PostgresDb) WriteState(partitionName string, time float64, state []float64) error
- type PostgresDbOutputFunction
func AddPartitionsToStateTimeStorage
func AddPartitionsToStateTimeStorage(storage *simulator.StateTimeStorage, partitions []*simulator.PartitionConfig, windowSizeByPartition map[string]int) *simulator.StateTimeStorageAddPartitionsToStateTimeStorage extends the state time storage with newly generated values from the specified partitions.
For each existing partition name, windowSizeByPartition[name] sets StateHistoryDepth for the FromStorageIteration replay (default 1).
func GetDataFrameFromPartition
func GetDataFrameFromPartition(storage *simulator.StateTimeStorage, partitionName string) dataframe.DataFrameGetDataFrameFromPartition converts simulation partition data into a Gota DataFrame for convenient data manipulation and analysis.
This function extracts time series data from a simulation partition and converts it into a structured DataFrame format. The resulting DataFrame has a “time” column followed by columns for each state dimension, making it easy to perform data analysis, visualization, and export operations.
DataFrame Structure:
- Column 0: “time” - Contains the time axis values
- Column 1+: State dimension columns labeled by their integer indices (0, 1, 2, …)
Parameters:
- storage: StateTimeStorage containing the simulation data
- partitionName: Name of the partition to extract data from
Returns:
- dataframe.DataFrame: Gota DataFrame with time and state columns
Example:
// Extract price data from simulation storage
df := GetDataFrameFromPartition(storage, "prices")
// Access time column
timeCol := df.Col("time")
// Access state columns
price1Col := df.Col("0") // First price dimension
price2Col := df.Col("1") // Second price dimension
// Perform analysis
meanPrice1 := price1Col.Mean()
maxPrice2 := price2Col.Max()Use Cases:
- Data visualization and plotting
- Statistical analysis and computation
- Data export to various formats (CSV, JSON, etc.)
- Integration with data analysis tools
- Time series analysis and forecasting
Performance:
- O(n * m) time complexity where n is number of samples, m is state dimensions
- Memory usage: O(n * m) for the resulting DataFrame
- Efficient for moderate-sized datasets (< 1M samples)
Error Handling:
- Panics if partition name is not found in storage
- Provides helpful error messages with available partition names
func NewLinePlotFromDataFrame
func NewLinePlotFromDataFrame(df *dataframe.DataFrame, xAxis string, yAxis string, groupBy ...string) *charts.LineNewLinePlotFromDataFrame renders a line chart from a dataframe using the specified X and Y columns.
Usage hints:
- Optionally split by a single groupBy column into multiple series.
func NewLinePlotFromPartition
func NewLinePlotFromPartition(storage *simulator.StateTimeStorage, xRef DataRef, yRefs []DataRef, fillYRefs []FillLineRef) *charts.LineNewLinePlotFromPartition renders a multi-series line chart from storage using an X reference and one or more Y references.
Usage hints:
- yRefs may contain multiple series each; one line per series is added.
- Optional filled bands can be added via fillYRefs.
func NewScatterPlotFromDataFrame
func NewScatterPlotFromDataFrame(df *dataframe.DataFrame, xAxis string, yAxis string, groupBy ...string) *charts.ScatterNewScatterPlotFromDataFrame renders a scatter plot using columns of a dataframe.
Usage hints:
- Optionally provide a single groupBy column to split series.
func NewScatterPlotFromPartition
func NewScatterPlotFromPartition(storage *simulator.StateTimeStorage, xRef DataRef, yRefs []DataRef) *charts.ScatterNewScatterPlotFromPartition renders a scatter plot from storage-backed DataRef axes.
Usage hints:
- X-axis must reference a single series (typically time).
- Each DataRef in yRefs may contain multiple series; a series is added for each.
func NewStateTimeStorageFromCsv
func NewStateTimeStorageFromCsv(filePath string, timeColumn int, stateColumnsByPartition map[string][]int, skipHeaderRow bool) (*simulator.StateTimeStorage, error)NewStateTimeStorageFromCsv creates a StateTimeStorage from CSV data.
This function reads time series data from a CSV file and organizes it into partitions for use in stochadex simulations. It supports multiple partitions with different column configurations.
Parameters:
- filePath: Path to the CSV file to read (must exist and be readable)
- timeColumn: Index of the column containing timestamps (0-based indexing)
- stateColumnsByPartition: Map of partition names to column indices for their state values
- skipHeaderRow: Whether to skip the first row as headers (recommended for CSV files with headers)
Returns:
- *StateTimeStorage: Storage containing the loaded time series data, organized by partition
- error: Any error encountered during file reading or parsing
CSV Format Requirements:
- Time column must contain parseable float64 values
- State columns must contain parseable float64 values
- All rows must have the same number of columns
- Missing or malformed values will cause parsing errors
Example:
// Load data from a CSV with time in column 0, prices in columns 1-2, volumes in column 3
storage, err := NewStateTimeStorageFromCsv(
"market_data.csv",
0, // time in first column
map[string][]int{
"prices": {1, 2}, // prices partition uses columns 1 and 2
"volumes": {3}, // volumes partition uses column 3
},
true, // skip header row
)
if err != nil {
log.Fatal("Failed to load CSV data:", err)
}Error Handling:
- File not found: Returns error with file path
- CSV parsing errors: Returns error with parsing details
- Invalid numeric values: Returns error with conversion details
- Inconsistent row lengths: Returns error with row information
Performance Notes:
- Loads entire file into memory (consider file size for large datasets)
- O(n) time complexity where n is the number of rows
- Memory usage: O(n * m) where m is the total number of state columns
func NewStateTimeStorageFromJsonLogEntries
func NewStateTimeStorageFromJsonLogEntries(filename string) (*simulator.StateTimeStorage, error)NewStateTimeStorageFromJsonLogEntries reads a file up to a given number of iterations into a simulator.StateTimeStorage struct.
func NewStateTimeStorageFromPartitions
func NewStateTimeStorageFromPartitions(partitions []*simulator.PartitionConfig, termination simulator.TerminationCondition, timestep simulator.TimestepFunction, initTime float64) *simulator.StateTimeStorageNewStateTimeStorageFromPartitions generates a new simulator.StateTimeStorage by running a simulation with the specified partitions configured.
func NewStateTimeStorageFromPostgresDb
func NewStateTimeStorageFromPostgresDb(db *PostgresDb, partitionNames []string, startTime float64, endTime float64) (*simulator.StateTimeStorage, error)NewStateTimeStorageFromPostgresDb reads from a PostgreSQL database over a pre-defined time interval into a simulator.StateTimeStorage struct.
func SetPartitionFromDataFrame
func SetPartitionFromDataFrame(storage *simulator.StateTimeStorage, partitionName string, df dataframe.DataFrame, overwriteTime bool)SetPartitionFromDataFrame updates a partition’s values from a Gota dataframe with schema [time, 0, 1, …]. If overwriteTime is true, the storage’s time vector is replaced with the “time” column.
func WriteStateTimeStorageToPostgresDb
func WriteStateTimeStorageToPostgresDb(db *PostgresDb, storage *simulator.StateTimeStorage)WriteStateTimeStorageToPostgresDb writes all of the data in the state time storage to a PostgreSQL database.
type AppliedGrouping
AppliedGrouping configures a grouping transformation on data.
type AppliedGrouping struct {
GroupBy []DataRef
Precision int
}type ColourGenerator
ColourGenerator iterates over the default ECharts categorical palette.
type ColourGenerator struct {
// contains filtered or unexported fields
}func (*ColourGenerator) Next
func (cg *ColourGenerator) Next() stringNext returns the next colour in the ECharts palette, cycling when the end is reached.
type DataPlotting
DataPlotting declares optional transformations for plotting, such as treating a reference as time and restricting to a time index range.
type DataPlotting struct {
IsTime bool
TimeRange *IndexRange
}type DataRef
DataRef identifies a subset of data stored in StateTimeStorage. It can reference the special time axis or one or more value indices of a partition. Optional plotting hints may be supplied via Plotting.
type DataRef struct {
PartitionName string
ValueIndices []int
Plotting *DataPlotting
}func (*DataRef) GetFromStorage
func (d *DataRef) GetFromStorage(storage *simulator.StateTimeStorage) [][]float64GetFromStorage returns the entire referenced series. For a time reference, this is a single series containing all times; for a value reference, this is one series per value index.
func (*DataRef) GetSeriesNames
func (d *DataRef) GetSeriesNames(storage *simulator.StateTimeStorage) []stringGetSeriesNames returns human-readable series labels for plotting. Time references are labeled “time”; value references are labeled as “<partition> <index>”.
func (*DataRef) GetTimeIndexFromStorage
func (d *DataRef) GetTimeIndexFromStorage(storage *simulator.StateTimeStorage, timeIndex int) []float64GetTimeIndexFromStorage returns the data at a specific time index. For a time reference, this is a single-element slice containing the time value; for a value reference, this is the row slice for that time index.
func (*DataRef) GetValueIndices
func (d *DataRef) GetValueIndices(storage *simulator.StateTimeStorage) []intGetValueIndices returns the referenced value indices, defaulting to all indices within the partition when ValueIndices is nil.
type FillLineRef
FillLineRef specifies an upper and lower bound series used to fill a confidence region in a line plot.
type FillLineRef struct {
Upper DataRef
Lower DataRef
}type GroupedStateTimeStorage
GroupedStateTimeStorage is a representation of simulator.StateTimeStorage which has already had a grouping transformation applied to it.
type GroupedStateTimeStorage struct {
Storage *simulator.StateTimeStorage
// contains filtered or unexported fields
}func NewGroupedStateTimeStorage
func NewGroupedStateTimeStorage(applied AppliedGrouping, storage *simulator.StateTimeStorage) *GroupedStateTimeStorageNewGroupedStateTimeStorage creates a new GroupedStateTimeStorage given the provided simulator.StateTimeStorage and applied grouping.
func (*GroupedStateTimeStorage) GetAcceptedValueGroupLabels
func (g *GroupedStateTimeStorage) GetAcceptedValueGroupLabels() []stringGetAcceptedValueGroupLabels returns the unique group labels that were found in the data which are typically used for labelling plots.
func (*GroupedStateTimeStorage) GetAcceptedValueGroups
func (g *GroupedStateTimeStorage) GetAcceptedValueGroups(tupIndex int) []float64GetAcceptedValueGroups returns the unique groups that were found in the data which are typically used to configure group aggregation partitions.
func (*GroupedStateTimeStorage) GetAcceptedValueGroupsLength
func (g *GroupedStateTimeStorage) GetAcceptedValueGroupsLength() intGetAcceptedValueGroupsLength returns the number of accepted value groups (equivalent to the length of the state vector in simulation partition).
func (*GroupedStateTimeStorage) GetGroupTupleLength
func (g *GroupedStateTimeStorage) GetGroupTupleLength() intGetGroupTupleLength returns the length of tuple in the grouping index construction.
func (*GroupedStateTimeStorage) GetGroupingPartition
func (g *GroupedStateTimeStorage) GetGroupingPartition(tupIndex int) stringGetGroupingPartitions returns the partition used in the data for grouping.
func (*GroupedStateTimeStorage) GetGroupingValueIndices
func (g *GroupedStateTimeStorage) GetGroupingValueIndices(tupIndex int) []float64GetGroupingValueIndices returns the value indices used in the data for grouping.
func (*GroupedStateTimeStorage) GetPrecision
func (g *GroupedStateTimeStorage) GetPrecision() intGetPrecision returns the requested float precision for grouping.
type IndexRange
IndexRange represents an inclusive-exclusive [Lower, Upper) span of indices. It is commonly used to clip time-series windows for plotting.
type IndexRange struct {
Lower int
Upper int
}type PostgresDb
PostgresDb is a struct which can be configured to define interactions with a PostgresSQL database.
type PostgresDb struct {
User string
Password string
Dbname string
TableName string
// DB is the database/sql handle used for reads and writes. Set it to your own handle —
// sql.Open with any DSN or driver (a remote TimescaleDB or another Postgres-wire database
// with host/port/sslmode, or a pooled *sql.DB) — to use the clean database/sql write path.
// Leave it nil and OpenTableConnection opens a local Postgres from User/Password/Dbname.
DB *sql.DB
}func NewPostgresDb
func NewPostgresDb(db *sql.DB, tableName string) *PostgresDbNewPostgresDb returns a PostgresDb backed by a caller-provided database/sql handle — the clean database/sql write path. The handle may target any Postgres-wire database (Postgres, TimescaleDB, QuestDB, …) opened with any DSN or driver; tableName is the destination table.
func (*PostgresDb) OpenTableConnection
func (p *PostgresDb) OpenTableConnection() errorOpenTableConnection connects to the PostgreSQL database or creates it if it doesn’t exist.
func (*PostgresDb) ReadStateInRange
func (p *PostgresDb) ReadStateInRange(partitionName string, startTime float64, endTime float64) (*sql.Rows, error)ReadStateInRange retrieves all entries between a specified start and end time range for a given partition.
func (*PostgresDb) WriteState
func (p *PostgresDb) WriteState(partitionName string, time float64, state []float64) errorWriteState writes a new partition state value to the database.
type PostgresDbOutputFunction
PostgresDbOutputFunction writes the data from the simulation to a PostgresSQL database when the simulator.OutputCondition is met.
type PostgresDbOutputFunction struct {
// contains filtered or unexported fields
}func NewPostgresDbOutputFunction
func NewPostgresDbOutputFunction(db *PostgresDb) *PostgresDbOutputFunctionNewPostgresDbOutputFunction creates a new PostgresDbOutputFunction.
func (*PostgresDbOutputFunction) Configure
func (p *PostgresDbOutputFunction) Configure(*simulator.Settings)func (*PostgresDbOutputFunction) Output
func (p *PostgresDbOutputFunction) Output(partitionName string, state []float64, cumulativeTimesteps float64)Generated by gomarkdoc