flexmeasures.data.models.planning.scheduling_problem

Solver-agnostic preparation of the device scheduler’s inputs.

flexmeasures.data.models.planning.linear_optimization.device_scheduler() (Pyomo) and flexmeasures.data.models.planning.highspy_optimization.device_scheduler_highspy() (direct HiGHS) build the same mathematical model in two very different representations, so the model construction itself is necessarily written twice. Everything around it is not: normalising arguments, resolving stock groups, converting legacy commitments, deriving Big-Ms, and turning solver output back into schedules and costs is plain pandas/numpy work with no solver in it.

Keeping that work here means the two backends cannot drift apart on input handling — only on the model, which is what the equivalence tests in tests/test_highspy_equivalence.py compare. It also gives both backends a single place to grow support for a new scheduling feature’s inputs.

Functions

flexmeasures.data.models.planning.scheduling_problem.aggregate_commodity_costs(commitments: list[DataFrame], subcommitment_costs: dict) → dict: Sum sub-commitment costs per commodity, skipping commitments without one.

flexmeasures.data.models.planning.scheduling_problem.aggregate_subcommitment_costs(subcommitment_costs: dict, commitment_mapping: dict, merged_constituents: dict | None = None, deviations: dict | None = None) → dict

Sum sub-commitment costs back onto the commitments they were split from.

A merged sub-commitment stands for several commitments at once, so its realised cost is shared out by their own prices against the deviation they share: a commitment priced up_i and down_i carries u * up_i + d * down_i of it. That is exact rather than apportioned, because the deviation is one and the same for every commitment merged into it, which is why merging them costs nothing in what can be reported afterwards.

Costs come back ordered by commitment, since callers read them off alongside the commitments themselves.

Parameters:: deviations – per sub-commitment index, its realised (upwards, downwards) deviation, signed as the solver holds them.

flexmeasures.data.models.planning.scheduling_problem.convert_commitments_to_subcommitments(dfs: list[DataFrame]) → tuple[list[DataFrame], dict[int, int]]

Transform commitments, each specifying a group for each time step, to sub-commitments, one per group.

‘Groups’ are a commitment concept (grouping time slots of a commitment), making it possible that deviations/breaches can be accounted for properly within this group (e.g. highest breach per calendar month defines the penalty). Here, we define sub-commitments, by separating commitments by group and by direction of deviation (up, down).

We also enumerate the time steps in a new column “j”.

For example, given contracts A and B (represented by 2 DataFrames), each with 3 groups, we return (sub)commitments A1, A2, A3, B1, B2 and B3, where A,B,C is the enumerated contract and 1,2,3 is the enumerated group.

flexmeasures.data.models.planning.scheduling_problem.deviation_price(commitment: DataFrame, column: str) → float

The single deviation price that the optimizers apply to a commitment, for the given price column.

A commitment carries one pair of deviation variables, priced by one pair of prices, so the price in its first row stands for the whole commitment. convert_commitments_to_subcommitments guarantees that is well defined, by rejecting a group whose prices differ per row. A missing column, or a missing price, means no price at all.

flexmeasures.data.models.planning.scheduling_problem.interchangeable_subcommitments(commitments: list[DataFrame], device_group_lookup: dict[int, dict]) → list[list[int]]

Sub-commitments that impose the identical constraint, grouped, singletons dropped.

Such sub-commitments pin the same deviation in any bounded solution, because their constraints say the same thing, so one of them can carry the group: the duplicates add a variable pair and a constraint row each without adding any information. Replacing a group by one sub-commitment carrying the summed prices is what lets a commitment that is not convex on its own be priced against a partner that more than compensates, since there is then one deviation to inflate rather than one each (GH#2534).

flexmeasures.data.models.planning.scheduling_problem.loss_coefficients(efficiency: float) → tuple[float, float]

Coefficients (a, b) of one step of the stock recursion, for how=”linear”.

stock[j] = a * stock[j-1] + b * change[j]

Mirrors apply_stock_changes_and_losses(), which we cannot call here because it expects numbers, while change[j] may be a Pyomo expression.

flexmeasures.data.models.planning.scheduling_problem.merge_interchangeable_subcommitments(commitments: list[DataFrame], commitment_mapping: dict[int, int], device_group_lookup: dict[int, dict], worth_merging) → tuple[list[DataFrame], dict[int, int], dict[int, dict], dict[int, list[tuple]]]

Replace each group of interchangeable sub-commitments that worth_merging selects by one carrying their summed prices.

The duplicates say nothing the first one does not, so the merged sub-commitment constrains the solver exactly as the group did, with one pair of deviation variables and one constraint row in place of one each. What the group’s members do not share is the level of their prices, so the merged sub-commitment is priced on their sum, and each member’s own prices are recorded against it so that its share of the cost can be worked out again after the solve.

Parameters:: worth_merging – called with a group’s member indices; a group it rejects is left alone, so a problem that gains nothing keeps its model unchanged.
Returns:: the sub-commitments, their mapping back to the original commitments, their device-group lookup, and, per merged index, the (original index, upwards price, downwards price) of each member.

flexmeasures.data.models.planning.scheduling_problem.planned_power_per_device(power_per_device, start, end, resolution) → list[Series]: Turn each device’s planned power values into a time series.

flexmeasures.data.models.planning.scheduling_problem.prepare_scheduling_problem(device_constraints: list[DataFrame], ems_constraints: DataFrame | list[DataFrame], commitment_quantities: list[Series] | None = None, commitment_downwards_deviation_price: list[Series] | list[float] | None = None, commitment_upwards_deviation_price: list[Series] | list[float] | None = None, commitments: list[DataFrame] | list[Commitment] | None = None, initial_stock: float | list[float] = 0, stock_groups: dict[int, list[int]] | None = None, ems_constraint_groups: list[list[int]] | None = None, device_power_bands: list[list[tuple[float, float]] | None] | None = None, coupling_groups: dict[str, list[tuple[int, float]]] | None = None, balance_groups: dict[str, list[int]] | None = None) → SchedulingProblem: Normalise and validate device_scheduler’s arguments into a SchedulingProblem.

Note

This adds a “stock delta” column to the passed device_constraints DataFrames in place, as the schedulers have always done.

flexmeasures.data.models.planning.scheduling_problem.solver_options(solver_name: str) → dict

The solver options to apply, for the given solver.

HiGHS (whether reached through Pyomo as appsi_highs or directly as highspy – both match on “highs”) gets a tight-tolerance profile, so the two backends cannot disagree on tolerances and silently produce different schedules. Operator-configured options are applied last, so they win.

flexmeasures.data.models.planning.scheduling_problem.validate_highs_options(options: dict) → None

Raise if HiGHS would refuse any of these options.

Pyomo’s appsi_highs interface applies solver options without checking HiGHS’ return status, so an unknown name, an invalid value, or a feature missing from the installed HiGHS build is otherwise ignored without a word. That silently turns a mis-typed option into a no-op, and a benchmark of it into a false negative. Probing a throwaway Highs instance surfaces the rejection instead.

Classes

class flexmeasures.data.models.planning.scheduling_problem.SchedulingProblem(start: object, end: object, resolution: object, device_constraints: list[~pandas.core.frame.DataFrame], ems_constraints_list: list[~pandas.core.frame.DataFrame], ems_constraint_device_groups: list[list[int]], device_to_group: dict[int, str], group_to_devices: dict[str, list[int]], commitments: list[~pandas.core.frame.DataFrame], commitment_mapping: dict[int, int], device_group_lookup: dict[int, dict], merged_constituents: dict[int, list[tuple]], convex_cost_curve: bool, Md: float, Mc: float, band_lookup: dict[int, list[tuple[float, float]]], coupling_device_specs: list[tuple[int, int, float]], balance_group_specs: list[list[int]], initial_stock: float | list[float], original_commitments: list[~pandas.core.frame.DataFrame] = <factory>)

Everything both scheduler backends need before building their model.

Produced by prepare_scheduling_problem(); see device_scheduler’s docstring for what the underlying arguments mean.

Md: float: Big-Ms bounding the search space for device power (Md) and commitment deviations (Mc)

__init__(start: object, end: object, resolution: object, device_constraints: list[~pandas.core.frame.DataFrame], ems_constraints_list: list[~pandas.core.frame.DataFrame], ems_constraint_device_groups: list[list[int]], device_to_group: dict[int, str], group_to_devices: dict[str, list[int]], commitments: list[~pandas.core.frame.DataFrame], commitment_mapping: dict[int, int], device_group_lookup: dict[int, dict], merged_constituents: dict[int, list[tuple]], convex_cost_curve: bool, Md: float, Mc: float, band_lookup: dict[int, list[tuple[float, float]]], coupling_device_specs: list[tuple[int, int, float]], balance_group_specs: list[list[int]], initial_stock: float | list[float], original_commitments: list[~pandas.core.frame.DataFrame] = <factory>) → None

balance_group_specs: list[list[int]]: device lists of the balance groups (internal commodity nodes), empty groups dropped

band_lookup: dict[int, list[tuple[float, float]]]: device index -> its signed power bands (S2 operation modes)

commitments: list[DataFrame]: Sub-commitments (one per commitment group and deviation direction), and the mapping from each sub-commitment index back to its original commitment index

property commodity_devices: dict

commodity -> set(device indices).

Computed on demand: only the EMS-level flow commitment constraints need it, and the per-row scan is not cheap enough to pay for unconditionally.

convex_cost_curve: bool: Whether every commitment’s deviation prices describe a convex cost curve (a non-convex curve needs binary commitment-sign variables). A merged sub-commitment is judged on the summed prices it carries, which is the point of merging it.

coupling_device_specs: list[tuple[int, int, float]]: (group index, device index, coefficient) triples for hard flow-coupling constraints

device_constraints: list[DataFrame]: Device constraints, with a “stock delta” column guaranteed to be present

device_group_lookup: dict[int, dict]: sub-commitment index -> {device group label -> member device indices}

device_to_group: dict[int, str]: device -> its primary stock group key, and stock group key -> member devices

ems_constraints_list: list[DataFrame]: EMS constraints, normalised to a list, plus the device indices each applies to

initial_stock_of(d) → float

The initial stock of device d, defaulting to 0.

Device indices reaching this from a commitment’s “device” column may be numpy floats, hence the cast.

merged_constituents: dict[int, list[tuple]]: Per merged sub-commitment index, the (original index, upwards price, downwards price) of each commitment merged into it, so that its share of the realised cost can be worked out again.

original_commitments: list[DataFrame]: The commitments as passed in, before the sub-commitment split. Only kept to derive commodity_devices lazily.

start: object: Timing, taken from the first device