MGA Compass API

Exploring a set from Python

Exploring a near-optimal set from Python, e.g. in a notebook.

load() reads a set in the near-optimal set format (mga_format) and returns a Space: the set’s designs in PoI space, ready to navigate:

import mga_compass as mc

space = mc.load("model-energy/")
space            # the PoIs and the cost, with their units and ranges
space.points     # one row per design
space.optimum    # the cheapest design

result = space.navigate(space.optimum, {
    "wind": ("increase", 6),     # by at least 6, in wind's unit
    "TOTEX": ("keep", "3%"),     # within 3% of the start's cost
})
result           # start and target side by side, and what had to be adjusted
result.traverse  # the path from start to target: cost along it, with a lower bound
space.navigate(result, {...})    # carry on from the target

space.outputs()                       # the set's detailed outputs
space.output("generation", result)    # one of them, for the target
mga_compass.space.Delta = float | str

How far to move a PoI or the cost: a number in its unit, or a percentage of its value at the start, such as "20%".

mga_compass.space.Request = typing.Literal['increase', 'decrease', 'keep'] | tuple[typing.Literal['increase', 'decrease', 'keep'], float | str]

What to ask of one PoI or the cost: (direction, delta), or a direction alone, meaning a delta of 0.

mga_compass.space.load(source, directory=None, *, units='display', slack=None, solver_name='highs', solver_options=None)[source]

Load a near-optimal set as a Space.

source is the set’s directory, its zip archive, or the URL of a published archive (https:// or file://). An archive is unpacked into directory, replacing whatever is there; the set’s outputs are read from that directory later, so it has to stay. The set is checked against its manifest’s checksums before anything is used.

See Space for units, slack, solver_name and solver_options.

Parameters:
Return type:

Space

class mga_compass.space.Space(nos, directory, *, units='display', slack=None, solver_name='highs', solver_options=None)[source]

Bases: object

The designs of a near-optimal set in PoI space, to navigate.

Navigation moves within the convex hull of these designs: every mix of them is a design too, at most as expensive as the same mix of their costs.

  • units: "display" (default) gives every PoI and the cost in the unit the set says to show it in (its manifest’s display, e.g. GW instead of MW); "model" keeps the units of the model. Navigation finds the same designs either way; values and deltas are just in these units.

  • slack: the slack level to explore, one of the set’s. By default its largest, using all designs. A smaller one keeps the optimum and the designs found at slack levels up to it.

  • solver_name, solver_options: the LP solver navigation uses; the options default to default_solver_options().

Parameters:
  • nos (mga_format.NearOptimalSet)

  • directory (str | Path)

  • units (dict[str, str])

  • slack (float | None)

  • solver_name (str)

  • solver_options (dict | None)

unit_system: Literal['display', 'model']
cost: str
pois: list[str]
title: str

What the set is of, from its manifest; may be empty.

description: str

The model behind the set, from its manifest; may be empty.

units: dict[str, str]
points: DataFrame

One row per design (index: its point id), one column per PoI plus the cost.

duals: DataFrame

Each design’s duals (index: its point id), one column per PoI: a subgradient of the cost at the design, in cost units per PoI unit.

property optimum: Series

The cheapest design: its PoI values and its cost.

summary()[source]

Each PoI and the cost: its unit, the range of the designs, and its value at the optimum.

Return type:

DataFrame

plot_ranges(axes=None)[source]

Each PoI’s range over the space’s designs, as a bar from its smallest to its largest value, with the optimum marked: one panel per unit. A PoI of a single component takes the colour the set gives that component. Needs matplotlib (pip install mga-compass[notebook]). Returns the panels’ axes.

plot_projection(x, y, navigation=None, ax=None)[source]

The space projected onto two of its columns, PoIs or the cost: the outline of the designs’ convex hull there, the designs, and the optimum. With navigation, also its start, target and the path between them.

The space is the convex hull of the designs, and the projection of a convex hull is the hull of the projected points: the outline is exact for the space navigation searches. Designs inside it lie deeper along the other columns. The true near-optimal region of the model is larger; the hull is an inner approximation of it. Needs matplotlib and scipy (pip install mga-compass[notebook]). Returns the axes.

Parameters:
navigate(start, request)[source]

Move from design start as request asks, to the cheapest design that meets it.

start is a design (a row of points, optimum, or any mix of designs with all its values), a point id, or the result of an earlier navigation, to carry on from its target.

request maps PoIs, and the cost if wanted, to (direction, delta):

  • ("increase", d): raise it by at least d, then as far as possible;

  • ("decrease", d): lower it by at least d, then as far as possible;

  • ("keep", d): keep it within d of its start value, then as close to it as possible.

d is in the unit of units, or a percentage of the start value such as "20%"; a direction alone means d = 0. The order of request is the priority order: every request is met first, then each is pushed (“as far”, “as close”) as well as the ones before it allow, so a keep listed before an increase limits it more than one after it. When a request cannot be met in full, it is met as far as possible and Navigation.changes says how. PoIs left out are free: they move however the requests make cheapest.

The path from start to target is computed too: Navigation.traverse.

Parameters:
Return type:

Navigation

traverse(start, target)[source]

The path from design start to design target: see Traverse.

Parameters:
Return type:

Traverse

outputs()[source]

The set’s detailed outputs, one row each: the unit (in this space’s units), the index sets besides point, and the description.

Return type:

DataFrame

plots()[source]

The plots the set declares for its outputs, one row each: the type, the output, the set along x and the set stacked, if any. Draw one with plot().

Return type:

DataFrame

plot(name, design, ax=None)[source]

Draw plot name of the set (see plots()) for a design, as the web platform does: its output summed over the sets other than x and stack, members in the sets’ order, with the sets’ labels and colours. design is as for output(). Needs matplotlib (pip install mga-compass[notebook]). Returns the axes.

Parameters:
output(name, design)[source]

Detailed output name (see outputs()) of a design, such as hourly generation per technology.

A design here is a mix of the set’s designs, and its output is the same mix of their outputs. design is a navigation result (its target), the weights of a design (e.g. result.alpha or result.traverse.alpha_at(0.5)), a point id, or a design’s PoI values, whose cheapest mix is found first.

Returns a Series indexed by the output’s sets in their order (unstack it for a table), in this space’s units. Reads only the outputs of the designs in the mix.

Parameters:
Return type:

Series

class mga_compass.space.Navigation(space, start, target, requested, applied, traverse)[source]

Bases: object

The result of Space.navigate().

Parameters:
space: Space
start: Series

The design navigation started from.

target: Series

The design it reached: the cheapest one meeting the requests.

requested: Constraints

The requests, as constraints: one row per PoI and the cost, in priority order, with direction ">=", "<=", "==" or "" (free).

applied: Constraints

The constraints as navigation could meet them.

traverse: Traverse

The path from start to target.

changes: list[str]

How requests that could not be met in full were adjusted, in words.

property alpha: Alpha

The target as weights of the space’s designs (point id -> weight).

to_frame()[source]

Each PoI and the cost: its unit, start and target values, the change, and what was requested of it.

Return type:

DataFrame

class mga_compass.space.Traverse(space, breakpoints)[source]

Bases: object

The path from one design to another, computed by Space.traverse().

The PoIs move in a straight line from start to target as beta goes from 0 to 1. The cost along the path is the cheapest mix of the space’s designs with those PoIs, refined at breakpoints until it improves by less than 0.1%: an upper bound on the true cost. lower_bound gives a matching lower bound from the designs’ duals; the true cost lies between the two.

Parameters:
space: Space
breakpoints: list[Breakpoint]

Where the cost curve bends, from beta = 0 (the start) to beta = 1 (the target), each with its weights and its design.

property betas: list[float]
to_frame()[source]

One row per breakpoint (index: beta): the PoIs, the cost (upper bound) and the cost’s lower bound there.

Return type:

DataFrame

at(beta)[source]

The design at beta (0 to 1): its PoIs and cost, between the neighbouring breakpoints.

Parameters:

beta (float)

Return type:

Series

alpha_at(beta)[source]

The weights of the design at beta: the neighbouring breakpoints’ weights, mixed linearly.

Parameters:

beta (float)

Return type:

Alpha

weights_at(beta)[source]

The set’s designs mixed at beta, heaviest first (index: point): each one’s weight, and the slack, kind and key it has in the set (key, the producer’s identifier of its solve, only if the set records it). Weights below 1e-9, the solver’s round-off, are left out.

Parameters:

beta (float)

Return type:

DataFrame

property lower_bound: Series[source]

A lower bound on the cost along the path: its value (index: beta) where it bends, at 0, at 1 and in between; it is linear in between.

The largest of the bounds the designs’ duals give, cost >= cost_i + dual_i . (PoIs - PoIs_i), for each design with a weight at some breakpoint, and of the optimal cost: no design is cheaper.

lower_bound_at(beta)[source]

The cost’s lower bound at beta.

Parameters:

beta (float)

Return type:

float

plot(ax=None, beta=None)[source]

Plot the cost along the path, with its lower bound below it, the optimal cost and the space’s budget (the optimal cost plus its slack). With beta, mark the design there. Needs matplotlib (pip install mga-compass[notebook]). Returns the axes.

Parameters:

beta (float | None)

plot_pois(beta=None, axes=None)[source]

Bars of each PoI, one panel per unit, in the PoIs’ colours.

Without beta, the start (pale) and the target side by side. With beta, the design there, with dashed lines at the start’s and the target’s values: the bar moves between them as beta goes from 0 to 1. Returns the panels’ axes.

Parameters:

beta (float | None)

plot_radar(beta=1.0, ax=None)[source]

A radar chart of the PoIs at the start and at beta, by default the target, each PoI scaled from the space’s smallest (0) to largest value (1). Returns the axes.

Parameters:

beta (float)

cost_at(beta)[source]

The cost at beta: the upper bound, linear between breakpoints.

Parameters:

beta (float)

Return type:

float

largest_gap()[source]

How far the cost is above its lower bound at most, anywhere on the path. Both are linear between their bends, so checking those is enough.

Return type:

float

Navigation and interpolation

mga_compass.core.configure_solver_logging(log_path)[source]

Route linopy/solver logs to a file instead of the console. Call this once at startup.

Parameters:

log_path (Path)

Return type:

None

exception mga_compass.core.SolverError[source]

Bases: RuntimeError

An LP did not solve to optimality. linopy reports a solver that gives up (e.g. HiGHS’ “unknown”) as status “ok” and still fills in a solution – all zeros – so every solve’s termination condition has to be checked.

mga_compass.core.navigate(points, constraints, obj_label, solver_name='highs', solver_options=None)[source]
Parameters:
Return type:

tuple[Constraints, Series[float]]

mga_compass.core.interpolate(points, point_start, point_end, obj_label, threshold=0.001, solver_name='highs', solver_options=None)[source]
Parameters:
  • points (DataFrame[float])

  • point_start (Series[float])

  • point_end (Series[float])

  • obj_label (str)

  • threshold (float)

  • solver_name (str)

  • solver_options (dict | None)

Return type:

list[Breakpoint]

mga_compass.core.alpha_2_point(alpha, points)[source]
Parameters:
Return type:

Series[float]

mga_compass.core.point_2_alpha(point, points, obj_label, solver_name='highs', solver_options=None)[source]
Parameters:
  • point (Series[float])

  • points (DataFrame[float])

  • obj_label (str)

  • solver_name (str)

  • solver_options (dict | None)

Return type:

Alpha

mga_compass.core.describe_changes(input_constraints, output_constraints)[source]

Compare input and output constraints and return (code, message) tuples for each change.

Parameters:
Return type:

list[ConstraintChange]

mga_compass.core.linear_lower_bounds(points, breakpoints, duals, obj_label, optimum=None)[source]

Lines in beta below the cost along the path of breakpoints.

Point i, with duals mu_i (a subgradient of the cost at it), gives cost(z) >= cost_i + mu_i . (z - z_i); along the path, z is linear in beta, and so is this bound. Only the points with a weight at some breakpoint are used. With optimum, the optimal cost, the constant line cost >= optimum is added too: no design is cheaper. points and duals are indexed by point id, as the breakpoints’ weights are.

Parameters:
Return type:

list[LinearBound]

mga_compass.core.build_outer_approximation(outer_approximation)[source]
Parameters:

outer_approximation (DataFrame)

Return type:

Model

mga_compass.core.navigate_outer_approximation(outer_approximation, input_constraints, obj_label, solver_name='highs', solver_options=None)[source]
Parameters:
  • outer_approximation (Model)

  • input_constraints (Constraints)

  • obj_label (str)

  • solver_name (str)

  • solver_options (dict | None)

Return type:

tuple[Constraints, Series[float]]

mga_compass.core.upper_envelope(bounds)[source]
Parameters:

bounds (list[LinearBound])

Return type:

list[LowerBoundPoint]

Data schemes

exception mga_compass.schemes.ConstraintsIndexError[source]

Bases: ValueError

exception mga_compass.schemes.ConstraintsColumnsError[source]

Bases: ValueError

exception mga_compass.schemes.ConstraintsDirectionError[source]

Bases: ValueError

exception mga_compass.schemes.ConstraintsDeltaError[source]

Bases: ValueError

exception mga_compass.schemes.ConstraintsValueError[source]

Bases: ValueError

class mga_compass.schemes.Constraints(data=None, index=None, columns=None, dtype=None, copy=None)[source]

Bases: DataFrame

Parameters:
  • index (Axes | None)

  • columns (Axes | None)

  • dtype (Dtype | None)

  • copy (bool | None)

copy(deep=True)[source]

Make a copy of this object’s indices and data.

When deep=True (default), a new object will be created with a copy of the calling object’s data and indices. Modifications to the data or indices of the copy will not be reflected in the original object (see notes below).

When deep=False, a new object will be created without copying the calling object’s data or index (only references to the data and index are copied). With Copy-on-Write, changes to the original will not be reflected in the shallow copy (and vice versa). The shallow copy uses a lazy (deferred) copy mechanism that copies the data only when any changes to the original or shallow copy are made, ensuring memory efficiency while maintaining data integrity.

Note

In pandas versions prior to 3.0, the default behavior without Copy-on-Write was different: changes to the original were reflected in the shallow copy (and vice versa). See the Copy-on-Write user guide for more information.

Parameters

deepbool, default True

Make a deep copy, including a copy of the data and the indices. With deep=False neither the indices nor the data are copied.

Returns

Series or DataFrame

Object type matches caller.

See Also

copy.copy : Return a shallow copy of an object. copy.deepcopy : Return a deep copy of an object.

Notes

When deep=True, data is copied but actual Python objects will not be copied recursively, only the reference to the object. This is in contrast to copy.deepcopy in the Standard Library, which recursively copies object data (see examples below).

While Index objects are copied when deep=True, the underlying numpy array is not copied for performance reasons. Since Index is immutable, the underlying data can be safely shared and a copy is not needed.

Since pandas is not thread safe, see the gotchas when copying in a threading environment.

Copy-on-Write protects shallow copies against accidental modifications. This means that any changes to the copied data would make a new copy of the data upon write (and vice versa). Changes made to either the original or copied variable would not be reflected in the counterpart. See Copy_on_Write for more information.

Examples

>>> s = pd.Series([1, 2], index=["a", "b"])
>>> s
a    1
b    2
dtype: int64
>>> s_copy = s.copy(deep=True)
>>> s_copy
a    1
b    2
dtype: int64

Due to Copy-on-Write, shallow copies still protect data modifications. Note shallow does not get modified below.

>>> s = pd.Series([1, 2], index=["a", "b"])
>>> shallow = s.copy(deep=False)
>>> s.iloc[1] = 200
>>> shallow
a    1
b    2
dtype: int64

When the data has object dtype, even a deep copy does not copy the underlying Python objects. Updating a nested data object will be reflected in the deep copy.

>>> s = pd.Series([[1, 2], [3, 4]])
>>> deep = s.copy()
>>> s[0][0] = 10
>>> s
0    [10, 2]
1     [3, 4]
dtype: object
>>> deep
0    [10, 2]
1     [3, 4]
dtype: object
Parameters:

deep (bool)

Return type:

Constraints

changed(other)[source]
Parameters:

other (Constraints)

Return type:

Series

validate(points)[source]
Parameters:

points (DataFrame)

Return type:

None

exception mga_compass.schemes.NegativeAlphaError[source]

Bases: ValueError

exception mga_compass.schemes.AlphaSumError[source]

Bases: ValueError

class mga_compass.schemes.Alpha(data=None, *args, **kwargs)[source]

Bases: Series

class mga_compass.schemes.Breakpoint(beta, alpha, point)[source]

Bases: NamedTuple

Parameters:
beta: float

Alias for field number 0

alpha: Alpha

Alias for field number 1

point: Series[float]

Alias for field number 2

class mga_compass.schemes.LinearBound(slope, intercept)[source]

Bases: NamedTuple

Parameters:
slope: float

Alias for field number 0

intercept: float

Alias for field number 1

class mga_compass.schemes.LowerBoundPoint(beta, value)[source]

Bases: NamedTuple

Parameters:
beta: float

Alias for field number 0

value: float

Alias for field number 1

class mga_compass.schemes.ConstraintChange(code, message, values, dim)[source]

Bases: NamedTuple

Parameters:
code: str

Alias for field number 0

message: str

Alias for field number 1

values: list[float]

Alias for field number 2

dim: str

Alias for field number 3

Near-optimal sets

Navigation and plotting on a set in the near-optimal set format.

Using a near-optimal set (mga_format) for navigation and plots.

The format keeps outputs on disk, one row group per point. A navigated or interpolated point is a mix of only a few sample points (its weights alpha), so plotting it reads just those points’ outputs: memory stays small however large the set is.

mga_compass.near_optimal.navigable_points(nos)[source]

The points as navigation expects them: indexed by point, one column per PoI plus the cost column. All points are used, so the space navigated is the set’s largest slack level.

Parameters:

nos (NearOptimalSet)

Return type:

DataFrame

mga_compass.near_optimal.display_factor(nos, name)[source]

The factor quantity name (a PoI, the cost or an output) is shown with; 1 when the manifest’s display has no entry for it.

Parameters:
Return type:

float

mga_compass.near_optimal.display_units(nos)[source]

The unit each PoI and the cost is shown in: its display unit if it has one, else its own unit.

Parameters:

nos (NearOptimalSet)

Return type:

dict[str, str]

mga_compass.near_optimal.display_points(nos)[source]

navigable_points() with every column multiplied by its display factor. Navigating in these units gives the same weights as in model units.

Parameters:

nos (NearOptimalSet)

Return type:

DataFrame

mga_compass.near_optimal.category_styles(nos)[source]

Display labels and colours of set members, from every set’s optional label and color columns, keyed by the member as a string.

Parameters:

nos (NearOptimalSet)

Return type:

tuple[dict[str, str], dict[str, str]]

mga_compass.near_optimal.mix_output(directory, nos, output, alphas)[source]

Output output mixed with each of the weights alphas: one DataArray per alpha, over the output’s sets without point, in the sets’ member order. Reads only the points with a weight in some alpha.

A set member with no rows at all in these points does not exist for this output (e.g. a storage technology in a generation output) and is left out; a combination missing for some points only counts as zero for them.

Parameters:
Return type:

list[DataArray]

mga_compass.near_optimal.plot_data(directory, nos, plot_name, alphas, display=False)[source]

The data of plot plot_name for each of the weights alphas: {"x_dim": [...], "data": [...]}, one entry of data per alpha. An entry is {category: values over x} for a plot with stack, else the values over x. Sets other than x and stack are summed over. With display, values are multiplied by the output’s display factor.

Parameters:
Return type:

dict

Solvers

mga_compass.solvers.FEASIBILITY_TOLERANCE = 1e-07

Feasibility tolerance of every LP: HiGHS’ own default. Needs data whose dimensions span similar ranges (see the dataset build scripts).

mga_compass.solvers.default_solver_options(solver_name, feasibility_tolerance=1e-07)[source]
Parameters:
  • solver_name (str)

  • feasibility_tolerance (float)

Return type:

dict