Learning Invariant Sets from Failure

Safety-critical control systems require the state to remain within a prescribed set of admissible states at all times. We seek the invariant set and the control inputs that ensure invariance when the system dynamics are unknown.

Preprint

The control problem

Consider the discrete-time controlled nonlinear time-invariant dynamical system

Here, x(k) ∈ ℝ^n_x is the state and u(k) ∈ ℝ^n_u is the control input. The mapping f : ℝ^n_x × ℝ^n_u → ℝ^n_x is Lipschitz continuous with f(0, 0) = 0. The state is constrained to a compact set X containing the origin in its interior, and the input is constrained to a compact set U(x), with 0 ∈ U(0).

We wish to choose inputs that achieve a control objective while satisfying these constraints indefinitely. In model predictive control (MPC), we minimize a cost over a finite prediction horizon, apply the first optimal input, and solve again at the next observed state. The question is whether repeatedly solving a finite-horizon problem is enough to keep the system safe.

An illustrative application: drone motion

For simplicity, we illustrate this problem using the constrained double integrator from our numerical example. We interpret x₁ as displacement from a reference height, x₂ as vertical velocity, and u as the input:

The drone is a schematic visualization of this discrete-time model. We use it to examine complete trajectories under MPC, first with exact dynamics and then with model mismatch. The invariant-set definitions that follow apply to the general nonlinear system (1).

From failing trajectories to invariant constraints

We choose a quadratic cost to regulate the example system to the origin. Play each case to follow the drone, its position over time, and its state-space trajectory.

The first and third cases continue until a state constraint is violated. The second and fourth show the corresponding remedies. All four start from the same state with the same cost, horizon, and physical constraints.

With exact dynamics and an initially feasible problem, an invariant terminal set provides an admissible continuation at every step.^[1] Under model mismatch, we must also account for uncertainty in the transitions. Robust MPC imposes conditions that hold for a specified uncertainty set.^[2]

The cost of requiring safety for every possible model

The robust remedy preserves safety, but it can exclude states that are invariant for the actual dynamics. The state (0, 9) lies in the true MCI and outside the robust set: under α = 1, braking keeps the drone within constraints, whereas under α = 1.2, even maximum braking reaches x₁ = 15.6.

Same invariant state, different inputs

Must we discard the state (0, 9)? For the actual dynamics, the answer is no. The state admits an input that preserves safety, even though another admissible input leads to failure.

The MCI set only characterizes where the system can remain safe but not which inputs at each state ensure invariance. In this example, we can retain the initial state if we restrict the first input to [−5, −3.5], rather than excluding the state itself.

With known dynamics, we can check which admissible inputs keep the next state in the MCI. With an approximate model, that check can be wrong, as the MPC example shows. We therefore seek both the invariant set and the control inputs that ensure invariance at each state, using observed transitions in place of a known dynamics model.

To represent this information, we must retain the association between each state and its admissible invariance-preserving inputs. We first state the definitions in state space, then extend the representation to include the input.

Control invariance

Consider system (1) with state and input constraints X and U(x) for all x ∈ ℝ^n_x. A set C ⊆ X is called control invariant if, for every x ∈ C, there exists u ∈ U(x) such that f(x, u) ∈ C.

A set X_∞ ⊆ X is called the maximal control invariant set (MCI) if

1. X_∞ is control invariant, and
2. if C ⊆ X is any control invariant set, then C ⊆ X_∞.

A set C ⊆ X is safe if it is a control invariant set with respect to (1) and U(x) for all x ∈ C.

The MCI set can be computed via the one-step predecessor operator. For any set Ω ⊆ ℝ^n_x, define

as the set of all states from which there exists an admissible input that steers the system into Ω in one step. The MCI set can be obtained as the fixed point of the recursive iteration

Invariance-preserving inputs

The MCI set only characterizes where the system can remain safe but not which inputs at each state ensure invariance. The existential quantifier in (2) discards all control information during the recursion.

We now define the invariance-preserving input set.

Consider system (1) and a set C ⊆ ℝ^n_x. The invariance-preserving input set of C under dynamics f is the set-valued map U^f_C : ℝ^n_x → 2^ℝn_u defined by

For convenience, we write U^f_∞(x) ≜ U^f_X∞(x).

Given an initial state x(0) ∈ X_∞, a controller that selects u(k) from U^f_∞(x(k)) ∩ U(x(k)) for all k ∈ ℤ₊, ensures that x(k) remains in X_∞.

In the next section, we show that both X_∞ and U^f_∞(x) ∩ U(x) can be obtained simultaneously by extending the computation of invariant sets to the joint state-control space.

The joint state-control space

We now return to the general nonlinear system (1). The definitions and invariant-set characterization below apply to this system. We continue to use the double integrator to visualize the sets.

Let z = [x^⊤ u^⊤]^⊤ ∈ ℝ^{n_x + n_u} denote a state-control vector, and let Z ⊆ ℝ^{n_x + n_u} be a set in the joint state-control space. We now define the two operations that allow us to decompose Z into the state and control spaces, respectively.

The state projection of Z is the mapping Π_x : 2^{ℝn_x + n_u} → 2^ℝn_x and, for a given x ∈ Π_x(Z), the x-section of Z is the mapping U_Z : ℝ^n_x → 2^ℝn_u, given by

Note that we do not use the control projection Π_u(Z) since it would yield all control inputs in Z irrespective of the state. The x-section U_Z(x) preserves the coupling between states and controls in Z. Using these operations and given the state and input constraint sets X and U(x) for all x ∈ X, we construct the joint constraint set in the state-control space

We now define the notion of state-control invariant sets.

Consider system (1) and the joint constraint set Z given above. A set C ⊆ Z is called state-control invariant if, for every z ∈ C, there exists u⁺ ∈ ℝ^n_u such that [f(x, u)^⊤ (u⁺)^⊤]^⊤ ∈ C.

A set Z_∞ ⊆ Z is called the maximal state-control invariant set if

1. Z_∞ is state-control invariant, and
2. if C ⊆ Z is any state-control invariant set, then C ⊆ Z_∞.

Similarly to the state-space case, Z_∞ can be computed using a predecessor operator in the joint state-control space. For any set Ω ⊆ ℝ^{n_x + n_u}, define

that is, the set of all state-control vectors for which the next state f(x, u) has a non-empty x-section, i.e., U_Ω(f(x, u)) ≠ ∅. The MSCI is then obtained as the fixed point of the recursive iteration

At each step, Ω_k+1 retains only those state-control vectors in Ω_k for which an input exists that, when paired with the next state, lies within Ω_k. State-control vectors that lack such an input are discarded.

The MSCI Z_∞ admits the following properties:

MSCI and state projection

From failures to halfspaces

We now specialize to deterministic LTI systems, f(x, u) = Ax + Bu, with polytopic constraints, and develop the failure-aware iterative learning (FAIL) algorithm.

At iteration ℓ, we write the current projected set as X_ℓ−1 = {x : H_proj^ℓ−1x ≤ g_proj^ℓ−1}. For linear dynamics, the predecessor is

The jth constraint of the projected set induces a predecessor halfspace with normal

Examining which constraints defined by X_ℓ−1 were violated allows us to identify which predecessor constraints were violated and their corresponding (g_proj^ℓ−1)_j without knowing the dynamics.

A learning iteration can recover several halfspaces from the same failing trajectory. After each update, we check the stored trajectory against the updated set and continue learning until no new failures are identified. Below, we examine the regression for one halfspace at a time.

Next, we learn a_j^★ without knowing the dynamics. The z-space normal a_j^★ satisfies

for every time step t along any trajectory of (1). The left-hand side involves the unknown a_j^★, while the right-hand side is computed using (H_proj^ℓ−1)_j and x(t+1). Hence, learning a_j^★ is a linear regression with z(t) being the regressor in ℝ^{n_x + n_u}, a_j^★ being the parameter to estimate, and (H_proj^ℓ−1)_jx(t+1) being a measurement.

Let p ≜ n_x + n_u. To learn a_j^★, we require p linearly independent state-control samples. We form a window of p samples from available trajectory data

where t₁, …, t_p are time indices drawn from the failing trajectory or from other available trajectories. For these deterministic LTI dynamics, s = Z a_j^★ holds. If rank(Z) = p, the estimate of the normal vector â_j is given by

Fit the halfspace yourself

Adjust the three components of â_j to fit the observed transitions. The normal sets the orientation of the boundary; the offset translates it. The plots show which pairs your halfspace retains and removes.

Each regression identifies p coefficients for one violated facet. Identifying (A, B) would require n_xp coefficients before computing the predecessor sets.

The algorithm

Using the state projection and x-section of P_ℓ, the state and input constraints at iteration ℓ are

The x-section U_ℓ(x) is the algorithm's current estimate of the admissible invariance-preserving inputs at state x ∈ X_ℓ.

At iteration ℓ, a one-step failing state-input vector z^ℓ = [(x^ℓ)^⊤ (u^ℓ)^⊤]^⊤, with x^ℓ ∈ X_ℓ−1 and u^ℓ ∈ U_ℓ−1(x^ℓ), satisfies

Our goal is to exclude such states and inputs from the learned polytope. For each recovered halfspace, we apply the update

Each recovered halfspace contains Z_∞ and excludes the observed failing pair. Thus, Z_∞ ⊆ P_ℓ ⊆ P_ℓ−1: the iterates are decreasing outer approximations of the MSCI.

A learning iteration can generate more than one halfspace. The visualization advances through individual halfspace updates: trajectory 3 produces two halfspaces, and the final failing trajectory, trajectory 6, produces three. Trajectory 4 remains within the current set and produces none.

What the sections are for

At convergence, the section U_L(x) contains exactly the admissible invariance-preserving inputs. Starting in the learned state projection and choosing an input from this section at every step keeps the trajectory within constraints.

Numerical results

We now validate FAIL using constant-input and random admissible controllers to generate failing trajectories. The interactive figures simulate the same dynamics and compute their displayed values from the double-integrator dynamics.

We validate FAIL on the double-integrator example, with n_x = 2, n_u = 1, and p = 3. The bounds |x₁| ≤ 15, |x₂| ≤ 10, and |u| ≤ 5 define an initial polytope P₀ with six constraints. FAIL does not know (A, B); we use these matrices only to simulate the system and compute the reference invariant sets.

Generating failing trajectories

We first apply the constant-input controllers u(k) ≡ −5 and u(k) ≡ 5 from x₀ = [0, 0]^⊤. We then sample initial states from Π_x(P_ℓ) and inputs uniformly from U_ℓ(x(k)). Each trajectory runs for up to T = 15 steps and terminates when the state first leaves the current projected set.

Learning the MSCI

From six failing trajectories, we learn eight predecessor halfspaces in eight iterations. Together with the six initial constraints, these recover all 14 halfspaces of Z_∞. At every iteration, we verify Z_∞ ⊆ P_ℓ and that the learned halfspace matches an MSCI constraint within a numerical tolerance of 10⁻³. At termination, P₈ and Z_∞ have Hausdorff distance zero, as do Π_x(P₈) and X_∞.

We ran these experiments on a laptop with an Intel i9-11980HK CPU and an NVIDIA RTX 3080 GPU. The times above are from this experiment, not timings of the interactive figures.

At each iteration, we test up to 1,200 trajectories using the random admissible controller. At L = 8, none of the 1,200 trajectories leaves Π_x(P₈). The learned sections satisfy U₈(x) = U^f_∞(x) ∩ U(x), giving the admissible inputs that preserve invariance.

Learned sets at iteration 8

View the 1,200 non-failing trajectories

We generate 1,200 trajectories using the random admissible controller at L = 8. None leaves Πₓ(P₈). We also compare P₈ directly with Z∞ and obtain a Hausdorff distance of zero.

What remains

The FAIL recovery guarantees rely on deterministic linear dynamics, exact measurements, permissible failures, and sufficiently informative data. The intermediate sets are outer approximations and may still admit failures. Extending the recovery and containment guarantees to noisy systems is future work. In higher dimensions, computing the state projection remains a computational bottleneck.

Contributors

Ahmad Amine, Nick-Marios T. Kokolakis, Ugo Rosolia, Truong X. Nghiem, and Rahul Mangharam.

Citation

BibTeX

@misc{amine2026failure,
  title={Failure-Aware Iterative Learning of State-Control Invariant Sets},
  author={Amine, Ahmad and Kokolakis, Nick-Marios T. and Rosolia, Ugo and Nghiem, Truong X. and Mangharam, Rahul},
  year={2026},
  eprint={2604.06776},
  archivePrefix={arXiv},
  primaryClass={eess.SY},
  doi={10.48550/arXiv.2604.06776}
}

MLA

Amine, Ahmad, et al. “Failure-Aware Iterative Learning of State-Control Invariant Sets.” arXiv preprint arXiv:2604.06776, 2026.

Preprint