Blending Neural Control Density Functions for Stabilization and Safety

TL;DR

Lyapunov functions are the standard certificate for neural-network control, but they cannot certify stability when a system has multiple equilibria, and controller/certificate pairs generally can't be combined. We use density functions — a dual certificate to Lyapunov functions — to fix both problems. We introduce Neural Control Density Functions (NCDFs), a novel exponential parameterization that provably satisfies the integrability condition density certificates require, prove that blending several NCDF controllers yields a region of attraction containing the union of the constituent regions of attraction, and extend the framework to jointly certify safety via control barrier functions.

Motivation

Classical techniques in control synthesis, such as sum-of-squares programming, linear quadratic regulators, and semidefinite programming, are quite expensive and do not scale well to high-dimensional systems. Recent advances in deep learning have shown that Lyapunov-based methods can be used to learn stabilizing controllers for high-dimensional systems. However, Lyapunov-based methods do not work well in the presence of multiple equilibria in the system. Furthermore, combining two Lyapunov-based controllers does not guarantee the stability of the combined controller — instead, a new Lyapunov function must be learned for the combined controller. In this work, we propose the use of density functions as a dual alternative to Lyapunov functions. Density functions can be used to certify the stability of systems with multiple equilibria, and they also allow multiple controllers to be combined while still guaranteeing the stability of the combined controller.

Density Functions — A Dual Alternative

We utilize the idea of density function to address the challenges of Lyapunov function.

Stability via Density Function (Rantzer, 2001)

A system is a.e. stable if there exists $\rho$ such that:

$\rho(x) > 0$ for all $x$
$\int_{\|x\| \ge 1} \frac{\rho(x)\|f(x)+g(x)u(x)\|}{\|x\|} < \infty$
$\nabla \cdot \big[\rho(x)(f(x)+g(x)u(x))\big] > 0$ for almost all $x$

Stability via Lyapunov Function

A system is stable if there exists $V$ such that:

$V(x) > 0$ for all $x \ne 0$
$V(0) = 0$
$\nabla V(x)^\top \big[f(x)+g(x)u(x)\big] < 0$ for all $x$

3D surface plots comparing a density function and a Lyapunov function certifying stability of the inverted pendulum, next to a phase portrait of the system. — The idea is density of particles should increase as you are near the point of stability compared to the lyapunov function where energy decrease as you are near the point of stability.

As a result of the above Certification:

Density functions can certify stability of systems with multiple equilibria as a consequence of almost everywhere stability.
Density functions allow for blending multiple controllers while still guaranteeing stability of the combined controller.
The divergence criterion is convex in the pair $(\rho, u \rho)$ enabling the search of pair verifying the inequality.

Neural Control Density Functions (NCDFs)

The density function $\rho$ must satisfy the integrability condition. Note that the divergence condition associated with density function is convex in the pair $(\rho, u \rho)$. Thus, we set $\psi(x) := \rho(x)u(x)$ and search over the pair $(\rho, \psi)$ where the control can always be recovered as $u=\psi/ \rho$. We propose a novel exponential parameterization that provably satisfies this condition:

$$\rho(x) = \dfrac{a(x)}{\exp\!\left(\|x\|^2 + \|b(x)\|^2\right)} \qquad \psi(x) = \dfrac{c(x)}{\exp\!\left(\|x\|^2 + \|b(x)\|^2\right)}$$

The search over $(\rho,\psi)$ reduces to learning $(a, b, c)$, each parameterized using a neural network. We train with a counterexample-guided (CEGIS) loop: an SMT falsifier (dReal) searches for states that violate the density or divergence conditions, and every counterexample found is added back to the training set, similar to prior work like Neural Lyapunov Control.

$$\mathcal{L}(\theta) = \mathbb{E}\big[\max(0, -\rho_\theta(x)) + \max\!\big(0, -\nabla\cdot[\rho_\theta f + g\psi_\theta](x)\big)\big]$$

Inverted pendulum stabilizing under the learned NCDF controller.

Expanding Regions of Attraction by Blending

Because the stability condition is linear in $\rho$, several density/controller pairs $\{\rho_i, u_i\}_{i=1}^N$ — each individually stabilizing — can be combined into one blended controller:

$$\rho^\star(x) = \sum_{i=1}^{N} \rho_i(x), \qquad u^\star(x) = \sum_{i=1}^{N} \frac{\rho_i(x)\,u_i(x)}{\rho^\star(x)}$$

The resultant blended controller $u^\star$ is guaranteed to be stabilizing and the certificate for the blended controller is simply the sum of the individual certificates. Let $D_i \subset Q_i$ be the sublevel set of the density function $\rho_i$ and $Q_i$ be the region of attraction corresponding to the $i^{th}$ controller, and let $D^\star \subset Q^\star$ be the sublevel set of the blended density function $\rho^\star$. Then, we can show that:

$$\bigcup_i D_i \;\subseteq\; D^\star \subseteq Q^\star$$

In other words, blending controllers with density functions can only help — the blended system's certified RoA contains the union of every constituent RoA.

Regions of attraction from individual NCDFs vs. the blended (mixed) NCDF vs. Yang et al. (2024)'s Lyapunov Control, on the inverted pendulum and path-tracking tasks.

Density Functions for Safety

A natural question to ask is whether density functions can be used to certify safety as well. We show that the answer is yes, and we can jointly certify stability and safety by combining density functions with control barrier functions (CBFs). We say for a system $\dot{x} = f(x) + g(x)u$ with a safe set $\mathcal{C} = \{x : h(x) \ge 0\}$, the system is safe if there exists a control $u$ such that:

$$ L_f h(x) + L_g h(x) u + \alpha(h(x)) \ge 0, \quad \forall x \in \mathcal{C}$$ $$ \rho(x) \big(L_f h(x) + \alpha(h(x))\big) + \psi(x)L_g h(x) \ge 0, \quad \forall x \in \mathcal{C}$$

The second condition above is a density-based CBF condition that guarantees safety of the system. We can then train a safe-stabilizing NCDF controller by adding the density-based CBF condition to the loss function:

$$\mathcal{L}_{\text{safe}}(\theta) = \mathcal{L}(\theta) \;+\; \mathbb{E}\Big[\max\big(0,\, -\big[\rho(x)\big(L_f h(x)+\alpha(h(x))\big) + \psi(x) L_g h(x)\big]\big)\Big]$$

Particles avoiding unsafe set and heading towards point of stability.

Particles avoiding multiple unsafe set and heading towards point of stability.

References

A. Rantzer. A dual to Lyapunov's stability theorem. Systems & Control Letters, 42(3):161–168, 2001.
A. Rantzer and F. Ceragioli. Smooth blending of nonlinear controllers using density functions. In 2001 European Control Conference (ECC), pp. 2851–2853. IEEE, 2001.
Y.-C. Chang, N. Roohi, and S. Gao. Neural Lyapunov control. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019.
A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada. Control barrier functions: Theory and applications. In 2019 18th European Control Conference (ECC), pp. 3420–3431. IEEE, 2019.
L. Yang, H. Dai, Z. Shi, C.-J. Hsieh, R. Tedrake, and H. Zhang. Lyapunov-stable neural control for state and output feedback: A novel formulation. In International Conference on Machine Learning (ICML), pp. 56033–56046. PMLR, 2024.

Bibtex

  @inproceedings{
      chaudhary2026blending,
      title={Blending Neural Control Density Functions for Stabilization and Safety},
      author={Sahil Chaudhary and Chaitanya Murti and Chiranjib Bhattacharyya},
      booktitle={Forty-third International Conference on Machine Learning},
      year={2026},
      url={https://openreview.net/forum?id=IxBsOvMFXc}
      }