GLL: A Differentiable Graph Learning Layer for Neural Networks

Wed, 11 Dec 2024 00:00:00 +0000

The graph learning layer (GLL) replaces the usual projection head and softmax classifier with graph-Laplacian label propagation. Samples in a minibatch become nodes of a similarity graph, allowing predictions to use relationships among examples instead of classifying each representation independently.

The main technical contribution is an exact end-to-end differentiation rule for a broad family of graph learning methods. By deriving the backward pass with the adjoint method, the graph solver can participate directly in neural-network training rather than being detached from the feature extractor or differentiated through a heuristic approximation.

Experiments show smoother transitions between classes, improved optimization and generalization, and stronger resistance to adversarial perturbations than a conventional softmax head. The framework supports both supervised and semi-supervised learning and is released with an open-source implementation.

Graph Neural Networks | Bohan Chen's Personal Webpage

GLL: A Differentiable Graph Learning Layer for Neural Networks