Lifelong Graph Learning

Published: by

Graph neural networks (GNNs) are powerful models for many graph-structured tasks. Existing models often assume that a complete structure of a graph is available during training. However, graph-structured data is often formed in a streaming fashion so that learning a graph continuously is often necessary.

A temporally growing graph, which is challenging to learn the graph in a sequential way.

Existing lifelong learning techniques are mostly designed for convolutional neural networks (CNNs), which assumes the new data samples are independent. However, in lifelong graph learning, nodes are connected and dynamically added. In this work, we have an important observation:

  • The number of nodes increases dynamically, while the number of node features is stable.

Therefore, we introduce feature graph, which takes features as nodes and turns nodes into independent graphs. This successfully converts the original problem of node classification to graph classification, in which the increasing nodes are turned into independent training samples.

Feature graph takes the features as nodes and turns nodes into graphs, resulting in a graph predictor instead of the node predictor. Take the node a with label in the regular graph as an example, its features are nodes in feature graph .

The feature adjacency is established via feature cross-correlation between and its neighbors to model feature “interaction.” This makes the lifelong learning techniques for CNN applicable to GNN, as the new nodes in a regular graph become individual training samples.

The relationship of a graph and feature graph.

Applications

1. Feature Matching

Image feature matching is crucial for many 3-D computer vision tasks including simultaneous localization and mapping (SLAM). As shown below, the interest point and their descriptors form an infinite temporal growing graph, in which the feature points are nodes and their descriptors are the node features. In this way, the problem of feature matching becomes edge prediction for a temporal growing graph.

Feature matching is a problem of edge prediction for temporal growing graph.

In an environment with dynamic illumination, feature graph network can also achieve much better accuracy and robustness, which demonstrate its effectiveness.

Feature fatching in an environment with dynamic illumination.

2. Distributed Human Action Recognition with Wearable Devices

Five sensors, each of which consists of a triaxial accelerometer and a biaxial gyroscope, are located at the left and right forearms, waist, left and right ankles, respectively. Each sensor produces 5 data streams and totally 5 × 5 data streams is available. 13 daily action categories are considered, including rest at standing (ReSt), rest at sitting (ReSi), rest at lying (ReLi), walk forward (WaFo), walk forward left-circle (WaLe), walk forward right-circle (WaRi), turn left (TuLe), turn right (TuRi), go upstairs (Up), go downstairs (Down), jog (Jog), jump (Jump), and push wheelchair (Push). Therefore, action recognition is a problem of sub-graph classification.

Lifelong human action recognition is a problem of lifelong sub-graph classification.

Our feature graph network (FGN) has a much higher and stable performance than all the other methods, including GCN, APPNP, and GAT. It also achieves a much higher final per-class precision in nearly all the categories.

  • FGN is the first method to bridge graph learning to lifelong learning via a novel graph topology.

Video

Source Code

Publications

  1. Lifelong Graph Learning.
    Chen Wang, Yuheng Qiu, Dasong Gao, Sebastian Scherer.
    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
    Selected as Oral Presentation (4.2%)