Graph neural networks (GNNs) are powerful models for many graph-structured tasks. Existing models often assume that a complete structure of a graph is available during training. However, graph-structured data is often formed in a streaming fashion so that learning a graph continuously is often necessary.
Existing lifelong learning techniques are mostly designed for convolutional neural networks (CNNs), which assumes the new data samples are independent. However, in lifelong graph learning, nodes are connected and dynamically added. In this work, we have an important observation:
- The number of nodes increases dynamically, while the number of node features is stable.
Therefore, we introduce feature graph, which takes features as nodes and turns nodes into independent graphs. This successfully converts the original problem of node classification to graph classification, in which the increasing nodes are turned into independent training samples.
The feature adjacency is established via feature cross-correlation between and its neighbors
to model feature “interaction.” This makes the lifelong learning techniques for CNN applicable to GNN, as the new nodes in a regular graph become individual training samples.
Applications
1. Feature Matching
Image feature matching is crucial for many 3-D computer vision tasks including simultaneous localization and mapping (SLAM). As shown below, the interest point and their descriptors form an infinite temporal growing graph, in which the feature points are nodes and their descriptors are the node features. In this way, the problem of feature matching becomes edge prediction for a temporal growing graph.
In an environment with dynamic illumination, feature graph network can also achieve much better accuracy and robustness, which demonstrate its effectiveness.
2. Distributed Human Action Recognition with Wearable Devices
Five sensors, each of which consists of a triaxial accelerometer and a biaxial gyroscope, are located at the left and right forearms, waist, left and right ankles, respectively. Each sensor produces 5 data streams and totally 5 × 5 data streams is available. 13 daily action categories are considered, including rest at standing (ReSt), rest at sitting (ReSi), rest at lying (ReLi), walk forward (WaFo), walk forward left-circle (WaLe), walk forward right-circle (WaRi), turn left (TuLe), turn right (TuRi), go upstairs (Up), go downstairs (Down), jog (Jog), jump (Jump), and push wheelchair (Push). Therefore, action recognition is a problem of sub-graph classification.
Our feature graph network (FGN) has a much higher and stable performance than all the other methods, including GCN, APPNP, and GAT. It also achieves a much higher final per-class precision in nearly all the categories.
- FGN is the first method to bridge graph learning to lifelong learning via a novel graph topology.
Video
Source Code
Publications
-
Lifelong Graph Learning.IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
