fully connected readout layer

In a fully connected readout layer we define

Where

• $C\in {\mathbb{R}}^{d\times n{d}_L}$
• $\text{vec}(x_{\ell })\in {\mathbb{R}}^{n{d}_L}$
  • in general $\text{vec}(\cdot )$ vectorizes ${\mathbb{R}}^{m\times n}$ matrices into ${\mathbb{R}}^{mn}$ vectors
• $\rho$ can be the identity or some other pointwise nonlinearity (ReLU, softmax, etc)

There are some downsides of a fully connected readout layer

• The number of parameters depends on $n$ - adding $n{d}_{L}d$ learning parameters, which grows with the graph size. This is not amenable to groups of large graphs
• No longer permutation invariant because of the $\text{vec}(\cdot )$ operation

• No longer transferrable across graphs.
  • unlike in GNNs, $C$ depends on $n$. So if the number of nodes $n$ changes, we have to relearn $C$

These make this a not-so-attractive option, so we usually use an aggregation readout layer