Nearest vertex attraction for actively reducing loss

In recent years the neural networks have received considerable success on classification tasks. The randomly initialized network is generally expected to adjust the weights for reducing the loss, which is also indicated as the total distance from the samples to the corresponding vertices. However, the output space of the conventional neural networks suffers from the fixed relation between the labels and vertices during learning. This case forces the mapped points around the unexpected vertex across the decision boundary into the neighborhood of the correct vertex, and simultaneously the boundary points cannot obtain the substantial rectification. Therefore, this study proposes a novel nearest vertex attraction (NVA) to actively adjust the relation between the categories and the output vertices for improving the neural network classifiers. The best relation allows that the data points can be attracted by the nearest vertices to minimize the total moving distances. In this way, the mapped points that are near to the vertices and the decision boundaries obtain the decent management. We evaluated the NVA with several conventional classification techniques and other neural network classifiers on 12 public UCI datasets. The numerical experiments demonstrate that the proposed method improves performance of the neural network classifiers on the involved benchmarks.