Deep learning meets extreme values: The Neural Hill estimator

Abstract

We introduce a novel neural Hill estimator that advances tail index estimation by leveraging deep learning to automate the selection of the crucial threshold. This method integrates extreme value theory with neural networks to intelligently identify the optimal number of upper-order statistics, k, by directly analyzing key features extracted from the observed data. Under standard second-order regular variation conditions, we establish the estimator’s consistency and asymptotic normality, thereby providing robust theoretical guarantees for its effectiveness. Our comprehensive simulation study demonstrates that the estimator performs well across diverse scenarios involving extreme values and varying tail thickness. The results show substantial improvements in accuracy and stability compared with traditional methods. We also illustrate the practical utility of this approach through two real-world examples: an analysis of three major European stock indices and the modeling of large insurance claims. These examples confirm that the method performs reliably in challenging practical contexts while remaining computationally efficient.

Keywords

• Heavy-tailed distributions
• Hill estimator
• robust estimation
• tail index estimation
• threshold selection

References

1. [1] S. Achar, Neural-Hill: A novel algorithm for efficient scheduling of IoT-Cloud resources to maintain scalability, IEEE Access 11, 26502–26511, 2023.