An approach to apply deep learning on insurance mortality data
This is the product of about 80 hours of work to implement an idea for neural nets on ILEC mortality data.
It makes use of
- Use of categorical embeddings to encode categories
- For one model, a neural network architecture to take input embeddings along with issue age and duration and predict mortality.
- For the other model, a neural network architecture which takes the input embeddings and uses this to control a spline basis for issue age and duration.
The first model did not perform as well as the second. The nearly piecewise linear nature of the neural network prediction surface could not cope with the curves in the data, especially at the early durations.
The second model performed much better, as the neural network did not need to learn curves, just how to translate the learned categorical embeddings into controlling the splines themselves.
- The table surfaces can get jagged when claims are especially sparse. This requires that some sort of regularization be implemented. One way is the way regularization is implemented in mgcv::gam. See Wood's GAM book for more information. In terms of implementation, in a GAM, the square root of the penalty matrix times the penalty lambda is appended to the model matrix, 0s are appended to the response vector, and the problem is treated as a linear model. The same could be done here, although it would require appending almost 10000 copies of the penalty matrix and managing the optimization of the lambda parameter.
- The embedding spaces for both splines and intercept need to be explored more carefully. While fuzzy, there is structure here.
- Any single embedding may carry only a modest amount of claims, and the claim distribution may be quite sparse. The reason one gets a smooth table is that information is being borrowed from cells that are nearby in the embedding space. If two cells are close in the embedding space, that suggests they are similar in some way. It would be interesting if someone built a browser for the embedding space to explore the characteristics of nearby cells for a given cell. In some sense, this is a soft grouping of data.
- Is the architecture of the models adequate, optimal, and so on? For example, I used an embedding dimension of 9 for the categories. Is this too much? Too little? The same goes for the breadth and depth of the neural networks.
- Would any part of the model architectural benefit from other layer types, such as normalization, drop out, and so on?
There are two notebooks.
- PoissonGLMTests.ipynb: implements Poisson GLMs in PyTorch with tests and comparisons of performance and outputs
- PoissonNN.ipynb: implements the neural network models on the ILEC data
I did all of this on machines with an RTX 4090 with 24GB VRAM. I had begun this using a Ryzen 9 3950x with 128GB memory. In early December, I upgraded to a Core Ultra 265K with 96GB memory. I ran everything on a Docker instance on top of Windows. The Dockerfile is included in ./Dockerfile. It contains a lot of things that aren't needed for this, since this is my generic analytics workspace.
The second spline-based model takes about
Python resources required for setup are in setup.py.
I also called out to R at times, and I used the data.table package. R will also require the IRkernel package to interface with Jupyter notebooks.
I opted not to include a requirements.txt or a setup.py. The reason for this is the peculiarities of using the nVidia Docker container. Instead, I have put the packages in the earliest cells of each notebook. This can be changed later if there is sufficient interest to do so.
I mentioned Simon Prince's book, Understanding Deep Learning, which has accessible and mathematically driven explanations for deep learning and other AI model types. Highly recommended.
I also mentioned some references on topological data analysis in the neural nets notebook.
There is also a book by Mario Wuttrich and Michael Merz, (Statistical Foundations of Actuarial Learning and its Applications )[https://link.springer.com/book/10.1007/978-3-031-12409-9]. It is heavy with graduate-level and later mathematics and statistical content. Not an introductory text. However, it does discuss the use of embeddings and neural networks, among other model types. All of the book is devoted to actuarial applications.