Sub Module: Hyperparameter Tuning

[DiogenesAnalytics]

Problem

Would be good to have some hyperparameter tuning tools available for finding optimal hyperparameters for the different autoencoder variants.

References

• https://neptune.ai/blog/hyperparameter-tuning-in-python-complete-guide
• https://stats.stackexchange.com/questions/407303/the-best-number-of-nodes-in-bottleneck-layer-in-autoencoder
• https://stats.stackexchange.com/questions/243567/is-there-a-minimum-image-size-for-successful-deep-auto-encoders
• https://stackoverflow.com/questions/63003776/how-to-choose-matching-layer-sizes-in-a-convolutional-autoencoder
• https://machinelearningmastery.com/grid-search-hyperparameters-deep-learning-models-python-keras/
• https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)
• https://datascience.stackexchange.com/questions/36741/how-to-make-it-possible-for-a-neural-network-to-tune-its-own-hyper-parameters
• https://datascience.stackexchange.com/questions/116846/hyperparameter-tuning-vs-regularization
• https://stats.stackexchange.com/questions/418525/optimizing-parameters-for-cnn-autoencoder-based-on-training-and-validation-loss
• https://stats.stackexchange.com/questions/235114/parameters-tuning-for-auto-encoders
• https://stackoverflow.com/questions/49823192/autoencoder-gridsearch-hyperparameter-tuning-keras

[DiogenesAnalytics]

Research Papers

PDFs of various research articles related to hyperparameter tuning.

List

• An Empirical Study of the Impact of Hyperparameter Tuning and Model Optimization on the Performance Properties of Deep Neural Networks.pdf
• Hyperparameter Tuning of Artificial Neural Networks for Well Production Estimation Considering the Uncertainty in Initialized Parameters.pdf

[DiogenesAnalytics]

Default Autoencoder Hyperparameters

Here we discuss some tentative solutions to finding the default values for more complex autencoders.

Problem

How to determine what the "default" model parameters (layer size, kernel size, etc ...) should be.

Solution 1: Perform Grid Search

Can use Grid Search or something similar to search for the optimal model hyperparameters. Can literally write up a Jupyter Notebook to demo/run the tuning code.

References

• https://keras.io/keras_tuner/