DOI: 10.1016/j.earscirev.2019.103076
论文题名: Impact of deep learning-based dropout on shallow neural networks applied to stream temperature modelling
作者: Piotrowski A.P. ; Napiorkowski J.J. ; Piotrowska A.E.
刊名: Earth Science Reviews
ISSN: 00128252
出版年: 2020
卷: 201 语种: 英语
中文关键词: Atmosphere-hydrosphere interactions
; Deep learning
; Dropout
; Shallow artificial neural networks
; Stream temperature modelling
英文关键词: artificial neural network
; atmosphere-hydrosphere interaction
; machine learning
; regression analysis
; river discharge
; streamwater
; temperature gradient
英文摘要: Although deep learning applicability in various fields of earth sciences is rapidly increasing, shallow multilayer-perceptron neural networks remain widely used for regression problems. Despite many clear distinctions between deep and shallow neural networks, some techniques developed for deep learning may help improve shallow models. Dropout, a simple approach to avoid overfitting by randomly skipping some nodes in a net during each training iteration, is among methodological features that made deep learning networks successful. In this study we give a review of dropout methods and empirically show that, when used together with early-stopping, dropout and its variant dropconnect could improve performance of shallow multi-layer perceptron neural networks. Shallow neural networks are applied to streamwater temperature modelling problems in six catchments, based on air temperature, river discharge and declination of the Sun. We found that when training of a particular neural network architecture that includes at least a few hidden nodes is repeated many times, dropout reduces the number of models that perform poorly on testing data, and hence improves the mean performance. If the number of inputs or hidden nodes is very low, dropout only disturbs training. However, nodes need to be dropped out with a much lower probability than in the case of deep neural networks (about 1%, instead of 10–50% for deep learning), due to a much smaller number of nodes in the network. Larger probabilities of dropping out nodes hinder convergence of the training algorithm and lead to poor results for both calibration and testing data. Dropconnect turned out to be slightly more effective than dropout. © 2019 The Authors Citation statistics:
资源类型: 期刊论文
标识符: http://119.78.100.158/handle/2HF3EXSE/166218
Appears in Collections: 气候变化与战略
There are no files associated with this item.
作者单位: Institute of Geophysics, Polish Academy of Sciences, Ks. Janusza 64, Warsaw, 01-452, Poland; Faculty of Polish Studies, University of Warsaw, Krakowskie Przedmiescie 26/28, Warsaw, 00-927, Poland
Recommended Citation:
Piotrowski A.P.,Napiorkowski J.J.,Piotrowska A.E.. Impact of deep learning-based dropout on shallow neural networks applied to stream temperature modelling[J]. Earth Science Reviews,2020-01-01,201