Benchmarking Training Methodologies for Dense Neural Networks

Isaac Tonkin; Geoff Harris; Volodymyr Novykov

doi:10.1007/978-3-031-08530-7_59

Benchmarking Training Methodologies for Dense Neural Networks

Isaac Tonkin, Geoff Harris, Volodymyr Novykov

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review

Abstract

Multi-Layer Perceptrons (MLP) trained using Back Propagation (BP) and Extreme Learning Machine (ELM) methodologies on highly non-linear, two-dimensional functions are compared and benchmarked. To ensure validity, identical numbers of trainable parameters were used for each approach. BP training combined with an MLP structure used many hidden layers, while ELM training can only be used on the Single Layer, Feed Forward (SLFF) neural network topology. For the same number of trainable parameters, ELM training was more efficient, using less time to train the network, while also being more effective in terms of the final value of the loss function.

Original language	English
Title of host publication	Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence - 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022, Proceedings
Subtitle of host publication	Proceedings of the 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022
Editors	Hamido Fujita, Philippe Fournier-Viger, Moonis Ali, Yinglin Wang
Publisher	Springer
Pages	707-713
Number of pages	7
ISBN (Electronic)	978-3-031-08530-7
ISBN (Print)	978-3-031-08529-1
DOIs	https://doi.org/10.1007/978-3-031-08530-7_59
Publication status	Published - 30 Aug 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13343 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Access to Document

10.1007/978-3-031-08530-7_59

Cite this

Tonkin, I., Harris, G., & Novykov, V. (2022). Benchmarking Training Methodologies for Dense Neural Networks. In H. Fujita, P. Fournier-Viger, M. Ali, & Y. Wang (Eds.), Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence - 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022, Proceedings: Proceedings of the 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022 (pp. 707-713). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13343 LNAI). Springer. https://doi.org/10.1007/978-3-031-08530-7_59

Tonkin, Isaac ; Harris, Geoff ; Novykov, Volodymyr. / Benchmarking Training Methodologies for Dense Neural Networks. Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence - 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022, Proceedings: Proceedings of the 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022. editor / Hamido Fujita ; Philippe Fournier-Viger ; Moonis Ali ; Yinglin Wang. Springer, 2022. pp. 707-713 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{0789fd9883ec456eb40a452083f6a9b4,

title = "Benchmarking Training Methodologies for Dense Neural Networks",

abstract = "Multi-Layer Perceptrons (MLP) trained using Back Propagation (BP) and Extreme Learning Machine (ELM) methodologies on highly non-linear, two-dimensional functions are compared and benchmarked. To ensure validity, identical numbers of trainable parameters were used for each approach. BP training combined with an MLP structure used many hidden layers, while ELM training can only be used on the Single Layer, Feed Forward (SLFF) neural network topology. For the same number of trainable parameters, ELM training was more efficient, using less time to train the network, while also being more effective in terms of the final value of the loss function.",

author = "Isaac Tonkin and Geoff Harris and Volodymyr Novykov",

note = "Publisher Copyright: {\textcopyright} 2022, Springer Nature Switzerland AG.",

year = "2022",

month = aug,

day = "30",

doi = "10.1007/978-3-031-08530-7_59",

language = "English",

isbn = "978-3-031-08529-1",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "707--713",

editor = "Hamido Fujita and Philippe Fournier-Viger and Moonis Ali and Yinglin Wang",

booktitle = "Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence - 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022, Proceedings",

address = "Germany",

}

Tonkin, I, Harris, G & Novykov, V 2022, Benchmarking Training Methodologies for Dense Neural Networks. in H Fujita, P Fournier-Viger, M Ali & Y Wang (eds), Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence - 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022, Proceedings: Proceedings of the 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13343 LNAI, Springer, pp. 707-713. https://doi.org/10.1007/978-3-031-08530-7_59

Benchmarking Training Methodologies for Dense Neural Networks. / Tonkin, Isaac; Harris, Geoff; Novykov, Volodymyr.
Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence - 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022, Proceedings: Proceedings of the 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022. ed. / Hamido Fujita; Philippe Fournier-Viger; Moonis Ali; Yinglin Wang. Springer, 2022. p. 707-713 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13343 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review

TY - GEN

T1 - Benchmarking Training Methodologies for Dense Neural Networks

AU - Tonkin, Isaac

AU - Harris, Geoff

AU - Novykov, Volodymyr

PY - 2022/8/30

Y1 - 2022/8/30

N2 - Multi-Layer Perceptrons (MLP) trained using Back Propagation (BP) and Extreme Learning Machine (ELM) methodologies on highly non-linear, two-dimensional functions are compared and benchmarked. To ensure validity, identical numbers of trainable parameters were used for each approach. BP training combined with an MLP structure used many hidden layers, while ELM training can only be used on the Single Layer, Feed Forward (SLFF) neural network topology. For the same number of trainable parameters, ELM training was more efficient, using less time to train the network, while also being more effective in terms of the final value of the loss function.

AB - Multi-Layer Perceptrons (MLP) trained using Back Propagation (BP) and Extreme Learning Machine (ELM) methodologies on highly non-linear, two-dimensional functions are compared and benchmarked. To ensure validity, identical numbers of trainable parameters were used for each approach. BP training combined with an MLP structure used many hidden layers, while ELM training can only be used on the Single Layer, Feed Forward (SLFF) neural network topology. For the same number of trainable parameters, ELM training was more efficient, using less time to train the network, while also being more effective in terms of the final value of the loss function.

UR - http://www.scopus.com/inward/record.url?scp=85137984731&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-08530-7_59

DO - 10.1007/978-3-031-08530-7_59

M3 - Conference contribution

SN - 978-3-031-08529-1

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 707

EP - 713

BT - Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence - 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022, Proceedings

A2 - Fujita, Hamido

A2 - Fournier-Viger, Philippe

A2 - Ali, Moonis

A2 - Wang, Yinglin

PB - Springer

ER -

Tonkin I, Harris G, Novykov V. Benchmarking Training Methodologies for Dense Neural Networks. In Fujita H, Fournier-Viger P, Ali M, Wang Y, editors, Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence - 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022, Proceedings: Proceedings of the 35th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2022. Springer. 2022. p. 707-713. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-08530-7_59

Benchmarking Training Methodologies for Dense Neural Networks

Abstract

Publication series

Access to Document

Other files and links

Fingerprint

Cite this