# Welcome to EpyNN’s documentation!

**EpyNN is written in pure Python/NumPy.**

If you use EpyNN in academia, please cite:

Malard F., Danner L., Rouzies E., Meyer J. G., Lescop E., Olivier-Van Stichelen S. **EpyNN: Educational python for Neural Networks**, 2021, *Submitted*.

Please email to fmalard@epynn.net or solivier@mcw.edu for any comments.

## What is EpyNN?

EpyNN is a production-ready but first **Educational python resource for Neural Networks**. EpyNN is designed for Supervised Machine Learning (SML) approaches by means of Neural Networks.

EpyNN includes **scalable**, **minimalistic** and **homogeneous** implementations of major Neural Network architectures in **pure Python/Numpy**. Because EpyNN is meant to be **educational**, it contains several dummy and real world examples of problems including data processing schemes and pipelines for training and prediction.

## Do I need EpyNN?

EpyNN is intended for **teachers**, **students**, **scientists**, or more generally anyone with minimal skills in Python programming **who wish to understand** and build from basic implementations of NN architectures.

Although EpyNN can be used for production, it is meant to be a library of **homogeneous architecture templates** and **practical examples** which is expected to save an important amount of time for people who wish to learn, teach or **develop from scratch**.

## Is EpyNN reliable?

EpyNN has been cross-validated against TensorFlow/Keras API and provides identical results for identical configurations in the limit of float64 precision.

Herein we report evaluation for **264** distinct combinations of:

Architecture schemes.

Number of units in layers.

Activation functions.

Loss functions.

Comparison was achieved by computing the Root Mean Square Deviation (RMSD) between:

EpyNN and TensorFlow/Keras averaged per-sample loss computed from output probabilities using TensorFlow/Keras loss function.

EpyNN and TensorFlow/Keras averaged per-sample per-output probabilities.

RMSD distribution for two training epochs with learning rate set to 0.01, for all experiments, shown below.

Note that the minimal error considering float64 precision and RMS operation is 1e-08. For the sake of rigor, RMSD = 0.0 was counted as RMSD < 1e-08.

Architecture schemes:

**(Dense)n**: Perceptron (n = 1) to Deep Feed-Forward (n > 2).**(RNN)n_Dense**: With one (n = 1) and two (n = 2)*RNN*layers.**(LSTM)n_Dense**: With one (n = 1) and two (n = 2)*LSTM*layers.**(GRU)n_Dense**: With one (n = 1) and two (n = 2)*GRU*layers.**(Convolution_Pooling)n_Dense**: With one (n = 1) and two (n = 2) Convolution_Pooling blocks.

Note that combinations of those are likely to be as reliable, but the exponential number of possibilities makes them unreported here.

Activation functions:

**Sigmoid**: For output layer.**Hyperbolic tangent**: For output layer and hidden layers.**Softmax**: For output layer.**ReLU**: For*Convolution*and hidden*Dense*layer.**ELU**: For hidden*Dense*layer.

Loss functions:

**MSE**: With sigmoid, tanh and softmax in the output*Dense*layer.**MAE**: With sigmoid, tanh and softmax in the output*Dense*layer.**MSLE**: With sigmoid and softmax in the output*Dense*layer.**BCE**: With sigmoid and softmax in the output*Dense*layer.**CCE**: With softmax in the output*Dense*layer.

These experiments, including all executable codes, can be downloaded below: