A major issue in machine learning is to select the best hyperparameters of a predictive model without over-fitting. In this paper, we propose to study through a principled way the hyperparameter optimization in a neural network designed for a classification problem on cosmetic formulation data. Specifically, we propose to make use of Bayesian optimisation (BO) to automatically choose the next hyperparameter set to try based on previous observation and a surrogate function of the model. This BO-based hyperparameter selection method is compared to the popular grid search method. A 2-hidden-layer fully-connected neural network is trained on cosmetic formulation data. Extensive experiments show that hyperparameters found with Bayesian optimization outperform the grid search hyperparameter set. Moreover, Bayesian optimization only needs 80 evaluations while 140 are used for the grid search.