Evaluation of Normalization Techniques on Neural Networks for the Prediction of 305-Day Milk Yield

Abstract

In this study, the impact of data preprocessing on the prediction of 305-day milk yield using neural networks were investigated with regard to the effect of different normalization techniques. Eight normalization techniques “Z-Score, Min-Max, D-Min-Max, Median, Sigmoid, Decimal Scaling, Median and MAD, Tanh-Estimators" and five different back propagation algorithms “Levenberg-Marquardt (LM), Bayesian Regularization (BR), Scaled Conjugate Gradient (SCG), Conjugate Gradient Back propagation with Powell-Beale Restarts (CGB) and Brayde Fletcher Gold Farlo Shanno Quasi Newton Back propagation (BFG)” were examined and tested comparatively for the analysis. Neural network architecture was optimized and tested with several experiments. Results of the analysis show that applying different normalization techniques affect the performance and the distribution of outputs influences the learning process of the neural network. The magnitude of the effects varied with the type of back propagation algorithms, activation functions, and network's architectural structure. According to the results of the analysis, the most successful performance value in the 305-day milk yield estimation was obtained by using the neural network structured by using the Decimal Scaling normalization technique with the Bayesian Regulation algorithm (R2Adj = 0.8181, RMSE= 0.0068, MAPE= 160.42 for test set; R2Adj =0.8141, RMSE= 0.0067, MAPE= 114.12 for validation set).

Keywords

• 305-day milk yield
• agricultural data
• back propagation algorithms
• data pre-processing
• neural network
• normalization

References

1. Akıllı A (2019). Analysis of agricultural data with multivariate nonlinear fuzzy regression method. PhD Thesis, Kırşehir Ahi Evran University, Institute of Science, Department of Zootechnics, Kırşehir.
2. Atıl H and Akıllı A (2016). Comparison of artificial neural network and K-means for clustering dairy cattle. IJSAMI, 2 (1): 40-52.
3. Brown-Brandl TM, Jones DD and Woldt WE (2005). Evaluating modelling techniques for cattle heat stress prediction. Biosystems Engineering, 91 (4): 513-524.
4. Chen LJ, Cui LY, Xing L and Han LJ (2008). Prediction of the nutrient content in dairy manure using artificial neural network modeling. J. Dairy. Sci. 91: 4822-4829.
5. Cihan P, Kalıpsız O and Gökçe E (2017). Effect of normalization techniques on artificial neural network and feature selection performance in animal disease diagnosis. Electronic Turkish Studies, 12 (11): 59-70.
6. Dong R and Zhao G (2014). The use of artificial neural in vitro rumen methane production using the CNCPS carbohydrate fractions as dietary variables. Livest. Prod. Sci. 162: 159-167.
7. Dongre VB, Gandhi RS, Singh A and Ruhil AP (2012). Comparative efficiency of artificial neural networks and multiple linear regression analysis for prediction of first lactation 305-day milk yield in Sahiwal cattle. Livest. Sci. 147: 192–197.
8. Edriss MA, Hosseinnia P, Edrisi M, Rahmani HR and Nilforooshan MA (2008). Prediction of second parity milk performance of dairy cows from first parity information using artificial neural network and multiple linear regression methods. Asian Journal of Animal and Veterinary Advances, 3 (4): 222-229.

9. Eesa AS and Arabo WK (2017). Normalization method for backpropagation: A comparative study. Science Journal of University of Zakho, 5 (4): 319-323.
10. Gandhi RS, Monalisa D, Dongre VB, Ruhil AP, Singh A and Sachdeva GK (2012). Prediction of first lactation 305-day milk yield based on monthly test day records using artificial neural networks in Sahiwal cattle. Indian Journal of Dairy Science, 65 (3): 229-233.
11. Gandhi RS, Raja TV, Ruhil AP and Kumar A (2010). Artificial neural network versus multiple regression analysis for prediction of lifetime milk production in Sahiwal cattle. Journal of Applied Animal Research, 38 (2): 233-237.
12. Gorgulu O (2012). Prediction of 305-day milk yield in Brown Swiss cattle using artificial neural networks. South African Journal of Animal Science, 42 (3): 280-287.
13. Grzesiak W, Lacroix R, Wójcik J and Blaszczyk P (2003). A comparison of neural network and multiple regression predictions for 305-day lactation yield using partial lactation records. Canadian Journal of Animal Science, 83 (2): 307-310.
14. Grzesiak W, Blaszczyk P and Lacroix R (2006). Methods of predicting milk yield in dairy cows- Predictive capabilities of Wood’s lactation curve and artificial neural networks (ANNs). Comput. Electron. Agric. 54: 69-83.
15. Hampel FR, Ronchetti EM, Rousseeuw PJ and Stahel WA (1986). Robust statistics: The approach based on influence functions. John Wiley and Sons: Canada.
16. Han J and Kamber M (2006). Data mining: Concepts and techniques. 2nd ed, Elsevier: San Francisco, USA.
17. Hassan KJ, Samarasinghe S and Lopez- Benavidest MG (2009). Use of neural networks to detect minor and major pathogens that cause bovine mastitis. J. Dairy. Sci. 92: 1493-1499.
18. Hosseinia P, Edrisi M, Edriss MA and Nilforooshan MA (2007). Prediction of second parity milk yield and fat percentage of dairy cows based on first parity information using neural network system. JApSc, 7 (21): 3274-3279.
19. Ince D and Sofu A (2013). Estimation of lactation milk yield of Awassi sheep with artificial neural network modeling. Small Ruminant Research, 113 (1): 15-19.
20. Jain A, Nandakumar K and Ross A (2005). Score normalization in multimodal biometric systems. Pattern recognition, 38 (12): 2270-2285.
21. Jain YK and Bhandare SK (2011). Min max normalization based data perturbation method for privacy protection. IJCCT 2 (8): 45-50.
22. Jayalakshmi T and Santhakumaran, A (2011). Statistical normalization and back propagation for classification. IJCTE. 3 (1): 1793-8201.
23. Kandanaarachchi S, Muñoz MA, Hyndman RJ and Smith-Miles K (2019). On normalization and algorithm selection for unsupervised outlier detection. Data Mining and Knowledge Discovery, 34 (2): 309-354.
24. Karadas K, Tariq M, Tariq MM and Eyduran, E (2017). Measuring predictive performance of data mining and artificial neural network algorithms for predicting lactation milk yield in indigenous Akkaraman sheep. Pakistan Journal of Zoology, 49 (1): 1-7.
25. Kominakis AP, Abas Z, Maltaris I and Rogdakis E (2002). A preliminary study of the application of artificial neural networks to prediction of milk yield in dairy sheep. Computers and electronics in agriculture, 35 (1): 35-48.
26. Kong LN, Li JB, Li RL, Zhao XX, Ma YB, Sun SH and Zhong JF (2018). Estimation of 305-day milk yield from test-day records of Chinese Holstein cattle. Journal of Applied Animal Research, 46 (1): 791-797.
27. Mostert BE, Theron HE, Kanfer FHJ and van Marle Koster E (2006). Comparison of breeding values and genetic trends for production traits estimated by a lactation model and a fixed regression test- day model. S. Afr. J. Anim. Sci. 36: 71–78.
28. Murphy MD, O’Mahony MJ, Shalloo L, French P and Upton J (2014). Comparison of modelling techniques for milk-production forecasting. J Dairy Sci. 97 (6): 3352-3363.
29. Nayak SC, Misra BB and Behera HS (2014). Impact of data normalization on stock index forecasting. Int. J. Comp. Inf. Syst. Ind. Manag. Appl. 6: 357-369.
30. Negnevitsky M (2002). Artificial intelligence, a guide to intelligent systems. 2nd ed, Pearson Education: Harlow.
31. Njubi DM, Wakhungu JW and Badamana MS (2010). Use of test-day records to predict first lactation 305-day milk yield using artificial neural network in Kenyan Holstein–Friesian dairy cows. Tropical animal health and production, 42 (4): 639-644.
32. Njubi DM, Wakhungu J and Badamana MS (2009). Milk yield prediction in Kenyan Holstein-Friesian cattle using computer neural networks system. Livestock Research for Rural Development 21 (4).
33. Oztemel E (2002). Yapay sinir ağları. Papatya Yayıncılık, İstanbul, Turkey.
34. Pan J, Zhuang Y and Fong S (2016). The impact of data normalization on stock market prediction: Using SVM and technical indicators. In International Conference on Soft Computing in Data Science, 2nd Ed: Berry MW, Mohamed AH, Yap BW, Springer: Singapore, 72-88.
35. Panigrahi S and Behera HS (2013). Effect of normalization techniques on univariate time series forecasting using evolutionary higher order neural network. IJEAT, 3 (2): 280-285.
36. Ruhil AP, Gandhi RS, Monalisa D, Behra K and Raja TV (2011). Prediction of lactation yield based on partial lactation records using artificial neural networks. In Proc. 5th National Conference on Computing for Nation Development, INDIACom-2011.
37. Russell SJ and Norvig P (2016). Artificial intelligence: A modern approach. Pearson Education Limited: Malaysia.
38. Salehi F, Lacroix R and Wade KM (1998). Effects of learning parameters and data presentation on the performance of backpropagation networks for milk yield prediction. Transactions of the ASAE, 41 (1): 253.
39. Sanzogni L, and Kerr D (2001). Milk production estimates using feed forward artificial neural networks. Computers and Electronics in Agriculture, 32 (1): 21-30.
40. Savegnago RP, Nunes BN, Caetano SL, Ferraudo AS, Schmidt GS, Ledur MC and Munari DP (2011). Comparison of logistic and neural network models to fit to the egg production curve of White Leghorn hens. Poultry science, 90 (3): 705-711.
41. Shahinfar S, Mehrabani-Yeganeh H, Lucas C, Kalhor A, Kazemian M and Weigel KA (2012). Prediction of breeding values for dairy cattle using artificial neural networks and neuro-fuzzy systems. Comput. Math. Meth. Med. 2012: 127130.
42. Shalabi AL, Shaaban Z and Kasasbeh B (2006). Data mining a preprocessing engine. Journal of Computer Science, 2 (9): 735-739.
43. Shanker M, Hu MY and Hung MS (1996). Effect of data standardization on neural network training. Omega, 24 (4): 385-397.
44. Sharma AK, Sharma RK and Kasana HS (2006). Empirical comparisons of feed-forward connectionist and conventional regression models for prediction of first lactation 305-day milk yield in Karan Fries dairy cows. Neural Computing & Applications, 15 (3-4): 359-365.
45. Sharma AK, Sharma RK and Kasana HS (2007). Prediction of first lactation 305-day milk yield in Karan Fries dairy cattle using ANN modeling. Appl. Soft Comput. 7: 1112-1120.
46. Sola J and Sevilla J (1997). Importance of input data normalization for the application of neural networks to complex industrial problems. IEEE Transactions on nuclear science 44 (3): 1464-1468.
47. Tahmoorespur M, Hosseinnia P, Teimurian M and Aslaminejad AA (2012). Predictions of 305-day milk yield in Iranian Dairy cattle using test-day records by artificial neural network. Indian Journal of Animal Sciences, 82 (5): 511-516.
48. Takma Ç, Atıl H and Aksakal V (2012). Comparison of multiple linear regression and artificial neural network models goodness of fit to lactation milk yields. J Fac Vet Med Kafkas Univ, 18 (6): 941-944.
49. Torres M, Hervás C and Amador F (2005). Approximating the sheep milk production curve through the use of artificial neural networks and genetic algorithms. Computers & Operations Research, 32 (10): 2653-2670.
50. Yang XZ, Lacroix R and Wade KM (2000). Investigation into the production and conformation traits associated with clinical mastitis using artificial neural networks. Can. J. Anim. Sci. 80: 415-426.
51. Zhang G, Patuwo BE and Hu MY (1998). Forecasting with artificial neural networks: The state of the art. Int. J. Forecast, 14 (1): 35-62.