A NEW DESIGN OF TANGENT HYPERBOLIC FUNCTION GENERATOR WITH APPLICATION TO THE NEURAL NETWORK IMPLEMENTATIONS

Abstract

A CMOS hyperbolic tangent function generator circuit suitable for the implementation of analog neural networks is presented. In order to obtain an accurate yet simple circuit realization, a judiciously chosen symmetrical Padé approximation of the hyperbolic tangent function is proposed. As an illustrative application, we set up an application in which the proposed circuit is used as the nonlinear block of a two-layer neural network. Simulation results using Spectre Simulation tool in Cadence design environment with 0.18µm CMOS process verify proper operation of the proposed circuit as well as the neural network built around. These results demonstrate the validity of the theoretical analysis and the feasibility of the proposed circuit.

Keywords

• Tangent hyperbolic function generator,
• Padé approximation
• Activation function
• Neural network implementation

References

1. Weng, J.H., Lin, D.Y. (2014). Sine function generator. Int. Sym. on Bio. and Bioinf. (ISBB), 11-14.
2. Enqvist, M., Schoukens, J., Pintelon, R. (2007). Detection of unmodeled nonlinearities using correlation methods. IEEE Instrumentation and Measurement Tech. Conference (IMTC), Poland.
3. Esfahani, A.F., Dreesen, P., Tiels, K., Noël, J., Schoukens, J. (2018). Parameter reduction in nonlinear state-space identification of hysteresis. Mechanical Sys. and Signal Proces., 104, 884-895.
4. Wang, M., Liu, B., Foroosh, H. (2018). Look-up table unit activationfunction for deep convolutional neural networks. IEEE Winter Conf. on Applications of Computer Vision (WACV), 1225–1233.
5. Gomar, S., Mirhassani, M., Ahmadi, M. (2016). Precise digital implementations of hyperbolic tanh and sigmoid function. 50th Asilomar Conference on Signals, Systems and Computers, 1586–1589.
6. Kutluoglu, D., Toker, A., Ozoguz, S. (2017). CMOS Current Mode Exponential Function Generator Circuit Using Padé Approximation. 10th Int. Conf. on Elec.l and Elec. Eng. (ELECO), 1202-1206.
7. Robles, M.C., Serrano, L. (2009). Accurate differential tanh(nx) implementation. Int. J. Circ. Theor. Appl. 37, 613-629.
8. Abuelma’atti, M.T., Al-Yahia N.M. (2008). An Improved Universal CMOS Current-Mode Analog Function Synthesizer: Performance Analysis. International Journ. of Electronics, 95(1), 1127-1148.

9. Shamsi,J., Amirsoleimani, A., Mirzakuchaki, S., Ahmadi, A., Alirezaee, S., Ahmadi. M. (2015) Hyperbolic Tangent Passive Resistive-Type Neuron. IEEE Int. Symp. on Cir. and Sys. (ISCAS).
10. Namin, A.H., Leboeuf, K., Muscedere,R., Wu, H., Ahmad, M. (2009). Efficient hardware implementation of the hyperbolic tangent sigmoid function. Int. Symp. on Cir. and Sys. (ISCAS).
11. Erdener, Ö., Ozoguz, S. (2016). A new neuron and synapse model suitable for low power VLSI implementation. Analog Integr. Circ. Sig. Process, 89, 749–770.
12. Lin K.J., Liu M.K. (2007). CMOS current-mode exponential circuit using Padé approximation. Chung Hua Journal of Science and Engineering, 5, 77-80.
13. McCabe, J. H. (2009). On the Padé table for e(x) and the simple continued fractions for e and e(L/M). The Ramanujan Journal, 19, 95-105.
14. BakerJr., G.A., Graves-Morris, P.R., Padé Approximants, Cambridge Uni. Press, Cambridge, 1996.
15. Saatlo, A.N., Ozoguz, S., Minaei, S. (2015). Applications of a CMOS Current Squaring Circuit in Analog Signal Processing. 38th Int. Conference on Telecom. and Signal Processing (TSP), 339-343.
16. Alikhani, A., Ahmadi, A. (2012). A novel current-mode four-quadrant CMOS analog multiplier/divider. Inter. Journal of Electronics and Communications (AEÜ), 66 (7), 581-586.
17. Naderi, A., Khoei, A., Hadidi, K., Ghasemzadeh, H. (2009). A New High Speed and Low Power Four-Quadrant CMOS Analog Multiplier in Current-Mode. Int. Journal of Elec.and Com. (AEÜ), 63(9), 769-775.
18. Salama, M.K., Soliman, A.M. (2003). Low-Voltage Low-Power CMOS RF Four-Quadrant Multiplier. International Journal of Electronics and Communications (AEÜ), 57(1), 74-78.
19. Han, G., Sanchez-Sinencio, E. (1998). CMOS transconductance multiplier a tutorial, IEEE Trans. Circuits Syst. II 45 (12), 1550–1563.
20. Leboeuf, K., Namin, A., Muscedere, R., Wu, H., Ahmadi, M. (2008). High speed VLSI implementation of the hyperbolic tangent sigmoid function. Int. Conf. Converg. Hybrid Inf. Technol., 1070–1073.
21. Namin,A., Leboeuf,K., Muscedere,R., Wu,H., Ahmadi,M. (2009).Efficient hardware implementation of the hyperbolic tangent sig. function. IEEE Int. Symp. Circuits Syst.,2117–2120.
22. Heidari, M., Shamsi, H. (2019). Analog programmable neuron and case study on VLSI implementation of Multi-Layer Perceptron (MLP). Microelectronics Journal, 84, 36–47.
23. Ghomi, A., Dolatshahi, M. (2018). Design of a new CMOS Low-Power Analogue Neuron. IETE Journal of Research, 64(1), 67-75.