NOISE CANCELLATION WITH LMS VARIANTS

The noise is one of the important factors that affect negatively the operation of circuits and systems. There are different methods and techniques for cancelling/reducing/de-noising these unwanted signals that mix with the original signal. In this study, an application/simulator that performs noise cancellation operations with least mean-squares algorithms was designed. The application, which has powerful visual support and a user-friendly interactive interface, can make single or comparative noise cancellation operations and performance analyzes with different algorithms on noisy signals. The effects of the algorithm parameters can be observed, information about the related algorithms can be obtained via the developed application that can also be used for educational purpose.


INTRODUCTION
The main purpose of the noise cancellation/reduction process, which is used in wide areas such as communication, biomedical, control, signal/image processing, is to reduce or completely eliminate the undesired signal (noise) in the desired signal. The fundamental process used in denoising in digital signal processing is filtering. For this purpose, finite impulse response (FIR) and infinite impulse response (IIR) digital filters are used. However, problems are encountered in de-noising process if filters which have fixed coefficients are used, and the performance rate decreases. Adaptive filters have been used frequently in recent years to eliminate these problems.
The adaptive filters are filters with self-adjusting characteristics. These filters, which can be designed as analog, digital or hybrid (mixed), have advantages and disadvantages compared to each other. For example, while analog filters consume low power and respond quickly, almost all of them are implemented as digital (FIR or IIR) due to the complexity of the optimization algorithms.
As a result of the rapid development in the computer world, computer-aided technologies (CAx) concepts (such as computer-aided engineering (CAE), computer-aided design (CAD), computer-aided manufacturing (CAM), computer-aided software engineering (CASE), computeraided instruction (CAI), computer-aided learning (CAL) etc.) have currently gained significant place. Simulators, applications, web pages, etc. are developed in many areas using computer software (Maddala, 2010 In this study, an application/simulator that performs noise cancellation operations with different variants of the least mean-squares (LMS) algorithm is designed. The application, which is designed using MATLAB App Designer (MathWorks, 2018), can be also used for educational purposes, can perform single or comparative noise cancellation operations, observe the effects of algorithm parameters, and examine the information about algorithms. This paper is organized as follows: In Section 2, adaptive filters and theirs application areas are summarized. In Section 3 , LMS algorithm and its variants are briefly explained. In Section 4, noise cancellation process with LMS is presented. In Section 5, designed application/simulator is explained and sample applications are given. Finally, Section 6 contains conclusions.

ADAPTIVE FILTER
The commonly used adaptive filter structure is given in Fig. 1. The error signal here is used to create the objective/performance function that the related adaptive algorithm will handle. Minimizing this function ensures filter output matches the desired signal. The mean square of the error signal is the popularly used cost function. Table 1 shows the basic application classes of adaptive filtering (Widrow and Stearns, 1985;Morales, 2011;Haykin, 2014;Diniz, 2020).

Figure 1:
The commonly used adaptive filter structure As can be seen from Table 1, adaptive filters have wide usage areas: audio and video signal processing, noise and echo cancellation, channel estimation, channel equalization, modeling, inverse modeling, system identification, linear prediction, signal enhancement, active noise control, adaptive notch filter, jammer suppression, speech coding, adaptive line enhancement

LEAST MEAN-SQUARES ALGORITHM
LMS, one of the most used algorithms in adaptive filter applications, was developed in 1960 by Prof. Bernard Widrow and his Ph.D. student Ted Hoff. The main properties of this algorithm, of which many variants have been developed, can be listed as simplicity, ease of implementation, low computational complexity/cost, robustness etc. (Farhang-Boroujeny, 2013; Poularikas, 2015; Diniz, 2020).

Figure 2: Adaptive linear combiner (ALC) in transversal form
The adaptive linear combiner (adaptive FIR filter, adaptive transversal filter, ADALINE) as shown in Fig. 2 The output of the filter is written by Eq. (2), with filter input and tap-weight (coefficients) vector.
The error signal, its square and the mean square (squared) error (expectation of the error power or energy) are defined by Eq. (3)(4)(5).
Since the correlation matrix is a symmetric and positive definite matrix then Eq. (5) is a quadratic function of . This function has only one minimum solution known as the optimum Wiener solution (Wiener weight vector). This solution is obtained by equating the derivative of the Eq. (5) to zero. Thus, the gradient of the mean square error function in Eq. (6) is obtained and its solution is given in Eq. (7).
Generally, Eq. (7) (Wiener-Hopf equations) solution is not applicable and alternative solutions should be used as in Table 2

Method/algorithm Iteration equation Solution iteration
Newton's method in gradient search ] is the gradient operator, the following LMS recursion obtained. [ [ Replacing the iteration index by the time index in Eq. (10) is rewritten as Eq. (11).

NOISE CANCELLATION PROCESS WITH LMS
Adaptive noise cancellation (ANC) is used in wide area to reduce noise from a corrupted signal. The aim of an ANC is to cancel the noise from a signal adaptively to improve the SNR. Fig. 3 shows the block diagram of adaptive noise cancellation. The system output in Fig. 3 is written by Eq. (12) (Widrow et al., 1975).
The minimum output power of system is In order to provide { 2 } to be minimum, {( 0 − ) 2 } must also be minimum in Eq. (15), hence

DESIGNED APPLICATION
In this study, an application/simulator was designed using MATLAB App Designer (MathWorks, 2018). With this application, noise cancellation operations can be performed with different variants of LMS algorithms. In the application, which includes information about algorithms, the effects of parameter changes can also be observed. In addition, single or comparative analyzes can be made on internal (sample or user-defined) and external (uploaded by the user) signals, and the results are displayed both numerically and graphically. Thus, the performance effects of both of the methods and the parameters related to the methods can be observed. In addition, it supports better understanding of both noise cancellation operations and LMS algorithms with its use in the field of education.
The main screen of the designed application is shown in Fig. 4 and its menus are given in Table 4. The user/student can select the sample signal for analysis from the "Signal" drop-down list (combobox), generate different types of signals (generating sinusoidal, exponential, constant signals by entering their parameters) (Fig. 5a) or load signals from the file. Likewise, users can select the sample noise from the "Noise" drop-down list, generate noise themselves (generating noise by entering the scaling and shifting parameters in equation { (0,1) − }) (Fig. 5b) or load noise from the file. After each process, the related signal/noise graphics are simultaneously seen/plotted on the main screen. Finally, noise cancellation is performed by selecting LMS or one of its variants from the "Algorithm" drop-down list. If it is desired, comparative analysis can also be made with the "Comparative" option here. In addition, by selecting "Error analysis" or "Parameter analysis" from the menu that opens with the mouse right-click on the "de-noised signal" graph, error graphics/values and the effects of parameter changes can be viewed both numerically and graphically on the related screens. The flowchart of single analysis process in designed application is given in Fig. 6.  Figure 5:

(a) (b)
The signal and noise generation screens When "Least Mean-Square (LMS)" is selected from the "Algorithm" drop-down list, the screen for setting the related algorithm parameters appears (Fig. 7b). After the related parameters are entered, the result screen in Fig. 7c is obtained. When the mouse is right-clicked on the "De-noise signal" graph and "Error analysis" is selected, the screen in Fig. 7d appears. This screen displays "learning curve", "absolute error" and "squared error" graphics and mean absolute error (MAE), mean squared error (MSE), sum absolute error (SAE) and sum squared error (SSE) performance values. In addition, by changing the starting point of the samples with related slider or spinner, the change of this error values can be monitored (Fig. 7e). On the other hand, if "Parameter analysis" is selected, comparative performance analysis can be performed by entering changeable parameter values of the related algorithm ( Fig. 7f-g). In addition to the comparative performance graphics, the minimum values in the relevant error table are automatically colored on this screen. For "Comparative analysis", algorithms are selected on the algorithm selection screen, their parameters are set and the comparison screen in Fig. 7h appears. On this screen, the MSE values of each algorithm are listed in the table, and de-noised signals and error graphs are also plotted comparatively. Also, re-analysis can be made by changing the parameters (Fig. 7h). Figure 7: Screenshots of the sample applications Summarization of comparison this study with similar studies in noise cancellation field are given below:  Generally, several methods or LMS variants are examined and performance comparisons are made in the studies. In the application developed in this study, there are many LMS variants, and single or comparative analysis can be performed.  In the studies, methods/methods are usually encoded in MATLAB environment (or ready-made functions are used) or SIMULINK models are prepared. In this study, an interactive interface is designed for the methods.  In some studies, hardware applications were carried out using FPGA or DSP kits. The hardware implementation is not included in the study in this article.  The general comparison of the simulator developed in the reference (Maddala, 2010) and the simulator designed in this study is given in Table 4. The interactive application developed has features that can be used both in practice and education. For practical applications, the most appropriate algorithm and parameter values can be determined by analyzing the relevant noisy signals (by creating or loading) in the simulator. It can be used effectively in the field of education thanks to its capabilities such as including lectures, showing parameter effects in detail, and making comparisons.

CONCLUSIONS
In this study, an application/simulator was carried out for noise cancellation, in one of the most important application areas of adaptive filters. With this application, which includes many variants of the LMS algorithm; the sample, user-defined or loading noisy signals can be de-noised easily and quickly with the selected algorithm or algorithms. It is obvious that LMS algorithms and noise cancellation processes can be better understood by using the developed interactive application especially in the field of engineering education. The ability of the simulator to give opportunity to students/users to examine lectures, change algorithm parameters, and observe single or comparative results both numerically and graphically will contribute greatly to learning and comprehension. As a future study, it is aimed to add more LMS variants to the application/simulator and move it to the web environment.