Real time implementation for fault diagnosis and condition monitoring approach using image processing in railway switches

Today, railway transportation is one of the transport modes commonly used. Compared to other transport modes, railway traffic is highly critical. Multiple railway vehicles run constantly on one or two lines. Rail switch passages are used to prevent locomotives from colliding with one another and avoid traffic disruptions. Through switch passages, locomotives pass from one line to another. Friction between rail and wheels on switch passages is considerably high. This friction leads to failures on switch passages. Unless these failures are diagnosed early and remedied, significant accidents emerge. In this study, a new approach based on image processing has been presented for detection of rail switch passages on railway lines. A test vehicle has been created in order to test the proposed approach and apply it on a real-time system. Railway line is monitored by digital cameras fixed on this test vehicle. Image-processing approach is developed on the real-time images captured from the railway line and the switch passages on the line are detected. In addition, by specifying the train route, the fault which occurring at the point of the switches is detected. The image-processing approach consists of three main parts including pre-processing, feature extraction and processing of the features obtained. At the pre-processing stage, the basic image processing methods are used. At the feature extraction stage, Canny edge extraction algorithm is used and hence the edges in the image are detected. Hough transform method is used at the stage of processing of the extracted features. Following Hough transform stage, straight lines and angles of these lines are obtained on the image. Taking into account the angle of each straight line, the junction points of the lines are calculated. Thus, rail switch passage and switch types are detected. The proposed image-processing approach is highly fast and real time-based. Compared to the existing studies in the literature, it is seen that the proposed method gives fast and successful results. This study intends to diagnose the failures on switch passages early and prevent potential accidents.


Introduction
Railway transportation is commonly used in the freight and passenger transport. Railway vehicles provide transportation using a particular rail line [1,2]. These vehicles are preferred by people because they can carry quite heavy loads and large number of passengers. It is very important for rails to be robust because railway vehicles provide transportation on the rails [3]. The railway lines are the most critical components for this transportation. The switch has been developed for more than one train can use the same railway line. Trains can change their direction, give way to another train or proceed to the standby points at the station due to the switch system. The development of the switch system has made the railway line quite utilizable. Therefore, the switch system has a great importance for the railway transportation. Excessive friction and wear occur during switching [4]. Such undesirable situations lead to major wear on the rail in time. Major accidents may occur in the event of failure in the early diagnosis of these wears. The accidents that may occur can cause loss of life and injury for passengers. However, the railway line hinders the traffic for a while during the accident, and major financial losses are formed. Today, the railway line is inspected at regular intervals, and the faults which may arise are diagnosed early. There are many literature studies related to switch system. Johansson et al. [5] conducted a study for the detection of the corrosion faults caused by friction and wear during switching. In the study, Switches & Crossings components are monitored. They modelled the contact of the wheel with the rail by simulation during switching. As a result of modelling, observation was made according to the loads that may occur on the rails. Bocciolone et all [6] proposed a signal processing-based method for the detection of the faults that occur on the rail surface. In the proposed method, the vibration signals made by the train while going on the railway line were used. Fault detection is performed by improving the signal processing methods through vibration signals. Palsson [7] completed a thesis study on optimization in the Switches and Crossing sections on the railways. In this thesis study, Switches and Crossing components were monitored and the faults that occurred in these components were examined. The changes in the Switches and Crossing areas were observed during switch transitions. The load distribution from one rail to another rail was examined. In addition to these studies, many studies have been conducted for the detection of the faults occurring on the rail surface and in its components. Quingyong et all. [8] an image processing based contactless method is proposed and detected faults on rails in real-time. Rail surface is extracted by using railway images. Surface faults are determined by contrast stretching of the obtained surface. This method, is used only railway surface to detect faults. Limin et all. [9], proposed a method that detects faults on the surface by using machine vision techniques. The roughness and cracks are determined with image processing algorithms. Edge detection and feature extraction methods are used. Ying et al. [10] proposed an automatic monitoring approach to detect and evaluate the railway components. The proposed method is working on a train with 16km/h speed. Ray component detection is carried out by applying Sobel edge detection and Hough transform methods. Status and positions of the _______________________________________________________________________________________________________________________________________________________________ Figure 1. General architecture of the railway inspection system proposed in the literature [10] Ashwani et all. [11]. Has proposed an image segmentation based method for the detection of railway rail failures. Using the Maximum Stable Extremal Region Marking (MSER) method, he has identified defects on the track surface. The flow diagram of the proposed method is given in Figure. Pavel at all [12], he performed component detection and fault detection on the railway line. By detecting the rail track surface and connecting elements, defects in these components detect shape disturbances. Li at all. [13] proposed a line detection method for the rail switches tracking system. Basic image processing techniques are used in this method. Canny edge extraction is used in this study, the Zhang-Suen Thinning method is used to reduce the thickness of the edges, and the probabilistic Hough Transform is used for line detection. Zhiquan at all [14] detects rail tracks using a HOG-based method on the railway line. It divides the image into small flags and removes the rail line on the image. The obtained rail track detected the route of the train using switch passages. It also suggests a method that can work on images taken under different conditions. In this study, an image processing-based method was proposed for the detection of the crossing points on the railways. Images were taken for different types of switches using a camera fixed on the train. The switch crossings were determined using image processing algorithms on the images taken. By following the switch crossing points, the train route is also determined. In addition, by switch crossing points are examined and condition monitoring and malfunction detection are performed.

Types of Railway Switches
The apparatus for changing the way that allow railway vehicles moving on the rail to pass from one rail to another rail are called "switch". In switch systems, the change of way of railway vehicles is carried out by ensuring the movement of the Switch part of the junctions with human or electric power [15]. Each switch has a right track and a diverted track. The sleepers of the switches on conventional lines are wooden. The radius of the curve located on the diverted track that provides transition to secondary roads is R=300 m, the maximum length standard of single turnout is 34.20 m. The sleepers of the switches on highspeed trains are concrete, the radius of the curve located on the diverted track that provides transition to secondary roads is R=1500 m, the maximum length standard of single turnout is 72 m.
Railways have switch crossings for the control of the traffic and a line to be used by more than one trains. A train moving on a line can easily switch to the other line due to switch crossings [16]. An example of a switch crossing system and components is given in Figure 3. As it is seen in Figure 3, the switch crossing system has many components. The components of Switch Blades, Toe, Stretcher and Heel serve as a switching. The components of Crossing, Check Rails and Wing Rails allow train to switch to another rail in a secure way. The components of Check Rails and Wing Rails are the most important components that prevent the derailment of the train. The switch crossings are very important on the railway line. Major faults may occur as a result of the failures of the components on the switch crossings. Therefore, the railway line especially the switch crossings should consistently be inspected. It is required to make the necessary maintenance even considering the small faults. On switch crossings, to which direction the train will go is determined using the components of Switch Blades, Toe, Stretcher and Heel. The changes that occur on the switch crossings when a train goes straight or to the right direction on an example railway line are seen in Figure 4. In the sample images given in Figure 4, a train coming from X direction in Figure 4.a goes straight ahead, namely to the Y direction. In Figure 4.b, a train coming from X direction turns right, namely to the Z direction. There are many types of switch crossings which are commonly used on the railways. These types of switches vary depending on the number and position of the railway line. The types of switch crossings and their sample images are given in Figure 5. The sample images of the switch crossing types given in Figure 5 are presented in Figure 6.

The Method Proposed for Switch
In this study, an image processing-based method was developed for detecting the switch crossings on the railway line. In the proposed method, feature extraction was performed by performing pre-processing on the image. The lines on the image are obtained as a result of feature extraction. The intersection points are calculated by considering the start and end positions and slopes of the lines obtained. The calculated intersection points are called switch crossing zone. The flow chart showing the main stages of the proposed method is presented in Figure 7. represents blue image matrix. The image smoothing process is performed by applying gauss filter to the image in gray format obtained. A gauss mask in 5x5 size is used in the Gauss filter. After the gauss masking process, edge extraction is performed by using the canny masking matrix of the image from X and Y direction. The image is formed according to certain angle limits by calculating the gradient magnitude on the image matrices obtained. The edge extraction process is achieved more successfully because Canny edge extraction algorithm uses the Gauss filter before performing edge extraction on the image. Canny edge extraction algorithm gives better results compared to other edge extraction algorithms. Hough transform is used to obtain the lines on the image after the edge extraction process [22]. Hough transform is generally used in detecting geometric models on the image in combination with the edge extraction methods. Hough transform is applied onto the images, the edge information of which are obtained. The method transforms the problem of finding a shape into a problem of finding a density by transferring the information in the image space to the parameter space [23][24][25]. Hough transform does not require that the points on the image are side-by-side or connected to detect an image. The pseudo code of Hough transform used in the proposed method is given in Figure 8.  [24] A binary-based image obtained as a result of edge extraction is primarily used in the pseudo code of Hough transform given in Figure 7. The straight lines are expressed as in equation 1.
This equation has two parameters to specify any straight right. These parameters are slope ( m ) and junction point (  point of these two lines is E. The junction point of two lines two points of which are given is shown in the analytical plane as in Figure 9.
The switch crossing zone is detected on the image using

Experimental Results
In this study, an image taking environment was created to take sample images on the railway line. The images of the railway are taken by the camera installed on the train as in Figure 10. In the proposed method, the switch crossings were detected by taking images in different situations on the railway line. The dimensions of the images taken are 640 x 480 pixels. The colour image was firstly transformed into gray image, and then edge extraction was performed. The lines are obtained by Hough transform on the image obtained. The images used during realization of the proposed method are given in Figure 11. The results were obtained by performing edge extraction on the image given in Figure 11. These results are presented in Figure 12.  Figure 13. The switch crossings in Figure 13 were detected. The image of a normal railway without switch crossing is presented in Figure  13.f. No switch crossing was detected there. The switch crossings were detected successfully in other images. Within the context of this study, many images were obtained for the test procedure.
There are 100 switch images on these images. These switch images belong to different types. The numbers of successful and unsuccessful images when the proposed method was applied on these images are given in Table 1.  As it is seen in Table 1, the proposed method is highly successful in the images belonging to the types of single left and right switch. The success ratios of other types of switch are not very good because they are a little more complicated and due to the clarity of the image. The paths of images where switch passes are detected and their routes has determined. Switch passage was determined on the sample images given in Figure 6.c and d and their routes were determined. The images in Figures 14.a and c were obtained by applying the proposed method to the sample image in Figure 6.c. In Figure  14.c, the trains are on the right-hand side, as the left-hand rail joins the main and neighbour railways. By applying the proposed method to the sample image in Figure 6.d, the images in Figure  14.b and d are obtained. In figure 14d, the right rail is connected to the main and neighbour railway lines, so the train route is on the left. The route of the train is determined by taking into account the intersections that occur at the crossing point of the switches. If the images do not intersect the main and neighbour rail tracks, or if the two are intersecting, there may be a sign of malfunction. The performance of the proposed method is given in Table 2.  Determine route, fault detection 26, 9 4,8 The performance test was conducted on a computer with i5 processor 2.60 Ghz, 8 GB RAM and 64 bit operating system.

Conclusion
An image processing based method for the detection of railway switch crossings was proposed in this study. In the proposed method, the switch crossings were detected by image processing methods using railway images. The route of the train is determined by considering the switch crossing points. It can be detected if there is an abnormal condition at switch crossing points during the defined route. Canny edge extraction and Hough transform were used as the image processing methods. When the studies in the literature are analyzed, it is seen that there is not a study on the detection of switch crossings and condition monitoring by using image processing based methods. It is seen that the proposed method is successful when its results are taken into account. In the proposed method, it is very important to determine the route during switch crossing. Failure detection can also be done by following the switch crossing points.