Image Processing Based Level Crossing Detection and Foreign Objects Recognition Approach in Railways

: Level crossings are an important part of rail and road transportation and are areas where serious accidents occur in. Most of the accidents in railway transportation are happening in the level crossings. In this paper, a vision-based method is proposed for the prevention of these accidents in the level crossing. With this method, which is based on image processing, the condition monitoring of the level crossing is performed. The obstacles in the level crossing are detected and the estimated distance of these obstacles to the camera is calculated in the proposed method. In order to detect the obstacles in the level crossing, the level crossing in the railway image is determined first. YCbCr color transformation, edge extraction, filtering and Hough transformation have been applied to the image in the detection of the level crossing. The detected level crossing area has been labeled as the grade crossing in the image. It has been checked whether or not it has obstacles at the level crossing. HSV color transformation, image difference extraction, gradient calculation, filtering, detection of connected components and feature extraction have been applied to object detection. A single camera has been used in the proposed method to calculate the distance between the detected foreign object and the camera. The number of pixels covered by the object in the image is taken into account in calculating the distance between the object and the camera. The distance of objects at different distances from the camera is calculated in proportion to the number of pixels in the reference image. This study provides an improvement in this area due to the fact that studies on the literature related to the determination of the level crossing and foreign objects in the level crossing based image processing are not enough.


Introduction
The level crossings are often used in railway and road transportation. Due to frequent use of the level crossings, serious accidents occur in these crossings. Due to the use of people, motor vehicles and trains, serious accidents in these areas can occur. The presence of foreign objects in the crossings threatens the safety of transportation. Then warnings should be given to the relevant units after it is determined whether the foreign objects in these passageways cause to an accident or not. It is aimed to prevent the accidents that may be experienced by using the image processing techniques in order to detect by how far the foreign object is located. There are studies in the literature related to the topic. Dwarakanath et al. [1] proposed automatic railway gate control at a level crossing and dealt with detection of obstacle on track by utilizing the automatic railway gate control at the level crossing. The arrival of the train was detected by the sensors placed on the side of the tracks. To identify obstacles to the railway line is concerned with two things; firstly any obstacle on the track is detected by using the sensors located at the front end of the train and secondly, to convey the obstacle detection message to the nearby railway station through GSM technology. The proposed system uses infra-red sensors to detect the arrival and departure of trains at the railway level crossing. This method tackled with automatic railway gate control at a level crossing changing the gates operated by the gatekeepers. Silar and Dobrovolny [2] recommended algorithm for obstacle detection and objects tracking in a railway crossing area. The object tracking was based on template matching and the sum of absolute differences. The object tracking was enforced for better reliability of recommended system. The results of proposed algorithm were verified in real traffic scenarios consisted of two railway crossings in the Czech Republic. Amaral et al. [3] presented a system for obstacle detection in railway level crossings from 3D point clouds gained with curved 2D laser scanners. This recommended system was able to gain highly dense and correct point clouds, allowing the detection of small obstacles, like rocks laying near the rail. Throughout an offline training phase, this system learned a background model of the level crossing from a set of point clouds. Then the obstacle was compared online with the background model and identified as the occupied area. To diminish the requirement for manual onsite calibration, this system automatically forecasted the pose of the level crossing and railway according to the laser scanner. Kim and Cohn [4] set up a camera in front of a locomotive to investigate the level crossing traffic accidents. After the ignition of the signal, it provided a computer vision system that automatically detects the possible after-accident scenes by detecting the effectiveness of the vehicles passing in front of the train. In addition, it provides a quick algorithm to detect moving objects recorded with a moving camera with minimum calculation. Their algorithm works in pseudo real-time and thinks that its algorithm can be applied to real-time applications such as ______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ , 1 Computer Engineering, Firat University, Elazig -23100, TURKEY collision warnings in the near future with the enhancement of hardware technology. Heavisides et al. [5] presented the risk of a level crossing, which is the subject of a range of current, high-profile, industry programs in the UK. This study is concerned with a concerted effort to establish a common basis for modelling riskier in level crossings and evaluating all crossings. Another purpose of this study was to use the detection of obstacles to provide an early warning for the train driver approaching an object on the road. Based on the findings of this research, they developed a basic specification for a complete barrier transition with obstacle detection. Regardless of the specifics of the specification, obstacle detection has shown the potential to be an important feature of the level crossing design and renovation programs. Pu et al. [6] designed a study to develop an advanced system. This system was found to have moving obstacles on the railway crossing. The authors set up a grayscale CCD camera and developed a graphical user interface to manipulate the level crossing images. The software is programmed to perform various image processing techniques such as image subtraction, quadrature, morphological transformation and segmentation to track moving obstacles. Portions of the monitored image around the rails were labelled as alert and alarm zones where detected obstacles would trigger the sirens of this system. Under diverse lighting conditions for the model cars with different colors, experiments on the developed system indicated that the level of luminous was a significant factor affecting the average alert accuracy rate but the color of cars was not. Overall, the average warning accuracy rate reached 97.8%. The promising results of the disinfection capacity of the system may threaten the system deserves full scale development at railway transit sites to provide effective protection against unpredictable, preventable events. The flow diagram of this system [6] is shown in Fig. 1. In this study, the obstacle have been detected at the railway crossing and the distance between the camera and the detected object have been calculated by contactless image processing techniques. It has been determined whether or not the object that may interfere with transportation or cause an accident. Preprocessing, feature extraction, Hough transformation, elimination of noisy pixels and some image processing techniques have been used during the detection of the level crossing. The railroad image must be clear of the noisy pixels of the scene so that the level crossing can be correctly detected. During the elimination of the noisy pixels, it should be taken into account to prevent the pixels which are important for the level crossing from being destroyed. Image processing filters have been applied to the image for this purpose. A single camera has been used in the proposed method to calculate the distance between the perceived foreign object and the camera. The number of pixels covered by the object in view is considered when calculating the distance between the object and the camera. The distances of objects from different distances from the camera are calculated in proportion to the number of pixels in the reference image.

The Types of Level Crossing
The average speeds in the railway sector and the intensity of traffic on railway lines have increased considerably in recent years and have resulted in new and greater risks. Furthermore, the increase in traffic intensity has also decreased considerably the time intervals used for safety between train services. New safety strategies and methods are required to ensure that the railway's usual safe travel image is not lost and risks are kept within certain limits. The level crossing accidents are defined as the collision of one or more road vehicles using a railway vehicle with a passenger car or other users of the passenger car (e.g., pedestrians -animals -other objects). It is possible to see the significance of the level crossing accidents when you look at the Accident Statistics in Railways. Within the last 10 years, the railway accidents in the level crossing have a share of 35% [7]. The types of the level crossing are free (cross-marked), blind without bell, secured barrier, automatic barrier and other. The coating type of the level crossings can ben wood, stabilization, concrete, tire, soil -ballasts, asphalt, parquet stone, steel coating.

The Proposed Approach
In this study, the condition monitoring of the level crossing has been determined based on image processing. It is determined a vision based whether or not there is an obstacle at the level crossing that is detected by using image processing techniques. Then the calculation of the distance between the obstacle and the camera detected has been done. The distance calculation uses the number of pixels covered by the object in the image and the camera distance information from the reference image. The block diagram of the proposed work is shown in Fig. 4

The Level Crossing Detection
To extract region of the level crossing, we applied several imageprocessing techniques to railroad images. These imageprocessing techniques are respectively image enhancement, color conversion, edge detection, Gaussian filter, gradient computing, Hough transform, etc. In the first step of this detection method, the images of the railway have been taken from the camera. Image enhancement techniques have been applied to taken images. Image sharpening and contrast enhancement have been performed in order to advance the images.  Fig. 5. The pseudo code of Hough transform [22].
Image sharpening increments the contrast between bright and dark regions to expose features [10]. Contrast, the image quality is an important factor in the evaluation of subjective. The contrast is created by the difference in brightness reflected from two adjacent surfaces [11]. In other words, the contrast is the difference between the visual features that distinguish the object from other objects and the background. Contrast in the visual sense is determined by the difference in color and brightness of the object and other objects [11]. The image is converted from the RGB color space to the YCbCr color space after applying the image enhancement process. The YCbCr color space is widely utilized for digital video [12]. YCbCr color space is one of the popular color space in computing [13]. In this color space, the luminance information is stored as a single component (Y), and chrominance information is stored as two color-difference components (Cb and Cr). Cb represents the difference between the blue component and a reference value. Cr represents the difference between the red component and a reference value [12][13][14]. It can be calculated Y, Cb, and Cr values as in (1) In the next step of the proposed method, the edge detection is performed on the image. Prewitt edge detection algorithm is utilized for this. Prewitt mask is a discrete differentiation operator. This operator uses two 3 × 3 masks to calculate approximate derivative values in the horizontal and vertical direction [15,16]. Fig. 6 appears how to calculate the new value of a pixel horizontally and vertically. When Prewitt edge detection mask average is taken, it gives equal weight to all pixels [17]. Coefficients in the matrix core as shown in the mask is 1. The important idea is that of central differences. This mask when convolved with an image, it performs a two-dimensional spatial gradient [17]. Gradient magnitude and x and y-direction gradients of this image are then calculated [18][19][20]. Then a Gaussian filter is applied to the image. Gaussian filters are a class of linear smoothing filters of a Gaussian function with selected weights according to the shape. The Gaussian smoothing filter is a very good filter to remove noise from normal distribution [21]. Then Hough transform is implemented to detect line in the image. The Hough transform is used to detect the straight lines and parametric curves in an image [22][23][24]. The pseudo code of Hough transform algorithm is shown in Fig. 8. After the lines in the image have been determined, the level crossing has been extracted by cropping the image of the level crossing region.

Foreign Object Detection
After the level crossing is determined, it has been determined whether there are foreign objects on the grade crossing. It is a common practice to detect foreign objects by using imageprocessing techniques. The block diagram of the algorithm used for foreign object detection is shown in Fig. 9. Several image processing techniques such as color conversion, image filtering, and segmentation have been applied to the image of the grade crossing taken as input in the first step of the algorithm. The input image received in RGB format during color conversion is converted to HSV format. HSV color space consists of three channels [25]. These channels are Hue (H), Saturation (S), and Value (V). Hue channel defines the dominant wavelength of the color such as green, blue, yellow, etc. In addition, this channel has a value between 0-360 angular. Saturation channel identifies the vividness of the color. High saturation reasons to vivid colors and low saturation ensures that the color approaches to shades of  gray. While value channel defines white rate of within color. Therefore, it identifies the bright of color. V channel image is used in this study. Then the input image received in RGB format during color conversion is converted to grayscale color space. It is subtracted the grayscale image from the V channel image. The gradient magnitude of the image obtained in the previous step has been calculated and the Median filter is applied to the result image. The connected components of the result image is found and this components are compared with certain threshold values. Finally, it is labelled as a foreign object.

Distance Measurement Between
After the level crossing is determined, it has been determined whether there are foreign objects on the grade crossing.

Experimental Results
References need not be cited in text. When they are, they appear Other than books, capitalize only the first word in a paper title,.

Conclusions
References need not be cited in text. When they are, they appear Other than books, capitalize only the first word in a paper title, except for proper nouns