An Image Enhancement Method For Night-Way Images

—Image processing has a wide range of applications especially in our daily lives. Image processing is not common in sensitive industrial applications. Because of these applications, very high percentage of success is requested. Also these applications work in real-time. However, it can be widely used in many daily routines (driving, entrance to the workplace/ exit, control of multimedia devices, security applications, identification applications, etc.). Especially Advanced Driver Assistance Systems (ADAS) is a popular working area for image processing. Strip tracking systems, pedestrian detection systems, reading of traffic signs and signals are based on image processing. In this study, the visibility levels of road images at In the brightness level is low because of insufficient of light sources (headlights and road lighting) which are often used to increase the driver's view. On the other hand, adversely affects the view of driver which the headlight of coming vehicles from opposite directions, poorly structured road lighting and etc. Especially the vehicle headlights coming from the opposite direction take the eye of the drivers and cause the level of view to decrease. Intense dark areas and light sources are in the image together. special to these images requires the use of an adaptive improvement method. This is because, when classical image are the visibility levels of the dark areas are increased, and the shining regions are more likely to shine and the visibility level decreases in these regions.


I. INTRODUCTION
IGHT DRIVING support systems for drivers are now a new study area for Advanced Driver Assistance Systems (ADAS). When night driving support systems in the literature are examined; The driver is intended to perceive factors such as pedestrian, animal, warning sign, or road condition which the driver cannot perceive (such as being dark, the headlights can only illuminate certain areas, other light sources in the field of view adversely affect the driver's view, etc.). It is seen that night driving support systems are considered as transferring the warning information obtained using sensors (infrared / thermal vision, radar, ultrasonic sensors, etc.) that can detect these factors. The data is transferred to the driver via audio/visual warning systems or screen. The need for different analyzes and syntheses on the image for different road conditions, will cause the use of many different sensors or image processing methods / classifiers / machine learning methods. For example, infrared / thermal cameras can be used to detect pedestrian / animals or other objects. Infrared / thermal image will not be affected by the visible light in the environment, it provides us with information on the identification of pedestrians, animals, and objects. However, it does not include some data that we can obtain in a visible light environment. In this case, the driver needs to look at the infrared image being transmitted to the road and to the screen at the same time in order to receive all the information. but its continuity is not possible. Therefore, the warning system developed should automatically alert the driver.
However, the warning system cannot differentiate for different situations. The support system to be developed will warn the driver in the same way for an adult, child, elderly, or animal at the side of the road considering the current technologies. This will lead the driver to question the effectiveness of the warning level of the support system.
Whereas, all of these data can be used to improve the quality of the driver's view. The driver can make his own decision without the need for another support system for the analysis and synthesis of the image. Thus, the data obtained can be used effectively. At this point, it would be more useful to provide the driver with night vision support instead of or in addition to night driving support to prevent traffic accidents. Night Vision Assistance Systems (NVAS), the driver to provides driving the vehicle by looking at the enhancement image. In this case, the driver will not look at the road. Instead the driver looking at the screen presented to him will use the vehicle. There are advantages to using a screen view instead of An Image Enhancement Method For Night-Way Images B. TURAN  N a road view. At the same time, there are also disadvantages. Any problems that may occur on the screen may cause the sight to disappear completely. In addition, the change of the driver's point of view does not have an effect on the image, this is also may make it difficult to use. With another approach, NVASs can also be used via transparent screen glasses which can be used as screens but do not obstruct normal view. Thus, even if the NVAS is switched off completely, the driver does not experience any loss of view. However, he can adjust the viewpoint and orientation as he wishes. The data obtained by the sensors and cameras are created on the glasses and added to the normal view. To use this method effectively, transparent screen technologies need to work decisively. In addition, these technologies need to be accessible in terms of cost. But it is clear that it will not be an appropriate method unless these conditions are met.
Drivers, in case of hesitation or low visibility, can be imaging an enhancement image via on a screen besides the your field of views, until the technology level is available for the aforementioned applications. This way, the driver can continue his journey by looking at the view support screen when the road view is too low or When the sight is completely absent. Or, when the field of view is limited, can follow the enhancement view of the dark areas by the occasionally view looking at the support screen.
Driver night vision support areas: • Light sources that cause glare: Pupils are growing to collect better data in a structurally dark environment. However, if an intense light from any light source in the dark environment is directed towards the eye, the pupil becomes smaller and the perception becomes weaker because the light from other objects is less.
a-Front and rear headlights of traffic vehicles: Front headlights of vehicles which coming from the opposite direction (long-short). Divided roads bring down in a positively this glare, while traffic density increases in a negatively this glare. b-Road lightings: Although road lighting is intended to provide driver view support, there are places where it may adversely affect view. Especially when over-lighting in urban road lighting.
c-Illuminated billboards: The billboards of the facilities / businesses, emitting a large amount of light unconsciously located at the side of the road adversely affect the driver's view.
d-Field lighting: Intense and incorrect lighting of roadside construction sites, factories, private properties and so on. adversely affects the view of the driver. e-Sunlight: during the daytime driving the vehicle towards the sun (east-west) during certain hours, the quality of vision is very low due to the intense light.
• Areas outside the illumination area: There are areas within the driver's field of view that are not illuminated by the vehicle headlights. Providing opinion support in these areas will affect traffic safety positively. For example, a long headlight provides driver support by illuminating the area that the short headlight cannot illuminate, but adversely affects the traffic safety, as to dazzle the opposite direction driver eye. For this reason, NVASs can be developed with image processing methods or infrared illumination for these regions without using visible light.
In the study, it was aimed to eliminate the decrease in the quality of vision caused by the light sources that caused the glare at night driving and to increase the quality of vision in areas outside the lighting area.

II. PRIOR WORK
As the main subject of the study is to improve road images at night. Therefore, one of the objectives is to increase the visibility levels of pedestrians (all living beings) on the road. Because night-time driving is unsafe due to poor view of low contrast in the road image [1]. There are many studies in the literature about pedestrian detection [2,3,4,5,6]. In some of these studies fixed camera [2] and in the other studies carcamera [3,4,5,6] have been used. As a common feature, it was used IR images in all studies. In some of them near-IR (NIR) imager and far-IR (FIR) images [3], in others visible images and FIR images are used and compared. Especially when the aim is pedestrian detection, it is seen that FIR images came to the fore. Good results in areas where the visible light is very low in night road images and the fact that no IR light source is needed is a great advantage for the studies with FIR. However, if the FIR images which since there are visible off-band images are transferred to the driver as it is, it is difficult for the driver to drive the vehicle just by looking at these images. But, images can be processed and transferred to the driver via a separate screen with machine learning methods that detect pedestrians.
ADAS is an popular study area in recent years for researchers. Academic studies have been carried out to examine the close history of driver support systems [7] and to examine systems developed to strengthen night vision [8]. Some driver assistance systems to enhance night vision, that pedestrian (all vivid forms) images obtained from thermal cameras should be added onto the non-thermal camera image and suggest transferred to drivers [9]. In this way,, thermal and non-thermal camera images are added to each other and a driver support system is created and patented [10]. In these studies, the non-thermal camera images are enhancement and then the pedestrian images obtained with FIR cameras are added to the enhancement non-thermal camera image and support is provided to the driver by increasing the visibility of low visibility pedestrians images. One of the reasons for the decrease in visibility on night trips is the opposite headlights. Studies have been conducted to prevent the headlights from taking the eyes of the drivers coming from the opposite direction [11]. In some of these studies, the aim is not only to improve the image but also to support the driver in different areas (Lane monitoring and warning) [12,13]. In general, ADAS aims to warn the driver in many different areas (lane tracking and warning, distance tracking and warning, pedestrian detection, identification and warning of traffic information sign, detection and warning of traffic signals, and etc.) and to operate these systems on the same platform.
The topic discussed in the study is related to image enhancement. For this reason, during the preliminary study, image enhancement studies were examined. Especially complex images [14,15,16,17], foggy images [18,19] night images [20,21,22] / night road images [23,24,25], dimmed images with local bright areas [26,27], infrared images [28] were examined. There is no standard enhancement method (contrast enhancement, histogram equalization, sharpening, time / frequency domain filters and others) that can be applied to each image (dark, light, foggy, complex, and etc.) and can be given good results. Therefore, researchers are working on new / adaptive enhancement methods. However, better results can be obtained when specific methods are developed for specific images instead of adaptive enhancement methods for all images. On night trips (usually these trips are intercity), we can find and filter the light sources which into the driver's pointview image. Thus, the filtered image can be transferred to the driver by means of augmented reality applications to provide feedback to the driver. However, it is clearly seen in Fig.1 and Video 1 (clickhere) that the glare caused by the source cannot be obstructed even if the light source is filtered (dimmed). Because the glare has already occured on the camera sensor during image capture. In order to prevent glare, the light from the light sources must never reach the camera sensor. For example, it is very difficult to enhancement of the image given in figure (see Fig.2c) and to perform this process using simple image processing methods. Light source at the image given in figure (see Fig.2d), can also be filtered by simple image processing methods. However, the source is filtered, although the glare caused by the source cannot be prevented. Using a larger filter or an adaptive filter when the location of the source is detected will complicate the process. And filtering on larger areas will reduce the field of view. Because in this case, the filter means that all data in the filtered region is lost. Therefore, instead of improving the captured image interfering during the capture of the image can lead to a more efficient solution. Figures (see Fig.3a) and (see Fig.3b) images were captured in a similar manner to the image of figure (see Fig.2d) when the camera's flash was on and there was a light source in the environment. However, an object is placed in front of the source during image capture. As a result of this process, the resulting image appears to be of similar quality to that of figure (see Fig.2b) (no light source in the captured image). Figure (see Fig.3c) and (see Fig.3d) images were captured in a similar manner to the image of figure (see Fig.2c) when the camera's flash was off and there was a light source in the environment. However, an object was placed in front of the source during image capture. Even if obtained image is not in quality image in the figure (see Fig.2b) (no light source in the received image) as a result of this operation, It is clearly seen that it is much better quality than the image in figure (see Fig.2c). It is aimed to improve the night road images recorded in the study. Therefore, the effect of glare on the camera sensor during recording has already been recorded. Therefore, eliminate glare is not included in the study. Instead, it is the ultimate goal of the study to reduce glare in glare-forming images, and also to improve image quality in poorly damaged areas due to glare or insufficient light intensity. a.
b. Fig. 4. Night road image histograms [31,32] When the histograms of the night road images given in figure (see Fig.4) are examined, it is generally seen that there is an agglomeration in the dark area. For such images, contrast stretching, histogram equalization, sharpening can usually be sufficient to improve the image. However, as can be seen in figure (see Fig.3), standard contrast stretching, histogram equalization, sharpening, and etc. image processing applications are not sufficient to improve such images. These images are usually dark, but also include light sources directed to the driver in different paths and time zones. Thus, the driver's view is negatively influenced by the lack of light sources that support his view, and by the presence of light sources that negatively affect his view. The focus of the study is the adaptive improvement of these images, which are constantly changing.

A. Recommended Method
In the study to enhancement images, when calculating the new value of pixel: • The light sources size in the image, • Of the pixels whose value is to be calculated, distance to the light source.
• Neighboring pixels value's It is thought that it should be used as parameter of each. Thus, for each pixel, the new pixel value can be calculated in an adaptive manner. As a result, unnecessary glare and dark areas can be removed within the enhanced image.
To run the method, the color image is first converted to a binary image using a gray image followed by a threshold value of 0,9. In this image the center coordinates, boundaries and fields of light sources are calculated. The edges, numbers, center coordinates, and area covered of the light sources in the image were found using the Matlan funtions (bwboundaries, bwlabel, regionprops). Matlab codes prepared are below. As far as is observed, the glare in the night road images occurs around the light source. The glare is inversely proportional with the distance from the light source, and it is proportional the size of the light source. In accordance with this information, it is tried to develop an equation which uses the distance of the pixel from light sources, the size of these light sources as parameter and operates according to the intensity level of the pixels within the image.
In the equation (1), the distance of the pixel to the nearest light source and the radius of this source are used as parameters.
The equation produces results in the range 0-1. It is normalized with 1/20 due to its logarithmic properties. Has a sigmoid structure. Thus, it performs an effective classification in the range in which it is normalized. As grows (as the distance from the source increases), the multiplier converges to 1. As decreases, the multiplier converges to 0. Convergence to 0 does not cause problems because the pixels remaining within the light source limits are not processed during the process. Since it produces results in O-1 range, it decreases the glare of the glowing regions and ensures that other regions are not affected. Thus, the parameters used in the equation affect the pixel values as adaptive. However, in the image processing, constant c coefficient is used when image enhancement is performed by using logarithmic and force transformation.
Logarithmic transformation ( , ) = * ( ( , ) + )  5. Example images processed with logarithmic and force transforms [31,32,33,34] An adaptive coefficient of improvement in logarithmic and force transformations is not used. Logarithmic or exponential processing of pixel value, it results in a direct proportion to the pixel value. Logarithmic transformation in some images commonly used in the study, some of them give good results in the force conversion, while in some images both are poor. Sample image results processed by logarithmic and force transformation are given in figure (see Fig.5). Images processed by equation (1) are shown in figures (see Fig.7, Fig.8, and Fig.9). The equation (1) that is used decreases only the brightness of the glowing regions as adaptive. However, another aim of the study is to increase the brightness levels of the regions with low visibility. For this, the equation needs to be developed. It is especially desirable to increase the brightness in areas where the image is dark. Thus, the visibility level of the regions with low visibility is increased, but the visibility levels of other regions will not be affected. It can use the pixels value to achieve of this effect. However, this study, 4 neighboring averages of the pixel were used in order to prevent the process to be negatively affected by noise. The 4 neighboring averages of the pixel was normalized with 1/255. And to the equation was added as + − ( , )/ (4). The value of the added portion converges to 1.36 as the average grows. The average as it shrinks converges to 2. Thus, visibility of regions with low visibility is increased more than in other regions. Equation (4) is obtained by using neighboring pixel averages as parameters. Images processed by equation (4) are given in figures (see Fig.7, Fig.8, and Fig.9). In such images, setting the value of pixels to increase the visibility level with only 4 neighboring pixel averages is a deficiency. Despite and are used to update the values of glare pixels, they was not used to update the values of pixels with low visibility. 2 was normalized with and added to the equation as 1 + −( 2 ) (5). In cases where the light source is small and the pixels are far from the source, the equation is converges to 2. In cases where the source is large and the pixel is close to the source, the equation converges to 1. Thus, the part added to the equation will contribute to a more effective increase of the visibility level of the pixels in regions with low visibility, similar to the 4 neighboring average. Instead of using a constant coefficient as in logarithmic and force transformations, when the equation is created in this way, it gets special adaptive properties for night road images. The final form of the equation is given below (5).
The equation was applied to different night road images recorded by the researcher and collected from the internet. Examples of the general night road images obtained were shown in figure (see Fig.7), Video 2 (clickhere) and Video 3 (clickhere). In addition, the developed equation was applied to the images containing the detail (human, animal, and etc.) in the regions with low visibility. The operation of the proposed method is given in figure (see Fig.6). The resulting images were shown in figure (see Fig.8 and Fig.9).   7) and video 1 examined, of the developed method is seen that stable and good results on night road images. Image at the stage of improvement is processed separately for each pixel. It is considered that if necessary work is done, it can be used as image filter. When examined in figure (see Fig.8 and Fig.9), it has been clearly observed that the level of visibility within the image of human/animal/object in areas of low visibility is increased.
In the study, it has been tried to develop an image enhancement method to increase the visibility level in the regions with low visibility on night road images. At the same time, this method should ensure reduces glare in the regions where glare occurs, or should ensure at least the glare does not increase. As a result, the expectation in the output image is generally a decrease in the pixel brightness values of the regions around the light sources. In addition, it is aimed to increase the brightness value in dark regions where visibility is low. Especially in the output image, changes in the pixel values in the regions with these characteristics are expected.   Structural similarity maps include changes in pixel values. For this reason, the structural similarity map is created of the output images which was obtained. Brightness, contrast, and structural components are used when evaluating structural similarity [29]. The Structural Similarity Index Maps (SSIM) contain visual information about the regions that have been modified by processing the output and input images together. It is considered that it would be meaningful to evaluate the results of SSIM index map and output image together in the specific image enhancement studies as in the study. Because the location of the change of pixel values is important in specific image enhancement studies. SSIM and mean SSIM equations are given below (6),(7),(8), (9), (10). SSIM defined as [29]: The structural similarity is calculated for each pixel and the structural similarity is generated in the index map. In the study, the structural similarity index maps were created by using input and output images and given in figures (see Fig.7, Fig.8, and Fig.9). The structural similarity index maps have value of 1 in unchanged pixels. If the value is in the changing takes a value between 0-1. If the change in the pixel value is small, it produces a value close to 1, and the change in the pixel value is big, it produces a value close to 0. Thus, the regions where the change is very high are expressed as dark, and the regions where there is little change are expressed as light colors. Figures (see Fig.7, Fig.8, and Fig.9) output images and the SSIM index map together examined, there is no seen change in the regions around the light sources or there is seen change in the direction of glare reduction. Whereas, in regions with low visibility (dark), it is seen that there is more change (in the direction of increasing the level of brightness). The structural similarity mean (mean SSIM) can be taken in order to give an overall quality measure of the image studied [24,29]. The mean SSIM equation (11) is given below for color images. Mean SSIM defined as [29]: , [29] Mean SSIM quality values are given in Table 1.
In addition, peak signal to noise ratio (PSNR) was used as the criterion of success in the evaluation of the study. PSNR adaptive histogram equalization [17,28,30] is widely used as a measure of success in studies. The PSNR is based on the input and output images, and acts as the error between the input image pixel value and the output image pixel value. For this reason, it is reasonable to use it MSE is defined as [28]: Generally, quality criteria are expected to be high in image enhancement studies. This is also the case for PSNR [17,28,30] and SSIM [24]. High PSNR and mean SSIM values mean that the amount of change in pixels is low. For example, when there is no change between input and output pixel values, PSNR takes infinite and SSIM takes 1. So they get the maximum value they can get. Therefore, it is generally considered appropriate to take reasonable values in line with the changes expected from the improvement process. In image enhancement studies, the quality criteria obtained from the methods that are expected to do the same work are compared [16,17,24,27,28,29,30]. In this study, a specific study was performed to improve night road images, so no comparison could be made with any method. However, it can be said that PSNR and SSIM values are within reasonable limits when compared with the values in studies using PSNR and mean SSIM [17,24,28,30].