Automatic Detection of Regulatory Traffic Signs via Circle Detection by Post Edge Detection Applied to Straight Line Hough Transform

Automatic traffic sign recognition is a crucial application of Driver Assistance Systems for safe and comfortable driving conditions. The major step of traffic sign recognition is to detect and localize the traffic signs if they exist. An important portion of traffic signs (i


Introduction
Advanced driver assistant systems are vital applications which are mostly dedicated to decrease the risk of traffic accidents. Traffic accidents are among the most important risks for human health throughout the world and they are major social and economic issues for all the countries [1]. With the current advancements in technology, different solutions have been proposed to reduce the risk of traffic accidents. Advanced Driver Assistance Systems (ADAS) are among these solutions which prevent accidents or mitigate their consequences [2]. Driver Assistance Systems are designed to guarantee the comfort and the safety of the drivers [3]. An important component of ADAS is the traffic sign detection and recognition (TSDR) systems [3].
In the literature, there exist important number of studies which design systems or algorithms for TSDR technology.
The first attempts on automated traffic sign detection can be traced back to 1984 in Japan [4]. Since this date, researchers throughout the world tried to find different TSDR solutions for the traffic signs in various countries. In [4], for Malaysian traffic signs, the authors built an automatic TSDR system by combining color segmentation, shape matching and support vector machine methods. In [3], for German traffic signs, the authors designed an automatic TSDR system by moment invariants and again support vector machines. In [5], for Chinese and German traffic signs, a real-time detection and recognition system is created based on simplified Gabor wavelets and convolutional neural networks. In [6], the traffic signs in European Union and the United States are compared to each other. It is mentioned that because they include additional text or warnings, traffic signs in United States are not straightforward compared to those in European Union. Finally, because of the increasing number of studies on this

Abstract
Automatic traffic sign recognition is a crucial application of Driver Assistance Systems for safe and comfortable driving conditions. The major step of traffic sign recognition is to detect and localize the traffic signs if they exist. An important portion of traffic signs (i.e. regulatory traffic signs) has circular shape. They include vital information about the traffic rules and regulations (especially the speed limits). Therefore, this study introduces an innovative circular detection method to detect and localizes the circular traffic signs. In the literature, it is known that a circle creates a distinctive sinusoidal structure in Straight Line Hough Transform (SLHT). This study exploits this notion in circle detection by trying to catch a part of the envelopes of this sinusoidal structure. First, post edge detection is applied to SLHT, and this new image is called H-Edge image. Then, edge linking is performed on H-Edge image to obtain multiple candidate curves. By sinusoid curve fitting and sinusoidal normalization, the curves belonging to the sinusoidal structure are identified, so the circle in the original image is detected. Furthermore, a new iterative linear image segmentation method which is based on finding local minima in SLHT is proposed. Combining these methods and color filtering, a new innovative method for circular traffic sign detection is obtained. For certain sample images, the new method successfully detects and localizes circular traffic signs, and it is shown that it can perform faster than Circular Hough Transform methods.
Keywords: Automatic traffic sign detection; Advanced driver assistant systems; Circle detection; Image segmentation; Straight Line Hough Transform subject, a literature review study [7] is presented which provides an overview of traffic sign detection and classification methods from some recent studies.
TSDR systems have two main stages i.e. the first stage to detect and localize the traffic sign if they exist, and the second stage is to classify and recognize the detected traffic signs [8]. In the case of any false detection by the detection system, the performance of the overall TSDR system can be quite low because the classifiers of recognition stage are not usually trained on false alarms. Therefore, as the primary stage, the detection of the traffics signs is significantly important for the success of the overall TSDR system.
Traffic sign detection can be handled by different ways such as only color-based detection, only shape-based detection, both color and shape-based detection and finally learning based methods [8,7]. Using only color or shape based detection is not suitable because there exist a high risk of false alarms when the image or the video frame contains other objects having same colors and shapes with the traffic signs [8]. Therefore, first using the color detection and then engaging shape detection prevents any possible intervention of other objects [8].
The dominant colors of the traffics signs are red, blue and finally yellow [7], so the color based detection is primarily focused on these colors. And, the major shapes of the traffics signs are triangle, rectangle, octagon and finally circle [8]. There are five basic traffic sign classes i.e. warning, regulatory, obligatory, priority and informative [6]. Among these groups, the regulatory traffic signs include very vital information about the traffic rules and regulations (especially the speed limits) that the drivers must follow. And, with a few exception, all the regulatory traffic signs has a circular structure. Therefore, circle detection is an important component of detection of regulatory traffic signs. Therefore, this study introduces an advanced circle detection algorithm to more effectively detect the regulatory traffic signs in TSDR systems.
In image processing, Hough transform (or similarly Radon transform [9]) had initially intended to detect lines within images. Existence of a line within the edge detected image results in a peak value at the Hough transform [10]. After introduction of Hough transform, extensions to the Hough Transform emerged in to order to generalize the idea of Hough Transform to detect various different shapes and structures. These shapes include analytical curves such as circles, ellipses or parabolas as well as arbitrarily complex shapes [11]. Therefore, in order to prevent the ambiguity, the original Hough transform is sometimes referred to as Straight Line Hough Transform (SLHT).
While the extensions to Hough transform to detect different shapes and structures are useful to detect only that specific shape or structure, Hough transform for straight line detection has a strong capability to provide various information about the original image. Previously, circle detection by SLHT was discussed by means of binary (not a gray scale) Hough transform which was obtained by pixel contiguity [12]. However, no recent study exist to extend the idea of circle detection by SLHT or Radon Transform in the relevant way. Moreover, even though it was directly or indirectly observed that local minima in SLHT represents the linear gaps in the images [13], this idea has not been engaged in segmentation of images.
Finally, by collecting all these ideas into one framework, this study introduces a new method for multiple circle object detection and image segmentation by means of only Straight Line Hough Transform. In this paper, first the background of this study is briefly discussed. Next, the new proposed method is introduced and explained in detail. Then, the new method is used in detecting the regulatory traffic signs in the sample images. Consequently, the execution time analysis which compares the new method with some previous methods is presented.

Background
In this part, the background of this research is provided. First, the basics of Straight Line Hough Transform (SLHT) are presented. Then, the previous research about use of SLHT in circle detection and image processing is briefly mentioned. In this study, MATLAB® software is used [14].

Straight Line Hough Transform (SLHT)
In SLHT, a point within an edge detected image is represented by means of all possible lines which pass through this point, where lines are represented by the polar parameters as shown in Eq. (1) and in Fig.1.a: where is the distance of the line to the origin, and is the angle between the x-axis and the direction passing from the origin and perpendicularly intersecting the line ( Fig.1.a). Therefore, for a binary image which includes only a single point ( , ), the SLHT contains a curve representing the pairs of 's and 's which satisfies Eq. (2): Based on this equation, it can be shown that each point in an edge detected image is mapped to a certain sinusoid in SLHT space. Therefore, SLHT of any binary image is just a collection of sinusoids with different phases and amplitudes.
When three points are aligned along a line as shown in Fig.1.b, the SLHT sinusoids of these points will intersect just at the polar parameters of this particular line as shown in Fig.1.c. Therefore, in complicated images, finding the locations of the peaks within SLHT allows us to find the lines within the original image. While presenting the Hough transform, the range of is set as (-90,90], and the range of depends on the size of the original image. The axis of is usually reverted.

Use of SLHT in Circle Detection and Image Segmentation
Although SLHT is originally developed for line detection, it can also used in detecting other objects. In [15], it is visually demonstrated that given a certain shape in the original image, the Hough transform of has a distinct structure. And, the envelopes of this structure can provide necessary information to detect the shape in the image. In [12], it is first discussed that the circles can be detected by SLHT. First, they mention about "Hough function" as the function of polar parameters of the tangent lines to the curve existing in the image. After determining the tangent lines to curve based on pixel contiguity [16], they create a binary image as the representation of SLHT of the curve. Therefore, in their study, SLHT of the curve only includes Hough Function of the curve in the binary form without any gray colors. They mention about the necessity to detect Hough functions in Hough Space, so by selecting three points from Hough functions, they extract the necessary parameters for the representation of the curve (circle or ellipse) existing in the image. In [17], again by means of binary SLHT (not gray scale SLHT or accumulator SLHT), ellipses are detected which are existing in an image.
In spite of these early studies, there exist only few recent studies which try to improve the concept of "circle detection by SLHT" with the current computational opportunities and software from the perspective mentioned above. Some studies have focused on circle detection by SLHT or Radon Transform [18,19,20], yet they are building algorithms from very different perspectives which are not matching the ideas mentioned in this study.
Image segmentation is the process which aims to divide the image into meaningful parts to be able to manage the image more effectively. In the previous literature, there are studies which engages Hough transform to segment the structures like scattered blood cells [21,22], however, these studies do not engage the local or global minimum in 2-D SLHT (or Radon transform) space to linearly and iteratively segment the images. There are studies which engages local minima in 2-D Radon Transform space only to average the blur lengths in motion blur estimation [13]. Consequently, this paper will introduce a new advanced circle detection and images segmentation method based on SLHT.

Methodology
In this part, the new proposed methods for circle detection and image segmentation which are primarily based on SLHT are introduced and explained in detail. First, the necessary terminology together with necessary mathematical background are introduced. Then, the proposed circle detection method is comprehensively explained. Finally, image segmentation based on SLHT is discussed.

Terminology and Mathematical Details of Hough Transform of Circular Shapes
In this section, the terminology and some observations related to SLHT are presented.
Hough Curve: In the literature [15], Hough Curve is the name of a single sinusoid in SLHT space corresponding to a unique point within the edge detected binary image.
A single point within an image creates a sinusoid (also called sinogram in Radon transform) within SLHT. The Hough curve of a single point ( , ) can be written as: where √ 2 + 2 is the distance of this point to the origin. Consequently, based on trigonometric identities, the following equation which relates to can be written: where and can be defined as follows: Therefore, it can be observed that the Hough curve of a single point is a cosine function (i.e. a sinusoid) with the parameters and [15]. Therefore, all the Hough curves are sinusoids.
Hough Stripe: An arbitrary closed curve in Fig.2.a produces a stripe within SLHT as can be seen in Fig.2.b. In this paper, this structure is named "Hough Stripe" of this specific closed curve. Hough stripe is constituted of Hough curves corresponding to the points lying along this closed curve. Hough stripe depends on the characteristics as well as the place of the closed curve within the image.
The vertical thickness of Hough stripe at a specific value represents the projected size of the closed convex shape with respect to angle as in Fig.2.c. Therefore, an ellipse as shown in Fig.2.c has a non-uniform stripe as can be seen in Fig.2.d. This non-uniform Hough stripe occurs because of the changing thickness of the ellipse with respect to different directions. Finally, because a circle as shown in Fig.2.e has always the same thickness from all directions, it must have a uniform stripe as seen in Fig.2.f.
As can be seen in Fig.2.f, upper and lower edges of the Hough stripe of a circle are more intense with respect to the interior region. Edges of the Hough stripe represent the tangent lines of any closed shape at the corresponding angle [23]. Because the closed shape can be approximated as a tangent line around a specific point, all of the neighboring points slightly contribute to the tangent line, which at the end makes the edges of the Hough stripe more intense. Finally, a circle has a Hough stripe with regular distinctive edges, and this property will be utilized in this study to detect the edges of Hough stripes of the circles by post edge detection applied to SLHT.
The edges of a Hough stripe of a circle must follow the pattern of the Hough curve of the center point of this circle. Therefore, the edges of Hough stripes can be formulated as the vertically shifted versions [12] of the sinusoid of the center point as follows: where R is radius of the corresponding circle.
H-Edge Image: In this study, the circle is detected by applying a post edge detection to the Hough transform. Consequently, the image obtained by applying edge detection to the Hough transform is called "H-Edge image" throughout the paper.

Circle Detection by Post Edge Detection on SLHT
In this section, the proposed circle detection method is described in detail. Fig.3.a demonstrates a sample image which contains a circle, and Fig.3.b shows its Hough transform. The Hough transform of this image can look complicated at the first glance, however, it is not difficult to observe the uniform Hough Stripe of the circle which is pointed by the arrows in Fig.3.b. To emphasize again, having uniform intense edges is the major property what makes the circle's Hough stripe distinguishable. The main idea of the new method is that the circle in the original image will be detected with all its parameters (i.e. coordinates of the center and the radius) if even a small part of any of two edges (upper or lower envelopes) of Hough stripe of the circle is extracted.
The procedure starts first by applying post edge detection to SLHT image as shown in Fig.3.c. The reason of using the term 'post' to describe this edge detection process is that edge detection is normally applied to the original images before Hough transforms. Therefore, this special image in Fig.3.c can be named Hough Transform Edge image or namely H-Edge image. H-Edge image consists of numerous edges which potentially include the parts of upper and lower envelopes of the Hough stripe of the circle. The points in H-edge image can be concatenated by edge linking [24] as seen in Fig.3.d to produce a list of curves which are all candidates to be a part of Hough stripe of the circle. Therefore, the edge links in Fig.3.d are all subjected to the following sinusoidal curve fitting: (̂,̂,) = argmin ( , , ) ∑( cos( − ) + =1 − ) 2 (8) where ( , )'s are the points of the associated edge link which contains different points. Some of the candidate edge links can be removed if they have only a few points (i.e. if they are not long enough for valid sinusoidal fitting) and if they produce large residual errors after sinusoidal fitting. In Fig.3.e, the remaining candidate edge links are presented where the edge link colored in red is the curve which belongs to the Hough stripe of the circle.
After conducting curve fitting for each edge links and obtaining a list of fitted sinusoids, the candidate circles corresponding to each of these fitted sinusoids can be obtained by means of the following equations: ̂= || (11) where (̂,̂) is the estimated center of the possible circle and ̂ is the estimated radius of this circle. Based on these parameters, the possible circles which are associated with the edge links in Fig.3.e are drawn on the original image as shown in Fig.3.f. As seen, the candidate detected circle #9 coincides with the real circle in the original image. Now, it is necessary to eliminate the fitted sinusoids which do not correspond to a circle in the original image. In order to achieve this task, the H-edge image is re-organized according to the inverted versions of the fitted sinusoids. This process is called "Sinusoid Normalization of the H-Edge Image". In Fig.3.g and Fig.3.h, the aim of this process is illustrated. When the H-Edge image is normalized with the inverted sinusoid, the edges of the Hough stripe are vertically linearized. Consequently, when the plot of sum of each row of the H-edge image after sinusoid normalization is obtained as shown in Fig.3.i, two (or more) peaks appear which corresponds to the upper and lower edges of the Hough stripe of the circle. If the fitted sinusoid does not correspond to any circle in the original image, these peaks cannot be observed.
Finally, in Fig.3.j, sinusoid normalization of H-edge with respect to the fitted sinusoid #9 is depicted where several peaks values appear in the plot of sum of each row as can be seen in Fig.3.k. When the plots of sum of each row are obtained for all fitted sinusoids, it can be observed that only the plot associated with the fitted sinusoid #9 contains significant peaks as seen in Fig.3.l. This means that only the fitted sinusoid #9 is associated with a circle in the original image. As s result, the circle in the original image is detected.

Image Segmentation by Detecting Minima of SLHT
In Fig.4.c, the iterative segmentation is depicted by means of different thickness for the lines segmenting the image. The thickest line is the first level segmentation, and the other lines are the segmentations for the next levels. Image segmentation stops when there exists no significant minimum point in the Hough transform. If the circle detection method described in Section 3.2 is applied to each sub-image obtained after image segmentation, all the circles within the image can be conveniently detected as shown in Fig.4.c. Consequently, by linearly dividing complex images, the complexity of Hough transform is reduced to make the Hough stripes of the circles more visible. Finally, combination of these two tech-niques provides us with a very effective way for multiple circle detection in traffic sign detection for regulatory traffic signs.

Detection of Regulatory Traffic Signs
As described in Section 1, the major aim of this study is to improve Driver Assistant Systems by introducing an effective circle detection method. In this part, the new circle detection method proposed in Section 3 will be applied to two different RGB images shown in Fig.5.a and Fig.5.b to test its success in detecting the regulatory traffic signs. In this section, first the RGB images which simulate the images obtained by a dash camera within a car are constructed. Two different photo-realistic RGB images are constructed which contain different regulatory traffic signs with different size and colors. In Fig.5.a, there is a speed limit traffic sign with a red color located near a two lane road. In Fig.5b, there are three different speed limit traffic signs with orange LED lights located above a highway. In each image, there exist also other objects just like trees, other vehicles etc. to make the images more complicated.
The flowchart of the overall traffic sign detection algorithm is depicted in Fig.6. First, the RGB images are subjected to color filtering as seen in Fig.5.c and Fig.5.d. The RGB images are first converted to HSV images, and then red and orange colors in the images are maintained while the other colors are filtered out. For other traffic signs, color filtering can be also applied for blue and yellow colors. After color filtering, the images are ready for edge detection. In Fig5.e and Fig5.f, the images after edge detection process are depicted. Because these images are too large and complex, the complexity of these images can be reduced by image segmentation as described in Section 3.3.
Finally, the circle detection procedure for each sub-image can applied to detect the circular structures of the traffic signs. As seen in Fig5.e and Fig5.f, all the regulatory traffic signs are successfully detected. Fig.6 also summarizes the steps of the proposed circle detection as a flowchart. Fig.6. The flowchart of the overall process to automatically detect the regulatory traffic signs.

Execution Time Analysis of Circle Detection
In this part, the execution time of the new circle detection method is compared to those of certain Circular Hough Transform methods. Circular Hough Transform is a well-known method which requires a peak detection procedure within a three-dimensional space [11] to detect the circles. Therefore, it has quite different detection philosophy than the new method described in this study.
First for the sample images shown in Fig.5, the sub-images are obtained after color filtering, image segmentation and edge detection. Then, the execution times of circle detection methods are recorded for only the sub-images which contain circular traffic signs. In Fig.7, the average execution times of the circle detection algorithms are presented. There exist two different Circular Hough Transform methods. First method is the 'Hough Transform for Circles' method by [25]. The second method is the MATLAB's built-in function (i.e. imfindcircles) [14] which is basically based on Circular Hough Transform.
As mentioned in Section 3.2, the new method is primarily based on the SLHT of the images; therefore, the resolution of the SLHT affects the execution time of the new method. The resolution of the SLHT can be adjusted in two ways. Both the size of interval (namely ) and the size of interval (namely ) can be increased to decrease the resolution of SLHT and to attain an optimum execution time. Of course, a very low resolution of SLHT destroys the circle detection capability of the new method. However, it is observed that the new method can be quite successful in detecting circles with moderate levels of SLHT resolution.
In Fig.7, the method labeled as CD-SLHT( =1) is the new method i.e. circle detection by SLHT where the interval is set to be 1 degree. The execution time of the new method is plotted with respect to increasing interval. The method labeled as CD-SLHT( =3) is the new method where the interval is set to be 3 degrees. As can be seen, the execution time of the new method is decreasing with the increasing the size of both and intervals while the new method preserves its capability to detect the existing circles. Finally, as can be seen, the new method i.e. CD-SLHT can perform faster than both of the Circular Hough Transform methods. Therefore, the new method appears as an effective circle detection method with an efficient execution time.

Conclusion
In this paper, a new effective method for automatic detection and localization of circular traffic signs is proposed, explained in detail and finally tested on photo-realistic sample images. The rate of traffic accidents can be significantly reduced by means of Advanced Driver Assistant Systems. However, Driver assistant systems should be quite reliable to guide the drivers throughout their trips. Any false alarms by the driver assistant systems would confuse the drivers. Completely robust driver assistant systems can only be achieved by rigorous research on image processing, machine learning and so on.
In this study, the circle detection algorithm is based on Straight Line Hough Transform, and it provides a more organized and advanced solution compared to previous related studies. As discussed in Section 1, traffic signs can be in different shapes including triangle, rectangle, octagon and finally circle. Because Hough transform is basically designed for detecting the lines or line segments within the images, traffic signs with triangular or rectangular shapes can be very conveniently detected via Hough Transform. Consequently, because this study constructs an algorithm to detect circles via SLHT, all the traffics signs can be detected by means of a single framework namely SLHT.
Future research can include building a complete traffic sign detection as well as recognition system by enriching the current algorithm with new image processing or machine learning tools such as Deep Neural Networks and so on. Hough transform based traffic sign detection algorithm can be coupled by different machine learning tools where Hough transform is utilized in feature extraction. Hough transform of detected traffic signs can be also used in recognition of these signs. Finally, driver assistant systems can take the advantage of a new advanced TSDR system which is primarily based on SLHT.