Color Based Object Separation in Conveyor Belt Using PLC

In today's production systems, industrial automation systems are preferred in order to provide high-efficiency and high-quality production, and therefore it is continuously developing. PLC is the basis of programmable industrial automation systems. In the study, in the PLC controlled conveyor belt system, the process of separating the objects according to their color by image processing was realized in real-time. In the image processing application, recognition is recognized by separating objects according to their colors. The process of sending objects separated by classes to different containers in the system is also done in real-time. The conveyor belt system used in the study was controlled by S7-1200 PLC. Image processing was performed in Matlab. Communication between Matlab and PLC has been used with the OPC interface application, which is widely used in the industry. This study, experimental prototype of an automation system that is widely used in industrial applications, has been successfully made. Matlab program was used in the image processing part of the conveyor belt object separation system, PLC program was integrated into communication and control part, and it was seen in experimental studies that the system was working efficiently in real-time. It has been observed that the image processing algorithm for this study has been successfully performed to sort objects according to their colors, PLC-OPC system communication, and separation of objects according to specified positions. As a result of the studies on the experimental prototype, it is foreseen that this system can be applied to a commercial or industrial system.


Introduction
Automation systems take the place of manpower in production. The control of the working systems is carried out in a high quality and efficient manner without the need of manpower with automation systems. Programmable Logical Controller (PLC) is the basis of industrial automation systems. While PLCs offer a cost-effective solution for the control of complex systems, they offer advantages such as applicability to other systems quickly and easily, flexibility in system integration, resistance to electrical noise, vibration, and impact [1]. With the widespread use of automation systems, the workload demanded by PLCs has become too heavy to be performed by a single processor. In most cases, the meaning integrity and requirements of the process are overcome by using multiple PLCs. In the communication of these PLCs, it is mainly realized with the interface that serves high data rates such as Profinet, Modbus, and Profibus. With the additional modules added to the PLC, communication with RS232, RS485 protocols, and GPRS (General Packet Radio Service) networks can also be realized. In literature studies, object separation with PLC, conveyor belt system applications with PLC, image processing with conveyor belt, quality control in image processing system with conveyor belt, and so on. both academic and industrial studies are available in many different fields [2][3][4][5][6][7]. In the thesis prepared by Turgay Erdoğan in 2012, real-time object recognition and separation of objects were performed with the camera image taken from the band system using artificial neural networks, and studies were performed on various learning methods and performance [8]. In the article published by Sedat Metlek and Bayram Cetişli in 2012, the control of the band system created by using Festo experiment sets was carried out with the digital camera and image processing [9]. In the article published by Filiz Harmankuyu et al., The separation of three different objects in the tape system was controlled by using PLC and Arduino by separating the objects in Matlab environment according to their color and shape using a webcam [10].
In this study, as shown in Figure 1, in the PLC controlled conveyor belt system, the process of separating the objects according to their color was carried out in real-time. Another part of the study, image processing was carried out in Matlab environment. In the image processing stage, the object separation process is performed by taking the image of the objects on the conveyor belt in real-time and determining them according to the colors of the objects with the image processing algorithm. The purposes defined in Matlab are transferred to the PLC by OLE for Process Control (OPC) and carried out for object separation on the conveyor belt in accordance with the object information.

Operator Panel and OPC
OPC (Ole for Process Control) is a standard for the transmission of real-time data. It is used for communication of SCADA () modules such as PLC, actuator, and sensor used in industrial automation systems. Prior to OPC, each additional module manufacturer defined a unique communication structure. In this case, a software company developing an HMI had to customize the software is developed for each different PLC brand and model. By standardizing OPC communication, for example, HMI software can be developed as the standard for all PLCs. The OPC server is a protocol converter that converts communication of modules that do not conform to the OPC standard to the OPC standard. The first OPC standards were built on Microsoft COM. After 2010, the OPC Foundation began publishing OPC UA (Unified Architecture) features. With OPC UA server,  Web services and SOA (Service Oriented Architecture) architecture instead of COM,  Collection of OPC data models (DA, A&E, HDA) under a single service model,  Can be applied on platforms that contain embedded modules other than Microsoft,  Reliable and efficient access to highly structured data, are intended. The OPC server is, in short, a software application. Provides access to a set of data sources (read / write) or communication path. If the OPC client is, OPC is software used by an application to communicate with the server. The client acts as a data receiver, initiates communication, and provides control of the communication. Figure 2 shows the connection between the OPC server and the client. The OPC DA (Data Access) standard provides communication between the client and server and performs real-time data read / write functionality. Matlab -OPC Toolbox is an OPC DA client and can be connected to compatible servers. This enables the OPC server to be accessed in Matlab and read and write OPC data directly.

OPC Configuration Between PLC-Matlab
To enable OPC communication between PLC-MATLAB, the TIA Portal, and Matlab programs must be pre-set. The necessary actions are given below. Since the PLC project will be created in TIA Portal, OPC configuration is performed in the TIA Portal project. After a new project is created in TIA, PLC and PC station are added to the project via the "add new device ında tab and 'controller' and 'PC station 'interfaces used in the system. In the beginning, the empty PC station is added to the "IE General" communication module and "OPC server from the Catalog Hardware Catalog" tab. As a result of this process, Figure 3 is given the PC station, which forms the OPC structure created in the project. In the project tree for the IP configuration of the PLC, the IP assignment is performed by following the "Properties >> General >> Profinet interface >> Ethernet address" tabs of the PLC's properties and clicking the "add new subnet" button to assign a subnet to the PROFINET interface. The "Properties >> General >> Profinet interface >> Protection" tabs are followed, and the Allow PLC, HMI, OPC access via PUT / GET communication tab is activated. Figure 4 also shows the interface for the IP assignment. For the PC's network connection configuration, TCP / IPv4 from the Ethernet features of the PC's network and internet settings is first turned on, and an IP assignment is performed on the same network as the PLC. Figure 5 shows the interface where the IP assignment of the PC added to the network is presented. The IP assignment of the IE General communication module added to the PC station for OPC configuration is made from Properties >> General >> Profinet interface >> Ethernet address IE from the IE properties. Figure 6 shows the assignment of the IP address of the PC to the IE General module for OPC communication.   In order to configure the variables between PLC and Matlab in OPC, the 'Properties >> General >> S7 >> OPC tags un tabs of the OPC server are followed in Figure 9, and the configured option is activated. The XDB file is then created, as shown in Figure 10, to create the variables. The XDB file is created by selecting >> Properties >> General >> XDB Configuration PC from the PC station properties. In addition to the TIA Portal program, the Station configurator editor is opened from Simatic Net software for data access with the OPC server. With the interface shown in Figure 10, the XDB file containing IE General and OPC Server configurations is loaded by selecting 'import Station.' After these configurations are performed, and the PLC program is written, the PLC program is loaded to the PLC, and the OPC configuration is loaded to the OPC server. When loading into OPC, the OPC Station interface is created as in Figure 11. This indicates that the configuration of the OPC is correct. Figure 11. A structured view of the OPC Station interface OPC Scout software is supported by Simatic Net and used as an OPC client. Using the OPC Scout, the data of the S7-1200 CPU is accessed via the OPC server. The OPC server consists of three objects: OPC Server, OPC Group, and OPC Item. The OPC Item object is the smallest unit used for reading and writing data and is associated with the client's variables. The OPC Item object is accessed using group objects and is not available directly. Figure 12 Access to item objects using OPC Scout is right in 'DA view 1'; the connection of the variables through OPC is correct, and the 'Quality' value should be good as shown in Figure 13. Thus, it shows that the necessary configuration for PLC communication with Matlab over OPC is performed correctly [11][12].

Object Separation with Image Processing
Image processing is a process other than "signal processing", a study for converting data into a readable format on another device after capture, measurement and evaluation, or transferring it from one electronic medium to another. Image processing is mainly used to process existing images that have been recorded, i.e. to modify, alienate or improve existing images and graphics. In recent years, image processing applications are widely used in many fields. Some of these are industry, health, military, security, quality control, object separation, etc. there are many different fields. In some applications, defining colors has a very important place. In order to define colors, mathematical models which are expressed as color space have been developed. Color spaces are created to represent all colors. Color spaces are designed in 3D. Because according to Grassmann's first law, which forms the basis of the science of colorimetry, three independent variables are needed to determine a color [13][14][15][16]. Extracting information about the object contained in an image from the image is one of the primary purposes of image processing. The most basic approach for separating an object from an image is the extraction of visual attributes (such as shape, color, texture). Color spaces are mathematical models used to define colors. Color spaces are designed in three dimensions to represent all colors. Because according to Grassmann's first law, three independent variables are needed to determine a color. The place of colors in the color space is determined according to these variables [17]. Each color space has its own standards for color rendering. While creating color spaces, it can be transformed into another color space by linear or non-linear methods [18]. Commonly used color spaces, RGB (Red, Green, Blue), HSV (Hue, Saturation, Value) and NTSC (National Television System Committee) are the color spaces [19][20][21][22].
In this study, it is preferred to use HSV color space because it is closer to the human vision mechanism and provides color determination by looking at color essence and saturation levels without being affected by brightness changes. The HSV color space consists of three bands: H (color extract), S (saturation), and V (brightness). It is the color extract that determines the color and is defined by angular values and takes values between 0°-360°. Saturation refers to the vividness of color. A low saturation value brings the color closer to gray tones, while a high saturation value makes the color more vivid. Brightness determines the ratio of white in color, i.e., the intensity of the color. Figure 14. HSV color space Equations 1, 2, and 3 are used to convert an image of the RGB color space to the HSV color space. In this study, two different materials, yellow and gray, were used, and other materials were identified as foreign materials. A web camera is installed in the apparatus comprising the tape system so as to see the beginning of the system from the top. The transfer of the received image to the computer was realized via the USB connection. The image processing algorithm was executed via Matlab. Since the received image is in RGB format, the HSV color space is provided first. After the threshold values were determined for the objects used in the application, the acquired images were scanned pixel by pixel, and the pixels of the corresponding color were identified and masked. In the obtained image, the noise effect is suppressed by filtering. The detection of the object was carried out by removing unconnected small pieces.
The outputs of the program developed in Matlab are shown in Figure 15, Figure 16, and Figure 17.

Operating Principle of the System
Conveyor belts are transfer systems used to transport loads or equipment from the ground or at a certain height, at spaced distances. The conveyor belt system realized in this study is given in Figure 18. There are two conveyor belts in the system. In work done, the start command given to the system by pressing the start button ensures that the operations take place sequentially. With the start command, the upper band starts to work, and the camera sensor at the entrance of the system informs whether the object has arrived. Upon receipt of the object arrival information, the upper band moves for a while and then stops. This ensures that the object is entirely in front of the camera and, thus, possible errors that can result from misalignment. When the object is completed in front of the camera, the command 'Take Image' is sent from PLC to Matlab via the OPC server. The color of the object in the received image is determined by the developed image processing algorithm, and the object information is sent to the PLC through the OPC server. The information obtained from the PLC is written to an array, such as first in first out (FIFO). The reason for writing to the array is that there is a possibility of a new object coming in front of the camera until the object is imaged. When the first sensor (yellow object separation sensor) sees the object, the yellow object is decomposed from the upper band to the lower band by activating the corresponding yellow object output of the yellow object while the object information in the recorded sequence. If the object information in the recorded array is not the yellow object information, the object continues to move through the tape. The second sensor (gray object separation sensor) sees the object this time. If the object information in the recorded sequence is the gray object, the corresponding gray object output of the PLC is activated, and the gray object is decomposed from the upper band to the lower band. If the object information in the recorded array is not the information of the gray object, the object continues to move through the tape and is parsed as faulty material. As the gray object coming to the bottom belt moves through the band, it collects the yellow object at the end of the band. Thus, the separation of objects occurs. These operations continue as long as the stop button is not pressed. The system implemented in Figure 19, and the operating algorithm of the system is given in Figure 20.

Conclusion and Suggestion
In this study, the data obtained as a result of the image processing algorithm developed in the Matlab environment is sent to Siemens S7-1200 PLC via the OPC server, and the control of the experimental setup is synchronized in real-time. The control of the designed band system, image acquisition, and separation are performed in real-time. The web camera used in the system was used to differentiate colored objects, and it was found to be sufficient to recognize the object successfully. It was observed that the developed image processing algorithm was distinguished according to the colors of the objects in 5, 6 seconds. It is thought that using a highquality industrial camera to reduce this time will give more successful results. For this reason, it has been determined that using highresolution cameras and different lenses for industrial applications, more prosperous and faster responses can be obtained than applications. It was determined that the image processing application developed in different color spaces could be compared to success rates. In addition, synchronous operation of the image processing application with PLC, which is a must in many industrial applications, has been realized.
As a result of these studies, PLCs, which are among the indispensable products in the industrial automation sector, were processed in real-time in a conveyor system, image processing, and OPC was operated synchronously on a prototype test apparatus. It has been seen that the image processing algorithm for this study has been successfully performed to sort objects according to their colors, PLC-OPC system communication, and separation of objects according to specified positions. As a result of the studies on the experimental prototype, it is foreseen that this system can be applied to a commercial or industrial system.