Accuracy comparison of interior orientation parameters from different photogrammetric software and direct linear transformation method

The integration of computer vision algorithms and photogrammetric methods leads to procedures that increasingly automate the image-based 3D modeling process. The main objective of photogrammetry is to obtain a three-dimensional model using terrestrial or aerial images. Calibration of the camera and detection of the orientation parameters are important for obtaining accurate and reliable 3D models. For this purpose, many methods have been developed in the literature. However, since each method has different mathematical background, calibration results may be different. In this study, the effect of camera interior orientation parameters obtained from different methods on the accuracy of three-dimensional model will be examined. In this context, a test area consisting of 21 points was used. The test network was coordinated in a local coordinate system using geodetic methods. Some points of the test area were selected as the check point and accuracy analysis was performed. Direct Linear Transformation (DLT) method, MATLAB, Agisoft Lens, Photomodeler, 3D Flow Zephyr software were analysed. The lowest error value of 7.7 cm was achieved by modelling with Agisoft Lens.


INTRODUCTION
Photogrammetry involves scientific methods that calculate three-dimensional coordinates of an object by measuring the corresponding points in overlapping images. The mathematical relationship between an image point and an object point is derived by equinox linear equations based on central projection (Akcay et al. 2017). Photogrammetry is the most reliable and useful method for 3D modelling of the real world. Recording of historical artefacts (Duran and Aydar 2012;Ulvi and Toprak 2016), 3D modelling of the surface (Nex and Remondino 2014;Yemenicioglu et al. 2016), medical studies (Reis 2018) and in different situations where measurement must be made without contact with objects (Linder 2009), photogrammetry is widely used. Camera calibration is the determination of internal orientation parameters of the camera by the 3D coordinates of a point in space and the corresponding image coordinates (Song et al. 2013).
There are many studies about camera calibration that is used for enhancing 3D modelling. Zhao et al. (2015) were developed faster calibration method. They used a matching method based on heterodyne multi-frequency phase-shifting. Root mean square error (RMSE) was obtained as 2.5 cm. In another study, self-calibration of range cameras was realised using bundle adjustment (Lichti et al. 2010). 3-D coordinate errors were reduced by up to 74%. In addition to these studies, there are also comprehensive studies that examine calibration methods in general. In the study conducted by Hemayed (2003), self-calibration methods for determining interior parameters were examined. In another large-scale study, calibration methods were examined as traditional camera calibration method, camera self-calibration method, and camera calibration method based on active vision (Song et al. 2013).
Among the existing methods, Structure from Motion (SFM) is a popular algorithm. This algorithm creates 3D models using photographs taken from different angles of an object. The positional accuracy of the created models is affected by camera calibration. Therefore, accurate calibration of the camera is important in terms of 3D modelling. In this study, the effect of camera calibration values obtained from different popular software on 3D model accuracy was investigated. MATLAB, Agisoft Lens, Photomodeler, 3D Flow Zephyr and Direct Linear Transformation (DLT) methods have been selected. A three dimensional test area was created to evaluate the calibration results.

Material
Application was carried out with a Nikon D800 camera ( Figure 1). The camera with a variable lens is set to a focal length of 24 mm.

Figure 1. Nikon D800 Digital SLR Camera
Within the scope of the study, a test network was established. The test area contains 21 points ( Figure 2). The local coordinates of the points were determined by geodetic measurements using Total Station. It has 3 mm + 2 ppm distance accuracy and 3" (0.9 mgon) angle accuracy. Points with different height values have been established for an accurate assessment. The height varies between the lowest and the highest point by 10 cm. Photos of the test area were taken at a distance of approximately 50 cm.

Camera Calibration
The camera calibration is one of the classic problems of the field of photogrammetry. Calibration of a camera can be regarded as the inverse of photogrammetric process. In the photogrammetric process, orientation parameters are known and coordinates of the object points are searched, but in camera calibration, the coordinates of the object points are known and the elements of the internal orientation are searched (Kraus 1993). Camera parameters can vary with temperature, humidity, atmospheric pressure, and the camera must be calibrated from time to time for the detection of parameters. (Song et al. 2013). Since the study was carried out in a laboratory environment and its atmospheric conditions were standard laboratory conditions (25 °C at 100 kPa).
Camera calibration is performed to obtain the interior orientation parameters of the camera. With these parameters obtained as a result of the calibration, the spatial beam is fixed to the projection centre (Ozdemir and Duran 2017). Interior orientation parameters are calibrated focal length c, coordinates of principal point coordinates (x0, y0) and distortion parameters. When the camera focuses on a point, the focal length is represented by c. The focal length should be precisely determined because it affects the coordinates due to the mathematical model of photogrammetry. Most of the cameras used in photogrammetry produce photographs which can also be considered central projections of sufficiently accurate spatial bodies. The central point of the central projection is called the projection centre. The projection centre's projection point on the image is called the principal point.
Radial distortion is the image displacement that occurs when the rays coming from different angles to the lens focus on or behind the projection plane due to angular magnification caused by the lens. Radial distortion affects the position of the point on the image radially. Radial distortion should be modelled with high accuracy because of its positional effect on coordinates. The tangential distortion occurs if the lens elements and the centres of the image sensor are not coincident and their planes are not parallel (Ozdemir and Duran 2017). The image coordinates with radial and tangential distortion (x', y') formulas are shown in equation (1) and (2).

APPLICATION
In the scope of the study, the camera calibration was performed. Thus, the internal orientation parameters of the camera have been determined. The camera was calibrated using each method and software. Each software has a calibration pattern. The calibration process is performed and the results are shown below.

Calibration with Agisoft Lens
Calibration with Agisoft Lens was performed on the computer screen using the test area of the software. The test area, similar to the chessboard, is shown in the Figure 3. The captured images are used in the calibration process via the software interface. 13 photos of the calibration paper were taken from different angles and evaluated through the software. As a result of the process, interior orientation parameters were calculated. The interior orientation parameters are calculated in Agisoft Lens as pixels are transferred to the Table 1 in mm.

Calibration with Photomodeler
Calibration with Photomodeler software was done by printing the calibration network of the software on A4 paper. There are 100 control points on the calibration paper (Figure 4). 13 photos of the calibration paper were taken from different angles and evaluated through the software. Calibration results are kept in a file with the specific extension of the software and the parameters are given in the metric system ( Figure 5). The calibration results are shown in Table 2.

Calibration with 3D Flow Zephyr
The program uses the Procedural Perlin Noise image (Figure 6), unlike other software for calibration. The image is reflected directly on the computer screen via the software. 13 photos were obtained.  After the photos are uploaded, the software performs the calibration on a single window. The obtained calibration values were shown in Table 3.

Calibration with MATLAB
Camera Calibrator is used in the computer vision toolbox for calibration via MATLAB. MATLAB Camera Calibrator estimates camera interior orientation, exterior orientation, and lens distortion parameters. The program benefits from previous studies for necessary calculations (Zhang 2000).
The test area used by the program is in the form of a chessboard. A checkerboard image can be created within the software in different sizes and dots. The left half of the checkerboard image is in black and white and the right half is in black and grey to define the coordinate system (Figure 7).

Figure 7. MATLAB calibration test area
When the software completes the calibration process, it sends the results and errors to the MATLAB workspace as a variable. The interior orientation parameters produced with MATLAB are shown in Table 4. Direct Linear Transformation (DLT) method is a linear calibration method. It was developed in 1971 by Abdel-Aziz and Karara (2015). The major advantage of this method is that the solution is linear and does not have an approximate value problem. With DLT equations, it is possible to reach the space coordinates directly from the image coordinates (Tasdemir et al. 2009). In addition to the parameters added to the 11 parameters, DLT equations are given in the following equations. There are 16 parameters in direct linear transformation method. 11 are used for conversion.
The parameters from L1 to L11 are the camera calibration parameters. L12, L13, L14 related to radial distortion, L15, L16 are the parameters related to tangential distortion. The parameters were calculated using MATLAB (Table 5). The calculated parameters are as follows. Calibration with DLT was performed on the prepared 3D test area. In the equations (3) and (4), the unknown object coordinates are x, y, z. At least three equations are required to solve a system with three unknowns. It is not possible to solve the system, since two equations for a point can be obtained from one image. However, four equations can be obtained for one point from two images and x, y, z unknowns can be calculated. For 3D coordinate calculation, DLT parameters must be calculated on at least two images. Below are the points of the two images ( Figure 8).
The interior orientation parameters obtained by using the DLT parameters are as follows (Table  6).

Comparison of Calibration Parameters
The obtained calibration parameters were compared and visualized using graphs. For focal length, the methods gave similar results except DLT. The focal length that was computed by DLT, had higher value. The closest value to the prior focal length value (24 mm) was the computed focal length by 3D Flow Zephyr software. The proximity to the prior value is not meaningful for photogrammetry. The method that can best detect the change in focal length gives more accurate results.     Significant results were obtained at the tangential distortion. While the methods outside the DLT were calculated to be almost 0, DLT calculated high value tangential distortion. It is note that the interior orientation parameters have been calculated with different values for each method. The effects of the changes on the accuracy of the 3D model to be produced was examined.

RESULTS and DISCUSSION
A test area of 21 points was used in the study. 11 points of the test area were identified as control points and 10 points as check points. A 3D model was created in Agisoft Photscan by using 15 images taken with Nikon D800 camera (Figure 9). The model is coordinated and scaled in the local coordinate system. Agisoft Photscan is a software that uses Structure from Motion (SFM) algorithm. Features are automatically extracted from the images and matched.
Camera calibration parameters can be entered as input to Agisoft Lens. Calibration parameters calculated from other software was given to the software by converting to pixel value. The software calculates and corrects 3D coordinates according to the calibration values. In addition, root mean square error (RMSE) was calculated by the software. The DLT method and the coordinates were calculated using its mathematical model. The root mean squared error values of the 10 check points was shown in Table 7.
The highest accuracy in terms of both planimetry and altimeter accuracy has been obtained with Agisoft Lens software. RMSEx, RMSEY and RMSEZ values were 0.011 m, 0.019 m and 0.110 m respectively. Calibration values calculated with Photomodeler, 3D Flow Zephyr and MATLAB are different especially in terms of image main point coordinates. This difference has led to incorrect calculation of the coordinates. The DLT method gave similar results to Agisoft Lens in terms of planimetric accuracy. However, the accuracy of the Z value is 4.629 m. Therefore, it is understood that DLT method cannot be used in terms of height evaluation.

CONCLUSION
The camera calibration technique has significant research and application value in the field of computer vision, and its precision directly influences its effect in three-dimensional modeling. In this study, the effects of different calibration methods on accuracy of 3D model were investigated. Mathematical reasons behind the different results of different methods should be examined. Because this situation affects the accuracy of 3D models. It should not be deduced that the software made the wrong calculation. The scope of the study can be extended by evaluating each software in itself. In future studies, accuracy research can be done for large digital elevation models. Similar studies can also be done for calibration of systems such as UAV. However, as a result of this study, inferences can be obtained for future studies.