ACTIVATION OF B-SPLINE BASES IN MACHINE LEARNING USE AS A FUNCTION Oybek Mallaev 1[0000-0002-1918-0641] , Fayzullajon Botirov1[0000-0002-2362-4153] Javohir Nurmurodov 1[0000-0002-9941-3969] Xudoyberdi Mamirov1[ 0000-0002-9850-1908] Sirojiddin Qobilov 1[0000-0002-1984-3855] 1. Department of Artificial Intelligence, Tashkent University of Information Technologies named after Muhammad al Khwarizmi , Tashkent, Uzbekistan nurmurodov1994@gmail. c om Abstract In the last decade, artificial intelligence and machine learning have become the most used areas for solving complex real-world problems. In particular, the problems that are considered very difficult or in some cases impossible by computers are becoming more and more interesting both scientifically and industrially. An example is Google's AlphaGo beating the world's best Go player using deep neural networks. At the core of neural networks is the socalled activation function of neurons. This feature is critical to network performance, but is often overlooked. In most cases, one of the non-customizable functions is selected as the activation function. There have been studies using adaptive sigmoid or ReLU functions, but this has drawbacks because the fit of data from one limited region affects the overall results. Therefore, the article suggested using flexible spline functions with free nodes. Research in the field of spline calculation is developing and expanding the scope of its application, and it is the quadratic B-splines that accept the approximation condition that have been studied the most. This paper considers B-splines as a basic activation function for machine learning, and thus makes it easy to use any degree of polynomial. The article provided information about regression splines and their advantages over linear and polynomial regression. Another way to produce splines is called spline grinding. It works similar to Ridge/Lasso adjustment. Applying these methods to data sets with high variability can make a difference. It is an efficient way to use splines as an activation function in neural network construction. In the course of this study, it was proposed to use the neural network built using B spline based on the geophysical signals of the probes to predict the location of underground ores, and a mathematical model and algorithm were developed. Based on graphs and tables, a comparative analysis was conducted and absolute and relative errors were evaluated. 1. Introduction Today's in the day scientists by geophysics in the field a lot research take is going Especially husband under fossil wealth in determining very a lot difficulties surface is coming Especially radiation its rays distributor radioactive of elements husband in old age how in depth location to determine present until the day problematic matter being is coming Various algorithms are proposed by scientists to solve this problem. However, there is a need for new models and methods in predicting the location of chemical elements in the earth's crust using terrestrial radiation signals, as there are large errors in the models and methods used today. One of these methods is machine learning. In machine learning , linear regression is one of the first algorithms to be studied . When it is applied to different data sets various advantages and limitations have been identified. It plays a key role in linking the linear relationship between local and global variables. As an improvement to this model, multivariate regression is used and it often gives better results. But using polynomial regression in data sets with high variability leads to overfitting. And when neural networks are built, they don't work well with the information in their hidden layers. Therefore, the use of regression spline functions is now defined . At the core of neural networks is the so-called activation function of neurons. This function is the most important for the fast and accurate operation of the neural network. In many cases, adaptive functions are chosen as activation functions. For example, sigmoid or relay functions are used. Their main disadvantage is that the values of the functions always lie between 0 and 1. These values are more needed when solving logistic regression problems. In order to process signals and make predictions correctly, the use of these functions in neural networks leads to an increase in absolute errors in the approximation process. Therefore, we suggest using flexible spline functions with free nodes. Splines are mainly used as activation functions in neural networks, and they have a certain universal approximation property. It is recommended to use B-splines to avoid overfitting in these functions . This is because B-splines satisfy the approximation condition and have less flexibility. In the functions that satisfy the interpolation condition , the feature of flexibility occurs more. 2. Spline functions. Univariate B-spline functions are said to be defined with respect to the segments of a smooth curve connecting a set of node points. Each part of the spline between two consecutive nodes is called a slice. In each slice, the spline is represented by a polynomial function of degree d . In the framework of the quadratic spline K , it focuses on the second-order polynomials, d = 2. Bi , 2 ( t ) = t − ti t −t Bi , 1 ( t ) + i +3 Bi +1, 1 ( t ) ti + 2 − ti ti +3 − ti +1 t − ti t − ti , ti + 2 − ti ti +1 − ti ti + 2 − t t − ti t − ti +1 t −t + i +3 , Bi , 2 ( t ) = ti + 2 − ti +! ti + 2 − ti ti + 2 − ti +1 ti +3 − ti +! t −t t −t i +3 i +3 , ti +3 − ti +1 ti +3 − ti + 2 0, (1) if t ti , ti +1 if t ti +! , ti + 2 (2) if t ti + 2 , ti +3 акс холда If we open the above formula in the time interval [0, 3], the following expression will appear. (t ) = t 1 2 if → t 0,1 2 2(t ) = 6 − 2t2−3t 3(t ) = 9 − 62t + t 2 if → t 1, 2 2 (3) if → t 2, 3 If we enter a value in the interval [0,3] with a step of 0.1, a graph in the form of Figure 1 will appear. Figure 1.1. A repeating B - spline in intervals. The spline function iterates over every three intervals and recovers the value of the signals. In the process of restoring the value of signals, it satisfies the approximation condition, and therefore, this function is less accurate than the functions subject to the interpolation condition. Based on this function, we perform one-dimensional geophysical signal recovery. i+d c(t ) = Qi Bi , d ( t ) (4) i Here is an intrinsic parameter of the spatial system that follows the t-spline curve, the c ( t ) = xc ( t ) , yc ( t ) coordinates of the spline curve evaluated at t in the Cartesian coordinate system x; y . Qi - the coordinates of the nodal point, n - the number of corresponding nodal points, d - the degree of the parametric curve, Bi , d ( t ) d-level mixed functions. Figure 1.2. Reconstruction of univariate signals based on a quadratic B spline. The graph above represents the approximation of the geophysical signal. Table 1.1. Results of approximation of univariate geophysical signal on the basis of quadratic B spline. No x i (t) b i (t) B i (t) 1. 0.381000 0.374333 0.377500 2. 0.428000 0.443667 0.404500 3. 0.381000 0.364667 0.404500 4. 0.432000 0.434500 0.406500 5. 0.468000 0.476000 0.450000 6. 0.456000 0.457833 0.462000 7. 0.433000 0.430500 0.444500 8. 0.425000 0.417167 0.429000 9. 0.464000 0.474333 0.444500 10. 0.441000 0.432333 0.452500 11. 0.470000 0.479167 0.455500 12. 0.444000 0.436667 0.457000 13. 0.462000 0.469667 0.453000 14. 0.434000 0.432333 0.448000 15. 0.416000 0.403833 0.425000 16. 0.471000 0.483500 0.443500 17. 0.451000 0.452500 0.461000 18. 0.422000 0.412167 0.436500 19. 0.452000 0.468000 0.437000 20. 0.386000 0.373000 0.419000 21. 0.398000 0.396833 0.392000 22. 0.417000 0.424500 0.407500 23. 0.391000 0.382500 0.404000 24. 0.416000 0.411000 0.403500 25. 0.471000 0.483500 0.443500 26. 0.451000 0.452500 0.461000 27. 0.422000 0.412167 0.436500 28. 0.452000 0.468000 0.437000 29. 0.386000 0.373000 0.419000 30. 0.398000 0.396833 0.392000 31. 0.417000 0.424500 0.407500 32. 0.391000 0.376500 0.404000 Table dvgi X i (t) - the value of the incoming signal. B i (t) is a three-point spline coefficient. C i (t) – signal approximation results. Table 1.2. Estimation of the errors of the approximation results of the one-variable geophysical signal based on the quadratic B-spline. No 1 B i (t) 0.017 C i (t) 0.0155 2 3.8% 3.5% In the above table 1 - an absolute mistake. 2 - relative error. Bivariate B-spline functions are the most important factor in the separate organization of nodes along the principal x and directions. y Indeed, to construct a parametric surface, it is essential to have an organized mesh network of nodes. A grid of nodes corresponds to a set of orthonormal coordinates of the corresponding nodes x , y, z . This distribution of nodes indicates the discretization of the parametric surface problem in slices. A slice is defined between the four nodes at its corners. It is assumed that there is an internal parametrization of the space in which the set of nodes belongs to a single plane. u , v Based on assumptions and considerations, we present the general formula of a parametric surface using a two-dimensional quadratic spline. i +d j +d S ( u, v ) = Qi , j Bi , d ( u ) B j , d ( v ) i (5) j Here: • u is the internal parameter of the spatial system that follows the parametric surface in the first direction, • v is an internal parameter of the spatial system that follows the parametric surface in the second direction, • S ( u, v ) = xs (u, v); ys (u, v); zs (u, v) Cartesian coordinate system evaluated at parametric surface coordinates , • d is the level of the spline in both directions, • Qi , j - management point coordinates , • Bi , d ( u ) and Bi , d ( v ) their own relevant internal parameters are B-spline functions of d -degree for . S ( u, v ) - this C 1 of the class function because _ bik vadrati k splines from the borders except all parametric on surfaces the first of derivatives continuity guarantees . If the reconstruction of two-dimensional signals is carried out using the formula (5) above, a graph in the form of figure 1.3 will appear. 1. Fig. 3 . Quadratic B - spline based bivariate signal reconstruction. The graph above illustrates the digital processing of two-variable geophysical signals. The highest peaks on this graph indicate that the amount of radiation is high and that the chemical elements are found in abundance in that location. No 1 C i (t) 0. 7 2 0.59 % In the above table 1 - an absolute mistake. 2 - relative error. As can be seen in the table, the absolute error is high. In order to reduce this error, we will consider building a neural network using B-spline functions and predicting the location of underground radioactive elements with this help. 3. Building a neural network using the B-spline function. BSNT , the following sequence is performed for each function. For this we use the following algorithm r 2 N = (i ( x) − yi ) i ( x) wi N i =0 r 2 N = (i +1 ( x) − yi +1 ) i +1 ( x) wi +1 N i =0 r wi + 2 ( x) = (6) 2 N (i + 2 ( x) − yi +2 ) i + 2 ( x) N i =0 Using the derivative to find the minimum and maximum values of each function, the weight of the constructed neuron is searched. 2 N (i ( x) − yi ) i ( x) wi = wi − N i =0 wi+1 = wi+1 − 2 N (i+1 ( x) − yi+1 ) i+1 ( x) N i =0 w i + 2 = wi + 2 − 2 N (i+2 ( x) − yi+2 ) i+2 ( x) N i =0 (7) In the process of the neuron's backward lo k al va global gradients appears b dies. During the descent from the local gradient to the global gradient, there are cases where the value of the gradient is not updated. In order to avoid this problem - the correct choice of the learning step plays an important role, and it is defined to take it in the interval [0,1]. Figure 1.4 . Note the gradient in one-variable signals . During the global gradient search, it is checked using the value function to see if it has reached the optimal value. Reading continues until its value reaches zero, or the required error, and each value check is one epoch. 1 . Figure 5. Minimization of the loss value based on the value function . rloss - training is carried out until the smallest value is reached. At each epoch, the gradient value is updated, increasing the prediction accuracy of the neural network. When the optimal value of the gradient is reached, the training process ends and the result is passed to the next hidden layer. Yi+1 Y Yi+2 back forward X Xi+1 Xi+2 1 . Figure 6 . B SNN forward and backward graph. B -spline based neural networks on the above univariate signals Once the global values of the gradient are found, we update the function values and try to recover the signal again. Figure 1.7. Reconstruction of univariate signals using neural networks based on quadratic B-splines. Let's compare the values of the signals recovered with the help of neural networks according to the table to see if they correspond to the real value. Table 1.2 . Results of approximation of univariate geophysical signal based on quadratic B - spline. No x i (t) B i (t) NT C i (t) 1. 0.381000 0.374333 0.409000 2. 0.428000 0.443667 0.404167 3. 0.381000 0.364667 0.399583 4. 0.432000 0.434500 0.455250 5. 0.468000 0.476000 0.466917 6. 0.456000 0.457833 0.444167 7. 0.433000 0.430500 0.423833 8. 0.425000 0.417167 0.445750 9. 0.464000 0.474333 0.453333 10. 0.441000 0.432333 0.455750 11. 0.470000 0.479167 0.457917 12. 0.444000 0.436667 0.453167 13. 0.462000 0.469667 0.451000 14. 0.434000 0.432333 0.418083 15. 0.416000 0.403833 0.443667 16. 0.471000 0.483500 0.468000 17. 0.451000 0.452500 0.432333 18. 0.422000 0.412167 0.440083 19. 0.452000 0.468000 0.420500 20. 0.386000 0.373000 0.384917 21. 0.398000 0.396833 0.410667 22. 0.417000 0.424500 0.403500 23. 0.391000 0.382500 0.396750 24. 0.416000 0.411000 0.447250 25. 0.471000 0.483500 0.468000 26. 0.451000 0.452500 0.432333 27. 0.422000 0.412167 0.440083 28. 0.452000 0.468000 0.420500 29. 0.386000 0.373000 0.384917 30. 0.398000 0.396833 0.410667 31. 0.417000 0.424500 0.400500 32. 0.391000 0.376500 0.424833 table, NT C i (t) is a one-variable geophysical signal restored on the basis of a neural network. 4. Neural network construction using bivariate B-spline function. In the process of digital processing of two-dimensional signals, neuron construction is carried out in the same sequence. To do this, the neural network built on the x -axis is now built on the u -axis, which is done as follows. Figure 1.8. Graph of processes performed in two-variable layers in BSNN. Neural network consists of I-layer input variables, II-layer hidden layer and III-layer output variables. In the hidden layer, a sequence of algorithms operates, similar to a neural network built for processing single-variable signals. While the neural network moves along the x -axis for univariate signals, it moves along the x- and u-axis for bivariate signals. And the result is transmitted to the outgoing network in the form of a matrix. Figure 1.9. A graph of part of a two-variable layer in BSNN. Since there are mainly three functions used when building neural networks even with a two-dimensional Bspline, once again the following sequence is performed for these functions. r 2 N = (i ( x) − yi ) i ( x) wi N i =0 r 2 N = (i +1 ( x) − yi +1 ) i +1 ( x) wi +1 N i =0 r wi + 2 ( x) = (7) 2 N (i + 2 ( x) − yi + 2 ) i + 2 ( x) N i =0 Using the derivative to find the minimum and maximum values of each function, the weight of the neural network is searched. wi = wi − 2 N (i ( x) − yi ) i ( x) N i =0 wi +1 = wi +1 − 2 N (i +1 ( x) − yi +1 ) i +1 ( x) N i =0 wi + 2 = wi + 2 − 2 N (i + 2 ( x) − yi + 2 ) i + 2 ( x) N i =0 (8) Local and global gradients appear in the neuron's backward movement process. During the descent from the local gradient to the global gradient, there are cases where the value of the gradient is not updated. In order to avoid this problem - the correct choice of the learning step plays an important role, and it is defined to take it in the interval [0,1]. In neural regression algorithms, the use of local gradients and iterative functions to increase accuracy and speed in the process of approaching global gradients seems to be the right solution. We will try to predict the location of underground layers using radiation signals with the help of a constructed neural network. Figure 2.1. Reconstruction of bivariate signals using neural networks based on quadratic B-splines. 5. Comparative analysis. Let's compare the values of the signals recovered with the help of neural networks according to the table to see if they correspond to the real value. We perform error estimation using Tables 1.2. 1.2 - table. Estimation of absolute and relative errors in restoration of one-variable geophysical signals. No 1 2 B i (t) 0.017 C i (t) 0.0155 NT C i (t) 0.00878 3.8% 3.5% 1.7% In the above table 1 - an absolute mistake. 2 - relative error. Table 1.3. assessment of absolute and relative errors in the restoration of two-dimensional geophysical signals. No 1 2 C i (t) 0. 7 NT C i (t) 0. 1 0.5 4% 0.084 % C i ( t ) – signal approximation results. NT C i (t) represents a one- and two-variable geophysical signal reconstructed on the basis of a neural network. As can be seen from the results in the table , the prediction error is improved by 0.509% if the prediction is made using BSNT . Figure 2.2. Prediction of location of chemical elements using neural networks based on quadratic B splines. The peaks in Figure 1.4 indicate the location of the ores. Prediction with the help of neural networks is much more convenient and reduces unnecessary costs. 5. Conclusion Today, artificial intelligence methods are entering all fields. The effectiveness of these methods is to minimize the error by using the most probable values. Using predictive values to generate desired results with high accuracy. In the article, the B-spline function was selected from several activation functions and the construction of a neural network was considered using this function. With the help of the built neuron, the prediction of underground ores was made using radiation signals. A relative error of 1.8% percent accuracy was achieved in one-dimensional radiation signal recovery and digital processing. A relative error of 0.456% accuracy was achieved in digital processing of two-dimensional signals. Increased accuracy to 0.509% in the prediction of ores located in the earth. References [1] Z. Hakimjon and A. Bunyod, "Biomedical signals interpolation spline models," in 2019 International Conference on Information Science and Communications Technologies (ICISCT) , Nov. 2019, pp. 1–3, doi: 10.1109/ICISCT47635.2019.9011926. [2] H. Zaynidinov, S. Bakhromov, B. Azimov, and S. Makhmudjanov, "Comparative analysis of spline methods in digital processing of signals," Adv. Sci. Technol. Most. Syst. , vol. 5, no. 6, 2020, doi: 10.25046/aj0506180. [3] M. Singh, H. Zaynidinov, M. Zaynutdinova, and D. Singh, "Bi-cubic spline based temperature measurement in the thermal field for navigation and time system," J. Appl. Sci. Most. , vol. 22, no. 3, 2019, doi: 10.6180/jase.201909_22(3).0019. [4] D. Singh, M. Singh, and Z. Hakimjon, "B-Spline approximation for polynomial splines," in SpringerBriefs in Applied Sciences and Technology , 2019. [5] D. Singh, M. Singh, and Z. Hakimjon, "Evaluation methods of spline," in SpringerBriefs in Applied Sciences and Technology , 2019. [6] H. Zaynidinov, S. Ibragimov, G. Tojiboyev, and J. Nurmurodov, "Efficiency of Parallelization of Haar Fast Transform Algorithm in Dual-Core Digital Signal Processors," in 2021 8th International Conference on Computer and Communication Engineering (ICCCE ), Jun. 2021, pp. 7–12, doi: 10.1109/ICCCE50029.2021.9467190. [7] H. Zaynidinov, S. Ibragimov, and G. Tojiboyev, "Comparative Analysis of the Architecture of Dual-Core Blackfin Digital Signal Processors," in 2021 International Conference on Information Science and Communications Technologies (ICISCT), Nov. 2021, pp. 1–4, doi: 10.1109/ICISCT52966.2021.9670135.
