Aiming at the compounding problem of extreme pressure and anti-wear additives in gear oil, a prediction model of lubricating oil friction and wear performance based on Generalized Regression Neural Network optimized by Grey Wolf Optimizer was proposed. The four materials sulfurized isobutylene (T321), tricresyl phosphate (T306), isooctyl acid thiophospholipid octadecylamine (T308) and molybdenum dialkyl dithiocarbamate (MoDTC) commonly used in gear oil were selected as additives. Design the experimental scheme by orthogonal experiment method, add additives to base oil PAO at different mass fractions to prepare lubricating oil samples to be tested. Use MFT-R4000 reciprocating friction and wear tester to test the tribological properties of the mixed oil sample on the steel-steel friction pair. The Generalized Regression Neural Network proposed by American scholar Specht is composed of four parts: input layer, parameter layer, summation layer and output layer, it had strong nonlinear mapping ability, high fault tolerance and robustness, and had good prediction effect on nonlinear problems and small sample datasets, at the same time, compared with Radial Basis Function neural network and Back Propagation neural network, it had more advantages in computing speed and learning ability, and was widely used in prediction, control, system identification and other fields. By establishing a GRNN model, the effects of different ratios of the four additives on the mean friction coefficient and wear volume performance were predicted, and a Support Vector Machine model was established to compare and analyze the accuracy of the prediction results of the Generalized Regression Neural Network model. Grey Wolf Optimizer is a swarm intelligence optimization algorithm inspired by the predatory activities of gray wolves in nature. Compared with other intelligent optimization algorithms, this algorithm has obvious advantages such as simple operation, few parameters, and easy programming. Therefore, Grey Wolf Optimizer is used to optimize the spread value of Generalized Regression Neural Network to overcome the influence of subjective factors and improve the prediction accuracy of the model. The Hold-out method and Leave-one-out cross validation method are used to evaluate the generalization ability of the model on the small sample dataset, and reduce the risk of model overfitting. The research results showed that: Generalized Regression Neural Network and Support Vector Machine can relatively accurately predict the mean friction coefficient and wear volume of lubricating oil. In contrast, Generalized Regression Neural Network had better effect and was more effective for small sample lubricating oil performance prediction problems. After introducing the Grey Wolf Optimizer to optimize the spread value of the Generalized Regression Neural Network, the coefficient of determination of the model was significantly improved, and the coefficient of determination of the GWO-GRNN mean friction coefficient prediction model reached 96%, the coefficient of determination of the wear volume prediction model reached 91%; indicating that the model can relatively accurate predict the tribological properties of gear oil extreme pressure and anti-wear additives in the case of small samples. This method provided a new method for compound research of extreme pressure and anti-wear additives in gear oil, which was helpful to improve the research and development efficiency of lubricating oil.