ReaxFF is a computationally efficient force field to simulate complex reactive dynamics in extended molecular models with diverse chemistries, if reliable force-field parameters are available for the chemistry of interest. If not, they must be optimized by minimizing the error ReaxFF makes on a relevant training set. Because this optimization is far from trivial, many methods, in particular, genetic algorithms (GAs), have been developed to search for the global optimum in parameter space. Recently, two alternative parameter calibration techniques were proposed, that is, Monte-Carlo force field optimizer (MCFF) and covariance matrix adaptation evolutionary strategy (CMA-ES). In this work, CMA-ES, MCFF, and a GA method (OGOLEM) are systematically compared using three training sets from the literature. By repeating optimizations with different random seeds and initial parameter guesses, it is shown that a single optimization run with any of these methods should not be trusted blindly: nonreproducible, poor or premature convergence is a common deficiency. GA shows the smallest risk of getting trapped into a local minimum, whereas CMA-ES is capable of reaching the lowest errors for two-third of the cases, although not systematically. For each method, we provide reasonable default settings, and our analysis offers useful guidelines for their usage in future work. An important side effect impairing parameter optimization is numerical noise. A detailed analysis reveals that it can be reduced, for example, by using exclusively unambiguous geometry optimization in the training set. Even without this noise, many distinct near-optimal parameter vectors can be found, which opens new avenues for improving the training set and detecting overfitting artifacts.