To support the development of autonomous control system for the Transportable Fluoride-salt-cooled High-temperature Reactor(TFHR), a machine learning based reactor system performance prediction model is created. The prediction model consists of a reactor physics model and a thermal-hydraulic model, which are formulated using support vector machine(SVR) with training data generated by a RELAP model of TFHR primary system. A particle filtering method is used to estimate the model parameters with noisy instrument measurements. Verifications of the proposed models have been conducted using TFHR reactivity insertion events. Satisfactory performance in predicting the core behavior and estimating model parameters are concluded.