The present study aimed to construct prospective models for tumor grading of rectal carcinoma by using magnetic resonance (MR)-based radiomics features. A set of 118 patients with rectal carcinoma was analyzed. After imbalance-adjustments of the data using Synthetic Minority Oversampling Technique (SMOTE), the final data set was randomized into the training set and validation set at the ratio of 3:1. The radiomics features were captured from manually segmented lesion of magnetic resonance imaging (MRI). The most related radiomics features were selected using the random forest model by calculating the Gini importance of initial extracted characteristics. A random forest classifier model was constructed using the top important features. The classifier model performance was evaluated via receive operator characteristic curve and area under the curve (AUC). A total of 1,131 radiomics features were extracted from segmented lesion. The top 50 most important features were selected to construct a random forest classifier model. The AUC values of grade 1, 2, 3, and 4 for training set were 0.918, 0.822, 0.775, and 1.000, respectively, and the corresponding AUC values for testing set were 0.717, 0.683, 0.690, and 0.827 separately. The developed feature selection method and machine learning-based prediction models using radiomics features of MRI show a relatively acceptable performance in tumor grading of rectal carcinoma and could distinguish the tumor subjects from the healthy ones, which is important for the prognosis of cancer patients.