A new probability-based standard setting technique, the Objective Borderline Method (OBM), was introduced recently. This was based on a mathematical model of how test scores relate to student ability. The present study refined the model and tested it using 2500 simulated data-sets. The OBM was feasible to use. On average, the OBM performed well with specificity 0.88, sensitivity 0.51, false positive rate 3.4% and false negative rate 26%. These indices were insensitive to the borderline score range. This probability-based standard setting may be a useful addition to the range of standard setting methods available.