The method splits history into many train and test combinations, as in combinatorial-cross-validation. For each split, pick the configuration that ranked first in the training half, then see where it ranks in the test half. The share of splits where the winner lands in the bottom half of test performance is the PBO.
A PBO near 0.5 means your selection procedure is no better than picking at random, which is common for large parameter searches on short samples. A PBO below about 0.2 suggests the selection carries information.
Note what it measures: the process, not the strategy. It answers whether your way of choosing between candidates works, which is the question most researchers never ask.
Related: combinatorial-cross-validation, overfitting, deflated-sharpe-ratio, walk-forward-analysis