For most GLUE tasks, the 'good' subnetworks can be retrained to attain overall performance close to that of the full model, but so can randomly sampled subnetworks of the exact same size.