“Once barriers — which consist in a sense only in man’s ignorance of the possible — are torn down, they are not easily set up again.”