Key insight for noise-contrastive estimation: My objective tries to fit the ratio between data and noise samples, where the denominator (noise) is fixed to be the prior probability of noise—so it does me no good to blow up the numerator. Then some technical work to show that the best way to fit this ratio is to use the prior probability of data (though this seems intuitive).