Changes in phosphorylation of the carboxy-terminal domain (CTD) of RNA polymerase II (RNAP) are associated with transcription initiation, elongation and termination. Sites of active transcription are generally characterized by hyperphosphorylated RNAP, particularly at Ser 2 residues, whereas inactive or poised genes may lack RNAP or may bind Ser 5-phosphorylated RNAP at promoter proximal regions. Recent studies have demonstrated that silent developmental regulator genes have an unusual histone modification profile in ES cells, being simultaneously marked with Polycomb repressor-mediated histone H3K27 methylation, and marks normally associated with gene activity. Contrary to the prevailing view, we show here that this important subset of developmental regulator genes, termed bivalent genes, assemble RNAP complexes phosphorylated on Ser 5 and are transcribed at low levels. We provide evidence that this poised RNAP configuration is enforced by Polycomb Repressor Complex (PRC)-mediated ubiquitination of H2A, as conditional deletion of Ring1A and Ring1B leads to the sequential loss of ubiquitination of H2A, release of poised RNAP, and subsequent gene de-repression. These observations provide an insight into the molecular mechanisms that allow ES cells to self-renew and yet retain the ability to generate multiple lineage outcomes.