Protein synthesis in eukaryotes is controlled by signals and stresses via a common pathway, called the integrated stress response (ISR). Phosphorylation of the translation initiation factor eIF2 alpha at a conserved serine residue mediates translational control at the ISR core. To provide insight into the mechanism of translational control we have determined the structures of eIF2 both in phosphorylated and unphosphorylated forms bound with its nucleotide exchange factor eIF2B by electron cryomicroscopy. The structures reveal that eIF2 undergoes large rearrangements to promote binding of eIF2α to the regulatory core of eIF2B comprised of the eIF2B alpha, beta and delta subunits. Only minor differences are observed between eIF2 and eIF2αP binding to eIF2B, suggesting that the higher affinity of eIF2αP for eIF2B drives translational control. We present a model for controlled nucleotide exchange and initiator tRNA binding to the eIF2/eIF2B complex.