RNA polymerase II (Pol II) in Saccharomyces cerevisiae can terminate transcription via several pathways. To study how a mechanism is chosen, we analyzed recruitment of Nrd1, which cooperates with Nab3 and Sen1 to terminate small nucleolar RNAs and other short RNAs. Budding yeast contains three C-terminal domain (CTD) interaction domain (CID) proteins, which bind the CTD of the Pol II largest subunit. Rtt103 and Pcf11 act in mRNA termination, and both preferentially interact with CTD phosphorylated at Ser2. The crystal structure of the Nrd1 CID shows a fold similar to that of Pcf11, but Nrd1 preferentially binds to CTD phosphorylated at Ser5, the form found proximal to promoters. This indicates why Nrd1 cross-links near 5' ends of genes and why the Nrd1-Nab3-Sen1 termination pathway acts specifically at short Pol II-transcribed genes. Nrd1 recruitment to genes involves a combination of interactions with CTD and Nab3.