Evidence of microbial zonation in the open ocean is rapidly accumulating, but while the distribution of communities is often described according to depth, the other physical factors structuring microbial diversity and function remain poorly understood. Here we identify three different water masses in the North Water (eastern Canadian Arctic), defined by distinct temperature and salinity characteristics, and show that they contained distinct archaeal communities. Moreover, we found that one of the water masses contained an increased abundance of the archaeal alpha-subunit of the ammonia monooxygenase gene (amoA) and accounted for 70% of the amoA gene detected overall. This indicates likely differences in putative biogeochemical capacities among different water masses. The ensemble of our results strongly suggest that the widely accepted view of depth stratification did not explain microbial diversity, but rather that parent water masses provide the framework for predicting communities and potential microbial function in an Arctic marine system. Our results emphasize that microbial distributions are strongly influenced by oceanic circulation, implying that shifting currents and water mass boundaries resulting from climate change may well impact patterns of microbial diversity by displacing whole biomes from their historic distributions. This relocation could have the potential to establish a substantially different geography of microbial-driven biogeochemical processes and associated oceanic production.