Olfaction---how molecules are perceived as odors to humans---remains poorly understood. Recently, the principal odor map (POM) was introduced to digitize the olfactory properties of single compounds. However, smells in real life are not pure single molecules, but complex mixtures of molecules, whose representations remain relatively under-explored due to limited data in olfactory mixtures. We introduce POMMix, a mixture model extension of POM which leverages mono-molecular olfactory data to build meaningful mixture representations of smells. Our model builds upon the symmetries of the problem space in a hierarchical manner: (1) graph neural networks for building mono-molecular embeddings, (2) attention mechanisms for aggregating molecular representations into mixture representations, and (3) cosine prediction heads to encode olfactory perceptual distance in the mixture embedding space. POMMix achieves state-of-the-art performance across multiple datasets. We perform comprehensive ablation studies of the components of POMMix to understand the contribution of each component. We evaluate the generalizability of the model, explore olfactory phenomena with the representations, and analyze the interpretability of the representations. Our work advances the effort to digitize olfaction, highlighting the synergy of domain expertise and deep learning in crafting mixture representations in low-data regimes.