Understanding and modeling protein-ligand interactions is fundamental to modern drug discovery and design. Virtually every method employed in drug discovery - from experimental bioassays to computational techniques such as QSAR, docking, and activity prediction - relies on accurate models of these interactions. Recent advances in deep learning have greatly enhanced our ability to model protein-ligand interactions, as evidenced by innovations including graph neural networks for activity prediction, diffusion-based docking methods, geometric deep learning for binding pocket detection, contrastive learning for affinity prediction and virtual screening, and most recently foundational models for molecular structure prediction of biological complexes. In this work, we propose VN-EGNNrank, a novel ligand-conditioned binding site prediction method that combines a geometric architecture for protein encoding, a specialized ligand encoder, and a contrastive objective function to effectively align binding pocket and ligand representations in a shared latent space. Our experiments show that incorporating ligand information significantly enhances binding pocket ranking compared to ligand-agnostic models, and VN-EGNNrank achieves performance comparable to -- or even exceeding -- that of the much larger blind docking model DiffDock, while maintaining high computational efficiency suitable for large-scale virtual screening.