This work proposes FedMoDN, a novel federated modular neural network architecture for collaborative learning across all features of an imperfectly interoperable distributed dataset. Here, distributed data centers that collect variable combinations of features are able to use the full complement of their features with minimal exposure to biased missingness. Our approach enables data owners collecting different feature subsets to train a joint model without sharing, discarding, or imputing any data.We evaluate the robustness of our approach through experiments that mirror realistic challenges encountered with medical data, particularly in resource-limited settings. Our results show that this modular approach is significantly more robust than a monolithic neural network when dealing with missing data, systematic bias, or heterogeneous feature subsets.