In recent years, there have been various efforts to product thinking and decentralized data loading using data meshes. However in deep learning, data loading is still challenging to master at scale. In this talk, we present our decentralized data loading solution and show why flexibility and collaboration is key to enable novel ML use cases. We hope to make large-scale model training accessible to a wider community and move towards more sustainable ML.