Phoenix: Federated Learning for Generative Diffusion Model - Stanley Jothiraj, Fiona Victoria (My University, 2023)
Generative AI has made impressive strides in enabling users to create diverse and realistic visual content such as images, videos, and audio. However, training generative models on large centralized datasets can pose challenges in terms of data privacy, security, and accessibility. Federated learning is an approach that uses decentralized techniques to collaboratively train a shared deep learning model while retaining the training data on individual edge devices to preserve data privacy. This paper proposes a novel method for training a Denoising Diffusion Probabilistic Model (DDPM) across multiple data sources, using federated learning techniques. Diffusion models, a newly emerging generative model, show promising results in achieving superior quality images than Generative Adversarial Networks (GANs). Our proposed method Phoenix is an unconditional diffusion model that leverages strategies to improve the data diversity of generated samples even when trained on data with statistical heterogeneity (Non-IID data). We demonstrate how our approach outperforms the default diffusion model in a federated learning setting. These results are indicative that high-quality samples can be generated by maintaining data diversity, preserving privacy, and reducing communication between data sources, offering exciting new possibilities in the field of generative AI.
Thesis (Master's)--University of Washington, 2023
Keywords: Diffusion Model; Federated Learning; Generative AI; Non-IID Data; Artificial intelligence; Computer science and engineering