Step by Step Guide to Build and Compare FedAvg and FedProx Federated Learning on Non-IID CIFAR-10 with NVIDIA FLARE

['In this tutorial, we build an advanced federated learning experiment with NVIDIA FLARE, comparing FedAvg and FedProx on a non-IID CIFAR-10 setup. The client data is split using a Dirichlet distribution to simulate realistic label imbalance across federated sites. We utilize the NVFlare Job API to define and launch federated jobs, while the Client API handles local training, model exchange, and communication between simulated clients and the server.

Finally, we run both algorithms on the same partitioned dataset and visualize how their global model accuracy evolves across communication rounds.', 'To begin, we install the required libraries and import the main packages needed for the federated learning experiment. We define the experiment settings, including the number of clients, training rounds, batch size, learning rate, and non-IID Dirichlet alpha value. We also create the required data and results folders, then download CIFAR-10 once, so that all simulated clients can reuse the same dataset safely.', "Next, we create the client-side training script as a separate Python file, allowing NVFlare to import and run it inside each simulated client process.

We define a small CNN model for CIFAR-10 classification and add a deterministic Dirichlet partitioning function to create non-IID client datasets. We also define an evaluation function that measures the global model's accuracy on the shared CIFAR-10 test set.", "We then build the main client training workflow that runs inside every NVFlare site. We initialize the NVFlare Client API, identify the current client site, load that client's non-IID data shard, and prepare the local model, optimizer, and loss function.

We then receive global weights from the server, evaluate the model, train locally with either FedAvg or FedProx, and send the updated model back for aggregation.", 'On the server-side, we define the federated experiment using the NVFlare Job API. We create a FedAvg job, attach the client training script to each simulated site, and pass all experimental arguments, such as alpha, learning rate, local epochs, and the FedProx mu value. We then run two experiments on the same non-IID setup: one with standard FedAvg and another with FedProx.

We load the saved accuracy logs for both FedAvg and FedProx and sort them by communication round. We plot the global test accuracy curves to visually compare how the two methods perform on non-IID CIFAR-10 data.', 'In conclusion, we completed an end-to-end federated learning workflow using NVIDIA FLARE, from preparing non-IID CIFAR-10 client shards to training and comparing FedAvg and FedProx. We saw how the Job API helps us configure server-side orchestration, while the Client API lets each simulated client receive global weights, train locally, and send updated models back for aggregation.