Federated Learning

General
1

“Federated learning can be used to train LLMs without having to share data. In federated learning, each device trains a local model on its own data.”

In this article (Maximizing ROI with Privacy in Mind: Leveraging Federated Learning for Large Enterprise Language Models) Dr. Vimal Rawat talks about this in the context of companies training AI without giving up control of their data, but I think it’s equally as applicable to a network of individuals.

This is a big key to enabling Step 7 of The Digital Brain Battle Plan: Enabling Digital Brain Owners to Trade their Data, Knowledge, Information and Use Cases → https://www.linkedin.com/pulse/war-our-digital-brains-david-waring-czu7e/

2

I need to spend a few hours of concentrated time going through this paper because it seems like they have a federated learning system already up and running that they are open sourcing: The Future of Large Language Model Pre-training is Federated

Another interesting idea from the paper is incentivizing indivduals or companies that are participating in the federation with their data not via crypto tokens but by getting access to compute on the model.

I would like at this similarly to how Jigsaw a lead sharing platform that was aquired by salesforce did it. You earned credits by submitting leads to the system.