Federated Learning

The question was simple to say and annoying to answer. If you train one model across a bunch of tiny devices that never share their data, how many of those devices can be lying before the model is useless?

Each device learns from what it sees, sends what it learned to a server, and the server averages everyone together and sends it back. Nothing private ever leaves the device, which is the whole appeal. The catch is that the server has to trust whatever comes in, and a device that lies gets averaged into everything.

My whole summer came down to a model with 512 numbers in it, small enough to fit on a chip with kilobytes of memory, and a setting for how many of the five devices were lying. I ran every attack I could find against every defense I could find and watched the accuracy.

With a few liars the model was fine. With a few more, most of the defenses fell apart and the model was guessing. The one defense that held was the simplest, it ignored everyone except the device that agreed most with the others.

Averaging is a trust decision. When the model is tiny, one bad update is a big part of everything it knows, so blending in a lie costs more than throwing away a truth.

I spent a summer in Miami breaking a very small model on purpose, and the defense that held was the one that trusted a single device.

Personal