Toxicity is usually measured by the LD50: the amount needed to kill half the creatures (usually mice) to whom a poison is given. Amusingly, the worst found so far is the bacterial protein botulin – Botox. Based on mice, its LD50 when injected into humans is less than 0.1 micrograms.
For poisons from animals, batrachotoxin is often mentioned. Various estimates of its equivalent LD50 give somewhat under 100 micrograms: almost 1000 times less toxic. Famous from Phyllobates ‘poison dart’ frogs and certain birds, it’s now thought they ingest batrachotoxin from insects.
However, palytoxin probably beats this. The human LD50 of this amazingly intricate non-protein chemical has been calculated as about 4 micrograms. It comes from a Hawaiian soft coral, but this animal may be obtaining palytoxin from non-animal sources, namely certain dinoflagellate algae.