Obituary: Farewell to robots.txt (1994-2025)

Henning Fries:

The protocol taught us that technology can be based on human values like ethics and morality. It showed that voluntary compliance works when all parties benefit. Its greatest achievement was perhaps preserving the internet for three decades from what it has become today – a soulless extraction machine.