The Problem: Scaling Services on the Internet
The popularity of the Web and other Internet services puts a lot of stress on servers and network infrastructure.
Demand is highly dynamic. Flash crowds are very large and move very quickly. Therefore, overprovisioning does not work.
Caching (either in the browser or by proxies) is used for the Web. Although it reduces the amount of traffic drastically, it is not a satisfactory solution for many reasons. For example:
How is stale data purged?
Content providers want hits on site (e.g. for advertisement banners), and they need to meter the hits for revenue.
What about non-Web traffic?