Today I learned – while building this site, natch – that Firefox have a special cache for favicons, which the normal cache refreshing techniques doesn’t clear.
It took a couple of searches to find a frustrated Stack Overflow question explaining the behaviour, and providing a workaround during development: