Few weeks ago Munin was flooding me with message alerts about IO latency, read or write, going too high. Ironically it turned out Munin was the culprit. And the solution was to use rrdcached.
Lets see a Munin graph that shows pretty well the difference using rrdcached made:
Around week 21, when I deployed this optimization, writes dropped quite a lot and reads stabilized.
Now follows the step-by-step guide to use rrdcached (with Gentoo).