It's definitely worth it if you are starting to notice it from the ground looking up. My roofer told me that instead of seeing it as a way of making your roof look nice, it's better to think of it as a preventative measure. A full roofing replacement later down the line will be much more expensive than roof moss removal now. There is no best time of year to remove moss from a roof either.