There are a lot of people asking the same questions right now. My own thought is that you need to look at all the plugins that you use and see what the
vB 4 plans are with them. For those that are not going to be updated you have to ask yourself if they will be missed by the site members. One thing that I've not really seen touched upon at all is the usability changes that have been put in place. How comfortable will your members be with the changes? Will you have to spend countless hours explaining how to do work with the latest version. This depends on you user community and how quickly they can learn the new and/or changed functions.
I know this is no a definitive yes or no answer but each site has to assess quite a few different parameters before deciding to make the leap or not. Personally, I think the process from beta to RC to Gold is rushed and I will not be moving to 4.x for a while.