I think it depends on how one interprets justice. If the point of justice is to exact retribution for the wrongs done to the victim, then certainly the victim should be able to decline that payback.

But IMHO, the point of the justice system isn't to avenge past wrongs, it's to prevent future ones. The point of jailing Polanski is to discourage the next guy who's thinking about taking advantage of some under-age girl. While the court should be considerate to those involved, there are future victims who are also affected by leniency.

And, yes, there is a problem with 'victimless crimes'; I think the solution there is to include prior consent as a defense.