Too many plugins and site speed


To set up a WPMU site you need a lot of plugins, especially if you want all of the features available. However, how can you also have a fast site? Is it possible to have 50 to 100 plugins enabled and still have a fast site?