Whiteface is one of the best mountains on the East coast simply because of its size and challenging terrain. Beginners get somewhat of a raw deal if the mountain isn't getting much snow, since a lot of trails will be closed. I've always considered Whiteface a more advanced mountain.
It's a good bet to go later in the season, say late January or February. I spent New Years week at a rented cabin near the mountain and it was one of the funnest trips I've had in a while, we even were blessed with six inches of new snow.
Even if you don't ski or snowboard there is a lot to do in town, from ice skating to dog sledding, and even bob sledding. The town itself has a European feel to it because of all the Olympic facilities and while we chose to party at home, it's a safe bet that there should be something to do nightlifewise. Chances are you'll be too tired from skiing to care either way.