The landmark's storied history includes political fundraisers, reports of ghosts and a raid by FBI agents investigating an art fraud scheme.