Yes, it is alot of fun! We race with about 5-6 racers, mostly using Carrera cars grouped by body style, DTM, LMP, GT etc. You really have to pay attention to where everyone is, at all times too. Like you said, it tends to age the cars some, but then I have my D1XX race cars, and then my analog queens.
We run non magnet, and if there's 3-4, we'll put a magnet ghost car out just to mix things up some.