Tomato can definitely do this part. "Access Restriction" - just create a rule that applies at all times and specify that it's for a specific machine and then type in the MAC address.

The logging is going to be more difficult. You can turn on logging for outbound traffic, but it's going to get huge very quickly with that many connections. You'll want to save the log data remotely and then I suppose you can search through it for signs of the abuse.

If you're running managed switches you may want to do some of the setup before the traffic even gets to the router.