You should take care not to reject mail as a result of spam filtering if it is forwarded from “friendly” sources, such as:
Your backup MX hosts, if any. Supposedly, these have already filtered out most of the junk (see Multiple Incoming Mail Exchangers).
Mailing lists, to which you or your users subscribe. You may still filter such mail (it may not be as criticial if it ends up in a black hole). However, if you reject the mail, you may end up causing the list server to automatically unsubscribe the recipient.
Other accounts belonging to the recipient. Again, rejections will generate collateral spam, and/or create problems for the host that forwards the mail.
You may see a logistical issue with the last two of these sources: They are specific to each recipient. How to you allow each user to specify which hosts they want to whitelist, and then use such individual whitelists in a system-wide SMTP-time filtering setup? If the message is forwarded to several recipients at your site (as may often be true in the case of a mailing list), how do you decide whose whitelist to use?
There is no magic bullet here. This is one of those situations where we just have to do a bit of work. You can decide to accept all mails, regardless of spam classification, so long as it is sent from a host in the whitelist of any one of the recipients. For instance, in response to each RCPT TO: command, we can match the sending host against the corresponding user's whitelist. If found, set a flag that will prevent a subsequent rejection. Effectively, you are using an aggregate of each recipient's whitelist.
The implementation appendices cover this in more detail.