The problem is that the large email purveyors like AOL, Yahoo, Microsoft, etc, use large server farms that balance the load between multiple hosts, so when the mail retries it comes from different IP addresses on every retry. Microsoft, for example, lists thousands of IP addresses as part of their email service.
Greylisting by IP address hasn't got a chance of working in that environment.
When I ran my own mailserver I had greylisting that only worked by sender mail address. Additionally, it did the usual SPF checking etc.
This did not cause the abovementioned problem, but I'm not sure it added much spam prevention. I had other methods to detect trojaned PCs with bad SMTP senders (e.g. doing PIPELINING without having negotiated it) and that was much more effective.
Rob