As a followup to my previous entry with the same subject (Part 1), We have discovered more information. It turns out that the router alert packets weren't the cause of the high utilization, but instead mearly a piece of the larger puzzle.
The CPU utilization issue became so unbearable that something had to be done, and unfortunately, modifying all Macintosh systems on the network to stop using mDNS wasn't an option. A few of the systems were changed with no noticable impact, but that could be because the number of systems changed constituted less than 1% of the systems at that campus. Instead something else would have to be tried... Somthing on the network...
Being confident that the previous troubleshooting was accurate, and with assurances that nothing production other than GhostCast was using multicast, multicast routing was turned off on the switch. This was a bold and rather drastic move, but it should solve the problem, right? Wrong. The number of packets hitting the CPU with the router alert option set remained constant, and CPU usage continued to behave as it had before. If I had thought thiss through a bit more, I would have realized that traffic to the 224.0.0.x/24 multicast address range are meant to stay local, therefore the traffic would continue.
The next step was to create an access list that blocks the mDNS group address destination address. After that is created, it is added as a Port ACL on each client facing switch port. Suddenly, we now have a real change in status. CPU Usage dropped from 100% to 21% as soon as the interface range command was completed.
The big question here is why did this work, and what caused the switches poor performance? The 4500 is capable of handling much higher volumes of multicast traffic, and it has distributed hardware processing of multicast. It turns out that the 224.0.0.0/24 range is reserved for L2 local multicast, such as routing protocols, All routers, All hosts, etc. Because of this fact, the 4500 was designed to send all multicast traffic destined to any address in this range directly to the CPU weather it was needed/subscribed, or not. I think an inbound 224.0.0.0/24 multicast filter should be considered a basic security requirement for every network in order to prevent inadvertant or intentional DoS against the switched infrastructure regardless of weather multicast is officially in use on the network!
Leave a comment