We've got this new Xen server and needed to make sure that we weren't making the kernels out of sync with the physical box's kernel version. For this reason we needed to remove the kernel upgrades from yum so that this didn't get installed/upgraded by default.

Modifying the /etc/yum.conf file and adding in this exception in the [main] section did the trick:
exclude=kernel*

Makes it a lot easier to keep track of those files you want to leave as they are.