I fully agree with it being yellow. By far, the most common isotope of uranium is uranium-238, which is indeed radioactive, but not dangerously radioactive. In this list, lead is listed as a yellow because it can give you heavy metal poisoning. In this scenario, the uranium would cause more damage to your body by damaging it as lead would (heavy metal poisoning affecting brain, kidneys, liver, etc) before the radiation would ever have an impact on your body.
I have a few of these!