We don't need to cast to the type of the template argument because it is always an unsigned number. When passed an enumeration type, MSVC's debug formatter would get confused, so this fixes issues like looking at a llvm::PointerIntPair<const ContentCache *, 3, CharacteristicKind> such as what's used in SrcMgr::FileInfo.