sd-netlink: revamp message serial handling

sd-netlink: revamp message serial handling

Let's use uint32_t everywhere to maintain the seqno, since that's what
the kernel does. Prviously in the reply_callback logic we used 64bit,
for no apparent reason.

Using 32bit also provides us with the benefit that we can avoid using
uint64_hash_ops, and can use trivial_hash_ops instead for the reply
hashmap, so that we can store the seqno in the key pointer directly.

While we are at it, let's make sure we never run into serial collisions
internally (32bit is a lot, but not that much), and let's put a limit on
outstanding serials, to catch programming errors.

(cherry picked from commit b522c4b92a1a8999e008002f0a30acbaf58b55e4)