Yubikey seems to self-assign to the previous user when unassigning it and use it to validate a token.
To reproduce it:
Import a CSV yubico file with yubikey information
Assign a yubikey to a user
Validate a token with the RADIUS PAP Tester and the yubikey inserted - do not close the RADIUS PAP Tester
Unassign the yubikey (do not delete it), do not unplug it
Validate another token with the RADIUS PAP Tester and the yubikey inserted
Refresh the AD Users and Computers window, tokens node
The yubikey will appear assigned to the previous user
See the attached video.