Switching visual modes while already in a different visual mode deselects selection and does nothing instead

Reproduction example:

1. enter character selection visual mode
2. move cursor around
3. activate linewise selection visual mode with V

Selection deselects. Should switch to linewise selection visual mode instead, as if you had entered linewise selection visual mode using the start and end markers from step 1. Then if you do:

4. move cursor around
5. enter block selection visual mode

It would then switch to block selection visual mode, using the new start and end markers from step 3.