It is related to nystagmus. I thought I only did it on purpose, but when I am tired I also sometimes do it involuntarily.

From the link:

A slightly different form of nystagmus may be produced voluntarily by some (8% of) people. Some can sustain it for up to 35 seconds, but most average around 5 seconds.[19][20][21]