It's known that there are ways to losslessly rotate a JPEG image. One tool supporting lossless rotation is jpegtran, a free software command line app coming with the JPEG libraries. By default, jpegtran copies JPEG comments only: EXIF data is not copied. No warning is printed. Sadly I learned that only recently. To rotate image I am now using this command:

# jpegtran -rotate 90 -copy all -perfect \
    -optimize -progressive INPUT.jpg > OUTPUT.jpg

The -copy all is turn on copying of EXIF data.