For some reason with CloudFront there is an intermediate HTTP redirect when using routing rule redirects; that is:
https://example.com/foo.html redirects to
http://example.com/bar.html redirects to
This happens for sites already deployed using the original implementation. I opened a Stack Overflow question to try to understand this more.
This ticket can also make another improvement: making sure that alt domains use HTTPS when redirecting if possible. Thus there are three areas to check that redirect directly from HTTP to HTTPS:
Redirecting directly to HTTPS for alt domains.
Redirecting directly to HTTPS for routing rules.
Redirecting directly to HTTPS for object redirects. This seems to happen automatically if the object redirect just uses a URI reference (path).
I may have been incorrect about intermediate HTTP redirects for S3 object redirects. As long as the redirect location is a path and not a full URI, S3 will send it back literally as the HTTP Location header and CloudFront should resolve it to the current URL, so it should stay at HTTPS as I noted elsewhere.
Apparently S3 generates full URLs for routing rules, however, so we need to set both the host and the protocol for routing rules, as the answer to my question on Stack Overflow explained.