For some reason with CloudFront there is an intermediate HTTP redirect when using routing rule redirects; that is:
https://example.com/foo.html redirects to
http://example.com/bar.html redirects to
This happens for sites already deployed using the original implementation. I opened a Stack Overflow question to try to understand this more.
This ticket can also make another improvement: making sure that alt domains use HTTPS when redirecting if possible. Thus there are three areas to check that redirect directly from HTTP to HTTPS:
Redirecting directly to HTTPS for alt domains.
Redirecting directly to HTTPS for routing rules.
Redirecting directly to HTTPS for object redirects. This seems to happen automatically if the object redirect just uses a URI reference (path).