By default OC set HTTPS only to pages with form submit data: login, register, cart and checkout. Product page is not secured with all the default settings. We are not very sure how you have set all site HTTP, but it is certain that this modification is the cause of non working reviews pagination.
P.S. Same problem occurs when using the default theme.