This issue was created because of a misunderstanding of the `Content > Annotations > Annotation display > Display annotations by default` parameter. I thought it would be used to also have the effect of the `Look & Feel > Presentation > Page tabs > Show page annotations` parameter.
Adding some info in the description of the first parameter could have probably prevented this misunderstanding.