I had a look at the cause: these headings aren't meant to be visible. In XWiki.ConfigurableClass, the sheet adds the "Customize <page>:" heading for each page contributing to a section except the first one, and its JSX ("Hide headers unless they have error messages under them") is supposed to remove them. But that script only removes the h1 elements whose id starts with hcustomize, while the #showHeading macro of XWiki.ConfigurableClassMacros renders a level 2 heading (== ... ==). So the headings always stay. It's still the case on master and on stable-17.10.x.
Note that the JSX is only loaded when one of the pages has a configurationClass.
Workaround used in XWiki Standard itself: XWiki.WYSIWYG.DefaultEditorConfig, whose section is shared with CKEditor.AdminSection, hides them in its SSX, each contribution displaying its own heading: