I tried to write a test for this, see https://github.com/xwiki/xwiki-platform/commit/254bcab007adb8726327afa75dc7eee764e1c273 , but it's not possible to open a new incognito window without creating a new WebDriver instance. The test is using the same user for now, which is not really verifying that the problem is not present. I left a TODO to improve the test later.