I played with this feature as it is so anoying. Here what I figured out:
If you switch ON "Lock after Leaving" and switch it OFF again, than it does what it is supposed to do.
So there seams to be an issue with keeping the setting or with the initial setting.
But at least this is a possibility to get is working.