Where you trying to use an in-and-out peak to create an auto-save mid-Helix fight?
Just trying to figure out how you were fighting Helix near a door to get kicked out.
It's possible Helix returned to his spawn point, or if he was walking back to it, he got stuck in a corner somewhere.
If your journal doesn't show you killed Helix, he should re-set when the map re-loads (inn save, 12-hour in game, etc). However, just to be safe, loading an auto-save or quick save, prior to entering the dungeon would be best.
What could have happened, is that the in-game time passed long enough for the map to want to reset when you left. So when you re-entered, the map was re-loaded as a fresh map. With the fight with Helix being underway, the game could have enter a limbo where Helix woudn't spawn (since he already spawned to fight you), but can never show him as being killed as you didn't kill him.
