Leaving out whether we were in full system or not as contextualizing state when decoding instructions was actually broken, so that part of the change is necessary. The performance cost was not measurable on x86. Having separate decode caches is an intrinsic part of the design we talked about before because se/fs has to be contextualizing information.

Also we don't *have* to have SE/FS in the decoder in the first place. Some ISAs (SPARC, for instance) take care of the SE vs. FS changes in behavior when handling the trap/exception/interrupt which is requesting a system call. When set up that way, the instruction objects themselves don't know which type of system they're part of. It gets a little more complicated when there are system call instructions which don't return Fault objects (X86 has these), but then we can just inject system call targets with pseudo instructions at them which invoke a system call in SE mode.

I wouldn't say it was "broken"... having SE and FS simultaneously in the same simulation is just a feature that's not supported.  Sort of like having multiple CPUs with different ISAs in the same simulation: it doesn't work, but that doesn't mean it's broken, just that the design doesn't even attempt to cover that case.

However, unlike having multiple CPUs with different ISAs, I can't even think of a use case for having SE and FS simultaneously in the same simulation.  Also, I doubt that just being able to dynamically decode instructions under either mode is enough to actually support SE and FS simultaneously anyway, especially in the same system where the simulator and the OS would have to fight over physical memory allocation etc.  So to me, making SE/FS part of the dynamic context is code and runtime overhead that has no purpose, and at most only gives the false appearance that we support something that doesn't actually work or make sense.  Thus even if the code and overhead is small, I don't like having it in there.

I'm trying to eliminate or significantly reduce the separation between SE mode and FS mode, so yes, it's something that I plan to support. A simple example of where that would be useful would be where you have some simple client apps you want to run in SE mode and a server you want to run in FS. I agree it's probably unlikely you'd want to switch the *same* cpu/system between SE and FS, but you might want to have both in two *different* systems in the same simulation. The distinction between SE and FS mode is fairly artificial anyway, so I want to be able to have things which are at other points on that spectrum (BIOS emulation mode, hypervisor emulation mode in SPARC, emulated option ROMs). In that vein I want to flush SE and FS mode out of the decoder entirely, but I'm not there yet.

The code has virtually no overhead, supports what I'm working towards, makes sense, and is something I intend to support.

We can't run client apps in SE mode because there's no network stack for them to use to communicate with the server.  Can you come up with a better example?

I don't think I need to come up with use cases to justify fixing a bug. I also don't think I have to prove something is a bug because "nobody will try that" while I work towards making it possible for somebody to try that or something similar.

I don't see it as fixing a bug, I see it as adding a feature.  I think new features should be justified when they add complexity.  I don't think "somebody might someday want to try something like this" with no specific reason why is a very good justification.

I've decided another one of these arguments is just not worth it, so I'm not going to bother. I'm going to keep my code, and you guys and I can do whatever we want separately.