kvm: Add support for multi-system simulation

Information
Submitter:	Andreas Sandberg
Repository:	gem5
Branch:	default
Bugs:
Depends On:
Reviewers
Groups:	Default
People:

Description

Changeset 10062:4fea5050f4f2
---------------------------
kvm: Add support for multi-system simulation

The introduction of parallel event queues added most of the support
needed to run multiple VMs (systems) within the same gem5
instance. This changeset fixes up signal delivery so that KVM's
control signals are delivered to the thread that executes the CPU's
event queue.

There are currently no facilities to multiplex between multiple KVM
CPUs in the same event queue, we are therefore limited to
configurations where there is only one KVM CPU per event queue. In
practice, this means that multi-system configurations can be
simulated, but not multiple CPUs in a shared-memory configuration.

Testing Done

Tested boot and interactive use of single CPU system -- works.
Tested boot of dual-system configuration and interactive use -- works.

Did not run regressions as the current tests do not test KVM functionality.

Also available from the multisys branch in git://github.com/andysan/gem5.git.

You comment that "we are currently limited to configurations where there is only one KVM CPU per event queue"... can you expand on this?  I.e., why can't I have multiple KVM CPUs per event queue and timeslice among them?  Or is that not what you meant?

Andreas Sandberg Feb. 10, 2014, 1:45 a.m. (Feb. 10, 2014, 1:45 a.m.)

The limitation at the moment is because we can't multiplex between KVM CPUs in the same event queue. This is caused by the way we simulated until the next event (we look into the future and request an exit when the next event is due), if there is another KVM CPU executing, we'll see that one on the same or a nearby tick. It might be possible to get multiplexing to work by ignoring KVM events when we look into the local event queue to calculate when to exit from KVM.

Steve Reinhardt Feb. 10, 2014, 6:18 a.m. (Feb. 10, 2014, 6:18 a.m.)

I see.  So this would be a limitation with the current single-threaded model as well, right?  I guess I was reading too much into it and thinking this was a new limitation due to the multiple queues.

Andreas Sandberg Feb. 10, 2014, 6:57 a.m. (Feb. 10, 2014, 6:57 a.m.)

That is correct. Without this patch, the KVM implementation only supports one CPU at a time.

src/cpu/kvm/base.hh (Diff revision 1)

seems to be a typo in this comment... should this be "is called as"?

Andreas Sandberg Feb. 10, 2014, 1:45 a.m. (Feb. 10, 2014, 1:45 a.m.)
```
Oopps. Thanks for spotting it.
```

src/cpu/kvm/base.cc (Diff revision 1)

I wasn't sure why you changed from sigprocmask to pthread_sigmask, but on googling for the answer I found this:
http://stackoverflow.com/questions/2170672/sigprocmask-causing-segfault

So it makes sense to me now, but might be worth a mention in the commit message.

Andreas Sandberg Feb. 10, 2014, 1:45 a.m. (Feb. 10, 2014, 1:45 a.m.)

That wasn't the problem I encountered. The problem I had was that I need to ensure that only the thread executing the KVM CPU's event queue should be affected by the new signal mask. sigprocmask might actually work -- I don't know -- but the man page is pretty clear on the subject:

"The use of sigprocmask() is unspecified in a multithreaded process; see pthread_sigmask(3)."

Steve Reinhardt Feb. 10, 2014, 6:17 a.m. (Feb. 10, 2014, 6:17 a.m.)

Right... I didn't necessarily mean the segfault, I just noticed at that link a similar comment about sigprocmask only being valid for single-threaded processes.  Anyway, adding a sentence like "Switched from sigprocmask() to pthread_sigmask() because the former is not compatible with multiple threads" to the commit message would be nice for those of us not familiar with these details.

Andreas Sandberg Feb. 10, 2014, 6:56 a.m. (Feb. 10, 2014, 6:56 a.m.)

I'll add that and include a line about the change in timer signal delivery.

Description:

~		Changeset 10062:ec5050f0c501
	~	Changeset 10062:4fea5050f4f2

		kvm: Add support for multi-system simulation

		The introduction of parallel event queues added most of the support
		needed to run multiple VMs (systems) within the same gem5
		instance. This changeset fixes up signal delivery so that KVM's
		control signals are delivered to the thread that executes the CPU's
		event queue.

~		Since CPUs must execute in their own threads, we are currently limited
~		to configurations where there is only one KVM CPU per event queue. In
	~	There are currently no facilities to multiplex between multiple KVM
	~	CPUs in the same event queue, we are therefore limited to
	+	configurations where there is only one KVM CPU per event queue. In
		practice, this means that multi-system configurations can be
		simulated, but not multiple CPUs in a shared-memory configuration.

Diff:

Revision 2 (+78 -22)

Show changes

	src/cpu/kvm/base.hh
	src/cpu/kvm/base.cc
	src/cpu/kvm/timer.cc

It looks like I was asking for a few clarifications in the commit message, but that's it.  I think the code itself is good to go.  I don't want to halt your momentum on this :)

You have a pending review.

Review Board 2.0.15

This change has been marked as submitted.

Screenshots

Files

Description:

Changeset 10062:ec5050f0c501

Changeset 10062:4fea5050f4f2

Diff:

Status: Closed (submitted)