Config: Make disk-image option support mutiple disks

Information
Submitter:	Jayneel Gandhi
Repository:	gem5
Branch:	default
Bugs:
Depends On:
Reviewers
Groups:	Default
People:

Description

Changeset 9008:940d21e358ea
---------------------------
Config: Make disk-image option support mutiple disks
Added support to specify mutiple disks provided on command line with
disk-image option. Made the required changes to SysConfig to support that.
Updated regressions to accomodate the stat name changes and change in
SysConfig. When you use this option, you have to specify disks in specific
order and the default disks for the system also needs to be specified.
The operating system disk is to be specified first in the comma-spearated list.
The new disks that you specify can be mounted by using mount command once the
operating system is booted. Like for X86 linux, you can mount the new disk from
/dev/hdc, if the option was used as --mem-disk=linux disk,swap disk,new disk.

Testing Done

All FS test passed except SPARC FS (FS files for SPARC no available)
All SE tests passed except eio tests.

Issue Summary

Description	From	Last Updated	Status
Leave one space after the comma.	Nilay Vaish	May 18, 2012, 3:25 a.m.	Resolved
The code for creation of disk array seems to be common to all the architectures. Can you pull this out ...	Nilay Vaish	May 18, 2012, 4:04 a.m.	Resolved
Is there any difference between the class CowMmDisk and CowIdeDisk?	Nilay Vaish	May 18, 2012, 3:26 a.m.	Resolved
Space between comma and the next object.	Nilay Vaish	May 18, 2012, 3:29 a.m.	Resolved
I think you inadvertently deleted a line here.	Nilay Vaish	May 19, 2012, 1:22 a.m.	Resolved
Can the comma appear on the previous line?	Nilay Vaish	May 19, 2012, 1:26 a.m.	Resolved
This appears to be the least pythonesque way to do this. how about for dn in self.diskname: disks.append(dn.strip()) return disks	Ali Saidi	May 20, 2012, 6:22 a.m.	Resolved
So why have you bothered adding the swap disk if you're going to just undo that here?	Ali Saidi	May 20, 2012, 6:18 a.m.	Open
if you specify more than one disk this is likely going to break sparc in general.	Ali Saidi	May 20, 2012, 12:28 a.m.	Open

configs/common/Benchmarks.py (Diff revision 1)

Leave one space after the comma.

Show all issues

configs/common/FSConfig.py (Diff revision 1)

The code for creation of disk array seems to be common to all the architectures. Can you pull this out into a separate function?

Show all issues

Jayneel Gandhi May 18, 2012, 4:05 a.m. (May 18, 2012, 4:05 a.m.)

All the systems except SPARC calls a common function called 'create_disks'. Uploading the new patch soon.

configs/common/FSConfig.py (Diff revision 1)

Is there any difference between the class CowMmDisk and CowIdeDisk?

Show all issues

Jayneel Gandhi May 18, 2012, 4:07 a.m. (May 18, 2012, 4:07 a.m.)

CowMmDisk is re-introduced in the new patch. This was the problem with SPARC FS.

tests/configs/pc-o3-timing.py (Diff revision 1)

Space between comma and the next object.

Show all issues

Description:

~		Changeset 8984:a567fd6f8748
	~	Changeset 9008:868c58107488

		Config: Make disk-image option support mutiple disks
		Added support to specify mutiple disks provided on command line with
		disk-image option. Made the required changes to SysConfig to support that.
		Updated regressions to accomodate the stat name changes and change in
		SysConfig. When you use this option, you have to specify disks in specific
		order and the default disks for the system also needs to be specified.
		The operating system disk is to be specified first in the comma-spearated list.
		The new disks that you specify can be mounted by using mount command once the
		operating system is booted. Like for X86 linux, you can mount the new disk from
		/dev/hdc, if the option was used as --mem-disk=linux disk,swap disk,new disk.

Diff:

Revision 2 (+157 -151)

Show changes

	configs/common/Benchmarks.py
	configs/common/FSConfig.py
	configs/common/Options.py
	configs/example/fs.py
	tests/configs/pc-o3-timing.py
	tests/configs/pc-simple-atomic.py
	tests/configs/pc-simple-timing.py
	tests/configs/pc-simple-timing-ruby.py
	15 more

Change Summary:

Incorporated Nilay's changes

Testing Done:

~		build/ALPHA/tests/opt/quick/se/20.eio-short/alpha/eio/simple-atomic FAILED!
~		build/ALPHA/tests/opt/quick/se/20.eio-short/alpha/eio/simple-timing FAILED!
	~	All FS test passed except SPARC FS (FS files for SPARC no available)
	~	All SE tests passed except eio tests.
-		build/ALPHA/tests/opt/quick/se/30.eio-mp/alpha/eio/simple-atomic-mp FAILED!
-		build/ALPHA/tests/opt/quick/se/30.eio-mp/alpha/eio/simple-timing-mp FAILED!
-		build/SPARC/tests/opt/long/fs/80.solaris-boot/sparc/solaris/t1000-simple-atomic FAILED!
-		build/SPARC/tests/opt/quick/se/02.insttest/sparc/linux/inorder-timing FAILED!
-		build/SPARC/tests/opt/quick/se/02.insttest/sparc/linux/o3-timing FAILED!
-		build/SPARC/tests/opt/quick/se/02.insttest/sparc/linux/simple-atomic FAILED!
-		build/SPARC/tests/opt/quick/se/02.insttest/sparc/linux/simple-timing FAILED!
-		build/SPARC/tests/opt/quick/se/40.m5threads-test-atomic/sparc/linux/o3-timing-mp FAILED!
-		build/SPARC/tests/opt/quick/se/40.m5threads-test-atomic/sparc/linux/simple-atomic-mp FAILED!
-		build/SPARC/tests/opt/quick/se/40.m5threads-test-atomic/sparc/linux/simple-timing-mp FAILED!
-
-		Rest all passed. SPARC SE test failed due to tests not available with me. But changes are made to FS mode, SE tests wont be affected.

Unless required, try to not include the stat updates in patches. They kind of clutter
the patch.

configs/common/FSConfig.py (Diff revision 2)

I think you inadvertently deleted a line here.

Show all issues

configs/example/fs.py (Diff revision 2)

Can the comma appear on the previous line?

Show all issues

Description:

~		Changeset 9008:868c58107488
	~	Changeset 9008:b2a8dc3c8a2e

		Config: Make disk-image option support mutiple disks
		Added support to specify mutiple disks provided on command line with
		disk-image option. Made the required changes to SysConfig to support that.
		Updated regressions to accomodate the stat name changes and change in
		SysConfig. When you use this option, you have to specify disks in specific
		order and the default disks for the system also needs to be specified.
		The operating system disk is to be specified first in the comma-spearated list.
		The new disks that you specify can be mounted by using mount command once the
		operating system is booted. Like for X86 linux, you can mount the new disk from
		/dev/hdc, if the option was used as --mem-disk=linux disk,swap disk,new disk.

Diff:

Revision 3 (+157 -150)

Show changes

	configs/common/Benchmarks.py
	configs/common/FSConfig.py
	configs/common/Options.py
	configs/example/fs.py
	tests/configs/pc-o3-timing.py
	tests/configs/pc-simple-atomic.py
	tests/configs/pc-simple-timing.py
	tests/configs/pc-simple-timing-ruby.py
	15 more

This still seems like a a not completely adequate bandaid for a problem that goes a lot deeper than just specifying disk images on the command line.

Jayneel Gandhi May 20, 2012, 6:48 a.m. (May 20, 2012, 6:48 a.m.)

At UW we have a lot of different disk images that have different benchmark binaries. I didn't want to change the X86root and sawp disk every time, I want to run some other benchmark suite. So, I had first designed it provide that support for X86. I later on extended it for other systems, so that every one can use the functionality. Also, a lot of other users wanted to know how to add more disks to to the systems. I think they have their binaries in a new disk that they have created and wanted to mount that disk. I extended the patch to make that possible to the extent I have the files available to run FS modes.

I know the main problem is accessing the host file system, instead of going through the whole hassle of disk drives to have your own program binaries. I am not an expert in file systems and virtualization to comment on how can that be achieved in gem5. This was an intermediate solution that will relieve some problems that users face today. I know that simics used to provide access to the host file systems without any problems, but I suppose, gem5 full system did not have that ever built in.

A discussion on this is necessary moving forward. It is painful to manage different disk images or making new ones/adding more stuff to existing ones for your experiments. It requires root access to mount disks or use some virtual machines or use guestfs for those who have that installed to mount disks without root access. Especially, we need to think about a lot of gem5 users that are university students and placing these constraints is a barrier to get gem5 accepted easily.

configs/common/Benchmarks.py (Diff revision 3)

This appears to be the least pythonesque way to do this.

how about 
for dn in self.diskname:
   disks.append(dn.strip())
return disks

Show all issues

configs/common/FSConfig.py (Diff revision 3)

So why have you bothered adding the swap disk if you're going to just undo that here?

Show all issues

Jayneel Gandhi May 20, 2012, 6:48 a.m. (May 20, 2012, 6:48 a.m.)

If you look at the earlier implementation, sawp disk was added as disk2. But was never added as a simple disk. I wanted to be consistent with the earlier method.

But let me make a new disk drive and boot up full systems other than X86 like ALPHA and ARM where such a simple disk is created, to see how they will access the new disk drive when specified on command line. I have been using the implementation on X86 for some time and is working fully, but it does not have the simple disk creation.

configs/common/FSConfig.py (Diff revision 3)

if you specify more than one disk this is likely going to break sparc in general.

Show all issues

Jayneel Gandhi May 20, 2012, 6:48 a.m. (May 20, 2012, 6:48 a.m.)

I do not have the SPARC FS files to test. If any one can provide me access to them, I am willing to try and get it working for SPARC as well. I tried to provide a solution for SPARC which seems to consistent with the earlier implementation, but could not test.

Description:

~		Changeset 9008:b2a8dc3c8a2e
	~	Changeset 9008:940d21e358ea

		Config: Make disk-image option support mutiple disks
		Added support to specify mutiple disks provided on command line with
		disk-image option. Made the required changes to SysConfig to support that.
		Updated regressions to accomodate the stat name changes and change in
		SysConfig. When you use this option, you have to specify disks in specific
		order and the default disks for the system also needs to be specified.
		The operating system disk is to be specified first in the comma-spearated list.
		The new disks that you specify can be mounted by using mount command once the
		operating system is booted. Like for X86 linux, you can mount the new disk from
		/dev/hdc, if the option was used as --mem-disk=linux disk,swap disk,new disk.

Diff:

Revision 4 (+157 -150)

Show changes

	configs/common/Benchmarks.py
	configs/common/FSConfig.py
	configs/common/Options.py
	configs/example/fs.py
	tests/configs/pc-o3-timing.py
	tests/configs/pc-simple-atomic.py
	tests/configs/pc-simple-timing.py
	tests/configs/pc-simple-timing-ruby.py
	15 more

You have a pending review.

Review Board 2.0.15

Screenshots

Files

Issue Summary

Description:

Changeset 8984:a567fd6f8748

Changeset 9008:868c58107488

Diff:

Change Summary:

Testing Done:

Description:

Changeset 9008:868c58107488

Changeset 9008:b2a8dc3c8a2e

Diff:

Description:

Changeset 9008:b2a8dc3c8a2e

Changeset 9008:940d21e358ea

Diff: