
Brandon Potter got review request #3113!
ruby: allows multiple instances of ruby to be invoked
Review Request #3113 - Created Sept. 14, 2015 and discarded
Information | |
---|---|
Brandon Potter | |
gem5 | |
default | |
Reviewers | |
Default | |
Changeset 11093:3c32e2cb1c87 --------------------------- ruby: allows multiple instances of ruby to be invoked This patch removes the restriction that there can only be a single ruby system object in a given simulation. With the configuration supplied, it is possible to run an instance of ruby with any of the supplied topologies. The problem centers around the use of globals and static class members in the ruby source. Conceptually, this prohibits multi-instance simulations since the ruby systems end up sharing values which should be distinct. (The earliest manifestation of this problem occurs when important runtime components use shared variables for MachineType sanity checks which will trigger failures.) The idea behind the patch is to move as many as the statics/globals as necessary into the ruby system object. The ruby system object acts as the single point of reference for the child objects which needed the globals/statics. With multi-instance runs, each ruby system object will contain distinct values for the previous globals/statics. Unfortunately, this requires passing the ruby system pointer throughout the object hierarchy and may incur some minor performance overhead due to extra indirect references through the ruby system object.
gem5/util/regress (to check builds) and custom tests (which use the new configuration)
Issue Summary
Hi Brandon,
Does this mean we can compile Ruby once (for all protocols), and then dynamically instantiate it? If so, that's great!
The next step would be to build the entire src/mem tree once for all ISAs.
This patch really needs to be broken into smaller chunks. It contains numerous changes without sufficient description, which will make it very difficult to merge with. In general, these changes need better object-oriented design. Also, Reviewboard won't load/display changes to RubySystem.cc, which is pretty important for reviewing this patch.
I recommend splitting out the following as separate patches:
1) Move controller, directory, etc. variables into RubySystem, add accessors as appropriate, remove static designators as appropriate, add const designators as appropriate and initialize.
2) Move testAndRead and testAndWrite into DataBlk.
3) Distribute RubySystem pointers. To do this, introduce a heritable class (let's call it 'RubySystemComponent' for the purpose of discussion), which contains a RubySystem pointer and some miscellaneous functions (Feel free to ping me if you need more detail on how to do this. I feel pretty strongly that we should do this as described here):
A) Inherit from RubySystemComponent in all persistent simulation objects (simulated system components, but not state) that need a RubySystem pointer (e.g. CacheMemory, DirectoryMemory, PerfectCacheMemory, Network, NetworkInterface, etc.). This inherited type will at least signal how the RubySystem pointers should be distributed as compared to this patch, which just uses them wherever to get things working. Further, this will unify the naming of the RubySystem pointer, so that every component will reference it with the same variable name for consistency.
B) Make machineCount(), makeLineAddress(), mapAddressToRange(), and map_Address_to_Directory() (etc.?) functions of both the RubySystem and RubySystemComponent. This will eliminate the need to pass a RubySystem pointer or call these functions on a RubySystem pointer by encapsulating their functionality in the inherited class functions. In RubySystemComponent, call the appropriate RubySystem function through the RubySystem pointer.
C) Make MachineType_base* functions of RubySystem and RubySystemComponent. NOTE: This patch currently appears to make it so that multiple RubySystem instances must share a common coherence protocol. Eventually, a RubySystem instance might track the protocol it is simulating, so the MachineType_base* functions would need to be protocol-dependent. These can still be generated code but implement RubySystem/RubySystemComponent functions.
D) Avoid getRubySystem() functions. In a few cases, the presence of these appear to signal that a component is missing a RubySystem pointer when it should have one. In other cases, child classes (e.g. ports within other components) can have direct access to the pointer in the parent class, since there is sufficient encapsulation/abstraction of who owns and can access the pointer.I've provided lots of comments below, but I would strongly prefer that this patch be broken into smaller chunks.
-
src/cpu/testers/rubytest/Check.cc (Diff revision 1) -
In general, it doesn't make sense for transient state, like a SenderState, to carry a pointer to a persistent simulation object like the RubySystem. I'm not sure why is is necessary to add the RubySystem pointer here, but it should be reconsidered and removed if possible.
-
src/cpu/testers/rubytest/Check.cc (Diff revision 1) -
This patch uses getRubySystem() functions more common than it should. Based on the number of uses like this, Nilay is right: They're going to cause simulator performance degradation, because they're not just traversing a pointer, but also calling a function to get the pointer.
-
src/cpu/testers/rubytest/RubyTester.hh (Diff revision 1) -
Per above, a SenderState shouldn't have a RubySystem pointer.
-
src/mem/protocol/MESI_Three_Level-L0cache.sm (Diff revision 1) -
These constructor tags on MessageBuffer declarations are unnecessary, since the constructor is called from SWIG code. These should all be removed.
-
src/mem/protocol/MOESI_CMP_directory-L2cache.sm (Diff revision 1) -
I don't recall where we ended on the mapping function discussion. However, mapping functions will be specific to each RubySystem instance. I believe we had the preference that these calls end up looking like:
ruby_system.map_Address_to_Directory(address)
-
src/mem/protocol/MOESI_CMP_directory-L2cache.sm (Diff revision 1) -
This doesn't make sense. Cache entries should not need a pointer to the RubySystem. Certainly the caches have RubySystem pointers, right?
-
src/mem/protocol/RubySlicc_ComponentMapping.sm (Diff revision 1) -
Almost all of these functions are going to need to be RubySystem instance specific...
-
src/mem/protocol/RubySlicc_Exports.sm (Diff revision 1) -
Not sure what this means...? Why is this declaration necessary?
-
src/mem/protocol/RubySlicc_Exports.sm (Diff revision 1) -
Transient state shouldn't have pointers to persistent objects.
-
src/mem/protocol/RubySlicc_Types.sm (Diff revision 1) -
Transient state shouldn't have pointers to persistent objects.
-
src/mem/protocol/RubySlicc_Util.sm (Diff revision 1) -
These are RubySystem instance specific functions, since calculating the return values depends on a RubySystem's cache block size. These functions should be moved into RubySystem.
-
src/mem/ruby/SConscript (Diff revision 1) -
Is this correct? Nothing about the files is changed by this patch...
-
src/mem/ruby/common/Address.hh (Diff revision 1) -
See prior comment.
-
src/mem/ruby/common/DataBlock.cc (Diff revision 1) -
Pass around the cache line size? Sure, it makes sense to do so, since it is a property of the DataBlock. In contrast, passing around a RubySystem pointer just to get line size is very cumbersome.
-
src/mem/ruby/slicc_interface/RubySlicc_Util.hh (Diff revision 1) -
I don't even know why we have these functions around: DataBlk has functions to do reads/writes, and these largely replicate that functionality with a little extra for checking correct addresses.
-
src/mem/ruby/structures/DirectoryMemory.cc (Diff revision 1) -
Needs to be set in a different way with more appropriate abstraction (i.e. m_numa_high_bit should be private in RubySystem, and set once not by components outside RubySystem).
-
src/mem/ruby/structures/PerfectCacheMemory.hh (Diff revision 1) -
If there were appropriate inheritance unifying PerfectCacheMemory with the CacheMemory and DirectoryMemory, the assumptions would be the same about addresses being line addresses.
-
src/mem/ruby/structures/PerfectCacheMemory.hh (Diff revision 1) -
ENTRY should not require a RubySystem pointer.
-
src/mem/ruby/system/RubySystem.hh (Diff revision 1) -
Is m_num_directories not the same as something like MachineType_base_count(Directory)? If possible, I'd recommend trying to eliminate redundant variables here.
-
src/mem/ruby/system/RubySystem.hh (Diff revision 1) -
Make these private, and add accessor functions. These should not be directly accessible from other components. Where possible, initialize in constructor initializer lists and mark variables as const.
-
src/mem/ruby/system/Sequencer.cc (Diff revision 1) -
Transient state shouldn't have access to a RubySystem pointer.
-
src/mem/slicc/symbols/Type.py (Diff revision 1) -
Most of these changes shouldn't happen, since cache entries shouldn't have a RubySystem pointer.
-
src/mem/slicc/symbols/Type.py (Diff revision 1) -
These functions need to be moved into the RubySystem, since they are properties of each system instance. As evidence, in line 736, you're just calling a function of the RubySystem anyway.
-
src/python/swig/pyobject.cc (Diff revision 1) -
Is this necessary? If not, please remove.
Pretty cool that you have Ruby / gem5 working with multiple systems! Kudos for getting everything to actually work.
However, at a high-level, this patch makes the software engineer in me sad about the current state of Ruby. There are so many inter-dependencies on objects (mostly the RubySystem). Can you imagine the UML diagram of Ruby?! I'm not sure what to do about it, but it seems to me that we need a serious effort in the community to solve some of the inherent problems with Ruby's current codebase. This patch, and many of Nilay's patches as well, seem to just "patch" the underlying issues. I don't have any specific solutions, and I'm sorry to bring this rant up again...
-
configs/common/Options.py (Diff revision 1) -
Why add this option here? Why not in multi-system-se.py since that is the only file that uses this option?
-
src/mem/ruby/common/NetDest.hh (Diff revision 1) -
Does it make sense to leave these empty constructors around? Or are there times when a NetDest will be created without a RubySystem pointer?
I'm just commenting here, but there are other places too.
-
src/mem/ruby/common/NetDest.cc (Diff revision 1) -
Another high-level question that applies to more than just this line:
Why not have MachineType_base_count(machineType) as a method in RubySystem, instead of passing the RubySystem pointer around? It wouldn't really change much, but seems like it would be more readable.
Is there any way to make m_ruby_system implicit in SLICC? I really don't like having to change all of the protocols.
-
src/cpu/testers/rubytest/Check.cc (Diff revision 1) -
This needs to be reverted to writeAddr.
Diff: |
Revision 3 (+978 -469)
|
---|
Diff: |
Revision 4 (+793 -362)
|
---|
Diff: |
Revision 5 (+772 -352)
|
---|