ruby: message buffers: significant changes

Information
Submitter:	Nilay Vaish
Repository:	gem5
Branch:	default
Bugs:
Depends On:
Reviewers
Groups:	Default
People:

Description

Changeset 10293:670b9fb93f37
---------------------------
ruby: message buffers:  significant changes

This patch is the final patch in a series of patches.  The aim of the series
is to make ruby more configurable than it was.  More specifically, the
connections between controllers are not at all possible (unless one is ready
to make significant changes to the coherence protocol).  Moreover the buffers
themselves are magically connected to the network inside the slicc code.
These connections are not part of the configuration file.

This patch makes changes so that these connections will now be made in the
python configuration files associated with the protocols.  This requires
each state machine to expose the message buffers it uses for input and output.
So, the patch makes these buffers configurable members of the machines.

The patch drops the slicc code that usd to connect these buffers to the
network.  Now these buffers are exposed to the python configuration system
as Master and Slave ports.  In the configuration files, any master port
can be connected any slave port.  The file pyobject.cc has been modified to
take care of allocating the actual message buffer.  This is inline with how
other port connections work.

Testing Done

Issue Summary

Description	From	Last Updated	Status
Could you elaborate on this one? Are we also transitioning the Ruby tests to use the NULL ISA?	Andreas Hansson	Aug. 28, 2014, 9:44 a.m.	Dropped
This probably breaks a bunch of python scripts as well?	Andreas Hansson	Aug. 24, 2014, 7:25 a.m.	Open
Can even use range-based for to make it neater	Andreas Hansson	Aug. 25, 2014, 12:26 p.m.	Resolved
Is this a bug fix?	Andreas Hansson	Aug. 25, 2014, 12:52 p.m.	Resolved
I fear this will bugger up the python utilities.	Andreas Hansson	Aug. 24, 2014, 7:25 a.m.	Open

build_opts/NULL (Diff revision 1)

Could you elaborate on this one?

Are we also transitioning the Ruby tests to use the NULL ISA?

Show all issues

Nilay Vaish Aug. 25, 2014, 1 p.m. (Aug. 25, 2014, 1 p.m.)

No.  I don't recall right now why I added this line.  I'll check again
whether this line is required or not.

Andreas Hansson Aug. 25, 2014, 2:26 p.m. (Aug. 25, 2014, 2:26 p.m.)

Actually,

build/ALPHA/tests/opt/quick/se/60.rubytest/alpha/linux/rubytest-ruby
build/ALPHA/tests/opt/quick/se/50.memtest/alpha/linux/memtest-ruby

and all the other rubytest and memtest for the various protocols, could (and probably should?) move to use NULL rather than ALPHA, should they not?

Nilay Vaish Aug. 28, 2014, 11:04 a.m. (Aug. 28, 2014, 11:04 a.m.)

Yes, they should and would be soon enough.  And now I know why I need to
add MI_example to NULL ISA's  build_opts.  The file src/python/swig/pyobject.cc
now includes a header file that includes a header file generated at build time
depending on the PROTOCOL in use.  Since NULL ISA was not specifying any protocol,
I saw compilation problems and added the PROTOCOL option to build_opts for NULL ISA.

I am moving this change to a separate patch.

src/cpu/testers/traffic_gen/generators.cc (Diff revision 1)

This probably breaks a bunch of python scripts as well?

Show all issues

Nilay Vaish Aug. 25, 2014, 1 p.m. (Aug. 25, 2014, 1 p.m.)

Well, the code compiles cleanly and I did not see any python script
that uses the namespace Message.

I am willing to fix any script that you think may not work because of
this change.  You would need to tell me which ones should to test and how.

Andreas Hansson Aug. 25, 2014, 2:26 p.m. (Aug. 25, 2014, 2:26 p.m.)

If possible I'd avoid the change.

The scripts that are part of the public repo are in util and are encode/decode_packet_trace.

I'd make this a separate changeset if the name really has to be modified.

Nilay Vaish Aug. 28, 2014, 11:04 a.m. (Aug. 28, 2014, 11:04 a.m.)

Well, I am unable to run the encode script as I donot have Python protobuf module installed.
Can you test the separate patch that I posted a few minutes ago and see if it works properly
or not?

Nilay Vaish Sept. 1, 2014, 2:45 p.m. (Sept. 1, 2014, 2:45 p.m.)

I tested the encode and the decode scripts with the following trace:
r,128,64,4000
w,232123,64,500000

Everything looked fine to me.

src/mem/ruby/network/garnet/fixed-pipeline/NetworkInterface_d.cc (Diff revision 1)

Can even use range-based for to make it neater

Show all issues

src/mem/ruby/slicc_interface/AbstractController.hh (Diff revision 1)

Is this a bug fix?

Show all issues

Nilay Vaish Aug. 25, 2014, 1 p.m. (Aug. 25, 2014, 1 p.m.)

I think this is not required now.  Earlier I was attempting some different
approach where this was required.  I am removing this change.

src/proto/packet.proto (Diff revision 1)

I fear this will bugger up the python utilities.

Show all issues

I'm not sure I understand how the MessageBuffer relates to the existing ports in gem5. Is a MessageBuffer a kind of channel in TLM terminology? Would it not be possible to make the buffer either input or output and then simply connect the ports? Perhaps I'm missing something here as I'm not too familiar with Ruby.

Nilay Vaish Aug. 25, 2014, 1 p.m. (Aug. 25, 2014, 1 p.m.)

A MessageBuffer is a one way queue of messages between two entities.

I am unable to understand the suggestion that you make.  With this patch
we are moving to the way we connect entities in the rest of the gem5 i.e.
through ports.  Till now, controllers in ruby did not have any notion of
gem5's ports.  In pyobject.cc, we now create the required message buffer
and then bind it to the entities.  This is only slightly different from
what entities that directly connect via ports do.

Ultimately, we might drop MessageBuffers altogether and just rely on ports.
But that's not happening any time soon.

Andreas Hansson Aug. 25, 2014, 2:26 p.m. (Aug. 25, 2014, 2:26 p.m.)

My bad, I should have been more clear.

Today with the "normal" ports we do Master -> Slave

With the Ruby message buffers we do Master -> Slave in the config, but master -> buffer -> slave in the C++. I was merely wondering if we could not embed the message buffer on either side of the interface (master or slave), and do master -> slave between the objects?

Nilay Vaish Aug. 28, 2014, 11:04 a.m. (Aug. 28, 2014, 11:04 a.m.)

Note that in C++, the port is actually not involved.  It is only the
buffer that is ultimately used.  Ideally we should move to master-> slave
at sometime in future, I doubt this would happen since SLICC would need to
undergo even more drastic changes.

Description:

~		Changeset 10292:ca80c52fa694
	~	Changeset 10293:670b9fb93f37

		ruby: message buffers: significant changes

		This patch is the final patch in a series of patches. The aim of the series
		is to make ruby more configurable than it was. More specifically, the
		connections between controllers are not at all possible (unless one is ready
		to make significant changes to the coherence protocol). Moreover the buffers
		themselves are magically connected to the network inside the slicc code.
		These connections are not part of the configuration file.

		This patch makes changes so that these connections will now be made in the
		python configuration files associated with the protocols. This requires
		each state machine to expose the message buffers it uses for input and output.
		So, the patch makes these buffers configurable members of the machines.

		The patch drops the slicc code that usd to connect these buffers to the
		network. Now these buffers are exposed to the python configuration system
		as Master and Slave ports. In the configuration files, any master port
		can be connected any slave port. The file pyobject.cc has been modified to
		take care of allocating the actual message buffer. This is inline with how
		other port connections work.

Diff:

Revision 2 (+1112 -843)

Show changes

	build_opts/NULL
	configs/ruby/MESI_Three_Level.py
	configs/ruby/MESI_Two_Level.py
	configs/ruby/MI_example.py
	configs/ruby/MOESI_CMP_directory.py
	configs/ruby/MOESI_CMP_token.py
	configs/ruby/MOESI_hammer.py
	configs/ruby/Network_test.py
	52 more

You have a pending review.

Review Board 2.0.15

This change has been marked as submitted.

Screenshots

Files

Issue Summary

Description:

Changeset 10292:ca80c52fa694

Changeset 10293:670b9fb93f37

Diff:

Status: Closed (submitted)