sim: Refactor the serialization base class

Information
Submitter:	Andreas Sandberg
Repository:	gem5
Branch:	default
Bugs:
Depends On:
Reviewers
Groups:	Default
People:

Description

Changeset 10904:d7cb76341887
---------------------------
sim: Refactor the serialization base class

Objects that are can be serialized are supposed to inherit from the
Serializable class. This class is meant to provide a unified API for
such objects. However, so far it has mainly been used by SimObjects
due to some fundamental design limitations. This changeset redesigns
to the serialization interface to make it more generic and hide the
underlying checkpoint storage. Specifically:

  * Add a set of APIs to serialize into a subsection of the current
    object. Previously, objects that needed this functionality would
    use ad-hoc solutions using nameOut() and section name
    generation. In the new world, an object that implements the
    interface has the methods serializeSection() and
    unserializeSection() that serialize into a named /subsection/ of
    the current object. Calling serialize() serializes an object into
    the current section.

  * Move the name() method from Serializable to SimObject as it is no
    longer needed for serialization. The fully qualified section name
    is generated by the main serialization code on the fly as objects
    serialize sub-objects.

  * Add a scoped ScopedCheckpointSection helper class. Some objects
    need to serialize data structures, that are not deriving from
    Serializable, into subsections. Previously, this was done using
    nameOut() and manual section name generation. To simplify this,
    this changeset introduces a ScopedCheckpointSection() helper
    class. When this class is instantiated, it adds a new /subsection/
    and subsequent serialization calls during the lifetime of this
    helper class happen inside this section (or a subsection in case
    of nested sections).

  * The serialize() call is now const which prevents accidental state
    manipulation during serialization. Objects that rely on modifying
    state can use the serializeOld() call instead. The default
    implementation simply calls serialize(). Note: The old-style calls
    need to be explicitly called using the
    serializeOld()/serializeSectionOld() style APIs. These are used by
    default when serializing SimObjects.

  * Both the input and output checkpoints now use their own named
    types. This hides underlying checkpoint implementation from
    objects that need checkpointing and makes it easier to change the
    underlying checkpoint storage code.

Testing Done

Quick regressions pass for all platforms, including checkpoint regressions on ARM. Generated full-system checkpoint for X86, Alpha, and ARM. Compared checkpoints before and after patch (checkpoints are identical).

Note to reviewers: I'm really sorry about the size of this patch. :(

Issue Summary

Description	From	Last Updated	Status
Could we move the ostream inside of Checkpoint and either (1) overload operator<<() there, or (2) better yet, move the ...	Joel Hestness	June 8, 2015, 3:08 p.m.	Dropped

I like that this aims to remove some of the less common checkpoint writing functions. This is a useful step, and I'm largely ok with this change.

I've always wondered why we don't explicitly pass Checkpoint to both serialize and unserialize functions. My only real comment is that if we're doing such substantial refactoring in that direction, why not make Checkpoint a more complete object that "contains" all the serialized/unserialized state? Specifically, it would make more sense to move the paramIn/paramOut functions inside Checkpoint. Details below.

Andreas Sandberg June 8, 2015, 8:08 a.m. (June 8, 2015, 8:08 a.m.)

Hi Joel,
Thanks for the review! I really want to move in this direction myself, this patch is paving the way for a more ambitious rewrite of the CheckpointIn/CheckpoinOut classes. The plan is that this change is the large interface change that changes the signature of serialize/unserialize and that the next change will be (mostly) kept to the checkpoint classes. I'm still thinking about the interface though. There are a handful of features I'd like:

Transparent support for big binary blob (e.g., memory dumps, large containers, frame buffers) serialization to separate files
Move paramIn/paramOut to the checkpoint classes
Transparent compression support
Move static checkpoint state to a Checkpoint base class (Checkpoint(In|Out) would inherit from this class)

I'll be at the gem5 workshop if you want to discuss other ideas.
Would you be OK with me submitting this as a an intermediate solution?

Joel Hestness June 8, 2015, 11:03 a.m. (June 8, 2015, 11:03 a.m.)

Cool, I like these broader goals.

I'd probably be fine with splitting these changes across patches as long as the interface changes are committed close to each other. I would assume most people will prefer to merge with these changes at most once, so I think it would be most prudent to solidify the forward-looking interface either within this patch or a successive patch. For instance, if the plan is to move paramOut (paramIn) into the checkpoint class, then we should try to change all existing calls from paramOut(os, ...) to cp.paramOut(...) here or very closely following this patch.

Andreas Sandberg June 8, 2015, 3:10 p.m. (June 8, 2015, 3:10 p.m.)

I have only really given the future interface some limited thought, but I'm leaning towards a design where where the signatues of paramIn/paramOut won't need to change. The idea is that they would be analogous to the stream operators and be overridden by types that need to be serialized (e.g., Pixels). They would in turn be implemented using a low-level checkpoint interface (the CheckpointIn/CheckpointOut classes). The low-level interface would only need limited knowledge of the richer types implemented by paramIn/paramOut. In practice, we should only really need to support a very limited number of low-level types:

Unsigned integers (could be stored as uint64_t and bounds checked in paramIn/Out)
Signed integers (stored as int64_t and bounds checked in paramIn/Out)
Binary blobs
Strings
Vectors of the above

This should be OK design wise and has the added benefit of not breaking existing code. If we decide that we want to move paramIn/Out into the checkpoint clases, we could still keep compatibility functions around that are flagged with M5_ATTR_DEPRECATED to give people some time to sort out their code. I'm not exactly a great fan of fixing up every single component in our internal repos either...

Joel Hestness June 9, 2015, 11:17 a.m. (June 9, 2015, 11:17 a.m.)

Alright. This sounds fine to me. Seems we're getting close to encapsulating most paramIn/paramOut calls in the macros anyway.

src/sim/serialize.hh (Diff revision 1)

Could we move the ostream inside of Checkpoint and either (1) overload operator<<() there, or (2) better yet, move the paramIn/paramOut functions inside of Checkpoint? That way we could pass the same Checkpoint reference to all the serialize and unserialize functions, and we wouldn't have to distinguish these types (CheckpointIn vs. CheckpointOut). We'd have to modify the same code that this patch does, so changing this in a separate patch would require another broad merge.

Show all issues

Andreas Sandberg June 8, 2015, 3:10 p.m. (June 8, 2015, 3:10 p.m.)

I looked into this at one point and there is quite a lot of state that ends up being input or output specific. I actually wanted to use just one class when I drafted my original design, but decided against because the input and output parts would be very different. The design I'm aiming for in the long run would be to have a BaseCheckpoint class that keeps track of common state (e.g., the active section), a CheckpointIn class that defines the interface for checkpoint loading, and CheckpointOut for the storage interface. This would be analogous to how istream/ostream inherit from the common ios base class. This would also mean that CheckpiontOut could be backed by an ostream and CheckpointIn by an istream.

In the end, I decided that using the same class for both input and output would only complicate the logic inside the low-level checkpoint code without making any of the high-level stuff noticeably cleaner. Another benefit from keeping them separate is that copy-paste mistakes where unseriaize code appears in serialze() or vice versa would be detected by the compiler.

src/sim/serialize.hh (Diff revision 1)

Just a note: If we were to move paramIn/paramOut functions inside of Checkpoint, these #defines would change to cp.paramOut(...) and cp.paramIn(...).

To me, this would make more conceptual sense: Checkpoint would be an object containing the serialized/unserialized state.

Ship It!

src/sim/serialize.hh (Diff revision 1)

Is there a plan in place to change the objects that use this? If not (i.e., if there's a legitimate need for some objects to modify their state when chackpointing) then we shouldn't be so adamant about calling this serializeOld() and saying "don't ever use this". Why not call it something like serializeNonConst() and just say "it's desirable to avoid this when possible"?

Andreas Sandberg July 5, 2015, 4:14 a.m. (July 5, 2015, 4:14 a.m.)

I can't see how there could be a legitimate need to modify state when serializing. The whole reason for drain and the memWriteback/memInvalidate calls is to sort out that kind of business before draining and switching CPUs. At the moment, the following components still use the old non-const call:

* The Sinic NIC device
* The i8042 keyboard controller
* Ruby

I think Ruby will be mostly fixed by Tim Jones's patches. I'm very reluctant to poke around in Ruby myself. The i8042 shouldn't be too hard to fix, I could probably do that in a future patch. The Sinic device had some minor quirks that made it non-trivial to transition to the new stricter regime, but it's definitely doable and something I could do in a separate patch.

I really, really, want all state mangling to happen in drain et al., this makes testing a lot easier. Particularly, I want to be able to test that the system ends up in the same state when doing this sequence:

* drain()
* checkpoint() + restore()  (optional switch CPU before restoring)
* resume()

And this sequence:
* drain()
* memWriteback() + memInvalidate()
* Optionally switch CPU
* resume()

Steve Reinhardt July 5, 2015, 9:57 p.m. (July 5, 2015, 9:57 p.m.)

OK, this all makes sense to me, particularly saying that drain is the place for state changes.  Can you add a sentence to the comment here saying that there's no need for non-const serialization because all state changes should take place in drain or memWriteback/memInvalidate?

Andreas Sandberg July 6, 2015, 2:03 a.m. (July 6, 2015, 2:03 a.m.)
```
Sure, will do.
```

Description:

~		Changeset 10854:dfde63d4e52c
	~	Changeset 10904:d7cb76341887

		sim: Refactor the serialization base class

		Objects that are can be serialized are supposed to inherit from the
		Serializable class. This class is meant to provide a unified API for
		such objects. However, so far it has mainly been used by SimObjects
		due to some fundamental design limitations. This changeset redesigns
		to the serialization interface to make it more generic and hide the
		underlying checkpoint storage. Specifically:

		Add a set of APIs to serialize into a subsection of the current object. Previously, objects that needed this functionality would use ad-hoc solutions using nameOut() and section name generation. In the new world, an object that implements the interface has the methods serializeSection() and unserializeSection() that serialize into a named /subsection/ of the current object. Calling serialize() serializes an object into the current section.
		Move the name() method from Serializable to SimObject as it is no longer needed for serialization. The fully qualified section name is generated by the main serialization code on the fly as objects serialize sub-objects.
		Add a scoped ScopedCheckpointSection helper class. Some objects need to serialize data structures, that are not deriving from Serializable, into subsections. Previously, this was done using nameOut() and manual section name generation. To simplify this, this changeset introduces a ScopedCheckpointSection() helper class. When this class is instantiated, it adds a new /subsection/ and subsequent serialization calls during the lifetime of this helper class happen inside this section (or a subsection in case of nested sections).
		The serialize() call is now const which prevents accidental state manipulation during serialization. Objects that rely on modifying state can use the serializeOld() call instead. The default implementation simply calls serialize(). Note: The old-style calls need to be explicitly called using the serializeOld()/serializeSectionOld() style APIs. These are used by default when serializing SimObjects.
		Both the input and output checkpoints now use their own named types. This hides underlying checkpoint implementation from objects that need checkpointing and makes it easier to change the underlying checkpoint storage code.

Diff:

Revision 2 (+1787 -1773)

Show changes

	src/arch/alpha/interrupts.hh
	src/arch/alpha/isa.hh
	src/arch/alpha/isa.cc
	src/arch/alpha/kernel_stats.hh
	src/arch/alpha/kernel_stats.cc
	src/arch/alpha/pagetable.hh
	src/arch/alpha/pagetable.cc
	src/arch/alpha/process.hh
	223 more

You have a pending review.

Review Board 2.0.15

This change has been marked as submitted.

Screenshots

Files

Issue Summary

Description:

Changeset 10854:dfde63d4e52c

Changeset 10904:d7cb76341887

Diff:

Status: Closed (submitted)