[Local GC] Introduce standalone eventing design document (#15570)

* Typing

* First draft

* Update the doc based on feedback

* Next iteration based on feedback

* Iteration feedback

1 files changed, 237 insertions, 0 deletions

diff --git a/Documentation/design-docs/standalone-gc-eventing.md b/Documentation/design-docs/standalone-gc-eventing.md
new file mode 100644
index 0000000000..55f8056c23
--- /dev/null
+++ b/Documentation/design-docs/standalone-gc-eventing.md
@@ -0,0 +1,237 @@
+# Standalone GC Eventing Design
+
+Author: Sean Gillespie (@swgillespie) - 2017
+
+This document aims to provide a specification for how a standalone GC fires
+events that can be collected by trace collectors. Such a feature is highly desirable
+for a standalone GC since it is the primary way that is used to reason about
+GC performance. Since a standalone GC is not permitted to link against the rest of the runtime, all
+communication betwene the runtime and the GC most pass through dynamically-dispatched interfaces.
+
+## Definitions
+
+* An **event** is some unit of information that the runtime can issue if requested. In general,
+this is used for lightweight tracing. Managed code can issue events using
+`System.Diagnostics.Tracing.EventSource`. Native code (i.e. the runtime) issues events by calling macros
+that delegate the issuing of events to autogenerated code that is generated to interface with the
+underlying event implementation. Events are only issued if they are turned on; the mechanism by which
+events are turned on is not in the scope of this document
+* The **payload** of an event is some amount of data that is delivered with the event itself. Its size
+may be variable. Most events that are fired by the runtime have a schema (predefined layout), but it
+is not a requirement.
+
+## Goals
+
+The goal of this document is to describe a system that allows for the efficient firing of
+performance events by a standalone GC. This system must have three properties in order to be
+acceptable:
+
+1. It must be *efficient* to query whether or not a particular event is turned on. It is not acceptable
+to perform an indirection (i.e. cross the GC/EE interface boundary) in order to get this information.
+2. The cost of firing an event by a standalone GC should be comparable to the cost of firing an event
+without using a standalone GC.
+3. A standalone GC must be able to add new events without having to recompile the EE.
+
+## Querying Whether Events Are Enabled
+
+It is not acceptable to perform an indirection when querying whether or not a requested event is enabled. Therefore,
+it follows that the GC must maintain some state about what events are currently enabled. Events are enabled through
+*keywords* and *levels* on a particular provider; a particular event is enabled if the provider to which the event
+belongs has the event's keyword and level enabled.
+
+The GC fires events from two providers: the "main" provider, `Microsoft-Windows-DotNETRuntime`, and the "private"
+provider, `Microsoft-Windows-DotNETRuntimePrivate`. The GC must track the enabled keyword and level status of each
+provider separately. To accomplish this, the GC will contain a class with this signature:
+
+```c++
+enum GCEventProvider
+{
+    GCEventProvider_Default = 0,
+    GCEventProvider_Private = 1
+};
+
+
+class GCEventStatus
+{
+public:
+    // Returns true if the given keyword and level are enabled for the given provider,
+    // false otherwise.
+    static bool IsEnabled(GCEventProvider provider, int keyword, int level);
+
+    // Enables events with the given keyword and level on the given provider.
+    static bool Enable(GCEventProvider provider, int keyword, int level);
+
+    // Disables events with the given keyword and level on the given provider.
+    static bool Disable(GCEventProvider provider, int keyword, int level);
+};
+```
+
+The GC will use `GCEventStatus::IsEnabled` to query whether or not a particular event is enabled. Whenever the EE observes
+a change in what keywords or levels are enabled for a particular provider, it must inform the GC of the change so that
+it can update `GCEventStatus` using `Enable` and `Disable`. The exact mechanism by which the EE observes a change in the
+event state is described further below. ("Getting Informed of Changes to Event State").
+
+When the EE *does* observe a change in event state, it must inform the GC of these changes so that it can update its
+state accordingly. The following additions are made to the `IGCHeap` API surface area:
+
+```c++
+class IGCHeap
+{
+    // Enables or disables events with the given keyword and level on the default provider.
+    virtual void ControlEvents(bool enable, int keyword, int level) = 0;
+
+    // Enables or disables events with the given keyword and level on the private provider.
+    virtual void ControlPrivateEvents(bool enable, int keyword, int level) = 0;
+};
+```
+
+The currently enabled keywords and levels are encoded as bit vectors so that querying whether an event is enabled
+is efficient:
+
+```c++
+uint32_t enabledLevels[2];
+uint32_t enabledKeywords[2];
+
+bool GCEventStatus::IsEnabled(GCEventProvider provider, int keyword, int level)
+{
+    size_t index = static_cast<size_t>(provider);
+    return (enabledLevels[index] & level) && (enabledKeywords[index] & keyword);
+}
+```
+
+## Firing Events
+
+In order to fire an event, the GC will need to communicate with the EE in some way. The EE is ultimately responsible for routing the event to any appropriate subsystems (ETW, LTTNG, EventPipe) and the GC has no knowledge of what it is going to do with events that we give it.
+
+Events are divided into two categories: **known** events and **dynamic** (or **custom**) events. Known events are known
+to the EE and correspond one-to-one to individual event types fired by the underlying platform loggers. Dynamic events are
+events not known to the EE; their use and description is below in the "Dynamic Events" section.
+
+All events are fired through the `IGCToCLREventSink` interface, which is accessed through the `IGCToCLR` interface given
+to the GC on startup:
+
+```c++
+class IGCToCLREventSink
+{
+
+};
+
+class IGCToCLR
+{
+    virtual IGCToCLREventSink* EventSink() = 0;
+};
+```
+
+Every known event is fired through its own dedicated callback on `IGCToCLREventSink`. For example, the `GCEnd` event
+is fired through a callback like this:
+
+```c++
+class GCToCLREventSink : public IGCToCLREventSink { ... }
+
+GCToCLREventSink::FireGCEnd(uint32_t count, uint16_t depth)
+{
+    // ...
+}
+```
+
+`GCTOCLREventSink::FireGCEnd` is responsible for constructing and dispatching the event to platform loggers if the event
+is enabled. The principal advantage of having one callback per known event is that known events can reach into EE internals
+and add data to events that the GC otherwise would not be aware of. Concrete examples of this would be:
+* The addition of `ClrInstanceId` to the payload of many events fired by the GC
+* Getting human-readable type names for objects allocated on the heap
+* Correlating `GCStart` events with collections induced via ETW.
+
+## Defining and Firing Dynamic Events
+
+It is useful for a standalone GC to be able to fire events that the EE was not previously aware of. For example, it is useful for a GC developer to add some event-based instrumentation to the GC, especially when testing new features and ensuring that they work as expected. Furthermore, it is desirable for GCs that are shipped from this repository
+(the "CLR GC") to interopate seamlessly with future versions of the .NET Core EE, which implies that it should be possible
+for the GC within this repository to add new events without having to recompile the runtime.
+
+While it is possible for some eventing implementations to receive events that are created at runtime, not all eventing implementations (particularly LTTNG) are not flexible enough for this. In order to accomodate new events, another method is added to `IGCToCLREventSink`:
+
+```c++
+void IGCToCLREventSink::FireDynamicEvent(
+    /* IN */ const char* eventName,
+    /* IN */ void* payload,
+    /* IN */ size_t payloadSize
+);
+```
+
+A runtime implementing this callback will implement it by having a "catch-all" GC event whose schema is an arbitrary sequence of bytes. Tools can parse the (deliberately unspecified) binary format provided by GC events that use this mechanism in order to recover the data within the payload.
+
+Dynamic events will by fired by the GC whenever developers want to add a new event but don't want to force
+users to get a new version of the runtime in order to utilize the new event.
+
+## Getting Informed of Changes to Event State
+
+There are three mechanisms by which CoreCLR is able to log events: EventPipe, ETW, and LTTNG. When it comes to changing the state of events, EventPipe and ETW both allow users to attach callbacks that are invoked whenever events are enabled or disabled. For these two mechanisms, it is sufficient to use this callback mechanism to call
+IGCEventController::{Enable/Disable}Events from within such a callback in order to inform the GC of changes to tracing state.
+
+LTTNG does not have such a mechanism. In order to observe changes in the eventing state, LTTNG must be polled periodically.
+Other eventing components in CoreCLR must already poll LTTNG for changes in the eventing state, so this design can utilize
+that same poll to inform the GC of changes.
+
+## Implementation Concerns
+
+An implementation of this spec must take care not to perturb the existing GC code base too much. The GC fires events through
+the use of macros generated by the ETW message compiler and carefully mocked by code generation for the other platform
+logging implementations. The eventing scheme in this document will need to provide implementations for all eventing
+macros used by the GC ([1]).
+
+These headers are often auto-generated. We will need to take care to re-use existing code generators if possible - after all,
+we do want it to be easy to add new events. It will likely be difficult to balance auto-generated code with the need for
+subtle custom modifications to event dispatch code.
+
+Tools (specifically PerfView) will want to be enlightened about dynamic events fired by the runtime. The extent to which
+we want to make this experience nice is up to us, but we will most likely want to ship PerfView support for any dynamic event
+that we end up shipping with the CLR GC.
+
+[1]: https://github.com/dotnet/coreclr/blob/cab0db6345a7941f75d991281bcc0079d28ba182/src/gc/env/etmdummy.h#L5-L57 
+
+### Concrete Example: Porting a single known event to IGCToCLREventSink
+
+The following steps illustrate what needs to be done to bring a single event over to `IGCToCLREventSink`:
+
+Two things are needed by the GC in order to fire an event: a way to determine if the event is on, and a way to fire
+the event. Events can be fired even if they aren't enabled (they occasionally are); the platform loggers will ignore the
+event if it is not enabled. However, the GC generally avoids doing expensive eventing-related operations if an event is
+not on.
+
+The GC generally uses the `ETW_EVENT_ENABLED` macro to query whether an event is on. `ETW_EVENT_ENABLED` will be
+implemented in terms of `GCEventStatus::IsEnabled` above, so you will need to define appropriate macros in order for your
+event to work here. This will likely mean that you will need to define a macro that turns the name of your event into a
+pair of a level and a keyword, which will determine whether or not your event is enabled.
+
+To fire the event, you can add your event's callback to `IGCToCLREventSink`:
+
+```c++
+class IGCToCLREventSink
+{
+    virtual void FireYourEvent(
+        /* your event arguments here... */
+    ) = 0;
+};
+```
+
+You can then define the `FireYourEvent` macro in `src/gc/env/etmdummy.h` to point to your new method on `IGCToCLREventSink`
+Your implementation of `FireYourEvent` in the EE will need to calculate any EE-specific data (e.g. `ClrInstanceId`) and
+then forward the event arguments onto the platform logger, which can be done with the `Fire` macros that the EE has access
+to (that we implemented for the GC).
+
+### Concrete Example: Adding a dynamic event
+
+The following steps illustrate what needs to be done to add a new dynamic event:
+
+There are two things that need to be written: the `ETW_EVENT_ENABLED` support for your new event and the macro responsible
+for firing your event. `ETW_EVENT_ENABLED` can be implemented in the same manner as a known event, by introducing a macro
+for your event name that expands to your event's level and keyword.
+
+Firing of a dynamic event ultimately must call `FireDynamicEvent` with an event name and a serialized payload. Therefore,
+it is the responsibility of the GC to format an event's payload into a binary format. It is ultimately up to you(1) to write
+a function that serializes your event's arguments into a buffer and sends the buffer to `FireDynamicEvent`. This code in
+turn can be wired up to the GC codebase by defining a new `FireMyCustomEvent` macro whose arguments are forwarded onto
+the event serialization function.
+
+1. C++ has the ability to auto-generate large swaths of this code. The implementation of this spec will provide a series of
+composable helper functions that automate the serialization of arguments so that they do not have to be written by the
+developer adding a new event.