Event Journal

Purpose

The event journal is a lightweight, append-only file on disk that records every push event published to the event bus. It exists for one reason: crash recovery. When a subscriber (such as the context crate) crashes and reconnects, it needs the events it missed during the downtime. The journal provides those events.

The journal is not a general-purpose message queue. It does not support cross-subscriber routing, priority queues, or arbitrary retention queries. It is a write-ahead log for the event bus, optimized for sequential writes and bounded replay.

/var/log/portal/
  audit-app.jsonl          # Audit trail
  audit-compositor.jsonl   # Audit trail
  audit-system.jsonl       # Audit trail
  event-journal.bin        # Push event journal (this page)

Binary Format

The journal uses a binary format rather than JSON. At peak load the event bus can see 100 events per second during resize storms. Serializing 100 JSON objects per second is unnecessary overhead when the consumer (the replay path) already knows the schema. Binary headers are trivially cheap to append and trivially cheap to parse.

Each journal entry has a fixed-size header followed by variable-length fields:

/// Journal entry header: fixed size, followed by variable fields.
#[repr(C)]
pub struct JournalHeader {
    /// Magic bytes: b"PCPJ"
    pub magic: [u8; 4],
    /// Entry format version.
    pub version: u8,
    /// Event domain (as u8 enum discriminant).
    pub domain: u8,
    /// Event type string length.
    pub type_len: u16,
    /// Source string length.
    pub source_len: u16,
    /// Payload length (JSON bytes).
    pub payload_len: u32,
    /// Sequence number.
    pub sequence: u64,
    /// Timestamp (Unix epoch millis).
    pub timestamp_ms: i64,
    /// CRC32 of payload.
    pub checksum: u32,
}

/// Complete journal entry = JournalHeader + type_bytes + source_bytes + payload_bytes

The on-disk layout for a single entry:

Offset	Size	Field
0	4 bytes	Magic (PCPJ)
4	1 byte	Version
5	1 byte	Domain
6	2 bytes	Type string length
8	2 bytes	Source string length
10	4 bytes	Payload length
14	8 bytes	Sequence number
22	8 bytes	Timestamp (ms)
30	4 bytes	CRC32 checksum
34	type_len bytes	Event type string (UTF-8)
34 + type_len	source_len bytes	Source string (UTF-8)
…	payload_len bytes	Payload (JSON)

The CRC32 covers the payload bytes only. It catches disk corruption and partial writes but does not cover the header fields, which are validated separately by the magic bytes and version field.

Journal Index

The journal maintains an in-memory index that maps (source, sequence) to file offsets. This allows the replay path to seek directly to a specific sequence number without scanning the entire file from the beginning.

pub struct EventJournal {
    file: std::fs::File,
    path: PathBuf,
    /// Sequence number to file offset index.
    index: BTreeMap<(String, u64), u64>,  // (source, seq) -> offset
    /// Per-source high-water mark for retention trimming.
    high_water: HashMap<String, u64>,
}

The index is rebuilt on startup by scanning the journal file. This takes a few milliseconds for a typical 10 MiB journal and happens before the event bus accepts subscriptions.

Retention and Compaction

The journal is bounded. It does not grow without limit.

Policy	Value	Rationale
Maximum journal size	50 MiB	Matches the audit log ring buffer size
Retention per source	1 hour of events	Covers a full PCP restart cycle
Compaction trigger	> 40 MiB	Truncate entries older than the retention window
Compaction method	Truncate from head	Advance the read pointer; no file rewrite needed

Compaction is straightforward. Since the file is append-only, truncating old entries means advancing the start offset. The journal does not rewrite the file. It keeps a logical start offset and ignores entries before it. Physical truncation happens at the next clean shutdown or when the journal is rotated.

The high-water mark tracks the oldest sequence number that any active subscriber still needs. If no subscriber has a replay pointer pointing into a given range, those entries are eligible for truncation.

Replay Protocol

When a durable subscriber reconnects after a crash or PCP restart, it provides the sequence number of the last event it successfully processed. The event bus uses this number to scan the journal for events that occurred while the subscriber was down.

Subscriber (context crate)           Event Bus
     |                                   |
     |  subscribe({                      |
     |    durable: true,                 |
     |    last_sequence: 4782,           |
     |  })                               |
     | ---------------------------------->|
     |                                   |
     |                     [journal scan] |
     |                     find seq 4782  |
     |                     replay 4783+   |
     |                                   |
     |<---- event 4783 ------------------|
     |<---- event 4784 ------------------|
     |<---- event 4785 ------------------|
     |<---- ... --------------------------|
     |<---- event 4810 (current) --------|
     |                                   |
     |  acknowledge(seq: 4810)           |
     | ---------------------------------->|
     |                                   |
     |          [live delivery begins]  |
     |<---- event 4811 ------------------|
     |<---- event 4812 ------------------|

Replay is bounded to 1000 events per re-subscribe. If the gap between the subscriber’s last sequence and the current journal head exceeds 1000 events, the subscriber receives a SubscriptionGap notification instead of individual events. At that point, the subscriber must fall back to polling or a full state refresh to recover its view of the world.

Acknowledgment

After replay completes, the subscriber acknowledges the sequence number it has processed up to. This advances the subscriber’s replay pointer. If PCP crashes again before the acknowledgment, the next replay starts from the same point and delivers the same events again.

pub struct AckRequest {
    pub subscription_id: SubscriptionId,
    pub sequence: u64,
}

impl EventBus {
    fn acknowledge(&self, ack: AckRequest) -> Result<(), EventBusError>;
}

Acknowledgment is optional for non-durable subscriptions. A subscriber that does not need crash recovery (for example, a diagnostic logger) can skip it entirely and simply consume events from the channel until it shuts down.

Relationship to Subscription Lifecycle

The journal is the mechanism behind the SUSPENDED and REPLAYING states described in Channels & Overflow. When a subscriber crashes, its channel is dropped but the journal continues to append events for its subscribed domains. When the subscriber reconnects and provides its last sequence number, the journal enters the REPLAYING state and drains the retained events before resuming live delivery.

If the retention window expires (default one hour) before the subscriber reconnects, the journal entries for that period are purged. The subscriber transitions to EXPIRED and starts fresh on its next subscribe call.