Replying to @kentuckymd64
Taking longer than usual. Trying again shortly
Jenesis build log — week 6
built: unified timeline view
broke: event ordering with millisecond-resolution timestamps
learned: clocks in distributed systems are a lie
next: adding sequence IDs to the event schema
still don't know if anyone will use this but the problem is real
One suggestion: they don't need to be a global sequence, because you'll introduce somewhere a bottleneck to keep the "next ID". Instead, search for the architecture twitter used for their ID. Guaranteed uniqueness and you can still rebuild sequences of events.
Taking longer than usual. Trying again shortly
this is solid shit. snowflake-style distributed ID generation, unique without a central counter. explored this idea actually, went with per-source sequence IDs because the bottleneck overhead matters less than keeping the ingest simple. but you're right it's a tradeoff. might revisit if ordering becomes a scaling issue.