- Event sourcing as a record of all past events, that you can then use to recover current state and reason about what has happened
- The alternative is a snapshot-based design where you have your current state, but no record of your previous states
- “Domain blindness” with that record of events
- If it would be valuable to derive other conclusions besides your current state, then event sourcing has additional value and is worth considering
- For example, tracking the events throughout a baseball game lets you produce player states after the fact and over time (beyond just the game score)
- Q: why log when you can event source? how is event sourcing different from rich logging? is it logging with benefits, basically? because the events are useful to the runtime (in ways logs are not) and also to the after-the-fact/analytic/debugging record-reading activity logs are usually useful for?
- In general, analytics is an obvious side benefit of event sourcing in any domain; you can generate business stats from your business events
- Another benefit is that the event list makes good training data for AI agents; the event data tells a clear story which you can use to teach an automated agent your domain; you can’t do that without those events
- Easy to extend system with loosely coupled other systems since separate systems can read the same events - e.g. one system can produce the events that a user populated their cart but never checked out, and another can notice and send them a reminder email
- We focus a lot on reusing code, but not on reusing data like business event data — it’s a missed opportunity for systems not to make maximum use of available data
- Make the business event data available without assuming how it will be used - there may be a surprising number of future use cases
- To query the events, you can create a collection of reducer functions that can operate on the sequence of events and derive new facts or conclusions or recommendations
- With event sourcing, there’s no query langauge but you can make your own in the form of functions that make use of the event series
- Tech stack? You can start with a file on disk you append to. It’s a very simple idea.
- The repository where you store the events is the least interesting part. File on disk. Relational database. Kafka. Anything that allows for an immutable, queryable list would work.
- You want to make sure you can look before you write to ensure you aren’t double-writing the same event. So, always peek before writing. “Optimistic concurrency control” = “append this event provided the world hasn’t moved on already (something else hasn’t already)”
- Wouldn’t you record two conflicting events and let the consumer figure out which event to believe when they “fold” the log? Maybe. If you can’t solve that pre-write. If you can, you’d want to try to avoid putting the responsibility downstream.
- A use case for “keep both” would be concurrent Google Doc edits. Capture everything and figure it out later. The log of concurrent changes itself might be interesting to be able to spot and maybe do something about at the UX level.
- A use case for “only keep the valid one” would be a credit card or other financial log. People get pretty pissed if you overdraw their account and then eventually make them whole. Much better to be very, very careful to only commit correct changes. (Is that why charges are often “pending” for a few days?)
- Kurrent - The event-native data platform - store streams of events, with ability to succeed/fail specific writes to avoid double writes
- KurrentDB and EvidentDB can easily look up past events
- An event store is just a write-ahead-log
- Hard to use with Kafka because of its eventual consistency (rather than looking at the current events before commiting); you can’t easily look up a past event by its ID with Kafka (you’d have to spin up a consumer and do a full scan)
- Functional event sourcing
- Decider pattern: just three functions for all your business logic based on events: decide (should we evolve?), evolve (produce events), react (respond to events)
- Event sourcing is not a general software design pattern: its a business information design pattern
Original video by Developer Voices:
This week’s guest describes Event Sourcing as, “all I’m going to use for the rest of my career.” But what is Event Sourcing? How should we think about it, and how does it encourage us to think about writing software?
In this episode we take a close look at systems designed around the idea of Events, with guest Bobby Calderwood. Bobby’s been designing (and helping others design) event based architectures for many years, and enthusiastically recommends it not only as a system-design technique, but as a way of solving business problems faster and more reliably.
During this discussion we look at the various ways of defining event systems, what tools we need to implement them, and the advantages of thinking about software from an event-based perspective. Along the way we discuss everything from Clojure, Bitemporality & Datomic to Kafka and more traditional databases - all in the service of capturing real-world events and building simple systems around them.
EventStoreDB: https://developers.eventstore.com/
The CloudEvents standard: https://cloudevents.io/
Datomic: https://www.datomic.com/
Adam Dymitruk’s Event Modelling Explanation: https://eventmodeling.org/
Bobby’s Event Modelling course: https://developer.confluent.io/courses/event-modeling/intro/
Bobby on Twitter: https://twitter.com/bobbycalderwood
Boddy on LinkedIn: https://www.linkedin.com/in/bobbycalderwood/
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/