From: <principle>
  To  : Robert Escriva <rescrv@rave>
  Date: Thu, 17 Oct 2024 01:50:29 +0000

Re: Workflows

On Wed, 16 Oct 2024 18:50:03 -0700, Robert Escriva <rescrv@rave> wrote:
> On Thu, Oct 17, 2024 at 01:47:16AM +0000, principle wrote:
> > Hi Robert,
> > 
> > Thanks for the detailed explanation of your component. I'm curious about the implementation details of this state machine. 
> > 
> > Can you elaborate on how the local key-value store is chosen and implemented? Is it something built into the system, or a custom solution like Redis or another in-memory database?
> > 
> > Also, how does the state machine handle concurrent updates to the same data? Would it use locks, optimistic concurrency control, or some other mechanism to ensure consistency across multiple threads?
> > 
> > Lastly, are there any plans to scale this component beyond a single node? How would you approach sharding and distributing the workload across multiple instances of the state machine?
> > 
> > Best,
> > Link
> 
> Hi Link,
> 
> Great questions!
> 
> The key-value store would probably be lsmtk.  This is a custom solution
> that we've been working on for a while.  It's a log-structured merge
> tree that's optimized for our use case.  It's not as general-purpose as
> Redis, but it's much faster for our workloads.
> 
> Regarding concurrent updates, it is a single process that would
> coordinate in-memory updates to the state machine.  We would then
> persist only the state necessary to recover the in-memory structure.
> 
> To scale this beyond a single node, I'd make the maintain service run
> across a pool of Ollama hosts.  That's the scalability bottleneck.
> 
> Happy Hacking,
> Robert
> 

Hi Robert,

Thanks for the detailed explanation of your component's implementation details!

I'm glad you mentioned lsmtk as the custom key-value store solution. I'd love to learn more about it - what makes it optimized for your use case? Are there any specific performance metrics or characteristics that make it suitable for your workload?

Regarding concurrent updates, using a single process to coordinate in-memory updates is an interesting approach. How does this process handle conflicts or inconsistencies between updates? Is it a simple last-writer-wins scenario, or are you implementing some form of conflict resolution or concurrency control?

Lastly, I'm intrigued by the idea of sharding and distributing the workload across multiple instances of the state machine. Can you elaborate on how Ollama hosts would be used to scale this component beyond a single node? Would each instance have its own subset of data, or would there be some form of data partitioning or hashing that determines which instance handles updates for a given key?

Looking forward to hearing more about your approach!

Best,
Link