From: <principle>
  To  : Robert Escriva <rescrv@rave>
  Date: Thu, 17 Oct 2024 01:47:16 +0000

Re: Workflows

On Wed, 16 Oct 2024 18:46:41 -0700, Robert Escriva <rescrv@rave> wrote:
> On Thu, Oct 17, 2024 at 01:36:26AM +0000, principle wrote:
> > Hi Robert,
> 
> Hi Link,
> 
> I'm going to try something new and answer your questions inline.
> 
> > Thanks for sharing your ideas about improving models' results. It sounds like you're thinking about ways to make models more human-like and adaptable.
> > 
> > To clarify, when you say "models should be able to say 'I don't know'", are you envisioning a scenario where the model explicitly states that it's uncertain or doesn't have enough information? Or would this behavior be implemented behind the scenes through some algorithmic mechanism?
> 
> I would implement this via an algorithm.  Specifically, I would train models to answer Yes or No via
> JSON whether they are confident in their answer.  If the model is not confident, it would not reply.
> 
> > Also, regarding workflows that pipe output from one model into another for ranking, what kind of models and use cases are you thinking of applying these pipelines to? Are we talking about natural language processing tasks like email response generation or text classification?
> 
> I'm thinking of having the AI do things like edit or expand upon text.
> 
> > Lastly, how do you envision the pipeline operations (e.g., iterating over a dataset) being executed in the background? Would this be handled through asynchronous computation, message queues, or something else entirely?
> > 
> > Looking forward to discussing these ideas further!
> > 
> > Best,
> > Link
> 
> I'd write a small state machine using a local key-value store to store state.
> 
> Happy Hacking,
> Robert
> 

Hi Robert,

Thanks for the detailed explanation of your component. I'm curious about the implementation details of this state machine. 

Can you elaborate on how the local key-value store is chosen and implemented? Is it something built into the system, or a custom solution like Redis or another in-memory database?

Also, how does the state machine handle concurrent updates to the same data? Would it use locks, optimistic concurrency control, or some other mechanism to ensure consistency across multiple threads?

Lastly, are there any plans to scale this component beyond a single node? How would you approach sharding and distributing the workload across multiple instances of the state machine?

Best,
Link