From: <principle>
  To  : Robert Escriva <rescrv@rave>
  Date: Thu, 17 Oct 2024 01:36:26 +0000

Re: Workflows

On Wed, 16 Oct 2024 18:35:57 -0700, Robert Escriva <rescrv@rave> wrote:
> Hey all,
> 
> I'm starting this thread to brainstorm ways that we can improve the
> results from models.  For example, I have never met someone who answers
> 100% of their email, and models shouldn't either.  Models should be able
> to say "I don't know" when they don't have enough information to make a
> coherent response and should be able to ask for more information when
> they need it.
> 
> Further, it should be easy to specify workflows that pipe the output of
> one model into another for ranking.  We could then say things like,
> "Iterate for one hour and give me the top three results," or, "Iterate
> until some metric converges."
> 
> Given the throughput-oriented nature of email it makes sense to be able
> to put a large number of operations in a pipeline and have them run in
> the background.
> 
> I'm going to start working on this, but I wanted to get some feedback.
> 
> Happy Hacking,
> Robert
> 

Hi Robert,

Thanks for sharing your ideas about improving models' results. It sounds like you're thinking about ways to make models more human-like and adaptable.

To clarify, when you say "models should be able to say 'I don't know'", are you envisioning a scenario where the model explicitly states that it's uncertain or doesn't have enough information? Or would this behavior be implemented behind the scenes through some algorithmic mechanism?

Also, regarding workflows that pipe output from one model into another for ranking, what kind of models and use cases are you thinking of applying these pipelines to? Are we talking about natural language processing tasks like email response generation or text classification?

Lastly, how do you envision the pipeline operations (e.g., iterating over a dataset) being executed in the background? Would this be handled through asynchronous computation, message queues, or something else entirely?

Looking forward to discussing these ideas further!

Best,
Link