Docs: Progress bar for pipelines

[rgriff23]

We plan to add an optional progress bar to the run method in the dependency, fluxus: BCG-X-Official/fluxus#23

It will then be available to the run method in artkit and we should introduce it in the "Building Your First ARTKIT Pipeline" tutorial, specifically in the section where we introduce the core ARTKIT functions.

Once the progress bar is added and we have updated the fluxus version in artkit, we will re-open this issue.

[ParthJohri]

Goodmorning @rgriff23 , I would like to work on this issue, could you please assign it to me?

Also, if I have any doubts, can I reach out to you on this thread?

[rgriff23]

Hi @ParthJohri! You can certainly pick this up - I suggest tagging @seanggani for questions.

[seanggani]

Hi @ParthJohri , apologies for missing your comment! Please let us know if there's anything we can help with

[ParthJohri]

Thank you, @seanggani @rgriff23 for assigning the issue.

Screen Recording & Screenshots

I implemented this progress bar using progressbar2 package instead of tqdm on the page to show how to instantiate a Gemini model class to interact with the Gemini API:

Screen.Recording.2024-07-02.at.21.18.50.mov

So we have to do the same on the pipeline page right?
The progress bar for pipeline by this I understood when the user would run the pipeline, the progress should start and show the progress, right?

Could you please tell if I am understand it wrong or moving in right direction?

[rgriff23]

Thanks @ParthJohri, this is a great start! I'll defer to @seanggani and @breakbotz to review the implementation details.

My main suggestion is for the progress bar to be implemented as an optional argument for the run function, e.g., progress_bar=True displays a progress bar, and I'd suggest it defaults to False. This approach ensures that the progress bar is only shown when it is actually useful, avoiding unnecessary clutter when the pipeline runs quickly due to cached results (which is the case in all of our documentation tutorials) or because the user is testing out a simple pipeline.

I'd suggest we introduce the progress bar explicitly in the "Building Your First ARTKIT Pipeline" tutorial, in the section where we introduce the core ARTKIT functions. Otherwise, I don't think we should include it throughout our documentation tutorials since all the tutorials are using cached results to ensure reproducibility, and we don't want to display progress bars which could give a misleading perception of how fast the pipelines run when there is no cache.

[rgriff23]

@ParthJohri the core team will need to discuss implementation internally before we can give feedback for you to move forward. Thanks in advance for your patience!

[ParthJohri]

Thank you @rgriff23 for your reply, I would be waiting for the response to the implementation.

[rgriff23]

@ParthJohri thanks for making the first pass at solving this! After discussing internally, we realized this is a more complex implementation which needs to be implemented in a dependency called fluxus. The change will require a good understanding of the underlying architecture of fluxes, so we think it is best handled by our core team.

Conclusion from our technical discussion:

The challenge is, there is no straightforward way of telling how many results a flow will produce, given that individual steps or parallel steps can create multiple outputs for any one input. Thus, while it is easy to count how many run results are already complete, we cannot easily translate this to 100%. For a meaningful progress bar, we need to come up with progress estimates for all steps & propagate these along the flow. I.e. once we know the number of inputs, we estimate the number of outputs of every step to be 1, then update that estimate if the input/output ratio starts to be different. in most cases this will lead to a reasonable progress bar. In some cases, estimates may need to be updated and progress can go from say 80% back to 60%.

Some suggestions for alternative ways to contribute to artkit:

1. There are a couple of "good first issues" you could tackle. These aren't very interesting from an engineering perspective, but they would be useful contributions to the documentation:
   a. Add GitHub links to tutorials: Add links to Jupyter notebooks on GitHub at the top of Sphinx tutorials #46
   b. Add badges to README: Add Badges #29
2. For a more interesting engineering challenge, you could take on:
   • Add a new LLM connector for Cohere: Add LLM connector for Cohere #47
3. After we add a progress bar to the run function in fluxus and update the fluxus version used in artkit, we will want to add a demo and explanation to the artkit documentation. If you're interested, we can assign that to you.

Let me know what you think!

[rgriff23]

@ParthJohri thanks for making the first pass at solving this! After discussing internally, we realized this is a more complex change which needs to be implemented in a dependency called fluxus. The change will require a good understanding of the underlying architecture of fluxus, so we think it is best handled by our core team.

FYI - some details from our technical discussion:

The challenge is, there is no straightforward way of telling how many results a flow will produce, given that individual steps or parallel steps can create multiple outputs for any one input. Thus, while it is easy to count how many run results are already complete, we cannot easily translate this to 100%. For a meaningful progress bar, we need to come up with progress estimates for all steps & propagate these along the flow. I.e. once we know the number of inputs, we estimate the number of outputs of every step to be 1, then update that estimate if the input/output ratio starts to be different. in most cases this will lead to a reasonable progress bar. In some cases, estimates may need to be updated and progress can go from say 80% back to 60%.

Some suggestions for alternative ways to contribute to artkit:

1. There are a couple of "good first issues" you could tackle. These aren't very interesting from an engineering perspective, but they would be useful contributions to the documentation:
   • Add GitHub links to tutorials: Add links to Jupyter notebooks on GitHub at the top of Sphinx tutorials #46
   • Add badges to README: Add Badges #29
2. For a more interesting engineering challenge, you could take on:
   • Add a new LLM connector for Cohere: Add LLM connector for Cohere #47
3. After we add a progress bar to the run function in fluxus and update the fluxus version used in artkit, we will want to add a demo and explanation to the artkit documentation. I have rescoped this issue to be documentation-only and plan to re-open it once the progress bar is available. If you're interested in taking that on, we can keep you assigned.

Let me know what you think!

[ParthJohri]

Thank you @rgriff23 for the clarification, I would be glad to work on the beginner issues and make my way to understand the code base to contribute further.

I will be happy to be assigned the documentation and explanation part.

[rgriff23]

@ParthJohri great! Do you mind checking the beginner issues and commenting on the one you'd like to pick up first? I can't assign an external contributor until you engage with the issue.

[ParthJohri]

@ParthJohri great! Do you mind checking the beginner issues and commenting on the one you'd like to pick up first? I can't assign an external contributor until you engage with the issue.

Absolutely @rgriff23 thank you so much, also sorry for not being able to reply promptly, I am unwell at the moment.
I would surely look at other issues as soon as I feel better, pick up one and would tag you to get it assigned.