Missing examples in the spider2.0

[susnato]

Hi, first of all thanks a lot for the amazing project and all the work you guys have put into it!

I am trying to run the spider agent on spider 2 data -- I followed the README
But I am getting this error -

(spider3) (base) susnato@MCB spider-agent % python run.py --model gpt-4o --ch_only
[2025-01-24 23:36:51,188 INFO run/96-MainProcess] Args: Namespace(max_steps=30, max_memory_length=25, suffix='gpt-4-try1', model='gpt-4o', temperature=0, top_p=0.9, max_tokens=2500, stop_token=None, test_path='../../spider2/examples/spider2.jsonl', example_index='all', example_name='', overwriting=False, retry_failed=False, output_dir='output', plan=False, bq_only=False, local_only=False, dbt_only=False, sf_only=False, ch_only=True, pg_only=False)
[2025-01-24 23:36:51,189 INFO run/176-MainProcess] Running gpt-4o-gpt-4-try1/ch001
[2025-01-24 23:36:51,199 INFO run/187-MainProcess] Removed existing output/gpt-4o-gpt-4-try1/ch001
[2025-01-24 23:36:51,199 INFO spider_agent/74-MainProcess] Initializing...
Container gpt-4o-gpt-4-try1-ch001 stopped and removed.
[2025-01-24 23:36:54,471 INFO spider_agent/155-MainProcess] Connected to container[name=gpt-4o-gpt-4-try1-ch001, id=583149590fe5fcccf14a06ab777bd24e1e62539fbded555588a571f124c18a90] from image spider_agent_clickhouse-image ...
[2025-01-24 23:36:54,473 INFO spider_agent/80-MainProcess] Setting up environment...
Warning: ../../spider2/examples/ch001 is neither a file nor a directory.
Traceback (most recent call last):
  File "/Users/susnato/git_repos/Spider2/methods/spider-agent/run.py", line 262, in <module>
    test(args)
  File "/Users/susnato/git_repos/Spider2/methods/spider-agent/run.py", line 217, in test
    env = Spider_Agent_Env(
          ^^^^^^^^^^^^^^^^^
  File "/Users/susnato/git_repos/Spider2/methods/spider-agent/spider_agent/envs/spider_agent.py", line 82, in __init__
    self.setup_controller.setup(self.config)
  File "/Users/susnato/git_repos/Spider2/methods/spider-agent/spider_agent/controllers/setup.py", line 55, in setup
    config_function(self, **parameters)
  File "/Users/susnato/git_repos/Spider2/methods/spider-agent/spider_agent/configs/general.py", line 149, in copy_all_subfiles_clickhouse_setup
    sh_file_path = sh_files[0]
                   ~~~~~~~~^^^
IndexError: list index out of range
(spider3) (base) susnato@MCB spider-agent % 

It seems like out of 632 examples some of them are not present in the examples(only 617 present) even after I followed the instructions and downloaded them,
these are -
['postgres_mrr001', 'postgres_chinook001', 'postgres_shopify001', 'ch006', 'postgres_google_ads001', 'ch004', 'ch010', 'postgres_tickit001', 'postgres_airport001', 'ch003', 'postgres_airbnb001', 'ch009', 'postgres_tpch001', 'postgres_maturity001', 'postgres_playbook001', 'ch005', 'ch001']

Can you please help me solve this issue and point out any mistakes I made while downloading the data?

cc: @lfy79001

[lfy79001]

Hello, there may be some bugs in the CH and Postgres examples. We will fix them as soon as possible.
If you're studying text-to-SQL, we recommend using spider2-snow or spider2-lite.

For your issue, you can run the following command to skip the examples:
python run.py --model gpt-4o --bq_only --sf_only --local_only --dbt_only

[susnato]

Thanks for the response!! If you don't mind then I will keep the issue open so that others could refer from this too.