Gml 1852 performant doc ingestion pipeline in support ai

[luzhoutg]

No description provided.

[RobRossmiller-TG]

Looks good except for the one change regarding drop_old. Please delete old code instead of commenting it out.

There's a lot of duplication between this package and graphrag. In the future, it'd be good to have a common set of functions that both loaders can use. For example, make_headers is identical in both packages as well as the whole ecc_util.py module

[luzhoutg]

Looks good except for the one change regarding drop_old. Please delete old code instead of commenting it out.

There's a lot of duplication between this package and graphrag. In the future, it'd be good to have a common set of functions that both loaders can use. For example, make_headers is identical in both packages as well as the whole ecc_util.py module

[luzhoutg]

Looks good except for the one change regarding drop_old. Please delete old code instead of commenting it out.

There's a lot of duplication between this package and graphrag. In the future, it'd be good to have a common set of functions that both loaders can use. For example, make_headers is identical in both packages as well as the whole ecc_util.py module

removed all comment out code now. I will start to remove the duplicates after merging parker's and my PR to dev. it would be easier due to less conflicts.

[RobRossmiller-TG]

removed all comment out code now. I will start to remove the duplicates after merging parker's and my PR to dev. it would be easier due to less conflicts.

Absolutely agreed! The comment was as much a note for us to come back to as anything.