Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an infer mode guarantying that data sample is valid against an inferred schema #111

Open
anuveyatsu opened this issue Nov 23, 2017 · 1 comment
Labels

Comments

@anuveyatsu
Copy link
Member

By default, infer method reads only first 100 rows when generating schema. However, there is some situations when we need to increase that limit, e.g., when first 100 rows contain integers, but then there is a decimal numbers. If I set limit option when calling infer, it still returns schema that is not correct. May be I am doing something wrong:

https://runkit.com/anuveyatsu/tableschema-infer-not-working-properly

as you can see field type for "Value" is "integer", however, there are decimal numbers (eg, in row 215)

@roll
Copy link
Member

roll commented Dec 10, 2017

@anuveyatsu
I don't think it's a good term to name an inferred schema as correct or not correct. An inferring is intended to be a schema bootstrap step. And now it uses a fast algorithm based on type/format confidence.

I think it's better to re-formulate this issue to support infer mode that guarantee that provided sample is a valid against an inferred schema.

@roll roll changed the title Infer not generating correct schema Add an infer mode guarantying that data sample is valid against an inferred schema Dec 10, 2017
@roll roll added the feature label Dec 10, 2017
@roll roll closed this as completed Apr 6, 2020
@roll roll reopened this Apr 6, 2020
@roll roll added review and removed review labels Apr 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants