Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Detectors and Probers for target languages #53

Open
rstm-sf opened this issue Feb 23, 2019 · 7 comments
Open

Add Detectors and Probers for target languages #53

rstm-sf opened this issue Feb 23, 2019 · 7 comments

Comments

@rstm-sf
Copy link
Collaborator

rstm-sf commented Feb 23, 2019

Hello!

It may be worth adding the ability to determine the encoding if you know which target language?

@304NotModified
Copy link
Member

Hi,

Sorry for the late response.

What do you mean with this?

@rstm-sf
Copy link
Collaborator Author

rstm-sf commented Aug 31, 2019

Hello!

I created a pr #63 for ease of understanding.

In order to detect the encoding prober's objects are created. They are defined for multiple languages. With a small sample of characters to detect the encoding, conflicts may arise between the encodings due to the possibility of being a character code in different languages.

But, what if we need to define an encoding, knowing that it can belong to only one language? Then you can restrict yourself to probers only for a given language, reducing the likelihood of incorrect detections.

PS. Sorry for my english.

@304NotModified 304NotModified self-assigned this Aug 31, 2019
@304NotModified
Copy link
Member

304NotModified commented Sep 21, 2019

sound good, but now sure how easy it is to change that is this code base.

@304NotModified 304NotModified removed their assignment Sep 21, 2019
@rstm-sf
Copy link
Collaborator Author

rstm-sf commented Nov 9, 2019

It seems to me that first we need to try to single out single-byte probers by language, as models

@rstm-sf
Copy link
Collaborator Author

rstm-sf commented Feb 24, 2020

Hello, @304NotModified !

We can make breaking changes and override, using internal, everything that is in src/Core? This would make it easier to change the code.

@304NotModified
Copy link
Member

do you mean if making breaking changes in src/core is OK? I think it is. We should make them internal also

@rstm-sf
Copy link
Collaborator Author

rstm-sf commented Feb 24, 2020

I think it would be nice if we could just change the source in src/core without thinking about breaking changes. That is, change the modifier from public to internal.

I just have the idea of separating probers as models into languages (however, it will take a lot of time, there are about 100 of them). And it would be nice then to change the namespace

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants