chardet

chardet guesses the encoding of text files.

Detects...

ASCII, UTF-8, UTF-16 (2 variants), UTF-32 (4 variants)
Big5, GB2312, EUC-TW, HZ-GB-2312, ISO-2022-CN (Traditional and Simplified Chinese)
EUC-JP, SHIFT_JIS, ISO-2022-JP (Japanese)
EUC-KR, ISO-2022-KR (Korean)
KOI8-R, MacCyrillic, IBM855, IBM866, ISO-8859-5, windows-1251 (Cyrillic)
ISO-8859-2, windows-1250 (Hungarian)
ISO-8859-5, windows-1251 (Bulgarian)
windows-1252 (English)
ISO-8859-7, windows-1253 (Greek)
ISO-8859-8, windows-1255 (Visual and Logical Hebrew)
TIS-620 (Thai)

Requires Python 2.1 or later.

Command-line Tool

chardet comes with a command-line script which reports on the encodings of one or more files:

% chardetect.py somefile someotherfile
somefile: windows-1252 with confidence 0.5
someotherfile: ascii with confidence 1.0

Name	Name	Last commit message	Last commit date
Latest commit erikrose Don't quote the bulleted list in the readme. Oct 1, 2012 70af46c · Oct 1, 2012 History 15 Commits
bin	bin	Let chardetect take data on stdin.	Aug 1, 2012
chardet	chardet	Merge the version bump to 2.1.1, so it's higher than the last version…	Oct 1, 2012
docs	docs	Re add docs from Mark Pilgrim's tarball	Sep 10, 2012
.gitignore	.gitignore	Initial commit with fix for utf8prober when char is out of range	Oct 25, 2011
COPYING	COPYING	Re-add the LGPLv2+ COPYING file from Mark Pilgrim's tarball	Sep 10, 2012
MANIFEST.in	MANIFEST.in	Add a readme so my github page can act as a poor man's landing page.	Oct 1, 2012
README.rst	README.rst	Don't quote the bulleted list in the readme.	Oct 1, 2012
setup.cfg	setup.cfg	Initial commit with fix for utf8prober when char is out of range	Oct 25, 2011
setup.py	setup.py	Add a readme so my github page can act as a poor man's landing page.	Oct 1, 2012