a
=bK & @ s d Z ddlmZmZ ddlmZ G dd deZedddg d d
d gd ed
ddg dddgd edddg dddgd edddddgd d gd ed d dg d d!d"gd ed#d$dd%d&gd'd(gd ed)d*dd+d,gd-d.gd ed/d0dd%d&gd1gd2ed3d4dd5gd6d7gd ed8d9dg d d:d;gd edd?d@gd edAdBdg d dCdDgd edEdFdg d dGdHdIgd edJdKddLdMgdNdOgd edPdQdddgdRdSgd edTdUdddgdVdWgd edXdYdg d dZd[gd ed\d]dg d^d_d`gd edadbdg d^dcddgd ededfdg dgdhdigd edjdkdd%d&gdlgd2edmdndddgdodpgd edqdrdg d dsdtgd edudvdddgdwdxgd edydzdg d{d|d}gd ed~ddddgddgd edddddgddgd edddg dgdgdedddg dddgd edddg dddgd eddddgddgd d ZdS )z
Metadata about languages used by our model training code for our
SingleByteCharSetProbers. Could be used for other things in the future.
This code is based on the language metadata from the uchardet project.
)absolute_importprint_function)
ascii_lettersc s* e Zd ZdZd fdd Zdd Z ZS ) Languagea Metadata about a language useful for training models
:ivar name: The human name for the language, in English.
:type name: str
:ivar iso_code: 2-letter ISO 639-1 if possible, 3-letter ISO code otherwise,
or use another catalog as a last resort.
:type iso_code: str
:ivar use_ascii: Whether or not ASCII letters should be included in trained
models.
:type use_ascii: bool
:ivar charsets: The charsets we want to support and create data for.
:type charsets: list of str
:ivar alphabet: The characters in the language's alphabet. If `use_ascii` is
`True`, you only need to add those not in the ASCII set.
:type alphabet: str
:ivar wiki_start_pages: The Wikipedia pages to start from if we're crawling
Wikipedia for training data.
:type wiki_start_pages: list of str
NTc sr t t| || _|| _|| _|| _| jr@|r:|t7 }qLt}n |sLtd|rbd t
t |nd | _ || _
d S )Nz*Must supply alphabet if use_ascii is False )superr __init__nameiso_code use_asciicharsetsr
ValueErrorjoinsortedsetalphabetwiki_start_pages)selfr r
r r r r __class__ >/usr/lib/python3.9/site-packages/chardet/metadata/languages.pyr # s
zLanguage.__init__c C s& d | jjddd | j D S )Nz{}({})z, c s s( | ] \}}| d sd||V qdS )_z{}={!r}N)
startswithformat).0kvr r r 6 s
z$Language.__repr__..)r r __name__r __dict__items)r r r r __repr__4 s
zLanguage.__repr__)NNTNNN)r
__module__ __qualname____doc__r r"
__classcell__r r r r r s
r ArabicarF)z
ISO-8859-6z WINDOWS-1256ZCP720ZCP864ub ءآأؤإئابةتثجحخدذرزسشصضطظعغػؼؽؾؿـفقكلمنهوىيًٌٍَُِّu الصفحة_الرئيسية)r r
r r r r
Belarusianbe)
ISO-8859-5 WINDOWS-1251IBM866 MacCyrillicu АБВГДЕЁЖЗІЙКЛМНОПРСТУЎФХЦЧШЫЬЭЮЯабвгдеёжзійклмнопрстуўфхцчшыьэюяʼu! Галоўная_старонка Bulgarianbg)r+ r, IBM855ux АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЬЮЯабвгдежзийклмнопрстуфхцчшщъьюяu Начална_страницаCzechczTz
ISO-8859-2z WINDOWS-1250u< áčďéěíňóřšťúůýžÁČĎÉĚÍŇÓŘŠŤÚŮÝŽu Hlavní_stranaDanishda)
ISO-8859-1z ISO-8859-15 WINDOWS-1252u æøåÆØÅZForsideGermander6 r7 u äöüßÄÖÜzWikipedia:HauptseiteGreekelz
ISO-8859-7z WINDOWS-1253u αβγδεζηθικλμνξοπρσςτυφχψωάέήίόύώΑΒΓΔΕΖΗΘΙΚΛΜΝΞΟΠΡΣΣΤΥΦΧΨΩΆΈΉΊΌΎΏu Πύλη:ΚύριαEnglishenZ Main_Page)r r
r r r Esperantoeo
ISO-8859-3uD abcĉdefgĝhĥijĵklmnoprsŝtuŭvzABCĈDEFGĜHĤIJĴKLMNOPRSŜTUŬVZu Vikipedio:ĈefpaĝoSpanishesu ñáéíóúüÑÁÉÍÓÚÜzWikipedia:PortadaEstonianet)
ISO-8859-4 ISO-8859-13 WINDOWS-1257u6 ABDEGHIJKLMNOPRSTUVÕÄÖÜabdeghijklmnoprstuvõäöüZEsilehtFinnishfiu ÅÄÖŠŽåäöšžzWikipedia:EtusivuFrenchfru, œàâçèéîïùûêŒÀÂÇÈÉÎÏÙÛÊu Wikipédia:Accueil_principalu Bœuf (animal)Hebrewhez
ISO-8859-8z WINDOWS-1255u< אבגדהוזחטיךכלםמןנסעףפץצקרשתװױײu עמוד_ראשיCroatianhru@ abcčćdđefghijklmnoprsštuvzžABCČĆDĐEFGHIJKLMNOPRSŠTUVZŽZGlavna_stranica HungarianhuuP abcdefghijklmnoprstuvzáéíóöőúüűABCDEFGHIJKLMNOPRSTUVZÁÉÍÓÖŐÚÜŰu KezdőlapItalianitu ÀÈÉÌÒÓÙàèéìòóùZPagina_principale
Lithuanianlt)rF rG rE uR AĄBCČDEĘĖFGHIĮYJKLMNOPRSŠTUŲŪVZŽaąbcčdeęėfghiįyjklmnoprsštuųūvzžZPagrindinis_puslapisLatvianlvuX AĀBCČDEĒFGĢHIĪJKĶLĻMNŅOPRSŠTUŪVZŽaābcčdeēfgģhiījkķlļmnņoprsštuūvzžu
Sākumlapa
Macedonianmk)r+ r, r. r1 u| АБВГДЃЕЖЗЅИЈКЛЉМНЊОПРСТЌУФХЦЧЏШабвгдѓежзѕијклљмнњопрстќуфхцчџшu Главна_страницаDutchnlZ HoofdpaginaPolishpluR AĄBCĆDEĘFGHIJKLŁMNŃOÓPRSŚTUWYZŹŻaąbcćdeęfghijklłmnńoóprsśtuwyzźżu Wikipedia:Strona_główna
Portugueseptu0 ÁÂÃÀÇÉÊÍÓÔÕÚáâãàçéêíóôõúu Wikipédia:Página_principalRomanianrou ăâîșțĂÂÎȘȚu Pagina_principalăRussianru)r+ r, zKOI8-Rr. r- r1 u абвгдеёжзийклмнопрстуфхцчшщъыьэюяАБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯu# Заглавная_страницаSlovakskuD áäčďéíĺľňóôŕšťúýžÁÄČĎÉÍĹĽŇÓÔŔŠŤÚÝŽu Hlavná_stránkaSloveneslu8 abcčdefghijklmnoprsštuvzžABCČDEFGHIJKLMNOPRSŠTUVZŽZ Glavna_stranSerbiansrux АБВГДЂЕЖЗИЈКЛЉМНЊОПРСТЋУФХЦЧЏШабвгдђежзијклљмнњопрстћуфхцчџшu Главна_страна)r r
r r r Thaith)z ISO-8859-11zTIS-620ZCP874u กขฃคฅฆงจฉชซฌญฎฏฐฑฒณดตถทธนบปผฝพฟภมยรฤลฦวศษสหฬอฮฯะัาำิีึืฺุู฿เแโใไๅๆ็่้๊๋์ํ๎๏๐๑๒๓๔๕๖๗๘๙๚๛u หน้าหลักTurkishtr)r@ z
ISO-8859-9z WINDOWS-1254uR abcçdefgğhıijklmnoöprsştuüvyzâîûABCÇDEFGĞHIİJKLMNOÖPRSŞTUÜVYZÂÎÛZ Ana_Sayfa
Vietnameseviz WINDOWS-1258uH aăâbcdđeêghiklmnoôơpqrstuưvxyAĂÂBCDĐEÊGHIKLMNOÔƠPQRSTUƯVXYu Chữ_Quốc_ngữ) r' r) r/ r2 r4 r8 r: r< r> rA rC rH rJ rL rN rP rR rT rV rX rZ r\ r^ r` rb rd rf rh rj rl rn N) r% Z
__future__r r stringr objectr Z LANGUAGESr r r r s ,