~֨ d Z ddlmZ ddlmZmZ G d d Zi d edddg d d
d g d
ed
ddg dddg d edddg dddg d edddddgd d g d ed d dg d d!d"g d# ed#d$dg d d%d&g d' ed'd(dd)d*gd+d,g d- ed-d.dg d/d0g1 d2 ed2d3dd4gd5d6g d7 ed7d8dg d d9d:g d; ed;dd?g d@ ed@dAdg d dBdCg dD edDdEdg d dFdGdHg dI edIdJddKdLgdMdNg dO edOdPdddgdQdRg dS edSdTdddgdUdVg dW edWdXdg d dYdZg ed[d\dg d]d^d_g ed`dadg d]dbdcg edddedg dfdgdhg edidjdg d/dkg1 edldmdddgdndog edpdqdg d drdsg edtdudddgdvdwg edxdydg dzd{d|g ed}d~dddgddg edddddgddg edddg dfdg edddg dddg edddg dddg eddddgddg dZdS )z
Metadata about languages used by our model training code for our
SingleByteCharSetProbers. Could be used for other things in the future.
This code is based on the language metadata from the uchardet project.
)
ascii_letters)ListOptionalc e Zd ZdZ d
dee dee dedeee dee d eee d
df fd
Zd
efd Z xZ
S )Languagea Metadata about a language useful for training models
:ivar name: The human name for the language, in English.
:type name: str
:ivar iso_code: 2-letter ISO 639-1 if possible, 3-letter ISO code otherwise,
or use another catalog as a last resort.
:type iso_code: str
:ivar use_ascii: Whether or not ASCII letters should be included in trained
models.
:type use_ascii: bool
:ivar charsets: The charsets we want to support and create data for.
:type charsets: list of str
:ivar alphabet: The characters in the language's alphabet. If `use_ascii` is
`True`, you only need to add those not in the ASCII set.
:type alphabet: str
:ivar wiki_start_pages: The Wikipedia pages to start from if we're crawling
Wikipedia for training data.
:type wiki_start_pages: list of str
NTnameiso_code use_asciicharsetsalphabetwiki_start_pagesreturnc X t || _ || _ || _ || _ | j r|r |t z
}nt }n|st d |r/d t t | nd | _ || _ d S )Nz*Must supply alphabet if use_ascii is False )
super__init__r r r
r r
ValueErrorjoinsortedsetr r
)selfr r r
r r r
__class__s l/builddir/build/BUILD/cloudlinux-venv-1.0.10/venv/lib/python3.11/site-packages/chardet/metadata/languages.pyr zLanguage.__init__# s
"
> K
)M)( K IJJ J:BLs8}} 5 5666
0 c d d | j D }| j j d| dS )Nz, c 3 T K | ]#\ }}| d | d|V $dS )_=N)
startswith).0kvs r z$Language.__repr__..< sU
!Q!,,sBSBS
LL1LL
r ())r __dict__itemsr __name__)r param_strs r __repr__zLanguage.__repr__; s] II
%)]%8%8%:%:
.)88I8888r )NNTNNN) r(
__module__ __qualname____doc__r strboolr r r*
__classcell__)r s @r r r s , #"& (,"&041 1sm1 3-1 1
49% 1 3-
1 #49-1
1 1 1 1 1 109# 9 9 9 9 9 9 9 9r r ArabicarF)z
ISO-8859-6z WINDOWS-1256CP720CP864ub ءآأؤإئابةتثجحخدذرزسشصضطظعغػؼؽؾؿـفقكلمنهوىيًٌٍَُِّu الصفحة_الرئيسية)r r r
r r r
Belarusianbe)
ISO-8859-5 WINDOWS-1251IBM866 MacCyrillicu АБВГДЕЁЖЗІЙКЛМНОПРСТУЎФХЦЧШЫЬЭЮЯабвгдеёжзійклмнопрстуўфхцчшыьэюяʼu! Галоўная_старонка Bulgarianbg)r7 r8 IBM855ux АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЬЮЯабвгдежзийклмнопрстуфхцчшщъьюяu Начална_страницаCzechczTz
ISO-8859-2z WINDOWS-1250u< áčďéěíňóřšťúůýžÁČĎÉĚÍŇÓŘŠŤÚŮÝŽu Hlavní_stranaDanishda)
ISO-8859-1z ISO-8859-15 WINDOWS-1252MacRomanu æøåÆØÅForsideGermandeu äöüßẞÄÖÜzWikipedia:HauptseiteGreekelz
ISO-8859-7z WINDOWS-1253u αβγδεζηθικλμνξοπρσςτυφχψωάέήίόύώΑΒΓΔΕΖΗΘΙΚΛΜΝΞΟΠΡΣΣΤΥΦΧΨΩΆΈΉΊΌΎΏu Πύλη:ΚύριαEnglishen)rB rC rD Main_Page)r r r
r r
Esperantoeo
ISO-8859-3uD abcĉdefgĝhĥijĵklmnoprsŝtuŭvzABCĈDEFGĜHĤIJĴKLMNOPRSŜTUŬVZu Vikipedio:ĈefpaĝoSpanishesu ñáéíóúüÑÁÉÍÓÚÜzWikipedia:PortadaEstonianet)
ISO-8859-4 ISO-8859-13 WINDOWS-1257u6 ABDEGHIJKLMNOPRSTUVÕÄÖÜabdeghijklmnoprstuvõäöüEsilehtFinnishfiu ÅÄÖŠŽåäöšžzWikipedia:EtusivuFrenchfru, œàâçèéîïùûêŒÀÂÇÈÉÎÏÙÛÊu Wikipédia:Accueil_principalu Bœuf (animal)Hebrewhez
ISO-8859-8z WINDOWS-1255u< אבגדהוזחטיךכלםמןנסעףפץצקרשתװױײu עמוד_ראשיCroatianhru@ abcčćdđefghijklmnoprsštuvzžABCČĆDĐEFGHIJKLMNOPRSŠTUVZŽGlavna_stranica HungarianhuuP abcdefghijklmnoprstuvzáéíóöőúüűABCDEFGHIJKLMNOPRSTUVZÁÉÍÓÖŐÚÜŰu KezdőlapItalianitu ÀÈÉÌÒÓÙàèéìòóùPagina_principale
Lithuanianlt)rU rV rT uR AĄBCČDEĘĖFGHIĮYJKLMNOPRSŠTUŲŪVZŽaąbcčdeęėfghiįyjklmnoprsštuųūvzžPagrindinis_puslapisLatvianlvuX AĀBCČDEĒFGĢHIĪJKĶLĻMNŅOPRSŠTUŪVZŽaābcčdeēfgģhiījkķlļmnņoprsštuūvzžu
Sākumlapa
Macedonianmk)r7 r8 r: r= u| АБВГДЃЕЖЗЅИЈКЛЉМНЊОПРСТЌУФХЦЧЏШабвгдѓежзѕијклљмнњопрстќуфхцчџшu Главна_страницаDutchnl HoofdpaginaPolishpluR AĄBCĆDEĘFGHIJKLŁMNŃOÓPRSŚTUWYZŹŻaąbcćdeęfghijklłmnńoóprsśtuwyzźżu Wikipedia:Strona_główna
Portugueseptu0 ÁÂÃÀÇÉÊÍÓÔÕÚáâãàçéêíóôõúu Wikipédia:Página_principalRomanianrou ăâîșțĂÂÎȘȚu Pagina_principalăRussianru)r7 r8 zKOI8-Rr: r9 r= u абвгдеёжзийклмнопрстуфхцчшщъыьэюяАБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯu# Заглавная_страницаSlovakskuD áäčďéíĺľňóôŕšťúýžÁÄČĎÉÍĹĽŇÓÔŔŠŤÚÝŽu Hlavná_stránkaSloveneslu8 abcčdefghijklmnoprsštuvzžABCČDEFGHIJKLMNOPRSŠTUVZŽ Glavna_stranSerbiansrux АБВГДЂЕЖЗИЈКЛЉМНЊОПРСТЋУФХЦЧЏШабвгдђежзијклљмнњопрстћуфхцчџшu Главна_страна)r r r r r
Thaith)z ISO-8859-11zTIS-620CP874u กขฃคฅฆงจฉชซฌญฎฏฐฑฒณดตถทธนบปผฝพฟภมยรฤลฦวศษสหฬอฮฯะัาำิีึืฺุู฿เแโใไๅๆ็่้๊๋์ํ๎๏๐๑๒๓๔๕๖๗๘๙๚๛u หน้าหลักTurkishtr)rO z
ISO-8859-9z WINDOWS-1254uR abcçdefgğhıijklmnoöprsştuüvyzâîûABCÇDEFGĞHIİJKLMNOÖPRSŞTUÜVYZÂÎÛ Ana_Sayfa
Vietnameseviz WINDOWS-1258uH aăâbcdđeêghiklmnoôơpqrstuưvxyAĂÂBCDĐEÊGHIKLMNOÔƠPQRSTUƯVXYu Chữ_Quốc_ngữ)rf ri rk rm rp rr rt rv rx rz r} r r r N)r- stringr typingr r r LANGUAGES r r r s ! ! ! ! ! ! ! ! !
19 19 19 19 19 19 19 19h^
hh
BAAu9: ^
((
HHH V=>
^
*
999 L;<
+^
: XX
/O*+
;^
J
hh
JJJ #
K^
Z
hh
JJJ$01
[^
j XX
/ T/0
k^
z xx
;;;% {^
H
W/0 I^
Z xx
JJJ/-.
[^
j
>>> J# k^
~ xx
JJJ'-.
^
N
hh
JJJ?8:JK
O^
^
hh
/O-.
_^
n
/S+, o^
@
/c% A^
R xx
JJJ/-.
S^
b (
>>>e01 x
>>>k& (
HHH P9:
X
;;;' h
/e56 (
JJJC89
/'./
x
X?@ h
/W,-
x
/K() x
LHHH56
H
444 Y45
x
===e% (
" !![./- M^
^
^
r