Ra!:{ d dl Z d dlmZ d dlmZ d dlmZ d dlmZ d dl m
Z
d dl m Z m
Z
mZmZmZmZmZ dd lmZmZ dd
lmZ dd lmZmZmZ G d d
Z G d d Z ee e f Z ee Z! G d d Z"dS ) N)Counter)aliases)sha256)dumps)sub)AnyDictIteratorListOptionalTupleUnion )NOT_PRINTABLE_PATTERNTOO_BIG_SEQUENCE)
mess_ratio) iana_nameis_multi_byte_encoding
unicode_rangec e Zd Z d*dededededddee f d Zd
e d efd Z
d
e d efd
Z e d efd Z
e d efd Ze d efd Ze d efd Zd efdZd efdZd+dZe d efd Ze d ee fd Ze d efd Ze d efd Ze d ee fd Ze d efd Ze d efd Z e d efd Z e d efd Z e d efd Z e d efd Z e d ed fd Z!e d efd! Z"e d ee fd" Z#e d ee fd# Z$d,d$Z%d,d%Z&d-d'ed efd(Z'e d efd) Z(dS ). CharsetMatchNpayloadguessed_encodingmean_mess_ratiohas_sig_or_bom languagesCoherenceMatchesdecoded_payloadc || _ || _ || _ || _ || _ d | _ g | _ d| _ d | _ d | _ || _
d S )N ) _payload _encoding_mean_mess_ratio
_languages_has_sig_or_bom_unicode_ranges_leaves_mean_coherence_ratio_output_payload_output_encoding_string)selfr r r r r r s y/builddir/build/BUILD/imunify360-venv-2.6.2/opt/imunify360/venv/lib/python3.11/site-packages/charset_normalizer/models.py__init__zCharsetMatch.__init__ s[
) / # - # %(" # $ & otherreturnc t |t sGt d t |j t | j | j |j k o| j |j k S )Nz&__eq__ cannot be invoked on {} and {}.)
isinstancer TypeErrorformatstr __class__encoding fingerprintr, r0 s r- __eq__zCharsetMatch.__eq__( ss % .. 8??((#dn*=*=
}.X43CuGX3XXr/ c N t |t st t | j |j z
}t | j |j z
}|dk r<|dk r6|dk r | j |j k r| j |j k S | j |j k S | j |j k S )zQ
Implemented to make sorted available upon CharsetMatches items.
g{Gz?g{Gz?r )r3 r
ValueErrorabschaos coherencemulti_byte_usage)r, r0 chaos_differencecoherence_differences r- __lt__zCharsetMatch.__lt__1 s % .. tzEK 788 "4>EO#C D D d " "';d'B'B 3&&4>U_+L+L,u/EEE>EO3 3zEK''r/ c j dt t | t | j z z
S )N ?)lenr6 rawr, s r- rA z CharsetMatch.multi_byte_usageD s& ST^^c$(mm333r/ c p t j dt t t | d S )z
Check once again chaos in decoded text, except this time, with full content.
Use with caution, this can be very slow.
Notice: Will be removed in 3.0
z=chaos_secondary_pass is deprecated and will be removed in 3.0rF )warningswarnDeprecationWarningr r6 rI s r- chaos_secondary_passz!CharsetMatch.chaos_secondary_passH s6
K
#d))S)))r/ c : t j dt dS )zy
Coherence ratio on the first non-latin language detected if ANY.
Notice: Will be removed in 3.0
z)r5 r8 r9 rI s r- __repr__zCharsetMatch.__repr__u s .55dmTEUVVVr/ c t |t r|| k r't d |j d |_ | j | d S )Nz;Unable to add instance <{}> as a submatch of a CharsetMatch)r3 r r= r5 r7 r+ r' appendr: s r- add_submatchzCharsetMatch.add_submatchx sk % .. %4-- MTTO
E"""""r/ c | j S N)r" rI s r- r8 zCharsetMatch.encoding s
~ r/ c g }t j D ]F\ }}| j |k r| | &| j |k r| | G|S )z
Encoding name are known by many name, using this could help when searching for IBM855 when it's listed as CP855.
)r itemsr8 r] )r,
also_known_asups r- encoding_aliasesz CharsetMatch.encoding_aliases sn
MOO ( (DAq}!! $$Q'''' !## $$Q''' r/ c | j S r` r% rI s r- bomzCharsetMatch.bom ##r/ c | j S r` rh rI s r- byte_order_markz CharsetMatch.byte_order_mark rj r/ c $ d | j D S )z
Return the complete list of possible languages found in decoded sequence.
Usually not really useful. Returned list may be empty even if 'language' property return something != 'Unknown'.
c g | ]
}|d S )r ).0es r-
z*CharsetMatch.languages.. s ...!...r/ r$ rI s r- r zCharsetMatch.languages s /.do....r/ c | j shd| j v rdS ddlm}m} t | j r || j n || j }t | dk sd|v rdS |d S | j d d S )z
Most probable language found in decoded sequence. If none were detected or inferred, the property will return
"Unknown".
asciiEnglishr )encoding_languagesmb_encoding_languagesz Latin BasedUnknown)r$ could_be_from_charsetcharset_normalizer.cdrw rx r r8 rG )r, rw rx r s r- languagezCharsetMatch.language s $444 y
X W W W W W W W *$-887%%dm444''
66
9~~""my&@&@ y Q< q!!$$r/ c | j S r` )r# rI s r- r? zCharsetMatch.chaos s $$r/ c : | j sdS | j d d S )Nr r r rs rI s r- r@ zCharsetMatch.coherence s# 3q!!$$r/ c 4 t | j dz d S Nd )ndigits)roundr? rI s r-
percent_chaoszCharsetMatch.percent_chaos s TZ#%q1111r/ c 4 t | j dz d S r )r r@ rI s r- percent_coherencez CharsetMatch.percent_coherence s T^c)15555r/ c | j S )z+
Original untouched bytes.
)r! rI s r- rH zCharsetMatch.raw s
} r/ c | j S r` )r' rI s r- submatchzCharsetMatch.submatch s
|r/ c 2 t | j dk S Nr )rG r' rI s r- has_submatchzCharsetMatch.has_submatch s 4< 1$$r/ c | j | j S d t | D }t t d |D | _ | j S )Nc , g | ]}t | S ro )r )rp chars r- rr z*CharsetMatch.alphabets.. s-
$(M$
r/ c h | ]}||S ro ro )rp rs r- z)CharsetMatch.alphabets.. s +L+L+L!!+LA+L+L+Lr/ )r& r6 sortedlist)r, detected_rangess r- alphabetszCharsetMatch.alphabets sj +' '
,/II
&d+L+L+L+L+L&M&M N N ##r/ c 6 | j gd | j D z S )z
The complete list of encoding that output the exact SAME str result and therefore could be the originating
encoding.
This list does include the encoding available in property 'encoding'.
c g | ] }|j
S ro )r8 )rp ms r- rr z6CharsetMatch.could_be_from_charset.. s "D"D"D!1:"D"D"Dr/ )r" r' rI s r- rz z"CharsetMatch.could_be_from_charset s% "D"Dt|"D"D"DDDr/ c | S z>
Kept for BC reasons. Will be removed in 3.0.
ro rI s r- firstzCharsetMatch.first r/ c | S r ro rI s r- bestzCharsetMatch.best r r/ utf_8r8 c | j | j |k r/|| _ t | |d | _ | j S )z
Method to get re-encoded bytes payload using given target encoding. Default to UTF-8.
Any errors will be simply ignored by the encoder NOT replaced.
Nreplace)r* r6 encoder) )r, r8 s r- outputzCharsetMatch.output sI
(D,AX,M,M$,D !#&t99#3#3Hi#H#HD ##r/ c h t | S )zw
Retrieve the unique SHA256 computed using the transformed (re-encoded) payload. Not the original one.
)r r hexdigestrI s r- r9 zCharsetMatch.fingerprint
s&
dkkmm$$..000r/ r` )r0 r r1 N)r1 r )r ))__name__
__module__ __qualname__bytesr6 floatboolr r. objectr; rD propertyrA rN rP r rV rY r[ r^ r8 r rf ri rl r r| r? r@ r r rH r r r rz r r r r9 ro r/ r- r r s~ *.' '' ' '
' &
' "# ' ' ' '2YF Yt Y Y Y Y(F (t ( ( ( (& 4% 4 4 4 X4
*e
*
*
* X
* U X 67 6 6 6 X 6 W# W W W W # # # # # X
$s)
X
$T $ $ $ X$ $ $ $ $ X$ /49 / / / X/ %# % % % X%6 %u % % % X% %5 % % % X%
2u 2 2 2 X2 65 6 6 6 X6 U X $~. X %d % % % X% $49 $ $ $ X $ EtCy E E E XE $ $s $ $ $ $ $ 1S 1 1 1 X1 1 1r/ r c e Zd ZdZddee fdZdee fdZde e
e f defdZ de
fd Z
defd
Zdeddfd Zded fd
Zded fdZdS )CharsetMatchesz
Container with every CharsetMatch items ordered by default from most probable to the less one.
Act like a list(iterable) but does not implements all related methods.
Nresultsc 6 |rt | ng | _ d S r` )r _results)r, r s r- r. zCharsetMatches.__init__ s +2:w
r/ r1 c # $ K | j E d {V d S r` r rI s r- __iter__zCharsetMatches.__iter__ s& = r/ itemc t |t r
| j | S t |t r't |d }| j D ]}||j v r|c S t )z
Retrieve a single item either by its position or encoding name (alias may be used here).
Raise KeyError upon invalid index or encoding not present in results.
F)r3 intr r6 r rz KeyError)r, r results r- __getitem__zCharsetMatches.__getitem__! sv
dC '=& & dC " T5))D -
"
"6777!MMM 8r/ c * t | j S r` rG r rI s r- __len__zCharsetMatches.__len__/ s 4=!!!r/ c 2 t | j dk S r r rI s r- __bool__zCharsetMatches.__bool__2 s 4=!!A%%r/ c t |t s4t d t |j t
|j t k rB| j D ]:}|j
|j
k r(|j |j k r| | dS ;| j
| t | j | _ dS )z~
Insert a single match. Will be inserted accordingly to preserve sort.
Can be inserted as a submatch.
z-Cannot append instance '{}' to CharsetMatchesN)r3 r r= r5 r6 r7 rG rH r r r9 r? r^ r] r )r, r matchs r- r] zCharsetMatches.append5 s
$ -- ?FF''
tx== , , ,
$(888U[DJ=V=V&&t,,,FF
T""" t}--
r/ r c . | j sdS | j d S )zQ
Simply return the first match. Strict equivalent to matches[0].
Nr r rI s r- r zCharsetMatches.bestI s } 4}Q r/ c * | S )zP
Redundant method, call the method best(). Kept for BC reasons.
)r rI s r- r zCharsetMatches.firstQ s yy{{r/ r` )r r r __doc__r r r. r
r r r r6 r r r r r] r r r ro r/ r- r r s%
; ;\ 2 ; ; ; ;!(<0 ! ! ! ! c3h L " " " " "&$ & & & &.< .D . . . .( h~. x/ r/ r c e Zd Zdedee dee dee dedee deded ed
ee d efd Ze d
e
ee f fd Z d
efdZ
dS )CliDetectionResultpathr8 rf alternative_encodingsr| r r r? r@ unicode_path is_preferredc || _ |
| _ || _ || _ || _ || _ || _ || _ || _ | | _ | | _
d S r` ) r r r8 rf r r| r r r? r@ r ) r, r r8 rf r r| r r r? r@ r r s r- r. zCliDetectionResult.__init__] s\ (
0 %:"
" ,
" (r/ r1 c | j | j | j | j | j | j | j | j | j | j | j
d S )N r r8 rf r r| r r r? r@ r r r rI s r- __dict__zCliDetectionResult.__dict__w sO I
$ 5%)%?
" 1Z - -
r/ c 0 t | j dd S )NT ) ensure_asciiindent)r r rI s r- to_jsonzCliDetectionResult.to_json s T]a@@@@r/ N)r r r r6 r r r r r. r r r r r ro r/ r- r r \ s )) 3-) s) )
$Cy )
) 9) ) ) ) sm) ) ) ) )4
$sCx.
X
A A A A A A Ar/ r )#rK collectionsr encodings.aliasesr hashlibr jsonr rer typingr r r
r r r
r constantr r mdr utilsr r r r r r6 r CoherenceMatchr r ro r/ r- r s % % % % % % D D D D D D D D D D D D D D D D D D = = = = = = = = C C C C C C C C C CD1 D1 D1 D1 D1 D1 D1 D1N@ @ @ @ @ @ @ @F sEz"' ,A ,A ,A ,A ,A ,A ,A ,A ,A ,Ar/