Pډb d Z ddlmZ ddlZddlZddlZddlZddlmZ ddl m
Z
ddl m Z ddl
mZ ddlmZmZ dd lmZ ddlZddlZdd
lmZ ed ee Zg d Z G d
de Z e ddddd e ddddd e dddd d e d d d d!d" e d#d$d%d&d' e d(d)d*d+d, gZ d- e D Z dVd3Z dWdXd:Z ej! ej" ej# ej$ ej% ej& ej' d;Z( ed< d=D Z) ej* d>ej+ ? Z,dYd@Z-dZdDZ.d[dFZ/d\dGZ0 e dHI d[dJ Z1d]dMZ2d^dOZ3 G dP dQej4 Z5d_dUZ6dS )`a2 Unicode and some other ASCII characters can be used to create programs that run
much different compared to what a human reader would expect from them.
PEP 672 lists some examples.
See: https://www.python.org/dev/peps/pep-0672/
The following checkers are intended to make users are aware of these issues.
) annotationsN) OrderedDict)Iterable) lru_cache)detect_encoding)
NamedTupleTypeVar)nodes)checkers_StrLike)
u u u u u u u u u u c V e Zd ZU dZded< ded< ded< ded< ded< d d Zd d
Zd S )
_BadCharz/Representation of an ASCII char considered bad.strname unescapedescapedcode help_textreturnc ( d| j d| j dS )z0Used for the detailed error message description.z Invalid unescaped character z, use "z
" instead.)r r selfs i/builddir/build/BUILD/cloudlinux-venv-1.0.10/venv/lib/python3.11/site-packages/pylint/checkers/unicode.py descriptionz_BadChar.descriptionC s.
-49
-
-L
-
-
-
c d| j S )z2Used to generate the human readable error message.zinvalid-character-)r r s r
human_codez_BadChar.human_codeJ s /DI///r N)r r )__name__
__module__ __qualname____doc____annotations__r r r r r r : sq 99
IIINNNLLL
IIINNN
0 0 0 0 0 0r r backspacez\bE2510zUMoves the cursor back, so the character after it will overwrite the character before.zcarriage-return
z\rE2511z]Moves the cursor to the start of line, subsequent characters overwrite the start of the line.subz\x1AE2512zcCtrl+Z "End of text" on Windows. Some programs (such as type) ignore the rest of the file after it.escz\x1BE2513zNCommonly initiates escape codes which allow arbitrary control of the terminal.nul z\0E2514z Mostly end of input for python.zzero-width-spaceu z\u200BE2515z9Invisible space character could hide real code execution.c i | ]
}|j | S r# )r ).0chars r
r6 s DDD$ DDDr linecodecr r intc t | t r%t | | |d }n| }| d }||k r| d }t | S )z?Get the length of a string like line as displayed in an editor.replace
r' )
isinstancebytes _remove_bomdecoderstriplen)r7 r8 decodedstrippeds r _line_lengthrE ss $ dE**11%CC~~d##H7??4(( x==r search_dictdict[_StrLike, _BadChar]new_linebyte_str_lengthdict[int, _BadChar]c p i }| D ]\ }}|| vr
|j dk r+| | rt | d|z z
}nd}d}| || } | dk r>| |k rt | |z }
|||
< | dz }| || } | dk >|S )zGet all occurrences of search dict keys within line.
Ignores Windows end of line and can handle bytes as well as string.
Also takes care of encodings for which the length of an encoded code point does not
default to 8 Bit.
r' Nr rF )itemsr endswithrB findr9 ) r7 rG rI rJ result
search_forr5
ignore_posstartposcols r _map_positions_to_resultrW s #%F ' - - / / / /
D T ! ! >T ! !dmmH&=&= ! TQ %88JJ Jii
E**Aggj #/00 "s !GE))J..C
Agg Mr )utf-8utf-16utf-32utf-16leutf-16beutf-32leutf-32bec # 4 K | ]}t | |fV d S N) UNICODE_BOMS)r4 r8 s r rb sB " " %% " " " " " "r )r] r^ rX r[ r\ z%utf[ -]?(8|16|32)[ -]?(le|be|)?(sig)?)flagsc \ t d| S )zDMake sure the codec name is always given as defined in the BOM dict.zutf-\1\2)UTF_NAME_REGEX_COMPILEDr) lowerr8 s r _normalize_codec_namerh s$ " & &{E : : @ @ B BBr encodedr> encodingc |t vr| S t | }| | r| t | d S | S )z$Remove the bom if given from a line.N)ra
startswithrB )ri rj boms r r? r? sL |##
x
C# #s3xxzz"" Nr stringc H t | | | S )z#Encode a string but remove the BOM.)r? encodern rj s r _encode_without_bomrr s v}}X.. 9 99r c b | d rdS | d rdS dS )z6Return how many byte are usually(!) a character point.rZ rY rM rF rl rg s r _byte_to_str_lengthrv s= !! q !! q 1r i )maxsizec " t | | S )z3A cached version of encode used for search pattern.)rr rq s r _cached_encode_searchry s vx 0 00r steamIterable[bytes]c # K | d s | d s
| E d{V dS d d | D }t d| }d} | || }|dk r |||t | z V n||d r ||d V dS |t | z }d)zHandle line ending for UTF16 and UTF32 correctly.
Currently, Python simply strips the required zeros after
after the
line ending. Leading to lines that can't be decoded properly
rY rZ Nr c 3 K | ]}|V d S r` r# )r4 r7 s r rb z,_fix_utf16_32_line_stream.. s" 22D4222222r r< r )rl joinry rP rB )rz r8 contentrI rT rU s r _fix_utf16_32_line_streamr s H % % (e.>.>x.H.H ( ((22E22222(u55
(,,x//Caxx ecCMM&9 9::::: 566? *!%&&/)))#h--'E
(r
first_linec t D ] \ }}| | r|c S t d )a Try to extract the codec (unicode only) by checking for the BOM.
For details about BOM see https://unicode.org/faq/utf_bom.html#BOM
Args:
first_line: the first line of a file
Returns:
a codec name
Raises:
ValueError: if no codec was found
z-No BOM found. Could not detect Unicode codec.)BOM_SORTED_TO_CODECrN rl
ValueError)r rm r8 s r extract_codec_from_bomr sX *//11
U % % LLL D
E
EEr c e Zd ZdZdZddddd eD Zed!d Zed!d
Z e
d"d Z ed#d Z d$dZ
d%dZd%d Zd&d Zd S )'UnicodeCheckera Check characters that could be used to hide bad code to humans.
This includes:
- Bidirectional Unicode (see https://trojansource.codes/)
- Bad ASCII characters (see PEP672)
If a programmer requires to use such a character they should use the escaped
version, that is also much easier to read and does not depend on the editor used.
The Checker also includes a check that UTF-16 and UTF-32 are not used to encode
Python files.
At the time of writing Python supported only UTF-8. See
https://stackoverflow.com/questions/69897842/ and https://bugs.python.org/issue1503789
for background.
unicode_checker)z?UTF-16 and UTF-32 aren't backward compatible. Use UTF-8 insteadinvalid-unicode-codeczFor compatibility use UTF-8 instead of UTF-16/UTF-32. See also https://bugs.python.org/issue1503789 for a history of this issue. And https://softwareengineering.stackexchange.com/questions/102205/ for some possible problems when using UTF-16 for instance.)z_Contains control characters that can permit obfuscated code executed differently than displayedbidirectional-unicodea bidirectional unicode are typically not displayed characters required to display right-to-left (RTL) script (i.e. Chinese, Japanese, Arabic, Hebrew, ...) correctly. So can you trust this code? Are you sure it displayed correctly in all editors? If you did not write it or your language is not RTL, remove the special characters, as they could be used to trick you into executing code, that does something else than what it looks like.
More Information:
https://en.wikipedia.org/wiki/Bidirectional_text
https://trojansource.codes/)z2PEP8 recommends UTF-8 as encoding for Python filesbad-file-encodingzsPEP8 recommends UTF-8 default encoding for Python files. See https://peps.python.org/pep-0008/#source-file-encoding)E2501E2502C2503c v i | ]6}|j | | |j f7S r# )r r r r )r4 bad_chars r r6 zUnicodeChecker.n sU
M$$&&##%%"
r r8 r r boolc V | d p| d S )NrY rZ ru rg s r _is_invalid_codecz UnicodeChecker._is_invalid_codecx s) ))GU-=-=h-G-GGr c , | d S )Nutfru rg s r _is_unicodezUnicodeChecker._is_unicode| s &&&r r7 r> rK c | |d }t }t ||d S # t $ r |}i }t D ]K}t j t 5 ||t |j | < ddd n # 1 swxY w Y Lt ||t d| t | cY S w xY w)zFind all matches of BAD_CHARS within line.
Args:
line: the input
codec: that will be used to convert line/or search string into
Return:
A dictionary with the column offset and the BadASCIIChar
strict)errorsr< N)rJ )
r@ BAD_ASCII_SEARCH_DICTrW UnicodeDecodeError BAD_CHARS
contextlibsuppressry r rv )clsr7 r8 line_searchrG line_search_bytesearch_dict_byter5 s r _find_line_matchesz!UnicodeChecker._find_line_matches s2 ++eH+==K/K+KdKK K! $ 68 !
();<< %-dneDD
, %dE22 3E : :
s3 .1 0C!B: CB
C
B
2CCstream
io.BytesIOtuple[str, int]c D t | j \ }}t | pd}nf# t $ rY}| d t | }d}n# t $ r}||d}~ww xY wY d}~nd}~ww xY wt | |fS )a Determine the codec from the given stream.
first tries https://www.python.org/dev/peps/pep-0263/
and if this fails also checks for BOMs of UTF-16 and UTF-32
to be future-proof.
Args:
stream: The byte stream to analyse
Returns: A tuple consisting of:
- normalized codec name
- the line in which the codec was found
Raises:
SyntaxError: if failing to detect codec
rF r N)r readlinerB SyntaxErrorseekr r rh )r r8 linescodec_definition_lineeves r _determine_codecz UnicodeChecker._determine_codec s $ +6?;;LE5 %(JJO! ! !
KKNNN
.v/@/@AA()%%
R
&%%%% %U++-BBBs8 (+
BB #A/.B /
B9A<<BB Br r9 Nonec |dk rEd}| | rd}| |||t j j dd dS dS )z Check validity of the codec.rX r r Nr7
end_lineno
confidence
col_offsetend_col_offset)r add_messagepylint
interfacesHIGH)r r8 r msgs r _check_codeczUnicodeChecker._check_codec sv G %C%%e,,
.- +0!,1 #
r linenoc | || }| D ]\\ }}| | ||t j j |dz |t |j z dz ]dS )z Look for chars considered bad.rF r N) r rN r r r r r rB r )r r7 r r8 matchesrV r5 s r _check_invalid_charsz#UnicodeChecker._check_invalid_chars s ))$66
IC !! !!,1 7 "S %8%8 81 <
r c | | sdS t D ]O}t || |v r;| d||dt || t
j j dS PdS )z2Look for Bidirectional Unicode, if we use unicode.Nr r )r7 r r r r )r BIDI_UNICODEry r rE r r r )r r7 r r8 dangerouss r _check_bidi_charsz UnicodeChecker._check_bidi_chars s && F% I$Y66$>>
+ % !#/e#<#< % 0 5 ! ' ? r node nodes.Modulec | 5 }| | \ }}| || | d t t || d D ]I\ }}|dk rt
|| }| ||| | ||| J ddd dS # 1 swxY w Y dS )z3Perform the actual check by checking module stream.r rF )rT N) r r r r enumerater r? r r )r r r r8
codec_liner r7 s r process_modulez UnicodeChecker.process_module s3
[[]]
?f $ 5 5f = = E: eZ 0 0 0 KKNNN !*)&%88! ! !
?
? Q;;&tU33D&&tVU;;;))$>>>>
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?s B-CCCN)r8 r r r )r7 r> r8 r r rK )r r r r )r8 r r r9 r r )r7 r> r r9 r8 r r r )r r r r )r r r r! r r msgs staticmethodr r classmethodr r r r r r r# r r r r - sA & D
*
G3 3 V
&
W3 Dj H H H \H ' ' ' \' ! ! ! [!F &C &C &C \&CP "
4? ? ? ? ? ?r r linterpylint.lint.PyLinterr c J | t | d S r` )register_checkerr )r s r registerr s$
N6 2 233333r )r7 r r8 r r r9 )rF )
r7 r rG rH rI r rJ r9 r rK )r8 r r r )ri r> rj r r r> )rn r rj r r r> )r8 r r r9 )rz r{ r8 r r r{ )r r> r r )r r r r )7r!
__future__r codecsr iore collectionsr collections.abcr functoolsr tokenizer typingr r astroidr
pylint.interfacesr pylint.lintr r r> r r r r r rE rW BOM_UTF8 BOM_UTF16 BOM_UTF32 BOM_UTF16_LE BOM_UTF16_BE BOM_UTF32_LE BOM_UTF32_BEra r compile
IGNORECASEre rh r? rr rv ry r r BaseRawFileCheckerr r r# r r r s
# " " " " "
# # # # # # $ $ $ $ $ $ $ $ $ $ $ $ & & & & & & & & 7:sE * *
*0 0 0 0 0z 0 0 0.
H
H
%
H
-
H
H
)
H C a9
t ED)DDD & # # # # #N _ #### "k " " K" " " %"*+2=
C C C C
: : : :
41 1 1 1
( ( ( (<F F F F*i? i? i? i? i?X0 i? i? i?X4 4 4 4 4 4r