= , d Z ddlZddlZddlZddlZddlZddlmZ ddlmZ ddlm Z ddlm
Z
ddlm Z ddlm Z dd l
mZ dd
l
mZ dd lmZ ej e Z eej ej g Z eej ej ej ej g Ze
e ee eef f Z e e
e e
e e f Z G d d
Z dej de!fdZ"dej de!fdZ#dej de!fdZ$dede defdZ%de defdZ&de de fdZ'dS )zFModule containing our file processor that tokenizes a file for checks. N)Any)Dict) Generator)List)Optional)Tuple)defaults)utils) LoadedPluginc | e Zd ZdZdZ d(dedej dee e ddfdZ
e de e j
fd Zej d
eded fd Zd)d
Zd)dZd)dZdeddfdZdeddfdZd)dZde fdZ de j fdZ de!eeef fdZ"de j
deeddf fdZ#de$ee%f de$ee&f de$ee&f fd Z'dee j
ddf fd Z(d ed ede$eef fd Z)d
edee fd!Z*defd"Z+de e fd#Z,de e fd$Z-de e fd%Z.de%fd&Z/d)d'Z0dS )*
FileProcessora> Processes a file and holdes state.
This processes a file by generating tokens, logical and physical lines,
and AST trees. This also provides a way of passing state about the file
to checks expecting that state. Any public attribute on this object can
be requested by a plugin. The known public attributes are:
- :attr:`blank_before`
- :attr:`blank_lines`
- :attr:`checker_state`
- :attr:`indent_char`
- :attr:`indent_level`
- :attr:`line_number`
- :attr:`logical_line`
- :attr:`max_line_length`
- :attr:`max_doc_length`
- :attr:`multiline`
- :attr:`noqa`
- :attr:`previous_indent_level`
- :attr:`previous_logical`
- :attr:`previous_unindented_logical_line`
- :attr:`tokens`
- :attr:`file_tokens`
- :attr:`total_lines`
- :attr:`verbose`
FNfilenameoptionslinesreturnc || _ || _ ||n| | _ | d| _ d| _ i | _ i | _ |j | _ d| _
d| _ |j | _ d| _
d| _ |j | _ |j | _ d| _ d| _ d| _ d| _ g | _ t- | j | _ |j | _ ddi| _ d| _ d| _ dS )z]Initialice our file processor.
:param filename: Name of the file to process
Nr F
logical lines) r r
read_linesr
strip_utf_bom blank_before blank_lines_checker_states
checker_state hang_closing indent_char indent_level indent_size line_number logical_linemax_line_lengthmax_doc_length multilineprevious_indent_levelprevious_logical previous_unindented_logical_linetokenslen total_linesverbose
statistics _file_tokens_noqa_line_mapping)selfr r r s b/builddir/build/BUILD/cloudlinux-venv-1.0.10/venv/lib/python3.11/site-packages/flake8/processor.py__init__zFileProcessor.__init__< s
# /UUT__5F5F
:< -/ # 0*. ". & 6 % 4 %&" " 02-02 tz?? *A.@D<@ c | j =t | j t t j fd | _ | j S )z-Return the complete set of tokens for a file.Nc " t S N)next) line_iters r/ z+FileProcessor.file_tokens..~ s
i r1 )r, iterr listtokenizegenerate_tokens)r. r6 s @r/ file_tokenszFileProcessor.file_tokensx sR $ TZ((I $()@)@)@)@AA! !D r1 r )NNNc # < K || _ d| _ dV d| _ dS )z2Context-manager to toggle the multiline attribute.TNF)r r# )r. r s r/ inside_multilinez FileProcessor.inside_multiline s+
'
r1 c d| _ dS )z)Reset the blank_before attribute to zero.r N)r r. s r/ reset_blank_beforez FileProcessor.reset_blank_before s r1 c | j d= dS )z-Delete the first token in the list of tokens.r N)r' r@ s r/ delete_first_tokenz FileProcessor.delete_first_token s KNNNr1 c & | xj dz
c_ dS )z&Note that we visited a new blank line. N)r r@ s r/ visited_new_blank_linez$FileProcessor.visited_new_blank_line s A r1 mappingc |d d \ }}| j |dz
}t |d| | _ | j | j k r| j | _ dS dS )z:Update the indent level based on the logical line mapping.r rE N)r
expand_indentr r r )r. rG start_row start_col
start_lines r/ update_statezFileProcessor.update_state sg !(A IZ A
.
)*ZiZ*@ A A t / / / $ 0D 0 /r1 pluginc f d|j v r'| j |j i | _ dS dS )z2Update the checker_state attribute for the plugin.r N)
parametersr
setdefault
entry_namer )r. rN s r/ update_checker_state_forz&FileProcessor.update_checker_state_for sA f / / /!%!5!@!@!2" "D 0 /r1 c | j r+| j | _ | j | _ | j s | j | _ d| _ g | _ dS )zoRecord the previous logical line.
This also resets the tokens list and the blank_lines count.
r N)r r r$ r% r& r r' r@ s r/ next_logical_linez FileProcessor.next_logical_line sP
J)-):D &$($5D !$
J8<8I5 r1 c g }g }g }d}dx}}| j D ]\ }}} }
} |t v r|sd| fg}|t j k r| | ?|t j k rt
| }|rO| \ } }
|| k r1|dz
}|dz
}| j | | }|dk s|dvr |dvrd| }n||
k r
| ||
|z }| | |t | z
}| ||
f |
\ }}|||fS )z4Build the mapping, comments, and logical line lists.r NrE ,z{[(}]) ) r' SKIP_TOKENSr: COMMENTappendSTRING
mutate_stringr r( )r. logicalcommentsrG length previous_rowprevious_column
token_typetextstartendlinerJ start_column row_index column_index
previous_texts r/ build_logical_line_tokensz'FileProcessor.build_logical_line_tokens sy #%)-- 26+ 2 2 .JeS$[((
' u:,X ---%%%X_,,$T**
E,1)L 9,, ,q 0I#2Q#6L$(Jy$9,$GM$++%U22t57H7H )4zz$ 44 <=DD NN4 c$ii F NNFC= ) ) ).1 +\??'))r1 c Z t j d | j S )z5Build an abstract syntax tree from the list of lines.r )astparsejoinr r@ s r/ build_astzFileProcessor.build_ast s y,,---r1 c | \ }}}d | }d | | _ | j dxx dz
cc< || j |fS )z2Build a logical line from the current tokens list.r r rE )rm rq r r+ )r. r` r_ mapping_listjoined_commentss r/ build_logical_linez FileProcessor.build_logical_line sm *.*H*H*J*J''< ''(++ GGG , , (((A-((( 1<??r1 tokenc # K t |j d |j d D ]$}| j |dz
V | xj dz
c_ %dS )z|Split a physical line's line based on new-lines.
This also auto-increments the line number for the caller.
r rE N)rangerf rg r r )r. rw line_nos r/
split_linezFileProcessor.split_line se U[^UYq\:: " "G*Wq[) ) ) ) ! " "r1 rP argumentsc i }| D ]L\ }}||v r
t | | ||< # t $ r! |r t d| Y Iw xY w|S )z8Generate the keyword arguments for a list of parameters.zPPlugin requested optional parameter "%s" but this is not an available parameter.)itemsgetattrAttributeErrorLOGwarning)r. rP r| retparamrequireds r/ keyword_arguments_forz#FileProcessor.keyword_arguments_for s ) / / 1 1
OE8 !!
$T5 1 1E
!
KKB
s 6(A! A!c # K t j | j D ]:}|d d | j k r dS | j | |V ;dS )z'Tokenize the file and yield the tokens. r N)r: r; next_liner) r' r\ )r. rw s r/ r; z FileProcessor.generate_tokens sj -dn== EQx{T --- K u % % %KKKK r1 min_linemax_linec t ||dz }d | j |dz
| }t || S )NrE r )ry rq r dictfromkeys)r. r r
line_rangejoineds r/ _noqa_line_rangez FileProcessor._noqa_line_range sJ 8X\22
HqL8$;<==}}Z000r1 c ~ | j | j }i }t | j dz }d}|D ]\ }}\ }}\ } }}|t j k r n}t
|| }t || }|t j t j fv rB|
| || t | j dz }d}|dk r)|
| || || _ n## t j t f$ r
i | _ Y nw xY w| j | S )z7Retrieve the line which will be used to determine noqa.Nr )r- r< r( r r: ENDMARKERminmaxNLNEWLINEupdater
TokenError SyntaxErrorget)
r. r r< r r r tp_s_linee_lines
r/
noqa_line_forzFileProcessor.noqa_line_for s_ " *
. " . tz??Q. :E & &6B;FA QX /// "8V 4 4H "8V 4 4Hhk8+; <<<
4#8#88#L#LMMM#&tz??Q#6#%
r>>JJt 4 4Xx H HIII*-''9 '5
-
-
- +-'''
-B &**;777s D D"!D"c | j | j k rdS | j | j }| xj dz
c_ | j #|dd t j v r
|d | _ |S )z Get the next line from the list.r rE Nr )r r) r r r
WHITESPACE)r. rh s r/ r zFileProcessor.next_line= sj t / / /2z$*+ A #RaRH4G(G(G #AwD r1 c | j | j dk r(| j j pd| _ | }n| }|S )z%Read the lines for this file checker.N-stdin)r r stdin_display_nameread_lines_from_stdinread_lines_from_filename)r. r s r/ r zFileProcessor.read_linesG sP = DMS$8$8 L ; FwDM..00EE1133E r1 c J t j | j 5 }| cddd S # 1 swxY w Y dS # t t
f$ rH t | j d 5 }| cddd cY S # 1 swxY w Y Y dS w xY w)zRead the lines for a file.Nzlatin-1)encoding)r: openr readlinesr UnicodeError)r. fds r/ r z&FileProcessor.read_lines_from_filenameP s7 & t}--
&||~~
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&\* & & & dmi888
&B||~~
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
&
& &sU A < A A A A A 'B"0B
B"B B"B B"!B"c ( t j S )z Read the lines from standard in.)r
stdin_get_linesr@ s r/ r z#FileProcessor.read_lines_from_stdin[ s $&&&r1 c | j j s t d | j D rdS t d | j D r t d dS dS )zCheck if ``flake8: noqa`` is in the file to be ignored.
:returns:
True if a line matches :attr:`defaults.NOQA_FILE`,
otherwise False
c 3 T K | ]#}t j | V $d S r4 )r NOQA_FILEmatch.0rh s r/ z3FileProcessor.should_ignore_file..f sD 1
1
/3H $ $T * *1
1
1
1
1
1
r1 Tc 3 T K | ]#}t j | V $d S r4 )r r searchr s r/ r z3FileProcessor.should_ignore_file..j s3 HHT#**400HHHHHHr1 z[Detected `flake8: noqa` on line with code. To ignore an error on a line use `noqa` instead.F)r disable_noqaanyr r r r@ s r/ should_ignore_filez FileProcessor.should_ignore_file_ s |( S 1
1
7;z1
1
1
.
.
4
HHTZHHH
H
H KK6
55r1 c | j sdS t | j d d }|dvrdS |dk r | j d dd | j d< dS | j d dd dk r | j d dd | j d< dS dS )z-Strip the UTF bom from the lines of the file.Nr ) r rE u )r ord)r.
first_bytes r/ r zFileProcessor.strip_utf_boms s z FA q)**
^ + + F JqM!"" -DJqMMM
Z]2A2
.
0
0 JqM!"" -DJqMMM 1
0r1 r4 )r N)1__name__
__module__ __qualname____doc__noqastrargparse Namespacer r r0 propertyr: TokenInfor<
contextlibcontextmanagerintr r> rA rC rF _LogicalMappingrM r rS rU _Logicalrm ro ASTrr r rv r{ r boolr r r; r r r r r r r r r1 r/ r
r
s 8 D &* :A :A:A #:A S " :A
:A :A :A :Ax !T("4 5 ! ! ! X! # $ 1O 1 1 1 1 1| !*8 !* !* !* !*F.37 . . . .@E#sO*C$D @ @ @ @ " ' " 3d? # " " " "dO S>
c3h .8+=tT+I!J 1 1 1S#X 1 1 1 1
&8 &8# &8 &8 &8 &8P3 DI &$s) & & & &'tCy ' ' ' 'D (. . . . . .r1 r
rw r c | d t v p1| d | d d d dk S )z+Check if the token is an end-of-line token.r r rE Nz\
)r lstriprw s r/ is_eol_tokenr s< 8w L%(58A;=="9"@"@"B"Bf"LLr1 c B | d t j k o d| d v S )z$Check if this is a multiline string.r
rE )r: r] r s r/ is_multiline_stringr s! 8x & ;458+;;r1 c | d t v S )z0Check if the token type is a newline token type.r )r r s r/ token_is_newliner s 8w r1 current_parentheses_count
token_textc * |dv r| dz S |dv r| dz
S | S )z Count the number of parentheses.z([{rE rX r )r r s r/ count_parenthesesr s4 U(1,, u (1,, $$r1 rh c F t | d S )zReturn the amount of indentation.
Tabs are expanded to the next multiple of 8.
>>> expand_indent(' ')
4
>>> expand_indent('\t')
8
>>> expand_indent(' \t')
8
>>> expand_indent(' \t')
16
)r(
expandtabs)rh s r/ rI rI s tq!! " ""r1 re c | | d dz }t | dz
}| dd dv r
|dz
}|dz }| d| d||z
z z | |d z S )zReplace contents with 'xxx' to prevent syntax matching.
>>> mutate_string('"abc"')
'"xxx"'
>>> mutate_string("'''abc'''")
"'''xxx'''"
>>> mutate_string("r'abc'")
"r'xxx'"
r rE N)z"""z'''r x)indexr( )re rf rg s r/ r^ r^ s{
JJtBx 1 $E
d))a-C BCCyN""
q <#u- -STT
::r1 )(r r ro r loggingr: typingr r r r r r flake8r r
flake8.plugins.finderr getLoggerr r frozensetr r INDENTDEDENTrZ r r r r r
r r r r r r rI r^ r r1 r/ r s L L
. . . . . .g!!
)X[("23
4
4i
[("HOX_E uS%S/ 123 cDI6 7d. d. d. d. d. d. d. d.N M* Mt M M M M