K!Dn S ( d Z ddlZddlZddlZddlZddlZddlZddlZddl Z ddl
m Z ddl
m Z ddl
m
Z
ddl
mZ ddl
mZ ddlmZ dd lmZ dd
lmZ dd lmZ dd lmZ dd
lmZ ddlmZ ddlmZ e
ee e e e ee f Z ej e Z!ej" hZ# G d d Z$ G d d Z%d dZ&de deej' j( fdZ)de de de fdZ*de%dee e e e e f f fd Z+d e d ej, dee e f fd Z-dS )!z$Checker Manager and Checker classes. N)Any)Dict)List)Optional)Tuple)defaults)
exceptions) processor)utils) expand_paths)Checkers) LoadedPlugin)StyleGuideManagerc e Zd ZdZdededdfdZddZdefdZ d e
d
e defd Z dd e
ee
ddfd
Zdeeef fdZddZddZddZdd e
ee
ddfdZddZdS )ManageraR Manage the parallelism and checker instances for each plugin and file.
This class will be responsible for the following:
- Determining the parallelism of Flake8, e.g.:
* Do we use :mod:`multiprocessing` or is it unavailable?
* Do we automatically decide on the number of jobs to use or did the
user provide that?
- Falling back to a serial way of processing files if we run into an
OSError related to :mod:`multiprocessing`
- Organizing the results of each checker so we can group the output
together and make our output deterministic.
style_guidepluginsreturnNc || _ |j | _ || _ | | _ g | _ g | _ ddddd| _ t t j
| j j | j j | _ dS )z Initialize our Manager instance.r )files
logical linesphysical linestokensN)
r optionsr
_job_countjobs
_all_checkerscheckers
statisticstuple itertoolschainexcludeextend_exclude)selfr r s `/builddir/build/BUILD/cloudlinux-venv-1.0.10/venv/lib/python3.11/site-packages/flake8/checker.py__init__zManager.__init__@ s '"* OO%% 02+-
ODL 0$,2M N N
c | j D ]1}t j D ]"}| j |xx |j | z
cc< #2| j dxx t | j z
cc< d S )Nr )r r STATISTIC_NAMESr len)r% checker statistics r& _process_statisticszManager._process_statisticsV s } L LG % 5
L
L ***g.@.KK****
L C
$6$66 r( c t j dk r t d dS t j | j j r t d dS | j j r t d dS | j j }|j
r% t j S # t $ r Y dS w xY w|j
S )NforkzGThe multiprocessing module is not available. Ignoring --jobs arguments.r z]The --jobs option is not compatible with supplying input using - . Ignoring --jobs arguments.zcThe --diff option was specified with --jobs but they are not compatible. Ignoring --jobs arguments.)multiprocessingget_start_methodLOGwarningr is_using_stdinr filenamesdiffr is_auto cpu_countNotImplementedErrorn_jobs)r% r s r& r zManager._job_count\ s + - - 7 7 KK-
1 6 7 7 KK=
1 < KKF
1| <
&0222&
qq
{s *B= =
C
C filenameresultsc
j | j }d}|D ]&\ }}}}} || |||||| z
}'|S )Nr )coder< line_number
column_numbertext
physical_line)r handle_error)
r% r< r= r reported_results_count
error_coder@ columnrB rC s
r& _handle_resultszManager._handle_results si & !" FM BZfdM "k&>&> ! ' $ +
'? ' '
" " &%r( pathsc : | j j } fdt | j j j j j j j D _ d j D _ t
dt j dS )z Create checkers for each file.Nc H g | ] }t |j j S ))r< r r ) FileCheckerr r ).0r< r% s r&
z)Manager.make_checkers.. sG
!
r( )rI stdin_display_namefilename_patternsr# is_running_from_diffc g | ] }|j | S )should_process)rM cs r& rN z)Manager.make_checkers.. s KKKq!:JKKKKr( zChecking %d files) r r6 r rO r< r# r7 r r r3 infor+ r% rI s ` r&
make_checkerszManager.make_checkers s =L*E
)#'<#B"&,"7 %)\%6
LKD$6KKK
$c$-&8&899999r( c dx}}| j D ]}}t |j d }|j }| j | 5 || || z
}ddd n # 1 swxY w Y |t | z
}~||fS )a Report all of the errors found in the managed file checkers.
This iterates over each of the checkers and reports the errors sorted
by line number.
:returns:
A tuple of the total results found and the results reported.
r c " | d | d fS )N rS )tups r& z Manager.report.. s s1vs1v>N r( )keyN)r sortedr= display_namer processing_filerH r+ )r% results_reported
results_foundr, r= r< s r& reportzManager.report s ,-,=) * *G W_2N2NOOOG +H!11(;;
L
L D$8$87$K$KK
L
L
L
L
L
L
L
L
L
L
L
L
L
L
L S\\ )MM /00s A,,A0 3A0 c t j t }t j t }t | j }|| dS d} | t | j t t | j | j }|D ]}|\ }}} |||< | ||< | |
d}|s(| |
n0# |s)| |
w w xY w| j D ]#}
|
j }|| |
_ || |
_ $dS )z Run the checkers in parallel.NF) chunksizeT) collections defaultdictlistdict_try_initialize_processpoolr
run_serialimap_unordered _run_checksr calculate_pool_chunksizer+ closejoin terminatera r= r ) r%
final_resultsfinal_statisticspool pool_closedpool_mapretr< r= r r, s r& run_parallelzManager.run_parallel s T_SjkoSpSp
6A6Md6S6S +4955 < OO F **
2
&& + H
8
803-':*1
h'-7 ** JJLLL IIKKK K
} < % $ 7D &*n&G&G&I&I"ID 03DN4H0I0IDO , - - - & %r( c d| j S )z)Provide helpful debugging representation.zFileChecker for )r< r s r& __repr__zFileChecker.__repr__, s 1$-111r( c t j | j | j S # t $ r:}| dddt
| j d| Y d }~d S d }~ww xY w)NE902r : )r
FileProcessorr< r OSErrorre typer )r% es r& r zFileChecker._make_processor0 s{
*4=$,GG G
KK1a)9&@&@Q&@&@ A A A44444 s !
A%/A A%rF r@ rG rB c || dd \ }}t | d r"| j | j | }nd}| j |||||f |S )z2Report an error by storing it in the results list.N r[ r
)splithasattrr
noqa_line_forr= append)r% rF r@ rG rB lines r& re zFileChecker.report= s #zz#q 1 1 J 4 % % $.*D>// <> ZZ!^
$Qx &),&7&7# /// 0F : C m #FF{r( c , | j J | j }| j j D ]d}| || } | }n# t $ r |}Y nw xY w|D ] \ }}}}| d||| !edS )z1Run all checks expecting an abstract syntax tree.N)treerF r@ rG rB )r
build_astr r r r r re ) r% astr r, runnerr@ offsetrB _s r& run_ast_checkszFileChecker.run_ast_checks s ~)))n&&((l' FnnV#n66G
! !
!
!
!
!28
.fdA # +!
s A A-,A-c \ | j J | j \ }}}|sdS | j | t d| | j j D ]}| j | | || pd}|D ]\\ }}t || \ }} || cxk rdk r n nt d| | d|| | ]| j
dS )z(Run all checks expecting a logical line.NzLogical line: "%s")r rS r z#position of error out of bounds: %sr )r
build_logical_line update_stater3 debugr r r update_checker_state_forr find_offsetr4 re next_logical_line)
r% commentsr mappingr r= r rB r@ r s
r& run_logical_checksz FileChecker.run_logical_checks s] ~)))*..*K*K*M*M', F ##G,,, & (;(;(=(=>>>l/ F N 3 3F ; ; ;nnV,nGGM2G '
-8-I-I* ] -4444144444KK EvNNN # +(
((*****r( rC c l | j J | j j D ]}| j | | || }|hd} |d }n# t
t f$ r Y nw xY wt |t r|f}|D ]*}|\ }}| d| j j
|| +dS )zoRun all checks for a given physical line.
A single physical check may return multiple errors.
N)rC r r ) r
r rC r r
IndexError TypeErrorr r re r@ )r% rC r resultr
result_singlerB s r& run_physical_checksz FileChecker.run_physical_checks s
~)))l0 F N 3 3F ; ; ;^^F-^HHF! $
$*1IMM"I. D mS11 ' $YF%+ M*7'M4KK#'$(N$> , ! % s AA+*A+c | j J d}| j }| j }d}| D ]}|dxx dz
cc< | || |dd \ }}|t j k rt j || }n/|dk r)t j | r| | |d }|j r6|
|j d | dS dS ) zProcess tokens and trigger checks.
Instead of using this directly, you should use
:meth:`flake8.checker.FileChecker.run_checks`.
Nr r r[ r\ r )
r
r generate_tokenscheck_physical_eolr OPcount_parenthesestoken_is_newlinehandle_newliner r r r )r% parensr file_processor
prev_physicalr
token_typerB s r& process_tokenszFileChecker.process_tokens s. ~)))_
#3355 % %E x A % # #E= 9 9 9 $QqSz JX[(("4VTBB1 -e44 4''
333 !!HMM & $ $^%9"%= > > > # # % % % % % & &r( c
| j J | | n# t t j f$ r}t
|t j rdnd}| | \ }}| |||t | j
d|j d | j | j
| j fcY d}~S d}~ww xY w| j j d }|| j d< | j | j
| j fS )z Run checks against the file.Nr E999r r r )r
r r r r r r r re r r r r< r= r )r% r r? r rG
logical_liness r& r| zFileChecker.run_checks s! ~))) @ ! ! ! ! ! ! !X 01 @ @ @ '8+> ? ?K66VD ::1==KC KKc6d1gg.>+M+M!&)+M+M N N N=$,? ? ? ? ? ? ? @ 1/B
+8(}dlDO;;s (4 C
BC
CCr c ` | j J |t j k r/| | j dS t | j j dk r4| j | j dS | dS )z3Handle the logic when encountering a newline token.Nr[ ) r
r NEWLINEr reset_blank_beforer+ r visited_new_blank_linedelete_first_token)r% r s r& r zFileChecker.handle_newline s ~))) ) ) ) # # % % % N - - / / / / /
&
'
'1
,
, N 1 1 3 3 3 N - - / / / / / # # % % % % %r( r r c | j J t j | r@|d dk r| | dS | |d dS t j | rw|d d }| j | 5 | j | D ]}| | ddd dS # 1 swxY w Y dS dS )z@Run physical checks if and only if it is at the end of the line.Nr r r\ r )r@ )r
is_eol_tokenr is_multiline_stringinside_multiline
split_line)r% r r line_nor s r& r z FileChecker.check_physical_eol, sP ~))) !% ( ( 3 Qx2~~(( 77777((q22222
*5
1
1 3 AhqkG 00W0EE
3
3 N 5 5e < < 3 3D,,T22223
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3 3 3s 3C C #C r )"r r r r r r
argparse Namespacer' r r r
r r r re r r r staticmethodr r r r r r r r r r| r r TokenInfor rS r( r& rL rL s% EEJ J J
# J
J J J J22# 2 2 2 2 )*A!B SM
* 3 3 2 8y 8U38_ 8 8 8 \8t *+ + + +4 >& & & &6 ? < < < < & & & & & & 3 ' 38; 3
3 3 3 3 3 3r( rL r c X t j t j t j dS )z:Ensure correct signaling of ^C using multiprocessing.Pool.N)signalSIGINTSIG_IGNrS r( r&
_pool_initr L s
M&- 00000r( job_countc t j | t S # t $ r}|j t
vr Y d}~nd}~wt $ r Y nw xY wdS )z@Return a new process pool instance if we are able to create one.N)r1 Poolr r errnoSERIAL_RETRY_ERRNOS ImportError)r errs r& rl rl Q sy
#Iz::: 9 / / / 0 / / / /
4s
A
:
A
A
num_checkersnum_jobsc . t | |dz z d S )a Determine the chunksize for the multiprocessing Pool.
- For chunksize, see: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.imap # noqa
- This formula, while not perfect, aims to give each worker two batches of
work.
- See: https://github.com/pycqa/flake8/issues/829#note_18878876
- See: https://github.com/pycqa/flake8/issues/197
r\ r[ )max)r r
s r& rp rp ` s |1 -q 1 11r( r, c * | S r )r| )r, s r& ro ro l s r( r r c t | t r| S |D ]}|d }| |k r
|d } nd}dx} }|d |d | z |z
fS )z*Find the offset tuple for a single offset.r r[ )r r )r r )r r r token_offsetpositions r& r r p s &%
" " Qx \ ! ! QxH E " !! QK!v- < ==r( r ).r r rh r r! loggingmultiprocessing.poolr1 r r typingr r r r r flake8r r r
r flake8.discover_filesr flake8.plugins.finderr
r flake8.style_guider r r r getLoggerr r3 ENOSPCr r rL r rv r rl rp ro _LogicalMappingr rS r( r& r s * *
. . . . . . * * * * * * . . . . . . 0 0 0 0 0 0
uS#sC# 67
8g!!
L
`# `# `# `# `# `# `# `#Fy3 y3 y3 y3 y3 y3 y3 y3x 1 1 1 1
o"'( 23 2# 2# 2 2 2 2 sGT#s(^/K)L > >#3>
38_> > > > > >r(