s(mr x d dl Z d dlZd dlZd dlmZ d dlmZmZmZm Z d dl
m Z d dl m
Z
mZ G d d ZdS ) N)Path)DictListOptionalAny)drop_privileges)DOCROOT_EXCLUDE_DIRSDOCROOT_MAX_DEPTHc e Zd ZdZdej fdZddededede d e
e ee f f
d
Z
ed fded e de d ee fd
Zdeded efdZdeded efdZdS )DocrootProcessorzO
Processes individual docroot to collect .htaccess files and metadata.
loggerc || _ d S )N)r
)selfr
s ,py/cl_website_collector/docroot_processor.py__init__zDocrootProcessor.__init__ s
docrootdomainsusernametimeoutreturnc < t j }|||g g dddd} | j d| t | 5 | |t
|dz
}| j dt
| | |D ] }| j d| |s | j d | n|D ]}t j |z
|k r#d
|d < | j d | nb | j d
| t | } | }
|
r#t | d n|} |
r2|d |
|| | d t | rVt j | t j r7|
|| } |d | || |
d n| j d| n# t$ $ r'}
| j d||
Y d}
~
d}
~
ww xY wddd n # 1 swxY w Y t
|d |d< t j |z
|d< | j d|d ||d n3# t$ $ r&}
| j d||
Y d}
~
nd}
~
ww xY w|S )ab
Collect .htaccess file paths from a docroot without reading file contents.
Args:
docroot: Document root path
domains: Domain names
username: Owner username
timeout: Processing timeout in seconds
Returns:
Dictionary with collected file paths or None if failed
Fr )r r r htaccess_file_pathssymlinkstimeout_reachedprocessing_time_secondshtaccess_files_foundz Finding .htaccess files in %s ) max_depthr z Found %d .htaccess files in %sz - %sz No .htaccess files found in %sTr z@[WEBSITE-COLLECTOR] Timeout reached while collecting paths in %sz Collecting .htaccess path: %sstrictr )linktargetr )location file_path real_path
is_symlinkzCannot read file: %sz0[WEBSITE-COLLECTOR] Error collecting path %s: %sNr r z2Collected %d .htaccess file paths from %s in %.2fsz3[WEBSITE-COLLECTOR] Error processing docroot %s: %s)timer
debugr _find_htaccess_filesr
lenerrorr r( strresolveappend_normalize_pathexistsosaccessR_OK Exception)r r r r r
start_timeresulthtaccess_filesr& pr( r' r% es r collect_htaccess_pathsz'DocrootProcessor.collect_htaccess_paths s Y[[
#% $'($%
6 a K =w G G G ** *
p *
p!%!:!:7N_ipstit!:!u!u !!"BCDWDWY`aaa!/ ; ;IK%%h ::::% #pK%%&FPPPP%3 p p 9;;3g==8z9DocrootProcessor._find_htaccess_files.. s0 ZZZ$2P2PQUWX2Y2YZ1ZZZr z .htaccessz([WEBSITE-COLLECTOR] Error walking %s: %s)r) r3 walkr
r- pathrelpathcountsepr is_filer4 r. r5 r0 r6 ) r r r r r7 r9 dirsfilesdepthr& r; rD s ` @r r+ z%DocrootProcessor._find_htaccess_filesn s Y[[
V%'WW%5%5
>
>!dE9;;+g55K%%&RT[\\\E 7?? EE GOOD' : : @ @ H HEI%% DG [ZZZZdZZZQQQ %'' $T
[ 8I "))++ > Ic)nnbg > >>&--c)nn=== V V V K H'ST U U U U U U U U V s D3E
E= E88E= parent_pathdirnamec L t | |z }| d }t D ]`}t | }t | t
j t | z s|j |j k r dS an# t $ r Y dS w xY wdS )a3
Check if directory should be excluded based on DOCROOT_EXCLUDE_DIRS.
Supports both plain directory names (e.g. "node_modules") and nested
paths (e.g. "wp-content/cache"). The check is performed against the
full candidate path composed from parent_path and dirname.
Fr! T) r r/ r r. endswithr3 rJ namer6 )r rO rP candidatecandidate_normalized exclude_dirpatterns r rA z*DocrootProcessor._should_exclude_directory s [))G3I#,#4#4E#4#B#B 3
{++,--66rvG 7LMM !',66 44 7
55 us B
B B
B! B!r& c t t | t | S # t $ r t | j cY S w xY w)z:
Normalize file path relative to docroot.
)r. r relative_to
ValueErrorrS )r r& r s r r1 z DocrootProcessor._normalize_path s_ (tI224==AABB B ( ( ( ??' ' ' ' (s ;> A A N)r )__name__
__module__ __qualname____doc__loggingLoggerr r. listintr r r r< r
r r+ boolrA r1 r@ r r r r s/ w~ R Rc RD RC RZ] Rgo cNh R R R Rh CTdf % % C % C % ^a % kopskt % % % % NS 3 4 ,( (s (s ( ( ( ( ( (r r )r_ r3 r) pathlibr typingr r r r clcommon.clpwdr cl_website_collector.constantsr r
r r@ r r rh s , , , , , , , , , , , , * * * * * * R R R R R R R Ra( a( a( a( a( a( a( a( a( a(r