a
lj
D @ s U d Z ddlZddlZddlZddlZddlZddlmZ ddlm Z m
Z
m Z m Z m
Z
mZmZmZmZ ddlmZ ddlmZmZ ddlmZ ddlmZ dd lm Z dd
l m Z dd l m Z e rdd lm!Z! dd
l"m#Z# dgZ$e
ee%e&f Z'e e&gee&ef f Z(e)dej*Z+e)dZ,e)dZ-ej.ej/ej0ej1ej2ej3ej4ej5ej6ej7ej8ej9ej:ej;gZd< ej?ej@ejAgZBee dej=f e>d< dd eBD ZCG dd dZDG dd dZEdBe&eed eej eFeeD d d dZGe&e'd d d!ZHdCeEeej eDd"d#d$ZIdDee&e%ed eej ee dd%d&d'ZJdEeEeej ed ee eeD d(d)d*ZKG d+d, d,ZLed e(d-d.d/ZMe&ee&e&f d0d1d2ZNG d3d4 d4eOZPejQd5d6d7ZRe'e'd8d9d:ZSe'e'd8d;d<ZTe'e'd8d=d>ZUe&eee&e&f d?d@dAZVdS )Fz
Requirements file parsing
N)Values)
TYPE_CHECKINGAnyCallableDictIterableIteratorListOptionalTuple)
cmdoptions)InstallationErrorRequirementsFileParseError) SearchScope)
PipSession)raise_for_status) auto_decode)get_url_scheme)NoReturn)
PackageFinderparse_requirementsz^(http|https|file):z (^|\s+)#.*$z#(?P\$\{(?P[A-Z0-9_]+)\}).SUPPORTED_OPTIONSSUPPORTED_OPTIONS_REQc C s g | ]}t | jqS )strdest).0or r >/usr/lib/python3.9/site-packages/pip/_internal/req/req_file.py
N r c @ s8 e Zd Zdeeeeeeeef ee ddddZdS )ParsedRequirementN) requirement is_editable
comes_from
constraintoptions line_sourcereturnc C s( || _ || _|| _|| _|| _|| _d S N)r" r# r$ r& r% r' )selfr" r# r$ r% r& r' r r r __init__R s zParsedRequirement.__init__)NN) __name__
__module__ __qualname__r boolr
r r r+ r r r r r! Q s r! c @ s$ e Zd ZeeeeeddddZdS )
ParsedLineN)filenamelinenoargsoptsr% r( c C sZ || _ || _|| _|| _|r0d| _d| _|| _n&|jrPd| _d| _|jd | _nd| _d S )NTFr )r1 r2 r4 r% is_requirementr# r" Z editables)r* r1 r2 r3 r4 r% r r r r+ d s zParsedLine.__init__)r, r- r. r intr r/ r+ r r r r r0 c s r0 Fr )r1 sessionfinderr& r% r( c c sF t |}t||}|| |D ]"}t||||d}|dur |V q dS )aq Parse a requirements file and yield ParsedRequirement instances.
:param filename: Path or url of requirements file.
:param session: PipSession instance.
:param finder: Instance of pip.index.PackageFinder.
:param options: cli options.
:param constraint: If true, parsing a constraint file rather than
requirements file.
)r& r8 r7 N)get_line_parserRequirementsFileParserparse handle_line) r1 r7 r8 r& r% line_parserparserZ parsed_line
parsed_reqr r r r ~ s
)contentr( c C s, t | dd}t|}t|}t|}|S )zxSplit, filter, and join lines, and return a line iterator
:param content: the content of the requirements file
)start) enumerate
splitlines
join_linesignore_commentsexpand_env_variables)r@
lines_enumr r r
preprocess s
rI )liner& r( c C s d | jrdnd| j| j}| jr8t| j| j|| jdS |rJt|| j i }t
D ],}|| j j v rR| j j | rR| j j | ||< qRd| j d| j }t| j| j|| j||dS d S )Nz{} {} (line {})z-cz-r)r" r# r$ r% line of )r" r# r$ r% r& r' ) formatr% r1 r2 r# r! r" r Zcheck_install_build_globalr4 SUPPORTED_OPTIONS_REQ_DEST__dict__)rJ r& Zline_comes_fromZ req_optionsr r' r r r handle_requirement_line s6 rP )r4 r1 r2 r8 r& r7 r( c s8 r4| j r| j _ | jr4 j fdd| jD |r4|j}|j}| jrT| jg}| jdu rbg }| jrt|| j | jr| jd }tj
tj |} tj | |}
tj
|
r|
}|| |r|| t||d} | |_| jr| | jr| |r4| jp g D ]$} d| d| }
|j| |
d qd S ) Nc 3 s | ]}| j vr|V qd S r) )features_enabled)r fr& r r s z%handle_option_line..Tr )
find_links
index_urlsrK rL )source)require_hashesrQ extendrU rV index_urlno_indexZextra_index_urlsospathdirnameabspathjoinexistsappendZupdate_index_urlsr search_scopepreZset_allow_all_prereleases
prefer_binaryZset_prefer_binaryZ
trusted_hostsZadd_trusted_host)r4 r1 r2 r8 r& r7 rU rV valueZreq_dirZrelative_to_reqs_filerc hostrW r rS r handle_option_line sL
rh )rJ r& r8 r7 r( c C s4 | j rt| |}|S t| j| j| j||| dS dS )a Handle a single parsed requirements line; This can result in
creating/yielding requirements, or updating the finder.
:param line: The parsed line to be processed.
:param options: CLI options.
:param finder: The finder - updated by non-requirement lines.
:param session: The session - updated by non-requirement lines.
Returns a ParsedRequirement object if the line is a requirement line,
otherwise returns None.
For lines that contain requirements, the only options that have an effect
are from SUPPORTED_OPTIONS_REQ, and they are scoped to the
requirement. Other options from SUPPORTED_OPTIONS may be present, but are
ignored.
For lines that do not contain requirements, the only options that have an
effect are from SUPPORTED_OPTIONS. Options from SUPPORTED_OPTIONS_REQ may
be present, but are ignored. These lines may contain multiple options
(although our docs imply only one is supported), and all our parsed and
affect the finder.
N)r5 rP rh r4 r1 r2 )rJ r& r8 r7 r? r r r r< s
r< c @ s` e Zd ZeeddddZeeee dddZ
eeee ddd Z eeee dd
d Z dS ) r: N)r7 r= r( c C s || _ || _d S r) )_session _line_parser)r* r7 r= r r r r+ < s z RequirementsFileParser.__init__)r1 r% r( c c s | ||E dH dS )z*Parse a given file, yielding parsed lines.N)_parse_and_recurse)r* r1 r% r r r r; D s z RequirementsFileParser.parsec c s | ||D ]}|js|jjs&|jjr|jjr@|jjd }d}n|jjd }d}t|rjtj ||}n t|st
j t
j
||}| ||E d H q |V q d S )Nr FT) _parse_filer5 r4 requirements constraints SCHEME_REsearchurllibr; urljoinr\ r] r` r^ rk )r* r1 r% rJ Zreq_pathZnested_constraintr r r rk H s(
z)RequirementsFileParser._parse_and_recursec c s t || j\}}t|}|D ]l\}}z| |\}} W n> tyt }
z&d| d|
j } t| W Y d }
~
n
d }
~
0 0 t|||| |V q d S )NzInvalid requirement:
)get_file_contentri rI rj OptionParsingErrormsgr r0 ) r* r1 r% _r@ rH line_numberrJ args_strr4 erv r r r rl g s z"RequirementsFileParser._parse_file)
r, r- r. r
LineParserr+ r r/ r r0 r; rk rl r r r r r: ; s r: )r8 r( c s t tt tf d fdd }|S )NrJ r( c sJ t }| }d |_ r j|_t| \}}|t||\}}||fS r) ) build_parserZget_default_valuesrZ Zformat_controlbreak_args_options
parse_argsshlexsplit)rJ r> defaultsry Z options_strr4 rw r8 r r
parse_line~ s z#get_line_parser..parse_line)r r r )r8 r r r r r9 } s r9 r| c C sf | d}g }|dd }|D ]2}|ds6|dr< qRq || |d q d|d|fS )zBreak up the line into an args and options string. We only want to shlex
(and then optparse) the options, not the args. args can contain markers
which are corrupted by shlex.
N-z--r )r
startswithrb popr` )rJ tokensr3 r& tokenr r r r~ s
r~ c @ s e Zd ZeddddZdS )ru N)rv r( c C s
|| _ d S r) )rv r* rv r r r r+ s zOptionParsingError.__init__)r, r- r. r r+ r r r r ru s ru )r( c C sJ t jdd} tt }|D ]}| }| | qttdddd}|| _| S )z7
Return a parser for parsing requirement lines
F)Zadd_help_optionr )r* rv r( c S s t |d S r) )ru r r r r parser_exit s z!build_parser..parser_exit)optparse OptionParserr r Z
add_optionr r exit)r> Zoption_factoriesZoption_factoryoptionr r r r r} s r} )rH r( c c s d}g }| D ]v\}}| dr(t|rjt|r:d| }|r^|| |d|fV g }q||fV q |sr|}||d q |r|d|fV dS )zJoins a line ending in '' with the previous line (except when following
comments). The joined line takes on the index of the first line.
N\r )endswith
COMMENT_REmatchrb r` strip)rH Zprimary_line_numberZnew_linerx rJ r r r rE s
rE c c s4 | D ]*\}}t d|}| }|r||fV qdS )z1
Strips comments and filter empty lines.
r N)r subr )rH rx rJ r r r rF s
rF c c sL | D ]B\}}t |D ]$\}}t|}|s.q|||}q||fV qdS )a Replace all environment variables that can be retrieved via `os.getenv`.
The only allowed format for environment variables defined in the
requirement file is `${MY_VARIABLE_1}` to ensure two things:
1. Strings that contain a `$` aren't accidentally (partially) expanded.
2. Ensure consistency across platforms for requirement files.
These points are the result of a discussion on the `github pull
request #3514 `_.
Valid characters in variable names follow the `POSIX standard
`_ and are limited
to uppercase letter, digits and the `_` (underscore).
N)
ENV_VAR_REfindallr\ getenvreplace)rH rx rJ Zenv_varZvar_namerf r r r rG s
rG )urlr7 r( c
C s t | }|dv r.|| }t| |j|jfS z:t| d }t| }W d n1 s\0 Y W n2 ty } zt d| W Y d}~n
d}~0 0 | |fS )a Gets the content of a file; it may be a filename, file: URL, or
http: URL. Returns (location, content). Content is unicode.
Respects # -*- coding: declarations on the retrieved files.
:param url: File path or url.
:param session: PipSession instance.
)httphttpsfilerbNz"Could not open requirements file: )
r getr r textopenr readOSErrorr
)r r7 schemeZresprR r@ excr r r rt s
.$rt )NNF)N)NNN)NNN)W__doc__r r\ rer urllib.parserq r typingr r r r r r r r
r Zpip._internal.clir Zpip._internal.exceptionsr
r Z!pip._internal.models.search_scoper Z pip._internal.network.sessionr Zpip._internal.network.utilsr Z pip._internal.utils.encodingr Zpip._internal.utils.urlsr r Z"pip._internal.index.package_finderr __all__r6 r Z ReqFileLinesr{ compileIro r r rZ Zextra_index_urlr[ rn rm ZeditablerU Z no_binaryZ only_binaryre rX rd Z trusted_hostZuse_new_featurer ZOption__annotations__Zinstall_optionsZglobal_optionshashr rN r! r0 r/ r rI rP rh r< r: r9 r~ Exceptionru r r} rE rF rG rt r r r r s ,
1 ? ,B