a
/h| @ s d Z ddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl
Z
ddl Z ddl m
Z
ddlmZ ddlmZ ddlmZ ddlmZ dd lmZmZmZmZmZmZm Z m Z m Z m Z m Z m!Z!m"Z"m#Z# dd
l$m%Z% dd l&m'Z' dd l(m)Z)m*Z*m+Z+m,Z, d
Z-e. Z/dddZ0ej1dkr:dd Z2dd Z dd Z3dd Z4dpdd Z5d d Z6d d Z7d!d" Z8d#d$ Z9d%d& Z:d'd( Z;dqd)d*Zd/d0 Z?d1d2 Z@d3d4 ZAd5d6 ZBd7d8 ZCd9d: ZDeEd;ZFdd? ZHdrdAdBZIdCdD ZJdEdF ZKdGdH ZLdIdJ ZMdKdL ZNdMdN ZOejPdOdP ZQdQdR ZRdsdSdTZSdUdV ZTdtdXdYZUdZd[ ZVd\d] ZWd^Xd_ZYeYd` ZZeYda Z[dbdc Z\ddde Z]dfdg Z^e_dhZ`e_diZadjdk Zbdldm Zcdndo ZddS )uz
requests.utils
~~~~~~~~~~~~~~
This module provides utility functions that are used within Requests
that are also useful for external consumption.
N) OrderedDict __version__)certs)to_native_string)parse_http_list)quoteurlparsebytesstrunquote
getproxies proxy_bypass
urlunparse
basestring
integer_typesis_py3proxy_bypass_environmentgetproxies_environmentMappingcookiejar_from_dict)CaseInsensitiveDict)
InvalidURL
InvalidHeaderFileModeWarningUnrewindableBodyError)z.netrc_netrcP i )httphttpswin32c C s zt rdd l}ndd l}W n ty. Y dS 0 z6||jd}t||dd }||dd }W n tyz Y dS 0 |r|sdS | d}|D ]T}|dkrd| vr d S |
dd
}|
d d }|
d
d}t || t j
r d S qdS )Nr Fz;Software\Microsoft\Windows\CurrentVersion\Internet SettingsZ ProxyEnableZ
ProxyOverride;z.Tz\.*z.*?)r winregZ_winreg ImportErrorOpenKeyHKEY_CURRENT_USERintZ QueryValueExOSErrorsplitreplacerematchI)hostr' ZinternetSettingsZ proxyEnableZ
proxyOverridetest r4 2/usr/lib/python3.9/site-packages/requests/utils.pyproxy_bypass_registry0 sF
r6 c C s t rt| S t| S dS )zReturn True, if the host should be bypassed.
Checks proxy settings gathered from the environment, if specified,
or the registry.
N)r r r6 )r2 r4 r4 r5 r W s r c C s t | dr| } | S )z/Returns an internal sequence dictionary update.items)hasattrr7 )dr4 r4 r5 dict_to_sequencec s
r: c C s. d }d}t | dr t| }n`t | dr.| j}nNt | dr|z | }W n tjyX Y n$0 t|j}d| jvr|t
dt t | drz | }W n" t
tfy |d ur|}Y n\0 t | dr|d u rz&| dd | }| |pd W n t
tfy d}Y n0 |d u r d}td|| S )
Nr __len__lenfilenoba% Requests has determined the content-length for this request using the binary size of the file: however, the file has been opened in text mode (i.e. without the 'b' flag in the mode). This may lead to an incorrect content-length. In Requests 3.0, support will be removed for files in text mode.tellseek )r8 r< r= ioUnsupportedOperationosfstatst_sizemodewarningswarnr r? r, IOErrorr@ max)oZ total_lengthZcurrent_positionr= r4 r4 r5 super_lenl sB
rM Fc
C s t jd}|dur |f}ndd tD }zddlm}m} d}|D ]B}zt j|}W n typ Y W dS 0 t j |rD|} qqD|du rW dS t
| } | j }
z:|| |
} | r| d rdnd} | | | d fW W S W n |t
fy |r Y n0 W n ttfy Y n0 dS ) z;Returns the Requests tuple auth for a given url from netrc.ZNETRCNc s s | ]}d |V qdS )z~/{}N)format).0fr4 r4 r5 z!get_netrc_auth..r )netrcNetrcParseErrorr rA )rD environget NETRC_FILESrS rT path
expanduserKeyErrorexistsr
hostnameZauthenticatorsrJ r( AttributeError)
urlZ raise_errorsZ
netrc_fileZnetrc_locationsrS rT Z
netrc_pathrP locZrir2 r Zlogin_ir4 r4 r5 get_netrc_auth s: r` c C sB t | dd}|r>t|tr>|d dkr>|d dkr>tj|S dS )z0Tries to guess the filename of the given object.nameNr <>)getattr
isinstancer rD rX basename)objra r4 r4 r5 guess_filename s
ri c C s t j| r| S t j| \}}|rPt j|sPt j|\}}d||g}q t|s^| S t|}|| vrx| S t
}t jj|g|dR }t j|s|j ||d}|S )zReplace nonexistent paths that look like they refer to a member of a zip
archive with the location of an extracted copy of the target, or else
just return the provided path unchanged.
/)rX ) rD rX r[ r- joinzipfile
is_zipfileZipFileZnamelisttempfileZ
gettempdirextract)rX archivememberprefixZzip_filetmpZextracted_pathr4 r4 r5 extract_zipped_paths s
ru c C s. | du r dS t | ttttfr&tdt| S )a Take an object and test to see if it can be represented as a
dictionary. Unless it can not be represented as such, return an
OrderedDict, e.g.,
::
>>> from_key_val_list([('key', 'val')])
OrderedDict([('key', 'val')])
>>> from_key_val_list('string')
Traceback (most recent call last):
...
ValueError: cannot encode objects that are not 2-tuples
>>> from_key_val_list({'key': 'val'})
OrderedDict([('key', 'val')])
:rtype: OrderedDict
N+cannot encode objects that are not 2-tuples)rf r r boolr+
ValueErrorr valuer4 r4 r5 from_key_val_list s
r{ c C s@ | du r dS t | ttttfr&tdt | tr8| } t| S )a Take an object and test to see if it can be represented as a
dictionary. If it can be, return a list of tuples, e.g.,
::
>>> to_key_val_list([('key', 'val')])
[('key', 'val')]
>>> to_key_val_list({'key': 'val'})
[('key', 'val')]
>>> to_key_val_list('string')
Traceback (most recent call last):
...
ValueError: cannot encode objects that are not 2-tuples
:rtype: list
Nrv ) rf r r rw r+ rx r r7 listry r4 r4 r5 to_key_val_list s
r} c C sX g }t | D ]F}|dd |dd kr4dkrHn nt|dd }|| q |S )a Parse lists as described by RFC 2068 Section 2.
In particular, parse comma-separated lists where the elements of
the list may include quoted-strings. A quoted-string could
contain a comma. A non-quoted string could have quotes in the
middle. Quotes are removed automatically after parsing.
It basically works like :func:`parse_set_header` just that items
may appear multiple times and case sensitivity is preserved.
The return value is a standard :class:`list`:
>>> parse_list_header('token, "quoted value"')
['token', 'quoted value']
To create a header from the :class:`list` again, use the
:func:`dump_header` function.
:param value: a string with a list header.
:return: :class:`list`
:rtype: list
Nr rc ")_parse_list_headerunquote_header_valueappend)rz resultitemr4 r4 r5 parse_list_header= s ( r c C sx i }t | D ]f}d|vr"d||< q |dd\}} | dd | dd krVdkrjn nt| dd } | ||< q |S )a^ Parse lists of key, value pairs as described by RFC 2068 Section 2 and
convert them into a python dict:
>>> d = parse_dict_header('foo="is a fish", bar="as well"')
>>> type(d) is dict
True
>>> sorted(d.items())
[('bar', 'as well'), ('foo', 'is a fish')]
If there is no value for a key it will be `None`:
>>> parse_dict_header('key_without_value')
{'key_without_value': None}
To create a header from the :class:`dict` again, use the
:func:`dump_header` function.
:param value: a string with a dict header.
:return: :class:`dict`
:rtype: dict
=Nr rc r~ )r r- r )rz r r ra r4 r4 r5 parse_dict_header] s (
r c C s\ | rX| d | d kr dkrXn n4| dd } |rD| dd dkrX| dd d dS | S )
zUnquotes a header value. (Reversal of :func:`quote_header_value`).
This does not use the real unquoting but what browsers are actually
using for quoting.
:param value: the header value to unquote.
:rtype: str
r rc r~ r NrA z\\\z\")r. )rz Z is_filenamer4 r4 r5 r s
$ r c C s i }| D ]}|j ||j< q|S )zReturns a key/value dictionary from a CookieJar.
:param cj: CookieJar object to extract cookies from.
:rtype: dict
)rz ra )cj cookie_dictZcookier4 r4 r5 dict_from_cookiejar s r c C s
t || S )zReturns a CookieJar from a key/value dictionary.
:param cj: CookieJar to insert cookies into.
:param cookie_dict: Dict of key/values to insert into CookieJar.
:rtype: CookieJar
r )r r r4 r4 r5 add_dict_to_cookiejar s r c C sT t dt tjdtjd}tjdtjd}td}|| || || S )zlReturns encodings from given content string.
:param content: bytestring to extract encodings from.
zIn requests 3.0, get_encodings_from_content will be removed. For more information, please see the discussion on issue #2266. (This warning should only appear once.)z!])flagsz+]z$^<\?xml.*?encoding=["\']*(.+?)["\'>])rH rI DeprecationWarningr/ compiler1 findall)contentZ
charset_reZ pragma_reZxml_rer4 r4 r5 get_encodings_from_content s
r c
C s | d}|d |dd }}i }d}|D ]`}| }|r0|d }}|d} | dkr|d| |}|| d d |}||| < q0||fS ) zReturns content type and parameters from given header
:param header: string
:return: tuple containing content type and dictionary of
parameters
r# r r Nz"' Tr rc )r- stripfindlower)
headertokens content_typeparamsZ params_dictZitems_to_stripparamkeyrz Zindex_of_equalsr4 r4 r5 _parse_content_type_header s
r c C sP | d}|sdS t|\}}d|v r4|d dS d|v r@dS d|v rLdS dS ) z}Returns encodings from given HTTP Header Dict.
:param headers: dictionary to extract encoding from.
:rtype: str
z content-typeNcharsetz'"textz
ISO-8859-1zapplication/jsonutf-8)rV r r )headersr r r4 r4 r5 get_encoding_from_headers s
r c c sj |j du r | D ]
}|V qdS t|j dd}| D ]}||}|r4|V q4|jddd}|rf|V dS )zStream decodes a iterator.Nr. errorsrR T)final)encodingcodecsgetincrementaldecoderdecode)iteratorrr decoderchunkrvr4 r4 r5 stream_decode_response_unicode s
r c c sH d}|du s|dkr t | }|t | k rD| ||| V ||7 }q dS )z Iterate over slices of a string.r N)r< )stringZ slice_lengthposr4 r4 r5 iter_slices s r c C sz t dt g }t| j}|rJzt| j|W S tyH || Y n0 zt| j|ddW S t yt | j Y S 0 dS )zReturns the requested content back in unicode.
:param r: Response object to get unicode content from.
Tried:
1. charset from content-type
2. fall back and replace all unicode characters
:rtype: str
zIn requests 3.0, get_unicode_from_response will be removed. For more information, please see the discussion on issue #2266. (This warning should only appear once.)r. r N)
rH rI r r r r r UnicodeErrorr TypeError)r Ztried_encodingsr r4 r4 r5 get_unicode_from_response s
r zBABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~c C s | d}tdt|D ]}|| dd }t|dkr| rztt|d}W n typ td| Y n0 |tv r||| dd ||< qd|| ||< qd|| ||< qd |S ) zUn-escape any percent-escape sequences in a URI that are unreserved
characters. This leaves all reserved, illegal and non-ASCII bytes encoded.
:rtype: str
%r r rA z%Invalid percent-escape sequence: '%s'N )
r- ranger< isalnumchrr+ rx r UNRESERVED_SETrk )uripartsihcr4 r4 r5 unquote_unreserved> s
r c C s> d}d}zt t| |dW S ty8 t | |d Y S 0 dS )zRe-quote the given URI.
This function passes the given URI through an unquote/quote cycle to
ensure that it is fully and consistently quoted.
:rtype: str
z!#$%&'()*+,/:;=?@[]~z!#$&'()*+,/:;=?@[]~)safeN)r r r )r Zsafe_with_percentZsafe_without_percentr4 r4 r5 requote_uriV s r c C s d| > d || > }|S )a Converts a mask from /xx format to a int
to be used as a mask for IP's in int format
Example: if mask is 24 function returns 0xFFFFFF00
if mask is 24 and totalbits=128 function
returns 0xFFFFFF00000000000000000000000000
:rtype: int
r r4 )maskZ totalbitsbitsr4 r4 r5 _get_mask_bitsl s
r c C s | d\}}t| rXt|rXtdt| d }tt|}tdt|d }npt| rt|rtdt tj
| \}}|d> |A }tt|d}tdt tj
|\} }
| d> |
A }ndS ||@ ||@ kS )a3 This function allows you to check if an IP belongs to a network subnet
Example: returns True if ip = 192.168.1.1 and net = 192.168.1.0/24
returns False if ip = 192.168.1.1 and net = 192.168.100.0/24
returns True if ip = 1:2:3:4::1 and net = 1:2:3:4::/64
:rtype: bool
rj z>Lr z>QQ@ F) r- is_ipv4_addressstructunpacksocketZ inet_atonr r+ is_ipv6_address inet_ptonAF_INET6) ipnetZnetaddrr ZipaddrnetmasknetworkZ
ipaddr_msbZ
ipaddr_lsbZ network_msbZ network_lsbr4 r4 r5 address_in_networkz s" r c C s& ddd| > d A }t td|S )zConverts mask from /xx format to xxx.xxx.xxx.xxx
Example: if mask is 24 function returns 255.255.255.0
:rtype: str
l r r z>I)r Z inet_ntoar pack)r r r4 r4 r5 dotted_netmask s r c C s. zt t j| W n t jy( Y dS 0 dS z
:rtype: bool
FT)r r ZAF_INETerrorZ string_ipr4 r4 r5 r s
r c C s. zt t j| W n t jy( Y dS 0 dS r )r r r r r r4 r4 r5 r s
r c C sD | |kr dS z t t j| t t j|kW S ty> Y dS 0 dS )zW
Compare 2 IP's, uses socket.inet_pton to normalize IPv6 IPs
:rtype: bool
TFN)r r r r, )ar> r4 r4 r5 compare_ips s r c C s | ddkr| d\}}z t|}W n ty< Y dS 0 t|r\|dk sV|dkr~dS qt|rz|dk st|dkr~dS qdS ndS dS )zV
Very simple check of the cidr format in no_proxy variable.
:rtype: bool
rj r Fr r T)countr- r+ rx r r )Zstring_networkaddressr r4 r4 r5
is_valid_cidr s r c c st |du}|r"t j| }|t j| < z*dV W |rp|du rBt j| = qp|t j| < n"|rn|du rdt j| = n
|t j| < 0 dS )zSet the environment variable 'env_name' to 'value'
Save previous value, yield, and then restore the previous value stored in
the environment variable 'env_name'.
If 'value' is None, do nothingN)rD rU rV )Zenv_namerz Z
value_changed old_valuer4 r4 r5 set_environ s
r c
C sJ dd }|}|du r |d}t | }|jdu r2dS |rdd |dd d
D }t|jsft|jr|D ]2}t|rt|j|r dS qjt|j|rj dS qjnB|j}|j r|d
|j 7 }|D ] }|j |s| |r dS qt d|> zt
|j}W n ttjfy d }Y n0 W d n1 s20 Y |rFdS d S )
zL
Returns whether we should bypass proxies or not.
:rtype: bool
c S s t j| pt j| S N)rD rU rV upper)kr4 r4 r5 rR z'should_bypass_proxies..Nno_proxyTc s s | ]}|r|V qd S r r4 )rO r2 r4 r4 r5 rQ s z(should_bypass_proxies.. r ,z:{}F)r
r\ r. r- r r r r r portrN endswithr r r r Zgaierror) r^ r Z get_proxyZ no_proxy_argparsedZproxy_ipZhost_with_portr2 Zbypassr4 r4 r5 should_bypass_proxies s@
*r c C s t | |dri S t S dS )zA
Return a dict of environment proxies.
:rtype: dict
)r N)r r )r^ r r4 r4 r5 get_environ_proxies5 s r c C st |pi }t | }|jdu r.||j|dS |jd |j |jd|j dg}d}|D ]}||v rV|| } qpqV|S )zSelect a proxy for the url, if applicable.
:param url: The url being for the request
:param proxies: A dictionary of schemes or schemes and hosts to proxy URLs
Nallz://zall://)r
r\ rV scheme)r^ ZproxiesZurlpartsZ
proxy_keysproxyZ proxy_keyr4 r4 r5 select_proxyA s
r python-requestsc C s d| t f S )zO
Return a string representing the default user agent.
:rtype: str
z%s/%sr )ra r4 r4 r5 default_user_agent[ s r c C s t t dddddS )z9
:rtype: requests.structures.CaseInsensitiveDict
z, )gzipZdeflatez*/*z
keep-alive)z
User-AgentzAccept-EncodingZAcceptZ
Connection)r r rk r4 r4 r4 r5 default_headersd s r c
C s g }d}| |} | s|S td| D ]}z|dd\}}W n tyZ |d }}Y n0 d| di}|dD ]D}z|d\}} W n ty Y qY n0 | ||| |< qt|| q&|S ) zReturn a list of parsed link headers proxies.
i.e. Link: ; rel=front; type="image/jpeg",; rel=back;type="image/jpeg"
:rtype: list
z '"z, * '"r )r r/ r- rx r ) rz ZlinksZ
replace_charsvalr^ r linkr r r4 r4 r5 parse_header_linksp s&
r asciirA c C s | dd }|t jt jfv r dS |dd t jkr6dS |dd t jt jfv rRdS |t}|dkrhd S |dkr|ddd tkrd
S |d dd tkrd S |dkr|dd t krd
S |d d t krdS dS )z
:rtype: str
N zutf-32r z utf-8-sigrA zutf-16r r z utf-16-ber z utf-16-lez utf-32-bez utf-32-le)
r BOM_UTF32_LE BOM_UTF32_BEBOM_UTF8 BOM_UTF16_LE BOM_UTF16_BEr _null_null2_null3)dataZsampleZ nullcountr4 r4 r5 guess_json_utf s*
r c C s8 t | |\}}}}}}|s$|| }}t||||||fS )zGiven a URL that may or may not have a scheme, prepend the given scheme.
Does not replace a present scheme with the one provided as an argument.
:rtype: str
)r
r )r^ Z
new_schemer netlocrX r queryfragmentr4 r4 r5 prepend_scheme_if_needed s
r c C s@ t | }zt|jt|jf}W n ttfy: d}Y n0 |S )z{Given a url with authentication components, extract them into a tuple of
username,password.
:rtype: (str,str)
)r r )r
r
usernamepasswordr] r )r^ r Zauthr4 r4 r5 get_auth_from_url s
r s ^\S[^\r\n]*$|^$z^\S[^\r\n]*$|^$c C sd | \}}t |trt}nt}z||s4td| W n( ty^ td||t|f Y n0 dS )zVerifies that header value is a string which doesn't contain
leading whitespace or return characters. This prevents unintended
header injection.
:param header: tuple, in the format (name, value).
z7Invalid return character or leading space in header: %sz>Value for header {%s: %s} must be of type str or bytes, not %sN)rf r _CLEAN_HEADER_REGEX_BYTE_CLEAN_HEADER_REGEX_STRr0 r r type)r ra rz patr4 r4 r5 check_header_validity s
r c C sF t | \}}}}}}|s"|| }}|ddd }t|||||dfS )zW
Given a url remove the fragment and the authentication part.
:rtype: str
@r rc r )r
rsplitr )r^ r r rX r r r r4 r4 r5
urldefragauth s
r c C s^ t | jdd}|durRt| jtrRz|| j W qZ ttfyN tdY qZ0 ntddS )zfMove file pointer back to its recorded starting position
so it can be read again on redirect.
r@ Nz;An error occurred when rewinding request body for redirect.z+Unable to rewind request body for redirect.)re bodyrf Z_body_positionr rJ r, r )Zprepared_requestZ body_seekr4 r4 r5 rewind_body s r )F)F)r )N)r )e__doc__r
contextlibrB rD r/ r r sysro rH rl collectionsr r r r Z_internal_utilsr compatr r r r
r r r
r r r r r r r r r Zcookiesr Z
structuresr
exceptionsr r r r rW whereZDEFAULT_CA_BUNDLE_PATHZ
DEFAULT_PORTSplatformr6 r: rM r` ri ru r{ r} r r r r r r r r r r r frozensetr r r r r r r r r r contextmanagerr r r r r r r encoder r r r r r r r
r r r r r4 r4 r4 r5 s @
' =
3 #
%
=
&