B
/RoA @ sj d Z ddlZddlZddlZddlZddlZddlZddlZ ddl
m Z dddddd d
d d d
ddddddddgZ dZ
dZdZdZdZdZe ejj d d ejj D Zd Zd Zd Ze d!j Z e d"j Z e d#Z!e d$Z"d%d&d'hZ#dCd)d*Z$G d+d, d,ej%j&Z'd-d. Z(e'fd/d0Z)G d1d dej*Z+G d2d dZ,y ddl-Z-W n e.k
rd Y n X G d3d4 d4e,Z/e 0d4 G d5d de1Z2G d6d de2Z3G d7d d e2Z4G d8d de2Z5G d9d d e2Z6G d:d
d
e2Z7G d;d d e2Z8G dd de9Z;G d?d de9ZG dBd de?e=Z@e2ZAdS )Da
HTTP/1.1 client library
HTTPConnection goes through a number of "states", which define when a client
may legally make another request or fetch the response for a particular
request. This diagram details these state transitions:
(null)
|
| HTTPConnection()
v
Idle
|
| putrequest()
v
Request-started
|
| ( putheader() )* endheaders()
v
Request-sent
|\_____________________________
| | getresponse() raises
| response = getresponse() | ConnectionError
v v
Unread-response Idle
[Response-headers-read]
|\____________________
| |
| response.read() | putrequest()
v v
Idle Req-started-unread-response
______/|
/ |
response.read() | | ( putheader() )* endheaders()
v v
Request-started Req-sent-unread-response
|
| response.read()
v
Request-sent
This diagram presents the following rules:
-- a second request may not be started until {response-headers-read}
-- a response [object] cannot be retrieved until {request-sent}
-- there is no differentiation between an unread response body and a
partially read response body
Note: this enforcement is applied by the HTTPConnection class. The
HTTPResponse class does not enforce this state machine, which
implies sophisticated clients may accelerate the request/response
pipeline. Caution should be taken, though: accelerating the states
beyond the above pattern may imply knowledge of the server's
connection-close behavior for certain requests. For example, it
is impossible to tell whether the server will close the connection
UNTIL the response headers have been read; this means that further
requests cannot be placed into the pipeline until it is known that
the server will NOT be closing the connection.
Logical State __state __response
------------- ------- ----------
Idle _CS_IDLE None
Request-started _CS_REQ_STARTED None
Request-sent _CS_REQ_SENT None
Unread-response _CS_IDLE
Req-started-unread-response _CS_REQ_STARTED
Req-sent-unread-response _CS_REQ_SENT
N)urlsplit HTTPResponseHTTPConnection
HTTPException NotConnectedUnknownProtocolUnknownTransferEncodingUnimplementedFileModeIncompleteRead
InvalidURLImproperConnectionStateCannotSendRequestCannotSendHeaderResponseNotReady
BadStatusLine LineTooLongRemoteDisconnectederror responsesP i ZUNKNOWNZIdlezRequest-startedz Request-sentc C s i | ] }|j |qS )phrase).0vr r 0/opt/alt/python37/lib64/python3.7/http/client.py
j s r i i d s [^:\s][^:\r\n]*s \n(?![ \t])|\r(?![ \t\n])z[ - ]z[ - ]ZPATCHZPOSTZPUTdatac
C sf y
| dS tk
r` } z8t|j|j|j|jd| | |j|j |f dW dd}~X Y nX dS )z c C s, t | }d|d}tjj|d|S )aG Parses only RFC2822 headers from a file pointer.
email Parser wants to see strings rather than bytes.
But a TextIOWrapper around self.rfile would buffer too many bytes
from the stream, bytes which we later need to read as bytes.
So we read the correct bytes here, as bytes, for email Parser
to parse.
r8 z
iso-8859-1)_class)r> joindecodeemailparserZParserZparsestr)r< r? r= Zhstringr r r
parse_headers s
rD c s e Zd Zd@ddZdd Zdd Zd d
Zd d Z fd
dZ fddZ dd Z
dd Z dAddZ dd Z
dd Zdd Zd d Zd d Zd!d" Zd#d$ Zd%d& ZdBd(d)ZdCd*d+ZdD fd,d- Zd.d/ Zd0d1 Zd2d3 ZdEd4d5Zd6d7 Z d8d9 Z d:d; Z dd? Z Z!S )Fr r Nc C sR | d| _|| _|| _d | _| _t| _t| _t| _ t| _
t| _ t| _ t| _
d S )Nrb)Zmakefiler<
debuglevel_methodr= msg_UNKNOWNversionstatusreasonchunked
chunk_leftlength
will_close)r0 sockrF methodurlr r r __init__ s zHTTPResponse.__init__c C s t | jtd d}t|tkr*td| jdkrBtdt| |sNt dy|
d d\}}}W nF t k
r y|
d d\}}d}W n t k
r d}Y nX Y nX | d s|
t|y$t|}|d
k s|d krt|W n t k
r t|Y nX |||fS ) Nr* z
iso-8859-1z status liner zreply:z-Remote end closed connection without response zHTTP/r i )strr< r9 r: r, r rF printreprr split
ValueError
startswith _close_connr int)r0 r2 rJ rK rL r r r _read_status s2
zHTTPResponse._read_statusc C s | j d k rd S x<| \}}}|tkr(P t| j}| jdkrFtd| ~qW | | _| _| | _
|dkrrd| _ n| drd| _ nt
|t| j | _ | _| jdkrx&| j D ]\}}td|d | qW | j d }|r| d
krd | _d | _nd | _| | _d | _| j d
}| j d }|rv| jsvyt|| _W n tk
r` d | _Y nX | jdk r|d | _nd | _|tks|tksd| krdk sn | j dkrd| _| js| js| jd krd | _d S )Nr zheaders:)zHTTP/1.0zHTTP/0.9
zHTTP/1. zheader:r) ztransfer-encodingrM TFzcontent-lengthr HEAD) r= r_ ZCONTINUEr> r< rF rX coderK striprL rJ r\ r rD rH itemsgetr+ rM rN _check_closerP rO r^ r[ Z
NO_CONTENTZ NOT_MODIFIEDrG ) r0 rJ rK rL Zskipped_headershdrvalZtr_encrO r r r begin8 s\
zHTTPResponse.beginc C sv | j d}| jdkr.|r*d| kr*dS dS | j dr>dS |rRd| krRdS | j d}|rrd| krrdS dS )NZ
connectionra closeTFz
keep-alivezproxy-connection)r= rg rJ r+ )r0 ZconnZpconnr r r rh s
zHTTPResponse._check_closec C s | j }d | _ | d S )N)r< rl )r0 r< r r r r] s zHTTPResponse._close_connc s$ zt W d | jr | X d S )N)superrl r< r] )r0 ) __class__r r rl s zHTTPResponse.closec s t | jr| j d S )N)rm flushr< )r0 )rn r r ro s
zHTTPResponse.flushc C s dS )zAlways returns TrueTr )r0 r r r readable s zHTTPResponse.readablec C s
| j dkS )z!True if the connection is closed.N)r< )r0 r r r isclosed s zHTTPResponse.isclosedc C s | j d krdS | jdkr$| dS |d k rRt|}| |}t|d | S | jr`| S | j d krv| j
}n6y| | j }W n t k
r | Y nX d| _ | |S d S )Nr8 rc r )
r< rG r] bytearrayreadinto
memoryviewtobytesrM _readall_chunkedrO read
_safe_readr
)r0 amtbr1 sr r r rw s*
zHTTPResponse.readc C s | j dkrdS | jdkr$| dS | jr4| |S | jdk r^t|| jkr^t|d| j }| j |}|s||r|| n&| jdk r| j|8 _| js| |S )z^Read up to len(b) bytes into bytearray b and return the number
of bytes read.
Nr rc ) r< rG r] rM _readinto_chunkedrO r, rt rs )r0 rz r1 r r r rs s$
zHTTPResponse.readintoc C sp | j td }t|tkr$td|d}|dkrB|d | }y
t|dS tk
rj | Y nX d S )Nr* z
chunk size ;r ) r< r9 r: r, r findr^ r[ r] )r0 r2 ir r r _read_next_chunk_size s
z"HTTPResponse._read_next_chunk_sizec C s> x8| j td }t|tkr&td|s,P |dkrP qW d S )Nr* z trailer line)s
r7 r8 )r< r9 r: r, r )r0 r2 r r r _read_and_discard_trailer s z&HTTPResponse._read_and_discard_trailerc C sl | j }|sh|d k r | d y | }W n tk
rD tdY nX |dkrb| | d }|| _ |S )NrU r8 r )rN rx r r[ r
r r] )r0 rN r r r _get_chunk_left& s
z HTTPResponse._get_chunk_leftc C sd g }y8x,| }|d krP || | d| _qW d|S tk
r^ td|Y nX d S )Nr r8 )r r/ rx rN r@ r
)r0 valuerN r r r rv > s
z HTTPResponse._readall_chunkedc C s d}t |}yvxp| }|d kr$|S t||krL| |}|| | _|| S |d | }| |}||d }||7 }d| _qW W n( tk
r tt|d| Y nX d S )Nr )rt r r, _safe_readintorN r
bytes)r0 rz total_bytesmvbrN r1 temp_mvbr r r r| L s$
z HTTPResponse._readinto_chunkedc C sX g }xH|dkrL| j t|t}|s4td|||| |t|8 }qW d|S )aV Read the number of bytes requested, compensating for partial reads.
Normally, we have a blocking socket, but a read() can be interrupted
by a signal (resulting in a partial read).
Note that we cannot distinguish between EOF and an interrupt when zero
bytes have been read. IncompleteRead() will be raised in this
situation.
This function should be used when bytes "should" be present for
reading. If the bytes are truly not available (due to EOF), then the
IncompleteRead exception can be used to detect the problem.
r r8 )r< rw min MAXAMOUNTr
r@ r/ r, )r0 ry r{ chunkr r r rx d s
zHTTPResponse._safe_readc C s d}t |}xt|t|k rtt|k r@|dt }| j|}n | j|}|sjtt|d| t|||d }||7 }qW |S )z2Same as _safe_read, but for reading into a buffer.r N)rt r, r r< rs r
r )r0 rz r r r r1 r r r r { s zHTTPResponse._safe_readintoc C s | j dks| jdkrdS | jr(| |S | jdk rJ|dk sD|| jkrJ| j}| j |}|sh|rh| n | jdk r| jt|8 _|S )zvRead with at most one underlying system call. If at least one
byte is buffered, return that instead.
Nrc r8 r )r< rG rM _read1_chunkedrO read1r] r, )r0 r1 resultr r r r s
zHTTPResponse.read1c C s4 | j d ks| jdkrdS | jr(| |S | j |S )Nrc r8 )r< rG rM
_peek_chunkedpeek)r0 r1 r r r r s
zHTTPResponse.peekc s | j d ks| jdkrdS | jr*t |S | jd k rL|dk sF|| jkrL| j}| j |}|sj|rj| n | jd k r| jt|8 _|S )Nrc r8 r )r< rG rM rm r9 rO r] r, )r0 limitr )rn r r r9 s
zHTTPResponse.readlinec C sd | }|d ks|dkr dS d| kr0|ks6n |}| j|}| jt|8 _|s`td|S )Nr r8 )r r< r rN r, r
)r0 r1 rN rw r r r r s zHTTPResponse._read1_chunkedc C sB y | }W n tk
r dS X |d kr.dS | j|d | S )Nr8 )r r
r< r )r0 r1 rN r r r r s zHTTPResponse._peek_chunkedc C s
| j S )N)r< fileno)r0 r r r r s zHTTPResponse.filenoc C sF | j dkrt | j |p |}t|ts4t|ds8|S d|S dS )ax Returns the value of the header matching *name*.
If there are multiple matching headers, the values are
combined into a single string separated by commas and spaces.
If no matching header is found, returns *default* or None if
the *default* is not specified.
If the headers are unknown, raises http.client.ResponseNotReady.
N__iter__z, )r= r Zget_all
isinstancerW hasattrr@ )r0 r% defaultr= r r r getheader s
zHTTPResponse.getheaderc C s | j dkrt t| j S )z&Return list of (header, value) tuples.N)r= r listrf )r0 r r r
getheaders s
zHTTPResponse.getheadersc C s | S )Nr )r0 r r r r s zHTTPResponse.__iter__c C s | j S )aj Returns an instance of the class mimetools.Message containing
meta-information associated with the URL.
When the method is HTTP, these headers are those returned by
the server at the head of the retrieved HTML page (including
Content-Length and Content-Type).
When the method is FTP, a Content-Length header will be
present if (as is now usual) the server passed back a file
length in response to the FTP retrieval request. A
Content-Type header will be present if the MIME type can be
guessed.
When the method is local-file, returned headers will include
a Date representing the file's last-modified time, a
Content-Length giving file size, and a Content-Type
containing a guess at the file's type. See also the
description of the mimetools module.
)r= )r0 r r r info s zHTTPResponse.infoc C s | j S )aZ Return the real URL of the page.
In some cases, the HTTP server redirects a client to another
URL. The urlopen() function handles this transparently, but in
some cases the caller needs to know which URL the client was
redirected to. The geturl() method can be used to get at this
redirected URL.
)rS )r0 r r r geturl
s
zHTTPResponse.geturlc C s | j S )zuReturn the HTTP status code that was sent with the response,
or None if the URL is not an HTTP URL.
)rK )r0 r r r getcode s zHTTPResponse.getcode)r NN)N)r )r )r )N)"r4 r5 r6 rT r_ rk rh r] rl ro rp rq rw rs r r r rv r| rx r r r r9 r r r r r r r r r
__classcell__r r )rn r r s<
!K
"
c @ s
e Zd ZdZdZeZeZdZ dZ
e dd Z e dd Z
d ejd d
fd d Zd7d
dZdd Zdd Zdd Zdd Zdd Zdd Zdd Zd d Zd8d d!Zd9d"d#Zd$d% Z d&d' Z d(d) Z d*d+ Z d,d- Z d:d d.d/d0Z!d i fd d.d1d2Z"d3d4 Z#d5d6 Z$d S );r ra zHTTP/1.1r* r c C s t | tjS )zFTest whether a file-like object is a text or a binary stream.
)r io
TextIOBase)streamr r r
_is_textIO' s zHTTPConnection._is_textIOc C sd | dkr | tkrdS dS t| dr*dS yt| }|jS tk
rL Y nX t| tr`t| S dS )a Get the content-length based on the body.
If the body is None, we set Content-Length: 0 for methods that expect
a body (RFC 7230, Section 3.3.2). We also set the Content-Length for
any method if the body is a str or bytes-like object and not a file.
Nr rw ) upper_METHODS_EXPECTING_BODYr rt nbytes TypeErrorr rW r, )bodyrR Zmvr r r _get_content_length- s
z"HTTPConnection._get_content_lengthNi c C sn || _ || _|| _d | _g | _d | _t| _d | _d | _ d | _
i | _ | ||\| _
| _| | j
tj| _d S )N)timeoutsource_address blocksizerQ _buffer_HTTPConnection__response_CS_IDLE_HTTPConnection__staterG _tunnel_host _tunnel_port_tunnel_headers
_get_hostporthostport_validate_hostsocketZcreate_connection_create_connection)r0 r r r r r r r r rT M s zHTTPConnection.__init__c C s< | j rtd| ||\| _| _|r.|| _n
| j dS )aD Set up host and port for HTTP CONNECT tunnelling.
In a connection that uses HTTP CONNECT tunneling, the host passed to the
constructor is used as a proxy server that relays all communication to
the endpoint passed to `set_tunnel`. This done by sending an HTTP
CONNECT request to the proxy server when the connection is established.
This method must be called before the HTML connection has been
established.
The headers argument should be a mapping of extra HTTP headers to send
with the CONNECT request.
z.Can't set up tunnel for established connectionN)rQ RuntimeErrorr r r r clear)r0 r r r= r r r
set_tunnelc s zHTTPConnection.set_tunnelc C s |d kr| d}| d}||kryt||d d }W nH tk
r ||d d dkrh| j}ntd||d d Y nX |d | }n| j}|r|d dkr|d dkr|dd }||fS ) Nr) ]r* rV znonnumeric port: '%s'r [r )rfindr^ r[ default_portr )r0 r r r jr r r r { s
z HTTPConnection._get_hostportc C s
|| _ d S )N)rF )r0 levelr r r set_debuglevel s z HTTPConnection.set_debuglevelc C s d| j | jf }|d}| | x6| j D ](\}}d||f }|d}| | q0W | d | j| j| jd}| \}} }
| t
j j kr|
td| |
f xP|jtd } t| tkrtd | sP | d
krP | jd krtd | qW d S )
NzCONNECT %s:%d HTTP/1.0
asciiz%s: %s
zlatin-1s
)rR z Tunnel connection failed: %d %sr* z header line)s
r7 r8 r zheader:)r r r sendr rf response_classrQ rG r_ http
HTTPStatusZOKrl OSErrorre r< r9 r: r, r rF rX rA ) r0 Z connect_strZ
connect_bytesheaderr Z
header_strZ header_bytesresponserJ rd messager2 r r r _tunnel s2
zHTTPConnection._tunnelc C sB | | j| jf| j| j| _| jtjtj d | j
r>| dS )z3Connect to the host and port specified in __init__.r* N) r r r r r rQ Z
setsockoptr Z IPPROTO_TCPZ TCP_NODELAYr r )r0 r r r connect s
zHTTPConnection.connectc C sB t | _z | j}|r d| _| W d| j}|r s z/HTTPConnection._send_request..r r* r zaccept-encodingr zcontent-lengthztransfer-encodingFr z Unable to determine size of %rTzTransfer-EncodingrM zContent-Lengthr )r ) frozensetr r rF rX r rW rf r r' r ) r0 rR rS r r= r Z header_namesZskipsZcontent_lengthri r r r r r s0
z HTTPConnection._send_requestc C s | j r| j rd| _ | jtks&| j r0t| j| jdkrR| j| j| j| jd}n| j| j| jd}yLy | W n t
k
r | Y nX t | _|j
r| n|| _ |S | Y nX dS )a) Get the response from the server.
If the HTTPConnection is in the correct state, returns an
instance of HTTPResponse or of whatever object is returned by
the response_class variable.
If a request has not been sent or if a previous response has
not be handled, ResponseNotReady is raised. If the HTTP
response indicates that the connection should be closed, then
it will be closed before the response is returned. When the
connection is closed, the underlying socket is closed.
Nr )rR )r rq r r r rF r rQ rG rk ConnectionErrorrl r rP )r0 r r r r getresponse1 s,
zHTTPConnection.getresponse)NN)NF)FF)N)%r4 r5 r6 r r r r HTTP_PORTr r rF staticmethodr r r _GLOBAL_DEFAULT_TIMEOUTrT r r r r r rl r r r r r r r r r r r r r r r r r r r s@
&
6
~
.c sH e Zd ZdZeZdddejdfdddd fdd Z fddZ Z
S ) HTTPSConnectionz(This class allows communication via SSL.Ni )contextcheck_hostnamer c s t t| j||||| d |d k s2|d k s2|d k rHdd l}
|
dtd || _|| _|d krtt }|j
d k rtd|_
|j tj k} |d kr|j
}|r| std|s|r||| |j
d k rd|_
|| _|d k r|| j_
d S )N)r r zTkey_file, cert_file and check_hostname are deprecated, use a custom context instead.rU TzMcheck_hostname needs a SSL context with either CERT_OPTIONAL or CERT_REQUIRED)rm r rT warningswarnDeprecationWarningkey_file cert_filesslZ _create_default_https_contextZpost_handshake_authZ verify_modeZ CERT_NONEr r[ Zload_cert_chain_context) r0 r r r r r r r r r r Z will_verify)rn r r rT | s4
zHTTPSConnection.__init__c s6 t | jr| j}n| j}| jj| j|d| _dS )z(Connect to a host on a given (SSL) port.)server_hostnameN)rm r r r r
Z wrap_socketrQ )r0 r )rn r r r s
zHTTPSConnection.connect) r4 r5 r6 __doc__
HTTPS_PORTr r r rT r r r r )rn r r u s !r c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd Zdd ZdS )r c C s |f| _ || _d S )N)argsrJ )r0 rJ r r r rT s zUnknownProtocol.__init__N)r4 r5 r6 rT r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s& e Zd ZdddZdd Zdd ZdS ) r
Nc C s |f| _ || _|| _d S )N)r partialexpected)r0 r r r r r rT s zIncompleteRead.__init__c C s2 | j d k rd| j }nd}d| jjt| j|f S )Nz, %i more expectedrV z%s(%i bytes read%s))r rn r4 r, r )r0 er r r __repr__ s
zIncompleteRead.__repr__c C s t | S )N)rY )r0 r r r __str__ s zIncompleteRead.__str__)N)r4 r5 r6 rT r r r r r r r
s
c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r
N)r4 r5 r6 r r r r r
s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd Zdd ZdS )r c C s |s t |}|f| _|| _d S )N)rY r r2 )r0 r2 r r r rT s zBadStatusLine.__init__N)r4 r5 r6 rT r r r r r s c @ s e Zd Zdd ZdS )r c C s t | dt|f d S )Nz&got more than %d bytes when reading %s)r rT r: )r0 Z line_typer r r rT s zLineTooLong.__init__N)r4 r5 r6 rT r r r r r s c @ s e Zd Zdd ZdS )r c O s" t | d tj| f|| d S )NrV )r rT ConnectionResetError)r0 poskwr r r rT s zRemoteDisconnected.__init__N)r4 r5 r6 rT r r r r r s )r )Br Z email.parserrB Z
email.messager r rer Zcollections.abcr Z urllib.parser __all__r r
rI r r r globalsupdater __members__r r r r: r; compile fullmatchr r r r r r r' r ZMessager( r> rD BufferedIOBaser r r ImportErrorr r/ Exceptionr r r r r r r
r r
r r r r r r r r r r r E s
1 W 8