a
lj, @ s d dl mZmZmZ d dlmZ d dlmZm Z d dl
m Z ddl m
Z
ddl mZ ddl mZmZ dd l mZmZmZ dd
l mZmZ dd l mZ dd lmZ dd
lmZ eeZe dkre Z ne Z G dd de Z dS ) )absolute_importdivisionunicode_literals)unichr)deque OrderedDict) version_info )spaceCharacters)entities) asciiLettersasciiUpper2Lower)digits hexDigitsEOF)
tokenTypes
tagTokenTypes)replacementCharacters)HTMLInputStream)Trie) c sd e Zd ZdZd fdd Zdd Zdd Zdd
d Zd d
Zdd Z dd Z
dd Z dd Z dd Z
dd Zdd Zd d Zd d Zd d! Zd"d# Zd$d% Zd&d' Zd(d) Zd*d+ Zd,d- Zd.d/ Zd0d1 Zd2d3 Zd4d5 Z d6d7 Z d8d9 Z d:d; Z dd? Z!d@dA Z"dBdC Z#dDdE Z$dFdG Z%dHdI Z&dJdK Z'dLdM Z(dNdO Z)dPdQ Z*dRdS Z+dTdU Z,dVdW Z-dXdY Z.dZd[ Z/d\d] Z0d^d_ Z1d`da Z2dbdc Z3ddde Z4dfdg Z5dhdi Z6djdk Z7dldm Z8dndo Z9dpdq Z:drds Z;dtdu Zdzd{ Z?d|d} Z@d~d ZAdd ZBdd ZCdd ZDdd ZEdd ZFdd ZGdd ZHdd ZIdd ZJdd ZKdd ZL ZMS )
HTMLTokenizera This class takes care of tokenizing HTML.
* self.currentToken
Holds the token that is currently being processed.
* self.state
Holds a reference to the method to be invoked... XXX
* self.stream
Points to HTMLInputStream object.
Nc sJ t |fi || _|| _d| _g | _| j| _d| _d | _t t
| d S )NF) r streamparserZ
escapeFlagZ
lastFourChars dataStatestateescape currentTokensuperr __init__)selfr r kwargs __class__ C/usr/lib/python3.9/site-packages/pip/_vendor/html5lib/_tokenizer.pyr ( s zHTMLTokenizer.__init__c c sP t g | _| rL| jjr6td | jjddV q| jr
| j V q6q
dS )z This is where the magic happens.
We do our usually processing through the states and when we have a token
to return we yield the token which pauses processing until the next token
is requested.
ParseErrorr typedataN)r
tokenQueuer r errorsr poppopleftr! r% r% r& __iter__7 s
zHTMLTokenizer.__iter__c C s t }d}|rt}d}g }| j }||v rH|turH|| | j }q"td||}|tv rt| }| j t
d dd|id n d| krd ksn |d
krd }| j t
d dd|id nd | krd
ks>n d| krdks>n d| krdks>n d| kr*dks>n |t g dv rZ| j t
d dd|id z t |}W n< t
y |d }t d|d? B t d|d@ B }Y n0 |dkr| j t
d dd | j| |S )zThis function returns either U+FFFD or the character based on the
decimal or hexadecimal representation. It also discards ";" if present.
If not present self.tokenQueue.append({"type": tokenTypes["ParseError"]}) is invoked.
r' z$illegal-codepoint-for-numeric-entity charAsIntr) r* Zdatavarsi i �r i i )# i i i i i i i i i i i i i i i i i i i i i
i
i i i i i
i
i i i i i r6 i i i ;z numeric-entity-without-semicolonr( )r r r charr appendintjoinr r+ r frozensetchr
ValueErrorunget) r! ZisHexallowedradix charStackcr4 r? vr% r% r& consumeNumberEntityG sn
&
z!HTMLTokenizer.consumeNumberEntityFc C s d}| j g}|d tv sB|d tddfv sB|d urV||d krV| j |d n |d dkr d}|| j |d dv rd}|| j |r|d tv s|s|d tv r| j |d | |}n4| j t
d d
d | j | dd | }nf|d turDt
d |s0qD|| j q z$t
d |d d }t|}W n ty d }Y n0 |d ur>|d d
kr| j t
d dd |d d
kr|r|| tv s|| tv s|| dkr| j | dd | }n.t| }| j | |d ||d 7 }n4| j t
d dd | j | dd | }|r| jd d d |7 < n*|tv rd}nd}| j t
| |d d S )N&r <#F)xXTr' zexpected-numeric-entityr( r3 r> z named-entity-without-semicolon=zexpected-named-entityr* r SpaceCharacters
Characters)r r? r
r rF r@ r r rL r+ r r- rB entitiesTrieZhas_keys_with_prefixZlongest_prefixlenKeyErrorr r r ) r! allowedChar
fromAttributeoutputrI hexZ
entityNameZ entityLengthZ tokenTyper% r% r&
consumeEntity s~
zHTMLTokenizer.consumeEntityc C s | j |dd dS )zIThis method replaces the need for "entityInAttributeValueState".
T)rY rZ N)r] )r! rY r% r% r& processEntityInAttribute s z&HTMLTokenizer.processEntityInAttributec C s | j }|d tv r|d t|d< |d td krp|d }t|}t|t|krh||ddd ||d< |d td kr|d r| j td d d
|d r| j td d d
| j | | j
| _ dS )
zThis method is a generic handler for emitting the tags. It also sets
the state to "data" because that's what's needed after a token has been
emitted.
r) nameStartTagr* NrP EndTagr' zattributes-in-end-tagr( selfClosingz self-closing-flag-on-end-tag) r r translater
r attributeMaprW updater+ r@ r r )r! tokenrawr* r% r% r& emitCurrentToken s( z HTMLTokenizer.emitCurrentTokenc C s | j }|dkr | j| _n|dkr.| j| _n|dkrd| jtd dd | jtd dd n`|tu rpdS |t v r| jtd || j
t d
d n&| j
d }| jtd || d d
S ) NrM rN r' invalid-codepointr( rU FrT TrM rN ri ) r r? entityDataStater tagOpenStater+ r@ r r r
charsUntilr! r* charsr% r% r& r s.
zHTMLTokenizer.dataStatec C s | | j| _dS NT)r] r r r/ r% r% r& rl s z HTMLTokenizer.entityDataStatec C s | j }|dkr | j| _n|dkr.| j| _n|tkr:dS |dkrp| jtd dd | jtd d d nT|t v r| jtd
|| j
t d d n&| j
d }| jtd || d d S )
NrM rN Fri r' rj r( rU r7 rT Trk ) r r? characterReferenceInRcdatar rcdataLessThanSignStater r+ r@ r r
rn ro r% r% r& rcdataState" s.
zHTMLTokenizer.rcdataStatec C s | | j| _dS rq )r] rt r r/ r% r% r& rr ? s z(HTMLTokenizer.characterReferenceInRcdatac C s | j }|dkr | j| _nh|dkrR| jtd dd | jtd dd n2|tkr^dS | j d }| jtd || d d
S NrN ri r' rj r( rU r7 F)rN ri T) r r? rawtextLessThanSignStater r+ r@ r r rn ro r% r% r& rawtextStateD s"
zHTMLTokenizer.rawtextStatec C s | j }|dkr | j| _nh|dkrR| jtd dd | jtd dd n2|tkr^dS | j d }| jtd || d d
S ru ) r r? scriptDataLessThanSignStater r+ r@ r r rn ro r% r% r& scriptDataStateV s"
z HTMLTokenizer.scriptDataStatec C sr | j }|tkrdS |dkrL| jtd dd | jtd dd n"| jtd || j d d dS ) NFri r' rj r( rU r7 T)r r? r r+ r@ r rn r! r* r% r% r& plaintextStateh s
z HTMLTokenizer.plaintextStatec C s | j }|dkr | j| _n|dkr.| j| _n|tv rVtd |g ddd| _| j| _n|dkr| j
td dd | j
td
d d | j | _nt|d kr| j
td d
d | j | | j
| _n@| j
td dd | j
td
dd | j | | j | _dS )N!/r` F)r) r_ r* rb ZselfClosingAcknowledged>r' z'expected-tag-name-but-got-right-bracketr( rU z<>?z'expected-tag-name-but-got-question-markzexpected-tag-namerN T)r r? markupDeclarationOpenStater closeTagOpenStater r r tagNameStater+ r@ r rF bogusCommentStaterz r% r% r& rm w s>
zHTMLTokenizer.tagOpenStatec C s | j }|tv r0td |g dd| _| j| _n|dkrX| jtd dd | j | _nn|t
u r| jtd dd | jtd d
d | j | _n0| jtd d d |id
| j | | j | _dS )Nra Fr) r_ r* rb r~ r' z*expected-closing-tag-but-got-right-bracketr( z expected-closing-tag-but-got-eofrU |tkr| jtd d d | j | _n| jtd |d d
S Nr rU r( rN ri r' rj r7 eof-in-script-in-scriptT)
r r? r+ r@ r scriptDataDoubleEscapedDashStater (scriptDataDoubleEscapedLessThanSignStater r rz r% r% r& r s*
z*HTMLTokenizer.scriptDataDoubleEscapedStatec C s | j }|dkr2| jtd dd | j| _n|dkrZ| jtd dd | j| _n|dkr| jtd dd | jtd dd | j| _nF|t kr| jtd d d | j
| _n | jtd |d | j| _d
S r ) r r? r+ r@ r $scriptDataDoubleEscapedDashDashStater r r r r rz r% r% r& r s.
z.HTMLTokenizer.scriptDataDoubleEscapedDashStatec C s | j }|dkr*| jtd dd n|dkrR| jtd dd | j| _n|dkrz| jtd dd | j| _n|dkr| jtd dd | jtd d d | j| _nF|t kr| jtd d
d | j
| _n | jtd |d | j| _d S ) Nr rU r( rN r~ ri r' rj r7 r T) r r? r+ r@ r r r ry r r r rz r% r% r& r % s2
z2HTMLTokenizer.scriptDataDoubleEscapedDashDashStatec C sP | j }|dkr8| jtd dd d| _| j| _n| j | | j | _dS )Nr} rU r( r3 T)
r r? r+ r@ r r scriptDataDoubleEscapeEndStater rF r rz r% r% r& r > s
z6HTMLTokenizer.scriptDataDoubleEscapedLessThanSignStatec C s | j }|ttdB v rR| jtd |d | j dkrH| j | _
q| j | _
nB|t v r| jtd |d | j|7 _n| j
| | j | _
dS r )r r? r
rC r+ r@ r r r r r r r rF rz r% r% r& r I s
z,HTMLTokenizer.scriptDataDoubleEscapeEndStatec C s0 | j }|tv r$| j td n|tv rJ| jd |dg | j| _n|dkr\| n|dkrn| j
| _n|dv r| j t d dd | jd |dg | j| _n|d
kr| j t d d d | jd d dg | j| _nF|t
u r| j t d d
d | j| _n | jd |dg | j| _dS )NTr* r3 r~ r} )'"rS rN r' #invalid-character-in-attribute-namer( ri rj r7 z#expected-attribute-name-but-got-eof)r r? r
rn r r r@ attributeNameStater rh r r+ r r r rz r% r% r& r Y s<
z&HTMLTokenizer.beforeAttributeNameStatec C s | j }d}d}|dkr&| j| _n.|tv r\| jd d d || j td 7 < d}n|dkrjd}n|tv r|| j| _n|dkr| j | _n|d kr| j
t d
d d | jd d d d
7 < d}n|dv r | j
t d
dd | jd d d |7 < d}nH|t
u r6| j
t d
dd | j| _n | jd d d |7 < d}|r| jd d d t| jd d d< | jd d d D ]>\}}| jd d d |kr| j
t d
dd qҐq|r| dS )NTFrS r* rP r r~ r} ri r' rj r( r7 r r rN r zeof-in-attribute-namezduplicate-attribute)r r? beforeAttributeValueStater r r rn r
afterAttributeNameStater r+ r@ r r r rc r
rh )r! r* ZleavingThisStateZ emitTokenr_ _r% r% r& r w s^
z HTMLTokenizer.attributeNameStatec C sD | j }|tv r$| j td n |dkr8| j| _n|dkrJ| n|tv rp| jd |dg | j
| _n|dkr| j | _n|dkr| j t
d d d
| jd d dg | j
| _n|d v r| j t
d d
d
| jd |dg | j
| _nF|tu r$| j t
d dd
| j| _n | jd |dg | j
| _dS )NTrS r~ r* r3 r} ri r' rj r( r7 r z&invalid-character-after-attribute-namez expected-end-of-tag-but-got-eof)r r? r
rn r r rh r r r@ r r r+ r r r rz r% r% r& r s@
z%HTMLTokenizer.afterAttributeNameStatec C sh | j }|tv r$| j td n@|dkr8| j| _n,|dkrX| j| _| j | n |dkrj| j| _n|dkr| j
t d dd | n|d kr| j
t d d
d | j
d d d
d7 < | j| _n|dv r| j
t d dd | j
d d d
|7 < | j| _nL|tu rB| j
t d dd | j| _n"| j
d d d
|7 < | j| _dS )NTr rM r r~ r' z.expected-attribute-value-but-got-right-bracketr( ri rj r* rP r r7 )rS rN `z"equals-in-unquoted-attribute-valuez$expected-attribute-value-but-got-eof)r r? r
rn attributeValueDoubleQuotedStater attributeValueUnQuotedStaterF attributeValueSingleQuotedStater+ r@ r rh r r r rz r% r% r& r sF
z'HTMLTokenizer.beforeAttributeValueStatec C s | j }|dkr | j| _n|dkr0| d n|dkrj| jtd dd | jd d d d
7 < nN|t u r| jtd d d | j
| _n&| jd d d || j d 7 < d
S )Nr rM ri r' rj r( r* rP r r7 z#eof-in-attribute-value-double-quote)r rM ri T r r? afterAttributeValueStater r^ r+ r@ r r r r rn rz r% r% r& r s&
z-HTMLTokenizer.attributeValueDoubleQuotedStatec C s | j }|dkr | j| _n|dkr0| d n|dkrj| jtd dd | jd d d d
7 < nN|t u r| jtd d d | j
| _n&| jd d d || j d 7 < d
S )Nr rM ri r' rj r( r* rP r r7 z#eof-in-attribute-value-single-quote)r rM ri Tr rz r% r% r& r s&
z-HTMLTokenizer.attributeValueSingleQuotedStatec C s | j }|tv r | j| _n|dkr0| d n|dkrB| n|dv r|| jt d dd | j
d d d |7 < n|d
kr| jt d d d | j
d d d d 7 < nV|t u r| jt d d
d | j | _n.| j
d d d || j
tdtB 7 < dS )NrM r~ )r r rS rN r r' z0unexpected-character-in-unquoted-attribute-valuer( r* rP r ri rj r7 z eof-in-attribute-value-no-quotes)rM r~ r r rS rN r ri T)r r? r
r r r^ rh r+ r@ r r r r rn rC rz r% r% r& r s4
z)HTMLTokenizer.attributeValueUnQuotedStatec C s | j }|tv r | j| _n|dkr.| np|dkr@| j| _n^|tu rt| j t
d dd | j | | j | _n*| j t
d dd | j | | j| _dS )Nr~ r} r' z$unexpected-EOF-after-attribute-valuer( z*unexpected-character-after-attribute-valueT)
r r? r
r r rh r r r+ r@ r rF r rz r% r% r& r . s&
z&HTMLTokenizer.afterAttributeValueStatec C s | j }|dkr&d| jd< | n^|tu rZ| jtd dd | j | | j | _
n*| jtd dd | j | | j | _
dS )Nr~ Trb r' z#unexpected-EOF-after-solidus-in-tagr( z)unexpected-character-after-solidus-in-tag) r r? r rh r r+ r@ r rF r r r rz r% r% r& r B s
z&HTMLTokenizer.selfClosingStartTagStatec C sD | j d}|dd}| jtd |d | j | j| _dS )Nr~ ri r7 Commentr( T) r rn replacer+ r@ r r? r r rz r% r% r& r T s
z HTMLTokenizer.bogusCommentStatec C s | j g}|d dkrR|| j |d dkrPtd dd| _| j| _dS n|d dv rd}dD ](}|| j |d |vrfd } qqf|rtd
dd d dd | _| j| _dS n|d d krD| jd urD| jj j
rD| jj j
d j | jj j krDd}d
D ].}|| j |d |krd } q2q|rD| j
| _dS | jtd dd |rt| j | qZ| j| _dS )NrP r r r3 r( T)dD))oOrJ CtTyYpPeEFZDoctype)r) r_ publicIdsystemIdcorrect[)r r Ar r r r' zexpected-dashes-or-doctype)r r? r@ r r commentStartStater doctypeStater ZtreeZ openElements namespaceZdefaultNamespacecdataSectionStater+ rF r- r )r! rI matchedexpectedr% r% r& r c sZ
z(HTMLTokenizer.markupDeclarationOpenStatec C s | j }|dkr | j| _n|dkrN| jtd dd | jd d7 < n|dkr| jtd d d | j| j | j| _nP|t u r| jtd d
d | j| j | j| _n| jd |7 < | j
| _d S ) Nr ri r' rj r( r* r7 r~ incorrect-commenteof-in-commentT) r r? commentStartDashStater r+ r@ r r r r commentStaterz r% r% r& r s.
z HTMLTokenizer.commentStartStatec C s | j }|dkr | j| _n|dkrN| jtd dd | jd d7 < n|dkr| jtd d d | j| j | j| _nT|t u r| jtd d
d | j| j | j| _n | jd d| 7 < | j
| _d S ) Nr ri r' rj r( r* -�r~ r r T) r r? commentEndStater r+ r@ r r r r r rz r% r% r& r s.
z#HTMLTokenizer.commentStartDashStatec C s | j }|dkr | j| _n|dkrN| jtd dd | jd d7 < nT|tu r| jtd dd | j| j | j | _n | jd || j
d 7 < d
S ) Nr ri r' rj r( r* r7 r )r ri T) r r? commentEndDashStater r+ r@ r r r r rn rz r% r% r& r s$
zHTMLTokenizer.commentStatec C s | j }|dkr | j| _n|dkrV| jtd dd | jd d7 < | j| _nT|t u r| jtd dd | j| j | j
| _n | jd d| 7 < | j| _d S )
Nr ri r' rj r( r* r zeof-in-comment-end-dashT) r r? r r r+ r@ r r r r r rz r% r% r& r s$
z!HTMLTokenizer.commentEndDashStatec C s, | j }|dkr*| j| j | j| _n|dkrd| jtd dd | jd d7 < | j| _n|dkr| jtd d d | j | _n|d
kr| jtd d d | jd |7 < nj|t
u r| jtd d d | j| j | j| _n4| jtd d
d | jd d| 7 < | j| _dS )Nr~ ri r' rj r( r* u --�r| z,unexpected-bang-after-double-dash-in-commentr z,unexpected-dash-after-double-dash-in-commentzeof-in-comment-double-dashzunexpected-char-in-commentz--T) r r? r+ r@ r r r r r commentEndBangStater rz r% r% r& r s@
z HTMLTokenizer.commentEndStatec C s | j }|dkr*| j| j | j| _n|dkrN| jd d7 < | j| _n|dkr| jtd dd | jd d 7 < | j | _nT|t
u r| jtd d
d | j| j | j| _n | jd d| 7 < | j | _d S ) Nr~ r r* z--!ri r' rj r( u --!�z eof-in-comment-end-bang-stateT) r r? r+ r@ r r r r r r r rz r% r% r& r s,
z!HTMLTokenizer.commentEndBangStatec C s | j }|tv r | j| _nj|tu r\| jtd dd d| j d< | j| j | j
| _n*| jtd dd | j | | j| _dS )Nr' !expected-doctype-name-but-got-eofr( Fr zneed-space-after-doctypeT) r r? r
beforeDoctypeNameStater r r+ r@ r r r rF rz r% r% r& r s
zHTMLTokenizer.doctypeStatec C s | j }|tv rn|dkrT| jtd dd d| jd< | j| j | j| _n|dkr| jtd dd d | jd
< | j | _nR|t
u r| jtd d d d| jd< | j| j | j| _n|| jd
< | j | _d S )
Nr~ r' z+expected-doctype-name-but-got-right-bracketr( Fr ri rj r7 r_ r T) r r? r
r+ r@ r r r r doctypeNameStater rz r% r% r& r * s4
z$HTMLTokenizer.beforeDoctypeNameStatec C s | j }|tv r2| jd t| jd< | j| _n|dkrh| jd t| jd< | j | j | j
| _n|dkr| j t d dd | jd d7 < | j | _nh|t
u r| j t d dd d | jd
< | jd t| jd< | j | j | j
| _n| jd |7 < d S ) Nr_ r~ ri r' rj r( r7 zeof-in-doctype-nameFr T)r r? r
r rc r
afterDoctypeNameStater r+ r@ r r r r rz r% r% r& r D s0
z HTMLTokenizer.doctypeNameStatec C sH | j }|tv rn.|dkr8| j| j | j| _n |tu rd| jd< | j | | jt
d dd | j| j | j| _n|dv rd}d D ] }| j }||vrd} qq|r| j | _dS nD|d
v r
d}d D ] }| j }||vrd} qq|r
| j | _dS | j | | jt
d d d
|id d| jd< | j
| _dS )Nr~ Fr r' eof-in-doctyper( r T))uU)bB)lL)iIr sS)r r r r )mMz*expected-space-or-right-bracket-in-doctyper* r5 )r r? r
r+ r@ r r r r rF r afterDoctypePublicKeywordState afterDoctypeSystemKeywordStatebogusDoctypeState)r! r* r r r% r% r& r ] sT
z#HTMLTokenizer.afterDoctypeNameStatec C s | j }|tv r | j| _n|dv rP| jtd dd | j | | j| _nT|t u r| jtd dd d| j
d< | j| j
| j | _n| j | | j| _dS N)r r r' unexpected-char-in-doctyper( r Fr T) r r? r
"beforeDoctypePublicIdentifierStater r+ r@ r rF r r r rz r% r% r& r s&
z,HTMLTokenizer.afterDoctypePublicKeywordStatec C s | j }|tv rn|dkr0d| jd< | j| _n|dkrLd| jd< | j| _n|dkr| jt d dd d | jd
< | j| j | j
| _nh|t u r| jt d d d d | jd
< | j| j | j
| _n(| jt d d d d | jd
< | j | _d
S )Nr r3 r r r~ r' unexpected-end-of-doctyper( Fr r r T)
r r? r
r (doctypePublicIdentifierDoubleQuotedStater (doctypePublicIdentifierSingleQuotedStater+ r@ r r r r rz r% r% r& r s:
z0HTMLTokenizer.beforeDoctypePublicIdentifierStatec C s | j }|dkr | j| _n|dkrN| jtd dd | jd d7 < n|dkr| jtd d d d
| jd < | j| j | j| _nR|t u r| jtd d d d
| jd < | j| j | j| _n| jd |7 < d
S )Nr ri r' rj r( r r7 r~ r Fr r T
r r? !afterDoctypePublicIdentifierStater r+ r@ r r r r rz r% r% r& r s0
z6HTMLTokenizer.doctypePublicIdentifierDoubleQuotedStatec C s | j }|dkr | j| _n|dkrN| jtd dd | jd d7 < n|dkr| jtd d d d
| jd < | j| j | j| _nR|t u r| jtd d d d
| jd < | j| j | j| _n| jd |7 < d
S )Nr ri r' rj r( r r7 r~ r Fr r Tr rz r% r% r& r s0
z6HTMLTokenizer.doctypePublicIdentifierSingleQuotedStatec C s | j }|tv r | j| _n|dkr<| j| j | j| _n|dkrn| jt d dd d| jd< | j
| _n|dkr| jt d dd d| jd< | j | _nh|t u r| jt d d d d
| jd < | j| j | j| _n(| jt d dd d
| jd < | j
| _d S )
Nr~ r r' r r( r3 r r r Fr T)r r? r
-betweenDoctypePublicAndSystemIdentifiersStater r+ r@ r r r (doctypeSystemIdentifierDoubleQuotedState(doctypeSystemIdentifierSingleQuotedStater r rz r% r% r& r s>
z/HTMLTokenizer.afterDoctypePublicIdentifierStatec C s | j }|tv rn|dkr4| j| j | j| _n|dkrPd| jd< | j| _n|dkrld| jd< | j | _nh|t
kr| jt d dd d | jd
< | j| j | j| _n(| jt d d d d | jd
< | j | _d S )
Nr~ r r3 r r r' r r( Fr r T)
r r? r
r+ r@ r r r r r r r r rz r% r% r& r s2
z;HTMLTokenizer.betweenDoctypePublicAndSystemIdentifiersStatec C s | j }|tv r | j| _n|dv rP| jtd dd | j | | j| _nT|t u r| jtd dd d| j
d< | j| j
| j | _n| j | | j| _dS r ) r r? r
"beforeDoctypeSystemIdentifierStater r+ r@ r rF r r r rz r% r% r& r ) s&
z,HTMLTokenizer.afterDoctypeSystemKeywordStatec C s | j }|tv rn|dkr0d| jd< | j| _n|dkrLd| jd< | j| _n|dkr| jt d dd d | jd
< | j| j | j
| _nh|t u r| jt d d d d | jd
< | j| j | j
| _n(| jt d dd d | jd
< | j | _d S )
Nr r3 r r r~ r' r r( Fr r T)
r r? r
r r r r r+ r@ r r r r rz r% r% r& r = s:
z0HTMLTokenizer.beforeDoctypeSystemIdentifierStatec C s | j }|dkr | j| _n|dkrN| jtd dd | jd d7 < n|dkr| jtd d d d
| jd < | j| j | j| _nR|t u r| jtd d d d
| jd < | j| j | j| _n| jd |7 < d
S )Nr ri r' rj r( r r7 r~ r Fr r T
r r? !afterDoctypeSystemIdentifierStater r+ r@ r r r r rz r% r% r& r Z s0
z6HTMLTokenizer.doctypeSystemIdentifierDoubleQuotedStatec C s | j }|dkr | j| _n|dkrN| jtd dd | jd d7 < n|dkr| jtd d d d
| jd < | j| j | j| _nR|t u r| jtd d d d
| jd < | j| j | j| _n| jd |7 < d
S )Nr ri r' rj r( r r7 r~ r Fr r Tr rz r% r% r& r r s0
z6HTMLTokenizer.doctypeSystemIdentifierSingleQuotedStatec C s | j }|tv rn~|dkr4| j| j | j| _n^|tu rt| jt d dd d| jd< | j| j | j| _n | jt d dd | j
| _dS ) Nr~ r' r r( Fr r T) r r? r
r+ r@ r r r r r r rz r% r% r& r s$
z/HTMLTokenizer.afterDoctypeSystemIdentifierStatec C sZ | j }|dkr*| j| j | j| _n,|tu rV| j | | j| j | j| _n dS )Nr~ T) r r? r+ r@ r r r r rF rz r% r% r& r s
z HTMLTokenizer.bogusDoctypeStatec C s g }| | jd | | jd | j }|tkr>qvq|d dd dkrj|d d d |d< qvq| | qd|}|d}|dkrt|D ]}| j t d d
d q|
dd }|r| j t d
|d | j | _ dS )N]r~ rP z]]r3 ri r r' rj r( r7 rU T)
r@ r rn r? r rB countranger+ r r r r )r! r* r? Z nullCountr r% r% r& r s0
z HTMLTokenizer.cdataSectionState)N)NF)N__name__
__module__ __qualname____doc__r r0 rL r] r^ rh r rl rt rr rw ry r{ rm r r rs r r rv r r rx r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r
__classcell__r% r% r# r& r s H
P #
6 "- 3 r N) Z
__future__r r r Zpip._vendor.sixr rD collectionsr r sysr Z constantsr
r r r
r r r r r r Z _inputstreamr Z_trier rV dictrd objectr r% r% r% r& s