UJ,^K d dl mZmZmZ d dlmZ d dlmZm Z d dl
m Z ddl m
Z
ddl mZ ddl mZmZ dd l mZmZmZ dd
l mZmZ dd l mZ dd lmZ dd
lmZ ee Ze dk re Z ne Z G d de Z dS ) )absolute_importdivisionunicode_literals)unichr)deque OrderedDict) version_info )spaceCharacters)entities) asciiLettersasciiUpper2Lower)digits hexDigitsEOF)
tokenTypes
tagTokenTypes)replacementCharacters)HTMLInputStream)Trie) c e Zd ZdZdM fd Zd Zd ZdNdZd Zd Z d
Z
d Z d Z d
Z
d Zd Zd Zd Zd Zd Zd Zd Zd Zd Zd Zd Zd Zd Zd Z d Z d Z d Z d Z d! Z!d" Z"d# Z#d$ Z$d% Z%d& Z&d' Z'd( Z(d) Z)d* Z*d+ Z+d, Z,d- Z-d. Z.d/ Z/d0 Z0d1 Z1d2 Z2d3 Z3d4 Z4d5 Z5d6 Z6d7 Z7d8 Z8d9 Z9d: Z:d; Z;d< Z Z>d? Z?d@ Z@dA ZAdB ZBdC ZCdD ZDdE ZEdF ZFdG ZGdH ZHdI ZIdJ ZJdK ZKdL ZL xZMS )O
HTMLTokenizera This class takes care of tokenizing HTML.
* self.currentToken
Holds the token that is currently being processed.
* self.state
Holds a reference to the method to be invoked... XXX
* self.stream
Points to HTMLInputStream object.
Nc t |fi || _ || _ d| _ g | _ | j | _ d| _ d | _ t t | d S NF) r streamparser
escapeFlag
lastFourChars dataStatestateescape currentTokensuperr __init__)selfr r kwargs __class__s /builddir/build/BUILDROOT/alt-python311-pip-21.3.1-4.el9.x86_64/opt/alt/python311/lib/python3.11/site-packages/pip/_vendor/html5lib/_tokenizer.pyr& zHTMLTokenizer.__init__( sn %f7777 ^
!
mT""++----- c # f K t g | _ | r| j j r;t
d | j j d dV | j j ;| j r"| j V | j "| dS dS )z This is where the magic happens.
We do our usually processing through the states and when we have a token
to return we yield the token which pauses processing until the next token
is requested.
ParseErrorr typedataN)r
tokenQueuer" r errorsr poppopleftr' s r* __iter__zHTMLTokenizer.__iter__7 s )) jjll 0+$
\ ), 7ASAWAWXYAZAZ[[[[[ +$
\/
0o--///// /
0 jjll 0 0 0 0 0r+ c l t }d}|r t }d}g }| j }||v rD|t ur;| | | j }||v r |t u;t
d | | }|t v r:t | }| j t d dd|id nd|cxk rd k sn |d
k r.d }| j t d dd|id nd |cxk rd
k sBn d|cxk rdk s3n d|cxk rdk s$n d|cxk rdk sn |t g d v r+| j t d dd|id t | }n@# t $ r3 |dz
}t d|dz z t d|dz z z }Y nw xY w|dk rB| j t d dd | j | |S )zThis function returns either U+FFFD or the character based on the
decimal or hexadecimal representation. It also discards ";" if present.
If not present self.tokenQueue.append({"type": tokenTypes["ParseError"]}) is invoked.
r- z$illegal-codepoint-for-numeric-entity charAsIntr/ r0 datavarsi i �r
i i )# i i i i i i i i i i i i i i i i i i i i i
i
i i i i i
i
i i i i i r> i i i ;z numeric-entity-without-semicolonr. )r r r charr appendintjoinr r1 r frozensetchr
ValueErrorunget) r' isHexallowedradix charStackcr; rG vs r* consumeNumberEntityz!HTMLTokenizer.consumeNumberEntityG s GE
K 7llq|| Q ""A 7llq||
**E22 - - -(3D O " "J|,D$J1S((O**J|4L,L,N ,N O O O rNc))m) |, << |,66 |,33K%%immoo666 2779#5#5 5FF %j 1FK%%immoo666bggi
&> ? ??FF&&
<0H(?(A (A B B B !!)--//222 rwwy 1 11 T f %b )! , , , 6 , , , , , (( - ( O " "Jy,A6#R#R S S S S Ss !AI& & I54I5c 4 | |d dS )zIThis method replaces the need for "entityInAttributeValueState".
T)re rf N)rl )r' re s r* processEntityInAttributez&HTMLTokenizer.processEntityInAttribute s#
{$GGGGGr+ c | j }|d t v r |d t |d< |d t d k rZ|d }t | }t
| t
| k r | |ddd ||d< |d t d k r`|d r(| j t d d d
|d r(| j t d d d
| j | | j
| _ dS )
zThis method is a generic handler for emitting the tags. It also sets
the state to "data" because that's what's needed after a token has been
emitted.
r/ nameStartTagr0 NrZ EndTagr- zattributes-in-end-tagr. selfClosingz self-closing-flag-on-end-tag) r$ r translater r attributeMaprc updater1 rH r! r" )r' tokenrawr0 s r* emitCurrentTokenz HTMLTokenizer.emitCurrentToken sU
! &M] * * !&M 3 34D E EE&MV}
: 666Fm#C((s88c$ii''KKDDbD *** $f
V}
8 444= NO**J|4L4K,M ,M N N N ' UO**J|4L4R,T ,T U U U u%%%^
r+ c z | j }|dk r| j | _ n
|dk r
| j | _ n|dk rQ| j t d dd | j t d dd n|t u rdS |t v rJ| j t d || j
t d
z d nE| j
d }| j t d ||z d d
S ) NrW rX r- invalid-codepointr. r_ Fr^ TrW rX r{ ) r rG entityDataStater" tagOpenStater1 rH r r r
charsUntilr' r0 charss r* r! zHTMLTokenizer.dataState sl { !! 3;; -DJJ
S[[ *DJJ
X
O " "J|,D,?$A $A
B
B
B O " "J|,D,4$6 $6
7
7
7
7
S[[5
_
$
$
O " "J7H,I$(4;+A+A/SW+X+X$X$Z $Z
[
[
[
[ K**+?@@E O " "J|,D$(5L$2 $2
3
3
3tr+ c F | | j | _ dS NT)rl r! r" r5 s r* r~ z HTMLTokenizer.entityDataState s" ^
tr+ c ~ | j }|dk r| j | _ n|dk r
| j | _ n|t
k rdS |dk rQ| j t d dd | j t d d d n|t v rJ| j t d
|| j
t d z d nE| j
d }| j t d ||z d d S )
NrW rX Fr{ r- r| r. r_ r? r^ Tr} ) r rG characterReferenceInRcdatar" rcdataLessThanSignStater r1 rH r r r r s r* rcdataStatezHTMLTokenizer.rcdataState" sl { !! 3;; 8DJJ
S[[ 5DJJ
S[[5
X
O " "J|,D,?$A $A
B
B
B O " "J|,D,4$6 $6
7
7
7
7
_
$
$
O " "J7H,I$(4;+A+A/SW+X+X$X$Z $Z
[
[
[
[ K**+?@@E O " "J|,D$(5L$2 $2
3
3
3tr+ c F | | j | _ dS r )rl r r" r5 s r* r z(HTMLTokenizer.characterReferenceInRcdata? s# %
tr+ c | j }|dk r
| j | _ n|dk rQ| j t d dd | j t d dd nR|t k rdS | j d }| j t d ||z d d
S NrX r{ r- r| r. r_ r? F)rX r{ T) r rG rawtextLessThanSignStater" r1 rH r r r r s r* rawtextStatezHTMLTokenizer.rawtextStateD s { !! 3;; 6DJJ
X
O " "J|,D,?$A $A
B
B
B O " "J|,D,4$6 $6
7
7
7
7
S[[5K**?;;E O " "J|,D$(5L$2 $2
3
3
3tr+ c | j }|dk r
| j | _ n|dk rQ| j t d dd | j t d dd nR|t k rdS | j d }| j t d ||z d d
S r ) r rG scriptDataLessThanSignStater" r1 rH r r r r s r* scriptDataStatez HTMLTokenizer.scriptDataStateV s { !! 3;; 9DJJ
X
O " "J|,D,?$A $A
B
B
B O " "J|,D,4$6 $6
7
7
7
7
S[[5K**?;;E O " "J|,D$(5L$2 $2
3
3
3tr+ c | j }|t k rdS |dk rQ| j t
d dd | j t
d dd nC| j t
d || j d z d dS ) NFr{ r- r| r. r_ r? T)r rG r r1 rH r r r' r0 s r* plaintextStatez HTMLTokenizer.plaintextStateh s { !! 3;;5
X
O " "J|,D,?$A $A
B
B
B O " "J|,D,4$6 $6
7
7
7
7
O " "J|,D$(4;+A+A(+K+K$K$M $M
N
N
Ntr+ c B | j }|dk r| j | _ nq|dk r| j | _ n]|t
v r&t d |g ddd| _ | j | _ n.|dk r]| j
t d dd | j
t d
d d | j | _ n|d k rO| j
t d d
d | j | | j
| _ nv| j
t d dd | j
t d
dd | j | | j | _ dS )N!/rq F)r/ rp r0 rs selfClosingAcknowledged>r- z'expected-tag-name-but-got-right-bracketr. r_ z<>?z'expected-tag-name-but-got-question-markzexpected-tag-namerX T)r rG markupDeclarationOpenStater" closeTagOpenStater
r r$ tagNameStater1 rH r! rN bogusCommentStater s r* r zHTMLTokenizer.tagOpenStatew s { !! 3;; 8DJJ
S[[ /DJJ
\
!
!)3J)?)-r05| j t d dd | j dxx d z
cc< n| j dxx |z
cc< d
S ) Nr r- zeof-in-tag-namer. r r{ r| rp r? T)
r rG r beforeAttributeNameStater" ry r r1 rH r r! selfClosingStartTagStater$ r s r* r zHTMLTokenizer.tagNameState s' { !! ? " " 6DJJ
S[[ ! ! # # # #
S[[ O " "J|,D$5$7 $7
8
8
8 DJJ
S[[ 6DJJ
X
O " "J|,D,?$A $A
B
B
B f % % % 1 % % % % f % % % - % % % tr+ c | j }|dk rd| _ | j | _ nN| j t d dd | j | | j | _ dS Nr r: r_ rX r. T)
r rG temporaryBufferrcdataEndTagOpenStater" r1 rH r rN r r s r* r z%HTMLTokenizer.rcdataLessThanSignState sz { !! 3;;#%D 3DJJ O " "J|,Dc#R#R S S S K d # # # )DJtr+ c | j }|t v r | xj |z
c_ | j | _ nN| j t d dd | j | | j
| _ dS Nr_ r r. T) r rG r
r rcdataEndTagNameStater" r1 rH r rN r r s r* r z#HTMLTokenizer.rcdataEndTagOpenState s { !! < D ( 3DJJ O " "J|,Dd#S#S T T T K d # # # )DJtr+ c | j o9| j d | j k }| j }|t
v r+|r)t d | j g dd| _ | j | _ n|dk r+|r)t d | j g dd| _ | j | _ n|dk r?|r=t d | j g dd| _ |
| j | _ np|t v r| xj |z
c_ nV| j
t d d| j z d | j | | j | _ d
S Nrp rr Fr r r r_ r r. T)r$ lowerr r rG r r r r" r ry r! r
r1 rH rN r r' appropriater0 s r* r z#HTMLTokenizer.rcdataEndTagNameState s 'mD,=f,E,K,K,M,MQUQeQkQkQmQm,m { !! ? " "{ ")3H)=)-)=)+E!C !CD 6DJJ
S[[[[)3H)=)-)=)+E!C !CD 6DJJ
S[[[[)3H)=)-)=)+E!C !CD
! ! # # # DJJ
\
!
! D ( O " "J|,D,043G,G$I $I
J
J
J K d # # # )DJtr+ c | j }|dk rd| _ | j | _ nN| j t d dd | j | | j | _ dS r )
r rG r rawtextEndTagOpenStater" r1 rH r rN r r s r* r z&HTMLTokenizer.rawtextLessThanSignState sz { !! 3;;#%D 4DJJ O " "J|,Dc#R#R S S S K d # # # *DJtr+ c | j }|t v r | xj |z
c_ | j | _ nN| j t d dd | j | | j
| _ dS r ) r rG r
r rawtextEndTagNameStater" r1 rH r rN r r s r* r z$HTMLTokenizer.rawtextEndTagOpenState s { !! < D ( 4DJJ O " "J|,Dd#S#S T T T K d # # # *DJtr+ c | j o9| j d | j k }| j }|t
v r+|r)t d | j g dd| _ | j | _ n|dk r+|r)t d | j g dd| _ | j | _ n|dk r?|r=t d | j g dd| _ |
| j | _ np|t v r| xj |z
c_ nV| j
t d d| j z d | j | | j | _ d
S r )r$ r r r rG r r r r" r ry r! r
r1 rH rN r r s r* r z$HTMLTokenizer.rawtextEndTagNameState s 'mD,=f,E,K,K,M,MQUQeQkQkQmQm,m { !! ? " "{ ")3H)=)-)=)+E!C !CD 6DJJ
S[[[[)3H)=)-)=)+E!C !CD 6DJJ
S[[[[)3H)=)-)=)+E!C !CD
! ! # # # DJJ
\
!
! D ( O " "J|,D,043G,G$I $I
J
J
J K d # # # *DJtr+ c ~ | j }|dk rd| _ | j | _ n|dk r5| j t d dd | j | _ nN| j t d dd | j | | j
| _ dS ) Nr r: r r_ zDJJ
\
!
! O " "J|,DcTXj#Y#Y Z Z Z#'D >DJJ O " "J|,Dc#R#R S S S K d # # # 4DJtr+ c | j }|t v r|| _ | j | _ nN| j t d dd | j | | j
| _ dS r ) r rG r
r scriptDataEscapedEndTagNameStater" r1 rH r rN r r s r* r z.HTMLTokenizer.scriptDataEscapedEndTagOpenState s| { !! < #'D >DJJ O " "J|,Dd#S#S T T T K d # # # 4DJtr+ c | j o9| j d | j k }| j }|t
v r+|r)t d | j g dd| _ | j | _ n|dk r+|r)t d | j g dd| _ | j | _ n|dk r?|r=t d | j g dd| _ |
| j | _ np|t v r| xj |z
c_ nV| j
t d d| j z d | j | | j | _ d
S r )r$ r r r rG r r r r" r ry r! r
r1 rH rN r r s r* r z.HTMLTokenizer.scriptDataEscapedEndTagNameState s 'mD,=f,E,K,K,M,MQUQeQkQkQmQm,m { !! ? " "{ ")3H)=)-)=)+E!C !CD 6DJJ
S[[[[)3H)=)-)=)+E!C !CD 6DJJ
S[[[[)3H)=)-)=)+E!C !CD
! ! # # # DJJ
\
!
! D ( O " "J|,D,043G,G$I $I
J
J
J K d # # # 4DJtr+ c | j }|t t d z v r_| j t d |d | j dk r
| j | _
nu| j | _
nh|t v r9| j t d |d | xj |z
c_ n&| j
| | j | _
dS N)r r r_ r. scriptT)r rG r rK r1 rH r r r scriptDataDoubleEscapedStater" r r
rN r s r* r z.HTMLTokenizer.scriptDataDoubleEscapeStartState s { !! Oi
&;&;; < < O " "J|,Dd#S#S T T T#))++x77 ! >
! 8
\
!
! O " "J|,Dd#S#S T T T D ( K d # # # 4DJtr+ c | j }|dk r5| j t d dd | j | _ n|dk r5| j t d dd | j | _ n|dk rQ| j t d dd | j t d dd nh|t k r5| j t d d d | j | _ n(| j t d |d d
S Nr r_ r. rX r{ r- r| r? eof-in-script-in-scriptT)
r rG r1 rH r scriptDataDoubleEscapedDashStater" (scriptDataDoubleEscapedLessThanSignStater r! r s r* r z*HTMLTokenizer.scriptDataDoubleEscapedState sc { !! 3;; O " "J|,Dc#R#R S S S >DJJ
S[[ O " "J|,Dc#R#R S S S FDJJ
X
O " "J|,D,?$A $A
B
B
B O " "J|,D,4$6 $6
7
7
7
7
S[[ O " "J|,D$=$? $?
@
@
@ DJJ O " "J|,Dd#S#S T T Ttr+ c | j }|dk r6| j t d dd | j | _ n|dk r5| j t d dd | j | _ n|dk r]| j t d dd | j t d dd | j | _ nt|t k r5| j t d d d | j
| _ n4| j t d |d | j | _ d
S r ) r rG r1 rH r $scriptDataDoubleEscapedDashDashStater" r r r r! r s r* r z.HTMLTokenizer.scriptDataDoubleEscapedDashState st { !! 3;; O " "J|,Dc#R#R S S S BDJJ
S[[ O " "J|,Dc#R#R S S S FDJJ
X
O " "J|,D,?$A $A
B
B
B O " "J|,D,4$6 $6
7
7
7 :DJJ
S[[ O " "J|,D$=$? $?
@
@
@ DJJ O " "J|,Dd#S#S T T T :DJtr+ c 4 | j }|dk r*| j t d dd nN|dk r6| j t d dd | j | _ n|dk r5| j t d dd | j | _ n|dk r]| j t d dd | j t d d d | j | _ nt|t k r5| j t d d
d | j
| _ n4| j t d |d | j | _ d S ) Nr r_ r. rX r r{ r- r| r? r T) r rG r1 rH r r r" r r r r! r s r* r z2HTMLTokenizer.scriptDataDoubleEscapedDashDashState% s { !! 3;; O " "J|,Dc#R#R S S S S
S[[ O " "J|,Dc#R#R S S S FDJJ
S[[ O " "J|,Dc#R#R S S S -DJJ
X
O " "J|,D,?$A $A
B
B
B O " "J|,D,4$6 $6
7
7
7 :DJJ
S[[ O " "J|,D$=$? $?
@
@
@ DJJ O " "J|,Dd#S#S T T T :DJtr+ c | j }|dk r<| j t d dd d| _ | j | _ n&| j | | j | _ dS )Nr r_ r. r: T)
r rG r1 rH r r scriptDataDoubleEscapeEndStater" rN r r s r* r z6HTMLTokenizer.scriptDataDoubleEscapedLessThanSignState> sz { !! 3;; O " "J|,Dc#R#R S S S#%D
\
!
! O " "J|,Dd#S#S T T T D ( K d # # # :DJtr+ c | j }|t v r"| j t d n|t v r0| j d |dg | j | _ nT|dk r| n8|dk r| j
| _ n$|dv rW| j t d dd | j d |dg | j | _ n|d
k rW| j t d d d | j d d dg | j | _ nl|t u r5| j t d d
d | j | _ n.| j d |dg | j | _ dS )NTr0 r: r r )'"r] rX r- #invalid-character-in-attribute-namer. r{ r| r? z#expected-attribute-name-but-got-eof)r rG r r r
r$ rH attributeNameStater" ry r r1 r r r! r s r* r z&HTMLTokenizer.beforeAttributeNameStateY s { !! ? " " K " "?D 9 9 9 9
\
!
! f % , ,dBZ 8 8 8 0DJJ
S[[ ! ! # # # #
S[[ 6DJJ
)
)
) O " "J|,D$I$K $K
L
L
L f % , ,dBZ 8 8 8 0DJJ
X
O " "J|,D,?$A $A
B
B
B f % , ,h^ < < < 0DJJ
S[[ O " "J|,D$I$K $K
L
L
L DJJ f % , ,dBZ 8 8 8 0DJtr+ c | j }d}d}|dk r| j | _ n|t v rF| j d d dxx || j t d z z
cc< d}n8|dk rd}n.|t v r| j | _ n|dk r| j | _ n|d k rL| j
t d
d d | j d d dxx d
z
cc< d}n|dv rL| j
t d
dd | j d d dxx |z
cc< d}na|t u r5| j
t d
dd | j | _ n#| j d d dxx |z
cc< d}|r| j d d d t | j d d d< | j d d d D ]L\ }}| j d d d |k r*| j
t d
dd nM|r| dS )NTFr] r0 rZ r r r r{ r- r| r. r? r r rX r zeof-in-attribute-namezduplicate-attribute)r rG beforeAttributeValueStater" r
r$ r r afterAttributeNameStater r1 rH r r r! rt r ry )r' r0 leavingThisState emitTokenrp _s r* r z HTMLTokenizer.attributeNameStatew s { !! 3;; 7DJJ
\
!
! f %b )! , , , &&|T::1;
; , , , $
S[[ II
_
$
$ 5DJJ
S[[ 6DJJ
X
O " "J|,D,?$A $A
B
B
B f %b )! , , , 8 , , , $
_
$
$ O " "J|,D$I$K $K
L
L
L
f %b )! , , , 4 , , , $
S[[ O " "J|,D,C$E $E
F
F
F DJJ f %b )! , , , 4 , , , $
(
!&)"-a0::;KLL
f %b )! , ,V4SbS9
a$V,R03t;;O**J|4L,A,C ,C D D DE <
(%%'''tr+ c | j }|t v r"| j t d n|dk r| j | _ n|dk r| nq|t v r0| j d |dg | j
| _ n8|dk r| j | _ n$|dk rW| j t d d d
| j d d dg | j
| _ n|d v rW| j t d d
d
| j d |dg | j
| _ nl|t u r5| j t d dd
| j | _ n.| j d |dg | j
| _ dS )NTr] r r0 r: r r{ r- r| r. r? r z&invalid-character-after-attribute-namez expected-end-of-tag-but-got-eof)r rG r r r r" ry r
r$ rH r r r1 r r r! r s r* r z%HTMLTokenizer.afterAttributeNameState s { !! ? " " K " "?D 9 9 9 9
S[[ 7DJJ
S[[ ! ! # # # #
\
!
! f % , ,dBZ 8 8 8 0DJJ
S[[ 6DJJ
X
O " "J|,D,?$A $A
B
B
B f % , ,h^ < < < 0DJJ
_
$
$ O " "J|,D$L$N $N
O
O
O f % , ,dBZ 8 8 8 0DJJ
S[[ O " "J|,D$E$G $G
H
H
H DJJ f % , ,dBZ 8 8 8 0DJtr+ c | j }|t v r"| j t d n|dk r| j | _ n|dk r(| j | _ | j | ny|dk r| j | _ ne|dk r>| j
t d dd | n!|d k rV| j
t d d
d | j
d d d
xx dz
cc< | j | _ n|dv rV| j
t d dd | j
d d d
xx |z
cc< | j | _ nk|t u r5| j
t d dd | j | _ n-| j
d d d
xx |z
cc< | j | _ dS )NTr rW r r r- z.expected-attribute-value-but-got-right-bracketr. r{ r| r0 rZ r
r? )r] rX `z"equals-in-unquoted-attribute-valuez$expected-attribute-value-but-got-eof)r rG r r attributeValueDoubleQuotedStater" attributeValueUnQuotedStaterN attributeValueSingleQuotedStater1 rH r ry r$ r r! r s r* r z'HTMLTokenizer.beforeAttributeValueState sG { !! ? " " K " "?D 9 9 9 9
T\\ =DJJ
S[[ 9DJ K d # # # #
S[[ =DJJ
S[[ O " "J|,D$T$V $V
W
W
W ! ! # # # #
X
O " "J|,D,?$A $A
B
B
B f %b )! , , , 8 , , , 9DJJ
_
$
$ O " "J|,D$H$J $J
K
K
K f %b )! , , , 4 , , , 9DJJ
S[[ O " "J|,D$J$L $L
M
M
M DJJ f %b )! , , , 4 , , , 9DJtr+ c * | j }|dk r
| j | _ n|dk r| d n|dk rJ| j t d dd | j d d d xx d
z
cc< nz|t u r5| j t d d d | j
| _ n<| j d d d xx || j d z z
cc< d
S )Nr rW r{ r- r| r. r0 rZ r
r? z#eof-in-attribute-value-double-quote)r rW r{ T r rG afterAttributeValueStater" rn r1 rH r r$ r r! r r s r* r z-HTMLTokenizer.attributeValueDoubleQuotedState sD { !! 4<< 6DJJ
S[[ ) )# . . . .
X
O " "J|,D,?$A $A
B
B
B f %b )! , , , 8 , , , ,
S[[ O " "J|,D$I$K $K
L
L
L DJJ f %b )! , , , &&'<==1>
> , , ,tr+ c * | j }|dk r
| j | _ n|dk r| d n|dk rJ| j t d dd | j d d d xx d
z
cc< nz|t u r5| j t d d d | j
| _ n<| j d d d xx || j d z z
cc< d
S )Nr rW r{ r- r| r. r0 rZ r
r? z#eof-in-attribute-value-single-quote)r rW r{ Tr r s r* r z-HTMLTokenizer.attributeValueSingleQuotedState sD { !! 3;; 6DJJ
S[[ ) )# . . . .
X
O " "J|,D,?$A $A
B
B
B f %b )! , , , 8 , , , ,
S[[ O " "J|,D$I$K $K
L
L
L DJJ f %b )! , , , &&';<<1=
= , , ,tr+ c 2 | j }|t v r| j | _ nf|dk r| d nI|dk r| n-|dv rJ| j t d dd | j
d d d xx |z
cc< n|d
k rJ| j t d d d | j
d d d xx d z
cc< n|t u r5| j t d d
d | j | _ nQ| j
d d d xx || j
t d t z z z
cc< dS )NrW r )r r r] rX r r- z0unexpected-character-in-unquoted-attribute-valuer. r0 rZ r
r{ r| r? z eof-in-attribute-value-no-quotes)rW r r r r] rX r r{ T)r rG r r r" rn ry r1 rH r r$ r r! r rK r s r* r z)HTMLTokenizer.attributeValueUnQuotedState s { !! ? " " 6DJJ
S[[ ) )# . . . .
S[[ ! ! # # # #
.
.
. O " "J|,D$V$X $X
Y
Y
Y f %b )! , , , 4 , , , ,
X
O " "J|,D,?$A $A
B
B
B f %b )! , , , 8 , , , ,
S[[ O " "J|,D$F$H $H
I
I
I DJJ f %b )! , , ,t{7M7MGHH?Z8\ 8\ 1\
\ , , ,tr+ c | j }|t v r
| j | _ n|dk r| n|dk r
| j | _ n|t u rO| j t d dd | j | | j | _ nN| j t d dd | j | | j | _ dS )Nr r r- z$unexpected-EOF-after-attribute-valuer. z*unexpected-character-after-attribute-valueT)
r rG r r r" ry r r r1 rH r rN r! r s r* r z&HTMLTokenizer.afterAttributeValueState. s { !! ? " " 6DJJ
S[[ ! ! # # # #
S[[ 6DJJ
S[[ O " "J|,D$J$L $L
M
M
M K d # # # DJJ O " "J|,D$P$R $R
S
S
S K d # # # 6DJtr+ c | j }|dk r d| j d< | n|t u rO| j t d dd | j | | j | _
nN| j t d dd | j | | j | _
dS )Nr Trs r- z#unexpected-EOF-after-solidus-in-tagr. z)unexpected-character-after-solidus-in-tag) r rG r$ ry r r1 rH r rN r! r" r r s r* r z&HTMLTokenizer.selfClosingStartTagStateB s { !! 3;;/3D m , ! ! # # # #
S[[ O " "J|,D$I$K $K
L
L
L
K d # # # DJJ O " "J|,D$O$Q $Q
R
R
R K d # # # 6DJtr+ c | j d }| dd }| j t
d |d | j | j | _ dS )Nr r{ r? Commentr. T) r r replacer1 rH r rG r! r" r s r* r z HTMLTokenizer.bogusCommentStateT sz {%%c**||Hh// *D 9 9 ; ; ;
^
tr+ c | j g}|d dk r]| | j |d dk r#t d dd| _ | j | _ dS n|d dv rjd}dD ]<}| | j |d |vrd } n=|r&t d
dd d dd | _ | j | _ dS n|d d k r| j | j j j
r| j j j
d j | j j j k rSd}d
D ]>}| | j |d |k rd } n?|r| j
| _ dS | j t d dd |r.| j | |.| j | _ dS )NrZ r r r: r. T)dD))oOrS CtTyYpPeEFDoctype)r/ rp publicIdsystemIdcorrect[)r r Ar r r r- zexpected-dashes-or-doctype)r rG rH r r$ commentStartStater" doctypeStater tree openElements namespacedefaultNamespacecdataSectionStater1 rN r3 r )r' rR matchedexpecteds r* r z(HTMLTokenizer.markupDeclarationOpenStatec sS [%%''( R=C T[ - - / / 0 0 0}##-7 -BB$O$O! ! 3
t $ r]j
(
(G A
!1!1!3!3444 R=00 #GE 1
-7 -B-/15404%6 %6! " .
t
ms""k%k + &k +B/9T[=M=^^^G :
!1!1!3!3444 R=H,, #GE -
! 3
t
<(@ < > > ? ? ? / K immoo . . . /+
tr+ c | j }|dk r| j | _ n |dk r>| j t d dd | j dxx dz
cc< n|dk rT| j t d d d | j | j | j | _ n~|t u rT| j t d d
d | j | j | j | _ n!| j dxx |z
cc< | j
| _ d S ) Nr r{ r- r| r. r0 r? r incorrect-commenteof-in-commentT) r rG commentStartDashStater" r1 rH r r$ r! r commentStater s r* r z HTMLTokenizer.commentStartState sn { !! 3;; 3DJJ
X
O " "J|,D,?$A $A
B
B
B f % % % 1 % % % %
S[[ O " "J|,D$7$9 $9
:
:
: O " "4#4 5 5 5 DJJ
S[[ O " "J|,D$4$6 $6
7
7
7 O " "4#4 5 5 5 DJJ f % % % - % % % *DJtr+ c | j }|dk r| j | _ n |dk r>| j t d dd | j dxx dz
cc< n|dk rT| j t d d d | j | j | j | _ n|t u rT| j t d d
d | j | j | j | _ n$| j dxx d|z z
cc< | j
| _ d S ) Nr r{ r- r| r. r0 -�r r r T) r rG commentEndStater" r1 rH r r$ r! r r r s r* r z#HTMLTokenizer.commentStartDashState sr { !! 3;; -DJJ
X
O " "J|,D,?$A $A
B
B
B f % % % 2 % % % %
S[[ O " "J|,D$7$9 $9
:
:
: O " "4#4 5 5 5 DJJ
S[[ O " "J|,D$4$6 $6
7
7
7 O " "4#4 5 5 5 DJJ f % % %t 3 % % % *DJtr+ c | j }|dk r
| j | _ n|dk r>| j t d dd | j dxx dz
cc< n|t u rT| j t d dd | j | j | j | _ n0| j dxx || j
d z z
cc< d
S ) Nr r{ r- r| r. r0 r? r )r r{ T) r rG commentEndDashStater" r1 rH r r$ r r! r r s r* r zHTMLTokenizer.commentState s# { !! 3;; 1DJJ
X
O " "J|,D,?$A $A
B
B
B f % % % 1 % % % %
S[[ O " "J|,D,<$> $>
?
?
? O " "4#4 5 5 5 DJJ f % % % &&77*8
8 % % %tr+ c | j }|dk r
| j | _ n|dk rJ| j t d dd | j dxx dz
cc< | j | _ n|t u rT| j t d dd | j | j | j
| _ n$| j dxx d|z z
cc< | j | _ d S )
Nr r{ r- r| r. r0 r! zeof-in-comment-end-dashT) r rG r" r" r1 rH r r$ r r r! r s r* r$ z!HTMLTokenizer.commentEndDashState s# { !! 3;; -DJJ
X
O " "J|,D,?$A $A
B
B
B f % % % 2 % % % *DJJ
S[[ O " "J|,D$=$? $?
@
@
@ O " "4#4 5 5 5 DJJ f % % %t 3 % % % *DJtr+ c | j }|dk r-| j | j | j | _ ny|dk rK| j t d dd | j dxx dz
cc< | j | _ n(|dk r5| j t d d d | j | _ n|d
k r>| j t d d d | j dxx |z
cc< n|t u rT| j t d d d | j | j | j | _ nL| j t d d
d | j dxx d|z z
cc< | j | _ dS )Nr r{ r- r| r. r0 u --�r z,unexpected-bang-after-double-dash-in-commentr z,unexpected-dash-after-double-dash-in-commentzeof-in-comment-double-dashzunexpected-char-in-commentz--T) r rG r1 rH r$ r! r" r r commentEndBangStater r s r* r" z HTMLTokenizer.commentEndState s { !! 3;; O " "4#4 5 5 5 DJJ
X
O " "J|,D,?$A $A
B
B
B f % % % 3 % % % *DJJ
S[[ O " "J|,D$R$T $T
U
U
U 1DJJ
S[[ O " "J|,D$R$T $T
U
U
U f % % % - % % % %
S[[ O " "J|,D$@$B $B
C
C
C O " "4#4 5 5 5 DJJ
O " "J|,D$@$B $B
C
C
C f % % % 4 % % % *DJtr+ c | j }|dk r,| j | j | j | _ n|dk r"| j dxx dz
cc< | j | _ n|dk rJ| j t d dd | j dxx d z
cc< | j | _ n|t u rT| j t d d
d | j | j | j | _ n$| j dxx d|z z
cc< | j | _ d S ) Nr r r0 z--!r{ r- r| r. u --!�z eof-in-comment-end-bang-stateT) r rG r1 rH r$ r! r" r$ r r r r s r* r' z!HTMLTokenizer.commentEndBangState sq { !! 3;; O " "4#4 5 5 5 DJJ
S[[ f % % % . % % % 1DJJ
X
O " "J|,D,?$A $A
B
B
B f % % % 4 % % % *DJJ
S[[ O " "J|,D$C$E $E
F
F
F O " "4#4 5 5 5 DJJ f % % % 5 % % % *DJtr+ c | j }|t v r
| j | _ n|t
u r^| j t d dd d| j d< | j | j | j
| _ nN| j t d dd | j | | j | _ dS )Nr- !expected-doctype-name-but-got-eofr. Fr zneed-space-after-doctypeT) r rG r beforeDoctypeNameStater" r r1 rH r r$ r! rN r s r* r zHTMLTokenizer.doctypeState s { !! ? " " 4DJJ
S[[ O " "J|,D$G$I $I
J
J
J+0D i ( O " "4#4 5 5 5 DJJ O " "J|,D$>$@ $@
A
A
A K d # # # 4DJtr+ c | j }|t v rn&|dk r^| j t
d dd d| j d< | j | j | j | _ n|dk r?| j t
d dd d | j d
< | j | _ n}|t u r^| j t
d d d d| j d< | j | j | j | _ n|| j d
< | j | _ d S )
Nr r- z+expected-doctype-name-but-got-right-bracketr. Fr r{ r| r? rp r* T) r rG r r1 rH r r$ r! r" doctypeNameStater r s r* r+ z$HTMLTokenizer.beforeDoctypeNameState* sp { !! ? " "
S[[ O " "J|,D$Q$S $S
T
T
T+0D i ( O " "4#4 5 5 5 DJJ
X
O " "J|,D,?$A $A
B
B
B(0D f % .DJJ
S[[ O " "J|,D$G$I $I
J
J
J+0D i ( O " "4#4 5 5 5 DJJ(,D f % .DJtr+ c p | j }|t v r;| j d t
| j d< | j | _ nX|dk rY| j d t
| j d< | j | j | j
| _ n|dk rJ| j t d dd | j dxx dz
cc< | j | _ n|t u r| j t d dd d | j d
< | j d t
| j d< | j | j | j
| _ n| j dxx |z
cc< d S ) Nrp r r{ r- r| r. r? zeof-in-doctype-nameFr T)r rG r r$ rt r afterDoctypeNameStater" r1 rH r! r r- r r s r* r- z HTMLTokenizer.doctypeNameStateD s { !! ? " "(,(9&(A(K(KL\(](]D f % 3DJJ
S[[(,(9&(A(K(KL\(](]D f % O " "4#4 5 5 5 DJJ
X
O " "J|,D,?$A $A
B
B
B f % % % 1 % % % .DJJ
S[[ O " "J|,D$9$; $;
<
<
<+0D i ((,(9&(A(K(KL\(](]D f % O " "4#4 5 5 5 DJJ f % % % - % % %tr+ c ^ | j }|t v rn|dk r-| j | j | j | _ nU|t u rxd| j d< | j | | j t d dd | j | j | j | _ n|dv r9d}d D ]#}| j }||vrd} n$|r| j | _ dS n<|d
v r8d}d D ]#}| j }||vrd} n$|r| j | _ dS | j | | j t d d d
|id d| j d< | j
| _ dS )Nr Fr r- eof-in-doctyper. r T))uU)bB)lL)iIr sS)r r: r r )mMz*expected-space-or-right-bracket-in-doctyper0 r< )r rG r r1 rH r$ r! r" r rN r afterDoctypePublicKeywordState afterDoctypeSystemKeywordStatebogusDoctypeState)r' r0 r r s r* r/ z#HTMLTokenizer.afterDoctypeNameState] s { !! ? " "
S[[ O " "4#4 5 5 5 DJJ
S[[+0D i ( K d # # # O " "J|,D$4$6 $6
7
7
7 O " "4#4 5 5 5 DJJz!! !9 H ;++--D8++"' , !%!DDJ 4 ## !9 H ;++--D8++"' , !%!DDJ 4
K d # # # O " "J|,D$P%+TN$4 $4
5
5
5 ,1D i ( /DJtr+ c $ | j }|t v r
| j | _ n|dv rO| j t d dd | j | | j | _ n|t u r^| j t d dd d| j
d< | j | j
| j | _ n&| j | | j | _ dS N)r r r- unexpected-char-in-doctyper. r1 Fr T) r rG r "beforeDoctypePublicIdentifierStater" r1 rH r rN r r$ r! r s r* r? z,HTMLTokenizer.afterDoctypePublicKeywordState { !! ? " " @DJJ
Z
O " "J|,D$@$B $B
C
C
C K d # # # @DJJ
S[[ O " "J|,D$4$6 $6
7
7
7+0D i ( O " "4#4 5 5 5 DJJ K d # # # @DJtr+ c | j }|t v rnE|dk rd| j d< | j | _ n'|dk rd| j d< | j | _ n |dk r^| j t d dd d | j d
< | j | j | j
| _ n|t u r^| j t d d d d | j d
< | j | j | j
| _ n>| j t d d d d | j d
< | j | _ d
S )Nr r: r
r r r- unexpected-end-of-doctyper. Fr r1 rD T)
r rG r r$ (doctypePublicIdentifierDoubleQuotedStater" (doctypePublicIdentifierSingleQuotedStater1 rH r r! r rA r s r* rE z0HTMLTokenizer.beforeDoctypePublicIdentifierState s { !! ? " "
T\\,.D j ) FDJJ
S[[,.D j ) FDJJ
S[[ O " "J|,D$?$A $A
B
B
B+0D i ( O " "4#4 5 5 5 DJJ
S[[ O " "J|,D$4$6 $6
7
7
7+0D i ( O " "4#4 5 5 5 DJJ O " "J|,D$@$B $B
C
C
C+0D i ( /DJtr+ c | j }|dk r| j | _ n$|dk r>| j t d dd | j dxx dz
cc< n|dk r^| j t d d d d
| j d < | j | j | j | _ n||t u r^| j t d d d d
| j d < | j | j | j | _ n| j dxx |z
cc< d
S )Nr r{ r- r| r. r
r? r rH Fr r1 T
r rG !afterDoctypePublicIdentifierStater" r1 rH r r$ r! r r s r* rI z6HTMLTokenizer.doctypePublicIdentifierDoubleQuotedState { !! 4<< ?DJJ
X
O " "J|,D,?$A $A
B
B
B j ) ) )X 5 ) ) ) )
S[[ O " "J|,D$?$A $A
B
B
B+0D i ( O " "4#4 5 5 5 DJJ
S[[ O " "J|,D$4$6 $6
7
7
7+0D i ( O " "4#4 5 5 5 DJJ j ) ) )T 1 ) ) )tr+ c | j }|dk r| j | _ n$|dk r>| j t d dd | j dxx dz
cc< n|dk r^| j t d d d d
| j d < | j | j | j | _ n||t u r^| j t d d d d
| j d < | j | j | j | _ n| j dxx |z
cc< d
S )Nr r{ r- r| r. r
r? r rH Fr r1 TrL r s r* rJ z6HTMLTokenizer.doctypePublicIdentifierSingleQuotedState { !! 3;; ?DJJ
X
O " "J|,D,?$A $A
B
B
B j ) ) )X 5 ) ) ) )
S[[ O " "J|,D$?$A $A
B
B
B+0D i ( O " "4#4 5 5 5 DJJ
S[[ O " "J|,D$4$6 $6
7
7
7+0D i ( O " "4#4 5 5 5 DJJ j ) ) )T 1 ) ) )tr+ c * | j }|t v r| j | _ nb|dk r-| j | j | j | _ n/|dk r?| j t d dd d| j d< | j
| _ n|dk r?| j t d dd d| j d< | j | _ n|t u r^| j t d d d d
| j d < | j | j | j | _ n>| j t d dd d
| j d < | j
| _ d S )
Nr r r- rD r. r: r r r1 Fr T)r rG r -betweenDoctypePublicAndSystemIdentifiersStater" r1 rH r$ r! r (doctypeSystemIdentifierDoubleQuotedState(doctypeSystemIdentifierSingleQuotedStater rA r s r* rM z/HTMLTokenizer.afterDoctypePublicIdentifierState s { !! ? " " KDJJ
S[[ O " "4#4 5 5 5 DJJ
S[[ O " "J|,D$@$B $B
C
C
C,.D j ) FDJJ
S[[ O " "J|,D$@$B $B
C
C
C,.D j ) FDJJ
S[[ O " "J|,D$4$6 $6
7
7
7+0D i ( O " "4#4 5 5 5 DJJ O " "J|,D$@$B $B
C
C
C+0D i ( /DJtr+ c t | j }|t v rn|dk r,| j | j | j | _ n|dk rd| j d< | j | _ n|dk rd| j d< | j | _ n|t k r^| j t d dd d | j d
< | j | j | j | _ n>| j t d d d d | j d
< | j | _ d S )
Nr r r: r r r- r1 r. Fr rD T)
r rG r r1 rH r$ r! r" rS rT r r rA r s r* rR z;HTMLTokenizer.betweenDoctypePublicAndSystemIdentifiersState sK { !! ? " "
S[[ O " "4#4 5 5 5 DJJ
S[[,.D j ) FDJJ
S[[,.D j ) FDJJ
S[[ O " "J|,D$4$6 $6
7
7
7+0D i ( O " "4#4 5 5 5 DJJ O " "J|,D$@$B $B
C
C
C+0D i ( /DJtr+ c $ | j }|t v r
| j | _ n|dv rO| j t d dd | j | | j | _ n|t u r^| j t d dd d| j
d< | j | j
| j | _ n&| j | | j | _ dS rC ) r rG r "beforeDoctypeSystemIdentifierStater" r1 rH r rN r r$ r! r s r* r@ z,HTMLTokenizer.afterDoctypeSystemKeywordState) rF r+ c | j }|t v rnE|dk rd| j d< | j | _ n'|dk rd| j d< | j | _ n |dk r^| j t d dd d | j d
< | j | j | j
| _ n|t u r^| j t d d d d | j d
< | j | j | j
| _ n>| j t d dd d | j d
< | j | _ d S )
Nr r: r r r r- rD r. Fr r1 T)
r rG r r$ rS r" rT r1 rH r r! r rA r s r* rW z0HTMLTokenizer.beforeDoctypeSystemIdentifierState= s { !! ? " "
T\\,.D j ) FDJJ
S[[,.D j ) FDJJ
S[[ O " "J|,D$@$B $B
C
C
C+0D i ( O " "4#4 5 5 5 DJJ
S[[ O " "J|,D$4$6 $6
7
7
7+0D i ( O " "4#4 5 5 5 DJJ O " "J|,D$@$B $B
C
C
C+0D i ( /DJtr+ c | j }|dk r| j | _ n$|dk r>| j t d dd | j dxx dz
cc< n|dk r^| j t d d d d
| j d < | j | j | j | _ n||t u r^| j t d d d d
| j d < | j | j | j | _ n| j dxx |z
cc< d
S )Nr r{ r- r| r. r r? r rH Fr r1 T
r rG !afterDoctypeSystemIdentifierStater" r1 rH r r$ r! r r s r* rS z6HTMLTokenizer.doctypeSystemIdentifierDoubleQuotedStateZ rN r+ c | j }|dk r| j | _ n$|dk r>| j t d dd | j dxx dz
cc< n|dk r^| j t d d d d
| j d < | j | j | j | _ n||t u r^| j t d d d d
| j d < | j | j | j | _ n| j dxx |z
cc< d
S )Nr r{ r- r| r. r r? r rH Fr r1 TrZ r s r* rT z6HTMLTokenizer.doctypeSystemIdentifierSingleQuotedStater rP r+ c | j }|t v rn|dk r,| j | j | j | _ n|t u r^| j t d dd d| j d< | j | j | j | _ n4| j t d dd | j
| _ dS ) Nr r- r1 r. Fr rD T) r rG r r1 rH r$ r! r" r r rA r s r* r[ z/HTMLTokenizer.afterDoctypeSystemIdentifierState s { !! ? " "
S[[ O " "4#4 5 5 5 DJJ
S[[ O " "J|,D$4$6 $6
7
7
7+0D i ( O " "4#4 5 5 5 DJJ O " "J|,D$@$B $B
C
C
C /DJtr+ c < | j }|dk r,| j | j | j | _ nP|t u rF| j | | j | j | j | _ n dS )Nr T) r rG r1 rH r$ r! r" r rN r s r* rA z HTMLTokenizer.bogusDoctypeState s { !! 3;; O " "4#4 5 5 5 DJJ
S[[ K d # # # O " "4#4 5 5 5 DJJ tr+ c g } | | j d | | j d | j }|t k rnF|dk sJ |d dd dk r|d d d |d< n| | d | }| d }|d k rPt | D ]*}| j t d
d d +|
dd
}|r(| j t d |d | j | _ dS )NT]r rZ z]]r: r{ r r- r| r. r? r_ )
rH r r rG r rJ countranger1 r r r! r" )r' r0 rG nullCountr s r* r z HTMLTokenizer.cdataSectionState s & KK ..s33 4 4 4 KK ..s33 4 4 4;##%%Ds{{s{{{{8BCC=D(( #Bx}DHKK%%% & wwt}}JJx(( q==9%%
F
F&&
<0H0C(E (E F F F F<<(33D 3 O " "J|,D,0$2 $2
3
3
3^
tr+ )Nr )N__name__
__module__ __qualname____doc__r& r6 rU rl rn ry r! r~ r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r$ r" r' r r+ r- r/ r? rE rI rJ rM rR r@ rW rS rT r[ rA r
__classcell__)r) s @r* r r s
.
.
.
.
.
. 0 0 0 F F FPNT NT NT NT`H H H
$ $ $8 :
:
$ $
! ! !F 0 , 8 8 8 ( ( ,
8 * . 2 <4 4 4l @ D & & 2 ( $
+ + +Z . . $ & > . " 4 21 1 1f ( : 0 0 < 4 ( : 0 0 & r+ r N)
__future__r r r pip._vendor.sixr rL collectionsr r sysr constantsr r r
r r r r r r r _inputstreamr _trier r` dictru objectr r+ r* rt si B B B B B B B B B B ) ) ) ) ) ) * * * * * * * * & & & & & & 5 5 5 5 5 5 5 5 - - - - - - - - - - 0 0 0 0 0 0 0 0 , , , , , , ) ) ) ) ) ) tH~~ 6LL Ll l l l lF l l l l lr+