🔐 Sid Gifari File Manager Pro
v8.0.5 | 2026-06-14 16:59:57 | PHP 8.1.34
📂
/ (Root)
/
usr
/
lib
/
python3.9
/
site-packages
/
pip
/
_vendor
/
html5lib
/
__pycache__
📍 /usr/lib/python3.9/site-packages/pip/_vendor/html5lib/__pycache__
🔄 Refresh
✏️
Editing: _tokenizer.cpython-39.opt-1.pyc
Read Only
a �lj�, � @ s� d dl mZmZmZ d dlmZ d dlmZm Z d dl m Z ddl m Z ddl mZ ddl mZmZ dd l mZmZmZ dd l mZmZ dd l mZ dd lmZ dd lmZ ee�Ze dkr�e Z ne Z G dd� de �Z dS )� )�absolute_import�division�unicode_literals)�unichr)�deque� OrderedDict)� version_info� )�spaceCharacters)�entities)� asciiLetters�asciiUpper2Lower)�digits� hexDigits�EOF)� tokenTypes� tagTokenTypes)�replacementCharacters)�HTMLInputStream)�Trie)� � c sd e Zd ZdZd�� fdd� Zdd� Zdd� Zd�d d �Zd d � Zdd� Z dd� Z dd� Z dd� Z dd� Z dd� Zdd� Zd d � Zd d � Zd d!� Zd"d#� Zd$d%� Zd&d'� Zd(d)� Zd*d+� Zd,d-� Zd.d/� Zd0d1� Zd2d3� Zd4d5� Z d6d7� Z d8d9� Z d:d;� Z d<d=� Z d>d?� Z!d@dA� Z"dBdC� Z#dDdE� Z$dFdG� Z%dHdI� Z&dJdK� Z'dLdM� Z(dNdO� Z)dPdQ� Z*dRdS� Z+dTdU� Z,dVdW� Z-dXdY� Z.dZd[� Z/d\d]� Z0d^d_� Z1d`da� Z2dbdc� Z3ddde� Z4dfdg� Z5dhdi� Z6djdk� Z7dldm� Z8dndo� Z9dpdq� Z:drds� Z;dtdu� Z<dvdw� Z=dxdy� Z>dzd{� Z?d|d}� Z@d~d� ZAd�d�� ZBd�d�� ZCd�d�� ZDd�d�� ZEd�d�� ZFd�d�� ZGd�d�� ZHd�d�� ZId�d�� ZJd�d�� ZKd�d�� ZL� ZMS )�� HTMLTokenizera This class takes care of tokenizing HTML. * self.currentToken Holds the token that is currently being processed. * self.state Holds a reference to the method to be invoked... XXX * self.stream Points to HTMLInputStream object. Nc sJ t |fi |��| _|| _d| _g | _| j| _d| _d | _t t | �� � d S )NF) r �stream�parserZ escapeFlagZ lastFourChars� dataState�state�escape� currentToken�superr �__init__)�selfr r �kwargs�� __class__� �C/usr/lib/python3.9/site-packages/pip/_vendor/html5lib/_tokenizer.pyr ( s zHTMLTokenizer.__init__c c sP t g �| _| �� rL| jjr6td | jj�d�d�V q| jr | j�� V q6q dS )z� This is where the magic happens. We do our usually processing through the states and when we have a token to return we yield the token which pauses processing until the next token is requested. � ParseErrorr ��type�dataN)r � tokenQueuer r �errorsr �pop�popleft�r! r% r% r& �__iter__7 s zHTMLTokenizer.__iter__c C s� t }d}|rt}d}g }| j�� }||v rH|turH|�|� | j�� }q"td�|�|�}|tv r�t| }| j �t d dd|id�� �n d| kr�d ks�n |d kr�d }| j �t d dd|id�� n�d | kr�d k�s>n d| kr�dk�s>n d| k�rdk�s>n d| k�r*dk�s>n |t g d��v �rZ| j �t d dd|id�� z t |�}W n< t �y� |d }t d|d? B �t d|d@ B � }Y n0 |dk�r�| j �t d dd�� | j�|� |S )z�This function returns either U+FFFD or the character based on the decimal or hexadecimal representation. It also discards ";" if present. If not present self.tokenQueue.append({"type": tokenTypes["ParseError"]}) is invoked. � � � r' z$illegal-codepoint-for-numeric-entity� charAsInt�r) r* Zdatavarsi � i�� � � �r � � � � � i� i� )#� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� r6 i i � i� �;z numeric-entity-without-semicolonr( )r r r �charr �append�int�joinr r+ r � frozenset�chr� ValueError�unget) r! ZisHex�allowed�radix� charStack�cr4 r? �vr% r% r& �consumeNumberEntityG sn � � � �� � � � � � � & � z!HTMLTokenizer.consumeNumberEntityFc C s� d}| j �� g}|d tv sB|d tddfv sB|d urV||d krV| j �|d � �n |d dk�r d}|�| j �� � |d dv r�d}|�| j �� � |r�|d tv s�|s�|d tv r�| j �|d � | �|�}n4| j �t d d d �� | j �|� � � dd � |� }�nf|d tu�rDt �d � |���s0�qD|�| j �� � �q z$t �d � |d d� ��}t|�}W n t�y� d }Y n0 |d u�r>|d d k�r�| j �t d dd �� |d d k�r|�r|| tv �s�|| tv �s�|| dk�r| j �|� � � dd � |� }n.t| }| j �|� � � |d � ||d � �7 }n4| j �t d dd �� | j �|� � � dd � |� }|�r�| jd d d |7 < n*|tv �r�d}nd}| j �t | |d �� d S )N�&r �<�#F���)�x�XTr' zexpected-numeric-entityr( r3 r> z named-entity-without-semicolon�=zexpected-named-entityr* r �SpaceCharacters� Characters)r r? r r rF r@ r r rL r+ r r- rB � entitiesTrieZhas_keys_with_prefixZlongest_prefix�len�KeyErrorr r r ) r! � allowedChar� fromAttribute�outputrI �hexZ entityNameZ entityLengthZ tokenTyper% r% r& � consumeEntity� s~ � � � � � � � � � � zHTMLTokenizer.consumeEntityc C s | j |dd� dS )zIThis method replaces the need for "entityInAttributeValueState". T)rY rZ N)r] )r! rY r% r% r& �processEntityInAttribute� s z&HTMLTokenizer.processEntityInAttributec C s� | j }|d tv r�|d �t�|d< |d td krp|d }t|�}t|�t|�krh|�|ddd� � ||d< |d td kr�|d r�| j� td d d �� |d r�| j� td d d �� | j� |� | j | _ dS ) z�This method is a generic handler for emitting the tags. It also sets the state to "data" because that's what's needed after a token has been emitted. r) �name�StartTagr* NrP �EndTagr' zattributes-in-end-tagr( � selfClosingz self-closing-flag-on-end-tag) r r � translater r � attributeMaprW �updater+ r@ r r )r! �token�rawr* r% r% r& �emitCurrentToken� s( � � z HTMLTokenizer.emitCurrentTokenc C s� | j �� }|dkr | j| _n�|dkr.| j| _n�|dkrd| j�td dd�� | j�td dd�� n`|tu rpdS |t v r�| j�td || j � t d � d�� n&| j � d �}| j�td || d�� d S ) NrM rN � r' �invalid-codepointr( rU FrT T�rM rN ri ) r r? �entityDataStater � tagOpenStater+ r@ r r r � charsUntil�r! r* �charsr% r% r& r s. � � � �zHTMLTokenizer.dataStatec C s | � � | j| _dS �NT)r] r r r/ r% r% r& rl s z HTMLTokenizer.entityDataStatec C s� | j �� }|dkr | j| _n�|dkr.| j| _n�|tkr:dS |dkrp| j�td dd�� | j�td d d�� nT|t v r�| j�td || j � t d � d�� n&| j � d �}| j�td || d�� d S ) NrM rN Fri r' rj r( rU r7 rT Trk ) r r? �characterReferenceInRcdatar �rcdataLessThanSignStater r+ r@ r r rn ro r% r% r& � rcdataState" s. � � � �zHTMLTokenizer.rcdataStatec C s | � � | j| _dS rq )r] rt r r/ r% r% r& rr ? s z(HTMLTokenizer.characterReferenceInRcdatac C s� | j �� }|dkr | j| _nh|dkrR| j�td dd�� | j�td dd�� n2|tkr^dS | j �d �}| j�td || d�� d S � NrN ri r' rj r( rU r7 F)rN ri T) r r? �rawtextLessThanSignStater r+ r@ r r rn ro r% r% r& � rawtextStateD s" � � �zHTMLTokenizer.rawtextStatec C s� | j �� }|dkr | j| _nh|dkrR| j�td dd�� | j�td dd�� n2|tkr^dS | j �d �}| j�td || d�� d S ru ) r r? �scriptDataLessThanSignStater r+ r@ r r rn ro r% r% r& �scriptDataStateV s" � � �z HTMLTokenizer.scriptDataStatec C sr | j �� }|tkrdS |dkrL| j�td dd�� | j�td dd�� n"| j�td || j �d� d�� dS ) NFri r' rj r( rU r7 T)r r? r r+ r@ r rn �r! r* r% r% r& �plaintextStateh s � � �z HTMLTokenizer.plaintextStatec C s | j �� }|dkr | j| _n�|dkr.| j| _n�|tv rVtd |g ddd�| _| j| _n�|dkr�| j � td dd �� | j � td d d �� | j | _nt|d kr�| j � td d d �� | j � |� | j | _n@| j � td dd �� | j � td dd �� | j � |� | j | _dS )N�!�/r` F)r) r_ r* rb ZselfClosingAcknowledged�>r' z'expected-tag-name-but-got-right-bracketr( rU z<>�?z'expected-tag-name-but-got-question-markzexpected-tag-namerN T)r r? �markupDeclarationOpenStater �closeTagOpenStater r r � tagNameStater+ r@ r rF �bogusCommentStaterz r% r% r& rm w s> � � � � zHTMLTokenizer.tagOpenStatec C s� | j �� }|tv r0td |g dd�| _| j| _n�|dkrX| j�td dd�� | j | _nn|t u r�| j�td dd�� | j�td d d�� | j | _n0| j�td d d |id �� | j � |� | j | _dS )Nra F�r) r_ r* rb r~ r' z*expected-closing-tag-but-got-right-bracketr( z expected-closing-tag-but-got-eofrU �</z!expected-closing-tag-but-got-charr* r5 T) r r? r r r r� r r+ r@ r r rF r� rz r% r% r& r� � s0 � � � � z HTMLTokenizer.closeTagOpenStatec C s� | j �� }|tv r | j| _n�|dkr.| �� n~|tu rV| j�t d dd�� | j | _nV|dkrh| j | _nD|dkr�| j�t d dd�� | j d d 7 < n| j d |7 < d S ) Nr~ r' zeof-in-tag-namer( r} ri rj r_ r7 T) r r? r �beforeAttributeNameStater rh r r+ r@ r r �selfClosingStartTagStater rz r% r% r& r� � s& � �zHTMLTokenizer.tagNameStatec C sP | j �� }|dkr"d| _| j| _n*| j�td dd�� | j �|� | j | _dS �Nr} r3 rU rN r( T) r r? �temporaryBuffer�rcdataEndTagOpenStater r+ r@ r rF rt rz r% r% r& rs � s z%HTMLTokenizer.rcdataLessThanSignStatec C sX | j �� }|tv r*| j|7 _| j| _n*| j�td dd�� | j � |� | j | _dS �NrU r� r( T) r r? r r� �rcdataEndTagNameStater r+ r@ r rF rt rz r% r% r& r� � s z#HTMLTokenizer.rcdataEndTagOpenStatec C s | j o | j d �� | j�� k}| j�� }|tv rT|rTtd | jg dd�| _ | j| _n�|dkr�|r�td | jg dd�| _ | j | _n||dkr�|r�td | jg dd�| _ | � � | j | _nH|t v r�| j|7 _n0| j �td d| j d �� | j�|� | j| _d S � Nr_ ra Fr� r} r~ rU r� r( T)r �lowerr� r r? r r r� r r� rh r r r+ r@ rF rt �r! Z appropriater* r% r% r& r� � s: � � � � z#HTMLTokenizer.rcdataEndTagNameStatec C sP | j �� }|dkr"d| _| j| _n*| j�td dd�� | j �|� | j | _dS r� ) r r? r� �rawtextEndTagOpenStater r+ r@ r rF rw rz r% r% r& rv � s z&HTMLTokenizer.rawtextLessThanSignStatec C sX | j �� }|tv r*| j|7 _| j| _n*| j�td dd�� | j � |� | j | _dS r� ) r r? r r� �rawtextEndTagNameStater r+ r@ r rF rw rz r% r% r& r� s z$HTMLTokenizer.rawtextEndTagOpenStatec C s | j o | j d �� | j�� k}| j�� }|tv rT|rTtd | jg dd�| _ | j| _n�|dkr�|r�td | jg dd�| _ | j | _n||dkr�|r�td | jg dd�| _ | � � | j | _nH|t v r�| j|7 _n0| j �td d| j d �� | j�|� | j| _d S r� )r r� r� r r? r r r� r r� rh r r r+ r@ rF rw r� r% r% r& r� s: � � � � z$HTMLTokenizer.rawtextEndTagNameStatec C sx | j �� }|dkr"d| _| j| _nR|dkrJ| j�td dd�� | j| _n*| j�td dd�� | j � |� | j | _dS ) Nr} r3 r| rU z<!r( rN T) r r? r� �scriptDataEndTagOpenStater r+ r@ r �scriptDataEscapeStartStaterF ry rz r% r% r& rx , s z)HTMLTokenizer.scriptDataLessThanSignStatec C sX | j �� }|tv r*| j|7 _| j| _n*| j�td dd�� | j � |� | j | _dS r� ) r r? r r� �scriptDataEndTagNameStater r+ r@ r rF ry rz r% r% r& r� : s z'HTMLTokenizer.scriptDataEndTagOpenStatec C s | j o | j d �� | j�� k}| j�� }|tv rT|rTtd | jg dd�| _ | j| _n�|dkr�|r�td | jg dd�| _ | j | _n||dkr�|r�td | jg dd�| _ | � � | j | _nH|t v r�| j|7 _n0| j �td d| j d �� | j�|� | j| _d S r� )r r� r� r r? r r r� r r� rh r r r+ r@ rF ry r� r% r% r& r� E s: � � � � z'HTMLTokenizer.scriptDataEndTagNameStatec C sJ | j �� }|dkr2| j�td dd�� | j| _n| j �|� | j| _dS �N�-rU r( T) r r? r+ r@ r � scriptDataEscapeStartDashStater rF ry rz r% r% r& r� a s z(HTMLTokenizer.scriptDataEscapeStartStatec C sJ | j �� }|dkr2| j�td dd�� | j| _n| j �|� | j| _dS r� ) r r? r+ r@ r � scriptDataEscapedDashDashStater rF ry rz r% r% r& r� k s z,HTMLTokenizer.scriptDataEscapeStartDashStatec C s� | j �� }|dkr2| j�td dd�� | j| _n�|dkrD| j| _nn|dkrz| j�td dd�� | j�td dd�� n8|tkr�| j | _n&| j � d �}| j�td || d�� d S ) Nr� rU r( rN ri r' rj r7 )rN r� ri T) r r? r+ r@ r �scriptDataEscapedDashStater �"scriptDataEscapedLessThanSignStater r rn ro r% r% r& �scriptDataEscapedStateu s( � � �z$HTMLTokenizer.scriptDataEscapedStatec C s� | j �� }|dkr2| j�td dd�� | j| _n�|dkrD| j| _nn|dkr�| j�td dd�� | j�td dd�� | j| _n0|t kr�| j | _n | j�td |d�� | j| _d S ) Nr� rU r( rN ri r' rj r7 T) r r? r+ r@ r r� r r� r� r r rz r% r% r& r� � s& � � z(HTMLTokenizer.scriptDataEscapedDashStatec C s� | j �� }|dkr*| j�td dd�� n�|dkr<| j| _n�|dkrd| j�td dd�� | j| _nn|dkr�| j�td dd�� | j�td d d�� | j| _n0|t kr�| j | _n | j�td |d�� | j| _d S ) Nr� rU r( rN r~ ri r' rj r7 T) r r? r+ r@ r r� r ry r� r r rz r% r% r& r� � s* � � z,HTMLTokenizer.scriptDataEscapedDashDashStatec C s� | j �� }|dkr"d| _| j| _n\|tv rT| j�td d| d�� || _| j | _n*| j�td dd�� | j � |� | j | _dS r� ) r r? r� � scriptDataEscapedEndTagOpenStater r r+ r@ r � scriptDataDoubleEscapeStartStaterF r� rz r% r% r& r� � s z0HTMLTokenizer.scriptDataEscapedLessThanSignStatec C sP | j �� }|tv r"|| _| j| _n*| j�td dd�� | j � |� | j | _dS r� ) r r? r r� � scriptDataEscapedEndTagNameStater r+ r@ r rF r� rz r% r% r& r� � s z.HTMLTokenizer.scriptDataEscapedEndTagOpenStatec C s | j o | j d �� | j�� k}| j�� }|tv rT|rTtd | jg dd�| _ | j| _n�|dkr�|r�td | jg dd�| _ | j | _n||dkr�|r�td | jg dd�| _ | � � | j | _nH|t v r�| j|7 _n0| j �td d| j d �� | j�|� | j| _d S r� )r r� r� r r? r r r� r r� rh r r r+ r@ rF r� r� r% r% r& r� � s: � � � � z.HTMLTokenizer.scriptDataEscapedEndTagNameStatec C s� | j �� }|ttd�B v rR| j�td |d�� | j�� dkrH| j | _ q�| j | _ nB|t v r�| j�td |d�� | j|7 _n| j � |� | j | _ dS �N)r} r~ rU r( ZscriptT)r r? r rC r+ r@ r r� r� � scriptDataDoubleEscapedStater r� r rF rz r% r% r& r� � s z.HTMLTokenizer.scriptDataDoubleEscapeStartStatec C s� | j �� }|dkr2| j�td dd�� | j| _n�|dkrZ| j�td dd�� | j| _nt|dkr�| j�td dd�� | j�td dd�� n>|tkr�| j�td d d�� | j | _n| j�td |d�� d S � Nr� rU r( rN ri r' rj r7 �eof-in-script-in-scriptT) r r? r+ r@ r � scriptDataDoubleEscapedDashStater �(scriptDataDoubleEscapedLessThanSignStater r rz r% r% r& r� � s* � � � z*HTMLTokenizer.scriptDataDoubleEscapedStatec C s� | j �� }|dkr2| j�td dd�� | j| _n�|dkrZ| j�td dd�� | j| _n�|dkr�| j�td dd�� | j�td dd�� | j| _nF|t kr�| j�td d d�� | j | _n | j�td |d�� | j| _d S r� ) r r? r+ r@ r �$scriptDataDoubleEscapedDashDashStater r� r� r r rz r% r% r& r� s. � � � z.HTMLTokenizer.scriptDataDoubleEscapedDashStatec C s | j �� }|dkr*| j�td dd�� n�|dkrR| j�td dd�� | j| _n�|dkrz| j�td dd�� | j| _n�|dkr�| j�td dd�� | j�td d d�� | j| _nF|t kr�| j�td d d�� | j | _n | j�td |d�� | j| _d S ) Nr� rU r( rN r~ ri r' rj r7 r� T) r r? r+ r@ r r� r ry r� r r rz r% r% r& r� % s2 � � � z2HTMLTokenizer.scriptDataDoubleEscapedDashDashStatec C sP | j �� }|dkr8| j�td dd�� d| _| j| _n| j �|� | j | _dS )Nr} rU r( r3 T) r r? r+ r@ r r� � scriptDataDoubleEscapeEndStater rF r� rz r% r% r& r� > s z6HTMLTokenizer.scriptDataDoubleEscapedLessThanSignStatec C s� | j �� }|ttd�B v rR| j�td |d�� | j�� dkrH| j | _ q�| j | _ nB|t v r�| j�td |d�� | j|7 _n| j � |� | j | _ dS r� )r r? r rC r+ r@ r r� r� r� r r� r rF rz r% r% r& r� I s z,HTMLTokenizer.scriptDataDoubleEscapeEndStatec C s0 | j �� }|tv r$| j �td� �n|tv rJ| jd �|dg� | j| _n�|dkr\| � � n�|dkrn| j | _n�|dv r�| j �t d dd �� | jd �|dg� | j| _n�|d kr�| j �t d d d �� | jd �d dg� | j| _nF|t u �r| j �t d d d �� | j| _n | jd �|dg� | j| _dS )NTr* r3 r~ r} )�'�"rS rN r' �#invalid-character-in-attribute-namer( ri rj r7 z#expected-attribute-name-but-got-eof)r r? r rn r r r@ �attributeNameStater rh r� r+ r r r rz r% r% r& r� Y s< � � � z&HTMLTokenizer.beforeAttributeNameStatec C s� | j �� }d}d}|dkr&| j| _�n.|tv r\| jd d d || j �td� 7 < d}n�|dkrjd}n�|tv r|| j| _n�|dkr�| j | _n�|d kr�| j � t d d d �� | jd d d d 7 < d}n�|dv �r | j � t d dd �� | jd d d |7 < d}nH|t u �r6| j � t d dd �� | j| _n | jd d d |7 < d}|�r�| jd d d �t�| jd d d< | jd d d� D ]>\}}| jd d d |k�r�| j � t d dd �� �qҐq�|�r�| �� dS )NTFrS r* rP r r~ r} ri r' rj r( r7 �r� r� rN r� zeof-in-attribute-namezduplicate-attribute)r r? �beforeAttributeValueStater r r rn r �afterAttributeNameStater� r+ r@ r r r rc r rh )r! r* ZleavingThisStateZ emitTokenr_ �_r% r% r& r� w s^ � � � � � � z HTMLTokenizer.attributeNameStatec C sD | j �� }|tv r$| j �td� �n |dkr8| j| _�n|dkrJ| �� n�|tv rp| jd � |dg� | j | _n�|dkr�| j | _n�|dkr�| j � t d d d �� | jd � d dg� | j | _n�|d v r�| j � t d d d �� | jd � |dg� | j | _nF|tu �r$| j � t d dd �� | j| _n | jd � |dg� | j | _dS )NTrS r~ r* r3 r} ri r' rj r( r7 r� z&invalid-character-after-attribute-namez expected-end-of-tag-but-got-eof)r r? r rn r� r rh r r r@ r� r� r+ r r r rz r% r% r& r� � s@ � � � z%HTMLTokenizer.afterAttributeNameStatec C sh | j �� }|tv r$| j �td� �n@|dkr8| j| _�n,|dkrX| j| _| j �|� �n |dkrj| j| _n�|dkr�| j � t d dd�� | � � n�|d kr�| j � t d d d�� | j d d d d7 < | j| _n�|dv �r| j � t d dd�� | j d d d |7 < | j| _nL|tu �rB| j � t d dd�� | j| _n"| j d d d |7 < | j| _dS )NTr� rM r� r~ r' z.expected-attribute-value-but-got-right-bracketr( ri rj r* rP r r7 )rS rN �`z"equals-in-unquoted-attribute-valuez$expected-attribute-value-but-got-eof)r r? r rn � attributeValueDoubleQuotedStater �attributeValueUnQuotedStaterF � attributeValueSingleQuotedStater+ r@ r rh r r r rz r% r% r& r� � sF � � � � z'HTMLTokenizer.beforeAttributeValueStatec C s� | j �� }|dkr | j| _n�|dkr0| �d� n�|dkrj| j�td dd�� | jd d d d 7 < nN|t u r�| j�td d d�� | j | _n&| jd d d || j � d � 7 < d S )Nr� rM ri r' rj r( r* rP r r7 z#eof-in-attribute-value-double-quote)r� rM ri T� r r? �afterAttributeValueStater r^ r+ r@ r r r r rn rz r% r% r& r� � s& � � �z-HTMLTokenizer.attributeValueDoubleQuotedStatec C s� | j �� }|dkr | j| _n�|dkr0| �d� n�|dkrj| j�td dd�� | jd d d d 7 < nN|t u r�| j�td d d�� | j | _n&| jd d d || j � d � 7 < d S )Nr� rM ri r' rj r( r* rP r r7 z#eof-in-attribute-value-single-quote)r� rM ri Tr� rz r% r% r& r� s& � � �z-HTMLTokenizer.attributeValueSingleQuotedStatec C s | j �� }|tv r | j| _n�|dkr0| �d� n�|dkrB| �� n�|dv r|| j�t d dd�� | j d d d |7 < n�|d kr�| j�t d d d�� | j d d d d 7 < nV|t u r�| j�t d d d�� | j | _n.| j d d d || j � td�tB � 7 < dS )NrM r~ )r� r� rS rN r� r' z0unexpected-character-in-unquoted-attribute-valuer( r* rP r ri rj r7 z eof-in-attribute-value-no-quotes)rM r~ r� r� rS rN r� ri T)r r? r r� r r^ rh r+ r@ r r r r rn rC rz r% r% r& r� s4 � � � � z)HTMLTokenizer.attributeValueUnQuotedStatec C s� | j �� }|tv r | j| _n�|dkr.| �� np|dkr@| j| _n^|tu rt| j� t d dd�� | j � |� | j | _n*| j� t d dd�� | j � |� | j| _dS )Nr~ r} r' z$unexpected-EOF-after-attribute-valuer( z*unexpected-character-after-attribute-valueT) r r? r r� r rh r� r r+ r@ r rF r rz r% r% r& r� . s& � � z&HTMLTokenizer.afterAttributeValueStatec C s� | j �� }|dkr&d| jd< | �� n^|tu rZ| j�td dd�� | j �|� | j | _ n*| j�td dd�� | j �|� | j | _ dS )Nr~ Trb r' z#unexpected-EOF-after-solidus-in-tagr( z)unexpected-character-after-solidus-in-tag) r r? r rh r r+ r@ r rF r r r� rz r% r% r& r� B s � � z&HTMLTokenizer.selfClosingStartTagStatec C sD | j �d�}|�dd�}| j�td |d�� | j �� | j| _dS )Nr~ ri r7 �Commentr( T) r rn �replacer+ r@ r r? r r rz r% r% r& r� T s � z HTMLTokenizer.bogusCommentStatec C s� | j �� g}|d dkrR|�| j �� � |d dkrPtd dd�| _| j| _dS n�|d dv r�d}dD ](}|�| j �� � |d |vrfd } q�qf|r�td dd d dd �| _| j| _dS n�|d d k�rD| jd u�rD| jj j �rD| jj j d j | jj j k�rDd}d D ].}|�| j �� � |d |k�rd } �q2�q|�rD| j | _dS | j�td dd�� |�rt| j �|�� � �qZ| j| _dS )NrP r� r� r3 r( T)�d�D))�o�O�rJ �C��t�T��y�Y��p�P��e�EFZDoctype)r) r_ �publicId�systemId�correct�[)r� r� �Ar� r� r� r' zexpected-dashes-or-doctype)r r? r@ r r �commentStartStater � doctypeStater ZtreeZ openElements� namespaceZdefaultNamespace�cdataSectionStater+ rF r- r� )r! rI �matched�expectedr% r% r& r� c sZ ���� �z(HTMLTokenizer.markupDeclarationOpenStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� | j�| j� | j| _nP|t u r�| j�td d d�� | j�| j� | j| _n| jd |7 < | j | _d S ) Nr� ri r' rj r( r* r7 r~ �incorrect-comment�eof-in-commentT) r r? �commentStartDashStater r+ r@ r r r r � commentStaterz r% r% r& r� � s. � � � z HTMLTokenizer.commentStartStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� | j�| j� | j| _nT|t u r�| j�td d d�� | j�| j� | j| _n | jd d| 7 < | j | _d S ) Nr� ri r' rj r( r* � -�r~ r� r� T) r r? �commentEndStater r+ r@ r r r r r� rz r% r% r& r� � s. � � � z#HTMLTokenizer.commentStartDashStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < nT|tu r�| j�td dd�� | j�| j� | j | _n | jd || j � d � 7 < d S ) Nr� ri r' rj r( r* r7 r� )r� ri T) r r? �commentEndDashStater r+ r@ r r r r rn rz r% r% r& r� � s$ � � �zHTMLTokenizer.commentStatec C s� | j �� }|dkr | j| _n�|dkrV| j�td dd�� | jd d7 < | j| _nT|t u r�| j�td dd�� | j�| j� | j | _n | jd d| 7 < | j| _d S ) Nr� ri r' rj r( r* r� zeof-in-comment-end-dashT) r r? r� r r+ r@ r r r� r r rz r% r% r& r� � s$ � � z!HTMLTokenizer.commentEndDashStatec C s, | j �� }|dkr*| j�| j� | j| _n�|dkrd| j�td dd�� | jd d7 < | j| _n�|dkr�| j�td d d�� | j | _n�|d kr�| j�td d d�� | jd |7 < nj|t u r�| j�td d d�� | j�| j� | j| _n4| j�td d d�� | jd d| 7 < | j| _dS )Nr~ ri r' rj r( r* u --�r| z,unexpected-bang-after-double-dash-in-commentr� z,unexpected-dash-after-double-dash-in-commentzeof-in-comment-double-dashzunexpected-char-in-commentz--T) r r? r+ r@ r r r r r� �commentEndBangStater rz r% r% r& r� � s@ � � � � �z HTMLTokenizer.commentEndStatec C s� | j �� }|dkr*| j�| j� | j| _n�|dkrN| jd d7 < | j| _n�|dkr�| j�td dd�� | jd d 7 < | j | _nT|t u r�| j�td d d�� | j�| j� | j| _n | jd d| 7 < | j | _d S ) Nr~ r� r* z--!ri r' rj r( u --!�z eof-in-comment-end-bang-stateT) r r? r+ r@ r r r r� r r� r rz r% r% r& r� s, � � z!HTMLTokenizer.commentEndBangStatec C s� | j �� }|tv r | j| _nj|tu r\| j�td dd�� d| j d< | j�| j � | j | _n*| j�td dd�� | j � |� | j| _dS )Nr' �!expected-doctype-name-but-got-eofr( Fr� zneed-space-after-doctypeT) r r? r �beforeDoctypeNameStater r r+ r@ r r r rF rz r% r% r& r� s � � zHTMLTokenizer.doctypeStatec C s� | j �� }|tv rn�|dkrT| j�td dd�� d| jd< | j�| j� | j| _n�|dkr�| j�td dd�� d | jd < | j | _nR|t u r�| j�td d d�� d| jd< | j�| j� | j| _n|| jd < | j | _d S ) Nr~ r' z+expected-doctype-name-but-got-right-bracketr( Fr� ri rj r7 r_ r� T) r r? r r+ r@ r r r r �doctypeNameStater rz r% r% r& r� * s4 � � � z$HTMLTokenizer.beforeDoctypeNameStatec C s | j �� }|tv r2| jd �t�| jd< | j| _n�|dkrh| jd �t�| jd< | j� | j� | j | _n�|dkr�| j� t d dd�� | jd d7 < | j | _nh|t u r�| j� t d dd�� d | jd < | jd �t�| jd< | j� | j� | j | _n| jd |7 < d S ) Nr_ r~ ri r' rj r( r7 zeof-in-doctype-nameFr� T)r r? r r rc r �afterDoctypeNameStater r+ r@ r r r� r rz r% r% r& r� D s0 � � z HTMLTokenizer.doctypeNameStatec C sH | j �� }|tv r�n.|dkr8| j�| j� | j| _�n |tu r�d| jd< | j � |� | j�t d dd�� | j�| j� | j| _n�|dv r�d}d D ] }| j �� }||vr�d} q�q�|r�| j | _dS nD|d v �r d}d D ] }| j �� }||vr�d} q�q�|�r | j | _dS | j � |� | j�t d d d |id�� d| jd< | j | _dS )Nr~ Fr� r' �eof-in-doctyper( r� T))�u�U)�b�B)�l�L)�i�Ir� ��s�S)r� r� r� r� )�m�Mz*expected-space-or-right-bracket-in-doctyper* r5 )r r? r r+ r@ r r r r rF r � afterDoctypePublicKeywordState� afterDoctypeSystemKeywordState�bogusDoctypeState)r! r* r� r� r% r% r& r� ] sT � � z#HTMLTokenizer.afterDoctypeNameStatec C s� | j �� }|tv r | j| _n�|dv rP| j�td dd�� | j �|� | j| _nT|t u r�| j�td dd�� d| j d< | j�| j � | j | _n| j �|� | j| _dS � N)r� r� r' �unexpected-char-in-doctyper( r� Fr� T) r r? r �"beforeDoctypePublicIdentifierStater r+ r@ r rF r r r rz r% r% r& r� � s& � � z,HTMLTokenizer.afterDoctypePublicKeywordStatec C s� | j �� }|tv rn�|dkr0d| jd< | j| _n�|dkrLd| jd< | j| _n�|dkr�| j�t d dd�� d | jd < | j�| j� | j | _nh|t u r�| j�t d d d�� d | jd < | j�| j� | j | _n(| j�t d d d�� d | jd < | j | _d S )Nr� r3 r� r� r~ r' �unexpected-end-of-doctyper( Fr� r� r� T) r r? r r �(doctypePublicIdentifierDoubleQuotedStater �(doctypePublicIdentifierSingleQuotedStater+ r@ r r r r� rz r% r% r& r� � s: � � � z0HTMLTokenizer.beforeDoctypePublicIdentifierStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� d | jd < | j�| j� | j| _nR|t u r�| j�td d d�� d | jd < | j�| j� | j| _n| jd |7 < d S )Nr� ri r' rj r( r� r7 r~ r� Fr� r� T� r r? �!afterDoctypePublicIdentifierStater r+ r@ r r r r rz r% r% r& r� � s0 � � � z6HTMLTokenizer.doctypePublicIdentifierDoubleQuotedStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� d | jd < | j�| j� | j| _nR|t u r�| j�td d d�� d | jd < | j�| j� | j| _n| jd |7 < d S )Nr� ri r' rj r( r� r7 r~ r� Fr� r� Tr� rz r% r% r& r� � s0 � � � z6HTMLTokenizer.doctypePublicIdentifierSingleQuotedStatec C s | j �� }|tv r | j| _n�|dkr<| j�| j� | j| _n�|dkrn| j�t d dd�� d| jd< | j | _n�|dkr�| j�t d dd�� d| jd< | j | _nh|t u r�| j�t d d d�� d | jd < | j�| j� | j| _n(| j�t d dd�� d | jd < | j | _d S ) Nr~ r� r' r� r( r3 r� r� r� Fr� T)r r? r �-betweenDoctypePublicAndSystemIdentifiersStater r+ r@ r r r �(doctypeSystemIdentifierDoubleQuotedState�(doctypeSystemIdentifierSingleQuotedStater r� rz r% r% r& r� � s> � � � � z/HTMLTokenizer.afterDoctypePublicIdentifierStatec C s� | j �� }|tv rn�|dkr4| j�| j� | j| _n�|dkrPd| jd< | j| _n�|dkrld| jd< | j | _nh|t kr�| j�t d dd�� d | jd < | j�| j� | j| _n(| j�t d d d�� d | jd < | j | _d S ) Nr~ r� r3 r� r� r' r� r( Fr� r� T) r r? r r+ r@ r r r r� r� r r r� rz r% r% r& r� s2 � � z;HTMLTokenizer.betweenDoctypePublicAndSystemIdentifiersStatec C s� | j �� }|tv r | j| _n�|dv rP| j�td dd�� | j �|� | j| _nT|t u r�| j�td dd�� d| j d< | j�| j � | j | _n| j �|� | j| _dS r� ) r r? r �"beforeDoctypeSystemIdentifierStater r+ r@ r rF r r r rz r% r% r& r� ) s& � � z,HTMLTokenizer.afterDoctypeSystemKeywordStatec C s� | j �� }|tv rn�|dkr0d| jd< | j| _n�|dkrLd| jd< | j| _n�|dkr�| j�t d dd�� d | jd < | j�| j� | j | _nh|t u r�| j�t d d d�� d | jd < | j�| j� | j | _n(| j�t d dd�� d | jd < | j | _d S ) Nr� r3 r� r� r~ r' r� r( Fr� r� T) r r? r r r� r r� r+ r@ r r r r� rz r% r% r& r� = s: � � � z0HTMLTokenizer.beforeDoctypeSystemIdentifierStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� d | jd < | j�| j� | j| _nR|t u r�| j�td d d�� d | jd < | j�| j� | j| _n| jd |7 < d S )Nr� ri r' rj r( r� r7 r~ r� Fr� r� T� r r? �!afterDoctypeSystemIdentifierStater r+ r@ r r r r rz r% r% r& r� Z s0 � � � z6HTMLTokenizer.doctypeSystemIdentifierDoubleQuotedStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� d | jd < | j�| j� | j| _nR|t u r�| j�td d d�� d | jd < | j�| j� | j| _n| jd |7 < d S )Nr� ri r' rj r( r� r7 r~ r� Fr� r� Tr� rz r% r% r& r� r s0 � � � z6HTMLTokenizer.doctypeSystemIdentifierSingleQuotedStatec C s� | j �� }|tv rn~|dkr4| j�| j� | j| _n^|tu rt| j�t d dd�� d| jd< | j�| j� | j| _n | j�t d dd�� | j | _dS ) Nr~ r' r� r( Fr� r� T) r r? r r+ r@ r r r r r r� rz r% r% r& r� � s$ � �z/HTMLTokenizer.afterDoctypeSystemIdentifierStatec C sZ | j �� }|dkr*| j�| j� | j| _n,|tu rV| j �|� | j�| j� | j| _n dS )Nr~ T) r r? r+ r@ r r r r rF rz r% r% r& r� � s z HTMLTokenizer.bogusDoctypeStatec C s� g }|� | j�d�� |� | j�d�� | j�� }|tkr>qvq|d dd � dkrj|d d d� |d< qvq|� |� qd�|�}|�d�}|dkr�t|�D ]}| j� t d d d �� q�|� dd �}|r�| j� t d |d �� | j | _ dS )N�]r~ rP ���z]]r3 ri r r' rj r( r7 rU T) r@ r rn r? r rB �count�ranger+ r r� r r )r! r* r? Z nullCountr� r% r% r& r� � s0 � �z HTMLTokenizer.cdataSectionState)N)NF)N�__name__� __module__� __qualname__�__doc__r r0 rL r] r^ rh r rl rt rr rw ry r{ rm r� r� rs r� r� rv r� r� rx r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� � __classcell__r% r% r# r& r s� H P # 6 "- 3 r N) Z __future__r r r Zpip._vendor.sixr rD � collectionsr r �sysr Z constantsr r r r r r r r r r Z _inputstreamr Z_trier rV �dictrd �objectr r% r% r% r& �<module> s
💾 Save Changes
❌ Cancel