a
lj$ @ s d dl mZmZmZ d dlmZmZmZ d dlm Z m
Z
m Z d dl m
Z
mZmZ d dlmZmZmZ G dd deZG dd d eZG d
d d eZG d d
d
eZG dd deZG dd deZG dd deZdS ) )EUCTW_CHAR_TO_FREQ_ORDEREUCTW_TABLE_SIZE EUCTW_TYPICAL_DISTRIBUTION_RATIO)EUCKR_CHAR_TO_FREQ_ORDEREUCKR_TABLE_SIZE EUCKR_TYPICAL_DISTRIBUTION_RATIO)GB2312_CHAR_TO_FREQ_ORDERGB2312_TABLE_SIZE!GB2312_TYPICAL_DISTRIBUTION_RATIO)BIG5_CHAR_TO_FREQ_ORDERBIG5_TABLE_SIZE BIG5_TYPICAL_DISTRIBUTION_RATIO)JIS_CHAR_TO_FREQ_ORDERJIS_TABLE_SIZE JIS_TYPICAL_DISTRIBUTION_RATIOc @ sL e Zd ZdZdZdZdZdd Zdd Zd d
Z d d Z
d
d Z dd Z dS )CharDistributionAnalysisi gGz?g{Gz? c C s0 d | _ d | _d | _d | _d | _d | _| d S N)_char_to_freq_order _table_sizetypical_distribution_ratio_done _total_chars _freq_charsresetself r H/usr/lib/python3.9/site-packages/pip/_vendor/chardet/chardistribution.py__init__. s z!CharDistributionAnalysis.__init__c C s d| _ d| _d| _dS )z reset analyser, clear any stateF N)r r r r r r r r = s z CharDistributionAnalysis.resetc C sX |dkr| |}nd}|dkrT| jd7 _|| jk rTd| j| krT| jd7 _dS )z"feed a character with known length r r i N) get_orderr r r r )r charZchar_lenorderr r r feedF s
z CharDistributionAnalysis.feedc C sT | j dks| j| jkr | jS | j | jkrN| j| j | j | j }|| jk rN|S | jS )z(return confidence based on existing datar )r r MINIMUM_DATA_THRESHOLDSURE_NOr SURE_YES)r rr r r get_confidenceT s
z'CharDistributionAnalysis.get_confidencec C s | j | jkS r )r ENOUGH_DATA_THRESHOLDr r r r got_enough_datad s z(CharDistributionAnalysis.got_enough_datac C s dS )Nr" r )r byte_strr r r r# i s z"CharDistributionAnalysis.get_orderN)
__name__
__module__ __qualname__r, r) r( r' r r r& r+ r- r# r r r r r ( s r c s$ e Zd Z fddZdd Z ZS )EUCTWDistributionAnalysisc s$ t t| t| _t| _t| _d S r ) superr2 r r r r r r r r __class__r r r r s z"EUCTWDistributionAnalysis.__init__c C s0 |d }|dkr(d|d |d d S dS d S )Nr ^ r r" r r r.
first_charr r r r# x s z#EUCTWDistributionAnalysis.get_orderr/ r0 r1 r r#
__classcell__r r r4 r r2 q s r2 c s$ e Zd Z fddZdd Z ZS )EUCKRDistributionAnalysisc s$ t t| t| _t| _t| _d S r ) r3 r= r r r r r r r r r4 r r r s z"EUCKRDistributionAnalysis.__init__c C s0 |d }|dkr(d|d |d d S dS d S )Nr r7 r r8 r" r r9 r r r r# s z#EUCKRDistributionAnalysis.get_orderr; r r r4 r r= s r= c s$ e Zd Z fddZdd Z ZS )GB2312DistributionAnalysisc s$ t t| t| _t| _t| _d S r ) r3 r? r r r r r r
r r r4 r r r s z#GB2312DistributionAnalysis.__init__c C s> |d |d }}|dkr6|dkr6d|d | d S dS d S )Nr r r> r8 r7 r" r r r. r: second_charr r r r# s z$GB2312DistributionAnalysis.get_orderr; r r r4 r r? s r? c s$ e Zd Z fddZdd Z ZS )Big5DistributionAnalysisc s$ t t| t| _t| _t| _d S r ) r3 rB r r r r r r
r r r4 r r r s z!Big5DistributionAnalysis.__init__c C sX |d |d }}|dkrP|dkr:d|d | d d S d|d | d S ndS d S ) Nr r r8 ? @ r" r r@ r r r r# s z"Big5DistributionAnalysis.get_orderr; r r r4 r rB s rB c s$ e Zd Z fddZdd Z ZS )SJISDistributionAnalysisc s$ t t| t| _t| _t| _d S r ) r3 rG r r r r r r r r r4 r r r s z!SJISDistributionAnalysis.__init__c C sr |d |d }}|dkr0|dkr0d|d }n&|dkrR|dkrRd|d d }nd S || d
}|d krnd }|S ) Nr r r" rF r )r r. r: rA r% r r r r# s z"SJISDistributionAnalysis.get_orderr; r r r4 r rG s rG c s$ e Zd Z fddZdd Z ZS )EUCJPDistributionAnalysisc s$ t t| t| _t| _t| _d S r ) r3 rO r r r r r r r r r4 r r r s z"EUCJPDistributionAnalysis.__init__c C s0 |d }|dkr(d|d |d d S dS d S )Nr r7 r8 r r" r )r r. r$ r r r r# s z#EUCJPDistributionAnalysis.get_orderr; r r r4 r rO s rO N) Z euctwfreqr r r Z euckrfreqr r r Z
gb2312freqr r r
Zbig5freqr r r
Zjisfreqr r r objectr r2 r= r? rB rG rO r r r r s I