a
=*f L @ s d Z dZdZddlZddlZddlmZ ddlmZm Z m
Z
m Z m Z m
Z
ddlmZ dd lmZ d
d lmZ eeZd Zed
ejZejdddZd ddZ dd fddZ G dd de Z G dd de Z G dd d e Z!dS ) z'Cyril Jaquier and Fail2Ban Contributorsz Copyright (c) 2004 Cyril JaquierZGPL N)Lock )re DateTemplateDatePatternRegex
DateTai64n DateEpochRE_EPOCH_PATTERN)validateTimeZone)Utils ) getLogger z(?D r% c C sb || j }t|}|s^|t| d| j}t| dr>t|}|s^t| dsVt|}nt|}|S )Nr )namer r getattrregexhasattrr )r wrapr' template2r) r r r _getAnchoredTemplateD s
r- c @ sH e Zd ZdZdd Zedd Zdd Zg dZed d
Z d d Z
d
S )DateDetectorCachez7Implements the caching of the default templates list.
c C s t | _t | _d S N)r _DateDetectorCache__locklist _DateDetectorCache__templatesselfr r r __init__\ s zDateDetectorCache.__init__c C s^ | j r | j S | j8 | j r.| j W d S | | j W d S 1 sP0 Y dS )6List of template instances managed by the detector.
N)r2 r0 _addDefaultTemplater3 r r r templates` s zDateDetectorCache.templatesc C s` |j }|dsL|dsLt|drLt|}|j |krLd|_| jd | | jd | dS )z&Cache Fail2Ban's default template.
r" ^r) g Y@r r N)r'
startswithr* r- weight _DateDetectorCache__tmpcacheappend)r4 r r' r, r r r _cacheTemplatel s
z DateDetectorCache._cacheTemplate)zF%ExY(?P<_sep>[-/.])%m(?P=_sep)%d(?:T| ?)%H:%M:%S(?:[.,]%f)?(?:\s*%z)?z)(?:%a )?%b %d %k:%M:%S(?:\.%f)?(?: %ExY)?z$(?:%a )?%b %d %ExY %k:%M:%S(?:\.%f)?z1%d(?P<_sep>[-/])%m(?P=_sep)(?:%ExY|%Exy) %k:%M:%Sz=%d(?P<_sep>[-/])%b(?P=_sep)%ExY[ :]?%H:%M:%S(?:\.%f)?(?: %z)?z%m/%d/%ExY:%H:%M:%Sz %m-%d-%ExY %k:%M:%S(?:\.%f)?r z{^LN-BEG}%H:%M:%Sz^<%m/%d/%Exy@%H:%M:%S>z%Exy%Exm%Exd ?%H:%M:%Sz%b %d, %ExY %I:%M:%S %pz^%b-%d-%Exy %k:%M:%Sz6%ExY%Exm%Exd(?:T| ?)%ExH%ExM%ExS(?:[.,]%f)?(?:\s*%z)?z1(?:%Z )?(?:%a )?%b %d %k:%M:%S(?:\.%f)?(?: %ExY)?z1(?:%z )?(?:%a )?%b %d %k:%M:%S(?:\.%f)?(?: %ExY)?r c C s< t tjd tr6ttjD ]\}}t|}|tj|< qtjS )Nr )
isinstancer. DEFAULT_TEMPLATESstr enumerater )r4 idtr r r defaultTemplates s
z"DateDetectorCache.defaultTemplatesc C s> g g f| _ | jD ]}| | q| j d | j d | _| ` dS )z r2 )r4 rD r r r r7 s
z%DateDetectorCache._addDefaultTemplateN) __name__
__module__ __qualname____doc__r5 propertyr8 r> r@ rE r7 r r r r r. Y s
5
r. c @ s0 e Zd ZdZdZdd Zedd Zdd Zd S )
DateDetectorTemplateztUsed for "shallow copy" of the template object.
Prevents collectively usage of hits/lastUsed in cached templates
r hitslastUseddistancec C s || _ d| _d| _d| _d S )Nr rL )r4 r r r r r5 s z DateDetectorTemplate.__init__c C s | j | jj td| j S )Nr )rM r r; maxrO r3 r r r r; s zDateDetectorTemplate.weightc C s t | j|S )zF Returns attribute of template (called for parameters not in slots)
)r( r )r4 r' r r r __getattr__ s z DateDetectorTemplate.__getattr__N) rF rG rH rI __slots__r5 rJ r; rR r r r r rK s
rK c @ sz e Zd ZdZe Zdd ZdddZdd Zdd d Z e
d
d Z dd Z e
dd Z
e
jdd Z
dddZdd Zd S ) DateDetectorzjManages one or more date templates to find a date within a log line.
Attributes
----------
templates
c C s> t | _t | _d| _d| _d| _d| _d| _d | _ d | _
d S )Ni, )r N)rP NrP r ) r1 _DateDetector__templatesr _DateDetector__known_names_DateDetector__unusedTime_DateDetector__lastPos_DateDetector__lastEndPos_DateDetector__lastTemplIdx_DateDetector__firstUnusedZ_DateDetector__preMatch_DateDetector__default_tzr3 r r r r5 s zDateDetector.__init__Fc C sD |j }|| jv r$|rd S td| | j| | jt| d S )Nz(There is already a template with name %s)r' rV
ValueErroraddrU r= rK )r4 r ignoreDupr' r r r _appendTemplate s
z DateDetector._appendTemplatec s t |tr| }d|vr"| t }|s dv rR fdd}| | dS d v rl| j|dd dS d krtd
}n
t| }t | | | t
d t |d d
|j t
dt |d d
|j dS )a Add a date template to manage and use in search of dates.
Parameters
----------
template : DateTemplate or str
Can be either a `DateTemplate` instance, or a string which will
be used as the pattern for the `DatePatternRegex` template. The
template will then be added to the detector.
Raises
------
ValueError
If a template already exists with the same name.
r )r" z {DEFAULT}c s dkr| j tj@ S d S r! )flagsr
LINE_BEGIN)r r r r r% r& z-DateDetector.appendTemplate..Nz{DATE}F)preMatch allDefaultsz{NONE}z{UNB}^z date pattern `%r`: `%s`r z date pattern regex for %r: %s)r? rA r r r addDefaultTemplater r r` logSysinfor( r' debugr) )r4 r r Zfltr rc r appendTemplate s2
zDateDetector.appendTemplateNTc sh t | jdk}|rtjjntjjD ]@}|dur8||s8q" durTt| fddd}| j||d q"dS )z0Add Fail2Ban's default set of date templates.
r Nc s t fddS )Nc s
t S r/ )r ZunboundPattern)mr# r r r% 7 r& zCDateDetector.addDefaultTemplate....)RE_DATE_PREMATCHsubr# rd r# r r% 7 r& z1DateDetector.addDefaultTemplate..)r+ )r_ )lenrU rT _defCacher8 rE r- r` )r4 ZfilterTemplaterd re r_ r r ro r rg * s
z DateDetector.addDefaultTemplatec C s | j S )r6 )rU r3 r r r r8 ; s zDateDetector.templatesc
C s t | js| t tkr$tjndd }|td d| d}d}d}| j}|t | jk rd| j| }|j}|j t
j t
j B @ r|td d| |
|}|}n4| jd | jd } }
|td d
|| |
|| d | | jd || |
||
|
d | jd
|| d | | jd ksJ|| | d | jd kr| jd s||
|
d | jd ks||
d |
| jd kr| jd s|td d || |
|
|| |
}n |td d
|
|}|}|rZ| } | }
t | jdks0|j t
j t
j B @ s0| | jd kr>|
| jd kr>|td| n|td || |
|f}d}n
|td |s|tdt | j d }| jD ]2}||kr|d7 }q|td d||j |j}|
|}|r| } | }
|td|| |j| jd |j |d t | jkr q|j t
j t
j B @ r. q| d krZ|jrZ| j|d jjsZ q| |jksv| | jd kr|td | |d k r|| |
|f}d}|d7 }q q|d7 }q|s|d r|\}} }
}|td| | j| }|j}|r| jd7 _t |_| |_| j|kr2| jd7 _| || d | || f| _|
||
d ||
|
d f| _|r|| jkr| |}|| _||fS |td dS )a Attempts to find date on a log line using templates.
This uses the templates' `matchDate` method in an attempt to find
a date. It also increments the match hit count for the winning
template.
Parameters
----------
line : str
Line which is searched by the date templates.
Returns
-------
re.MatchObject, DateTemplate
The regex match returned from the first successfully matched
template.
c W s d S r/ r )argsr r r r% V r& z(DateDetector.matchTime..r z"try to match time for line: %.120sN)NrP rP rP z/ try to match last anchored template #%02i ...r zJ try to match last template #%02i (from %r to %r): ...%r==%r %s %r==%r...r z+ boundaries are correct, search in part %rz, boundaries show conflict, try whole searchz" matched last time template #%02izB ** last pattern collision - pattern change, reserve & search ...z8 ** last pattern not found - pattern change, search ...z search template (%i) ...z try template #%02i: %sz2 matched time template #%02i (at %r <= %r, %r) %sz1 ** distance collision - pattern change, reservez use best time template #%02iz
no template.)NN)rp rU rg rh getEffectiveLevellogLevellogrZ r ra r rb ZLINE_ENDZ matchDaterX rY isalnumr endr' rO rM timerN r[ _reorderTemplate) r4 linerv matchfoundZignoreBySearchrC ddtemplr rO endposr r r matchTimeA s
&
zDateDetector.matchTimec C s | j S r/ )r\ r3 r r r
default_tz s zDateDetector.default_tzc C s t || _d S r/ )r
r\ )r4 valuer r r r s c C s |du r| |}|d }|durzV|j||d | jd}|durvt tkrpttd|d |d d|j |W S W n t y Y n0 dS )a Attempts to return the date on a log line using templates.
This uses the templates' `getDate` method in an attempt to find
a date.
For the faster usage, always specify a parameter timeMatch (the previous tuple result
of the matchTime), then this will work without locking and without cycle over templates.
Parameters
----------
line : str
Line which is searched by the date templates.
Returns
-------
float
The Unix timestamp returned from the first successfully matched
template or None if not found.
Nr r )r z& got time %f for %r using template %s)
r ZgetDater\ rh rt ru rv groupr' r] )r4 r{ Z timeMatchr dater r r getTime s
zDateDetector.getTimec s r| j t tkr0ttd j j| j j| j k rR| j nd fdd}| sd krS d | sS = gd< | j t
k rֈ| j jr| j d7 _ qt tkrttd S S )zReorder template (bubble up) in template list if hits grows enough.
Parameters
----------
num : int
Index of template should be moved.
z% -> reorder template #%02i, hits: %rr c sL j } t tkr6ttd| j j | kpJ jkS )NzE -> compare template #%02i & #%02i, weight %.3f > %.3f, hits %r > %r)r; rh rt ru rv rM rN )Zpweightr~ numposr8 Zuntimer; r r _moveable
s
z0DateDetector._reorderTemplate.._moveabler r z" -> moved template #%02i -> #%02i) rU rh rt ru rv rM rN rW r; r[ rp )r4 r r r r r rz s. z DateDetector._reorderTemplate)F)NNT)N)rF rG rH rI r. rq r5 r` rk rg rJ r8 r r setterr rz r r r r rT s
*
$rT )N)"
__author__Z
__copyright__Z __license__copyry threadingr Z datetemplater r r r r r strptimer
Zutilsr Zhelpersr
rF rh ru compile
IGNORECASErm ZCacher r r- objectr. rK rT r r r r s$
n