a
=Æ*f¾o ã @ sà d Z dZdZddlZddlZddlZddlZddlZddlZddl m
Z
ddl m Z ddl
mZ dd lmZ dd
lmZ d d lmZmZmZ eeƒZd
d„ Zdd„ Zdd„ Ze e e¡ e de¡ dd„ Z G dd„ de ƒZ!dS )zSteven Hiscocksz"Copyright (c) 2013 Steven HiscocksZGPLé N©Úwraps)ÚRLocké )ÚMyTime)Ú
FailTicket)ÚUtilsé )Ú getLoggerÚ
uni_stringÚ
PREFER_ENCc C s t | tƒrt| ƒ} t| ƒS )z/Avoid errors on types unknown in json-adapters.)Ú
isinstanceÚsetÚlistr )Úx© r úˆ | |g|¢R i |¤ŽW | ¡ W d ƒ W d ƒ S | ¡ 0 W d ƒ n1 sx0 Y W d ƒ n1 s–0 Y d S ©N)Ú_lockÚ_dbÚcursorÚclose)ÚselfÚargsÚkwargsÚcur©Úfr r ÚwrapperH s
þ z"commitandrollback..wrapperr )r0 r1 r r/ r ÚcommitandrollbackG s r2 c @ sœ e Zd ZdZdZdZeeƒZdMdd„ZdNd d
„Z d d „ Z
e d
d„ ƒZ dd„ Z
e dd„ ƒZe dd„ ƒZejdd„ ƒZdOdd„ZedPdd„ƒZdd„ Zed d „ ƒZed d „ ƒZed d!„ ƒZed"d#„ ƒZedQd%d&„ƒZed'd(„ ƒZdRd*d+„ZedSd,d-„ƒZ ed.d/„ ƒZ d0d1„ Z edTd2d3„ƒZ ed4d5„ ƒZ ed6d7„ ƒZ!ed8d9„ ƒZ"edUd:d;„ƒZ#dd?„Z%edWd@dA„ƒZ&dXdBdC„Z'dYdEdF„Z(dGdH„ Z)dIdJ„ Z*edKdL„ ƒZ+d$S )ZÚ
Fail2BanDbaË Fail2Ban database for storing persistent data.
This allows after Fail2Ban is restarted to reinstated bans and
to continue monitoring logs from the same point.
This will either create a new Fail2Ban database, connect to an
existing, and if applicable upgrade the schema in the process.
Parameters
----------
filename : str
File name for SQLite3 database, which will be created if
doesn't already exist.
purgeAge : int
Purge age in seconds, used to remove old bans from
database during purge.
Raises
------
sqlite3.OperationalError
Error connecting/creating a SQLite3 database.
RuntimeError
If existing database fails to update to new schema.
Attributes
----------
filename
purgeage
r ))Z
fail2banDbz7CREATE TABLE IF NOT EXISTS fail2banDb(version INTEGER);)Zjailsz•CREATE TABLE IF NOT EXISTS jails(name TEXT NOT NULL UNIQUE, enabled INTEGER NOT NULL DEFAULT 1);CREATE INDEX IF NOT EXISTS jails_name ON jails(name);)ÚlogsaS CREATE TABLE IF NOT EXISTS logs(jail TEXT NOT NULL, path TEXT, firstlinemd5 TEXT, lastfilepos INTEGER DEFAULT 0, FOREIGN KEY(jail) REFERENCES jails(name) ON DELETE CASCADE, UNIQUE(jail, path),UNIQUE(jail, path, firstlinemd5));CREATE INDEX IF NOT EXISTS logs_path ON logs(path);CREATE INDEX IF NOT EXISTS logs_jail_path ON logs(jail, path);)Úbansa‚ CREATE TABLE IF NOT EXISTS bans(jail TEXT NOT NULL, ip TEXT, timeofban INTEGER NOT NULL, bantime INTEGER NOT NULL, bancount INTEGER NOT NULL default 1, data JSON, FOREIGN KEY(jail) REFERENCES jails(name) );CREATE INDEX IF NOT EXISTS bans_jail_timeofban_ip ON bans(jail, timeofban);CREATE INDEX IF NOT EXISTS bans_jail_ip ON bans(jail, ip);CREATE INDEX IF NOT EXISTS bans_ip ON bans(ip);)ÚbipsaZ CREATE TABLE IF NOT EXISTS bips(ip TEXT NOT NULL, jail TEXT NOT NULL, timeofban INTEGER NOT NULL, bantime INTEGER NOT NULL, bancount INTEGER NOT NULL default 1, data JSON, PRIMARY KEY(ip, jail), FOREIGN KEY(jail) REFERENCES jails(name) );CREATE INDEX IF NOT EXISTS bips_timeofban ON bips(timeofban);CREATE INDEX IF NOT EXISTS bips_ip ON bips(ip);é€Q é c C s, d| _ tƒ | _|| _|| _|| _| ¡ d S )Né
)Ú
maxMatchesr r' Ú _dbFilenameÚ _purgeAgeÚ_outDatedFactorÚ
_connectDB)r+ ÚfilenameZpurgeAgeZoutDatedFactorr r r Ú__init__¦ s zFail2BanDb.__init__Fc
C sÒ | j }z*tj|dtjd| _i | _t d|¡ W n< tjyl } z"t d||j
d ¡ ‚ W Y d }~n
d }~0 0 zdd l }d}W n t y” d}Y n0 | j
¡ }z¢z:| d¡ | d¡ |sÈ| d ¡ | d
¡ | d ¡ W n„ tjy t d | ¡ ¡ Y n¸ tjyb } zDt d
||j
d ¡ tj |¡s:‚ | ¡ d }| ¡ W Y d }~nbd }~0 0 | ¡ d }|tjkr¼| |¡}|tjkr¢t d||¡ nt dtj||¡ tdƒ‚W |r&t d¡ | j |dd t d¡ t d¡ | d¡ | ¡ D ]} t dd | ¡¡ q| j ¡ |rÎ|r<| d ¡ | ¡ nˆ|r®t d¡ | j |dd t d¡ t d¡ | d¡ | ¡ D ]} t dd | ¡¡ qŠ| j ¡ |rÌ|rÄ| d ¡ | ¡ 0 d S )NF)Zcheck_same_threadZ detect_typesz.Connected to fail2ban persistent database '%s'z9Error connecting to fail2ban persistent database '%s': %sr TzPRAGMA foreign_keys = ONzPRAGMA synchronous = OFFz PRAGMA journal_mode = MEMORYzPRAGMA temp_store = MEMORYú&SELECT version FROM fail2banDb LIMIT 1z"New database created. Version '%r'z3Error opening fail2ban persistent database '%s': %sz"Database updated from '%r' to '%r'zIDatabase update failed to achieve version '%r': updated from '%r' to '%r'zFailed to fully updatez# Create missing tables/indices ...)Ú incrementalz -> okz Check integrity ...zPRAGMA integrity_checkz -> %sú )!r; Úsqlite3ZconnectZPARSE_DECLTYPESr( Ú_bansMergedCacher ÚinfoZOperationalErrorr r, Ú__pypy__Ú ImportErrorr) ÚexecuteZwarningÚcreateDbÚErrorÚosÚpathÚisfiler* ÚrepairDBÚfetchoner3 Ú __version__ÚupdateDbÚ RuntimeErrorÚdebugÚ _createDbZfetchallÚjoinZcommit)
r+ ÚcheckIntegrityr? r! rG Zpypyr. ÚversionZ
newversionÚsr r r r> ® s þÿ
þ
ÿ
þ
ÿþ
ó
zFail2BanDb._connectDBc C s" t d¡ | j ¡ t d¡ d S )Nz Close connection to database ...z Connection to database closed.)r rT r( r* rF ©r+ r r r r* ÿ s
zFail2BanDb.closec C sB z| j W S ty< | jd t dt ¡ ¡ | _ | j Y S 0 d S )NÚ.z
%Y%m%d-%H%M%S)Z _Fail2BanDb__dbBackupFilenameÚAttributeErrorr; ÚtimeÚstrftimer ÚgmtimerZ r r r Ú_dbBackupFilename s
z Fail2BanDb._dbBackupFilenamec
C sT G dd„ dt ƒ}| j}d | _z(z®t d| j¡ tj | j¡s^t
| j| j¡ t d| j¡ ntj | j¡rxt | j¡ t
d| j| jf¡ t | j¡j}|r¸t d|¡ | jdd nt d |¡ |d
ƒ‚W nn t y< } zTtjd | j|jd t||ƒ o
t ¡ d
kd t | j¡ | jdd W Y d }~n
d }~0 0 W || _n|| _0 d S )Nc @ s e Zd ZdS )z,Fail2BanDb.repairDB..RepairExceptionN)Ú__name__Ú
__module__Ú __qualname__r r r r ÚRepairException
s rd z Trying to repair database %sú Database backup created: %szHf2b_db=$0; f2b_dbbk=$1; sqlite3 "$f2b_dbbk" ".dump" | sqlite3 "$f2b_db" z5 Repair seems to be successful, restored %d byte(s).T)rW z1 Repair seems to be failed, restored %d byte(s).z Recreate ...z/ Error repairing of fail2ban database '%s': %sr r9 r )r rO r rF r; rL rM rN r` ÚshutilÚmoveÚremover Z
executeCmdÚstatÚst_sizer> r r, r
r )r+ rd Z _repairDBZ
dbFileSizer! r r r rO s6 ÿ þ $zFail2BanDb.repairDBc C s | j S )z&File name of SQLite3 database file.
)r; rZ r r r r? / s zFail2BanDb.filenamec C s | j S )zPurge age in seconds.
)r< rZ r r r Úpurgeage5 s zFail2BanDb.purgeagec C s t |¡| _d S r& )r Z str2secondsr< )r+ Úvaluer r r rk ; s c C s@ t jD ]\}}| |¡ q| dt jf¡ | d¡ | ¡ d S )z8Creates a new database, called during initialisation.
z\INSERT INTO fail2banDb(version) SELECT ? WHERE NOT EXISTS (SELECT 1 FROM fail2banDb LIMIT 1)rA r )r3 Ú_CREATE_SCRIPTSÚ
executescriptrI rQ rP )r+ r. rB ÚnrY r r r rU ? s þ
zFail2BanDb._createDbc C s | ||¡S r& )rU )r+ r. rB r r r rJ L s zFail2BanDb.createDbc C s& | d|f¡ | ¡ }|d uo$|d S )NzQselect 1 where exists (select 1 from sqlite_master WHERE type='table' AND name=?)r )rI rP )r+ r. ÚtableÚresr r r Ú _tableExistsP s
ÿzFail2BanDb._tableExistsc
C sJ |t jkrtdƒ‚zât d| j|¡ tj | j¡sPt
| j | j¡ t d| j¡ |dk rx| |d¡rx|
dt jd ¡ |dk r | |d¡r |
d t jd ¡ |d
k rÞ| |d ¡sÞ|
d t jd ¡ | |d¡rÞ| d
¡ | d¡ | ¡ d W S tyD } z6tjd| j|jd t ¡ dkd | ¡ W Y d}~n
d}~0 0 dS )z…Update an existing database, called during initialisation.
A timestamped backup is also created prior to attempting the update.
zIAttempt to travel to future version of database ...how did you get here??z&Upgrade database: %s from version '%r're r r4 zÃBEGIN TRANSACTION;CREATE TEMPORARY TABLE logs_temp AS SELECT * FROM logs;DROP TABLE logs;%s;INSERT INTO logs SELECT * from logs_temp;DROP TABLE logs_temp;UPDATE fail2banDb SET version = 2;COMMIT;r8 r5 zØBEGIN TRANSACTION;CREATE TEMPORARY TABLE bans_temp AS SELECT jail, ip, timeofban, -2 as bantime, 1 as bancount, data FROM bans;DROP TABLE bans;%s;
INSERT INTO bans SELECT * from bans_temp;DROP TABLE bans_temp;COMMIT;r r6 z?BEGIN TRANSACTION;%s;
UPDATE fail2banDb SET version = 4;COMMIT;z›INSERT OR REPLACE INTO bips(ip, jail, timeofban, bantime, bancount, data) SELECT ip, jail, timeofban, bantime, bancount, data FROM bans order by timeofbanrA r z#Failed to upgrade database '%s': %sr9 r N)r3 rQ ÚNotImplementedErrorr rF r` rL rM rN rf Úcopyfiler? rr rn Ú _CREATE_TABSrI rP r r r; r, r rO )r+ r. rX r! r r r rR V sB
ÿù úý ÿ
þzFail2BanDb.updateDbc C s. | d|jf¡ |jdkr*| d|jf¡ dS )zmAdds a jail to the database.
Parameters
----------
jail : Jail
Jail to be added to the database.
z7INSERT OR IGNORE INTO jails(name, enabled) VALUES(?, 1)r z |du r| d¡ n| dt|ƒf ¡ tdd„ | ¡ D ƒƒS )zGet name of jails in database.
Currently only used for testing purposes.
Returns
-------
set
Set of jail names.
NzSELECT name FROM jailsz'SELECT name FROM jails WHERE enabled=%sc s s | ]}|d V qdS ©r Nr ©Ú.0Úrowr r r Ú À ó z*Fail2BanDb.getJailNames..)rI Úintr Ú fetchmany)r+ r. Zenabledr r r Ú getJailNames° s ÿzFail2BanDb.getJailNamesc C s | ||| ¡ | ¡ | ¡ ¡S )a7 Adds a log to the database.
Parameters
----------
jail : Jail
Jail that log is being monitored by.
container : FileContainer
File container of the log file being added.
Returns
-------
int
If log was already present in database, value of last position
in the log file; else `None`
)Ú_addLogÚ getFileNameÚgetPosÚgetHash©r+ r. ry Ú containerr r r ÚaddLog s zFail2BanDb.addLogr c C s€ d }| d|j|f¡ z| ¡ \}}W n ty< d }Y n0 |d u rh|sR|d urh| d|j|||f¡ |d ur|||kr|d }|S )NzBSELECT firstlinemd5, lastfilepos FROM logs WHERE jail=? AND path=?úUINSERT OR REPLACE INTO logs(jail, path, firstlinemd5, lastfilepos) VALUES(?, ?, ?, ?))rI rw rP Ú TypeError)r+ r. ry rw ÚposÚmd5Z lastLinePosZ firstLineMD5r r r rˆ Õ s" ý
þzFail2BanDb._addLogc C sF d}g }|dur$|d7 }| |j¡ | ||¡ tdd„ | ¡ D ƒƒS )zëGets all the log paths from the database.
Currently only for testing purposes.
Parameters
----------
jail : Jail
If specified, will only return logs belonging to the jail.
Returns
-------
set
Set of log paths.
zSELECT path FROM logsNz
WHERE jail=?c s s | ]}|d V qdS r r r€ r r r rƒ þ r„ z)Fail2BanDb.getLogPaths..)Úappendrw rI r r† )r+ r. ry ÚqueryÚ queryArgsr r r Ú getLogPathsè s zFail2BanDb.getLogPathsc C s" | ||| ¡ | ¡ | ¡ ¡ dS )zÎUpdates hash and last position in log file.
Parameters
----------
jail : Jail
Jail of which the log file belongs to.
container : FileContainer
File container of the log file being updated.
N)Ú
_updateLogr‰ rŠ r‹ rŒ r r r Ú updateLog s zFail2BanDb.updateLogc C s6 | d|||j|f¡ |js2| d|j|||f¡ d S )NzEUPDATE logs SET firstlinemd5=?, lastfilepos=? WHERE jail=? AND path=?r rv )r+ r. ry rw r‘ r’ r r r r—
s þ þzFail2BanDb._updateLogc C s | |||||¡S )aA Get journal position from database.
Parameters
----------
jail : Jail
Jail of which the journal belongs to.
name, time, iso :
Journal name (typically systemd-journal) and last known time.
Returns
-------
int (or float)
Last position (as time) if it was already present in database; else `None`
)rˆ ©r+ r. ry rw r] Zisor r r Ú
getJournalPos s zFail2BanDb.getJournalPosc C s | |||||¡ dS )zÕUpdates last position (as time) of journal.
Parameters
----------
jail : Jail
Jail of which the journal belongs to.
name, time, iso :
Journal name (typically systemd-journal) and last known time.
N)r— r™ r r r Ú
updateJournal) s zFail2BanDb.updateJournalc
C s t | ¡ ƒ}z| j||f= W n ty. Y n0 z| j|df= W n tyR Y n0 | ¡ }| d¡}| jrœ|r®t|ƒ| jkr®| ¡ }|| j d… |d< n|r®| ¡ }|d= | d|j
|t t |
¡ ƒƒ| |j ¡ ¡| ¡ |f¡ | d||j
t t |
¡ ƒƒ| |j ¡ ¡| ¡ |f¡ dS )z¤Add a ban to the database.
Parameters
----------
jail : Jail
Jail in which the ban has occurred.
ticket : BanTicket
Ticket of the ban to be added.
NÚmatcheszWINSERT INTO bans(jail, ip, timeofban, bantime, bancount, data) VALUES(?, ?, ?, ?, ?, ?)zbINSERT OR REPLACE INTO bips(ip, jail, timeofban, bantime, bancount, data) VALUES(?, ?, ?, ?, ?, ?))ÚstrZgetIDrE ÚKeyErrorZgetDataÚgetr: ÚlenÚcopyrI rw r… ÚroundZgetTimeÚ
getBanTimeÚactionsZ getBanCount)r+ r. ry ÚticketÚipÚdatarœ r r r ÚaddBan6 s<
(ÿþ(ÿþzFail2BanDb.addBanc G s€ d}d}|j g}t|ƒs4| ||¡ | ||¡ dS |d7 }|d7 }| d¡ |D ](}t|ƒ|d< | ||¡ | ||¡ qRdS )zðDelete a single or multiple tickets from the database.
Parameters
----------
jail : Jail
Jail in which the ticket(s) should be removed.
args : list of IP
IPs to be removed, if not given all tickets of jail will be removed.
z DELETE FROM bips WHERE jail = ?z DELETE FROM bans WHERE jail = ?Nz AND ip = ?Ú r )rw r rI r“ r )r+ r. ry r, Zquery1Zquery2r• r¦ r r r ÚdelBan] s
zFail2BanDb.delBanc C s„ d}g }|d ur$|d7 }| |j¡ |d urN|dkrN|d7 }| t ¡ | ¡ |d url|d7 }| t|ƒ¡ |d7 }t| ||¡ƒS )Nz,SELECT ip, timeofban, data FROM bans WHERE 1ú AND jail=?r ú AND timeofban > ?ú AND ip=?z ORDER BY ip, timeofban desc)r“ rw r r] r r rI )r+ r. ry Úbantimer¦ r” r• r r r Ú_getBansw s zFail2BanDb._getBansc K sB g }| j f i |¤ŽD ](\}}}| t||ƒ¡ |d |¡ q|S )a» Get bans from the database.
Parameters
----------
jail : Jail
Jail that the ban belongs to. Default `None`; all jails.
bantime : int
Ban time in seconds, such that bans returned would still be
valid now. Negative values are equivalent to `None`.
Default `None`; no limit.
ip : str
IP Address to filter bans by. Default `None`; all IPs.
Returns
-------
list
List of `Ticket`s for bans stored in database.
éÿÿÿÿ)r¯ r“ r ZsetData)r+ r- Úticketsr¦ Ú timeofbanr§ r r r ÚgetBansŠ s
zFail2BanDb.getBansc C s² | j ” d}|du s |dk rH||f}|| jv rH| j| W d ƒ S g }d}t| j|||dƒ}|r^|d d }g } d}
i } |D ]¾\} }
}| |krÈt||| ƒ}| |
¡ | |¡ | }g } d}
i } | dg ¡}| jt | ƒ }|dkrt |ƒ|kr|| } n|| d… | } |
| dd¡7 }
|
|d< | |d< |
|¡ |
}q†t| || d}| |¡ |r||du rr|n|| j|< |du rŠ|n|W d ƒ S 1 s¤0 Y dS )a} Get bans from the database, merged into single ticket.
This is the same as `getBans`, but bans merged into single
ticket.
Parameters
----------
jail : Jail
Jail that the ban belongs to. Default `None`; all jails.
bantime : int
Ban time in seconds, such that bans returned would still be
valid now. Negative values are equivalent to `None`.
Default `None`; no limit.
ip : str
IP Address to filter bans by. Default `None`; all IPs.
Returns
-------
list or Ticket
Single ticket representing bans stored in database per IP
in a list. When `ip` argument passed, a single `Ticket` is
returned.
Nr )r¦ ry r® rœ Úfailuresr ©r§ ) r' rE r r¯ r Z
setAttemptr“ rŸ r: r Úupdate)r+ r¦ ry r® ZcacheKeyr± r¥ ÚresultsZ
prev_baniprœ r´ ZtickdataÚbanipr² r§ Zprev_timeofbanÚmZmaxaddr r r Ú
getBansMerged¤ sN
zFail2BanDb.getBansMergedc C s¤ t |ƒ}|sd}nd}|d7 }|g}|sD|d urD|d7 }| |j¡ |d urf|d7 }| t ¡ | ¡ |d ur€|d7 }| |¡ |sŒ|d u r”|d7 }t| ||¡ƒS )Nz-SELECT bancount, timeofban, bantime FROM bipsz