ZF h d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlmZm Z d dl
m Z m Z d dl
mZ d dlmZ d dlmZmZmZmZmZ d dlmZmZmZmZ d dl m Z d d l m Z d d
l m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z( d d l)m*Z*m+Z+ d d l,m-Z- d d
l.m/Z/ d dl0m1Z1 ed Z2d Z3d Z4e G d d Z5de6fdZ7d Z8d&dZ9d&de e5 fdZ:d&dZ;d Zd Z?d Z@d ZAd ZBd! ZCeDd"k r eE ejF ZGe2H d#eG ee%eG 5 e d$ e d% e" ej ejI e3 ej ejJ e4 e jK ZLeLM eC ddd dS # 1 swxY w Y dS dS )' N)datetime timedelta)ListOptional) dataclass)or_) ScrapeResult DomainAlertssetup_database
session_scopecleanup_old_data) get_domains setup_loggersave_pid_and_lock intersect)init_sentry_client)get_pkg_version)PING_TIMEOUT_STATUS_CODE
SENTRY_DNS!ERROR_DOMAINS_PING_RETRY_INTERVAL ERROR_DOMAINS_ALERT_INTERVAL
WMT_LOCK_FILEPING_CONNECTIONS&LICENSE_EXPIRED_FAREWELL_LETTER_MARKERLICENSE_CHECK_PAUSE)NotifierSupportedNotificationTypes) ErrorReport)cfg)CloudlinuxLicenseLib wmt_scannerc t j d t j D }t dt
| d S )Nc & i | ]\ }}|d k ||S ) report_email ).0kvs /builddir/build/BUILDROOT/cl-web-monitoring-tool-3.2.3-1.el9.cloudlinux.x86_64/opt/cloudlinux/venv/lib/python3.11/site-packages/wmt/main.py
z reload_conf../ s( PPP41aA r? r% r+ r) r; r; < sX HHH#'M8C='''&*hsm***%)OXc])))))r+ r; returnc t t j dt } t | } n # t t
f$ r
t } Y nw xY wt
d| S )Nping_connections )getattrr r rD TypeError
ValueErrormax)values r) get_connection_limitrN E s` CG /1A B BE!E
z " ! ! ! ! q%==s 2 A
A
c K |4 d{V fd}| d r | }d| t d d z }n<| d r d| t d d z }| }n
d| }d| } || d{V }| |_ |j dk r|cddd d{V S || d{V }| |_ |cddd d{V S # 1 d{V swxY w Y dS )a
Main 'pinger'
1. Requests domains
- if domain responded - keep status code
- if no response for timeout - keep Timeout status code
- if unreachable (ConnectionError or so) - keep 523 status code
(same logic as go implementation)
Nc
, K t j } | 4 d {V }t |d rt |j n| }t ||j t dt j |z
z cd d d d {V S # 1 d {V swxY w Y d S # t j j
$ r t | t dz cY S t j
j $ r t | d cY S w xY w)Ntimeoutr< i )r= r> i )r= )timegethasattrr1 r< r; statusrD
concurrentfutures TimeoutErrorr aiohttpclient_exceptions ClientError)
target_urlstartresp final_url ping_timeoutsessions r) _fetchzping.._fetchX s IKKE
G";;z<;HH D18u1E1E UDH
:I+!),TTY[[55H-I)J)J %2
N
N
N'
6N9E9LN N N N N N ,8
G
G
G (
#FFFFFF
Gs; B< A!B)B< )
B33B< 6B37B< <.D,$DDzhttps://zhttp:// )
startswithlenr? r= ) r< ra rb semaphorerc https_urlhttp_url https_result http_results `` r) pingrl N s' & & & & & & & & G G G G G G( >>* % % 'I 3s:'7'7#88HH
^^I
&
& '"SY %99IHH(3((I&H #VI . ....... '* $ % , , E& & & & & & & & & & & & & &H #F8 , ,,,,,,, &) #M& & & & & & & & & & & & & & & & & & & & & & & & & & & & & &s BC%9C%%
C/2C/c # K t }||t | z }t dt | |D ]*}t j | st || || V +dS )z
ping_timeout: specified in config timeout time (s) for request
semaphore: semaphore obj to handle asyncio tasks
ping_target_domains: mostly needed for re-pinging error domains
Nz Those domains will be pinged: %s)r setr/ debugr1 r is_domain_ignoredrl )ra rb rg ping_target_domainsdomainsdomains r) executorsrt s mmG&3*+++
LL3S\\BBB A A$V,, Av|Wi@@ @ @ @A Ar+ c K t t | ||| }t | dk rg S d |D } t j t j | | d {V S # t j $ rj g }|D ]*}| s| +|D ]3} | | d {V # t j
$ r Y 0w xY w|cY S w xY w)Nr c 6 g | ]}t j | S r% )asyncio create_task)r& coros r)
z scrape_sites.. s# > > >4W
&
& > > >r+ rQ ) listrt rf rw wait_forgatherrY donecancelappendCancelledError) ping_site_timeout
ping_intervalrb rg rq
coroutinestasksresultstasks r) scrape_sitesr s1 i 17IGZ[[\\J
:! > >: > > >E %gne& rd ) is_resolvedF)synchronize_sessionN)r r )rn r addr r< r= r> r? r queryr
filterr in_updatedictr ) enginepingedrq finished_domainserrors_domainsresolvedrb resultrr unfinished_domains
unfinisheds r) manage_ping_resultsr sY uuNH v G' , ,F KK
$ 2!'!8
!7!E6: F F F#s**-3-Avz**
++++
l##
VL(,,X66
7
7
VDT***V
F
F
F!G G G G G G G G G G G G G G G" mmG&3*+++ W (88 v ', J KK " !
s$ C>D,,D03D0:*F11F58F5c t j t t z
}t | 5 }| t j t j t | t t j
|k t j dk }d |D cddd S # 1 swxY w Y dS )z
- gets websites that must NOT be included in alert email:
less than ERROR_DOMAINS_ALERT_INTERVAL passed or is_resolved marker was not
changed from last alerting
)hoursFc g | ] }|j
S r% r r& rows r) rz z%get_recent_alerts.. s 888 888r+ N)r nowr r r r r
r r r r{ keysr
alert_timer )r
alert_domainsrepeat_intervalrb recently_alerteds r) get_recent_alertsr s lnny7S'T'T'TTO v 9'"== )=>>
VL(,,T-2D2D2F2F-G-GHH //A$0E9; ;< < 98'7888 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9s B%C++C/2C/c Z t dt t | d | D }t
t j t j d|it j | S )z
prepares needed error report object with error domains
to be alerted and sends this mail
returns alerted domains
zAlerts will be sent for %sc
g | ]W\ }}t |d t t t | t | XS )z, )r< code count_errors)r joinmapr1 rn rf )r& rs codess r) rz zalert.. sh
FE 3sCJJ//00 U
r+ error_report target_email
from_emailreportnotification_type)
r/ r0 r1 r{ r r. r r r r r ALERTnotify) domains_datar s r) alertr s KK,D **,,--..0 0 0 * / / 1 1
L
%> L
5:
< < < =CFHHH r+ c ` t | | fd| D }|sOt dt t | t dS t | t j
}t | 5 }| t t j }d |D }|D ]}||v r^| t t j |k t' |d d| t || ddd dS # 1 swxY w Y dS )aJ
- gets recently alerted domains (those that must not be alerted again)
and does not include them for alerting
- calls alerting for left domains
- updates DomainAlerts table:
if website was not alerted -> adds new record
if website was alerted before -> updates alert time and is_resolved marker
c $ i | ] \ }}|v ||
S r% r% )r& r' r( r s r) r* z flush_alerts.. s* \\\A!K[B[B[1B[B[B[r+ zDAll domains "%s" were alerted or still not resolved in last %d hoursNc g | ] }|j
S r% r r s r) rz z flush_alerts.. s 000 000r+ F)r r )r r )r r. r/ r0 r1 r{ r r r r r r r r
with_entitiesr allr r r r ) r r domains_to_alertr rb websitesurlsrs r s @r) flush_alertsr s ) ??\\\\ )<)<)>)>\\\ Z] / / 1 122330 2 2 2
,..C v J'== ..
]</
0
0
SUU 10x000& J JF~~
l++VL 0F :;;VDCU C C CDDDD LC H H HIIII
J J J J J J J J J J J J J J J J J J Js 4C!F##F'*F'c 0 | rt j j rdS dS )NTF)r alert_notifications_enabled)
error_domainss r) should_be_repingedr s < t 5r+ c t j t r5t d t j t d S d S )Nz CloudLinux license was updated)ospathexistsr r/ r0 remover% r+ r) cleanup_farewell_letter_markerr sM w~~<== : 4555
899999: :r+ c d t j t s_t d t
t j t j i t j dS dS # t $ r t d Y dS w xY w)z@
Sends farewell letter once (if it was not sent before)
z/Going to send last email about expired license!r z$Error while managing farewell letterN)r r r r r/ warningr r r r r FAREWELLr Exception exceptionr% r+ r) manage_license_farewellr $ s Aw~~ DEE P NNL M M M - >"<"E
G
G
G HNvxxxxx
P P A A A?@@@@@@As BB $B/.B/c
: K t j }t j j dz } t t | t }t
j | }t
j | 4 d{V }t j
| }t t j j ||| d{V }t || } t | rt dt# | t j j dz }|t$ k rt j |z
}
t j t) t+ t$ |
z
d d d{V |
t$ k rot t j j |||| d{V } t || | } t/ | | }
nnt/ | | }
| } n|
rWt d
t# t1 |
t3 ||
ng } ddd d{V n# 1 d{V swxY w Y n*# t4 $ r t d Y nw xY w t j j dz }t j |z
}t) t+ ||z
d d }t j | d{V ||k rnmns# t j j dz }t j |z
}t) t+ ||z
d d }t j | d{V ||k rnmw xY w| S ) aF
Scanner logic:
1. Scrapes domains and obtains ping results;
2. Manage ping results (e.g: saving to DB)
3. In case error domains found -> start re-pinging
Re-pinging:
- in min(ping_interval, 5 mins)
- flush alerts if needed
4. Sleep for ping_interval until next ping iteration
< )limit) connectorNz8Those domains are unsuccessful: %s
Try to re-ping themTr
)rq z1Domains with unsuccessful status code found: "%s"z Error during ping iteration!) rS r r r r
rN rZ TCPConnector
ClientSessionrw Semaphorer ra r r r/ r0 r1 r sleepminrL r r r{ r r r )previously_erroredr r^ ping_interval_secondsconnections_limitr rb rg ping_resultr elapsed_for_pingping_retry_result retry_errorsr elapsed
sleep_times r) scrape_iterationr 4 s
IKKE G 1B 69&((( 0 2 2(/@AAA (9=== * ( * ( * ( * ( * ( * ( * ( )*;<( %m $EHX$XZ[ \ \^` a a ,.OOO6B # 46KWV_4A4F4F4H4H7 7 7 1 1 1 1 1 1 - ,? &(9=;M;M;O;O, , L -6m\,R,RM ! P )22Dm(T(T
-:* / 2 ! 8KK S #D ););)=)=$>$> ? ?A A A 777 &("'U* ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * ( * (V 9 9 97888889 $'G$9B$> !ikkE)G S!6!@! D DbIIJ-
++ + + + + + + +...
$'G$9B$> !ikkE)G S!6!@! D DbIIJ-
++ + + + + + + +...
sP AJ G$I=+J =
JJ
J J L( $J63L( 5J66L( (A0Nc K t } g }d} t rd}t || d{V }n~|dz
}|dk rt d |dk rt
d}t j j dz }t |t }t j | d{V )z
Main loop for wmt_scanner_solo service
each 'while: True' iteration returns errored domains
(domains that responded with non-200 status code)
r TNrH z)Seems your CloudLinux license is expired! r )
r r get_license_statusr r/ r r r r r r rw r )r r license_attemptr r s r) scrape_loopr s
FO, ! ! 4 4 6 6 , O'78JF'S'S!S!S!S!S!S!S q O !## JKKK !##')))"#$'G$9B$> ! 24GHHJ-
++ + + + + + + + ,r+ __main__zPID: %szweb-monitoring-toolzcl-web-monitoring-tool)N)Nrw r r7 rZ signalrS concurrent.futuresrW r r typingr r dataclassesr
sqlalchemyr wmt.dbr r
r r r
wmt.common.utilsr r r r clsentryr clsentry.utilsr wmt.common.constr r r r r r r r wmt.common.notificationr r wmt.common.reportr
wmt.commonr cllicenser r/ r5 r9 r; rD rN rl rt r r r r r r r r r r r@ r1 getpidpidr0 SIGUSR1SIGTERMget_event_looplooprun_until_completer% r+ r) r s
( ( ( ( ( ( ( ( ! ! ! ! ! ! ! ! ! ! ! ! ! ! U T T T T T T T T T T T ' ' ' ' ' ' * * * * * * I H H H H H H H ) ) ) ) ) ) * * * * * * m $ $9 9 9 * * * * * * * *c / / /d A A A A jno kA (' ' ' 'T 9 9 9 4 J J JD : : :
A
A
A H H H V, , ,6 z
#ibikk
C
KK 3 =# . . / /0*?+CDD% ' ' '
fnk222
fnh///%w%''
.../ / / / / / / / / / / / / / / / / / s )A/F%%F),F)