j d Z ddlZddlmZ ddlmZmZmZmZ ddl Z ddl m
Z
m Z m Z ddl
mZmZ e G d d Zd ee fd
Zd Zd eeee f fd ZdS )
zF
Represents storage where ssa data is collected, stored and extracted
N) dataclass)ListIteratorTupleDict)funccastdistinct)
session_scope
RequestResultc D e Zd ZU eed< ee ed< eed< eed< dS )
DomainData domain_namedomain_total_reqsis_a_wordpress_domain urls_numberN)__name__
__module__ __qualname__str__annotations__r intbool J/opt/cloudlinux/venv/lib64/python3.11/site-packages/ssa/modules/storage.pyr r sF Cy r r returnc # z K t | 5 }| t j t j dt j t j t j t j t j
t j t t j
t j t j dt j t j t j dt j }t! j |d }|D ]^\ }}t% | }d}dgdz }|D ]*\ } }
} } }
| |t' |
< t ||
}+t) ||| | V _ ddd dS # 1 swxY w Y dS )z7
Iterates data from database domain-by-domain.
%Hc | d S Nr r items r z#iter_domains_data../ s
VZ[\V] r )keyr )r r r r N)r queryr domainr strftime
created_atCountidmax wordpresscountr
pathgroup_byorder_by itertoolsgroupbytupler r )enginedbresults_by_hourresults_by_hour_groupedr groupdomain_results_by_hourr requests_number_by_hour_hour requests_num is_wordpressurlss r iter_domains_datarB s
v !" (( M$
8 9 9 J}' ( ( H], - - Jx
233 4 4
( M$
8 9 9
( M$
8 9 9
#,"3OI]I]"^"^"^ "9 K%*5\\ "K ()cBh #=S
5
594 |T5A'D 2 !+t 4 4 '"9&2 '
#! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !s FF00F47F4c
# K t | 5 }| t j t j dt j t j t t j t j d t j
t j d t j t t j t j d t j |k t j | t j t j dt j t j t j dt j }d}dgdz dgdz dgdz }}}|D ]q\ } }
} } }
|r/|| k r)|t+ |||| fV dgdz dgdz dgdz }}}| |t- |
< | |t- |
< |
|t- |
< | }r| t+ | ||| fV ddd dS # 1 swxY w Y dS ) z6
Iterates urls data from database url-by-url.
r url_throttled_reqsurl_total_reqs
url_slow_reqsNr r& )r0 rD rE rF )r r' r r0 r r) r* Sumr hitting_limits
sqlalchemyIntegerlabelr+ r, is_slow_requestfilterr( in_r1 r2 dictr )r6 r all_pathsr7 urls_data
previous_pathrD rE rF r0 r> url_throttled_req
url_total_req url_slow_reqs r iter_urls_datarV C s
v 1
"HH M$
8 9 9 HT ,j.@
u)** J
e$%% HT -z/AC C
eO$$
& K /
& " "9 - -
(
dM4L M M
(
dM4L M M
% ,
C"HqcBhb -:N KT ! ! FD$)=,
1 $!6!6#T&'9#1"/ & & & C"HqcBhb 5BN" -> s4yy )(5N3t99 %'3M#d)) $ MMD 1)'
Y1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
s I$JJ
Jc # K t | 5 }| t j t j t j |k t j }t j |d }|D ]\ }}|d |D fV ddd dS # 1 swxY w Y dS )zA
Get information about durations of requests url-by-url.
c | d S r! r r" s r r$ z#get_url_durations.. s
DG r c g | ]\ }}|S r r ).0r= durations r
z%get_url_durations.. s :::[Q:::r N)
r r' r r0 r[ rM r( r2 r3 r4 )r6 r r7 rQ durations_by_pathr% r: s r get_url_durationsr^ { s#
v ;"HH "
& K /
(
& - ++ - -+ ; ;JC::E:::: : : : : ;; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ;s BB77B;>B;)__doc__r3 dataclassesr typingr r r r rI r r r
ssa.dbr r r rB rV r r r^ r r r rc s, ! ! ! ! ! ! . . . . . . . . . . . . + + + + + + + + + + / / / / / / / / %*!5 % % % %N5
5
5
p;d3c
?.C ; ; ; ; ; ;r