You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Impact of the new feature
ReqMgr2/WMStats and other services consuming this data
Is your feature request related to a problem? Please describe.
We have been discussing on how to alleviate the load on WMStats - and anything/anyone consuming ACTIVE data - and one of the ideas is to reduce the number of workflows that are constantly monitored by WMStats (and cached by DataCache WMStatsServer application).
however, this will impact the data that is served by wmstatsserver app and the content provided, e.g. the protectedlfns and globallocks, used in the microservices for some dangerous and sensitive actions.
Describe the solution you'd like
Before we can shorten this list of ACTIVE status, we need to:
investigate what the impact is on the system
investigate if it could cause problems for those consuming the list of protected LFNs (output)
investigate if it could cause problems for those consuming the global locks (input + output)
if no negative side effects are expected, we should then stop monitoring workflows in status new or assignment-approvement in WMStats. This will also affect those querying ReqMgr2 with this query string status=ACTIVE.
Describe alternatives you've considered
At the current load (30k requests), the system is still behaving properly, but we could see a degradation at 40k or so. So we need to find ways to reduce the footprint on WMStats.
Additional context
Likely dependent on: #11241
and #11243
The text was updated successfully, but these errors were encountered:
There is a candidate fix for this, provided in #11263
But I don't feel like that is the best (or the most cost-benefit) way to address this issue. I think we should instead work at the foundations of the WMStats data structure and caching mechanism to decrease the memory footprint and data volume (network bandwidth).
Given that I haven't put any effort on it for the last month or so, I am unassigning myself.
Impact of the new feature
ReqMgr2/WMStats and other services consuming this data
Is your feature request related to a problem? Please describe.
We have been discussing on how to alleviate the load on WMStats - and anything/anyone consuming ACTIVE data - and one of the ideas is to reduce the number of workflows that are constantly monitored by WMStats (and cached by DataCache WMStatsServer application).
That could be done by removing
new
andassignment-approved
from the list of ACTIVE status defined here:https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/ReqMgr/DataStructs/RequestStatus.py#L71
however, this will impact the data that is served by
wmstatsserver
app and the content provided, e.g. theprotectedlfns
andgloballocks
, used in the microservices for some dangerous and sensitive actions.Describe the solution you'd like
Before we can shorten this list of ACTIVE status, we need to:
if no negative side effects are expected, we should then stop monitoring workflows in status
new
orassignment-approvement
in WMStats. This will also affect those querying ReqMgr2 with this query stringstatus=ACTIVE
.Describe alternatives you've considered
At the current load (30k requests), the system is still behaving properly, but we could see a degradation at 40k or so. So we need to find ways to reduce the footprint on WMStats.
Additional context
Likely dependent on:
#11241
and
#11243
The text was updated successfully, but these errors were encountered: