hostsharing/hs.hsadmin.ng
6
0
Fork 0
Code Pull Requests 2 Activity

better formatted SQL and documentation of performance analysis for SELECT

Browse Source
This commit is contained in:
Michael Hoennig 2024-08-22 17:39:54 +02:00
parent 6dd20db238
commit 897ab191c7
2 changed files with 80 additions and 138 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
doc/rbac-performance-analysis.md
View File

@ -380,13 +380,21 @@ We found some solution approaches:
1. Getting rid of the `rbacrole` and `rbacpermission` table and only having implicit roles with implicit grants (OWNER->ADMIN->AGENT->TENENT->REFERRER) by comparison of ordered enum values and fixed permission assignments (e.g. OWENER->DELETE, ADMIN->UPDATE etc.). We could also get rid of the table `rbacreferece` if we enter users as business objects. 1. Getting rid of the `rbacrole` and `rbacpermission` table and only having implicit roles with implicit grants (OWNER->ADMIN->AGENT->TENENT->REFERRER) by comparison of ordered enum values and fixed permission assignments (e.g. OWENER->DELETE, ADMIN->UPDATE etc.). We could also get rid of the table `rbacreferece` if we enter users as business objects.
This should reduce This should dramatically reduce the size of the table `rbackgrant` as well as the recusion levels.
But since we only apply this query once for each business query, that would only improve performance once we have way more objects in our system, but does not help our current problem.
It's quite some effort to implement even just a prototype, so we did not further explore this idea.
2. Adding the object type to the table `rbacObject` to reduce the size of the result of the recursive CTE query.
See chapter below.
### Adding The Object Type To The Table `rbacObject` ### Adding The Object Type To The Table `rbacObject`
This optimization idea came from Michael Hierweck. This optimization idea came from Michael Hierweck and was promising.
Its idea is to reduce the size of the result of the recursive CTE query and maybe even speed up that query itself. The idea is to reduce the size of the result of the recursive CTE query and maybe even speed up that query itself.
To evaluate this, I added a type column to the `rbacObject` table, initially as an enum hsHostingAssetType. Then I entered the type there for all rows from hs_hosting_asset. This means that 83,886 of 92,545 rows in `rbacobject` have a type set, leaving 8,659 without. To evaluate this, I added a type column to the `rbacObject` table, initially as an enum hsHostingAssetType. Then I entered the type there for all rows from hs_hosting_asset. This means that 83,886 of 92,545 rows in `rbacobject` have a type set, leaving 8,659 without.
@ -403,7 +411,11 @@ joined with business query for all `'EMAIL_ADDRESSES'`:
As you can see, the query is no problem at all for normal customers (in the example, yours truly). With Hostsharing (D-1000000-hsh) it is quite slow. As you can see, the query is no problem at all for normal customers (in the example, yours truly). With Hostsharing (D-1000000-hsh) it is quite slow.
My optimization, which is not easy to try out, would only improve the part of the recursive CTE query. This is not necessary at the moment, but perhaps if we grow significantly. Luckily this experiment also shows that it's not a big problem, having all hosting assets in the same database table.
Implementing this approach would be a bit difficult anyway, because we would need to transfer the type query parameter into the definition of the restricted view. We have not even the slightest idea how this could be done.
See the related queries in [recursive-cte-experiments-for-accessible-uuids.sql](../sql/recursive-cte-experiments-for-accessible-uuids.sql). They might have changed independently since this document was written, but you can still check out the old version from git.
## Summary ## Summary

198
sql/recursive-cte-experiments-for-accessible-uuids.sql
View File

@ -2,19 +2,55 @@
select * from hs_statistics_view; select * from hs_statistics_view;
rollback transaction;
begin transaction;
SET TRANSACTION READ ONLY;
call defineContext('performance testing', null, 'superuser-alex@hostsharing.net',
'hs_booking_project#D-1000000-hshdefaultproject:ADMIN');
-- 'hs_booking_project#D-1000300-mihdefaultproject:ADMIN');
select count(type) as counter, type from hs_hosting_asset_rv
group by type
order by counter desc;
commit transaction;
-- ======================================================== -- ========================================================
-- An example for a restricted view (_rv) as generated by our RBAC system:
drop view if exists hs_hosting_asset_example_rv;
create view hs_hosting_asset_example_rv as
with accessible_hs_hosting_asset_uuids as (
with recursive
recursive_grants as (
select distinct rbacgrants.descendantuuid,
rbacgrants.ascendantuuid,
1 as level,
true
from rbacgrants
where (rbacgrants.ascendantuuid = any (currentsubjectsuuids()))
--and rbacgrants.assumed
union all
select distinct g.descendantuuid,
g.ascendantuuid,
grants.level + 1 as level,
assertTrue(grants.level < 22, 'too many grant-levels: ' || grants.level)
from rbacgrants g
join recursive_grants grants on grants.descendantuuid = g.ascendantuuid
where g.assumed
),
grant_count as (
select count(*) as grant_count from recursive_grants
),
count_check as (
select assertTrue((select grant_count from grant_count) < 600000,
'too many grants for current subjects: ' || (select grant_count from grant_count)) as valid
)
select distinct perm.objectuuid
from recursive_grants
join rbacpermission perm on recursive_grants.descendantuuid = perm.uuid
join rbacobject obj on obj.uuid = perm.objectuuid
join count_check cc on cc.valid
where obj.objecttable::text = 'hs_hosting_asset'::text
and obj.type = 'EMAIL_ADDRESS'::hshostingassettype -- with/without this type condition
)
select target.*
from hs_hosting_asset target
where (target.uuid in (select accessible_hs_hosting_asset_uuids.objectuuid
from accessible_hs_hosting_asset_uuids));
-- end of the example view.
-- -------------------------------------------------------------------------------
rollback transaction;
DO language plpgsql $$ DO language plpgsql $$
DECLARE DECLARE
start_time timestamp; start_time timestamp;
@ -24,73 +60,30 @@ DECLARE
BEGIN BEGIN
start_time := clock_timestamp(); start_time := clock_timestamp();
call defineContext('performance testing', null, 'superuser-alex@hostsharing.net', CALL defineContext('performance testing', null, 'superuser-alex@hostsharing.net',
'hs_booking_project#D-1000000-hshdefaultproject:ADMIN'); 'hs_booking_project#D-1000000-hshdefaultproject:ADMIN');
-- 'hs_booking_project#D-1000300-mihdefaultproject:ADMIN'); -- 'hs_booking_project#D-1000300-mihdefaultproject:ADMIN');
SET TRANSACTION READ ONLY; SET TRANSACTION READ ONLY;
FOR i IN 0..0 LOOP FOR i IN 0..25 LOOP
letter := chr(i+ascii('a')); letter := chr(i+ascii('a'));
PERFORM count(*) from (
perform count(*) from ( -- An example for a business query based on the view:
select type, uuid, identifier, caption
from hs_hosting_asset_example_rv
where type = 'EMAIL_ADDRESS'
and identifier like letter || '%'
-- end of the business query example.
-- start of VIEW hs_hosting_asset_rv: ) AS timed;
with accessible_hs_hosting_asset_uuids as (
with recursive
recursive_grants as (
select distinct rbacgrants.descendantuuid,
rbacgrants.ascendantuuid,
1 as level,
true
from rbacgrants
where (rbacgrants.ascendantuuid = any (currentsubjectsuuids()))
--and rbacgrants.assumed
union all
select distinct g.descendantuuid,
g.ascendantuuid,
grants.level + 1 as level,
assertTrue(grants.level < 22, 'too many grant-levels: ' || grants.level)
from rbacgrants g
join recursive_grants grants on grants.descendantuuid = g.ascendantuuid
where g.assumed
),
grant_count as (
select count(*) as grant_count from recursive_grants
),
count_check as (
select assertTrue((select grant_count from grant_count) < 600000,
'too many grants for current subjects: ' || (select grant_count from grant_count)) as valid
)
select distinct perm.objectuuid
from recursive_grants
join rbacpermission perm on recursive_grants.descendantuuid = perm.uuid
join rbacobject obj on obj.uuid = perm.objectuuid
join count_check cc on cc.valid
where obj.objecttable::text = 'hs_hosting_asset'::text
and obj.type = 'EMAIL_ADDRESS'::hshostingassettype -- with/without this type condition
)
-- end of VIEW hs_hosting_asset_rv.
-- start of business query, usually based on a view according to the above CTE query:
select type,
target.uuid,
target.identifier,
target.caption
from hs_hosting_asset target
where (target.uuid in (select accessible_hs_hosting_asset_uuids.objectuuid
from accessible_hs_hosting_asset_uuids))
and target.type = 'EMAIL_ADDRESS'
and identifier like letter || '%'
-- end of business query.
) timed;
END LOOP; END LOOP;
end_time := clock_timestamp(); end_time := clock_timestamp();
total_time := end_time - start_time; total_time := end_time - start_time;
RAISE NOTICE 'average execution time: %', total_time/26;
RAISE NOTICE 'average execution time: %', total_time/26;
END; END;
$$; $$;
@ -103,75 +96,14 @@ $$;
-- ============================================================================= -- =============================================================================
rollback transaction; -- extending the rbacobject table:
begin transaction;
SET TRANSACTION READ ONLY;
call defineContext('performance testing', null, 'superuser-alex@hostsharing.net',
'hs_booking_project#D-1000000-hshdefaultproject:ADMIN');
-- 'hs_booking_project#D-1000300-mihdefaultproject:ADMIN');
with one_path as (with recursive path as (
-- Base case: Start with the row where ascending equals the starting UUID
select ascendantuuid,
descendantuuid,
array [ascendantuuid] as path_so_far
from rbacgrants
where ascendantuuid = any (currentsubjectsuuids())
union all
-- Recursive case: Find the next step in the path
select c.ascendantuuid,
c.descendantuuid,
p.path_so_far || c.ascendantuuid
from rbacgrants c
inner join
path p on c.ascendantuuid = p.descendantuuid
where c.ascendantuuid != all (p.path_so_far) -- Prevent cycles
)
-- Final selection: Output all paths that reach the target UUID
select distinct array_length(path_so_far, 1),
path_so_far || descendantuuid as full_path
from path
join rbacpermission perm on perm.uuid = path.descendantuuid
join hs_hosting_asset ha on ha.uuid = perm.objectuuid
-- JOIN rbacrole_ev re on re.uuid = any(path_so_far)
where ha.identifier = 'vm1068'
order by array_length(path_so_far, 1)
limit 1
)
select
(
SELECT ARRAY_AGG(re.roleidname ORDER BY ord.idx)
FROM UNNEST(one_path.full_path) WITH ORDINALITY AS ord(uuid, idx)
JOIN rbacrole_ev re ON ord.uuid = re.uuid
) AS name_array
from one_path;
commit transaction;
with grants as (
select uuid
from rbacgrants
where descendantuuid in (
select uuid
from rbacrole
where objectuuid in (
select uuid
from hs_hosting_asset
-- where type = 'DOMAIN_MBOX_SETUP'
-- and identifier = 'example.org|MBOX'
where type = 'EMAIL_ADDRESS'
and identifier='test@example.org'
))
)
select * from rbacgrants_ev gev where exists ( select uuid from grants where gev.uuid = grants.uuid );
alter table rbacobject alter table rbacobject
-- just for performance testing, we would need a joined enum or a varchar(16) which would make it slow -- just for performance testing, we would need a joined enum or a varchar(16) which would make it slow
add column type hshostingassettype; add column type hshostingassettype;
-- and fill the type column with hs_hosting_asset types:
rollback transaction; rollback transaction;
begin transaction; begin transaction;
call defineContext('setting rbacobject.type from hs_hosting_asset.type', null, 'superuser-alex@hostsharing.net'); call defineContext('setting rbacobject.type from hs_hosting_asset.type', null, 'superuser-alex@hostsharing.net');
@ -183,9 +115,7 @@ call defineContext('setting rbacobject.type from hs_hosting_asset.type', null, '
end transaction; end transaction;
select -- check the result:
(select count(*) from hs_office_relation) as "relation",
(select count(*) from hs_booking_item) as "booking item";
select select
(select count(*) as "total" from rbacobject), (select count(*) as "total" from rbacobject),