hostsharing/hs.hsadmin.ng
6
0
Fork 0
Code Pull Requests 3 Activity

import-unix-user-and-email-aliases #81

Merged
hsh-michaelhoennig merged 14 commits from import-unix-user-and-email-aliases into master 2024-08-01 13:12:58 +02:00
Conversation 0 Commits 14 Files Changed 50 +1132 -337
50 changed files with 1132 additions and 337 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

91
doc/rbac-performance-analysis.md
View File

@ -1,13 +1,16 @@
# RBAC Performance Analysis
This describes the analysis of the legacy-data-import which took way too long, which turned out to be a problem in the RBAC-access-rights-check.
This describes the analysis of the legacy-data-import which took way too long, which turned out to be a problem in the RBAC-access-rights-check as well as `EntityManager.persist` creating too many SQL queries.
## Our Performance-Problem
During the legacy data import for hosting assets we noticed massive performance problems. The import of about 2200 hosting-assets (IP-numbers, managed-webspaces, managed- and cloud-servers) as well as the creation of booking-items and booking-projects as well as necessary office-data entities (persons, contacts, partners, debitors, relations) **took 10-25 minutes**.
During the legacy data import for hosting assets we noticed massive performance problems. The import of about 2200 hosting-assets (IP-numbers, managed-webspaces, managed- and cloud-servers) as well as the creation of booking-items and booking-projects as well as necessary office-data entities (persons, contacts, partners, debitors, relations) **took 25 minutes**.
We could not find a pattern, why the import mostly took about 25 minutes, but sometimes took *just* 10 minutes. The impression that it had to do with too many other parallel processes, e.g. browser with BBB or IntelliJ IDEA was proved wrong, but stopping all unnecessary processes and performing the import again.
Importing hosting assets up to UnixUsers and EmailAddresses even **took about 100 minutes**.
(The office data import sometimes, but rarely, took only 10min.
We could not find a pattern, why that was the case. The impression that it had to do with too many other parallel processes, e.g. browser with BBB or IntelliJ IDEA was proved wrong, but stopping all unnecessary processes and performing the import again.)
## Preparation
@ -111,7 +114,23 @@ time gw-importHostingAssets
### Fetch the Query Statistics
And afterward we can query the statistics in PostgreSQL:
And afterward we can query the statistics in PostgreSQL, e.g.:
```SQL
WITH statements AS (
SELECT * FROM pg_stat_statements pss
)
SELECT calls,
total_exec_time::int/(60*1000) as total_exec_time_mins,
mean_exec_time::int as mean_exec_time_millis,
query
FROM statements
WHERE calls > 100 AND shared_blks_hit > 0
ORDER BY total_exec_time_mins DESC
LIMIT 16;
```
### Reset the Query Statistics
```SQL
SELECT pg_stat_statements_reset();
@ -272,6 +291,7 @@ The slowest query now was fetching Relations joined with Contact, Anchor-Person
We changed these mappings from `EAGER` (default) to `LAZY` to `@ManyToOne(fetch = FetchType.LAZY)` and got this result:
:::small
| query | calls | total (min) | mean (ms) |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------|-------------|----------|
| select hope1_0.uuid,hope1_0.familyname,hope1_0.givenname,hope1_0.persontype,hope1_0.salutation,hope1_0.title,hope1_0.tradename,hope1_0.version from public.hs_office_person_rv hope1_0 where hope1_0.uuid=$1 | 1015 | 4 | 238 |
@ -292,10 +312,69 @@ We changed these mappings from `EAGER` (default) to `LAZY` to `@ManyToOne(fetch
Now, finally, the total runtime of the import was down to 12 minutes. This is repeatable, where originally, the import took about 25mins in most cases and just rarely - and for unknown reasons - 10min.
### Importing UnixUser and EmailAlias Assets
But once UnixUser and EmailAlias assets got added to the import, the total time went up to about 110min.
This was not acceptable, especially not, considering that domains, email-addresses and database-assets are almost 10 times that number and thus the import would go up to over 1100min which is 20 hours.
In a first step, a `HsHostingAssetRawEntity` was created, mapped to the raw table (hs_hosting_asset) not to the RBAC-view (hs_hosting_asset_rv). Unfortunately we did not keep measurements, but that was only part of the problem anyway.
The main problem was, that there is something strange with persisting (`EntityManager.persist`) for EmailAlias assets. Where importing UnixUsers was mostly slow due to RBAC SELECT-permission checks, persisting EmailAliases suddenly created about a million (in numbers 1.000.000) SQL UPDATE statements after the INSERT, all with the same data, just increased version number (used for optimistic locking). We were not able to figure out why this happened.
Keep in mind, it's the same table with the same RBAC-triggers, just a different value in the type column.
Once `EntityManager.persist` was replaced by an explicit SQL INSERT - just for `HsHostingAssetRawEntity`, the total time was down to 17min. Thus importing the UnixUsers and EmailAliases took just 5min, which is an acceptable result. The total import of all HostingAssets is now estimated to about 1 hour (on my developer laptop).
Now, the longest running queries are these:
| No.| calls | total_m | mean_ms | query |
|---:|---------|--------:|--------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | 13.093 | 4 | 21 | insert into hs_hosting_asset( uuid, type, bookingitemuuid, parentassetuuid, assignedtoassetuuid, alarmcontactuuid, identifier, caption, config, version) values ( $1, $2, $3, $4, $5, $6, $7, $8, cast($9 as jsonb), $10) |
| 2 | 517 | 4 | 502 | select hore1_0.uuid,hore1_0.anchoruuid,hore1_0.contactuuid,hore1_0.holderuuid,hore1_0.mark,hore1_0.type,hore1_0.version from public.hs_office_relation_rv hore1_0 where hore1_0.uuid=$1 |
| 3 | 13.144 | 4 | 21 | call buildRbacSystemForHsHostingAsset(NEW) |
| 4 | 96.632 | 3 | 2 | call grantRoleToRole(roleUuid, superRoleUuid, superRoleDesc.assumed) |
| 5 | 120.815 | 3 | 2 | select * from isGranted(array[granteeId], grantedId) |
| 6 | 123.740 | 3 | 2 | with recursive grants as ( select descendantUuid, ascendantUuid from RbacGrants where descendantUuid = grantedId union all select "grant".descendantUuid, "grant".ascendantUuid from RbacGrants "grant" inner join grants recur on recur.ascendantUuid = "grant".descendantUuid ) select exists ( select $3 from grants where ascendantUuid = any(granteeIds) ) or grantedId = any(granteeIds) |
| 7 | 497 | 2 | 259 | select hoce1_0.uuid,hoce1_0.caption,hoce1_0.emailaddresses,hoce1_0.phonenumbers,hoce1_0.postaladdress,hoce1_0.version from public.hs_office_contact_rv hoce1_0 where hoce1_0.uuid=$1 |
| 8 | 497 | 2 | 255 | select hope1_0.uuid,hope1_0.familyname,hope1_0.givenname,hope1_0.persontype,hope1_0.salutation,hope1_0.title,hope1_0.tradename,hope1_0.version from public.hs_office_person_rv hope1_0 where hope1_0.uuid=$1 |
| 9 | 13.144 | 1 | 8 | SELECT createRoleWithGrants( hsHostingAssetTENANT(NEW), permissions => array[$7], incomingSuperRoles => array[ hsHostingAssetAGENT(NEW), hsOfficeContactADMIN(newAlarmContact)], outgoingSubRoles => array[ hsBookingItemTENANT(newBookingItem), hsHostingAssetTENANT(newParentAsset)] ) |
| 10 | 13.144 | 1 | 5 | SELECT createRoleWithGrants( hsHostingAssetADMIN(NEW), permissions => array[$7], incomingSuperRoles => array[ hsBookingItemAGENT(newBookingItem), hsHostingAssetAGENT(newParentAsset), hsHostingAssetOWNER(NEW)] ) |
That the `INSERT into hs_hosting_asset` (No. 1) takes up the most time, seems to be normal, and 21ms for each call is also fine.
It seems that the trigger effects (eg. No. 3 and No. 4) are included in the measure for the causing INSERT, otherwise summing up the totals would exceed the actual total time of the whole import. And it was to be expected that building the RBAC rules for new business objects takes most of the time.
In production, the `SELECT ... FROM hs_office_relation_rv` (No. 2) with about 0.5 seconds could still be a problem. But once we apply the improvements from the hosting asset area also to the office area, this should not be a problem for the import anymore.
## Further Options To Explore
1. Instead of separate SQL INSERT statements, we could try bulk INSERT.
2. We could use the SQL INSERT method for all entity-classes, or at least for all which have high row counts.
3. For the production code, we could use raw-entities for referenced entities, here usually RBAC SELECT permission is given anyway.
## Summary
That the import runtime is down to about 12min is repeatable, where originally, the import took about 25mins in most cases and just rarely - and for unknown reasons - just 10min.
### What we did Achieve?
In a first step, the total import runtime for office entities was reduced from about 25min to about 10min.
In a second step, we reduced the import of booking- and hosting-assets from about 100min (not counting the required office entities) to 5min.
### What Helped?
Merging the recursive CTE query to determine the RBAC SELECT-permission, made it more clear which business-queries take the time.
Avoiding EAGER-loading where not neccessary, reduced the total runtime of the import to about the half.
Avoiding EAGER-loading where not necessary, reduced the total runtime of the import to about the half.
The major improvement came from using direct INSERT statements, which then also bypassed the RBAC SELECT permission checks.
### What Still Has To Be Done?
Where this performance analysis was mostly helping the performance of the legacy data import, we still need measures and improvements for the productive code.
For sure, using more LAZY-loading also helps in the production code. For some more ideas see section _Further Options To Explore_.

9
src/main/java/net/hostsharing/hsadminng/hs/booking/item/HsBookingItemEntity.java
View File

@ -32,6 +32,7 @@ import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.OneToMany;
import jakarta.persistence.OneToOne;
import jakarta.persistence.PostLoad;
import jakarta.persistence.Table;
import jakarta.persistence.Transient;
import jakarta.persistence.Version;
@ -124,6 +125,14 @@ public class HsBookingItemEntity implements Stringifyable, RbacObject<HsBookingI
@Transient
private PatchableMapWrapper<Object> resourcesWrapper;
@Transient
private boolean isLoaded;
@PostLoad
public void markAsLoaded() {
this.isLoaded = true;
}
public PatchableMapWrapper<Object> getResources() {
return PatchableMapWrapper.of(resourcesWrapper, (newWrapper) -> {resourcesWrapper = newWrapper; }, resources );
}

16
src/main/java/net/hostsharing/hsadminng/hs/booking/item/validators/HsManagedWebspaceBookingItemValidator.java
View File

@ -38,8 +38,8 @@ class HsManagedWebspaceBookingItemValidator extends HsBookingItemEntityValidator
);
}
private static TriFunction<HsBookingItemEntity, IntegerProperty, Integer, List<String>> unixUsers() {
return (final HsBookingItemEntity entity, final IntegerProperty prop, final Integer factor) -> {
private static TriFunction<HsBookingItemEntity, IntegerProperty<?>, Integer, List<String>> unixUsers() {
return (final HsBookingItemEntity entity, final IntegerProperty<?> prop, final Integer factor) -> {
final var unixUserCount = ofNullable(entity.getRelatedHostingAsset())
.map(ha -> ha.getSubHostingAssets().stream()
.filter(subAsset -> subAsset.getType() == UNIX_USER)
@ -53,8 +53,8 @@ class HsManagedWebspaceBookingItemValidator extends HsBookingItemEntityValidator
};
}
private static TriFunction<HsBookingItemEntity, IntegerProperty, Integer, List<String>> databaseUsers() {
return (final HsBookingItemEntity entity, final IntegerProperty prop, final Integer factor) -> {
private static TriFunction<HsBookingItemEntity, IntegerProperty<?>, Integer, List<String>> databaseUsers() {
return (final HsBookingItemEntity entity, final IntegerProperty<?> prop, final Integer factor) -> {
final var dbUserCount = ofNullable(entity.getRelatedHostingAsset())
.map(ha -> ha.getSubHostingAssets().stream()
.filter(bi -> bi.getType() == PGSQL_USER || bi.getType() == MARIADB_USER )
@ -68,8 +68,8 @@ class HsManagedWebspaceBookingItemValidator extends HsBookingItemEntityValidator
};
}
private static TriFunction<HsBookingItemEntity, IntegerProperty, Integer, List<String>> databases() {
return (final HsBookingItemEntity entity, final IntegerProperty prop, final Integer factor) -> {
private static TriFunction<HsBookingItemEntity, IntegerProperty<?>, Integer, List<String>> databases() {
return (final HsBookingItemEntity entity, final IntegerProperty<?> prop, final Integer factor) -> {
final var unixUserCount = ofNullable(entity.getRelatedHostingAsset())
.map(ha -> ha.getSubHostingAssets().stream()
.filter(bi -> bi.getType()==PGSQL_USER || bi.getType()==MARIADB_USER )
@ -85,8 +85,8 @@ class HsManagedWebspaceBookingItemValidator extends HsBookingItemEntityValidator
};
}
private static TriFunction<HsBookingItemEntity, IntegerProperty, Integer, List<String>> eMailAddresses() {
return (final HsBookingItemEntity entity, final IntegerProperty prop, final Integer factor) -> {
private static TriFunction<HsBookingItemEntity, IntegerProperty<?>, Integer, List<String>> eMailAddresses() {
return (final HsBookingItemEntity entity, final IntegerProperty<?> prop, final Integer factor) -> {
final var unixUserCount = ofNullable(entity.getRelatedHostingAsset())
.map(ha -> ha.getSubHostingAssets().stream()
.filter(bi -> bi.getType() == DOMAIN_MBOX_SETUP)

72
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/HsHostingAsset.java Normal file
View File

@ -0,0 +1,72 @@
package net.hostsharing.hsadminng.hs.hosting.asset;
import net.hostsharing.hsadminng.hs.booking.item.HsBookingItemEntity;
import net.hostsharing.hsadminng.hs.booking.project.HsBookingProjectEntity;
import net.hostsharing.hsadminng.hs.office.contact.HsOfficeContactEntity;
import net.hostsharing.hsadminng.hs.validation.PropertiesProvider;
import net.hostsharing.hsadminng.rbac.rbacobject.RbacObject;
import net.hostsharing.hsadminng.stringify.Stringify;
import net.hostsharing.hsadminng.stringify.Stringifyable;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.UUID;
import static java.util.Collections.emptyMap;
import static net.hostsharing.hsadminng.stringify.Stringify.stringify;
public interface HsHostingAsset extends Stringifyable, RbacObject<HsHostingAsset>, PropertiesProvider {
Stringify<HsHostingAsset> stringify = stringify(HsHostingAsset.class)
.withProp(HsHostingAsset::getType)
.withProp(HsHostingAsset::getIdentifier)
.withProp(HsHostingAsset::getCaption)
.withProp(HsHostingAsset::getParentAsset)
.withProp(HsHostingAsset::getAssignedToAsset)
.withProp(HsHostingAsset::getBookingItem)
.withProp(HsHostingAsset::getConfig)
.quotedValues(false);
void setUuid(UUID uuid);
HsHostingAssetType getType();
HsHostingAsset getParentAsset();
void setIdentifier(String s);
String getIdentifier();
HsBookingItemEntity getBookingItem();
HsHostingAsset getAssignedToAsset();
HsOfficeContactEntity getAlarmContact();
List<? extends HsHostingAsset> getSubHostingAssets();
String getCaption();
Map<String, Object> getConfig();
default HsBookingProjectEntity getRelatedProject() {
return Optional.ofNullable(getBookingItem())
.map(HsBookingItemEntity::getRelatedProject)
.orElseGet(() -> Optional.ofNullable(getParentAsset())
.map(HsHostingAsset::getRelatedProject)
.orElse(null));
}
@Override
default Object getContextValue(final String propName) {
final var v = directProps().get(propName);
if (v != null) {
return v;
}
if (getBookingItem() != null) {
return getBookingItem().getResources().get(propName);
}
if (getParentAsset() != null && getParentAsset().getBookingItem() != null) {
return getParentAsset().getBookingItem().getResources().get(propName);
}
return emptyMap();
}
@Override
default String toShortString() {
return getType() + ":" + getIdentifier();
}
}

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/HsHostingAssetController.java
View File

@ -72,7 +72,7 @@ public class HsHostingAssetController implements HsHostingAssetsApi {
final var entity = mapper.map(body, HsHostingAssetEntity.class, RESOURCE_TO_ENTITY_POSTMAPPER);
final var mapped = new HostingAssetEntitySaveProcessor(entity)
final var mapped = new HostingAssetEntitySaveProcessor(em, entity)
.preprocessEntity()
.validateEntity()
.prepareForSave()
@ -133,7 +133,7 @@ public class HsHostingAssetController implements HsHostingAssetsApi {
new HsHostingAssetEntityPatcher(em, entity).apply(body);
final var mapped = new HostingAssetEntitySaveProcessor(entity)
final var mapped = new HostingAssetEntitySaveProcessor(em, entity)
.preprocessEntity()
.validateEntity()
.prepareForSave()
@ -162,5 +162,5 @@ public class HsHostingAssetController implements HsHostingAssetsApi {
@SuppressWarnings("unchecked")
final BiConsumer<HsHostingAssetEntity, HsHostingAssetResource> ENTITY_TO_RESOURCE_POSTMAPPER = (entity, resource)
-> resource.setConfig(HostingAssetEntityValidatorRegistry.forType(entity.getType())
.revampProperties(entity, (Map<String, Object>) resource.getConfig()));
.revampProperties(em, entity, (Map<String, Object>) resource.getConfig()));
}

50
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/HsHostingAssetEntity.java
View File

@ -8,15 +8,10 @@ import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import net.hostsharing.hsadminng.hs.booking.item.HsBookingItemEntity;
import net.hostsharing.hsadminng.hs.booking.project.HsBookingProjectEntity;
import net.hostsharing.hsadminng.hs.office.contact.HsOfficeContactEntity;
import net.hostsharing.hsadminng.hs.validation.PropertiesProvider;
import net.hostsharing.hsadminng.mapper.PatchableMapWrapper;
import net.hostsharing.hsadminng.rbac.rbacdef.RbacView;
import net.hostsharing.hsadminng.rbac.rbacdef.RbacView.SQL;
import net.hostsharing.hsadminng.rbac.rbacobject.RbacObject;
import net.hostsharing.hsadminng.stringify.Stringify;
import net.hostsharing.hsadminng.stringify.Stringifyable;
import org.hibernate.annotations.Type;
import jakarta.persistence.CascadeType;
@ -39,10 +34,8 @@ import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.UUID;
import static java.util.Collections.emptyMap;
import static net.hostsharing.hsadminng.rbac.rbacdef.RbacView.CaseDef.inCaseOf;
import static net.hostsharing.hsadminng.rbac.rbacdef.RbacView.Column.dependsOnColumn;
import static net.hostsharing.hsadminng.rbac.rbacdef.RbacView.ColumnValue.usingDefaultCase;
@ -70,17 +63,7 @@ import static net.hostsharing.hsadminng.stringify.Stringify.stringify;
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class HsHostingAssetEntity implements Stringifyable, RbacObject<HsHostingAssetEntity>, PropertiesProvider {
private static Stringify<HsHostingAssetEntity> stringify = stringify(HsHostingAssetEntity.class)
.withProp(HsHostingAssetEntity::getType)
.withProp(HsHostingAssetEntity::getIdentifier)
.withProp(HsHostingAssetEntity::getCaption)
.withProp(HsHostingAssetEntity::getParentAsset)
.withProp(HsHostingAssetEntity::getAssignedToAsset)
.withProp(HsHostingAssetEntity::getBookingItem)
.withProp(HsHostingAssetEntity::getConfig)
.quotedValues(false);
public class HsHostingAssetEntity implements HsHostingAsset {
@Id
@GeneratedValue
@ -136,14 +119,6 @@ public class HsHostingAssetEntity implements Stringifyable, RbacObject<HsHosting
this.isLoaded = true;
}
public HsBookingProjectEntity getRelatedProject() {
return Optional.ofNullable(bookingItem)
.map(HsBookingItemEntity::getRelatedProject)
.orElseGet(() -> Optional.ofNullable(parentAsset)
.map(HsHostingAssetEntity::getRelatedProject)
.orElse(null));
}
public PatchableMapWrapper<Object> getConfig() {
return PatchableMapWrapper.of(configWrapper, (newWrapper) -> {configWrapper = newWrapper;}, config);
}
@ -157,30 +132,9 @@ public class HsHostingAssetEntity implements Stringifyable, RbacObject<HsHosting
return config;
}
@Override
public Object getContextValue(final String propName) {
final var v = config.get(propName);
if (v != null) {
return v;
}
if (bookingItem != null) {
return bookingItem.getResources().get(propName);
}
if (parentAsset != null && parentAsset.getBookingItem() != null) {
return parentAsset.getBookingItem().getResources().get(propName);
}
return emptyMap();
}
@Override
public String toString() {
return stringify.apply(this);
}
@Override
public String toShortString() {
return type + ":" + identifier;
return stringify.using(HsHostingAssetEntity.class).apply(this);
}
public static RbacView rbac() {

2
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/HsHostingAssetRepository.java
View File

@ -39,7 +39,7 @@ public interface HsHostingAssetRepository extends Repository<HsHostingAssetEntit
return findAllByCriteriaImpl(projectUuid, parentAssetUuid, HsHostingAssetType.asString(type));
}
HsHostingAssetEntity save(HsHostingAssetEntity current);
HsHostingAssetEntity save(HsHostingAsset current);
int deleteByUuid(final UUID uuid);

16
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/HsHostingAssetType.java
View File

@ -384,29 +384,29 @@ class EntityTypeRelation<E, T extends Node> {
" *==> ");
}
static EntityTypeRelation<HsHostingAssetEntity, HsHostingAssetType> optionalParent(final HsHostingAssetType hostingAssetType) {
static EntityTypeRelation<HsHostingAsset, HsHostingAssetType> optionalParent(final HsHostingAssetType hostingAssetType) {
return new EntityTypeRelation<>(
OPTIONAL,
PARENT_ASSET,
HsHostingAssetEntity::getParentAsset,
HsHostingAsset::getParentAsset,
hostingAssetType,
" o..> ");
}
static EntityTypeRelation<HsHostingAssetEntity, HsHostingAssetType> requiredParent(final HsHostingAssetType hostingAssetType) {
static EntityTypeRelation<HsHostingAsset, HsHostingAssetType> requiredParent(final HsHostingAssetType hostingAssetType) {
return new EntityTypeRelation<>(
REQUIRED,
PARENT_ASSET,
HsHostingAssetEntity::getParentAsset,
HsHostingAsset::getParentAsset,
hostingAssetType,
" *==> ");
}
static EntityTypeRelation<HsHostingAssetEntity, HsHostingAssetType> assignedTo(final HsHostingAssetType hostingAssetType) {
static EntityTypeRelation<HsHostingAsset, HsHostingAssetType> assignedTo(final HsHostingAssetType hostingAssetType) {
return new EntityTypeRelation<>(
REQUIRED,
ASSIGNED_TO_ASSET,
HsHostingAssetEntity::getAssignedToAsset,
HsHostingAsset::getAssignedToAsset,
hostingAssetType,
" o--> ");
}
@ -416,11 +416,11 @@ class EntityTypeRelation<E, T extends Node> {
return this;
}
static EntityTypeRelation<HsHostingAssetEntity, HsHostingAssetType> optionallyAssignedTo(final HsHostingAssetType hostingAssetType) {
static EntityTypeRelation<HsHostingAsset, HsHostingAssetType> optionallyAssignedTo(final HsHostingAssetType hostingAssetType) {
return new EntityTypeRelation<>(
OPTIONAL,
ASSIGNED_TO_ASSET,
HsHostingAssetEntity::getAssignedToAsset,
HsHostingAsset::getAssignedToAsset,
hostingAssetType,
" o..> ");
}

32
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HostingAssetEntitySaveProcessor.java
View File

@ -1,24 +1,27 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.errors.MultiValidationException;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.hs.hosting.generated.api.v1.model.HsHostingAssetResource;
import net.hostsharing.hsadminng.hs.validation.HsEntityValidator;
import jakarta.persistence.EntityManager;
import java.util.Map;
import java.util.function.Function;
/**
* Wraps the steps of the pararation, validation, mapping and revamp around saving of a HsHostingAssetEntity into a readable API.
* Wraps the steps of the pararation, validation, mapping and revamp around saving of a HsHostingAsset into a readable API.
*/
public class HostingAssetEntitySaveProcessor {
private final HsEntityValidator<HsHostingAssetEntity> validator;
private final HsEntityValidator<HsHostingAsset> validator;
private String expectedStep = "preprocessEntity";
private HsHostingAssetEntity entity;
private final EntityManager em;
private HsHostingAsset entity;
private HsHostingAssetResource resource;
public HostingAssetEntitySaveProcessor(final HsHostingAssetEntity entity) {
public HostingAssetEntitySaveProcessor(final EntityManager em, final HsHostingAsset entity) {
this.em = em;
this.entity = entity;
this.validator = HostingAssetEntityValidatorRegistry.forType(entity.getType());
}
@ -37,15 +40,26 @@ public class HostingAssetEntitySaveProcessor {
return this;
}
/// validates the entity itself including its properties, but ignoring some error messages for import of legacy data
public HostingAssetEntitySaveProcessor validateEntityIgnoring(final String ignoreRegExp) {
step("validateEntity", "prepareForSave");
MultiValidationException.throwIfNotEmpty(
validator.validateEntity(entity).stream()
.filter(errorMsg -> !errorMsg.matches(ignoreRegExp))
.toList()
);
return this;
}
/// hashing passwords etc.
@SuppressWarnings("unchecked")
public HostingAssetEntitySaveProcessor prepareForSave() {
step("prepareForSave", "saveUsing");
validator.prepareProperties(entity);
validator.prepareProperties(em, entity);
return this;
}
public HostingAssetEntitySaveProcessor saveUsing(final Function<HsHostingAssetEntity, HsHostingAssetEntity> saveFunction) {
public HostingAssetEntitySaveProcessor saveUsing(final Function<HsHostingAsset, HsHostingAsset> saveFunction) {
step("saveUsing", "validateContext");
entity = saveFunction.apply(entity);
return this;
@ -60,7 +74,7 @@ public class HostingAssetEntitySaveProcessor {
/// maps entity to JSON resource representation
public HostingAssetEntitySaveProcessor mapUsing(
final Function<HsHostingAssetEntity, HsHostingAssetResource> mapFunction) {
final Function<HsHostingAsset, HsHostingAssetResource> mapFunction) {
step("mapUsing", "revampProperties");
resource = mapFunction.apply(entity);
return this;
@ -70,7 +84,7 @@ public class HostingAssetEntitySaveProcessor {
@SuppressWarnings("unchecked")
public HsHostingAssetResource revampProperties() {
step("revampProperties", null);
final var revampedProps = validator.revampProperties(entity, (Map<String, Object>) resource.getConfig());
final var revampedProps = validator.revampProperties(em, entity, (Map<String, Object>) resource.getConfig());
resource.setConfig(revampedProps);
return resource;
}

50
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HostingAssetEntityValidator.java
View File

@ -3,7 +3,7 @@ package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.booking.item.HsBookingItemEntity;
import net.hostsharing.hsadminng.hs.booking.item.HsBookingItemType;
import net.hostsharing.hsadminng.hs.booking.item.validators.HsBookingItemEntityValidatorRegistry;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType;
import net.hostsharing.hsadminng.hs.office.contact.HsOfficeContactEntity;
import net.hostsharing.hsadminng.hs.validation.HsEntityValidator;
@ -23,13 +23,13 @@ import static java.util.Arrays.stream;
import static java.util.Collections.emptyList;
import static java.util.Optional.ofNullable;
public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHostingAssetEntity> {
public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHostingAsset> {
static final ValidatableProperty<?, ?>[] NO_EXTRA_PROPERTIES = new ValidatableProperty<?, ?>[0];
private final ReferenceValidator<HsBookingItemEntity, HsBookingItemType> bookingItemReferenceValidation;
private final ReferenceValidator<HsHostingAssetEntity, HsHostingAssetType> parentAssetReferenceValidation;
private final ReferenceValidator<HsHostingAssetEntity, HsHostingAssetType> assignedToAssetReferenceValidation;
private final ReferenceValidator<HsHostingAsset, HsHostingAssetType> parentAssetReferenceValidation;
private final ReferenceValidator<HsHostingAsset, HsHostingAssetType> assignedToAssetReferenceValidation;
private final HostingAssetEntityValidator.AlarmContact alarmContactValidation;
HostingAssetEntityValidator(
@ -40,23 +40,23 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
this.bookingItemReferenceValidation = new ReferenceValidator<>(
assetType.bookingItemPolicy(),
assetType.bookingItemTypes(),
HsHostingAssetEntity::getBookingItem,
HsHostingAsset::getBookingItem,
HsBookingItemEntity::getType);
this.parentAssetReferenceValidation = new ReferenceValidator<>(
assetType.parentAssetPolicy(),
assetType.parentAssetTypes(),
HsHostingAssetEntity::getParentAsset,
HsHostingAssetEntity::getType);
HsHostingAsset::getParentAsset,
HsHostingAsset::getType);
this.assignedToAssetReferenceValidation = new ReferenceValidator<>(
assetType.assignedToAssetPolicy(),
assetType.assignedToAssetTypes(),
HsHostingAssetEntity::getAssignedToAsset,
HsHostingAssetEntity::getType);
HsHostingAsset::getAssignedToAsset,
HsHostingAsset::getType);
this.alarmContactValidation = alarmContactValidation;
}
@Override
public List<String> validateEntity(final HsHostingAssetEntity assetEntity) {
public List<String> validateEntity(final HsHostingAsset assetEntity) {
return sequentiallyValidate(
() -> validateEntityReferencesAndProperties(assetEntity),
() -> validateIdentifierPattern(assetEntity)
@ -64,7 +64,7 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
}
@Override
public List<String> validateContext(final HsHostingAssetEntity assetEntity) {
public List<String> validateContext(final HsHostingAsset assetEntity) {
return sequentiallyValidate(
() -> optionallyValidate(assetEntity.getBookingItem()),
() -> optionallyValidate(assetEntity.getParentAsset()),
@ -72,7 +72,7 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
);
}
private List<String> validateEntityReferencesAndProperties(final HsHostingAssetEntity assetEntity) {
private List<String> validateEntityReferencesAndProperties(final HsHostingAsset assetEntity) {
return Stream.of(
validateReferencedEntity(assetEntity, "bookingItem", bookingItemReferenceValidation::validate),
validateReferencedEntity(assetEntity, "parentAsset", parentAssetReferenceValidation::validate),
@ -86,17 +86,17 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
}
private List<String> validateReferencedEntity(
final HsHostingAssetEntity assetEntity,
final HsHostingAsset assetEntity,
final String referenceFieldName,
final BiFunction<HsHostingAssetEntity, String, List<String>> validator) {
final BiFunction<HsHostingAsset, String, List<String>> validator) {
return enrich(prefix(assetEntity.toShortString()), validator.apply(assetEntity, referenceFieldName));
}
private List<String> validateProperties(final HsHostingAssetEntity assetEntity) {
private List<String> validateProperties(final HsHostingAsset assetEntity) {
return enrich(prefix(assetEntity.toShortString(), "config"), super.validateProperties(assetEntity));
}
private static List<String> optionallyValidate(final HsHostingAssetEntity assetEntity) {
private static List<String> optionallyValidate(final HsHostingAsset assetEntity) {
return assetEntity != null
? enrich(
prefix(assetEntity.toShortString(), "parentAsset"),
@ -112,7 +112,7 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
: emptyList();
}
protected List<String> validateAgainstSubEntities(final HsHostingAssetEntity assetEntity) {
protected List<String> validateAgainstSubEntities(final HsHostingAsset assetEntity) {
return enrich(
prefix(assetEntity.toShortString(), "config"),
stream(propertyValidators)
@ -124,7 +124,7 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
// TODO.test: check, if there are any hosting assets which need this validation at all
private String validateMaxTotalValue(
final HsHostingAssetEntity hostingAsset,
final HsHostingAsset hostingAsset,
final ValidatableProperty<?, ?> propDef) {
final var propName = propDef.propertyName();
final var propUnit = ofNullable(propDef.unit()).map(u -> " " + u).orElse("");
@ -140,7 +140,7 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
: null;
}
private List<String> validateIdentifierPattern(final HsHostingAssetEntity assetEntity) {
private List<String> validateIdentifierPattern(final HsHostingAsset assetEntity) {
final var expectedIdentifierPattern = identifierPattern(assetEntity);
if (assetEntity.getIdentifier() == null ||
!expectedIdentifierPattern.matcher(assetEntity.getIdentifier()).matches()) {
@ -151,19 +151,19 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
return Collections.emptyList();
}
protected abstract Pattern identifierPattern(HsHostingAssetEntity assetEntity);
protected abstract Pattern identifierPattern(HsHostingAsset assetEntity);
static class ReferenceValidator<S, T> {
private final HsHostingAssetType.RelationPolicy policy;
private final Set<T> referencedEntityTypes;
private final Function<HsHostingAssetEntity, S> referencedEntityGetter;
private final Function<HsHostingAsset, S> referencedEntityGetter;
private final Function<S, T> referencedEntityTypeGetter;
public ReferenceValidator(
final HsHostingAssetType.RelationPolicy policy,
final Set<T> referencedEntityTypes,
final Function<HsHostingAssetEntity, S> referencedEntityGetter,
final Function<HsHostingAsset, S> referencedEntityGetter,
final Function<S, T> referencedEntityTypeGetter) {
this.policy = policy;
this.referencedEntityTypes = referencedEntityTypes;
@ -173,14 +173,14 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
public ReferenceValidator(
final HsHostingAssetType.RelationPolicy policy,
final Function<HsHostingAssetEntity, S> referencedEntityGetter) {
final Function<HsHostingAsset, S> referencedEntityGetter) {
this.policy = policy;
this.referencedEntityTypes = Set.of();
this.referencedEntityGetter = referencedEntityGetter;
this.referencedEntityTypeGetter = e -> null;
}
List<String> validate(final HsHostingAssetEntity assetEntity, final String referenceFieldName) {
List<String> validate(final HsHostingAsset assetEntity, final String referenceFieldName) {
final var actualEntity = referencedEntityGetter.apply(assetEntity);
final var actualEntityType = actualEntity != null ? referencedEntityTypeGetter.apply(actualEntity) : null;
@ -216,7 +216,7 @@ public abstract class HostingAssetEntityValidator extends HsEntityValidator<HsHo
static class AlarmContact extends ReferenceValidator<HsOfficeContactEntity, Enum<?>> {
AlarmContact(final HsHostingAssetType.RelationPolicy policy) {
super(policy, HsHostingAssetEntity::getAlarmContact);
super(policy, HsHostingAsset::getAlarmContact);
}
// hostmaster alert address is implicitly added where neccessary

9
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HostingAssetEntityValidatorRegistry.java
View File

@ -1,6 +1,7 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType;
import net.hostsharing.hsadminng.hs.hosting.generated.api.v1.model.HsHostingAssetResource;
import net.hostsharing.hsadminng.hs.validation.HsEntityValidator;
@ -12,7 +13,7 @@ import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.*;
public class HostingAssetEntityValidatorRegistry {
private static final Map<Enum<HsHostingAssetType>, HsEntityValidator<HsHostingAssetEntity>> validators = new HashMap<>();
private static final Map<Enum<HsHostingAssetType>, HsEntityValidator<HsHostingAsset>> validators = new HashMap<>();
static {
// HOWTO: add (register) new HsHostingAssetType-specific validators
register(CLOUD_SERVER, new HsCloudServerHostingAssetValidator());
@ -36,14 +37,14 @@ public class HostingAssetEntityValidatorRegistry {
register(IPV6_NUMBER, new HsIPv6NumberHostingAssetValidator());
}
private static void register(final Enum<HsHostingAssetType> type, final HsEntityValidator<HsHostingAssetEntity> validator) {
private static void register(final Enum<HsHostingAssetType> type, final HsEntityValidator<HsHostingAsset> validator) {
stream(validator.propertyValidators).forEach( entry -> {
entry.verifyConsistency(Map.entry(type, validator));
});
validators.put(type, validator);
}
public static HsEntityValidator<HsHostingAssetEntity> forType(final Enum<HsHostingAssetType> type) {
public static HsEntityValidator<HsHostingAsset> forType(final Enum<HsHostingAssetType> type) {
if ( validators.containsKey(type)) {
return validators.get(type);
}

4
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsCloudServerHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -16,7 +16,7 @@ class HsCloudServerHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile("^vm[0-9][0-9][0-9][0-9]$");
}
}

12
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsDomainDnsSetupHostingAssetValidator.java
View File

@ -1,7 +1,7 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import lombok.SneakyThrows;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.system.SystemProcess;
import java.util.List;
@ -59,12 +59,12 @@ class HsDomainDnsSetupHostingAssetValidator extends HostingAssetEntityValidator
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile("^" + Pattern.quote(assetEntity.getParentAsset().getIdentifier() + IDENTIFIER_SUFFIX) + "$");
}
@Override
public void preprocessEntity(final HsHostingAssetEntity entity) {
public void preprocessEntity(final HsHostingAsset entity) {
super.preprocessEntity(entity);
if (entity.getIdentifier() == null) {
ofNullable(entity.getParentAsset()).ifPresent(pa -> entity.setIdentifier(pa.getIdentifier() + IDENTIFIER_SUFFIX));
@ -73,7 +73,7 @@ class HsDomainDnsSetupHostingAssetValidator extends HostingAssetEntityValidator
@Override
@SneakyThrows
public List<String> validateContext(final HsHostingAssetEntity assetEntity) {
public List<String> validateContext(final HsHostingAsset assetEntity) {
final var result = super.validateContext(assetEntity);
// TODO.spec: define which checks should get raised to error level
@ -87,7 +87,7 @@ class HsDomainDnsSetupHostingAssetValidator extends HostingAssetEntityValidator
return result;
}
String toZonefileString(final HsHostingAssetEntity assetEntity) {
String toZonefileString(final HsHostingAsset assetEntity) {
// TODO.spec: we need to expand the templates (auto-...) in the same way as in Saltstack
return """
$ORIGIN {domain}.
@ -104,7 +104,7 @@ class HsDomainDnsSetupHostingAssetValidator extends HostingAssetEntityValidator
.replace("{userRRs}", getPropertyValues(assetEntity, "user-RR") );
}
private String fqdn(final HsHostingAssetEntity assetEntity) {
private String fqdn(final HsHostingAsset assetEntity) {
return assetEntity.getIdentifier().substring(0, assetEntity.getIdentifier().length()-IDENTIFIER_SUFFIX.length());
}
}

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsDomainHttpSetupHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -42,12 +42,12 @@ class HsDomainHttpSetupHostingAssetValidator extends HostingAssetEntityValidator
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile("^" + Pattern.quote(assetEntity.getParentAsset().getIdentifier() + IDENTIFIER_SUFFIX) + "$");
}
@Override
public void preprocessEntity(final HsHostingAssetEntity entity) {
public void preprocessEntity(final HsHostingAsset entity) {
super.preprocessEntity(entity);
if (entity.getIdentifier() == null) {
ofNullable(entity.getParentAsset()).ifPresent(pa -> entity.setIdentifier(pa.getIdentifier() + IDENTIFIER_SUFFIX));

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsDomainMboxSetupHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -20,12 +20,12 @@ class HsDomainMboxSetupHostingAssetValidator extends HostingAssetEntityValidator
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile("^" + Pattern.quote(assetEntity.getParentAsset().getIdentifier() + IDENTIFIER_SUFFIX) + "$");
}
@Override
public void preprocessEntity(final HsHostingAssetEntity entity) {
public void preprocessEntity(final HsHostingAsset entity) {
super.preprocessEntity(entity);
if (entity.getIdentifier() == null) {
ofNullable(entity.getParentAsset()).ifPresent(pa -> entity.setIdentifier(pa.getIdentifier() + IDENTIFIER_SUFFIX));

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsDomainSetupHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.List;
import java.util.regex.Pattern;
@ -22,7 +22,7 @@ class HsDomainSetupHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
public List<String> validateEntity(final HsHostingAssetEntity assetEntity) {
public List<String> validateEntity(final HsHostingAsset assetEntity) {
// TODO.impl: for newly created entities, check the permission of setting up a domain
//
// reject, if the domain is any of these:
@ -51,7 +51,7 @@ class HsDomainSetupHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return identifierPattern;
}
}

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsDomainSmtpSetupHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -20,12 +20,12 @@ class HsDomainSmtpSetupHostingAssetValidator extends HostingAssetEntityValidator
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile("^" + Pattern.quote(assetEntity.getParentAsset().getIdentifier() + IDENTIFIER_SUFFIX) + "$");
}
@Override
public void preprocessEntity(final HsHostingAssetEntity entity) {
public void preprocessEntity(final HsHostingAsset entity) {
super.preprocessEntity(entity);
if (entity.getIdentifier() == null) {
ofNullable(entity.getParentAsset()).ifPresent(pa -> entity.setIdentifier(pa.getIdentifier() + IDENTIFIER_SUFFIX));

10
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsEMailAddressHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType;
import java.util.regex.Pattern;
@ -11,7 +11,7 @@ import static net.hostsharing.hsadminng.hs.validation.StringProperty.stringPrope
class HsEMailAddressHostingAssetValidator extends HostingAssetEntityValidator {
private static final String UNIX_USER_REGEX = "^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9]+)?$"; // also accepts legacy pac-names
private static final String UNIX_USER_REGEX = "^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9][a-z0-9\\._-]*)?$"; // also accepts legacy pac-names
private static final String EMAIL_ADDRESS_LOCAL_PART_REGEX = "[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+"; // RFC 5322
private static final String EMAIL_ADDRESS_DOMAIN_PART_REGEX = "[a-zA-Z0-9.-]+";
private static final String EMAIL_ADDRESS_FULL_REGEX = "^" + EMAIL_ADDRESS_LOCAL_PART_REGEX + "@" + EMAIL_ADDRESS_DOMAIN_PART_REGEX + "$";
@ -29,7 +29,7 @@ class HsEMailAddressHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
public void preprocessEntity(final HsHostingAssetEntity entity) {
public void preprocessEntity(final HsHostingAsset entity) {
super.preprocessEntity(entity);
super.preprocessEntity(entity);
if (entity.getIdentifier() == null) {
@ -38,11 +38,11 @@ class HsEMailAddressHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile("^"+ Pattern.quote(combineIdentifier(assetEntity)) + "$");
}
private static String combineIdentifier(final HsHostingAssetEntity emailAddressAssetEntity) {
private static String combineIdentifier(final HsHostingAsset emailAddressAssetEntity) {
return emailAddressAssetEntity.getDirectValue("local-part", String.class) +
ofNullable(emailAddressAssetEntity.getDirectValue("sub-domain", String.class)).map(s -> "." + s).orElse("") +
"@" +

13
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsEMailAliasHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType;
import java.util.regex.Pattern;
@ -10,8 +10,11 @@ import static net.hostsharing.hsadminng.hs.validation.StringProperty.stringPrope
class HsEMailAliasHostingAssetValidator extends HostingAssetEntityValidator {
private static final String UNIX_USER_REGEX = "^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9]+)?$"; // also accepts legacy pac-names
private static final String UNIX_USER_REGEX = "^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9][a-z0-9\\._-]*)?$"; // also accepts legacy pac-names
private static final String EMAIL_ADDRESS_REGEX = "^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$"; // RFC 5322
private static final String INCLUDE_REGEX = "^:include:/.*$";
private static final String PIPE_REGEX = "^\\|.*$";
private static final String DEV_NULL_REGEX = "^/dev/null$";
public static final int EMAIL_ADDRESS_MAX_LENGTH = 320; // according to RFC 5321 and RFC 5322
HsEMailAliasHostingAssetValidator() {
@ -19,13 +22,13 @@ class HsEMailAliasHostingAssetValidator extends HostingAssetEntityValidator {
AlarmContact.isOptional(),
arrayOf(
stringProperty("target").maxLength(EMAIL_ADDRESS_MAX_LENGTH).matchesRegEx(UNIX_USER_REGEX, EMAIL_ADDRESS_REGEX)
stringProperty("target").maxLength(EMAIL_ADDRESS_MAX_LENGTH).matchesRegEx(UNIX_USER_REGEX, EMAIL_ADDRESS_REGEX, INCLUDE_REGEX, PIPE_REGEX, DEV_NULL_REGEX)
).required().minLength(1));
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
final var webspaceIdentifier = assetEntity.getParentAsset().getIdentifier();
return Pattern.compile("^"+webspaceIdentifier+"$|^"+webspaceIdentifier+"-[a-z0-9]+$");
return Pattern.compile("^"+webspaceIdentifier+"$|^"+webspaceIdentifier+"-[a-z0-9][a-z0-9\\._-]*$");
}
}

4
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsIPv4NumberHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -20,7 +20,7 @@ class HsIPv4NumberHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return IPV4_REGEX;
}
}

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsIPv6NumberHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.net.InetAddress;
import java.net.UnknownHostException;
@ -24,7 +24,7 @@ class HsIPv6NumberHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
public List<String> validateEntity(final HsHostingAssetEntity assetEntity) {
public List<String> validateEntity(final HsHostingAsset assetEntity) {
final var violations = super.validateEntity(assetEntity);
if (!isValidIPv6Address(assetEntity.getIdentifier())) {
@ -35,7 +35,7 @@ class HsIPv6NumberHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return IPV6_REGEX;
}

4
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsManagedServerHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -54,7 +54,7 @@ class HsManagedServerHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile("^vm[0-9][0-9][0-9][0-9]$");
}
}

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsManagedWebspaceHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -11,11 +11,11 @@ class HsManagedWebspaceHostingAssetValidator extends HostingAssetEntityValidator
super(
MANAGED_WEBSPACE,
AlarmContact.isOptional(),
NO_EXTRA_PROPERTIES);
NO_EXTRA_PROPERTIES); // TODO.impl: groupid missing, should be equal to main user
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
final var prefixPattern =
!assetEntity.isLoaded()
? assetEntity.getRelatedProject().getDebitor().getDefaultPrefix()

4
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsMariaDbDatabaseHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -18,7 +18,7 @@ class HsMariaDbDatabaseHostingAssetValidator extends HostingAssetEntityValidator
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
final var webspaceIdentifier = assetEntity.getParentAsset().getParentAsset().getIdentifier();
return Pattern.compile("^"+webspaceIdentifier+"$|^"+webspaceIdentifier+"_[a-z0-9_]+$");
}

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsMariaDbInstanceHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -19,7 +19,7 @@ class HsMariaDbInstanceHostingAssetValidator extends HostingAssetEntityValidator
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile(
"^" + Pattern.quote(assetEntity.getParentAsset().getIdentifier()
+ DEFAULT_INSTANCE_IDENTIFIER_SUFFIX)
@ -27,7 +27,7 @@ class HsMariaDbInstanceHostingAssetValidator extends HostingAssetEntityValidator
}
@Override
public void preprocessEntity(final HsHostingAssetEntity entity) {
public void preprocessEntity(final HsHostingAsset entity) {
super.preprocessEntity(entity);
if (entity.getIdentifier() == null) {
ofNullable(entity.getParentAsset()).ifPresent(pa -> entity.setIdentifier(

4
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsMariaDbUserHostingAssetValidator.java
View File

@ -1,7 +1,7 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hash.HashGenerator;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -26,7 +26,7 @@ class HsMariaDbUserHostingAssetValidator extends HostingAssetEntityValidator {
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
final var webspaceIdentifier = assetEntity.getParentAsset().getIdentifier();
return Pattern.compile("^"+webspaceIdentifier+"$|^"+webspaceIdentifier+"_[a-z0-9_]+$");
}

4
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsPostgreSqlDatabaseHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -21,7 +21,7 @@ class HsPostgreSqlDatabaseHostingAssetValidator extends HostingAssetEntityValida
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
final var webspaceIdentifier = assetEntity.getParentAsset().getParentAsset().getIdentifier();
return Pattern.compile("^"+webspaceIdentifier+"$|^"+webspaceIdentifier+"_[a-z0-9_]+$");
}

6
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsPostgreSqlDbInstanceHostingAssetValidator.java
View File

@ -1,6 +1,6 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -21,7 +21,7 @@ class HsPostgreSqlDbInstanceHostingAssetValidator extends HostingAssetEntityVali
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
return Pattern.compile(
"^" + Pattern.quote(assetEntity.getParentAsset().getIdentifier()
+ DEFAULT_INSTANCE_IDENTIFIER_SUFFIX)
@ -29,7 +29,7 @@ class HsPostgreSqlDbInstanceHostingAssetValidator extends HostingAssetEntityVali
}
@Override
public void preprocessEntity(final HsHostingAssetEntity entity) {
public void preprocessEntity(final HsHostingAsset entity) {
super.preprocessEntity(entity);
if (entity.getIdentifier() == null) {
ofNullable(entity.getParentAsset()).ifPresent(pa -> entity.setIdentifier(

4
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsPostgreSqlUserHostingAssetValidator.java
View File

@ -1,7 +1,7 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hash.HashGenerator;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import java.util.regex.Pattern;
@ -26,7 +26,7 @@ class HsPostgreSqlUserHostingAssetValidator extends HostingAssetEntityValidator
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
final var webspaceIdentifier = assetEntity.getParentAsset().getIdentifier();
return Pattern.compile("^"+webspaceIdentifier+"$|^"+webspaceIdentifier+"_[a-z0-9_]+$");
}

39
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsUnixUserHostingAssetValidator.java
View File

@ -1,13 +1,14 @@
package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hash.HashGenerator;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType;
import net.hostsharing.hsadminng.hs.validation.PropertiesProvider;
import jakarta.persistence.EntityManager;
import java.util.regex.Pattern;
import static net.hostsharing.hsadminng.hs.validation.EnumerationProperty.enumerationProperty;
import static net.hostsharing.hsadminng.hs.validation.BooleanProperty.booleanProperty;
import static net.hostsharing.hsadminng.hs.validation.IntegerProperty.integerProperty;
import static net.hostsharing.hsadminng.hs.validation.PasswordProperty.passwordProperty;
import static net.hostsharing.hsadminng.hs.validation.StringProperty.stringProperty;
@ -21,29 +22,39 @@ class HsUnixUserHostingAssetValidator extends HostingAssetEntityValidator {
HsHostingAssetType.UNIX_USER,
AlarmContact.isOptional(),
integerProperty("SSD hard quota").unit("GB").maxFrom("SSD").optional(),
integerProperty("SSD soft quota").unit("GB").maxFrom("SSD hard quota").optional(),
integerProperty("HDD hard quota").unit("GB").maxFrom("HDD").optional(),
integerProperty("HDD soft quota").unit("GB").maxFrom("HDD hard quota").optional(),
enumerationProperty("shell")
.values("/bin/false", "/bin/bash", "/bin/csh", "/bin/dash", "/usr/bin/tcsh", "/usr/bin/zsh", "/usr/bin/passwd")
booleanProperty("locked").readOnly(),
integerProperty("userid").readOnly().initializedBy(HsUnixUserHostingAssetValidator::computeUserId),
integerProperty("SSD hard quota").unit("MB").maxFrom("SSD").withFactor(1024).optional(),
integerProperty("SSD soft quota").unit("MB").maxFrom("SSD hard quota").optional(),
integerProperty("HDD hard quota").unit("MB").maxFrom("HDD").withFactor(1024).optional(),
integerProperty("HDD soft quota").unit("MB").maxFrom("HDD hard quota").optional(),
stringProperty("shell")
// TODO.spec: do we want to change them all to /usr/bin/, also in import?
.provided("/bin/false", "/bin/bash", "/bin/csh", "/bin/dash", "/usr/bin/tcsh", "/usr/bin/zsh", "/usr/bin/passwd")
.withDefault("/bin/false"),
stringProperty("homedir").readOnly().computedBy(HsUnixUserHostingAssetValidator::computeHomedir),
stringProperty("homedir").readOnly().renderedBy(HsUnixUserHostingAssetValidator::computeHomedir),
stringProperty("totpKey").matchesRegEx("^0x([0-9A-Fa-f]{2})+$").minLength(20).maxLength(256).undisclosed().writeOnly().optional(),
passwordProperty("password").minLength(8).maxLength(40).hashedUsing(HashGenerator.Algorithm.LINUX_SHA512).writeOnly());
// TODO.spec: public SSH keys?
// TODO.spec: public SSH keys? (only if hsadmin-ng is only accessible with 2FA)
}
@Override
protected Pattern identifierPattern(final HsHostingAssetEntity assetEntity) {
protected Pattern identifierPattern(final HsHostingAsset assetEntity) {
final var webspaceIdentifier = assetEntity.getParentAsset().getIdentifier();
return Pattern.compile("^"+webspaceIdentifier+"$|^"+webspaceIdentifier+"-[a-z0-9]+$");
return Pattern.compile("^"+webspaceIdentifier+"$|^"+webspaceIdentifier+"-[a-z0-9\\._-]+$");
}
private static String computeHomedir(final PropertiesProvider propertiesProvider) {
final var entity = (HsHostingAssetEntity) propertiesProvider;
private static String computeHomedir(final EntityManager em, final PropertiesProvider propertiesProvider) {
final var entity = (HsHostingAsset) propertiesProvider;
final var webspaceName = entity.getParentAsset().getIdentifier();
return "/home/pacs/" + webspaceName
+ "/users/" + entity.getIdentifier().substring(webspaceName.length()+DASH_LENGTH);
}
private static Integer computeUserId(final EntityManager em, final PropertiesProvider propertiesProvider) {
final Object result = em.createNativeQuery("SELECT nextval('hs_hosting_asset_unixuser_system_id_seq')", Integer.class)
.getSingleResult();
return (Integer) result;
}
}

10
src/main/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/README.md
View File

@ -1,9 +1,9 @@
### HsHostingAssetEntity-Validation
### HsHostingAsset-Validation
There is just a single `HsHostingAssetEntity` class for all types of hosting assets like Managed-Server, Managed-Webspace, Unix-Users, Databases etc. These are distinguished by `HsHostingAssetType HsHostingAssetEntity.type`.
There is just a single `HsHostingAsset` interface and `HsHostingAssetEntity` entity for all types of hosting assets like Managed-Server, Managed-Webspace, Unix-Users, Databases etc. These are distinguished by `HsHostingAssetType HsHostingAsset.type`.
For each of these types, a distinct validator has to be
implemented as a subclass of `HsHostingAssetEntityValidator` which needs to be registered (see `HsHostingAssetEntityValidatorRegistry`) for the relevant type(s).
implemented as a subclass of `HsHostingAssetValidator` which needs to be registered (see `HsHostingAssetValidatorRegistry`) for the relevant type(s).
### Kinds of Validations
@ -21,7 +21,7 @@ References in this context are:
- the Assigned-To-Hosting-Asset and
- the Contact.
The first parameters of the `HsHostingAssetEntityValidator` superclass take rule descriptors for these references. These are all Subclasses fo
The first parameters of the `HsHostingAssetValidator` superclass take rule descriptors for these references. These are all Subclasses fo
### Validation Order
@ -37,4 +37,4 @@ In general, the validation es executed in this order:
2. the limits of the parent entity (parent asset + booking item)
3. limits against the own own-sub-entities
This implementation can be found in `HsHostingAssetEntityValidator.validate`.
This implementation can be found in `HsHostingAssetValidator.validate`.

16
src/main/java/net/hostsharing/hsadminng/hs/validation/HsEntityValidator.java
View File

@ -2,6 +2,7 @@ package net.hostsharing.hsadminng.hs.validation;
import jakarta.persistence.EntityManager;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
@ -13,6 +14,9 @@ import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.Collections.emptyList;
import static net.hostsharing.hsadminng.hs.validation.ValidatableProperty.ComputeMode.IN_INIT;
import static net.hostsharing.hsadminng.hs.validation.ValidatableProperty.ComputeMode.IN_PREP;
import static net.hostsharing.hsadminng.hs.validation.ValidatableProperty.ComputeMode.IN_REVAMP;
// TODO.refa: rename to HsEntityProcessor, also subclasses
public abstract class HsEntityValidator<E extends PropertiesProvider> {
@ -106,21 +110,21 @@ public abstract class HsEntityValidator<E extends PropertiesProvider> {
throw new IllegalArgumentException("Integer value (or null) expected, but got " + value);
}
public void prepareProperties(final E entity) {
public void prepareProperties(final EntityManager em, final E entity) {
stream(propertyValidators).forEach(p -> {
if ( p.isWriteOnly() && p.isComputed()) {
entity.directProps().put(p.propertyName, p.compute(entity));
if (p.isComputed(IN_PREP) || p.isComputed(IN_INIT) && !entity.isLoaded() ) {
entity.directProps().put(p.propertyName, p.compute(em, entity));
}
});
}
public Map<String, Object> revampProperties(final E entity, final Map<String, Object> config) {
public Map<String, Object> revampProperties(final EntityManager em, final E entity, final Map<String, Object> config) {
final var copy = new HashMap<>(config);
stream(propertyValidators).forEach(p -> {
if (p.isWriteOnly()) {
copy.remove(p.propertyName);
} else if (p.isReadOnly() && p.isComputed()) {
copy.put(p.propertyName, p.compute(entity));
} else if (p.isComputed(IN_REVAMP)) {
copy.put(p.propertyName, p.compute(em, entity));
}
});
return copy;

24
src/main/java/net/hostsharing/hsadminng/hs/validation/IntegerProperty.java
View File

@ -7,7 +7,7 @@ import org.apache.commons.lang3.Validate;
import java.util.List;
@Setter
public class IntegerProperty extends ValidatableProperty<IntegerProperty, Integer> {
public class IntegerProperty<P extends IntegerProperty<P>> extends ValidatableProperty<P, Integer> {
private final static String[] KEY_ORDER = Array.join(
ValidatableProperty.KEY_ORDER_HEAD,
@ -19,10 +19,11 @@ public class IntegerProperty extends ValidatableProperty<IntegerProperty, Intege
private String minFrom;
private Integer max;
private String maxFrom;
private Integer factor;
private Integer step;
public static IntegerProperty integerProperty(final String propertyName) {
return new IntegerProperty(propertyName);
public static IntegerProperty<?> integerProperty(final String propertyName) {
return new IntegerProperty<>(propertyName);
}
private IntegerProperty(final String propertyName) {
@ -35,14 +36,19 @@ public class IntegerProperty extends ValidatableProperty<IntegerProperty, Intege
Validate.isTrue(max == null || maxFrom == null, "max and maxFrom are exclusive, but both are given");
}
public IntegerProperty minFrom(final String propertyName) {
public P minFrom(final String propertyName) {
minFrom = propertyName;
return this;
return self();
}
public IntegerProperty maxFrom(final String propertyName) {
public P maxFrom(final String propertyName) {
maxFrom = propertyName;
return this;
return self();
}
public P withFactor(final int factor) {
this.factor = factor;
return self();
}
@Override
@ -62,10 +68,10 @@ public class IntegerProperty extends ValidatableProperty<IntegerProperty, Intege
result.add(propertyName + "' is expected to be multiple of " + step + " but is " + propValue);
}
if (minFrom != null) {
validateMin(result, propertyName, propValue, propProvider.getContextValue(minFrom, Integer.class));
validateMin(result, propertyName, propValue, propProvider.getContextValue(minFrom, Integer.class) * ((factor != null) ? factor : 1));
}
if (maxFrom != null) {
validateMax(result, propertyName, propValue, propProvider.getContextValue(maxFrom, Integer.class, 0));
validateMax(result, propertyName, propValue, propProvider.getContextValue(maxFrom, Integer.class, 0) * ((factor != null) ? factor : 1));
}
}

23
src/main/java/net/hostsharing/hsadminng/hs/validation/PasswordProperty.java
View File

@ -1,8 +1,8 @@
package net.hostsharing.hsadminng.hs.validation;
import lombok.Setter;
import net.hostsharing.hsadminng.hash.HashGenerator;
import net.hostsharing.hsadminng.hash.HashGenerator.Algorithm;
import lombok.Setter;
import java.util.List;
import java.util.stream.Stream;
@ -13,7 +13,10 @@ import static net.hostsharing.hsadminng.mapper.Array.insertNewEntriesAfterExisti
@Setter
public class PasswordProperty extends StringProperty<PasswordProperty> {
private static final String[] KEY_ORDER = insertNewEntriesAfterExistingEntry(StringProperty.KEY_ORDER, "computed", "hashedUsing");
private static final String[] KEY_ORDER = insertNewEntriesAfterExistingEntry(
StringProperty.KEY_ORDER,
"computed",
hsh-michaelhoennig marked this conversation as resolved
hsh-marcsandlus commented 2024-08-01 10:29:15 +02:00
Review
Copy Link

computed muss raus

computed muss raus
hsh-michaelhoennig commented 2024-08-01 11:37:53 +02:00
Review
Copy Link

computed muss doch in die Ausgabe, also z.B. als computed=IN_PREP weil ja die intializedBy und renderedBy Lambdas sind, also Code, den ich gar nicht ausgeben kann. Nur hashedBy kann ausgeben werden, nämlich der Name des Hash-Algorithmus.

computed muss doch in die Ausgabe, also z.B. als computed=IN_PREP weil ja die intializedBy und renderedBy Lambdas sind, also Code, den ich gar nicht ausgeben kann. Nur hashedBy kann ausgeben werden, nämlich der Name des Hash-Algorithmus.
"hashedUsing");
private Algorithm hashedUsing;
@ -34,10 +37,11 @@ public class PasswordProperty extends StringProperty<PasswordProperty> {
public PasswordProperty hashedUsing(final Algorithm algorithm) {
this.hashedUsing = algorithm;
computedBy((entity)
-> ofNullable(entity.getDirectValue(propertyName, String.class))
.map(password -> HashGenerator.using(algorithm).withRandomSalt().hash(password))
.orElse(null));
computedBy(
ComputeMode.IN_PREP,
(em, entity) -> ofNullable(entity.getDirectValue(propertyName, String.class))
.map(password -> HashGenerator.using(algorithm).withRandomSalt().hash(password))
.orElse(null));
return self();
}
@ -69,9 +73,10 @@ public class PasswordProperty extends StringProperty<PasswordProperty> {
}
}
final long groupsCovered = Stream.of(hasLowerCase, hasUpperCase, hasDigit, hasSpecialChar).filter(v->v).count();
if ( groupsCovered < 3) {
result.add(propertyName + "' must contain at least one character of at least 3 of the following groups: upper case letters, lower case letters, digits, special characters");
final long groupsCovered = Stream.of(hasLowerCase, hasUpperCase, hasDigit, hasSpecialChar).filter(v -> v).count();
if (groupsCovered < 3) {
result.add(propertyName
+ "' must contain at least one character of at least 3 of the following groups: upper case letters, lower case letters, digits, special characters");
}
if (containsColon) {
result.add(propertyName + "' must not contain colon (':')");

5
src/main/java/net/hostsharing/hsadminng/hs/validation/PropertiesProvider.java
View File

@ -4,6 +4,7 @@ import java.util.Map;
public interface PropertiesProvider {
boolean isLoaded();
Map<String, Object> directProps();
Object getContextValue(final String propName);
@ -11,6 +12,10 @@ public interface PropertiesProvider {
return cast(propName, directProps().get(propName), clazz, null);
}
default <T> T getDirectValue(final String propName, final Class<T> clazz, final T defaultValue) {
return cast(propName, directProps().get(propName), clazz, defaultValue);
}
default <T> T getContextValue(final String propName, final Class<T> clazz) {
return cast(propName, getContextValue(propName), clazz, null);
}

47
src/main/java/net/hostsharing/hsadminng/hs/validation/ValidatableProperty.java
View File

@ -9,6 +9,7 @@ import net.hostsharing.hsadminng.hs.booking.item.HsBookingItemEntity;
import net.hostsharing.hsadminng.mapper.Array;
import org.apache.commons.lang3.function.TriFunction;
import jakarta.persistence.EntityManager;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
@ -19,6 +20,7 @@ import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.function.BiFunction;
import java.util.function.Function;
import static java.lang.Boolean.FALSE;
@ -46,11 +48,17 @@ public abstract class ValidatableProperty<P extends ValidatableProperty<?, ?>, T
private Set<String> requiresAtMaxOneOf;
private T defaultValue;
protected enum ComputeMode {
IN_INIT,
IN_PREP,
IN_REVAMP
}
@JsonIgnore
private Function<PropertiesProvider, T> computedBy;
private BiFunction<EntityManager, PropertiesProvider, T> computedBy;
@Accessors(makeFinal = true, chain = true, fluent = false)
private boolean computed; // used in descriptor, because computedBy cannot be rendered to a text string
private ComputeMode computed; // name 'computed' instead 'computeMode' for better readability in property description
@Accessors(makeFinal = true, chain = true, fluent = false)
private boolean readOnly;
@ -75,7 +83,7 @@ public abstract class ValidatableProperty<P extends ValidatableProperty<?, ?>, T
return null;
}
protected void setDeferredInit(final Function<ValidatableProperty<?, ?>[], T[]> function) {
protected void setDeferredInit(final Function<ValidatableProperty<?, ?>[], T[]> function) {
this.deferredInit = function;
}
@ -95,7 +103,6 @@ protected void setDeferredInit(final Function<ValidatableProperty<?, ?>[], T[]>
public P readOnly() {
this.readOnly = true;
optional();
return self();
}
@ -137,8 +144,8 @@ protected void setDeferredInit(final Function<ValidatableProperty<?, ?>[], T[]>
if (asTotalLimitValidators == null) {
asTotalLimitValidators = new ArrayList<>();
}
final TriFunction<HsBookingItemEntity, IntegerProperty, Integer, List<String>> validator =
(final HsBookingItemEntity entity, final IntegerProperty prop, final Integer factor) -> {
final TriFunction<HsBookingItemEntity, IntegerProperty<?>, Integer, List<String>> validator =
(final HsBookingItemEntity entity, final IntegerProperty<?> prop, final Integer factor) -> {
final var total = entity.getSubBookingItems().stream()
.map(server -> server.getResources().get(propertyName))
@ -169,11 +176,11 @@ protected void setDeferredInit(final Function<ValidatableProperty<?, ?>[], T[]>
return thresholdPercentage;
}
public ValidatableProperty<P, T> eachComprising(final int factor, final TriFunction<HsBookingItemEntity, IntegerProperty, Integer, List<String>> validator) {
public ValidatableProperty<P, T> eachComprising(final int factor, final TriFunction<HsBookingItemEntity, IntegerProperty<?>, Integer, List<String>> validator) {
if (asTotalLimitValidators == null) {
asTotalLimitValidators = new ArrayList<>();
}
asTotalLimitValidators.add((final HsBookingItemEntity entity) -> validator.apply(entity, (IntegerProperty)this, factor));
asTotalLimitValidators.add((final HsBookingItemEntity entity) -> validator.apply(entity, (IntegerProperty<?>)this, factor));
return this;
}
@ -235,8 +242,8 @@ protected void setDeferredInit(final Function<ValidatableProperty<?, ?>[], T[]>
protected abstract void validate(final List<String> result, final T propValue, final PropertiesProvider propProvider);
public void verifyConsistency(final Map.Entry<? extends Enum<?>, ?> typeDef) {
if (required == null && requiresAtLeastOneOf == null && requiresAtMaxOneOf == null) {
throw new IllegalStateException(typeDef.getKey() + "[" + propertyName + "] not fully initialized, please call either .required(), .optional(), .withDefault(...), .requiresAtLeastOneOf(...) or .requiresAtMaxOneOf(...)" );
if (required == null && requiresAtLeastOneOf == null && requiresAtMaxOneOf == null && !readOnly && defaultValue == null) {
throw new IllegalStateException(typeDef.getKey() + "[" + propertyName + "] not fully initialized, please call either .readOnly(), .required(), .optional(), .withDefault(...), .requiresAtLeastOneOf(...) or .requiresAtMaxOneOf(...)" );
}
}
@ -301,14 +308,26 @@ protected void setDeferredInit(final Function<ValidatableProperty<?, ?>[], T[]>
.toList();
}
public P computedBy(final Function<PropertiesProvider, T> compute) {
public P initializedBy(final BiFunction<EntityManager, PropertiesProvider, T> compute) {
return computedBy(ComputeMode.IN_INIT, compute);
}
public P renderedBy(final BiFunction<EntityManager, PropertiesProvider, T> compute) {
return computedBy(ComputeMode.IN_REVAMP, compute);
}
protected P computedBy(final ComputeMode computeMode, final BiFunction<EntityManager, PropertiesProvider, T> compute) {
this.computedBy = compute;
this.computed = true;
this.computed = computeMode;
return self();
}
public <E extends PropertiesProvider> T compute(final E entity) {
return computedBy.apply(entity);
public boolean isComputed(final ComputeMode computeMode) {
return computed == computeMode;
}
public <E extends PropertiesProvider> T compute(final EntityManager em, final E entity) {
return computedBy.apply(em, entity);
}
@Override

10
src/main/java/net/hostsharing/hsadminng/mapper/PatchableMapWrapper.java
View File

@ -3,6 +3,7 @@ package net.hostsharing.hsadminng.mapper;
import org.apache.commons.lang3.tuple.ImmutablePair;
import jakarta.validation.constraints.NotNull;
import java.util.Arrays;
import java.util.Collection;
import java.util.Map;
import java.util.Set;
@ -56,16 +57,19 @@ public class PatchableMapWrapper<T> implements Map<String, T> {
return "{\n"
+ (
keySet().stream().sorted()
.map(k -> " \"" + k + "\": " + optionallyQuoted(get(k))))
.map(k -> " \"" + k + "\": " + formatted(get(k))))
.collect(joining(",\n")
)
+ "\n}\n";
}
private Object optionallyQuoted(final Object value) {
if ( value instanceof Number || value instanceof Boolean ) {
private Object formatted(final Object value) {
if ( value == null || value instanceof Number || value instanceof Boolean ) {
return value;
}
if ( value.getClass().isArray() ) {
return "\"" + Arrays.toString( (Object[]) value) + "\"";
}
return "\"" + value + "\"";
}

27
src/main/java/net/hostsharing/hsadminng/stringify/Stringify.java
View File

@ -16,9 +16,8 @@ import static java.lang.Boolean.TRUE;
public final class Stringify<B> {
private final Class<B> clazz;
private final String name;
private Function<B, ?> idProp;
private Function<? extends B, ?> idProp;
private final List<Property<B>> props = new ArrayList<>();
private String separator = ", ";
private Boolean quotedValues = null;
@ -31,8 +30,16 @@ public final class Stringify<B> {
return new Stringify<>(clazz, null);
}
public <T extends B> Stringify<T> using(final Class<T> subClass) {
//noinspection unchecked
return (Stringify<T>) new Stringify<T>(subClass, null)
.withIdProp(cast(idProp))
.withProps(cast(props))
.withSeparator(separator)
.quotedValues(quotedValues);
}
private Stringify(final Class<B> clazz, final String name) {
this.clazz = clazz;
if (name != null) {
this.name = name;
} else {
@ -45,7 +52,7 @@ public final class Stringify<B> {
}
}
public Stringify<B> withIdProp(final Function<B, ?> getter) {
public Stringify<B> withIdProp(final Function<? extends B, ?> getter) {
idProp = getter;
return this;
}
@ -60,6 +67,11 @@ public final class Stringify<B> {
return this;
}
private Stringify<B> withProps(final List<Property<B>> props) {
this.props.addAll(props);
return this;
}
public String apply(@NotNull B object) {
final var propValues = props.stream()
.map(prop -> PropertyValue.of(prop, prop.getter.apply(object)))
@ -74,7 +86,7 @@ public final class Stringify<B> {
.map(propVal -> propName(propVal, "=") + optionallyQuoted(propVal))
.collect(Collectors.joining(separator));
return idProp != null
? name + "(" + idProp.apply(object) + ": " + propValues + ")"
? name + "(" + idProp.apply(cast(object)) + ": " + propValues + ")"
: name + "(" + propValues + ")";
}
@ -106,6 +118,11 @@ public final class Stringify<B> {
return this;
}
private <T> T cast(final Object object) {
//noinspection unchecked
return (T)object;
}
private record Property<B>(String name, Function<B, ?> getter) {}
private record PropertyValue<B>(Property<B> prop, Object rawValue, String value) {

15
src/main/resources/db/changelog/7-hs-hosting/701-hosting-asset/7010-hs-hosting-asset.sql
View File

@ -111,6 +111,21 @@ create trigger hs_hosting_asset_type_hierarchy_check_tg
--//
-- ============================================================================
--changeset hosting-asset-system-sequences:1 endDelimiter:--//
-- ----------------------------------------------------------------------------
CREATE SEQUENCE IF NOT EXISTS hs_hosting_asset_unixuser_system_id_seq
AS integer
MINVALUE 1000000
MAXVALUE 9999999
NO CYCLE
OWNED BY NONE;
--//
-- ============================================================================
--changeset hosting-asset-BOOKING-ITEM-HIERARCHY-CHECK:1 endDelimiter:--//
-- ----------------------------------------------------------------------------

4
src/test/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsEMailAddressHostingAssetValidatorUnitTest.java
View File

@ -39,7 +39,7 @@ class HsEMailAddressHostingAssetValidatorUnitTest {
assertThat(validator.properties()).map(Map::toString).containsExactlyInAnyOrder(
"{type=string, propertyName=local-part, matchesRegEx=[^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+$], required=true}",
"{type=string, propertyName=sub-domain, matchesRegEx=[^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+$]}",
"{type=string[], propertyName=target, elementsOf={type=string, propertyName=target, matchesRegEx=[^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9]+)?$, ^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$], maxLength=320}, required=true, minLength=1}");
"{type=string[], propertyName=target, elementsOf={type=string, propertyName=target, matchesRegEx=[^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9][a-z0-9\\._-]*)?$, ^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$], maxLength=320}, required=true, minLength=1}");
}
@Test
@ -73,7 +73,7 @@ class HsEMailAddressHostingAssetValidatorUnitTest {
assertThat(result).containsExactlyInAnyOrder(
"'EMAIL_ADDRESS:test@example.org.config.local-part' is expected to match any of [^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+$] but 'no@allowed' does not match",
"'EMAIL_ADDRESS:test@example.org.config.sub-domain' is expected to match any of [^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+$] but 'no@allowedeither' does not match",
"'EMAIL_ADDRESS:test@example.org.config.target' is expected to match any of [^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9]+)?$, ^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$] but 'garbage' does not match any");
"'EMAIL_ADDRESS:test@example.org.config.target' is expected to match any of [^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9][a-z0-9\\._-]*)?$, ^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$] but 'garbage' does not match any");
}
@Test

30
src/test/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsEMailAliasHostingAssetValidatorUnitTest.java
View File

@ -22,18 +22,24 @@ class HsEMailAliasHostingAssetValidatorUnitTest {
// then
assertThat(validator.properties()).map(Map::toString).containsExactlyInAnyOrder(
"{type=string[], propertyName=target, elementsOf={type=string, propertyName=target, matchesRegEx=[^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9]+)?$, ^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$], maxLength=320}, required=true, minLength=1}");
"{type=string[], propertyName=target, elementsOf={type=string, propertyName=target, matchesRegEx=[^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9][a-z0-9\\._-]*)?$, ^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$, ^:include:/.*$, ^\\|.*$, ^/dev/null$], maxLength=320}, required=true, minLength=1}");
}
@Test
void validatesValidEntity() {
void acceptsValidEntity() {
// given
final var emailAliasHostingAssetEntity = HsHostingAssetEntity.builder()
.type(EMAIL_ALIAS)
.parentAsset(TEST_MANAGED_WEBSPACE_HOSTING_ASSET)
.identifier("xyz00-office")
.config(Map.ofEntries(
entry("target", Array.of("xyz00", "xyz00-abc", "office@example.com"))
entry("target", Array.of(
"xyz00",
"xyz00-abc",
"office@example.com",
"/dev/null",
"|/home/pacs/xyz00/mailinglists/ecartis -s xyz00-intern"
))
))
.build();
final var validator = HostingAssetEntityValidatorRegistry.forType(emailAliasHostingAssetEntity.getType());
@ -46,14 +52,22 @@ class HsEMailAliasHostingAssetValidatorUnitTest {
}
@Test
void validatesProperties() {
void rejectsInvalidConfig() {
// given
final var emailAliasHostingAssetEntity = HsHostingAssetEntity.builder()
.type(EMAIL_ALIAS)
.parentAsset(TEST_MANAGED_WEBSPACE_HOSTING_ASSET)
.identifier("xyz00-office")
.config(Map.ofEntries(
entry("target", Array.of("xyz00", "xyz00-abc", "garbage", "office@example.com"))
entry("target", Array.of(
"/dev/null",
"xyz00",
"xyz00-abc",
"garbage",
"office@example.com",
":include:/home/pacs/xyz00/mailinglists/textfile",
"|/home/pacs/xyz00/mailinglists/executable"
))
))
.build();
final var validator = HostingAssetEntityValidatorRegistry.forType(emailAliasHostingAssetEntity.getType());
@ -63,11 +77,11 @@ class HsEMailAliasHostingAssetValidatorUnitTest {
// then
assertThat(result).containsExactlyInAnyOrder(
"'EMAIL_ALIAS:xyz00-office.config.target' is expected to match any of [^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9]+)?$, ^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$] but 'garbage' does not match any");
"'EMAIL_ALIAS:xyz00-office.config.target' is expected to match any of [^[a-z][a-z0-9]{2}[0-9]{2}(-[a-z0-9][a-z0-9\\._-]*)?$, ^[a-zA-Z0-9_!#$%&'*+/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$, ^:include:/.*$, ^\\|.*$, ^/dev/null$] but 'garbage' does not match any");
}
@Test
void validatesInvalidIdentifier() {
void rejectsInvalidIndentifier() {
// given
final var emailAliasHostingAssetEntity = HsHostingAssetEntity.builder()
.type(EMAIL_ALIAS)
@ -84,7 +98,7 @@ class HsEMailAliasHostingAssetValidatorUnitTest {
// then
assertThat(result).containsExactlyInAnyOrder(
"'identifier' expected to match '^xyz00$|^xyz00-[a-z0-9]+$', but is 'abc00-office'");
"'identifier' expected to match '^xyz00$|^xyz00-[a-z0-9][a-z0-9\\._-]*$', but is 'abc00-office'");
}
@Test

7
src/test/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsMariaDbUserHostingAssetValidatorUnitTest.java
View File

@ -4,6 +4,7 @@ import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity.HsHostingAssetEntityBuilder;
import org.junit.jupiter.api.Test;
import jakarta.persistence.EntityManager;
import java.util.HashMap;
import java.util.stream.Stream;
@ -24,6 +25,8 @@ class HsMariaDbUserHostingAssetValidatorUnitTest {
.caption("some valid test MariaDB-Instance")
.build();
private EntityManager em = null; // not actually needed in these test cases
private static HsHostingAssetEntityBuilder givenValidMariaDbUserBuilder() {
return HsHostingAssetEntity.builder()
.type(MARIADB_USER)
@ -46,7 +49,7 @@ class HsMariaDbUserHostingAssetValidatorUnitTest {
// then
assertThat(props).extracting(Object::toString).containsExactlyInAnyOrder(
"{type=password, propertyName=password, minLength=8, maxLength=40, writeOnly=true, computed=true, hashedUsing=MYSQL_NATIVE, undisclosed=true}"
"{type=password, propertyName=password, minLength=8, maxLength=40, writeOnly=true, computed=IN_PREP, hashedUsing=MYSQL_NATIVE, undisclosed=true}"
);
}
@ -58,7 +61,7 @@ class HsMariaDbUserHostingAssetValidatorUnitTest {
// when
// HashGenerator.nextSalt("Ly3LbsArtL5u4EVt"); // not needed for mysql_native_password
validator.prepareProperties(givenMariaDbUserHostingAsset);
validator.prepareProperties(em, givenMariaDbUserHostingAsset);
// then
assertThat(givenMariaDbUserHostingAsset.getConfig()).containsExactlyInAnyOrderEntriesOf(ofEntries(

7
src/test/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsPostgreSqlUserHostingAssetValidatorUnitTest.java
View File

@ -5,6 +5,7 @@ import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity.HsHostingAssetEntityBuilder;
import org.junit.jupiter.api.Test;
import jakarta.persistence.EntityManager;
import java.nio.charset.Charset;
import java.util.Base64;
import java.util.HashMap;
@ -27,6 +28,8 @@ class HsPostgreSqlUserHostingAssetValidatorUnitTest {
.caption("some valid test PgSql-Instance")
.build();
private EntityManager em = null; // not actually needed in these test cases
private static HsHostingAssetEntityBuilder givenValidMariaDbUserBuilder() {
return HsHostingAssetEntity.builder()
.type(PGSQL_USER)
@ -49,7 +52,7 @@ class HsPostgreSqlUserHostingAssetValidatorUnitTest {
// then
assertThat(props).extracting(Object::toString).containsExactlyInAnyOrder(
"{type=password, propertyName=password, minLength=8, maxLength=40, writeOnly=true, computed=true, hashedUsing=SCRAM_SHA256, undisclosed=true}"
"{type=password, propertyName=password, minLength=8, maxLength=40, writeOnly=true, computed=IN_PREP, hashedUsing=SCRAM_SHA256, undisclosed=true}"
);
}
@ -61,7 +64,7 @@ class HsPostgreSqlUserHostingAssetValidatorUnitTest {
// when
HashGenerator.nextSalt(new String(Base64.getDecoder().decode("L1QxSVNyTU81b3NZS1djNg=="), Charset.forName("latin1")));
validator.prepareProperties(givenMariaDbUserHostingAsset);
validator.prepareProperties(em, givenMariaDbUserHostingAsset);
// then
assertThat(givenMariaDbUserHostingAsset.getConfig()).containsExactlyInAnyOrderEntriesOf(ofEntries(

55
src/test/java/net/hostsharing/hsadminng/hs/hosting/asset/validators/HsUnixUserHostingAssetValidatorUnitTest.java
View File

@ -3,8 +3,14 @@ package net.hostsharing.hsadminng.hs.hosting.asset.validators;
import net.hostsharing.hsadminng.hash.HashGenerator;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import jakarta.persistence.EntityManager;
import jakarta.persistence.Query;
import java.util.HashMap;
import java.util.stream.Stream;
@ -15,7 +21,10 @@ import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.MANA
import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.UNIX_USER;
import static net.hostsharing.hsadminng.mapper.PatchMap.entry;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mock;
@ExtendWith(MockitoExtension.class)
class HsUnixUserHostingAssetValidatorUnitTest {
private final HsHostingAssetEntity TEST_MANAGED_SERVER_HOSTING_ASSET = HsHostingAssetEntity.builder()
@ -43,6 +52,18 @@ class HsUnixUserHostingAssetValidatorUnitTest {
)))
.build();
@Mock
EntityManager em;
@BeforeEach
void initMocks() {
final var nativeQueryMock = mock(Query.class);
lenient().when(nativeQueryMock.getSingleResult()).thenReturn(12345678);
lenient().when(em.createNativeQuery("SELECT nextval('hs_hosting_asset_unixuser_system_id_seq')", Integer.class))
.thenReturn(nativeQueryMock);
}
@Test
void preparesUnixUser() {
// given
@ -51,14 +72,15 @@ class HsUnixUserHostingAssetValidatorUnitTest {
// when
HashGenerator.nextSalt("Ly3LbsArtL5u4EVt");
validator.prepareProperties(unixUserHostingAsset);
validator.prepareProperties(em, unixUserHostingAsset);
// then
assertThat(unixUserHostingAsset.getConfig()).containsExactlyInAnyOrderEntriesOf(ofEntries(
entry("SSD hard quota", 50),
entry("SSD soft quota", 40),
entry("totpKey", "0x123456789abcdef01234"),
entry("password", "$6$Ly3LbsArtL5u4EVt$i/ayIEvm0y4bjkFB6wbg8imbRIaw4mAA4gqYRVyoSkj.iIxJKS3KiRkSjP8gweNcpKL0Q0N31EadT8fCnWErL.")
entry("password", "$6$Ly3LbsArtL5u4EVt$i/ayIEvm0y4bjkFB6wbg8imbRIaw4mAA4gqYRVyoSkj.iIxJKS3KiRkSjP8gweNcpKL0Q0N31EadT8fCnWErL."),
entry("userid", 12345678)
));
}
@ -87,8 +109,8 @@ class HsUnixUserHostingAssetValidatorUnitTest {
.identifier("abc00-temp")
.caption("some test UnixUser with invalid properties")
.config(ofEntries(
entry("SSD hard quota", 100),
entry("SSD soft quota", 200),
entry("SSD hard quota", 60000),
entry("SSD soft quota", 70000),
entry("HDD hard quota", 100),
entry("HDD soft quota", 200),
entry("shell", "/is/invalid"),
@ -104,11 +126,10 @@ class HsUnixUserHostingAssetValidatorUnitTest {
// then
assertThat(result).containsExactlyInAnyOrder(
"'UNIX_USER:abc00-temp.config.SSD hard quota' is expected to be at most 50 but is 100",
"'UNIX_USER:abc00-temp.config.SSD soft quota' is expected to be at most 100 but is 200",
"'UNIX_USER:abc00-temp.config.SSD hard quota' is expected to be at most 51200 but is 60000",
"'UNIX_USER:abc00-temp.config.SSD soft quota' is expected to be at most 60000 but is 70000",
"'UNIX_USER:abc00-temp.config.HDD hard quota' is expected to be at most 0 but is 100",
"'UNIX_USER:abc00-temp.config.HDD soft quota' is expected to be at most 100 but is 200",
"'UNIX_USER:abc00-temp.config.shell' is expected to be one of [/bin/false, /bin/bash, /bin/csh, /bin/dash, /usr/bin/tcsh, /usr/bin/zsh, /usr/bin/passwd] but is '/is/invalid'",
"'UNIX_USER:abc00-temp.config.homedir' is readonly but given as '/is/read-only'",
"'UNIX_USER:abc00-temp.config.totpKey' is expected to match any of [^0x([0-9A-Fa-f]{2})+$] but provided value does not match",
"'UNIX_USER:abc00-temp.config.password' length is expected to be at min 8 but length of provided value is 5",
@ -131,7 +152,7 @@ class HsUnixUserHostingAssetValidatorUnitTest {
// then
assertThat(result).containsExactly(
"'identifier' expected to match '^abc00$|^abc00-[a-z0-9]+$', but is 'xyz99-temp'");
"'identifier' expected to match '^abc00$|^abc00-[a-z0-9\\._-]+$', but is 'xyz99-temp'");
}
@Test
@ -142,7 +163,7 @@ class HsUnixUserHostingAssetValidatorUnitTest {
// when
HashGenerator.nextSalt("Ly3LbsArtL5u4EVt");
final var result = validator.revampProperties(unixUserHostingAsset, unixUserHostingAsset.getConfig());
final var result = validator.revampProperties(em, unixUserHostingAsset, unixUserHostingAsset.getConfig());
// then
assertThat(result).containsExactlyInAnyOrderEntriesOf(ofEntries(
@ -162,14 +183,16 @@ class HsUnixUserHostingAssetValidatorUnitTest {
// then
assertThat(props).extracting(Object::toString).containsExactlyInAnyOrder(
"{type=integer, propertyName=SSD hard quota, unit=GB, maxFrom=SSD}",
"{type=integer, propertyName=SSD soft quota, unit=GB, maxFrom=SSD hard quota}",
"{type=integer, propertyName=HDD hard quota, unit=GB, maxFrom=HDD}",
"{type=integer, propertyName=HDD soft quota, unit=GB, maxFrom=HDD hard quota}",
"{type=enumeration, propertyName=shell, values=[/bin/false, /bin/bash, /bin/csh, /bin/dash, /usr/bin/tcsh, /usr/bin/zsh, /usr/bin/passwd], defaultValue=/bin/false}",
"{type=string, propertyName=homedir, readOnly=true, computed=true}",
"{type=boolean, propertyName=locked, readOnly=true}",
"{type=integer, propertyName=userid, readOnly=true, computed=IN_INIT}",
"{type=integer, propertyName=SSD hard quota, unit=MB, maxFrom=SSD}",
"{type=integer, propertyName=SSD soft quota, unit=MB, maxFrom=SSD hard quota}",
"{type=integer, propertyName=HDD hard quota, unit=MB, maxFrom=HDD}",
"{type=integer, propertyName=HDD soft quota, unit=MB, maxFrom=HDD hard quota}",
"{type=string, propertyName=shell, provided=[/bin/false, /bin/bash, /bin/csh, /bin/dash, /usr/bin/tcsh, /usr/bin/zsh, /usr/bin/passwd], defaultValue=/bin/false}",
"{type=string, propertyName=homedir, readOnly=true, computed=IN_REVAMP}",
"{type=string, propertyName=totpKey, matchesRegEx=[^0x([0-9A-Fa-f]{2})+$], minLength=20, maxLength=256, writeOnly=true, undisclosed=true}",
"{type=password, propertyName=password, minLength=8, maxLength=40, writeOnly=true, computed=true, hashedUsing=LINUX_SHA512, undisclosed=true}"
"{type=password, propertyName=password, minLength=8, maxLength=40, writeOnly=true, computed=IN_PREP, hashedUsing=LINUX_SHA512, undisclosed=true}"
);
}
}

130
src/test/java/net/hostsharing/hsadminng/hs/migration/CsvDataImport.java
View File

@ -3,6 +3,8 @@ package net.hostsharing.hsadminng.hs.migration;
import com.opencsv.CSVParserBuilder;
import com.opencsv.CSVReader;
import com.opencsv.CSVReaderBuilder;
import lombok.SneakyThrows;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.rbac.context.ContextBasedTest;
import net.hostsharing.hsadminng.rbac.rbacobject.RbacObject;
import net.hostsharing.hsadminng.rbac.test.JpaAttempt;
@ -25,18 +27,24 @@ import java.io.InputStreamReader;
import java.io.Reader;
import java.io.StringReader;
import java.io.StringWriter;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.math.BigDecimal;
import java.time.LocalDate;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.UUID;
import java.util.stream.Collectors;
import static java.lang.Boolean.parseBoolean;
import static java.util.Arrays.asList;
import static java.util.Arrays.stream;
import static java.util.Objects.requireNonNull;
import static java.util.Optional.ofNullable;
import static net.hostsharing.hsadminng.mapper.Array.emptyArray;
import static org.apache.commons.lang3.StringUtils.isNotBlank;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assumptions.assumeThat;
@ -68,7 +76,7 @@ public class CsvDataImport extends ContextBasedTest {
@MockBean
HttpServletRequest request;
private static final List<AssertionError> errors = new ArrayList<>();
static final List<String> errors = new ArrayList<>();
public List<String[]> readAllLines(Reader reader) throws Exception {
@ -113,6 +121,16 @@ public class CsvDataImport extends ContextBasedTest {
return records.subList(1, records.size());
}
@SneakyThrows
public static String[] parseCsvLine(final String csvLine) {
try (final var reader = new CSVReader(new StringReader(csvLine))) {
return stream(ofNullable(reader.readNext()).orElse(emptyArray(String.class)))
.map(String::trim)
.map(target -> target.startsWith("'") && target.endsWith("'") ? target.substring(1, target.length()-1) : target)
.toArray(String[]::new);
}
}
String[] trimAll(final String[] record) {
for (int i = 0; i < record.length; ++i) {
if (record[i] != null) {
@ -124,23 +142,75 @@ public class CsvDataImport extends ContextBasedTest {
public <T extends RbacObject> T persist(final Integer id, final T entity) {
try {
final var asString = entity.toString();
if ( asString.contains("'null null, null'") || asString.equals("person()")) {
System.err.println("skipping to persist empty record-id " + id + " #" + entity.hashCode() + ": " + entity);
return entity;
if (entity instanceof HsHostingAsset ha) {
//noinspection unchecked
return (T) persistViaSql(id, ha);
}
//System.out.println("persisting #" + entity.hashCode() + ": " + entity);
em.persist(entity);
// uncomment for debugging purposes
// em.flush(); // makes it slow, but produces better error messages
// System.out.println("persisted #" + entity.hashCode() + " as " + entity.getUuid());
return persistViaEM(id, entity);
} catch (Exception exc) {
System.err.println("failed to persist #" + entity.hashCode() + ": " + entity);
System.err.println(exc);
errors.add("failed to persist #" + entity.hashCode() + ": " + entity);
errors.add(exc.toString());
}
return entity;
}
public <T extends RbacObject> T persistViaEM(final Integer id, final T entity) {
//System.out.println("persisting #" + entity.hashCode() + ": " + entity);
em.persist(entity);
// uncomment for debugging purposes
// em.flush(); // makes it slow, but produces better error messages
// System.out.println("persisted #" + entity.hashCode() + " as " + entity.getUuid());
return entity;
}
@SneakyThrows
public RbacObject<HsHostingAsset> persistViaSql(final Integer id, final HsHostingAsset entity) {
if (entity.getUuid() == null) {
entity.setUuid(UUID.randomUUID());
}
final var query = em.createNativeQuery("""
insert into hs_hosting_asset(
uuid,
type,
bookingitemuuid,
parentassetuuid,
assignedtoassetuuid,
alarmcontactuuid,
identifier,
caption,
config,
version)
values (
:uuid,
:type,
:bookingitemuuid,
:parentassetuuid,
:assignedtoassetuuid,
:alarmcontactuuid,
:identifier,
:caption,
cast(:config as jsonb),
:version)
""")
.setParameter("uuid", entity.getUuid())
.setParameter("type", entity.getType().name())
.setParameter("bookingitemuuid", ofNullable(entity.getBookingItem()).map(RbacObject::getUuid).orElse(null))
.setParameter("parentassetuuid", ofNullable(entity.getParentAsset()).map(RbacObject::getUuid).orElse(null))
.setParameter("assignedtoassetuuid", ofNullable(entity.getAssignedToAsset()).map(RbacObject::getUuid).orElse(null))
.setParameter("alarmcontactuuid", ofNullable(entity.getAlarmContact()).map(RbacObject::getUuid).orElse(null))
.setParameter("identifier", entity.getIdentifier())
.setParameter("caption", entity.getCaption())
.setParameter("config", entity.getConfig().toString())
.setParameter("version", entity.getVersion());
final var count = query.executeUpdate();
logError(() -> {
assertThat(count).isEqualTo(1);
});
return entity;
}
protected <E> String toFormattedString(final Map<Integer, E> map) {
if ( map.isEmpty() ) {
return "{}";
@ -215,12 +285,33 @@ public class CsvDataImport extends ContextBasedTest {
try {
assertion.run();
} catch (final AssertionError exc) {
errors.add(exc);
errors.add(exc.toString());
}
}
void logErrors() {
assumeThat(errors).isEmpty();
assertThat(errors).isEmpty();
}
void expectErrors(final String... expectedErrors) {
assertContainsExactlyInAnyOrderIgnoringWhitespace(errors, expectedErrors);
}
private static class IgnoringWhitespaceComparator implements Comparator<String> {
@Override
public int compare(String s1, String s2) {
return s1.replaceAll("\\s", "").compareTo(s2.replaceAll("\\s", ""));
}
}
public static void assertContainsExactlyInAnyOrderIgnoringWhitespace(final List<String> expected, final List<String> actual) {
final var sortedExpected = expected.stream().map(m -> m.replaceAll("\\s", "")).toList();
final var sortedActual = actual.stream().map(m -> m.replaceAll("\\s", "")).toArray(String[]::new);
assertThat(sortedExpected).containsExactlyInAnyOrder(sortedActual);
}
public static void assertContainsExactlyInAnyOrderIgnoringWhitespace(final List<String> expected, final String... actual) {
assertContainsExactlyInAnyOrderIgnoringWhitespace(expected, asList(actual));
}
}
@ -252,7 +343,7 @@ class Record {
}
String getString(final String columnName) {
return row[columns.indexOf(columnName)];
return row[columns.indexOf(columnName)].trim();
}
boolean isEmpty(final String columnName) {
@ -288,12 +379,17 @@ class Record {
}
}
@Retention(RetentionPolicy.RUNTIME)
@interface ContinueOnFailure {
}
class OrderedDependedTestsExtension implements TestWatcher, BeforeEachCallback {
private static boolean previousTestsPassed = true;
public void testFailed(ExtensionContext context, Throwable cause) {
previousTestsPassed = false;
@Override
public void testFailed(final ExtensionContext context, final Throwable cause) {
previousTestsPassed = previousTestsPassed && context.getElement().map(e -> e.isAnnotationPresent(ContinueOnFailure.class)).orElse(false);
}
@Override

114
src/test/java/net/hostsharing/hsadminng/hs/migration/HsHostingAssetRawEntity.java Normal file
View File

@ -0,0 +1,114 @@
package net.hostsharing.hsadminng.hs.migration;
import io.hypersistence.utils.hibernate.type.json.JsonType;
import lombok.AccessLevel;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import net.hostsharing.hsadminng.hs.booking.item.HsBookingItemEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAsset;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType;
import net.hostsharing.hsadminng.hs.office.contact.HsOfficeContactEntity;
import net.hostsharing.hsadminng.mapper.PatchableMapWrapper;
import org.hibernate.annotations.Type;
import jakarta.persistence.CascadeType;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import jakarta.persistence.FetchType;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.OneToMany;
import jakarta.persistence.OneToOne;
import jakarta.persistence.PostLoad;
import jakarta.persistence.Table;
import jakarta.persistence.Transient;
import jakarta.persistence.Version;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.UUID;
@Builder
@Entity
@Table(name = "hs_hosting_asset")
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class HsHostingAssetRawEntity implements HsHostingAsset {
@Id
@GeneratedValue
private UUID uuid;
@Version
private int version;
@OneToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "bookingitemuuid")
private HsBookingItemEntity bookingItem;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "parentassetuuid")
private HsHostingAssetRawEntity parentAsset;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "assignedtoassetuuid")
private HsHostingAssetRawEntity assignedToAsset;
@Column(name = "type")
@Enumerated(EnumType.STRING)
private HsHostingAssetType type;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "alarmcontactuuid")
private HsOfficeContactEntity alarmContact;
@OneToMany(cascade = CascadeType.REFRESH, orphanRemoval = true, fetch = FetchType.LAZY)
@JoinColumn(name = "parentassetuuid", referencedColumnName = "uuid")
private List<HsHostingAssetRawEntity> subHostingAssets;
@Column(name = "identifier")
private String identifier; // e.g. vm1234, xyz00, example.org, xyz00_abc
@Column(name = "caption")
private String caption;
@Builder.Default
@Setter(AccessLevel.NONE)
@Type(JsonType.class)
@Column(columnDefinition = "config")
private Map<String, Object> config = new HashMap<>();
@Transient
private PatchableMapWrapper<Object> configWrapper;
@Transient
private boolean isLoaded;
@PostLoad
public void markAsLoaded() {
this.isLoaded = true;
}
public PatchableMapWrapper<Object> getConfig() {
return PatchableMapWrapper.of(configWrapper, (newWrapper) -> {configWrapper = newWrapper;}, config);
}
@Override
public Map<String, Object> directProps() {
return config;
}
@Override
public String toString() {
return stringify.using(HsHostingAssetRawEntity.class).apply(this);
}
}

402
src/test/java/net/hostsharing/hsadminng/hs/migration/ImportHostingAssets.java
View File

@ -6,7 +6,6 @@ import net.hostsharing.hsadminng.hs.booking.item.HsBookingItemEntity;
import net.hostsharing.hsadminng.hs.booking.item.HsBookingItemType;
import net.hostsharing.hsadminng.hs.booking.item.validators.HsBookingItemEntityValidatorRegistry;
import net.hostsharing.hsadminng.hs.booking.project.HsBookingProjectEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetEntity;
import net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType;
import net.hostsharing.hsadminng.hs.hosting.asset.validators.HostingAssetEntitySaveProcessor;
import net.hostsharing.hsadminng.rbac.test.JpaAttempt;
@ -23,18 +22,23 @@ import org.springframework.test.annotation.Commit;
import org.springframework.test.annotation.DirtiesContext;
import java.io.Reader;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.concurrent.atomic.AtomicReference;
import java.util.function.Function;
import static java.util.Arrays.stream;
import static java.util.Map.entry;
import static java.util.Optional.ofNullable;
import static java.util.stream.Collectors.toMap;
import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.CLOUD_SERVER;
import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.EMAIL_ALIAS;
import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.IPV4_NUMBER;
import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.MANAGED_SERVER;
import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.MANAGED_WEBSPACE;
import static net.hostsharing.hsadminng.hs.hosting.asset.HsHostingAssetType.UNIX_USER;
import static net.hostsharing.hsadminng.mapper.PostgresDateRange.toPostgresDateRange;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assumptions.assumeThat;
@ -91,13 +95,15 @@ public class ImportHostingAssets extends ImportOfficeData {
static final Integer IP_NUMBER_ID_OFFSET = 1000000;
static final Integer HIVE_ID_OFFSET = 2000000;
static final Integer PACKET_ID_OFFSET = 3000000;
static final Integer UNIXUSER_ID_OFFSET = 4000000;
static final Integer EMAILALIAS_ID_OFFSET = 5000000;
record Hive(int hive_id, String hive_name, int inet_addr_id, AtomicReference<HsHostingAssetEntity> serverRef) {}
record Hive(int hive_id, String hive_name, int inet_addr_id, AtomicReference<HsHostingAssetRawEntity> serverRef) {}
static Map<Integer, HsBookingProjectEntity> bookingProjects = new WriteOnceMap<>();
static Map<Integer, HsBookingItemEntity> bookingItems = new WriteOnceMap<>();
static Map<Integer, Hive> hives = new WriteOnceMap<>();
static Map<Integer, HsHostingAssetEntity> hostingAssets = new WriteOnceMap<>(); // TODO.impl: separate maps for each type?
static Map<Integer, HsHostingAssetRawEntity> hostingAssets = new WriteOnceMap<>(); // TODO.impl: separate maps for each type?
@Test
@Order(11010)
@ -126,14 +132,13 @@ public class ImportHostingAssets extends ImportOfficeData {
void verifyIpNumbers() {
assumeThatWeAreImportingControlledTestData();
// no contacts yet => mostly null values
assertThat(firstOfEachType(5, IPV4_NUMBER)).isEqualToIgnoringWhitespace("""
{
1000363=HsHostingAssetEntity(IPV4_NUMBER, 83.223.95.34),
1000381=HsHostingAssetEntity(IPV4_NUMBER, 83.223.95.52),
1000402=HsHostingAssetEntity(IPV4_NUMBER, 83.223.95.73),
1000433=HsHostingAssetEntity(IPV4_NUMBER, 83.223.95.104),
1000457=HsHostingAssetEntity(IPV4_NUMBER, 83.223.95.128)
1000363=HsHostingAssetRawEntity(IPV4_NUMBER, 83.223.95.34),
1000381=HsHostingAssetRawEntity(IPV4_NUMBER, 83.223.95.52),
1000402=HsHostingAssetRawEntity(IPV4_NUMBER, 83.223.95.73),
1000433=HsHostingAssetRawEntity(IPV4_NUMBER, 83.223.95.104),
1000457=HsHostingAssetRawEntity(IPV4_NUMBER, 83.223.95.128)
}
""");
}
@ -154,7 +159,6 @@ public class ImportHostingAssets extends ImportOfficeData {
void verifyHives() {
assumeThatWeAreImportingControlledTestData();
// no contacts yet => mostly null values
assertThat(toFormattedString(first(5, hives))).isEqualToIgnoringWhitespace("""
{
2000001=Hive[hive_id=1, hive_name=h00, inet_addr_id=358, serverRef=null],
@ -184,13 +188,13 @@ public class ImportHostingAssets extends ImportOfficeData {
assertThat(firstOfEachType(3, CLOUD_SERVER, MANAGED_SERVER, MANAGED_WEBSPACE)).isEqualToIgnoringWhitespace("""
{
3000630=HsHostingAssetEntity(MANAGED_WEBSPACE, hsh00, HA hsh00, MANAGED_SERVER:vm1050, D-1000000:hsh default project:BI hsh00),
3000968=HsHostingAssetEntity(MANAGED_SERVER, vm1061, HA vm1061, D-1015200:rar default project:BI vm1061),
3000978=HsHostingAssetEntity(MANAGED_SERVER, vm1050, HA vm1050, D-1000000:hsh default project:BI vm1050),
3001061=HsHostingAssetEntity(MANAGED_SERVER, vm1068, HA vm1068, D-1000300:mim default project:BI vm1068),
3001094=HsHostingAssetEntity(MANAGED_WEBSPACE, lug00, HA lug00, MANAGED_SERVER:vm1068, D-1000300:mim default project:BI lug00),
3001112=HsHostingAssetEntity(MANAGED_WEBSPACE, mim00, HA mim00, MANAGED_SERVER:vm1068, D-1000300:mim default project:BI mim00),
3023611=HsHostingAssetEntity(CLOUD_SERVER, vm2097, HA vm2097, D-1101800:wws default project:BI vm2097)
3000630=HsHostingAssetRawEntity(MANAGED_WEBSPACE, hsh00, HA hsh00, MANAGED_SERVER:vm1050, D-1000000:hsh default project:BI hsh00),
3000968=HsHostingAssetRawEntity(MANAGED_SERVER, vm1061, HA vm1061, D-1015200:rar default project:BI vm1061),
3000978=HsHostingAssetRawEntity(MANAGED_SERVER, vm1050, HA vm1050, D-1000000:hsh default project:BI vm1050),
3001061=HsHostingAssetRawEntity(MANAGED_SERVER, vm1068, HA vm1068, D-1000300:mim default project:BI vm1068),
3001094=HsHostingAssetRawEntity(MANAGED_WEBSPACE, lug00, HA lug00, MANAGED_SERVER:vm1068, D-1000300:mim default project:BI lug00),
3001112=HsHostingAssetRawEntity(MANAGED_WEBSPACE, mim00, HA mim00, MANAGED_SERVER:vm1068, D-1000300:mim default project:BI mim00),
3023611=HsHostingAssetRawEntity(CLOUD_SERVER, vm2097, HA vm2097, D-1101800:wws default project:BI vm2097)
}
""");
assertThat(firstOfEachType(
@ -226,19 +230,18 @@ public class ImportHostingAssets extends ImportOfficeData {
void verifyPacketComponents() {
assumeThatWeAreImportingControlledTestData();
// no contacts yet => mostly null values
assertThat(firstOfEachType(5, CLOUD_SERVER, MANAGED_SERVER, MANAGED_WEBSPACE))
.isEqualToIgnoringWhitespace("""
{
3000630=HsHostingAssetEntity(MANAGED_WEBSPACE, hsh00, HA hsh00, MANAGED_SERVER:vm1050, D-1000000:hsh default project:BI hsh00),
3000968=HsHostingAssetEntity(MANAGED_SERVER, vm1061, HA vm1061, D-1015200:rar default project:BI vm1061),
3000978=HsHostingAssetEntity(MANAGED_SERVER, vm1050, HA vm1050, D-1000000:hsh default project:BI vm1050),
3001061=HsHostingAssetEntity(MANAGED_SERVER, vm1068, HA vm1068, D-1000300:mim default project:BI vm1068),
3001094=HsHostingAssetEntity(MANAGED_WEBSPACE, lug00, HA lug00, MANAGED_SERVER:vm1068, D-1000300:mim default project:BI lug00),
3001112=HsHostingAssetEntity(MANAGED_WEBSPACE, mim00, HA mim00, MANAGED_SERVER:vm1068, D-1000300:mim default project:BI mim00),
3001447=HsHostingAssetEntity(MANAGED_SERVER, vm1093, HA vm1093, D-1000000:hsh default project:BI vm1093),
3019959=HsHostingAssetEntity(MANAGED_WEBSPACE, dph00, HA dph00, MANAGED_SERVER:vm1093, D-1101900:dph default project:BI dph00),
3023611=HsHostingAssetEntity(CLOUD_SERVER, vm2097, HA vm2097, D-1101800:wws default project:BI vm2097)
3000630=HsHostingAssetRawEntity(MANAGED_WEBSPACE, hsh00, HA hsh00, MANAGED_SERVER:vm1050, D-1000000:hsh default project:BI hsh00),
3000968=HsHostingAssetRawEntity(MANAGED_SERVER, vm1061, HA vm1061, D-1015200:rar default project:BI vm1061),
3000978=HsHostingAssetRawEntity(MANAGED_SERVER, vm1050, HA vm1050, D-1000000:hsh default project:BI vm1050),
3001061=HsHostingAssetRawEntity(MANAGED_SERVER, vm1068, HA vm1068, D-1000300:mim default project:BI vm1068),
3001094=HsHostingAssetRawEntity(MANAGED_WEBSPACE, lug00, HA lug00, MANAGED_SERVER:vm1068, D-1000300:mim default project:BI lug00),
3001112=HsHostingAssetRawEntity(MANAGED_WEBSPACE, mim00, HA mim00, MANAGED_SERVER:vm1068, D-1000300:mim default project:BI mim00),
3001447=HsHostingAssetRawEntity(MANAGED_SERVER, vm1093, HA vm1093, D-1000000:hsh default project:BI vm1093),
3019959=HsHostingAssetRawEntity(MANAGED_WEBSPACE, dph00, HA dph00, MANAGED_SERVER:vm1093, D-1101900:dph default project:BI dph00),
3023611=HsHostingAssetRawEntity(CLOUD_SERVER, vm2097, HA vm2097, D-1101800:wws default project:BI vm2097)
}
""");
assertThat(firstOfEachType(
@ -262,58 +265,247 @@ public class ImportHostingAssets extends ImportOfficeData {
}
@Test
@Order(11400)
@Order(14010)
void importUnixUsers() {
try (Reader reader = resourceReader(MIGRATION_DATA_PATH + "/hosting/unixuser.csv")) {
final var lines = readAllLines(reader);
importUnixUsers(justHeader(lines), withoutHeader(lines));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Test
@Order(14019)
void verifyUnixUsers() {
assumeThatWeAreImportingControlledTestData();
assertThat(firstOfEachType(15, UNIX_USER)).isEqualToIgnoringWhitespace("""
{
4005803=HsHostingAssetRawEntity(UNIX_USER, lug00, LUGs, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/bash", "userid": 102090}),
4005805=HsHostingAssetRawEntity(UNIX_USER, lug00-wla.1, Paul Klemm, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/bash", "userid": 102091}),
4005809=HsHostingAssetRawEntity(UNIX_USER, lug00-wla.2, Walter Müller, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 8, "SSD soft quota": 4, "locked": false, "shell": "/bin/bash", "userid": 102093}),
4005811=HsHostingAssetRawEntity(UNIX_USER, lug00-ola.a, LUG OLA - POP a, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/usr/bin/passwd", "userid": 102094}),
4005813=HsHostingAssetRawEntity(UNIX_USER, lug00-ola.b, LUG OLA - POP b, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/usr/bin/passwd", "userid": 102095}),
4005835=HsHostingAssetRawEntity(UNIX_USER, lug00-test, Test, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 1024, "SSD soft quota": 1024, "locked": false, "shell": "/usr/bin/passwd", "userid": 102106}),
4005964=HsHostingAssetRawEntity(UNIX_USER, mim00, Michael Mellis, MANAGED_WEBSPACE:mim00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/bash", "userid": 102147}),
4005966=HsHostingAssetRawEntity(UNIX_USER, mim00-1981, Jahrgangstreffen 1981, MANAGED_WEBSPACE:mim00, { "SSD hard quota": 256, "SSD soft quota": 128, "locked": false, "shell": "/bin/bash", "userid": 102148}),
4005990=HsHostingAssetRawEntity(UNIX_USER, mim00-mail, Mailbox, MANAGED_WEBSPACE:mim00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/bash", "userid": 102160}),
4100705=HsHostingAssetRawEntity(UNIX_USER, hsh00-mim, Michael Mellis, MANAGED_WEBSPACE:hsh00, { "HDD hard quota": 0, "HDD soft quota": 0, "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/false", "userid": 10003}),
4100824=HsHostingAssetRawEntity(UNIX_USER, hsh00, Hostsharing Paket, MANAGED_WEBSPACE:hsh00, { "HDD hard quota": 0, "HDD soft quota": 0, "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/bash", "userid": 10000}),
4167846=HsHostingAssetRawEntity(UNIX_USER, hsh00-dph, hsh00-uph, MANAGED_WEBSPACE:hsh00, { "HDD hard quota": 0, "HDD soft quota": 0, "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/false", "userid": 110568}),
4169546=HsHostingAssetRawEntity(UNIX_USER, dph00, Reinhard Wiese, MANAGED_WEBSPACE:dph00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/bash", "userid": 110593}),
4169596=HsHostingAssetRawEntity(UNIX_USER, dph00-uph, Domain admin, MANAGED_WEBSPACE:dph00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "shell": "/bin/bash", "userid": 110594})
}
""");
}
@Test
@Order(14020)
void importEmailAliases() {
try (Reader reader = resourceReader(MIGRATION_DATA_PATH + "/hosting/emailalias.csv")) {
final var lines = readAllLines(reader);
importEmailAliases(justHeader(lines), withoutHeader(lines));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Test
@Order(14029)
void verifyEmailAliases() {
assumeThatWeAreImportingControlledTestData();
assertThat(firstOfEachType(15, EMAIL_ALIAS)).isEqualToIgnoringWhitespace("""
{
5002403=HsHostingAssetRawEntity(EMAIL_ALIAS, lug00, lug00, MANAGED_WEBSPACE:lug00, { "target": "[michael.mellis@example.com]"}),
5002405=HsHostingAssetRawEntity(EMAIL_ALIAS, lug00-wla-listar, lug00-wla-listar, MANAGED_WEBSPACE:lug00, { "target": "[|/home/pacs/lug00/users/in/mailinglist/listar]"}),
5002429=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00, mim00, MANAGED_WEBSPACE:mim00, { "target": "[mim12-mi@mim12.hostsharing.net]"}),
5002431=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-abruf, mim00-abruf, MANAGED_WEBSPACE:mim00, { "target": "[michael.mellis@hostsharing.net]"}),
5002449=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-hhfx, mim00-hhfx, MANAGED_WEBSPACE:mim00, { "target": "[mim00-hhfx, |/usr/bin/formail -I 'Reply-To: hamburger-fx@example.net' | /usr/lib/sendmail mim00-hhfx-l]"}),
5002451=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-hhfx-l, mim00-hhfx-l, MANAGED_WEBSPACE:mim00, { "target": "[:include:/home/pacs/mim00/etc/hhfx.list]"}),
5002452=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-empty, mim00-empty, MANAGED_WEBSPACE:mim00, { "target": "[]"}),
5002453=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-0_entries, mim00-0_entries, MANAGED_WEBSPACE:mim00, { "target": "[]"}),
5002454=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-dev.null, mim00-dev.null, MANAGED_WEBSPACE:mim00, { "target": "[/dev/null]"}),
5002455=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-1_with_space, mim00-1_with_space, MANAGED_WEBSPACE:mim00, { "target": "[|/home/pacs/mim00/install/corpslistar/listar]"}),
5002456=HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-1_with_single_quotes, mim00-1_with_single_quotes, MANAGED_WEBSPACE:mim00, { "target": "[|/home/pacs/rir00/mailinglist/ecartis -r kybs06-intern]"})
}
""");
}
// --------------------------------------------------------------------------------------------
@Test
@Order(18010)
void validateBookingItems() {
bookingItems.forEach((id, bi) -> {
try {
HsBookingItemEntityValidatorRegistry.validated(bi);
} catch (final Exception exc) {
System.err.println("validation failed for id:" + id + "( " + bi + "): " + exc.getMessage());
errors.add("validation failed for id:" + id + "( " + bi + "): " + exc.getMessage());
}
});
}
@Test
@Order(11410)
@Order(18020)
void validateHostingAssets() {
hostingAssets.forEach((id, ha) -> {
try {
new HostingAssetEntitySaveProcessor(ha)
new HostingAssetEntitySaveProcessor(em, ha)
.preprocessEntity()
.validateEntity();
.validateEntity()
.prepareForSave();
} catch (final Exception exc) {
System.err.println("validation failed for id:" + id + "( " + ha + "): " + exc.getMessage());
errors.add("validation failed for id:" + id + "( " + ha + "): " + exc.getMessage());
}
});
}
@Test
@Order(18999)
@ContinueOnFailure
void logValidationErrors() {
this.logErrors();
}
// --------------------------------------------------------------------------------------------
@Test
@Order(19000)
@Commit
void persistHostingAssetEntities() {
void persistBookingProjects() {
System.out.println("PERSISTING hosting-assets to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
System.out.println("PERSISTING booking-projects to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
jpaAttempt.transacted(() -> {
context(rbacSuperuser);
bookingProjects.forEach(this::persist);
}).assertSuccessful();
}
@Test
@Order(19010)
@Commit
void persistBookingItems() {
System.out.println("PERSISTING booking-items to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
jpaAttempt.transacted(() -> {
context(rbacSuperuser);
bookingItems.forEach(this::persistRecursively);
}).assertSuccessful();
}
@Test
@Order(19120)
@Commit
void persistCloudServers() {
System.out.println("PERSISTING cloud-servers to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
persistHostingAssetsOfType(CLOUD_SERVER);
}
@Test
@Order(19130)
@Commit
void persistManagedServers() {
System.out.println("PERSISTING managed-servers to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
persistHostingAssetsOfType(MANAGED_SERVER);
}
@Test
@Order(19140)
@Commit
void persistManagedWebspaces() {
System.out.println("PERSISTING managed-webspaces to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
persistHostingAssetsOfType(MANAGED_WEBSPACE);
}
@Test
@Order(19150)
@Commit
void persistIPNumbers() {
System.out.println("PERSISTING ip-numbers to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
persistHostingAssetsOfType(IPV4_NUMBER);
}
@Test
@Order(19160)
@Commit
void persistUnixUsers() {
System.out.println("PERSISTING unix-users to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
persistHostingAssetsOfType(UNIX_USER);
}
@Test
@Order(19170)
@Commit
void persistEmailAliases() {
System.out.println("PERSISTING email-aliases to database '" + jdbcUrl + "' as user '" + postgresAdminUser + "'");
persistHostingAssetsOfType(EMAIL_ALIAS);
}
@Test
@Order(19900)
void verifyPersistedUnixUsersWithUserId() {
assumeThatWeAreImportingControlledTestData();
assertThat(firstOfEachType(15, UNIX_USER)).isEqualToIgnoringWhitespace("""
{
4005803=HsHostingAssetRawEntity(UNIX_USER, lug00, LUGs, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/bash", "userid": 102090}),
4005805=HsHostingAssetRawEntity(UNIX_USER, lug00-wla.1, Paul Klemm, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/bash", "userid": 102091}),
4005809=HsHostingAssetRawEntity(UNIX_USER, lug00-wla.2, Walter Müller, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 8, "SSD soft quota": 4, "locked": false, "password": null, "shell": "/bin/bash", "userid": 102093}),
4005811=HsHostingAssetRawEntity(UNIX_USER, lug00-ola.a, LUG OLA - POP a, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/usr/bin/passwd", "userid": 102094}),
4005813=HsHostingAssetRawEntity(UNIX_USER, lug00-ola.b, LUG OLA - POP b, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/usr/bin/passwd", "userid": 102095}),
4005835=HsHostingAssetRawEntity(UNIX_USER, lug00-test, Test, MANAGED_WEBSPACE:lug00, { "SSD hard quota": 1024, "SSD soft quota": 1024, "locked": false, "password": null, "shell": "/usr/bin/passwd", "userid": 102106}),
4005964=HsHostingAssetRawEntity(UNIX_USER, mim00, Michael Mellis, MANAGED_WEBSPACE:mim00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/bash", "userid": 102147}),
4005966=HsHostingAssetRawEntity(UNIX_USER, mim00-1981, Jahrgangstreffen 1981, MANAGED_WEBSPACE:mim00, { "SSD hard quota": 256, "SSD soft quota": 128, "locked": false, "password": null, "shell": "/bin/bash", "userid": 102148}),
4005990=HsHostingAssetRawEntity(UNIX_USER, mim00-mail, Mailbox, MANAGED_WEBSPACE:mim00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/bash", "userid": 102160}),
4100705=HsHostingAssetRawEntity(UNIX_USER, hsh00-mim, Michael Mellis, MANAGED_WEBSPACE:hsh00, { "HDD hard quota": 0, "HDD soft quota": 0, "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/false", "userid": 10003}),
4100824=HsHostingAssetRawEntity(UNIX_USER, hsh00, Hostsharing Paket, MANAGED_WEBSPACE:hsh00, { "HDD hard quota": 0, "HDD soft quota": 0, "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/bash", "userid": 10000}),
4167846=HsHostingAssetRawEntity(UNIX_USER, hsh00-dph, hsh00-uph, MANAGED_WEBSPACE:hsh00, { "HDD hard quota": 0, "HDD soft quota": 0, "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/false", "userid": 110568}),
4169546=HsHostingAssetRawEntity(UNIX_USER, dph00, Reinhard Wiese, MANAGED_WEBSPACE:dph00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/bash", "userid": 110593}),
4169596=HsHostingAssetRawEntity(UNIX_USER, dph00-uph, Domain admin, MANAGED_WEBSPACE:dph00, { "SSD hard quota": 0, "SSD soft quota": 0, "locked": false, "password": null, "shell": "/bin/bash", "userid": 110594})
}
""");
}
@Test
@Order(19910)
void verifyBookingItemsAreActuallyPersisted() {
final var biCount = (Integer) em.createNativeQuery("SELECT count(*) FROM hs_booking_item", Integer.class).getSingleResult();
assertThat(biCount).isGreaterThan(isImportingControlledTestData() ? 5 : 500);
}
@Test
@Order(19920)
void verifyHostingAssetsAreActuallyPersisted() {
final var haCount = (Integer) em.createNativeQuery("SELECT count(*) FROM hs_hosting_asset", Integer.class).getSingleResult();
assertThat(haCount).isGreaterThan(isImportingControlledTestData() ? 20 : 10000);
}
// ============================================================================================
@Test
@Order(99999)
void logErrors() {
super.logErrors();
if (isImportingControlledTestData()) {
super.expectErrors("""
validation failed for id:5002452( HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-empty, mim00-empty, MANAGED_WEBSPACE:mim00, {
"target": "[]"
}
)): ['EMAIL_ALIAS:mim00-empty.config.target' length is expected to be at min 1 but length of [[]] is 0]""",
"""
validation failed for id:5002453( HsHostingAssetRawEntity(EMAIL_ALIAS, mim00-0_entries, mim00-0_entries, MANAGED_WEBSPACE:mim00, {
"target": "[]"
}
)): ['EMAIL_ALIAS:mim00-0_entries.config.target' length is expected to be at min 1 but length of [[]] is 0]"""
);
} else {
super.logErrors();
}
}
private void persistRecursively(final Integer key, final HsBookingItemEntity bi) {
@ -323,19 +515,21 @@ public class ImportHostingAssets extends ImportOfficeData {
persist(key, HsBookingItemEntityValidatorRegistry.validated(bi));
}
// ============================================================================================
private void persistHostingAssetsOfType(final HsHostingAssetType hsHostingAssetType) {
jpaAttempt.transacted(() -> {
context(rbacSuperuser);
hostingAssets.forEach((key, ha) -> {
if (ha.getType() == hsHostingAssetType) {
new HostingAssetEntitySaveProcessor(ha)
.preprocessEntity()
.validateEntity()
.prepareForSave()
.saveUsing(entity -> persist(key, entity))
.validateContext();
context(rbacSuperuser);
if (ha.getType() == hsHostingAssetType) {
new HostingAssetEntitySaveProcessor(em, ha)
.preprocessEntity()
.validateEntityIgnoring("'EMAIL_ALIAS:.*\\.config\\.target' .*")
.prepareForSave()
.saveUsing(entity -> persist(key, entity))
.validateContext();
}
}
}
);
}).assertSuccessful();
}
@ -346,7 +540,7 @@ public class ImportHostingAssets extends ImportOfficeData {
.map(this::trimAll)
.map(row -> new Record(columns, row))
.forEach(rec -> {
final var ipNumber = HsHostingAssetEntity.builder()
final var ipNumber = HsHostingAssetRawEntity.builder()
.type(IPV4_NUMBER)
.identifier(rec.getString("inet_addr"))
.caption(rec.getString("description"))
@ -402,12 +596,17 @@ public class ImportHostingAssets extends ImportOfficeData {
bookingItems.put(PACKET_ID_OFFSET + packet_id, bookingItem);
final var haType = determineHaType(basepacket_code);
logError(() -> assertThat(!free || haType == MANAGED_WEBSPACE || bookingItem.getRelatedProject().getDebitor().getDefaultPrefix().equals("hsh"))
.as("packet.free only supported for Hostsharing-Assets and ManagedWebspace in customer-ManagedServer, but is set for " + packet_name)
logError(() -> assertThat(!free || haType == MANAGED_WEBSPACE || bookingItem.getRelatedProject()
.getDebitor()
.getDefaultPrefix()
.equals("hsh"))
.as("packet.free only supported for Hostsharing-Assets and ManagedWebspace in customer-ManagedServer, but is set for "
+ packet_name)
.isTrue());
final var asset = HsHostingAssetEntity.builder()
.isLoaded(haType == MANAGED_WEBSPACE) // this turns off identifier validation to accept former default prefixes
final var asset = HsHostingAssetRawEntity.builder()
// this turns off identifier validation to accept former default prefixes
.isLoaded(haType == MANAGED_WEBSPACE)
.type(haType)
.identifier(packet_name)
.bookingItem(bookingItem)
@ -461,9 +660,9 @@ public class ImportHostingAssets extends ImportOfficeData {
case "DAEMON" -> "Daemons";
case "MULTI" -> "Multi";
case "CPU" -> "CPU";
case "RAM" -> returning("RAM", convert = v -> v/1024);
case "QUOTA" -> returning("SSD", convert = v -> v/1024);
case "STORAGE" -> returning("HDD", convert = v -> v/1024);
case "RAM" -> returning("RAM", convert = v -> v / 1024);
case "QUOTA" -> returning("SSD", convert = v -> v / 1024);
case "STORAGE" -> returning("HDD", convert = v -> v / 1024);
case "TRAFFIC" -> "Traffic";
case "OFFICE" -> returning("Online Office Server", convert = v -> v == 1);
@ -526,7 +725,7 @@ public class ImportHostingAssets extends ImportOfficeData {
case "SLAPLAT8H" -> "EXT8H";
default -> throw new IllegalArgumentException("unknown basecomponent_code: " + basecomponent_code);
};
if ( ofNullable(asset.getBookingItem().getResources().get(name)).map("BASIC"::equals).orElse(true) ) {
if (ofNullable(asset.getBookingItem().getResources().get(name)).map("BASIC"::equals).orElse(true)) {
asset.getBookingItem().getResources().put(name, slaValue);
}
} else if (name.startsWith("SLA")) {
@ -537,7 +736,90 @@ public class ImportHostingAssets extends ImportOfficeData {
});
}
<V> V returning(final V value, final Object... assignments) {
private void importUnixUsers(final String[] header, final List<String[]> records) {
final var columns = new Columns(header);
records.stream()
.map(this::trimAll)
.map(row -> new Record(columns, row))
.forEach(rec -> {
final var unixuser_id = rec.getInteger("unixuser_id");
final var packet_id = rec.getInteger("packet_id");
final var unixUserAsset = HsHostingAssetRawEntity.builder()
.type(UNIX_USER)
.parentAsset(hostingAssets.get(PACKET_ID_OFFSET + packet_id))
.identifier(rec.getString("name"))
.caption(rec.getString("comment"))
.isLoaded(true) // avoid overwriting imported userids with generated ids
.config(new HashMap<>(Map.ofEntries(
entry("shell", rec.getString("shell")),
// entry("homedir", rec.getString("homedir")), do not import, it's calculated
entry("locked", rec.getBoolean("locked")),
entry("userid", rec.getInteger("userid")),
entry("SSD soft quota", rec.getInteger("quota_softlimit")),
entry("SSD hard quota", rec.getInteger("quota_hardlimit")),
entry("HDD soft quota", rec.getInteger("storage_softlimit")),
entry("HDD hard quota", rec.getInteger("storage_hardlimit"))
)))
.build();
// TODO.spec: crop SSD+HDD limits if > booked
if (unixUserAsset.getDirectValue("SSD hard quota", Integer.class, 0)
> 1024*unixUserAsset.getContextValue("SSD", Integer.class, 0)) {
unixUserAsset.getConfig().put("SSD hard quota", unixUserAsset.getContextValue("SSD", Integer.class, 0)*1024);
}
if (unixUserAsset.getDirectValue("HDD hard quota", Integer.class, 0)
> 1024*unixUserAsset.getContextValue("HDD", Integer.class, 0)) {
unixUserAsset.getConfig().put("HDD hard quota", unixUserAsset.getContextValue("HDD", Integer.class, 0)*1024);
}
// TODO.spec: does `softlimit<hardlimit?` even make sense? Fix it in this or the other direction?
if (unixUserAsset.getDirectValue("SSD soft quota", Integer.class, 0)
> unixUserAsset.getDirectValue("SSD hard quota", Integer.class, 0)) {
unixUserAsset.getConfig().put("SSD soft quota", unixUserAsset.getConfig().get("SSD hard quota"));
}
if (unixUserAsset.getDirectValue("HDD soft quota", Integer.class, 0)
> unixUserAsset.getDirectValue("HDD hard quota", Integer.class, 0)) {
unixUserAsset.getConfig().put("HDD soft quota", unixUserAsset.getConfig().get("HDD hard quota"));
}
// TODO.spec: remove HDD limits if no HDD storage is booked
if (unixUserAsset.getContextValue("HDD", Integer.class, 0) == 0) {
unixUserAsset.getConfig().remove("HDD hard quota");
unixUserAsset.getConfig().remove("HDD soft quota");
}
hostingAssets.put(UNIXUSER_ID_OFFSET + unixuser_id, unixUserAsset);
});
}
private void importEmailAliases(final String[] header, final List<String[]> records) {
final var columns = new Columns(header);
records.stream()
.map(this::trimAll)
.map(row -> new Record(columns, row))
.forEach(rec -> {
final var unixuser_id = rec.getInteger("emailalias_id");
final var packet_id = rec.getInteger("pac_id");
final var targets = parseCsvLine(rec.getString("target"));
final var unixUserAsset = HsHostingAssetRawEntity.builder()
.type(EMAIL_ALIAS)
.parentAsset(hostingAssets.get(PACKET_ID_OFFSET + packet_id))
.identifier(rec.getString("name"))
.caption(rec.getString("name"))
.config(Map.ofEntries(
entry("target", targets)
))
.build();
hostingAssets.put(EMAILALIAS_ID_OFFSET + unixuser_id, unixUserAsset);
});
}
// ============================================================================================
<V> V returning(
final V value,
@SuppressWarnings("unused") final Object... assignments // DSL-hack: just used for side effects on caller-side
) {
return value;
}
@ -561,7 +843,7 @@ public class ImportHostingAssets extends ImportOfficeData {
};
}
private static HsHostingAssetEntity ipNumber(final Integer inet_addr_id) {
private static HsHostingAssetRawEntity ipNumber(final Integer inet_addr_id) {
return inet_addr_id != null ? hostingAssets.get(IP_NUMBER_ID_OFFSET + inet_addr_id) : null;
}
@ -569,7 +851,7 @@ public class ImportHostingAssets extends ImportOfficeData {
return hive_id != null ? hives.get(HIVE_ID_OFFSET + hive_id) : null;
}
private static HsHostingAssetEntity pac(final Integer packet_id) {
private static HsHostingAssetRawEntity pac(final Integer packet_id) {
return packet_id != null ? hostingAssets.get(PACKET_ID_OFFSET + packet_id) : null;
}
@ -582,7 +864,11 @@ public class ImportHostingAssets extends ImportOfficeData {
.filter(hae -> hae.getValue().getType() == t)
.limit(maxCount)
)
.collect(toMap(Map.Entry::getKey, Map.Entry::getValue)));
.collect(toMap(Map.Entry::getKey, Map.Entry::getValue, ImportHostingAssets::uniqueKeys, TreeMap::new)));
}
protected static <V> V uniqueKeys(final V v1, final V v2) {
throw new RuntimeException(String.format("Duplicate key for values %s and %s", v1, v2));
}
private String firstOfEachType(

9
src/test/java/net/hostsharing/hsadminng/hs/validation/PasswordPropertyUnitTest.java
View File

@ -5,6 +5,7 @@ import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import jakarta.persistence.EntityManager;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
@ -19,6 +20,7 @@ class PasswordPropertyUnitTest {
private final ValidatableProperty<PasswordProperty, String> passwordProp =
passwordProperty("password").minLength(8).maxLength(40).hashedUsing(LINUX_SHA512).writeOnly();
private final List<String> violations = new ArrayList<>();
private EntityManager em = null; // not actually needed in these test cases
@ParameterizedTest
@ValueSource(strings = {
@ -99,7 +101,12 @@ class PasswordPropertyUnitTest {
void shouldComputeHash() {
// when
final var result = passwordProp.compute(new PropertiesProvider() {
final var result = passwordProp.compute(em, new PropertiesProvider() {
@Override
public boolean isLoaded() {
return false;
}
@Override
public Map<String, Object> directProps() {

12
src/test/resources/migration/hosting/emailalias.csv Normal file
View File

@ -0,0 +1,12 @@
emailalias_id;pac_id;name;target
2403;1094;lug00;michael.mellis@example.com
2405;1094;lug00-wla-listar;|/home/pacs/lug00/users/in/mailinglist/listar
2429;1112;mim00;mim12-mi@mim12.hostsharing.net
2431;1112;mim00-abruf;michael.mellis@hostsharing.net
2449;1112;mim00-hhfx;"mim00-hhfx,""|/usr/bin/formail -I 'Reply-To: hamburger-fx@example.net' | /usr/lib/sendmail mim00-hhfx-l"""
2451;1112;mim00-hhfx-l;:include:/home/pacs/mim00/etc/hhfx.list
2452;1112;mim00-empty;
2453;1112;mim00-0_entries;""
2454;1112;mim00-dev.null; /dev/null
2455;1112;mim00-1_with_space;" ""|/home/pacs/mim00/install/corpslistar/listar"""
2456;1112;mim00-1_with_single_quotes;'|/home/pacs/rir00/mailinglist/ecartis -r kybs06-intern'
1 emailalias_id pac_id name target
2 2403 1094 lug00 michael.mellis@example.com
3 2405 1094 lug00-wla-listar |/home/pacs/lug00/users/in/mailinglist/listar
4 2429 1112 mim00 mim12-mi@mim12.hostsharing.net
5 2431 1112 mim00-abruf michael.mellis@hostsharing.net
6 2449 1112 mim00-hhfx mim00-hhfx,"|/usr/bin/formail -I 'Reply-To: hamburger-fx@example.net' | /usr/lib/sendmail mim00-hhfx-l"
7 2451 1112 mim00-hhfx-l :include:/home/pacs/mim00/etc/hhfx.list
8 2452 1112 mim00-empty
9 2453 1112 mim00-0_entries
10 2454 1112 mim00-dev.null /dev/null
11 2455 1112 mim00-1_with_space "|/home/pacs/mim00/install/corpslistar/listar"
12 2456 1112 mim00-1_with_single_quotes '|/home/pacs/rir00/mailinglist/ecartis -r kybs06-intern'

19
src/test/resources/migration/hosting/unixuser.csv Normal file
View File

@ -0,0 +1,19 @@
unixuser_id;name;comment;shell;homedir;locked;packet_id;userid;quota_softlimit;quota_hardlimit;storage_softlimit;storage_hardlimit
100824;hsh00;Hostsharing Paket;/bin/bash;/home/pacs/hsh00;0;630;10000;0;0;0;0
5803;lug00;LUGs;/bin/bash;/home/pacs/lug00;0;1094;102090;0;0;0;0
5805;lug00-wla.1;Paul Klemm;/bin/bash;/home/pacs/lug00/users/deaf;0;1094;102091;4;0;0;0
5809;lug00-wla.2;Walter Müller;/bin/bash;/home/pacs/lug00/users/marl;0;1094;102093;4;8;0;0
5811;lug00-ola.a;LUG OLA - POP a;/usr/bin/passwd;/home/pacs/lug00/users/marl.a;1;1094;102094;0;0;0;0
5813;lug00-ola.b;LUG OLA - POP b;/usr/bin/passwd;/home/pacs/lug00/users/marl.b;1;1094;102095;0;0;0;0
5835;lug00-test;Test;/usr/bin/passwd;/home/pacs/lug00/users/test;0;1094;102106;2000000;4000000;20;0
100705;hsh00-mim;Michael Mellis;/bin/false;/home/pacs/hsh00/users/mi;0;630;10003;0;0;0;0
5964;mim00;Michael Mellis;/bin/bash;/home/pacs/mim00;0;1112;102147;0;0;0;0
5966;mim00-1981;Jahrgangstreffen 1981;/bin/bash;/home/pacs/mim00/users/1981;0;1112;102148;128;256;0;0
5990;mim00-mail;Mailbox;/bin/bash;/home/pacs/mim00/users/mail;0;1112;102160;0;0;0;0
167846;hsh00-dph;hsh00-uph;/bin/false;/home/pacs/hsh00/users/uph;0;630;110568;0;0;0;0
169546;dph00;Reinhard Wiese;/bin/bash;/home/pacs/dph00;0;19959;110593;0;0;0;0
169596;dph00-uph;Domain admin;/bin/bash;/home/pacs/dph00/users/uph;0;19959;110594;0;0;0;0
1 unixuser_id name comment shell homedir locked packet_id userid quota_softlimit quota_hardlimit storage_softlimit storage_hardlimit
2 100824 hsh00 Hostsharing Paket /bin/bash /home/pacs/hsh00 0 630 10000 0 0 0 0
3 5803 lug00 LUGs /bin/bash /home/pacs/lug00 0 1094 102090 0 0 0 0
4 5805 lug00-wla.1 Paul Klemm /bin/bash /home/pacs/lug00/users/deaf 0 1094 102091 4 0 0 0
5 5809 lug00-wla.2 Walter Müller /bin/bash /home/pacs/lug00/users/marl 0 1094 102093 4 8 0 0
6 5811 lug00-ola.a LUG OLA - POP a /usr/bin/passwd /home/pacs/lug00/users/marl.a 1 1094 102094 0 0 0 0
7 5813 lug00-ola.b LUG OLA - POP b /usr/bin/passwd /home/pacs/lug00/users/marl.b 1 1094 102095 0 0 0 0
8 5835 lug00-test Test /usr/bin/passwd /home/pacs/lug00/users/test 0 1094 102106 2000000 4000000 20 0
9 100705 hsh00-mim Michael Mellis /bin/false /home/pacs/hsh00/users/mi 0 630 10003 0 0 0 0
10 5964 mim00 Michael Mellis /bin/bash /home/pacs/mim00 0 1112 102147 0 0 0 0
11 5966 mim00-1981 Jahrgangstreffen 1981 /bin/bash /home/pacs/mim00/users/1981 0 1112 102148 128 256 0 0
12 5990 mim00-mail Mailbox /bin/bash /home/pacs/mim00/users/mail 0 1112 102160 0 0 0 0
13 167846 hsh00-dph hsh00-uph /bin/false /home/pacs/hsh00/users/uph 0 630 110568 0 0 0 0
14 169546 dph00 Reinhard Wiese /bin/bash /home/pacs/dph00 0 19959 110593 0 0 0 0
15 169596 dph00-uph Domain admin /bin/bash /home/pacs/dph00/users/uph 0 19959 110594 0 0 0 0