Skip to content

test(federation): Cover identity provider API - #1644

Merged
gtema merged 1 commit into
mainfrom
integration-tests
Nov 14, 2026
Merged

test(federation): Cover identity provider API#1644
gtema merged 1 commit into
mainfrom
integration-tests

Conversation

@gtema

@gtema gtema commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

Add integration tests for the identity provider provider: get,
update, delete, list filters and paging, issuer lookup precedence
and issuer uniqueness on create and update. Split the tests into
create, get, update, delete, list and issuer submodules.

Signed-off-by: Artem Goncharov artem.goncharov@gmail.com

@gtema
test(federation): Cover identity provider API
40000a2 
Add integration tests for the identity provider provider: get,
update, delete, list filters and paging, issuer lookup precedence
and issuer uniqueness on create and update. Split the tests into
create, get, update, delete, list and issuer submodules.

Signed-off-by: Artem Goncharov <artem.goncharov@gmail.com>
@github-actions

github-actions Bot commented Nov 14, 2026

Copy link
Copy Markdown

🧪 Tempest Identity Compatibility Results (advisory, non-blocking)

rust

Totals
======
Ran: 133 tests in 38.0012 sec.
 - Passed: 89
 - Skipped: 4
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 40
Sum of execute time for each test: 33.8359 sec.
Failed test IDs 
===== FAILED TESTS (target=rust) =====
tempest.api.identity.admin.v3.test_credentials.CredentialsTestJSON.test_credentials_create_get_update_delete[id-7cd59bf9-bda4-4c72-9467-d21cab278355,smoke]
tempest.api.identity.admin.v3.test_domains_negative.DomainsNegativeTestJSON.test_delete_active_domain[gate,id-1f3fbff5-4e44-400d-9ca1-d953f05f609b,negative]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_create_domain_config_and_show_config_groups_and_options[id-9e3ff13c-f597-4f01-9377-d6c06c2a1477]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_create_update_and_delete_domain_config_groups_and_opts[id-c7510fa2-6661-4170-9c6b-4783a80651e9]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_show_default_group_config_and_options[id-11a02bf0-6f94-4380-b3b0-c8dc18fc0d22]
tempest.api.identity.admin.v3.test_default_project_id.TestDefaultProjectId.test_default_project_id[id-d6110661-6a71-49a7-a453-b5e26640ff6d]
setUpClass (tempest.api.identity.admin.v3.test_endpoint_groups.EndPointGroupsTest)
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_check_revoke_roles_on_projects_group[id-26021436-d5a4-4256-943c-ded01e0d4b45]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_check_revoke_roles_on_projects_user[id-18b70e45-7687-4b72-8277-b8f1a47d7591]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_list_check_revoke_roles_on_domains_group[id-c7a8dda2-be50-4fb4-9a9c-e830771078b1]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_list_check_revoke_roles_on_domains_user[id-4e6f0366-97c8-423c-b2be-41eae6ac91c8]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_list_revoke_user_roles_on_domain[id-3acf666e-5354-42ac-8e17-8b68893bcd36]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_list_revoke_user_roles_on_project_tree[id-9f02ccd9-9b57-46b4-8f77-dd5a736f3a06]
tempest.api.identity.admin.v3.test_list_projects.ListProjectsTestJSON.test_list_projects_with_parent[id-6edc66f5-2941-4a17-9526-4073311c1fac]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_create_and_show_consumer[id-c8307ea6-a86c-47fd-ae7b-5b3b2caca76d]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_delete_consumer[id-fdfa1b7f-2a31-4354-b2c7-f6ae20554f93]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_list_consumers[id-09ca50de-78f2-4ffb-ac71-f2254036b2b8]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_update_consumer[id-080a9b1a-c009-47c0-9979-5305bf72e3dc]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_assignments_for_domain_roles[id-3859df7e-5b78-4e4d-b10e-214c8953842a]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_assignments_for_implied_roles_create_delete[id-c8828027-df48-4021-95df-b65b92c7429e]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_implied_domain_roles[id-eb1e1c24-1bc4-4d47-9748-e127a1852c82]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_implied_roles_create_check_show_delete[id-c90c316c-d706-4728-bcba-eb1912081b69]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_role_create_update_show_list[id-18afc6c0-46cf-4911-824e-9989cc056c3a,smoke]
tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_get_trusts_all[id-4773ebd5-ecbf-4255-b8d8-b63e6f72b65d,smoke]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_create_is_domain_project[id-a7eb9416-6f9b-4dbb-b71b-7f73aaef59d5]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_create_with_parent[id-1854f9c0-70bc-4d11-a08a-1c789d339e3d]
tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_get_trusts_query[id-6268b345-87ca-47c0-9ce3-37792b43403a]
tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_trust_expire[id-0ed14b66-cefd-4b5c-a964-65759453e292]
tempest.api.identity.admin.v3.test_tokens.TokensV3TestJSON.test_get_available_domain_scopes[id-ec5ecb05-af64-4c04-ac86-4d9f6f12f185]
tempest.api.identity.v3.test_tokens.TokensV3Test.test_validate_token[id-a9512ac3-3909-48a4-b395-11f438e16260]
tempest.api.identity.admin.v3.test_tokens.TokensV3TestJSON.test_rescope_token[id-565fa210-1da1-4563-999b-f7b5b67cf112]
tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_trust_impersonate[id-5a0a91a4-baef-4a14-baba-59bf4d7fcace]
tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_trust_noimpersonate[id-ed2a8779-a7ac-49dc-afd7-30f32f936ed2]
tempest.api.identity.admin.v3.test_users_negative.UsersNegativeTest.test_create_user_for_non_existent_domain[id-e75f006c-89cc-477b-874d-588e4eab4b17,negative]
tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_create_application_credential_expires[id-852daf0c-42b5-4239-8466-d193d0543ed3]
tempest.api.identity.v3.test_catalog.IdentityCatalogTest.test_catalog_standardization[id-56b57ced-22b8-4127-9b8a-565dfb0207e2]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_media_types[id-657c1970-4722-4189-8831-7325f3bc4265,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_resources[id-b9232f5e-d9e5-4d97-b96c-28d3db4de1bd,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_identity_v3_existence[id-79aec9ae-710f-4c54-a4fc-3aa25b4feac3]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_list_api_versions[id-721f480f-35b6-46c7-846e-047e6acea0dc,smoke]
===== END FAILED TESTS (target=rust) =====

View full run

@github-actions

github-actions Bot commented Nov 14, 2026

Copy link
Copy Markdown

🦢 Load Test Results

Goose Attack Report

Plan Overview

Action Started Stopped Elapsed Users
Increasing 26-11-14 09:04:39 26-11-14 09:05:01 00:00:22 0 → 45
Maintaining 26-11-14 09:05:01 26-11-14 09:05:31 00:00:30 45
Decreasing 26-11-14 09:05:31 26-11-14 09:05:32 00:00:01 0 ← 45

Request Metrics

Method Name # Requests # Fails Average (ms) Min (ms) Max (ms) RPS Failures/s
DELETE DELETE /SCIM/v2/:domain_id/Users/:id (sync offboard) 1 0 1.47 1.47 1.47 0.03 0.00
DELETE DELETE /v3/OS-TRUST/trusts/:id 18 0 333 253 429 0.60 0.00
DELETE DELETE /v3/auth/tokens 71 0 239 54 385 2.37 0.00
DELETE DELETE /v3/auth/tokens (churn) 20 0 309 49 453 0.67 0.00
DELETE DELETE /v3/credentials/:id 20 0 215 7.22 312 0.67 0.00
DELETE DELETE /v3/endpoints/:id 28 0 188 17 271 0.93 0.00
DELETE DELETE /v3/groups/:id 30 0 253 16 432 1.00 0.00
DELETE DELETE /v3/groups/:id/users/:id 30 0 295 27 443 1.00 0.00
DELETE DELETE /v3/projects/:id (churn) 20 0 354 25 481 0.67 0.00
DELETE DELETE /v3/projects/:id (teardown) 1 0 99 99 99 0.03 0.00
DELETE DELETE /v3/projects/:id/users/:id/roles/:id (churn) 20 0 229 35 316 0.67 0.00
DELETE DELETE /v3/regions/:id 28 0 189 9.33 292 0.93 0.00
DELETE DELETE /v3/roles/:id (teardown) 1 0 154 154 154 0.03 0.00
DELETE DELETE /v3/services/:id 28 0 192 10 343 0.93 0.00
DELETE DELETE /v3/users/:id (churn) 38 0 360 12 536 1.27 0.00
DELETE DELETE /v3/users/:id (teardown) 2 0 158 147 170 0.07 0.00
DELETE DELETE /v4/mappings/rulesets/:mapping_id (cc teardown) 1 0 36 36 36 0.03 0.00
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 1 0 93 93 93 0.03 0.00
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (cc teardown) 1 0 40 40 40 0.03 0.00
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 1 0 87 87 87 0.03 0.00
GET 83 0 212 138 342 2.77 0.00
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 15 0 345 5.23 586 0.50 0.00
└─ 200 (80.0%) 12 430 326 586
└─ 429 (20.0%) 3 6.29 5.23 8.12
GET GET /SCIM/v2/:domain_id/Users/:id (sync reconcile show) 15 0 174 3.16 590 0.50 0.00
└─ 200 (33.3%) 5 509 458 590
└─ 429 (66.7%) 10 5.71 3.16 10
GET GET /v3/OS-TRUST/trusts (by trustor) 18 0 227 147 297 0.60 0.00
GET GET /v3/OS-TRUST/trusts/:id 18 0 252 171 360 0.60 0.00
GET GET /v3/auth/tokens (churn validate) 20 0 282 120 424 0.67 0.00
GET GET /v3/auth/tokens (trustor) 18 0 256 149 362 0.60 0.00
GET GET /v3/auth/tokens (validate new) 71 0 214 101 372 2.37 0.00
GET GET /v3/auth/tokens (validate revoked) 71 0 211 30 323 2.37 0.00
GET GET /v3/auth/tokens (validate token) 218 0 270 146 418 7.27 0.00
GET GET /v3/credentials (by user) 20 0 169 4.94 288 0.67 0.00
GET GET /v3/credentials/:id 20 0 161 6.18 247 0.67 0.00
GET GET /v3/endpoints (by service) 27 0 181 25 271 0.90 0.00
GET GET /v3/endpoints/:id 27 0 171 31 306 0.90 0.00
GET GET /v3/groups 30 0 169 23 267 1.00 0.00
GET GET /v3/groups/:id 30 0 172 31 252 1.00 0.00
GET GET /v3/groups/:id/users 30 0 272 106 402 1.00 0.00
GET GET /v3/projects/:id 161 0 183 123 290 5.37 0.00
GET GET /v3/projects/:id (catalog) 144 0 183 116 276 4.80 0.00
GET GET /v3/projects?domain_id 229 0 236 119 410 7.63 0.00
GET GET /v3/regions/:id 26 0 172 109 234 0.87 0.00
GET GET /v3/role_assignments 80 0 204 138 300 2.67 0.00
GET GET /v3/role_assignments?role.id 80 0 186 116 313 2.67 0.00
GET GET /v3/role_assignments?scope.domain.id 81 0 179 107 316 2.70 0.00
GET GET /v3/role_assignments?user.id 80 0 183 111 292 2.67 0.00
GET GET /v3/roles 289 0 209 123 335 9.63 0.00
GET GET /v3/roles/:id 199 0 151 86 325 6.63 0.00
GET GET /v3/services 27 0 174 24 267 0.90 0.00
GET GET /v3/services/:id 26 0 190 78 333 0.87 0.00
GET GET /v3/users/:id 308 0 184 114 329 10.27 0.00
GET GET /v3/users/:id (catalog) 142 0 186 106 301 4.73 0.00
GET GET /v3/users?domain_id 227 0 241 157 404 7.57 0.00
GET GET /v4/api-keys 96 0 158 91 255 3.20 0.00
GET GET /v4/api-keys/:client_id 96 0 157 92 243 3.20 0.00
GET GET /v4/mappings/rulesets 94 0 159 96 320 3.13 0.00
GET GET /v4/mappings/rulesets/:mapping_id 94 0 161 99 252 3.13 0.00
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 2712 0 11 1.94 99 90.40 0.00
GET GET /v4/oauth2/:domain_id/clients 94 0 161 105 293 3.13 0.00
GET GET /v4/oauth2/:domain_id/clients/:provider_id 95 0 159 103 238 3.17 0.00
GET GET /v4/oauth2/:domain_id/jwks 2712 0 11 1.93 81 90.40 0.00
GET GET /v4/scim_realms 61 0 151 102 253 2.03 0.00
GET GET /v4/scim_realms/:domain_id/:provider_id 61 0 163 111 245 2.03 0.00
HEAD HEAD /v3/groups/:id/users/:id 30 0 247 165 337 1.00 0.00
PATCH PATCH /SCIM/v2/:domain_id/Users/:id (sync attribute drift) 15 0 549 3.26 752 0.50 0.00
└─ 200 (93.3%) 14 588 451 752
└─ 429 (6.7%) 1 3.26 3.26 3.26
PATCH PATCH /v3/credentials/:id 20 0 276 8.2 390 0.67 0.00
PATCH PATCH /v3/groups/:id 30 0 242 39 352 1.00 0.00
PATCH PATCH /v3/services/:id 28 0 224 19 312 0.93 0.00
POST POST /v3/OS-TRUST/trusts 18 0 294 207 451 0.60 0.00
POST POST /v3/auth/tokens 71 0 158 98 242 2.37 0.00
POST POST /v3/auth/tokens (churn user) 20 0 1730 1284 2769 0.67 0.00
POST POST /v3/auth/tokens (password, invalid) 20 0 1624 1158 2520 0.67 0.00
POST POST /v3/auth/tokens (password, project-scoped) 19 0 1725 1135 2675 0.63 0.00
POST POST /v3/auth/tokens (password, unscoped) 19 0 1769 1358 3305 0.63 0.00
POST POST /v3/auth/tokens (rescope to system) 289 0 209 120 364 9.63 0.00
POST POST /v3/auth/tokens (rescope, invalid project) 165 0 183 98 326 5.50 0.00
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 36 0 1752 1241 2968 1.20 0.00
POST POST /v3/credentials 20 0 173 8.29 248 0.67 0.00
POST POST /v3/endpoints 26 0 210 145 310 0.87 0.00
POST POST /v3/groups 28 0 173 125 261 0.93 0.00
POST POST /v3/projects (churn) 18 0 189 146 276 0.60 0.00
POST POST /v3/projects (setup) 1 0 243 243 243 0.03 0.00
POST POST /v3/regions 26 0 182 133 242 0.87 0.00
POST POST /v3/services 26 0 192 127 343 0.87 0.00
POST POST /v3/users (churn) 38 0 1791 1315 3326 1.27 0.00
POST POST /v3/users (setup) 1 0 2636 2636 2636 0.03 0.00
POST POST /v4/api-keys/:client_id/revoke (teardown) 1 0 87 87 87 0.03 0.00
POST POST /v4/oauth2/:domain_id/device_authorization 1149 0 26 7.11 121 38.30 0.00
POST POST /v4/oauth2/:domain_id/token (client_credentials) 16 0 405 264 647 0.53 0.00
PUT PUT /v3/groups/:id/users/:id 29 0 238 180 306 0.97 0.00
PUT PUT /v3/projects/:id/users/:id/roles/:id (churn) 19 0 216 134 279 0.63 0.00
PUT PUT /v4/scim_realms/:domain_id/:provider_id 61 0 180 111 288 2.03 0.00
Aggregated 11544 0 116 1.47 3326 384.80 0.00

Response Time Metrics

Method Name 50%ile (ms) 60%ile (ms) 70%ile (ms) 80%ile (ms) 90%ile (ms) 95%ile (ms) 99%ile (ms) 100%ile (ms)
DELETE DELETE /SCIM/v2/:domain_id/Users/:id (sync offboard) 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47
DELETE DELETE /v3/OS-TRUST/trusts/:id 309 354 362 371 399 412 429 429
DELETE DELETE /v3/auth/tokens 237 251 264 268 293 313 358 385
DELETE DELETE /v3/auth/tokens (churn) 297 313 326 375 440 444 453 453
DELETE DELETE /v3/credentials/:id 222 234 268 272 276 301 312 312
DELETE DELETE /v3/endpoints/:id 196 202 218 224 245 261 271 271
DELETE DELETE /v3/groups/:id 257 268 293 305 334 346 432 432
DELETE DELETE /v3/groups/:id/users/:id 301 317 338 354 379 424 443 443
DELETE DELETE /v3/projects/:id (churn) 383 387 416 416 428 461 481 481
DELETE DELETE /v3/projects/:id (teardown) 99 99 99 99 99 99 99 99
DELETE DELETE /v3/projects/:id/users/:id/roles/:id (churn) 234 253 261 272 305 309 316 316
DELETE DELETE /v3/regions/:id 196 202 214 224 247 253 292 292
DELETE DELETE /v3/roles/:id (teardown) 154 154 154 154 154 154 154 154
DELETE DELETE /v3/services/:id 189 206 224 228 245 251 343 343
DELETE DELETE /v3/users/:id (churn) 379 387 399 412 453 457 536 536
DELETE DELETE /v3/users/:id (teardown) 147 147 147 170 170 170 170 170
DELETE DELETE /v4/mappings/rulesets/:mapping_id (cc teardown) 36 36 36 36 36 36 36 36
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 93 93 93 93 93 93 93 93
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (cc teardown) 40 40 40 40 40 40 40 40
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 87 87 87 87 87 87 87 87
GET 212 220 230 239 253 272 281 342
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 379 399 440 481 578 578 586 586
GET GET /SCIM/v2/:domain_id/Users/:id (sync reconcile show) 7.65 8.77 457 481 514 514 590 590
GET GET /v3/OS-TRUST/trusts (by trustor) 222 241 249 251 261 268 297 297
GET GET /v3/OS-TRUST/trusts/:id 249 261 272 285 285 293 360 360
GET GET /v3/auth/tokens (churn validate) 276 281 289 326 342 379 424 424
GET GET /v3/auth/tokens (trustor) 264 268 276 276 317 322 362 362
GET GET /v3/auth/tokens (validate new) 212 220 228 241 268 297 342 372
GET GET /v3/auth/tokens (validate revoked) 214 224 232 249 268 276 317 323
GET GET /v3/auth/tokens (validate token) 264 276 289 301 334 350 412 418
GET GET /v3/credentials (by user) 171 189 196 216 228 272 288 288
GET GET /v3/credentials/:id 169 175 194 198 200 220 247 247
GET GET /v3/endpoints (by service) 183 187 202 212 214 259 271 271
GET GET /v3/endpoints/:id 171 179 181 194 224 285 306 306
GET GET /v3/groups 169 187 189 206 216 234 267 267
GET GET /v3/groups/:id 171 181 194 210 220 239 252 252
GET GET /v3/groups/:id/users 281 285 293 313 317 362 402 402
GET GET /v3/projects/:id 179 185 198 218 239 245 268 290
GET GET /v3/projects/:id (catalog) 177 187 202 212 239 243 268 276
GET GET /v3/projects?domain_id 232 245 257 272 289 317 371 410
GET GET /v3/regions/:id 165 179 187 200 212 230 234 234
GET GET /v3/role_assignments 200 208 222 237 245 272 300 300
GET GET /v3/role_assignments?role.id 179 189 198 208 249 272 305 313
GET GET /v3/role_assignments?scope.domain.id 177 185 194 204 222 253 272 316
GET GET /v3/role_assignments?user.id 179 187 200 212 228 251 255 292
GET GET /v3/roles 204 214 228 243 259 285 313 335
GET GET /v3/roles/:id 148 153 165 179 198 212 243 325
GET GET /v3/services 171 183 194 204 245 253 267 267
GET GET /v3/services/:id 183 200 202 218 228 239 333 333
GET GET /v3/users/:id 179 185 198 212 230 253 301 329
GET GET /v3/users/:id (catalog) 181 194 204 218 239 253 281 301
GET GET /v3/users?domain_id 232 251 261 281 301 330 367 404
GET GET /v4/api-keys 155 167 181 189 204 216 245 255
GET GET /v4/api-keys/:client_id 153 159 175 187 196 214 241 243
GET GET /v4/mappings/rulesets 153 161 169 179 204 232 272 320
GET GET /v4/mappings/rulesets/:mapping_id 159 165 177 187 204 224 249 252
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 9.15 11 12 15 19 24 36 99
GET GET /v4/oauth2/:domain_id/clients 157 161 171 194 212 214 249 293
GET GET /v4/oauth2/:domain_id/clients/:provider_id 159 163 173 185 208 218 234 238
GET GET /v4/oauth2/:domain_id/jwks 9.15 11 12 15 19 24 37 81
GET GET /v4/scim_realms 144 155 165 171 198 210 220 253
GET GET /v4/scim_realms/:domain_id/:provider_id 155 163 175 196 218 228 243 245
HEAD HEAD /v3/groups/:id/users/:id 247 253 261 281 301 313 337 337
PATCH PATCH /SCIM/v2/:domain_id/Users/:id (sync attribute drift) 578 594 618 651 717 717 752 752
PATCH PATCH /v3/credentials/:id 305 313 330 350 367 371 390 390
PATCH PATCH /v3/groups/:id 255 259 272 281 301 338 352 352
PATCH PATCH /v3/services/:id 241 247 251 257 276 297 312 312
POST POST /v3/OS-TRUST/trusts 264 281 317 330 379 391 451 451
POST POST /v3/auth/tokens 155 163 169 181 204 220 241 242
POST POST /v3/auth/tokens (churn user) 1,696 1,729 1,761 1,810 2,056 2,572 2,769 2,769
POST POST /v3/auth/tokens (password, invalid) 1,614 1,614 1,679 1,696 1,876 2,441 2,520 2,520
POST POST /v3/auth/tokens (password, project-scoped) 1,679 1,729 1,810 1,860 2,023 2,179 2,675 2,675
POST POST /v3/auth/tokens (password, unscoped) 1,614 1,630 1,729 1,827 2,343 2,540 3,305 3,305
POST POST /v3/auth/tokens (rescope to system) 204 216 226 241 261 281 330 364
POST POST /v3/auth/tokens (rescope, invalid project) 175 185 202 216 234 257 285 326
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 1,630 1,663 1,745 1,942 2,179 2,376 2,968 2,968
POST POST /v3/credentials 189 196 200 212 232 234 248 248
POST POST /v3/endpoints 208 224 224 247 251 268 310 310
POST POST /v3/groups 163 175 179 191 212 253 261 261
POST POST /v3/projects (churn) 175 198 214 214 230 237 276 276
POST POST /v3/projects (setup) 243 243 243 243 243 243 243 243
POST POST /v3/regions 181 187 202 208 212 239 242 242
POST POST /v3/services 187 194 210 224 232 293 343 343
POST POST /v3/users (churn) 1,663 1,745 1,778 1,827 2,073 3,064 3,326 3,326
POST POST /v3/users (setup) 2,636 2,636 2,636 2,636 2,636 2,636 2,636 2,636
POST POST /v4/api-keys/:client_id/revoke (teardown) 87 87 87 87 87 87 87 87
POST POST /v4/oauth2/:domain_id/device_authorization 23 26 29 33 42 50 72 121
POST POST /v4/oauth2/:domain_id/token (client_credentials) 362 403 432 457 502 561 647 647
PUT PUT /v3/groups/:id/users/:id 239 243 249 268 289 305 306 306
PUT PUT /v3/projects/:id/users/:id/roles/:id (churn) 220 228 249 259 272 276 279 279
PUT PUT /v4/scim_realms/:domain_id/:provider_id 177 183 206 214 228 237 272 288
Aggregated 24 130 169 202 247 289 1,483 3,326

Status Code Metrics

Method Name Status Codes
DELETE DELETE /SCIM/v2/:domain_id/Users/:id (sync offboard) 1 [429]
DELETE DELETE /v3/OS-TRUST/trusts/:id 18 [204]
DELETE DELETE /v3/auth/tokens 71 [204]
DELETE DELETE /v3/auth/tokens (churn) 20 [204]
DELETE DELETE /v3/credentials/:id 20 [204]
DELETE DELETE /v3/endpoints/:id 28 [204]
DELETE DELETE /v3/groups/:id 30 [204]
DELETE DELETE /v3/groups/:id/users/:id 30 [204]
DELETE DELETE /v3/projects/:id (churn) 20 [204]
DELETE DELETE /v3/projects/:id (teardown) 1 [204]
DELETE DELETE /v3/projects/:id/users/:id/roles/:id (churn) 20 [204]
DELETE DELETE /v3/regions/:id 28 [204]
DELETE DELETE /v3/roles/:id (teardown) 1 [204]
DELETE DELETE /v3/services/:id 28 [204]
DELETE DELETE /v3/users/:id (churn) 38 [204]
DELETE DELETE /v3/users/:id (teardown) 2 [204]
DELETE DELETE /v4/mappings/rulesets/:mapping_id (cc teardown) 1 [204]
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 1 [204]
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (cc teardown) 1 [204]
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 1 [204]
GET 83 [200]
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 3 [429], 12 [200]
GET GET /SCIM/v2/:domain_id/Users/:id (sync reconcile show) 5 [200], 10 [429]
GET GET /v3/OS-TRUST/trusts (by trustor) 18 [200]
GET GET /v3/OS-TRUST/trusts/:id 18 [200]
GET GET /v3/auth/tokens (churn validate) 20 [200]
GET GET /v3/auth/tokens (trustor) 18 [200]
GET GET /v3/auth/tokens (validate new) 71 [200]
GET GET /v3/auth/tokens (validate revoked) 71 [404]
GET GET /v3/auth/tokens (validate token) 218 [200]
GET GET /v3/credentials (by user) 20 [200]
GET GET /v3/credentials/:id 20 [200]
GET GET /v3/endpoints (by service) 27 [200]
GET GET /v3/endpoints/:id 27 [200]
GET GET /v3/groups 30 [200]
GET GET /v3/groups/:id 30 [200]
GET GET /v3/groups/:id/users 30 [200]
GET GET /v3/projects/:id 161 [200]
GET GET /v3/projects/:id (catalog) 144 [200]
GET GET /v3/projects?domain_id 229 [200]
GET GET /v3/regions/:id 26 [200]
GET GET /v3/role_assignments 80 [200]
GET GET /v3/role_assignments?role.id 80 [200]
GET GET /v3/role_assignments?scope.domain.id 81 [200]
GET GET /v3/role_assignments?user.id 80 [200]
GET GET /v3/roles 289 [200]
GET GET /v3/roles/:id 199 [200]
GET GET /v3/services 27 [200]
GET GET /v3/services/:id 26 [200]
GET GET /v3/users/:id 308 [200]
GET GET /v3/users/:id (catalog) 142 [200]
GET GET /v3/users?domain_id 227 [200]
GET GET /v4/api-keys 96 [200]
GET GET /v4/api-keys/:client_id 96 [200]
GET GET /v4/mappings/rulesets 94 [200]
GET GET /v4/mappings/rulesets/:mapping_id 94 [200]
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 2,712 [200]
GET GET /v4/oauth2/:domain_id/clients 94 [200]
GET GET /v4/oauth2/:domain_id/clients/:provider_id 95 [200]
GET GET /v4/oauth2/:domain_id/jwks 2,712 [200]
GET GET /v4/scim_realms 61 [200]
GET GET /v4/scim_realms/:domain_id/:provider_id 61 [200]
HEAD HEAD /v3/groups/:id/users/:id 30 [204]
PATCH PATCH /SCIM/v2/:domain_id/Users/:id (sync attribute drift) 1 [429], 14 [200]
PATCH PATCH /v3/credentials/:id 20 [200]
PATCH PATCH /v3/groups/:id 30 [200]
PATCH PATCH /v3/services/:id 28 [200]
POST POST /v3/OS-TRUST/trusts 18 [201]
POST POST /v3/auth/tokens 71 [201]
POST POST /v3/auth/tokens (churn user) 20 [201]
POST POST /v3/auth/tokens (password, invalid) 20 [401]
POST POST /v3/auth/tokens (password, project-scoped) 19 [201]
POST POST /v3/auth/tokens (password, unscoped) 19 [201]
POST POST /v3/auth/tokens (rescope to system) 289 [201]
POST POST /v3/auth/tokens (rescope, invalid project) 165 [401]
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 36 [201]
POST POST /v3/credentials 20 [201]
POST POST /v3/endpoints 26 [201]
POST POST /v3/groups 28 [201]
POST POST /v3/projects (churn) 18 [201]
POST POST /v3/projects (setup) 1 [201]
POST POST /v3/regions 26 [201]
POST POST /v3/services 26 [201]
POST POST /v3/users (churn) 38 [201]
POST POST /v3/users (setup) 1 [201]
POST POST /v4/api-keys/:client_id/revoke (teardown) 1 [200]
POST POST /v4/oauth2/:domain_id/device_authorization 1,149 [200]
POST POST /v4/oauth2/:domain_id/token (client_credentials) 16 [200]
PUT PUT /v3/groups/:id/users/:id 29 [204]
PUT PUT /v3/projects/:id/users/:id/roles/:id (churn) 19 [204]
PUT PUT /v4/scim_realms/:domain_id/:provider_id 61 [200]
Aggregated 185 [401], 15 [429], 71 [404], 656 [201], 437 [204], 10,180 [200]

Transaction Metrics

Transaction # Times Run # Fails Average (ms) Min (ms) Max (ms) RPS Failures/s
ReadHeavy
0.0 1 0 3102 3102 3102 0.03 0.00
0.1 84 0 245 162 404 2.80 0.00
0.2 83 0 236 138 372 2.77 0.00
0.3 83 0 212 138 342 2.77 0.00
TokenLifecycle
1.0 1 0 3273 3273 3273 0.03 0.00
1.1 71 0 822 285 1036 2.37 0.00
ValidateToken
2.0 1 0 2975 2975 2975 0.03 0.00
2.1 218 0 270 146 418 7.27 0.00
UserCRUD
3.0 1 0 2987 2987 2987 0.03 0.00
3.1 1 0 2637 2637 2637 0.03 0.00
3.2 308 0 184 114 329 10.27 0.00
3.3 2 0 158 147 170 0.07 0.00
ProjectCRUD
4.0 1 0 2960 2960 2960 0.03 0.00
4.1 1 0 243 243 243 0.03 0.00
4.2 161 0 183 123 290 5.37 0.00
4.3 1 0 99 99 99 0.03 0.00
RoleCRUD
5.0 0 0 0 0 0 0.00 0.00
5.1 0 0 0 0 0 0.00 0.00
5.2 199 0 151 86 325 6.63 0.00
5.3 1 0 154 154 154 0.03 0.00
RoleList
6.0 0 0 0 0 0 0.00 0.00
6.1 289 0 209 123 335 9.63 0.00
RoleAssignmentList
7.0 0 0 0 0 0 0.00 0.00
7.1 80 0 204 138 300 2.67 0.00
7.2 80 0 183 111 292 2.67 0.00
7.3 80 0 180 109 316 2.67 0.00
7.4 80 0 187 116 313 2.67 0.00
UserRead
8.0 0 0 0 0 0 0.00 0.00
8.1 142 0 239 157 354 4.73 0.00
8.2 142 0 186 106 301 4.73 0.00
ProjectRead
9.0 0 0 0 0 0 0.00 0.00
9.1 143 0 237 151 410 4.77 0.00
9.2 144 0 183 116 276 4.80 0.00
PasswordAuth
10.0 18 0 1737 1358 3305 0.60 0.00
10.1 19 0 1725 1135 2675 0.63 0.00
PasswordAuthNegative
11.0 20 0 1624 1158 2520 0.67 0.00
Rescope
12.0 0 0 0 0 0 0.00 0.00
12.1 289 0 209 120 364 9.63 0.00
RescopeNegative
13.0 0 0 0 0 0 0.00 0.00
13.1 165 0 183 98 326 5.50 0.00
SystemScopeAuth
14.0 36 0 1752 1241 2968 1.20 0.00
GroupChurn
15.0 0 0 0 0 0 0.00 0.00
15.1 30 0 2055 1004 2360 1.00 0.00
CatalogChurn
16.0 0 0 0 0 0 0.00 0.00
16.1 28 0 2240 1005 2548 0.93 0.00
CredentialChurn
17.0 0 0 0 0 0 0.00 0.00
17.1 20 0 3161 1680 4575 0.67 0.00
TrustChurn
18.0 0 0 0 0 0 0.00 0.00
18.1 18 0 3497 3034 4669 0.60 0.00
IdentityChurn
19.0 0 0 0 0 0 0.00 0.00
19.1 20 0 3299 2405 4350 0.67 0.00
ApiKeyCRUD
20.0 0 0 0 0 0 0.00 0.00
20.1 0 0 0 0 0 0.00 0.00
20.2 95 0 158 92 243 3.17 0.00
20.3 96 0 158 91 255 3.20 0.00
20.4 1 0 87 87 87 0.03 0.00
MappingCRUD
21.0 0 0 0 0 0 0.00 0.00
21.1 0 0 0 0 0 0.00 0.00
21.2 94 0 161 99 253 3.13 0.00
21.3 94 0 159 96 320 3.13 0.00
21.4 1 0 93 93 93 0.03 0.00
OAuth2ClientCRUD
22.0 0 0 0 0 0 0.00 0.00
22.1 0 0 0 0 0 0.00 0.00
22.2 95 0 159 103 238 3.17 0.00
22.3 94 0 161 106 293 3.13 0.00
22.4 1 0 87 87 87 0.03 0.00
OAuth2ClientCredentials
23.0 0 0 0 0 0 0.00 0.00
23.1 0 0 0 0 0 0.00 0.00
23.2 16 0 405 264 647 0.53 0.00
23.3 1 0 76 76 76 0.03 0.00
OAuth2DeviceAuthorization
24.0 1149 0 26 7.13 121 38.30 0.00
OAuth2Discovery
25.0 2711 0 11 1.96 81 90.37 0.00
25.1 2712 0 11 1.95 99 90.40 0.00
ScimRealmRead
26.0 0 0 0 0 0 0.00 0.00
26.1 61 0 163 111 245 2.03 0.00
26.2 61 0 151 102 253 2.03 0.00
26.3 61 0 180 111 288 2.03 0.00
ScimSync
27.0 0 0 0 0 0 0.00 0.00
27.1 0 0 0 0 0 0.00 0.00
27.2 0 0 0 0 0 0.00 0.00
27.3 16 0 574 326 1009 0.53 0.00
27.4 15 0 841 458 1011 0.50 0.00
27.5 15 0 616 451 1005 0.50 0.00
27.6 1 0 1003 1003 1003 0.03 0.00
Aggregated 10451 0 131 1.95 4669 348.37 0.00

Scenario Metrics

Scenario # Users # Times Run Average (ms) Min (ms) Max (ms) Scenarios/s Iterations
ReadHeavy 2 83 693 470 892 2.77 41.50
TokenLifecycle 2 69 837 674 1036 2.30 34.50
ValidateToken 2 216 269 146 418 7.20 108.00
UserCRUD 2 306 184 114 329 10.20 153.00
ProjectCRUD 1 160 183 123 290 5.33 160.00
RoleCRUD 1 198 151 86 325 6.60 198.00
RoleList 2 287 209 123 335 9.57 143.50
RoleAssignmentList 2 79 749 522 933 2.63 39.50
UserRead 2 141 426 303 615 4.70 70.50
ProjectRead 2 143 419 296 614 4.77 71.50
PasswordAuth 2 17 3542 2719 5266 0.57 8.50
PasswordAuthNegative 1 19 1625 1158 2520 0.63 19.00
Rescope 2 287 209 120 364 9.57 143.50
RescopeNegative 1 164 183 98 326 5.47 164.00
SystemScopeAuth 2 34 1768 1393 2968 1.13 17.00
GroupChurn 2 28 2122 1862 2360 0.93 14.00
CatalogChurn 2 26 2312 1945 2548 0.87 13.00
CredentialChurn 2 18 3324 2836 4575 0.60 9.00
TrustChurn 2 16 3499 3034 4669 0.53 8.00
IdentityChurn 2 18 3378 2974 4350 0.60 9.00
ApiKeyCRUD 1 95 316 205 478 3.17 95.00
MappingCRUD 1 93 321 223 461 3.10 93.00
OAuth2ClientCRUD 1 94 320 215 463 3.13 94.00
OAuth2ClientCredentials 1 15 1957 1575 2406 0.50 15.00
OAuth2DeviceAuthorization 1 1148 26 7.13 121 38.27 1148.00
OAuth2Discovery 2 2710 22 5.29 108 90.33 1355.00
ScimRealmRead 1 60 492 351 640 2.00 60.00
ScimSync 1 15 2045 1877 2295 0.50 15.00
Aggregated 45 6539 204 5.29 5266 217.97 145.31

Error Metrics

Method Name # Error
POST POST /v3/auth/tokens (password, invalid) 40 401 Unauthorized: POST /v3/auth/tokens (password, invalid)
POST POST /v3/auth/tokens (rescope, invalid project) 330 401 Unauthorized: POST /v3/auth/tokens (rescope, invalid project)
GET GET /v3/auth/tokens (validate revoked) 142 404 Not Found: GET /v3/auth/tokens (validate revoked)
DELETE DELETE /SCIM/v2/:domain_id/Users/:id (sync offboard) 2 429 Too Many Requests: DELETE /SCIM/v2/:domain_id/Users/:id (sync offboard)
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 6 429 Too Many Requests: GET /SCIM/v2/:domain_id/Users (sync reconcile list)
GET GET /SCIM/v2/:domain_id/Users/:id (sync reconcile show) 20 429 Too Many Requests: GET /SCIM/v2/:domain_id/Users/:id (sync reconcile show)
PATCH PATCH /SCIM/v2/:domain_id/Users/:id (sync attribute drift) 2 429 Too Many Requests: PATCH /SCIM/v2/:domain_id/Users/:id (sync attribute drift)

View full report

@gtema
gtema added this pull request to the merge queue Nov 14, 2026
Merged via the queue into main with commit c2cd9a8 Nov 14, 2026
44 of 45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@gtema