diff --git a/.ai/skills/upload-custom-integration/SKILL.md b/.ai/skills/upload-custom-integration/SKILL.md new file mode 100644 index 000000000000..b6b3192fae06 --- /dev/null +++ b/.ai/skills/upload-custom-integration/SKILL.md @@ -0,0 +1,362 @@ +--- +name: upload-custom-integration +description: > + Automates uploading a custom integration to a + Cortex Platform tenant using `demisto-sdk upload-custom-integration`. Use + this skill whenever the user asks to "upload a custom integration", + "duplicate an integration and upload it", "push my integration copy to + Platform", or similar. Handles repo sync, demisto-sdk installation, + credential discovery, the `_copy` naming safety convention, and the upload + itself, pausing for human input only where CLI automation is not possible + (API key generation, ambiguous paths, or bypassing safety checks). +--- + +# Skill: Upload Custom Integration to Cortex Platform + +## 1. Agent Role & Hard Constraints + +You are acting as an automation agent for the Cortex Platform custom +integration upload workflow. Follow these constraints strictly: + +- **STRICT COMMAND CONSTRAINT:** Only ever run `demisto-sdk + upload-custom-integration` to perform the actual upload. **Never** use + plain `demisto-sdk upload` for this workflow — it lacks the `_copy` safety + validation. +- **IN-PLACE MUTATION ONLY:** Modify the target integration YAML directly. + Do not create a duplicate directory unless the user explicitly asks for + that instead. +- **`--force-id` GATE:** Only pass `--force-id` if the human explicitly and + unambiguously asks to bypass the `_copy` check in their prompt (e.g., "use + force-id", "bypass the copy check", "I don't want to rename the ID"). Never + infer this on your own. Always show the CLI's own warning text to the user + first (see Step 5). +- **NEVER silently overwrite `content/.env`.** Only append missing keys or + ask the human to fill them in; never delete or comment-out existing lines. +- **PAUSE FOR HUMAN INPUT** wherever a step genuinely requires a browser/UI + action (generating an API key) or an ambiguous decision (multiple YAML + files found, missing target path). Do not guess. + +--- + +## 2. Execution Workflow + +```mermaid +flowchart TD + A[Start] --> B[Step 1: Sync demisto content repo] + B --> C[Step 2: Set up Python venv & install demisto-sdk] + C --> D[Step 3: Verify or collect Cortex Platform credentials] + D --> E[Step 4: Apply the _copy marker to the target integration YAML] + E --> F[Step 5: Run upload-custom-integration] + F --> G[Step 6: Print verification summary] +``` + +--- + +## Step 1 — Repository Discovery & Synchronization + +Determine whether the current working directory is inside a clone of +`demisto/content`: + +```bash +git remote -v +``` + +**If already inside `demisto/content`:** + +```bash +git checkout master +git pull origin master --rebase +``` + +**If NOT inside `demisto/content`:** ask the user for the path to their +existing local clone first (most users already have one — check common +locations like `../content` relative to `demisto-sdk`, or ask). Only clone a +fresh copy if the user confirms they don't have one yet: + +```bash +ssh -T git@github.com 2>&1 # check SSH availability +# if SSH works: +git clone git@github.com:demisto/content.git +# otherwise: +git clone https://github.com/demisto/content.git +cd content +``` + +--- + +## Step 2 — Python Environment & demisto-sdk Dependency Check + +Assume a brand-new machine with nothing installed yet. `demisto-sdk` must +run in an **activated virtual environment** (or be otherwise available on +`PATH`) — never rely on a system-wide `pip install` without one, since that +can conflict with other Python tooling on the machine. + +### 2.1 Check for an existing virtual environment + +The `content` repo root contains a [`pyproject.toml`](../../../pyproject.toml) +and a [`poetry.toml`](../../../poetry.toml) with `in-project = true`, meaning +Poetry (if used) creates its virtualenv at `content/.venv`. Check what's +already there before creating anything new: + +```bash +# Is Poetry installed? +poetry --version + +# Is there already an in-project venv? +ls -d .venv 2>/dev/null && echo "Found existing .venv" +``` + +### 2.2 Decision logic — which path to take + +**If Poetry is installed (preferred — matches the repo's own tooling):** + +```bash +# Only if python --version reports < 3.10 or > 3.14 (pyproject.toml requires >=3.10,<3.15) +python3 --version + +# Install project dependencies into an in-project .venv (creates it if missing) +poetry install --with dev + +# Activate the venv for the current shell session +source .venv/bin/activate +``` + +**If Poetry is not installed and the user doesn't want to install it,** fall +back to a plain `venv` + `pip`: + +```bash +python3 -m venv .venv +source .venv/bin/activate +pip install --upgrade pip +pip install --upgrade demisto-sdk +``` + +> ⚠️ On Windows (non-WSL) shells, activation is +> `.venv\Scripts\activate` instead of `source .venv/bin/activate`. + +### 2.3 Confirm the venv is actually active before continuing + +Do not just assume activation succeeded — verify it: + +```bash +which python # should point inside content/.venv/bin/python +which demisto-sdk # should also resolve inside the same .venv +``` + +If either command resolves to a path **outside** `.venv`, the environment is +not active — stop and re-run the activation command from 2.2 before +proceeding. Every subsequent step in this skill (Step 5 in particular) must +be run from this same activated shell/session. + +### 2.4 Verify demisto-sdk itself + +```bash +demisto-sdk --version +``` + +**Decision logic:** + +- If `demisto-sdk` is **not installed**, or the reported version is **below + 1.39.10**, upgrade to the latest release from PyPI (inside the active venv): + + ```bash + pip install --upgrade demisto-sdk + ``` + +- After installing/upgrading, **verify the subcommand is actually + registered** (this is the real signal that the feature is available, + more reliable than the version number alone): + + ```bash + demisto-sdk upload-custom-integration --help + ``` + +--- + +## Step 3 — Cortex Platform Credentials (Human-in-the-Loop) + +The `upload-custom-integration` command reads `DEMISTO_BASE_URL`, +`DEMISTO_API_KEY`, and `XSIAM_AUTH_ID` from the environment. In this +workflow, they are expected to live in **`content/.env`**, which +`demisto-sdk` automatically loads on every command invocation. + +### 3.1 Locate and parse `content/.env` + +- Look for `.env` in the root of the `content` repo clone. +- If it does not exist, create an empty one. +- Parse the file, but **only consider active (non-commented) lines** — + ignore any line starting with `#`. + **Never uncomment, delete, or reorder these historical + profiles.** +- From the active (uncommented) lines only, extract current values (if any) + for: + - `DEMISTO_BASE_URL` + - `DEMISTO_API_KEY` + - `XSIAM_AUTH_ID` + +### 3.2 If all three active values are present and non-empty + +Proceed directly to Step 4 — no human interaction needed. + +### 3.3 If any value is missing or empty + +Pause and display this exact guidance to the user (this cannot be automated +because it requires UI interaction in the Cortex Platform console): + +``` +📋 [HUMAN ACTION REQUIRED]: Platform Credentials Configuration + +To allow demisto-sdk to authenticate with your Cortex Platform tenant, +please perform the following steps in your web console: + +1. Obtain API Base URL (DEMISTO_BASE_URL): + - Log in to your Cortex Platform instance. + - Navigate to: Settings → Configurations → API Keys. + - Click "Copy API URL" in the top right corner. + +2. Generate API Key (DEMISTO_API_KEY): + - Click "New Key" (top right corner). + - Set Key Type to "Standard". + - Set Role to "Instance Administrator". + - Click Generate and COPY THE KEY IMMEDIATELY from the modal popup + (it will not be shown again). + +3. Obtain Auth ID (XSIAM_AUTH_ID): + - Close the key popup to view the API Keys table. + - Find the row for your newly created key (check the "Created By" column). + - Copy the numeric value in the "ID" column (e.g., 4). + +4. Paste these three values back to me, or add them directly to + content/.env as a new (uncommented) block: + + DEMISTO_BASE_URL=https://api-your-tenant-url... + DEMISTO_API_KEY=your_copied_api_key_here + XSIAM_AUTH_ID=your_key_id_number_here + DEMISTO_VERIFY_SSL=False +``` + +Wait for the user's response. Once provided, append the new values as a new +block at the bottom of `content/.env` (do not touch existing lines). + +### 3.4 Verify before proceeding + +Re-parse `content/.env` and confirm `DEMISTO_BASE_URL`, `DEMISTO_API_KEY`, +and `XSIAM_AUTH_ID` are all present and non-empty before moving to Step 4. + +--- + +## Step 4 — Apply the `_copy` Marker to the Target Integration YAML + +1. **Resolve the target path.** Ask the user for the integration path if not + already given (e.g., `Packs/PolarSecurity/Integrations/PolarSecurity`). + - `upload-custom-integration` already accepts either a `.yml` file or a + directory (it resolves the single YAML inside automatically). You do + not need to duplicate that resolution logic — but if a directory + contains **more than one** `.yml` file, ask the user which one to + target rather than guessing. + +2. **Read the current values.** Parse the YAML (`ruamel.yaml` or `PyYAML`, + preserving formatting/comments if possible) and read: + - `commonfields.id` (fallback: top-level `id` for unified YAMLs) + - `name` + +3. **Apply the `_copy` suffix in place, only where missing:** + - If `commonfields.id` (or `id`) does not already end with `_copy`, + rename it to `{original_id}_copy`. + - If `name` does not already end with `_copy`, rename it to + `{original_name}_copy`. + - If a field **already** ends with `_copy`, leave it untouched (do not + double-suffix, e.g. never produce `Foo_copy_copy`). + - It is also strongly recommended (per the CLI's own help text) to + likewise suffix the `display` field with `_copy`, to avoid UI confusion + with the original marketplace integration. Apply the same + already-suffixed check before mutating it. + +4. **Write the file back to disk**, mutating the **original file the user + pointed to** — no new directory or file is created. + +#### Example transformation + +```yaml +# BEFORE +commonfields: + id: PolarSecurity +name: Polar Security +display: Polar Security + +# AFTER +commonfields: + id: PolarSecurity_copy +name: Polar Security_copy +display: Polar Security_copy +``` + +--- + +## Step 5 — Upload via `upload-custom-integration` + +Run this from the **same activated virtual environment/shell session** +established in Step 2 (`which demisto-sdk` should still resolve inside +`.venv`). If a new terminal was opened since then, re-activate first +(`source .venv/bin/activate`). + +Run: + +```bash +demisto-sdk upload-custom-integration -i +``` + +The CLI itself will re-validate the `_copy` marker and fail with a clear +error if Step 4 was somehow skipped or incomplete — treat that as a signal +to re-check Step 4, not as a reason to add `--force-id`. + +### Handling `--force-id` (only on explicit human request) + +Only if the user has **explicitly** asked, in their own words, to bypass the +`_copy` requirement: + +1. Show them the CLI's own built-in warning verbatim (do not paraphrase): + > Uploading a custom integration whose ID matches a marketplace + > integration ID will cause subsequent installations of the pack that + > contains that integration to fail with a system error. + > + > ACTION REQUIRED — before continuing, verify ALL of the following: + > 1. Your chosen ID is completely unique and does NOT match the original + > integration ID. + > 2. Your chosen ID does NOT match any other integration ID already + > present in the repository. + > 3. Your chosen ID does NOT match any integration ID published on the + > Marketplace. +2. Get explicit confirmation from the user that they've verified the above. +3. Only then run: + + ```bash + demisto-sdk upload-custom-integration -i --force-id + ``` + +--- + +## Step 6 — Post-Execution Summary Report + +On success, print a structured summary: + +``` +🚀 Custom Integration Upload Summary + +- Target Integration File: +- Custom Integration ID: +- Custom Display Name: +- Target Platform Tenant: +- Executed Command: demisto-sdk upload-custom-integration +- Status: SUCCESS ✅ + +Next Steps: +1. Log in to your Cortex Platform instance. +2. Go to Settings → Configuration → Data Sources & Integrations → Add New. +3. Locate your integration (e.g., ). It will display a + "Custom" badge next to its title. +4. Click Add to configure and use your custom duplicate instance. +``` + +On failure, surface the CLI's error output verbatim and stop — do not retry +with `--force-id` unless the human explicitly asks after seeing the error. diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 7afe27e19d29..86c5907623f0 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -23,8 +23,6 @@ /Packs/QRadar/Integrations/QRadar_v3/* @ilappe /Packs/Slack/Integrations/* @amshamah419 @rshunim /Packs/SplunkPy/Integrations/SplunkPy/* @ilappe -/Packs/MicrosoftExchangeOnPremise/Integrations @amshamah419 -/Packs/MicrosoftExchangeOnline/Integrations @amshamah419 /Packs/ContentManagement/* @DeanArbel @kamalq97 /Packs/TAXIIServer/Integrations/TAXII2Server/* @Ni-Knight /Packs/FeedTAXII/Integrations/FeedTAXII2/* @Ni-Knight @@ -33,8 +31,8 @@ # Important Scripts /Packs/CommonScripts/Scripts/SetGridField/* @melamedbn /Packs/CommonScripts/Scripts/ParseEmailFiles/* @moishce -/Packs/ApiModules/Scripts/MicrosoftApiModule @michal-dagan @inbalapt1 @noydavidi -/Packs/ApiModules/Scripts/MicrosoftGraphMailApiModule @michal-dagan @inbalapt1 @noydavidi +/Packs/ApiModules/Scripts/MicrosoftApiModule @noydavidi +/Packs/ApiModules/Scripts/MicrosoftGraphMailApiModule @noydavidi /Packs/ApiModules/Scripts/EWSApiModule @noydavidi /Packs/ApiModules/Scripts/TAXII2ApiModule/* @Ni-Knight /Packs/CommonScripts/Scripts/StixCreator/* @Ni-Knight @@ -61,9 +59,9 @@ validation_config.toml @yedidyacohenpalo # XDR Related -/Packs/CortexXDR/Integrations/ @jlevypaloalto -/Packs/Core/Integrations/ @jlevypaloalto -/Packs/ApiModules/Scripts/CoreIRApiModule/* @jlevypaloalto +/Packs/CortexXDR/Integrations/ @jlevypaloalto @Shellyber +/Packs/Core/Integrations/ @jlevypaloalto @Shellyber +/Packs/ApiModules/Scripts/CoreIRApiModule/* @jlevypaloalto @Shellyber # Demisto Class Packs/ApiModules/Scripts/DemistoClassApiModule/DemistoClassApiModule.py @dantavori @@ -77,27 +75,6 @@ Packs/ApiModules/Scripts/DemistoClassApiModule/DemistoClassApiModule.py @dantavo /Packs/ApiModules/Scripts/TAXII2ApiModule/* @MosheEichler /Packs/ApiModules/Scripts/NGINXApiModule/* @MosheEichler -# ML scripts -/Packs/ML/ @jlevypaloalto -/Packs/PhishingURL/ @jlevypaloalto -/Packs/Phishing/Scripts/PhishingDedupPreprocessingRule @jlevypaloalto -/Packs/Phishing/Scripts/FindDuplicateEmailIncidents @jlevypaloalto -/Packs/Base/Scripts/DrawRelatedIncidentsCanvas@jlevypaloalto -/Packs/Campaign/Scripts/FindEmailCampaign/ @jlevypaloalto -/Packs/CortexXDR/Scripts/DBotGroupXDRIncidents/ @jlevypaloalto -/Packs/Base/Scripts/FindSimilarIncidentsByText/ @jlevypaloalto -/Packs/Base/Scripts/DBotTrainTextClassifierV2/ @jlevypaloalto -/Packs/Base/Scripts/DBotShowClusteringModelInfo/ @jlevypaloalto -/Packs/Base/Scripts/DBotPredictPhishingWords/ @jlevypaloalto -/Packs/Base/Scripts/DBotFindSimilarIncidents/ @jlevypaloalto -/Packs/Base/Scripts/DBotTrainClustering/ @jlevypaloalto -/Packs/Base/Scripts/DBotBuildPhishingClassifier/ @jlevypaloalto -/Packs/Base/Scripts/DBotPreprocessTextData/ @jlevypaloalto -/Packs/Base/Scripts/DBotFindSimilarIncidentsByIndicators/ @jlevypaloalto -/Packs/Base/Scripts/DBotSuggestClassifierMapping/ @jlevypaloalto -/Packs/Base/Scripts/GetMLModelEvaluation/ @jlevypaloalto -/Packs/Base/Scripts/DBotMLFetchData/ @jlevypaloalto - # Platform Automation /Packs/AzureDevOps/ @omerKarkKatz /Packs/Jira/ @omerKarkKatz @@ -105,12 +82,11 @@ Packs/ApiModules/Scripts/DemistoClassApiModule/DemistoClassApiModule.py @dantavo /Packs/MicrosoftTeams/ @omerKarkKatz /Packs/Slack/ @omerKarkKatz -/Packs/PingIdentity @Shellyber -/Packs/Okta @Shellyber +/Packs/Okta/Integrations/Okta_v2 @Shellyber /Packs/Core @Shellyber /Packs/CortexXDR @Shellyber /Packs/SentinelOne @Shellyber -/Packs/CrowdStrikeFalcon @Shellyber @talzich @JasBeilin +/Packs/CrowdStrikeFalcon @Shellyber @talzich /Packs/CrowdStrikeFalconSandbox @Shellyber /Packs/CrowdStrikeFalconStreamingV2 @Shellyber /Packs/MicrosoftExchangeOnline @Shellyber @noydavidi diff --git a/.github/content_roles.json b/.github/content_roles.json index 73f19db3a454..95157b1832de 100644 --- a/.github/content_roles.json +++ b/.github/content_roles.json @@ -12,7 +12,8 @@ ], "CONTRIBUTION_TL": "kamalq97", "CONTRIBUTION_SECURITY_REVIEWER": ["Benimanela"], - "MAPPING_REVIEWER": ["eepstain"], + "CONTRIBUTION_DOCKERFILES_REVIEWER": "shmuel44", + "MAPPING_REVIEWER": ["OBavly"], "ON_CALL_DEVS": [ "jbabazadeh", "YairGlick", diff --git a/.github/github_workflow_scripts/check_if_connectus_approved_label_exists.py b/.github/github_workflow_scripts/check_if_connectus_approved_label_exists.py new file mode 100644 index 000000000000..f517ac4ffb36 --- /dev/null +++ b/.github/github_workflow_scripts/check_if_connectus_approved_label_exists.py @@ -0,0 +1,320 @@ +""" +Verifies that PRs which add a new XSOAR-supported integration, or modify the +`configuration:` block of an existing one, carry the `connectus-approved` label +before they can be merged. The check: + + 1. Detects integration YML files touched by the PR whose owning pack is + XSOAR-supported (``"support": "xsoar"`` in ``pack_metadata.json``). + 2. Flags the PR when such a file is newly added, or when the `configuration:` + block of an existing one differs between the PR base and head. Renamed and + copied files are compared against their pre-rename path, so a pure move + that leaves `configuration:` untouched does not trigger the gate. + 3. If the PR is flagged, ensures the `connectus-approved` label is set (added + once the Connectus approval flow has signed off on the change). + +Checks that cannot be completed - unreadable pack metadata, a YML that cannot be +fetched at either ref, or a failure to list the PR's files - fail safe: the +change is treated as gated. A transient API error must never silently let an +unapproved configuration change through. + +Exit codes: + 0 - No triggering change OR the `connectus-approved` label is set. + 1 - Triggering change exists but the `connectus-approved` label is missing. +""" + +import argparse +import json +import sys +from pathlib import PurePosixPath +from typing import Any + +import urllib3 +import yaml +from github import Github +from github.PullRequest import PullRequest +from github.Repository import Repository + +from utils import timestamped_print + +urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) +print = timestamped_print # noqa: A001 - intentional, matches existing scripts + +CONNECTUS_APPROVED_LABEL = "connectus-approved" + +PACKS_DIR = "Packs" +INTEGRATIONS_DIR = "Integrations" +PACK_METADATA_FILE = "pack_metadata.json" +XSOAR_SUPPORT = "xsoar" +CONFIGURATION_KEY = "configuration" +YML_SUFFIXES = (".yml", ".yaml") + +ADDED_STATUS = "added" +# GitHub reports a plain content edit as "modified"; "changed" is the rarer +# variant it emits for files whose mode/type changed alongside the content. +MODIFIED_STATUSES = ("modified", "changed") +# Both carry `previous_filename`, pointing at the path the content had at base. +RENAMED_STATUSES = ("renamed", "copied") + + +def arguments_handler() -> argparse.Namespace: + """Validates and parses script arguments. + + Returns: + Namespace: Parsed arguments object. + """ + parser = argparse.ArgumentParser(description=f"Check if {CONNECTUS_APPROVED_LABEL} label exists.") + parser.add_argument("-p", "--pr_number", required=True, help="The PR number to check if the label exists.") + parser.add_argument("-g", "--github_token", required=True, help="The GitHub token to authenticate the GitHub client.") + return parser.parse_args() + + +def is_integration_yml(file_path: str) -> bool: + """ + Return True if `file_path` is an integration's entity YML file, i.e. it matches + ``Packs//Integrations//.yml``. + + Matching on the full path shape (rather than "any YML under Integrations/") + deliberately excludes companion files that live in the same folder but are + not the integration definition - README.md, *_test.py, CHANGELOG.md, and + test-data YAMLs - none of which can contain a `configuration:` block. + + ``PurePosixPath`` is used because filenames come from GitHub's PR API and are + always forward-slash separated, regardless of the runner's OS. + """ + path = PurePosixPath(file_path) + parts = path.parts + + if len(parts) != 5: + return False + if parts[0] != PACKS_DIR or parts[2] != INTEGRATIONS_DIR: + return False + if path.suffix.lower() not in YML_SUFFIXES: + return False + # The YML must be named after its containing integration directory. + return path.stem == parts[3] + + +def get_pack_name(file_path: str) -> str: + """Return the pack name for a ``Packs//...`` path (empty string if not under Packs).""" + parts = PurePosixPath(file_path).parts + if len(parts) < 2 or parts[0] != PACKS_DIR: + return "" + return parts[1] + + +def get_head_repo(pr: PullRequest) -> Repository: + """ + Return the repository that hosts the PR's head commits. + + For a PR opened from a fork this is the fork, not the upstream repo - the + head SHA simply does not exist upstream, so reading head content from + ``pr.base.repo`` would 404 and silently skip the file. ``pr.head.repo`` is + None when the fork has been deleted; fall back to the base repo so we + degrade to the previous behaviour rather than raising. + """ + return pr.head.repo or pr.base.repo + + +def get_file_content_at_ref(repo: Repository, file_path: str, ref: str) -> str | None: + """ + Fetch the raw text content of `file_path` at the given git `ref` from `repo`. + + The caller must pass the repository that actually contains `ref`: the base + repo for `pr.base.sha`, and :func:`get_head_repo` for `pr.head.sha`. + + Returns None when the file does not exist at that ref (e.g. a newly added + file has no base version) or cannot be read. + """ + try: + contents = repo.get_contents(file_path, ref=ref) + except Exception as e: + print(f"Warning: Could not fetch '{file_path}' at ref {ref}: {e}") + return None + + if isinstance(contents, list): # path is a directory - not something we can parse + print(f"Warning: '{file_path}' at ref {ref} is a directory, skipping.") + return None + + try: + return contents.decoded_content.decode("utf-8") + except Exception as e: + print(f"Warning: Could not decode '{file_path}' at ref {ref}: {e}") + return None + + +def parse_yml(content: str, file_path: str) -> dict[str, Any] | None: + """Parse YAML `content` into a dict, returning None if it is malformed or not a mapping.""" + try: + parsed = yaml.safe_load(content) + except yaml.YAMLError as e: + print(f"Warning: Failed to parse YAML file {file_path}: {e}") + return None + return parsed if isinstance(parsed, dict) else None + + +def is_xsoar_supported_pack(pr: PullRequest, pack_name: str, support_cache: dict[str, bool]) -> bool: + """ + Return True if `pack_name`'s metadata declares ``"support": "xsoar"``. + + The metadata is read at the PR head SHA so that a pack newly added (or + re-leveled) by this very PR is evaluated against its proposed state. + Results are cached per pack so a PR touching several integrations in one + pack costs a single API call. + + Fail-safe: when the metadata cannot be fetched or parsed we cannot rule the + pack out, so it is treated as XSOAR-supported. Assuming "not supported" + would let a gated change through on a transient API error, whereas assuming + "supported" merely asks for a label on a pack that may not have needed one. + Do not flip this default. + """ + if pack_name in support_cache: + return support_cache[pack_name] + + supported = True + metadata_path = f"{PACKS_DIR}/{pack_name}/{PACK_METADATA_FILE}" + content = get_file_content_at_ref(get_head_repo(pr), metadata_path, pr.head.sha) + + if content is None: + print(f"Warning: Could not read {metadata_path}; treating pack '{pack_name}' as XSOAR-supported to be safe.") + else: + try: + metadata = json.loads(content) + supported = metadata.get("support") == XSOAR_SUPPORT + except (json.JSONDecodeError, AttributeError) as e: + print(f"Warning: Failed to parse {metadata_path}: {e}; treating pack '{pack_name}' as XSOAR-supported to be safe.") + + support_cache[pack_name] = supported + return supported + + +def get_configuration(parsed_yml: dict[str, Any] | None) -> Any: + """Return the `configuration` block of a parsed integration YML (empty list when absent).""" + if not parsed_yml: + return [] + return parsed_yml.get(CONFIGURATION_KEY, []) + + +def is_configuration_modified(pr: PullRequest, file_path: str, base_path: str | None = None) -> bool: + """ + Return True if the `configuration:` block of `file_path` differs between the + PR base and head. + + Comparing the parsed blocks (rather than the raw diff) means cosmetic YAML + changes elsewhere in the file - a `dockerimage` bump, a reworded + `description` - correctly do NOT trigger the gate, while any real change to + a configuration param does. + + `base_path` is the path the content had at the base ref. It differs from + `file_path` only for renamed/copied files and defaults to `file_path`. + + Fail-safe: if either side cannot be fetched the comparison is undecidable, + so True is returned - an unverifiable change still requires approval. + """ + base_path = base_path or file_path + base_content = get_file_content_at_ref(pr.base.repo, base_path, pr.base.sha) + head_content = get_file_content_at_ref(get_head_repo(pr), file_path, pr.head.sha) + + if base_content is None or head_content is None: + print(f"Warning: Could not compare '{file_path}' between base and head; requiring approval to be safe.") + return True + + base_configuration = get_configuration(parse_yml(base_content, file_path)) + head_configuration = get_configuration(parse_yml(head_content, file_path)) + + return base_configuration != head_configuration + + +def check_pr_contains_connectus_changes(pr: PullRequest) -> list[str]: + """ + Return a list of human-readable reasons why this PR requires Connectus approval + (empty when it does not). + + A reason is recorded when an integration YML in an XSOAR-supported pack is + either newly added, or has had its `configuration:` block modified. A rename + is resolved against the file's pre-rename path, so moving an integration + without touching `configuration:` is not a reason on its own. + """ + reasons: list[str] = [] + support_cache: dict[str, bool] = {} + + try: + files = list(pr.get_files()) + except Exception as e: + # Fail safe: if we cannot inspect the PR we require the label rather than + # silently letting a potentially-gated change through. + print(f"Error listing PR files: {e}. Requiring the label to be safe.") + return [f"- Could not inspect the PR's files ({e}); requiring '{CONNECTUS_APPROVED_LABEL}' to be safe."] + + for file in files: + if not is_integration_yml(file.filename): + continue + + pack_name = get_pack_name(file.filename) + if not is_xsoar_supported_pack(pr, pack_name, support_cache): + print(f"Skipping '{file.filename}': pack '{pack_name}' is not {XSOAR_SUPPORT}-supported.") + continue + + try: + if file.status == ADDED_STATUS: + reasons.append(f"- New XSOAR-supported integration added: {file.filename}") + elif file.status in RENAMED_STATUSES: + previous_filename = getattr(file, "previous_filename", None) + if not previous_filename or not is_integration_yml(previous_filename): + # Renamed from something that was not an integration definition + # (or GitHub did not tell us where from): the integration YML + # appears at this path for the first time, so treat it as new. + reasons.append(f"- New XSOAR-supported integration added: {file.filename}") + elif is_configuration_modified(pr, file.filename, base_path=previous_filename): + reasons.append( + "- Modified 'configuration' of XSOAR-supported integration: " + f"{file.filename} (renamed from {previous_filename})" + ) + elif file.status in MODIFIED_STATUSES and is_configuration_modified(pr, file.filename): + reasons.append(f"- Modified 'configuration' of XSOAR-supported integration: {file.filename}") + except Exception as e: + print(f"Warning: Error processing {file.filename}: {e}") + continue + + return reasons + + +def main() -> None: + """ + Checks that the "connectus-approved" label exists on a PR when required. If the + label exists the workflow passes; if it is missing (and required) the workflow fails. + """ + options = arguments_handler() + pr_number = int(options.pr_number) + + github_client: Github = Github(options.github_token, verify=False) + content_repo: Repository = github_client.get_repo("demisto/content") + pr: PullRequest = content_repo.get_pull(pr_number) + + pr_label_names = {label.name for label in pr.labels} + + print(f"Checking if {CONNECTUS_APPROVED_LABEL} label exist in PR {pr_number}") + if CONNECTUS_APPROVED_LABEL in pr_label_names: + print(f"SUCCESS: PR #{pr_number} has the required label: {CONNECTUS_APPROVED_LABEL}") + sys.exit(0) + + reasons = check_pr_contains_connectus_changes(pr) + if not reasons: + print( + "INFO: PR does not add a new XSOAR-supported integration or modify integration configuration params.\n" + f" The '{CONNECTUS_APPROVED_LABEL}' label is not required for this PR." + ) + sys.exit(0) + + reasons_text = "\n".join(reasons) + print( + f"ERROR: Required label '{CONNECTUS_APPROVED_LABEL}' is missing from PR #{pr_number}.\n" + " The following changes require Connectus approval:\n" + f"{reasons_text}\n" + " Please ask the Connectus team to review the changes and add the " + f"'{CONNECTUS_APPROVED_LABEL}' label once approved." + ) + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/.github/github_workflow_scripts/create_internal_pr.py b/.github/github_workflow_scripts/create_internal_pr.py index 05d99721395c..48e18fa486c9 100755 --- a/.github/github_workflow_scripts/create_internal_pr.py +++ b/.github/github_workflow_scripts/create_internal_pr.py @@ -237,7 +237,7 @@ def remove_branch_protection(repo, branch_name, t): def prepare_body(pr: PullRequest): - body = f"## Original External PR\r\n[external pull request]({pr.html_url})\r\n\r\n" + body = f"## Related PRs\r\n**Original External PR:** {pr.html_url}\r\n\r\n" if "## Contributor" not in pr.body: merged_pr_author = pr.user.login body += f"## Contributor\r\n@{merged_pr_author}\r\n\r\n" @@ -376,6 +376,26 @@ def main(): except Exception as e: print(f"{t.red}Mapping PR failed: {e}{t.normal}") + # Cross-reference main and mapping PRs when both were created from the same external PR + if len(created) == 2: + main_pr, mapping_pr = created[0], created[1] + try: + main_pr.edit( + body=main_pr.body.replace( + f"**Original External PR:** {merged_pr.html_url}", + f"**Original External PR:** {merged_pr.html_url}\r\n**Mapping Internal PR:** {mapping_pr.html_url}", + ) + ) + mapping_pr.edit( + body=mapping_pr.body.replace( + f"**Original External PR:** {merged_pr.html_url}", + f"**Original External PR:** {merged_pr.html_url}\r\n**Main Internal PR:** {main_pr.html_url}", + ) + ) + print(f"{t.cyan}Cross-referenced main PR #{main_pr.number} and mapping PR #{mapping_pr.number}{t.normal}") + except Exception as e: + print(f"{t.red}Failed to cross-reference PRs: {e}{t.normal}") + for pr in created: remove_branch_protection(content_repo, pr.head.ref, t) diff --git a/.github/github_workflow_scripts/github_workflow_scripts_tests/check_if_connectus_approved_label_test.py b/.github/github_workflow_scripts/github_workflow_scripts_tests/check_if_connectus_approved_label_test.py new file mode 100644 index 000000000000..8e5d074a9de1 --- /dev/null +++ b/.github/github_workflow_scripts/github_workflow_scripts_tests/check_if_connectus_approved_label_test.py @@ -0,0 +1,724 @@ +""" +Unit tests for ``github_workflow_scripts/check_if_connectus_approved_label_exists.py``. + +The script talks to the GitHub API via the ``PyGithub`` library; for unit +testing we never hit the network - we hand-build lightweight mock objects that +expose only the attributes the script actually reads. +""" + +import json + +import pytest + +MODULE = "github_workflow_scripts.check_if_connectus_approved_label_exists" + +XSOAR_METADATA = json.dumps({"name": "MyPack", "support": "xsoar"}) +PARTNER_METADATA = json.dumps({"name": "MyPack", "support": "partner"}) + +INTEGRATION_YML = "Packs/MyPack/Integrations/MyIntegration/MyIntegration.yml" +OLD_INTEGRATION_YML = "Packs/MyPack/Integrations/OldIntegration/OldIntegration.yml" +PACK_METADATA = "Packs/MyPack/pack_metadata.json" + + +def integration_yml(configuration: str, dockerimage: str = "demisto/python3:3.10.1") -> str: + """Build a minimal integration YML with the supplied ``configuration`` block.""" + return ( + "commonfields:\n" + " id: MyIntegration\n" + "name: MyIntegration\n" + f"configuration:\n{configuration}" + "script:\n" + " type: python\n" + f" dockerimage: {dockerimage}\n" + ) + + +ONE_PARAM = "- display: Server URL\n name: url\n type: 0\n required: true\n" +TWO_PARAMS = ONE_PARAM + "- display: API Key\n name: apikey\n type: 4\n required: true\n" + + +# --------------------------------------------------------------------------- +# Tiny stand-in objects (no PyGithub dependency at test-collection time). +# --------------------------------------------------------------------------- + + +class _MockFile: + def __init__(self, filename: str, status: str = "modified", previous_filename: str | None = None): + self.filename = filename + self.status = status + # PyGithub only exposes `previous_filename` on renamed/copied entries, so + # the attribute is left off entirely when no rename source is supplied. + if previous_filename is not None: + self.previous_filename = previous_filename + + +class _MockLabel: + def __init__(self, name: str): + self.name = name + + +class _MockContentFile: + def __init__(self, content: str): + self.decoded_content = content.encode("utf-8") + + +class _MockRepo: + """Serves file contents keyed by ``(path, ref)``; raises when the pair is unknown.""" + + def __init__(self, contents: dict[tuple[str, str], str] | None = None): + self._contents = contents or {} + + def get_contents(self, path: str, ref: str): + try: + return _MockContentFile(self._contents[(path, ref)]) + except KeyError: + raise Exception(f"404: {path}@{ref} not found") + + +class _MockRef: + def __init__(self, sha: str, repo: _MockRepo | None = None): + self.sha = sha + self.repo = repo + + +class _MockPullRequest: + """Minimal stand-in matching the surface area used by the script under test.""" + + BASE_SHA = "base-sha" + HEAD_SHA = "head-sha" + + def __init__( + self, + number: int = 42, + files=None, + labels=None, + contents=None, + files_raise: bool = False, + head_contents=None, + head_repo_deleted: bool = False, + ): + """ + `contents` populates the base repo. `head_contents`, when given, puts the + head content in a *separate* repo reachable only via ``pr.head.repo``, + which is how GitHub models a PR opened from a fork. When it is omitted + both refs are served by the base repo (same-repo PR). + """ + self.number = number + self._files = files or [] + self._files_raise = files_raise + self.labels = labels or [] + self.base = _MockRef(self.BASE_SHA, _MockRepo(contents)) + if head_repo_deleted: + # GitHub reports head.repo as None once the source fork is deleted. + self.head = _MockRef(self.HEAD_SHA, None) + else: + head_repo = _MockRepo(head_contents) if head_contents is not None else self.base.repo + self.head = _MockRef(self.HEAD_SHA, head_repo) + + def get_files(self): + if self._files_raise: + raise Exception("GitHub API is down") + return iter(self._files) + + +def _contents(base_yml: str | None = None, head_yml: str | None = None, metadata: str = XSOAR_METADATA) -> dict: + """Build the ``(path, ref) -> content`` map for a single-integration PR.""" + contents: dict[tuple[str, str], str] = {(PACK_METADATA, _MockPullRequest.HEAD_SHA): metadata} + if base_yml is not None: + contents[(INTEGRATION_YML, _MockPullRequest.BASE_SHA)] = base_yml + if head_yml is not None: + contents[(INTEGRATION_YML, _MockPullRequest.HEAD_SHA)] = head_yml + return contents + + +def _run_main(mocker, pr: _MockPullRequest) -> int: + """Run the script's ``main()`` against ``pr`` and return its exit code.""" + from github_workflow_scripts import check_if_connectus_approved_label_exists as script + + mock_repo = mocker.MagicMock() + mock_repo.get_pull.return_value = pr + mocker.patch.object(script, "Github", return_value=mocker.MagicMock(get_repo=lambda _: mock_repo)) + mocker.patch.object( + script, + "arguments_handler", + return_value=mocker.MagicMock(pr_number=str(pr.number), github_token="token"), + ) + + with pytest.raises(SystemExit) as exit_info: + script.main() + return exit_info.value.code + + +# --------------------------------------------------------------------------- +# is_integration_yml +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "file_path, expected", + [ + # The integration entity YML - the only shape we gate on. + (INTEGRATION_YML, True), + ("Packs/MyPack/Integrations/MyIntegration/MyIntegration.yaml", True), + # Companion files in the same folder are not the integration definition. + ("Packs/MyPack/Integrations/MyIntegration/MyIntegration.py", False), + ("Packs/MyPack/Integrations/MyIntegration/README.md", False), + ("Packs/MyPack/Integrations/MyIntegration/MyIntegration_test.py", False), + # A YML in the integration folder not named after it (e.g. test data). + ("Packs/MyPack/Integrations/MyIntegration/test_data/response.yml", False), + ("Packs/MyPack/Integrations/MyIntegration/command_examples.yml", False), + # Other content entities must not trigger the gate. + ("Packs/MyPack/Scripts/MyScript/MyScript.yml", False), + ("Packs/MyPack/Playbooks/MyPlaybook.yml", False), + ("Packs/MyPack/ModelingRules/MyRule/MyRule.yml", False), + ("Packs/MyPack/pack_metadata.json", False), + # Paths outside Packs/. + (".github/workflows/connectus-approved.yml", False), + ("Tests/conf.json", False), + ], +) +def test_is_integration_yml(file_path, expected): + """ + Given: + - A file path from a PR's changed-files list. + + When: + - Running is_integration_yml. + + Then: + - Only ``Packs//Integrations//.yml`` is recognised as + an integration definition. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import is_integration_yml + + assert is_integration_yml(file_path) is expected + + +@pytest.mark.parametrize( + "file_path, expected", + [ + (INTEGRATION_YML, "MyPack"), + ("Packs/OtherPack/pack_metadata.json", "OtherPack"), + ("Tests/conf.json", ""), + ("README.md", ""), + ], +) +def test_get_pack_name(file_path, expected): + """ + Given: + - A file path. + + When: + - Running get_pack_name. + + Then: + - The owning pack name is returned, or an empty string outside Packs/. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import get_pack_name + + assert get_pack_name(file_path) == expected + + +# --------------------------------------------------------------------------- +# The four mandated end-to-end scenarios +# --------------------------------------------------------------------------- + + +def test_new_xsoar_integration_without_label_fails(mocker, capsys): + """ + Given: + - A PR that adds a new integration YML in an XSOAR-supported pack. + - The 'connectus-approved' label is missing. + + When: + - Running the script's main function. + + Then: + - The check fails (exit code 1) and the log names the offending file. + """ + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="added")], + contents=_contents(head_yml=integration_yml(ONE_PARAM)), + ) + + assert _run_main(mocker, pr) == 1 + + output = capsys.readouterr().out + assert "ERROR" in output + assert "connectus-approved" in output + assert "New XSOAR-supported integration added" in output + assert INTEGRATION_YML in output + + +def test_modified_configuration_without_label_fails(mocker, capsys): + """ + Given: + - A PR that modifies the 'configuration' block of an existing integration + in an XSOAR-supported pack. + - The 'connectus-approved' label is missing. + + When: + - Running the script's main function. + + Then: + - The check fails (exit code 1) and the log names the offending file. + """ + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="modified")], + contents=_contents( + base_yml=integration_yml(ONE_PARAM), + head_yml=integration_yml(TWO_PARAMS), + ), + ) + + assert _run_main(mocker, pr) == 1 + + output = capsys.readouterr().out + assert "ERROR" in output + assert "connectus-approved" in output + assert "Modified 'configuration' of XSOAR-supported integration" in output + assert INTEGRATION_YML in output + + +def test_label_present_passes_despite_changes(mocker, capsys): + """ + Given: + - A PR that both adds a new integration and changes configuration params. + - The 'connectus-approved' label IS present. + + When: + - Running the script's main function. + + Then: + - The check passes (exit code 0) regardless of the detected changes. + """ + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="added")], + labels=[_MockLabel("connectus-approved")], + contents=_contents(head_yml=integration_yml(TWO_PARAMS)), + ) + + assert _run_main(mocker, pr) == 0 + + output = capsys.readouterr().out + assert "SUCCESS" in output + assert "ERROR" not in output + + +def test_no_triggering_changes_passes(mocker, capsys): + """ + Given: + - A PR that touches no integration YMLs (only a script and release notes). + - The 'connectus-approved' label is missing. + + When: + - Running the script's main function. + + Then: + - The check passes (exit code 0) because the label is not required. + """ + pr = _MockPullRequest( + files=[ + _MockFile("Packs/MyPack/Scripts/MyScript/MyScript.py", status="modified"), + _MockFile("Packs/MyPack/ReleaseNotes/1_0_1.md", status="added"), + ], + contents=_contents(), + ) + + assert _run_main(mocker, pr) == 0 + + output = capsys.readouterr().out + assert "INFO" in output + assert "not required" in output + assert "ERROR" not in output + + +# --------------------------------------------------------------------------- +# check_pr_contains_connectus_changes - supporting cases +# --------------------------------------------------------------------------- + + +def test_non_xsoar_pack_is_ignored(): + """ + Given: + - A PR adding a new integration to a partner-supported pack. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - No reasons are returned: the gate only covers XSOAR-supported packs. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="added")], + contents=_contents(head_yml=integration_yml(ONE_PARAM), metadata=PARTNER_METADATA), + ) + + assert check_pr_contains_connectus_changes(pr) == [] + + +def test_unchanged_configuration_is_ignored(): + """ + Given: + - A PR modifying an integration YML where only the dockerimage changed + and the 'configuration' block is identical. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - No reasons are returned: non-configuration edits don't require approval. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="modified")], + contents=_contents( + base_yml=integration_yml(ONE_PARAM, dockerimage="demisto/python3:3.10.1"), + head_yml=integration_yml(ONE_PARAM, dockerimage="demisto/python3:3.10.2"), + ), + ) + + assert check_pr_contains_connectus_changes(pr) == [] + + +def test_undecidable_diff_requires_the_label(): + """ + Given: + - A PR whose modified integration YML cannot be fetched at the base ref, + leaving the configuration comparison undecidable. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - The change is flagged: an unverifiable configuration edit must not slip + through unapproved just because the API call failed. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="modified")], + contents=_contents(head_yml=integration_yml(TWO_PARAMS)), # no base entry + ) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "Modified 'configuration'" in reasons[0] + + +def test_unparsable_yml_is_handled_gracefully(): + """ + Given: + - A PR whose modified integration YML is malformed at the head ref. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - Parsing fails softly; the malformed head yields an empty configuration, + which differs from the base, so the change is still flagged for review. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="modified")], + contents=_contents( + base_yml=integration_yml(ONE_PARAM), + head_yml="configuration: [unclosed", + ), + ) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "Modified 'configuration'" in reasons[0] + + +def test_unreadable_pack_metadata_is_treated_as_xsoar(): + """ + Given: + - A PR adding an integration whose pack_metadata.json cannot be read. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - The pack is assumed XSOAR-supported and the addition is flagged: we + cannot rule the pack out, so we fail safe rather than fail open. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="added")], + contents={}, # no metadata available at any ref + ) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "New XSOAR-supported integration added" in reasons[0] + + +def test_malformed_pack_metadata_is_treated_as_xsoar(): + """ + Given: + - A PR adding an integration whose pack_metadata.json is not valid JSON. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - The pack is assumed XSOAR-supported and the addition is flagged. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="added")], + contents=_contents(head_yml=integration_yml(ONE_PARAM), metadata='{"support": "xsoar"'), + ) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "New XSOAR-supported integration added" in reasons[0] + + +# --------------------------------------------------------------------------- +# Renamed / copied files +# --------------------------------------------------------------------------- + + +def _rename_contents(base_yml: str, head_yml: str, base_path: str = OLD_INTEGRATION_YML) -> dict: + """Build the content map for a rename: base content lives at the pre-rename path.""" + return { + (PACK_METADATA, _MockPullRequest.HEAD_SHA): XSOAR_METADATA, + (base_path, _MockPullRequest.BASE_SHA): base_yml, + (INTEGRATION_YML, _MockPullRequest.HEAD_SHA): head_yml, + } + + +def test_pure_rename_is_not_flagged(): + """ + Given: + - A PR that renames an integration YML without touching its + 'configuration' block. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - No reason is recorded: a move is not a configuration change. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="renamed", previous_filename=OLD_INTEGRATION_YML)], + contents=_rename_contents(integration_yml(ONE_PARAM), integration_yml(ONE_PARAM)), + ) + + assert check_pr_contains_connectus_changes(pr) == [] + + +def test_rename_with_configuration_change_is_flagged(): + """ + Given: + - A PR that renames an integration YML and also adds a configuration param. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - The change is flagged and the reason names both the new and old paths. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="renamed", previous_filename=OLD_INTEGRATION_YML)], + contents=_rename_contents(integration_yml(ONE_PARAM), integration_yml(TWO_PARAMS)), + ) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "Modified 'configuration'" in reasons[0] + assert INTEGRATION_YML in reasons[0] + assert OLD_INTEGRATION_YML in reasons[0] + + +def test_rename_from_non_integration_path_is_flagged_as_new(): + """ + Given: + - A PR that renames a non-integration file into the integration YML path, + so the integration definition appears there for the first time. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - It is flagged as a new integration rather than compared as a rename. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[ + _MockFile( + INTEGRATION_YML, + status="renamed", + previous_filename="Packs/MyPack/Integrations/MyIntegration/draft.yml", + ) + ], + contents=_contents(head_yml=integration_yml(ONE_PARAM)), + ) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "New XSOAR-supported integration added" in reasons[0] + + +def test_rename_without_previous_filename_is_flagged_as_new(): + """ + Given: + - A renamed integration YML for which GitHub reported no previous_filename. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - It is flagged as new rather than raising an AttributeError. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="renamed")], # no previous_filename attribute + contents=_contents(head_yml=integration_yml(ONE_PARAM)), + ) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "New XSOAR-supported integration added" in reasons[0] + + +def test_changed_status_is_compared_like_modified(): + """ + Given: + - A PR whose integration YML carries GitHub's rarer 'changed' status and + whose configuration block was edited. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - It is compared exactly like a 'modified' file and flagged. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="changed")], + contents=_contents( + base_yml=integration_yml(ONE_PARAM), + head_yml=integration_yml(TWO_PARAMS), + ), + ) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "Modified 'configuration'" in reasons[0] + + +def test_pack_support_lookup_is_cached(): + """ + Given: + - A PR adding two integrations that live in the same pack. + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - Both are flagged, and the pack metadata is fetched only once. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + second_yml = "Packs/MyPack/Integrations/OtherIntegration/OtherIntegration.yml" + pr = _MockPullRequest( + files=[_MockFile(INTEGRATION_YML, status="added"), _MockFile(second_yml, status="added")], + contents=_contents(head_yml=integration_yml(ONE_PARAM)), + ) + + metadata_fetches = [] + original_get_contents = pr.base.repo.get_contents + + def counting_get_contents(path, ref): + if path == PACK_METADATA: + metadata_fetches.append(ref) + return original_get_contents(path, ref) + + pr.base.repo.get_contents = counting_get_contents + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 2 + assert len(metadata_fetches) == 1 + + +def test_api_failure_requires_the_label(): + """ + Given: + - Listing the PR's files raises (e.g. a GitHub API outage). + + When: + - Running check_pr_contains_connectus_changes. + + Then: + - A reason is returned so the gate fails safe rather than letting a + potentially-gated change through unreviewed. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import check_pr_contains_connectus_changes + + pr = _MockPullRequest(files_raise=True) + + reasons = check_pr_contains_connectus_changes(pr) + assert len(reasons) == 1 + assert "Could not inspect" in reasons[0] + + +# --------------------------------------------------------------------------- +# Small helpers +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "parsed_yml, expected", + [ + ({"configuration": [{"name": "url"}]}, [{"name": "url"}]), + ({"name": "NoConfig"}, []), + (None, []), + ], +) +def test_get_configuration(parsed_yml, expected): + """ + Given: + - A parsed integration YML (or None when parsing failed). + + When: + - Running get_configuration. + + Then: + - The configuration block is returned, defaulting to an empty list. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import get_configuration + + assert get_configuration(parsed_yml) == expected + + +def test_label_name_is_stable(): + """ + Given: + - The workflow and the branch-protection rule both reference the label by name. + + When: + - Reading CONNECTUS_APPROVED_LABEL. + + Then: + - It is exactly 'connectus-approved'; any rename is a breaking change. + """ + from github_workflow_scripts.check_if_connectus_approved_label_exists import CONNECTUS_APPROVED_LABEL + + assert CONNECTUS_APPROVED_LABEL == "connectus-approved" diff --git a/.github/github_workflow_scripts/github_workflow_scripts_tests/create_internal_pr_test.py b/.github/github_workflow_scripts/github_workflow_scripts_tests/create_internal_pr_test.py new file mode 100644 index 000000000000..f3ba7e13d4d3 --- /dev/null +++ b/.github/github_workflow_scripts/github_workflow_scripts_tests/create_internal_pr_test.py @@ -0,0 +1,248 @@ +from unittest.mock import MagicMock, patch + +from github_workflow_scripts.create_internal_pr import ( + main, + prepare_body, + replace_fixes_with_relates_in_pr_body, +) + + +def _make_mock_pr(html_url: str, number: int, body: str = "", login: str = "contributor1"): + """Create a minimal mock PullRequest.""" + pr = MagicMock() + pr.html_url = html_url + pr.number = number + pr.body = body + pr.user.login = login + return pr + + +class TestCrossReferencePRs: + """Tests for the cross-reference logic that links main and mapping PRs to each other.""" + + EXTERNAL_URL = "https://github.com/demisto/content/pull/1234" + MAIN_URL = "https://github.com/demisto/content/pull/1235" + MAPPING_URL = "https://github.com/demisto/content/pull/1236" + + def test_cross_reference_main_pr_body_contains_mapping_link(self): + """ + Given: + - A main PR body generated from an external PR. + When: + - The cross-reference replace is applied to add the mapping PR link. + Then: + - The main PR body contains the mapping PR URL right after the external PR link. + """ + ext_pr = _make_mock_pr(self.EXTERNAL_URL, 1234, body="description\n\nrelates: #999") + body = prepare_body(ext_pr) + + updated = body.replace( + f"**Original External PR:** {self.EXTERNAL_URL}", + f"**Original External PR:** {self.EXTERNAL_URL}\r\n**Mapping Internal PR:** {self.MAPPING_URL}", + ) + + assert f"**Mapping Internal PR:** {self.MAPPING_URL}" in updated + + def test_cross_reference_mapping_pr_body_contains_main_link(self): + """ + Given: + - A mapping PR body generated from an external PR. + When: + - The cross-reference replace is applied to add the main PR link. + Then: + - The mapping PR body contains the main PR URL right after the external PR link. + """ + ext_pr = _make_mock_pr(self.EXTERNAL_URL, 1234, body="description\n\nrelates: #999") + body = replace_fixes_with_relates_in_pr_body(prepare_body(ext_pr)) + + updated = body.replace( + f"**Original External PR:** {self.EXTERNAL_URL}", + f"**Original External PR:** {self.EXTERNAL_URL}\r\n**Main Internal PR:** {self.MAIN_URL}", + ) + + assert f"**Main Internal PR:** {self.MAIN_URL}" in updated + + def test_cross_reference_links_are_grouped_at_top(self): + """ + Given: + - Both a main and mapping PR body after cross-referencing. + When: + - Checking the position of the Related PRs section. + Then: + - The Related PRs section (with all links) appears before the Contributor section. + """ + ext_pr = _make_mock_pr(self.EXTERNAL_URL, 1234, body="description\n\nrelates: #999") + body = prepare_body(ext_pr) + + updated = body.replace( + f"**Original External PR:** {self.EXTERNAL_URL}", + f"**Original External PR:** {self.EXTERNAL_URL}\r\n**Mapping Internal PR:** {self.MAPPING_URL}", + ) + + assert updated.index("## Related PRs") < updated.index("## Contributor") + assert updated.index("**Mapping Internal PR:**") < updated.index("## Contributor") + + def test_cross_reference_no_duplicate_external_link(self): + """ + Given: + - A PR body after cross-referencing. + When: + - Checking for the external PR marker. + Then: + - The Original External PR line appears exactly once. + """ + ext_pr = _make_mock_pr(self.EXTERNAL_URL, 1234, body="description\n\nrelates: #999") + body = prepare_body(ext_pr) + + updated = body.replace( + f"**Original External PR:** {self.EXTERNAL_URL}", + f"**Original External PR:** {self.EXTERNAL_URL}\r\n**Mapping Internal PR:** {self.MAPPING_URL}", + ) + + assert updated.count("**Original External PR:**") == 1 + + @patch("github_workflow_scripts.create_internal_pr.get_env_var") + @patch("github_workflow_scripts.create_internal_pr.Github") + @patch("github_workflow_scripts.create_internal_pr.separate_pr_files") + @patch("github_workflow_scripts.create_internal_pr.split_branch_with_git") + @patch("github_workflow_scripts.create_internal_pr.create_pr") + @patch("github_workflow_scripts.create_internal_pr.remove_branch_protection") + @patch("github_workflow_scripts.create_internal_pr.get_content_roles", return_value=None) + @patch("github_workflow_scripts.create_internal_pr.is_organization_member", return_value=False) + def test_main_calls_edit_on_both_prs_when_both_created( + self, + mock_is_org_member, + mock_get_content_roles, + mock_remove_protection, + mock_create_pr, + mock_split_branch, + mock_separate, + mock_github_cls, + mock_get_env_var, + ): + """ + Given: + - An external PR that contains both XSOAR and XSIAM files. + When: + - main() creates both a main PR and a mapping PR. + Then: + - edit() is called on both PRs to cross-reference each other. + - The main PR body contains the mapping PR URL. + - The mapping PR body contains the main PR URL. + """ + # -- setup env vars -- + import json + + payload = {"pull_request": {"number": 1234}} + + def env_side_effect(name): + if name == "EVENT_PAYLOAD": + return json.dumps(payload) + return "fake-token" + + mock_get_env_var.side_effect = env_side_effect + + # -- setup GitHub mocks -- + ext_pr = _make_mock_pr(self.EXTERNAL_URL, 1234, body="description\n\nrelates: #999") + ext_pr.base.ref = "contrib-branch" + ext_pr.labels = [] + ext_pr.assignees = [] + ext_pr.merged_by = None + ext_pr.get_review_requests.return_value = ([], []) + ext_pr.get_files.return_value = [] + + mock_repo = MagicMock() + mock_repo.get_pull.return_value = ext_pr + mock_github_cls.return_value.get_repo.return_value = mock_repo + + # -- both file types present -- + xsoar_file = MagicMock() + xsiam_file = MagicMock() + mock_separate.return_value = ([xsoar_file], [xsiam_file]) + mock_split_branch.return_value = ("contrib-branch-main", "contrib-branch-mapping") + + # -- mock created PRs -- + main_pr = _make_mock_pr(self.MAIN_URL, 1235) + main_pr.body = prepare_body(ext_pr) + main_pr.head.ref = "contrib-branch-main" + + mapping_pr = _make_mock_pr(self.MAPPING_URL, 1236) + mapping_pr.body = replace_fixes_with_relates_in_pr_body(prepare_body(ext_pr)) + mapping_pr.head.ref = "contrib-branch-mapping" + + mock_create_pr.side_effect = [main_pr, mapping_pr] + + # -- run -- + main() + + # -- assert edit was called on both PRs -- + assert main_pr.edit.call_count == 1 + assert mapping_pr.edit.call_count == 1 + + main_updated_body = main_pr.edit.call_args.kwargs.get("body", "") + mapping_updated_body = mapping_pr.edit.call_args.kwargs.get("body", "") + + assert f"**Mapping Internal PR:** {self.MAPPING_URL}" in main_updated_body + assert f"**Main Internal PR:** {self.MAIN_URL}" in mapping_updated_body + + @patch("github_workflow_scripts.create_internal_pr.get_env_var") + @patch("github_workflow_scripts.create_internal_pr.Github") + @patch("github_workflow_scripts.create_internal_pr.separate_pr_files") + @patch("github_workflow_scripts.create_internal_pr.create_pr") + @patch("github_workflow_scripts.create_internal_pr.remove_branch_protection") + @patch("github_workflow_scripts.create_internal_pr.get_content_roles", return_value=None) + @patch("github_workflow_scripts.create_internal_pr.is_organization_member", return_value=False) + def test_no_cross_reference_when_only_one_pr_created( + self, + mock_is_org_member, + mock_get_content_roles, + mock_remove_protection, + mock_create_pr, + mock_separate, + mock_github_cls, + mock_get_env_var, + ): + """ + Given: + - An external PR that contains only XSOAR files (no XSIAM). + When: + - main() creates only a main PR. + Then: + - edit() is never called (no cross-reference needed). + """ + import json + + payload = {"pull_request": {"number": 1234}} + + def env_side_effect(name): + if name == "EVENT_PAYLOAD": + return json.dumps(payload) + return "fake-token" + + mock_get_env_var.side_effect = env_side_effect + + ext_pr = _make_mock_pr(self.EXTERNAL_URL, 1234, body="description\n\nrelates: #999") + ext_pr.base.ref = "contrib-branch" + ext_pr.labels = [] + ext_pr.assignees = [] + ext_pr.merged_by = None + ext_pr.get_review_requests.return_value = ([], []) + ext_pr.get_files.return_value = [] + + mock_repo = MagicMock() + mock_repo.get_pull.return_value = ext_pr + mock_github_cls.return_value.get_repo.return_value = mock_repo + + # Only XSOAR files + xsoar_file = MagicMock() + mock_separate.return_value = ([xsoar_file], []) + + main_pr = _make_mock_pr(self.MAIN_URL, 1235) + main_pr.body = prepare_body(ext_pr) + main_pr.head.ref = "contrib-branch" + + mock_create_pr.return_value = main_pr + + main() + + main_pr.edit.assert_not_called() diff --git a/.github/workflows/connectus-approved.yml b/.github/workflows/connectus-approved.yml new file mode 100644 index 000000000000..af77a5ac5cad --- /dev/null +++ b/.github/workflows/connectus-approved.yml @@ -0,0 +1,37 @@ +name: connectus-approved label verification +on: + pull_request: + types: [opened, synchronize, labeled, unlabeled] + + +permissions: + contents: read + +jobs: + check_connectus_approved_label_job: + runs-on: ubuntu-latest + if: github.repository == 'demisto/content' + steps: + - name: Checkout + uses: actions/checkout@v4 + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.10" # The Python version set here is the minimum supported by content, if you change it here, please change all the places containing this comment. + - name: Setup Poetry + uses: Gr1N/setup-poetry@48b0f77c8c1b1b19cb962f0f00dff7b4be8f81ec # v9 + - name: Install python dependencies + run: | + poetry install --with github-actions + - name: set pythonpath + run: | + echo "PYTHONPATH=$GITHUB_WORKSPACE" >> $GITHUB_ENV + - name: Check if connectus-approved label exists + id: check_label + env: + PR_NUMBER: ${{ github.event.pull_request.number }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + echo "Checking label connectus-approved for: $PR_NUMBER" + cd .github/github_workflow_scripts + poetry run python check_if_connectus_approved_label_exists.py --pr_number $PR_NUMBER --github_token $GITHUB_TOKEN diff --git a/.gitignore b/.gitignore index beff941e8e33..81816dab0a69 100644 --- a/.gitignore +++ b/.gitignore @@ -116,7 +116,12 @@ report.md plans/ cooc_command_examples.txt +# Local investigation workspace (customer data, logs, dumps, scratch files). Never commit. +/Investigations/ + # IDEX custom tools and rules .roomodes .roo/ .idex/ +.worktree-context/ + diff --git a/.hooks/bootstrap b/.hooks/bootstrap index 76494dec8a25..97d3af924efd 100755 --- a/.hooks/bootstrap +++ b/.hooks/bootstrap @@ -21,7 +21,7 @@ PATH=~/.local/bin:$PATH if [[ "$1" == "-h" ]] || [[ "$1" == "--help" ]]; then cat << __EOF__ Setup development environment (run with no arguments): -* install pre-commit hooks (set NO_HOOKS=1 to skip) +* install pre-commit and pre-push hooks (set NO_HOOKS=1 to skip) __EOF__ exit 0 fi @@ -39,6 +39,14 @@ GIT_HOOKS_DIR="${PWD}/.git/hooks" if [ -n "${NO_HOOKS}" ]; then echo "Skipping hooks setup as environment variable NO_HOOKS is set" else + if [ ! -e "${GIT_HOOKS_DIR}/pre-push" ]; then + echo "Installing 'pre-push' hook" + ln -s "${HOOKS_DIR}/pre-push" "${GIT_HOOKS_DIR}/pre-push" + exit_on_error $? "Failed to install pre-push hook" + else + echo "Skipping install of pre-push hook as it already exists." + fi + if [ ! -e "${GIT_HOOKS_DIR}/pre-commit" ]; then echo "Installing 'pre-commit' hooks" ln -s "${HOOKS_DIR}/pre-commit" "${GIT_HOOKS_DIR}/pre-commit" diff --git a/.hooks/pre-push b/.hooks/pre-push new file mode 100755 index 000000000000..4d4a3e993a91 --- /dev/null +++ b/.hooks/pre-push @@ -0,0 +1,29 @@ +#!/bin/sh +# Block accidental direct pushes to protected branches of the official repos. +# Pushes to forks and any other remote are left alone. +# Bypass intentionally with: git push --no-verify + +# git invokes this hook as: pre-push +# Normalize the URL so the match can be anchored to the end: without this, +# '*demisto/content*' would also catch content-private, content-docs, etc. +repo=${2%/} +repo=${repo%.git} +case "$repo" in + *demisto/content|*cortex-content/content) ;; + *) exit 0 ;; +esac + +protected="master main" + +while read -r _local_ref _local_sha remote_ref _remote_sha; do + branch=${remote_ref#refs/heads/} + for p in $protected; do + if [ "$branch" = "$p" ]; then + echo "pre-push: refusing to push directly to '$p'." >&2 + echo " Open a PR, or use 'git push --no-verify' if you really mean it." >&2 + exit 1 + fi + done +done + +exit 0 diff --git a/.secrets_white_list.json b/.secrets_white_list.json index 393242ea65cd..034dae50d031 100644 --- a/.secrets_white_list.json +++ b/.secrets_white_list.json @@ -1,31 +1,32 @@ -{ - "files": [], - "scripts": [], - "iocs": [], - "falsePositives": [ - "https://api.criminalip.io", - "api.criminalip.io", - "CriminalIP.IP.score.outbound", - "CriminalIP.IP.whois.data.org_name", - "CriminalIP.IP.whois.data.postal_code", - "CriminalIP.IP.whois.data.longitude", - "CriminalIP.IP.whois.data.confirmed_time", - "CriminalIP.IP.vpn.data.confirmed_time", - "CriminalIP.IP.anonymous_vpn.data", - "CriminalIP.IP.anonymous_vpn.data.vpn_name", - "CriminalIP.IP.anonymous_vpn.data.vpn_url", - "CriminalIP.IP.anonymous_vpn.data.confirmed_time", - "CriminalIP.IP.webcam.data.image_path", - "CriminalIP.IP.webcam.data.open_port_no", - "CriminalIP.IP.webcam.data.confirmed_time", - "CriminalIP.IP.honeypot.data.log_date", - "CriminalIP.IP.honeypot.data.dst_port", - "CriminalIP.IP.honeypot.data.user_agent", - "CriminalIP.IP.honeypot.data.confirmed_time", - "CriminalIP.IP.port.data.confirmed_time", - "CriminalIP.IP.port.data.ssl_info_raw", - "CriminalIP.IP.port.data.technologies", - "CriminalIP.IP.port.data.technologies.tech_name", - "CriminalIP.IP.port.data.technologies.tech_logo_url" - ] +{ + "files": [], + "scripts": [], + "iocs": [], + "falsePositives": [ + "https://api-your-tenant-url", + "https://api.criminalip.io", + "api.criminalip.io", + "CriminalIP.IP.score.outbound", + "CriminalIP.IP.whois.data.org_name", + "CriminalIP.IP.whois.data.postal_code", + "CriminalIP.IP.whois.data.longitude", + "CriminalIP.IP.whois.data.confirmed_time", + "CriminalIP.IP.vpn.data.confirmed_time", + "CriminalIP.IP.anonymous_vpn.data", + "CriminalIP.IP.anonymous_vpn.data.vpn_name", + "CriminalIP.IP.anonymous_vpn.data.vpn_url", + "CriminalIP.IP.anonymous_vpn.data.confirmed_time", + "CriminalIP.IP.webcam.data.image_path", + "CriminalIP.IP.webcam.data.open_port_no", + "CriminalIP.IP.webcam.data.confirmed_time", + "CriminalIP.IP.honeypot.data.log_date", + "CriminalIP.IP.honeypot.data.dst_port", + "CriminalIP.IP.honeypot.data.user_agent", + "CriminalIP.IP.honeypot.data.confirmed_time", + "CriminalIP.IP.port.data.confirmed_time", + "CriminalIP.IP.port.data.ssl_info_raw", + "CriminalIP.IP.port.data.technologies", + "CriminalIP.IP.port.data.technologies.tech_name", + "CriminalIP.IP.port.data.technologies.tech_logo_url" + ] } \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md index 1c871a5d31e6..caac2f5e0ff4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -37,6 +37,7 @@ demisto-sdk pre-commit -a - **Error Handling**: Use `return_error("message")` for user-facing errors. Only raise exceptions for unexpected failures. - **Outputs**: Use `CommandResults` objects with `return_results()`: `return_results(CommandResults(outputs_prefix='MyPrefix', outputs=data))`. Output keys should be CamelCase. - **Logging**: Use `demisto.debug()` and `demisto.info()`. Avoid `print()`. +- **Local Investigation Files**: Put every non-content artifact (customer logs, war room exports, HAR files, API responses, scratch scripts, drafts of Jira comments or PR bodies, plans, reports) under `/Investigations/` at the repo root, one subfolder per ticket (for example `Investigations/XSUP-12345/`). The folder is gitignored. Never write these files anywhere else in the repo, and never move them into `Packs/` or other tracked paths. They may contain customer data or secrets. ## Architecture Notes diff --git a/Config/core_packs_platform_list.json b/Config/core_packs_platform_list.json index 9c999a8d4586..f4a2ab33353d 100644 --- a/Config/core_packs_platform_list.json +++ b/Config/core_packs_platform_list.json @@ -11,21 +11,6 @@ "cloud_posture" ] }, - { - "id": "AIAgents", - "supportedModules": [ - "agentix", - "asm", - "cloud", - "cloud_appsec", - "cloud_posture", - "cloud_runtime_security", - "edr", - "email_security", - "exposure_management", - "xsiam" - ] - }, { "id": "AutoFocus", "supportedModules": [ @@ -48,7 +33,8 @@ "cloud_runtime_security", "edr", "exposure_management", - "xsiam" + "xsiam", + "data_security" ] }, { @@ -104,8 +90,7 @@ "agentix", "cloud_posture", "xsiam", - "edr", - "agentix_xsiam" + "edr" ] }, { diff --git a/Packs/AIAgents/ReleaseNotes/1_0_39.md b/Packs/AIAgents/ReleaseNotes/1_0_39.md new file mode 100644 index 000000000000..918262cdd31e --- /dev/null +++ b/Packs/AIAgents/ReleaseNotes/1_0_39.md @@ -0,0 +1,3 @@ +## AI Agents + +- Removed the **AI Agents** pack from the platform core packs list. No functional changes in this release. diff --git a/Packs/AIAgents/ReleaseNotes/1_0_40.md b/Packs/AIAgents/ReleaseNotes/1_0_40.md new file mode 100644 index 000000000000..7007b3e49f26 --- /dev/null +++ b/Packs/AIAgents/ReleaseNotes/1_0_40.md @@ -0,0 +1,3 @@ +## AI Agents + +- Marked the pack as hidden. No functional changes in this release. diff --git a/Packs/AIAgents/pack_metadata.json b/Packs/AIAgents/pack_metadata.json index b87a7518ddcb..135fbaf001dc 100644 --- a/Packs/AIAgents/pack_metadata.json +++ b/Packs/AIAgents/pack_metadata.json @@ -2,7 +2,8 @@ "name": "AI Agents", "description": "AI Agents", "support": "xsoar", - "currentVersion": "1.0.38", + "hidden": true, + "currentVersion": "1.0.40", "serverMinVersion": "8.13", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", diff --git a/Packs/AMP/Integrations/AMPv2/AMPv2.yml b/Packs/AMP/Integrations/AMPv2/AMPv2.yml index 3d934c99a358..816dbada23a4 100644 --- a/Packs/AMP/Integrations/AMPv2/AMPv2.yml +++ b/Packs/AMP/Integrations/AMPv2/AMPv2.yml @@ -87,6 +87,13 @@ configuration: type: 13 required: false section: Collect +- display: Incidents Fetch Interval + name: incidentFetchInterval + defaultvalue: '1' + required: false + type: 19 + section: Collect + advanced: true - display: Fetch incidents name: isFetch type: 8 diff --git a/Packs/AMP/Integrations/CiscoAMPEventCollector/README.md b/Packs/AMP/Integrations/CiscoAMPEventCollector/README.md index 882fece76819..947e9ac46c17 100644 --- a/Packs/AMP/Integrations/CiscoAMPEventCollector/README.md +++ b/Packs/AMP/Integrations/CiscoAMPEventCollector/README.md @@ -1,8 +1,6 @@ This is the Cisco AMP event collector integration for Cortex XSIAM. This integration was integrated and tested with version v1 of CiscoAMPEventCollector. -This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM. - ## Configure Cisco AMP Event Collector in Cortex | **Parameter** | **Required** | diff --git a/Packs/AMP/ModelingRules/AMP/AMP.yml b/Packs/AMP/ModelingRules/AMP/AMP.yml index 2bc491899199..f463944bbcaa 100644 --- a/Packs/AMP/ModelingRules/AMP/AMP.yml +++ b/Packs/AMP/ModelingRules/AMP/AMP.yml @@ -1,4 +1,5 @@ fromversion: 8.3.0 +toversion: 8.14.0 id: AMP_ModelingRule name: AMP Modeling Rule rules: '' diff --git a/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11.xif b/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11.xif new file mode 100644 index 000000000000..53e961309a3a --- /dev/null +++ b/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11.xif @@ -0,0 +1,49 @@ +[MODEL: dataset = "cisco_secure_endpoint_raw"] +alter + external_ip = computer -> external_ip, + network_addresses_ips = arraymap(computer -> network_addresses[], "@element" -> ip), + first_ip_network_addresses = json_extract_scalar(arrayindex(computer -> network_addresses[], 0), "$.ip"), + file_description = concat(file -> file_name, " - ", file -> disposition), + cloud_ioc_description = cloud_ioc -> description, + remote_ip_network_info = network_info -> remote_ip, + protocol_network_info = network_info -> nfm.protocol +| alter + xdm.event.id = to_string(id), + xdm.event.type = event_type, + xdm.event.description = if(file_description != null and cloud_ioc_description != null, concat(cloud_ioc_description, "; ", file_description), coalesce(cloud_ioc_description, file_description )), + xdm.event.outcome = if(lowercase(event_type) contains "fail", XDM_CONST.OUTCOME_FAILED, lowercase(event_type) contains "succ", XDM_CONST.OUTCOME_SUCCESS, null), + xdm.event.outcome_reason = error -> description, + xdm.alert.original_threat_id = coalesce(detection_id, threat_hunting -> incident_hunt_guid), + xdm.alert.severity = severity, + xdm.alert.original_threat_name = detection, + xdm.alert.name = coalesce(detection, threat_hunting -> incident_title), + xdm.alert.description = threat_hunting -> incident_summary, + xdm.alert.mitre_tactics = arraymap(tactics -> [], trim("@element", "\"")), + xdm.alert.mitre_techniques = arraymap(techniques -> [], trim("@element", "\"")), + xdm.alert.risks = vulnerabilities -> [], + xdm.source.host.hostname = computer -> hostname, + xdm.source.user.username = computer -> user, + xdm.source.identity.username = computer -> user, + xdm.source.agent.identifier = connector_guid, + xdm.intermediate.ipv4 = if(external_ip ~= "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}", external_ip, null), + xdm.intermediate.ipv6 = if(external_ip ~= "[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}", external_ip, null), + xdm.source.ipv4 = if(first_ip_network_addresses ~= "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}", first_ip_network_addresses, null), + xdm.source.ipv6 = if(first_ip_network_addresses ~= "[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}", first_ip_network_addresses, null), + xdm.source.host.ipv4_addresses = arrayfilter(network_addresses_ips, "@element" ~= "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}"), + xdm.source.host.ipv6_addresses = arrayfilter(network_addresses_ips, "@element" ~= "[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}"), + xdm.source.host.mac_addresses = arraymap(computer -> network_addresses[], "@element" -> mac), + xdm.target.file.filename = file -> file_name, + xdm.target.file.path = file -> file_path, + xdm.target.file.md5 = file -> identity.md5, + xdm.target.file.sha256 = file -> identity.sha256, + xdm.source.process.pid = coalesce(to_integer(file -> parent.process_id), to_integer(network_info -> parent.process_id)), + xdm.source.process.command_line = command_line -> arguments, + xdm.source.process.executable.filename = coalesce(file -> parent.file_name, network_info -> parent.file_name), + xdm.source.process.executable.md5 = coalesce(file -> parent.identity.md5, network_info -> parent.identity.md5), + xdm.source.process.executable.sha256 = coalesce(file -> parent.identity.sha256, network_info -> parent.identity.sha256), + xdm.target.ipv4 = if(remote_ip_network_info ~= "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}", remote_ip_network_info, null), + xdm.target.ipv6 = if(remote_ip_network_info ~= "[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}[a-fA-F0-9\:]{1,5}", remote_ip_network_info, null), + xdm.target.port = to_integer(network_info -> remote_port), + xdm.target.url = network_info -> dirty_url, + xdm.network.http.url = network_info -> dirty_url, + xdm.network.ip_protocol = if(protocol_network_info = "HOPOPT", XDM_CONST.IP_PROTOCOL_HOPOPT, protocol_network_info = "ICMP",XDM_CONST.IP_PROTOCOL_ICMP, protocol_network_info = "IGMP",XDM_CONST.IP_PROTOCOL_IGMP, protocol_network_info = "GGP",XDM_CONST.IP_PROTOCOL_GGP, protocol_network_info = "IP",XDM_CONST.IP_PROTOCOL_IP, protocol_network_info = "ST",XDM_CONST.IP_PROTOCOL_ST, protocol_network_info = "TCP",XDM_CONST.IP_PROTOCOL_TCP, protocol_network_info = "CBT",XDM_CONST.IP_PROTOCOL_CBT, protocol_network_info = "EGP",XDM_CONST.IP_PROTOCOL_EGP, protocol_network_info = "IGP",XDM_CONST.IP_PROTOCOL_IGP, protocol_network_info = "BBN_RCC_MON",XDM_CONST.IP_PROTOCOL_BBN_RCC_MON, protocol_network_info = "NVP_II",XDM_CONST.IP_PROTOCOL_NVP_II, protocol_network_info = "PUP",XDM_CONST.IP_PROTOCOL_PUP, protocol_network_info = "ARGUS",XDM_CONST.IP_PROTOCOL_ARGUS, protocol_network_info = "EMCON",XDM_CONST.IP_PROTOCOL_EMCON, protocol_network_info = "XNET",XDM_CONST.IP_PROTOCOL_XNET, protocol_network_info = "CHAOS",XDM_CONST.IP_PROTOCOL_CHAOS, protocol_network_info = "UDP",XDM_CONST.IP_PROTOCOL_UDP, protocol_network_info = "MUX",XDM_CONST.IP_PROTOCOL_MUX, protocol_network_info = "DCN_MEAS",XDM_CONST.IP_PROTOCOL_DCN_MEAS, protocol_network_info = "HMP",XDM_CONST.IP_PROTOCOL_HMP, protocol_network_info = "PRM",XDM_CONST.IP_PROTOCOL_PRM, protocol_network_info = "XNS_IDP",XDM_CONST.IP_PROTOCOL_XNS_IDP, protocol_network_info = "TRUNK_1",XDM_CONST.IP_PROTOCOL_TRUNK_1, protocol_network_info = "TRUNK_2",XDM_CONST.IP_PROTOCOL_TRUNK_2, protocol_network_info = "LEAF_1",XDM_CONST.IP_PROTOCOL_LEAF_1, protocol_network_info = "LEAF_2",XDM_CONST.IP_PROTOCOL_LEAF_2, protocol_network_info = "RDP",XDM_CONST.IP_PROTOCOL_RDP, protocol_network_info = "IRTP",XDM_CONST.IP_PROTOCOL_IRTP, protocol_network_info = "ISO_TP4",XDM_CONST.IP_PROTOCOL_ISO_TP4, protocol_network_info = "NETBLT",XDM_CONST.IP_PROTOCOL_NETBLT, protocol_network_info = "MFE_NSP",XDM_CONST.IP_PROTOCOL_MFE_NSP, protocol_network_info = "MERIT_INP",XDM_CONST.IP_PROTOCOL_MERIT_INP, protocol_network_info = "DCCP",XDM_CONST.IP_PROTOCOL_DCCP, protocol_network_info = "3PC",XDM_CONST.IP_PROTOCOL_3PC, protocol_network_info = "IDPR",XDM_CONST.IP_PROTOCOL_IDPR, protocol_network_info = "XTP",XDM_CONST.IP_PROTOCOL_XTP, protocol_network_info = "DDP",XDM_CONST.IP_PROTOCOL_DDP, protocol_network_info = "IDPR_CMTP",XDM_CONST.IP_PROTOCOL_IDPR_CMTP, protocol_network_info = "TP",XDM_CONST.IP_PROTOCOL_TP, protocol_network_info = "IL",XDM_CONST.IP_PROTOCOL_IL, protocol_network_info = "IPV6",XDM_CONST.IP_PROTOCOL_IPV6, protocol_network_info = "SDRP",XDM_CONST.IP_PROTOCOL_SDRP, protocol_network_info = "IPV6_ROUTE",XDM_CONST.IP_PROTOCOL_IPV6_ROUTE, protocol_network_info = "IPV6_FRAG",XDM_CONST.IP_PROTOCOL_IPV6_FRAG, protocol_network_info = "IDRP",XDM_CONST.IP_PROTOCOL_IDRP, protocol_network_info = "RSVP",XDM_CONST.IP_PROTOCOL_RSVP, protocol_network_info = "GRE",XDM_CONST.IP_PROTOCOL_GRE, protocol_network_info = "DSR",XDM_CONST.IP_PROTOCOL_DSR, protocol_network_info = "BNA",XDM_CONST.IP_PROTOCOL_BNA, protocol_network_info = "ESP",XDM_CONST.IP_PROTOCOL_ESP, protocol_network_info = "AH",XDM_CONST.IP_PROTOCOL_AH, protocol_network_info = "I_NLSP",XDM_CONST.IP_PROTOCOL_I_NLSP, protocol_network_info = "SWIPE",XDM_CONST.IP_PROTOCOL_SWIPE, protocol_network_info = "NARP",XDM_CONST.IP_PROTOCOL_NARP, protocol_network_info = "MOBILE",XDM_CONST.IP_PROTOCOL_MOBILE, protocol_network_info = "TLSP",XDM_CONST.IP_PROTOCOL_TLSP, protocol_network_info = "SKIP",XDM_CONST.IP_PROTOCOL_SKIP, protocol_network_info = "IPV6_ICMP",XDM_CONST.IP_PROTOCOL_IPV6_ICMP, protocol_network_info = "IPV6_NONXT",XDM_CONST.IP_PROTOCOL_IPV6_NONXT, protocol_network_info = "IPV6_OPTS",XDM_CONST.IP_PROTOCOL_IPV6_OPTS, protocol_network_info = "CFTP",XDM_CONST.IP_PROTOCOL_CFTP, protocol_network_info = "SAT_EXPAK",XDM_CONST.IP_PROTOCOL_SAT_EXPAK, protocol_network_info = "KRYPTOLAN",XDM_CONST.IP_PROTOCOL_KRYPTOLAN, protocol_network_info = "RVD",XDM_CONST.IP_PROTOCOL_RVD, protocol_network_info = "IPPC",XDM_CONST.IP_PROTOCOL_IPPC, protocol_network_info = "SAT_MON",XDM_CONST.IP_PROTOCOL_SAT_MON, protocol_network_info = "VISA",XDM_CONST.IP_PROTOCOL_VISA, protocol_network_info = "IPCV",XDM_CONST.IP_PROTOCOL_IPCV, protocol_network_info = "CPNX",XDM_CONST.IP_PROTOCOL_CPNX, protocol_network_info = "CPHB",XDM_CONST.IP_PROTOCOL_CPHB, protocol_network_info = "WSN",XDM_CONST.IP_PROTOCOL_WSN, protocol_network_info = "PVP",XDM_CONST.IP_PROTOCOL_PVP, protocol_network_info = "BR_SAT_MON",XDM_CONST.IP_PROTOCOL_BR_SAT_MON, protocol_network_info = "SUN_ND",XDM_CONST.IP_PROTOCOL_SUN_ND, protocol_network_info = "WB_MON",XDM_CONST.IP_PROTOCOL_WB_MON, protocol_network_info = "WB_EXPAK",XDM_CONST.IP_PROTOCOL_WB_EXPAK, protocol_network_info = "ISO_IP",XDM_CONST.IP_PROTOCOL_ISO_IP, protocol_network_info = "VMTP",XDM_CONST.IP_PROTOCOL_VMTP, protocol_network_info = "SECURE_VMTP",XDM_CONST.IP_PROTOCOL_SECURE_VMTP, protocol_network_info = "VINES",XDM_CONST.IP_PROTOCOL_VINES, protocol_network_info = "TTP",XDM_CONST.IP_PROTOCOL_TTP, protocol_network_info = "NSFNET_IGP",XDM_CONST.IP_PROTOCOL_NSFNET_IGP, protocol_network_info = "DGP",XDM_CONST.IP_PROTOCOL_DGP, protocol_network_info = "TCF",XDM_CONST.IP_PROTOCOL_TCF, protocol_network_info = "EIGRP",XDM_CONST.IP_PROTOCOL_EIGRP, protocol_network_info = "OSPFIGP",XDM_CONST.IP_PROTOCOL_OSPFIGP, protocol_network_info = "SPRITE_RPC",XDM_CONST.IP_PROTOCOL_SPRITE_RPC, protocol_network_info = "LARP",XDM_CONST.IP_PROTOCOL_LARP, protocol_network_info = "MTP",XDM_CONST.IP_PROTOCOL_MTP, protocol_network_info = "AX25",XDM_CONST.IP_PROTOCOL_AX25, protocol_network_info = "IPIP",XDM_CONST.IP_PROTOCOL_IPIP, protocol_network_info = "MICP",XDM_CONST.IP_PROTOCOL_MICP, protocol_network_info = "SCC_SP",XDM_CONST.IP_PROTOCOL_SCC_SP, protocol_network_info = "ETHERIP",XDM_CONST.IP_PROTOCOL_ETHERIP, protocol_network_info = "ENCAP",XDM_CONST.IP_PROTOCOL_ENCAP, protocol_network_info = "GMTP",XDM_CONST.IP_PROTOCOL_GMTP, protocol_network_info = "IFMP",XDM_CONST.IP_PROTOCOL_IFMP, protocol_network_info = "PNNI",XDM_CONST.IP_PROTOCOL_PNNI, protocol_network_info = "PIM",XDM_CONST.IP_PROTOCOL_PIM, protocol_network_info = "ARIS",XDM_CONST.IP_PROTOCOL_ARIS, protocol_network_info = "SCPS",XDM_CONST.IP_PROTOCOL_SCPS, protocol_network_info = "QNX",XDM_CONST.IP_PROTOCOL_QNX, protocol_network_info = "AN",XDM_CONST.IP_PROTOCOL_AN, protocol_network_info = "IPCOMP",XDM_CONST.IP_PROTOCOL_IPCOMP, protocol_network_info = "SNP",XDM_CONST.IP_PROTOCOL_SNP, protocol_network_info = "COMPAQ_PEER",XDM_CONST.IP_PROTOCOL_COMPAQ_PEER, protocol_network_info = "IPX_IN_IP",XDM_CONST.IP_PROTOCOL_IPX_IN_IP, protocol_network_info = "VRRP",XDM_CONST.IP_PROTOCOL_VRRP, protocol_network_info = "PGM",XDM_CONST.IP_PROTOCOL_PGM, protocol_network_info = "L2TP",XDM_CONST.IP_PROTOCOL_L2TP, protocol_network_info = "DDX",XDM_CONST.IP_PROTOCOL_DDX, protocol_network_info = "IATP",XDM_CONST.IP_PROTOCOL_IATP, protocol_network_info = "STP",XDM_CONST.IP_PROTOCOL_STP, protocol_network_info = "SRP",XDM_CONST.IP_PROTOCOL_SRP, protocol_network_info = "UTI",XDM_CONST.IP_PROTOCOL_UTI, protocol_network_info = "SMP",XDM_CONST.IP_PROTOCOL_SMP, protocol_network_info = "SM",XDM_CONST.IP_PROTOCOL_SM, protocol_network_info = "PTP",XDM_CONST.IP_PROTOCOL_PTP, protocol_network_info = "ISIS",XDM_CONST.IP_PROTOCOL_ISIS, protocol_network_info = "FIRE",XDM_CONST.IP_PROTOCOL_FIRE, protocol_network_info = "CRTP",XDM_CONST.IP_PROTOCOL_CRTP, protocol_network_info = "CRUDP",XDM_CONST.IP_PROTOCOL_CRUDP, protocol_network_info = "SSCOPMCE",XDM_CONST.IP_PROTOCOL_SSCOPMCE, protocol_network_info = "IPLT",XDM_CONST.IP_PROTOCOL_IPLT, protocol_network_info = "SPS",XDM_CONST.IP_PROTOCOL_SPS, protocol_network_info = "PIPE",XDM_CONST.IP_PROTOCOL_PIPE, protocol_network_info = "SCTP",XDM_CONST.IP_PROTOCOL_SCTP, protocol_network_info = "FC",XDM_CONST.IP_PROTOCOL_FC, protocol_network_info = "RSVP_E2E_IGNORE",XDM_CONST.IP_PROTOCOL_RSVP_E2E_IGNORE, protocol_network_info = "MOBILITY",XDM_CONST.IP_PROTOCOL_MOBILITY, protocol_network_info = "UDPLITE",XDM_CONST.IP_PROTOCOL_UDPLITE, protocol_network_info = "MPLS_IN_IP",XDM_CONST.IP_PROTOCOL_MPLS_IN_IP, protocol_network_info = "MANET",XDM_CONST.IP_PROTOCOL_MANET, protocol_network_info = "HIP",XDM_CONST.IP_PROTOCOL_HIP, protocol_network_info = "SHIM6",XDM_CONST.IP_PROTOCOL_SHIM6, protocol_network_info = "WESP",XDM_CONST.IP_PROTOCOL_WESP, protocol_network_info = "ROHC",XDM_CONST.IP_PROTOCOL_ROHC, protocol_network_info = "RESERVED",XDM_CONST.IP_PROTOCOL_RESERVED,to_string(protocol_network_info)); \ No newline at end of file diff --git a/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11.yml b/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11.yml new file mode 100644 index 000000000000..65a27a90711f --- /dev/null +++ b/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: AMP_ModelingRule +name: AMP Modeling Rule +rules: '' +schema: '' +tags: +supportedModules: +- xsiam diff --git a/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11_schema.json b/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11_schema.json new file mode 100644 index 000000000000..ab0aa9406dcb --- /dev/null +++ b/Packs/AMP/ModelingRules/AMP_2_11/AMP_2_11_schema.json @@ -0,0 +1,68 @@ +{ + "cisco_secure_endpoint_raw": { + "computer": { + "type": "string", + "is_array": false + }, + "id": { + "type": "int", + "is_array": false + }, + "event_type": { + "type": "string", + "is_array": false + }, + "file": { + "type": "string", + "is_array": false + }, + "detection_id": { + "type": "string", + "is_array": false + }, + "severity": { + "type": "string", + "is_array": false + }, + "detection": { + "type": "string", + "is_array": false + }, + "connector_guid": { + "type": "string", + "is_array": false + }, + "tactics": { + "type": "string", + "is_array": false + }, + "techniques": { + "type": "string", + "is_array": false + }, + "cloud_ioc": { + "type": "string", + "is_array": false + }, + "command_line": { + "type": "string", + "is_array": false + }, + "vulnerabilities": { + "type": "string", + "is_array": false + }, + "error": { + "type": "string", + "is_array": false + }, + "threat_hunting": { + "type": "string", + "is_array": false + }, + "network_info": { + "type": "string", + "is_array": false + } + } +} \ No newline at end of file diff --git a/Packs/AMP/ReleaseNotes/2_1_18.md b/Packs/AMP/ReleaseNotes/2_1_18.md new file mode 100644 index 000000000000..2118a24eef7a --- /dev/null +++ b/Packs/AMP/ReleaseNotes/2_1_18.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: AMP Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/AMP/ReleaseNotes/2_1_19.md b/Packs/AMP/ReleaseNotes/2_1_19.md new file mode 100644 index 000000000000..832d96690f20 --- /dev/null +++ b/Packs/AMP/ReleaseNotes/2_1_19.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### Cisco AMP v2 + +- Updated the Fetch Incidents configuration parameters. diff --git a/Packs/AMP/ReleaseNotes/2_1_20.md b/Packs/AMP/ReleaseNotes/2_1_20.md new file mode 100644 index 000000000000..011b377a5023 --- /dev/null +++ b/Packs/AMP/ReleaseNotes/2_1_20.md @@ -0,0 +1,3 @@ +## Cisco AMP + +- Documentation and metadata improvements. diff --git a/Packs/AMP/pack_metadata.json b/Packs/AMP/pack_metadata.json index 684966d70c86..fda076784a95 100644 --- a/Packs/AMP/pack_metadata.json +++ b/Packs/AMP/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Cisco AMP", "description": "Uses CISCO AMP Endpoint", "support": "xsoar", - "currentVersion": "2.1.17", + "currentVersion": "2.1.20", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -22,7 +22,6 @@ "marketplacev2", "platform" ], - "defaultDataSource": "CiscoAMPEventCollector", "supportedModules": [ "agentix", "xsiam" diff --git a/Packs/ANYRUN/.pack-ignore b/Packs/ANYRUN/.pack-ignore index 8e1dff94f7fd..a8736b726396 100644 --- a/Packs/ANYRUN/.pack-ignore +++ b/Packs/ANYRUN/.pack-ignore @@ -4,6 +4,42 @@ ignore=RM104 [file:AnyRunFeed.yml] ignore=IN122 +[file:ANYRUN_Detonate_URL_Android_for_XDR.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_URL_Windows.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_File_Windows_for_XDR.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_File_Android_for_XDR.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_URL_Linux_for_XDR.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_URL_Windows_for_XDR.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_File_Linux_for_XDR.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_File_Android.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_File_Linux.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_URL_Linux.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_URL_Android.yml] +ignore=PB106 + +[file:ANYRUN_Detonate_File_Windows.yml] +ignore=PB106 + [known_words] Ubuntu VM diff --git a/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed.py b/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed.py index 79f131422d70..2515112e8aad 100644 --- a/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed.py +++ b/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed.py @@ -1,14 +1,14 @@ from datetime import datetime -import demistomock as demisto -from CommonServerPython import * - +from anyrun import RunTimeException from anyrun.connectors import FeedsConnector from anyrun.iterators import FeedsIterator -from anyrun import RunTimeException + +import demistomock as demisto +from CommonServerPython import * DATE_TIME_FORMAT = "%Y-%m-%d %H:%M:%S" -VERSION = "PA-XSOAR:2.3.0" +VERSION = "PA-XSOAR:2.5.0" def test_module(params: dict) -> str: # pragma: no cover @@ -77,17 +77,39 @@ def convert_indicators(indicators: list[dict]) -> list[dict]: for indicator in indicators: indicator_type, indicator_value = extract_indicator_data(indicator) + fields: dict[str, Any] = { + "firstseenbysource": indicator.get("created"), + "first_seen": indicator.get("created"), + "modified": indicator.get("modified"), + "last_seen": indicator.get("modified"), + "vendor": "ANY.RUN", + "source": "ANY.RUN TI Feed", + "tags": indicator.get("labels") or [], + "publications": [ + { + "title": ref.get("source_name") or "", + "link": ref.get("url") or "", + "source": "ANY.RUN TI Feed", + "timestamp": indicator.get("created"), + } + for ref in indicator.get("external_references") or [] + if ref.get("url") + ], + } + if indicator_type == "domain-name": + fields["communitynotes"] = [ + { + "notes": ref["url"], + "timestamp": indicator.get("created"), + } + for ref in indicator.get("external_references") or [] + if ref.get("url") + ] + indicator_payload = { "value": indicator_value, "type": {"ipv4-addr": "IP", "url": "URL", "domain-name": "Domain"}.get(indicator_type), - "fields": { - "firstseenbysource": indicator.get("created"), - "first_seen": indicator.get("created"), - "modified": indicator.get("modified"), - "last_seen": indicator.get("modified"), - "vendor": "ANY.RUN", - "source": "ANY.RUN TI Feed", - }, + "fields": fields, } converted_indicators.append(indicator_payload) diff --git a/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed.yml b/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed.yml index dcc8c3f852a9..c7d09c597f56 100644 --- a/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed.yml +++ b/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed.yml @@ -71,7 +71,6 @@ configuration: - never - interval - indicatorType - - suddenDeath - display: "" name: feedExpirationInterval type: 1 @@ -95,6 +94,14 @@ configuration: type: 0 section: Collect additionalinfo: Supports CSV values. +- additionalinfo: Whether the feed should pull only new or modified indicators that have been sent from the integration. As the determination if the indicator is new or modified happens on the 3rd-party vendor's side, and only indicators that are new or modified are sent to Cortex XSOAR, all indicators coming from these feeds are labeled new or modified. + defaultvalue: 'true' + display: Incremental Feed + hidden: true + name: feedIncremental + type: 8 + section: Collect + required: false - display: Traffic Light Protocol Color name: tlp_color options: @@ -110,7 +117,7 @@ description: Threat Intelligence Feeds provide data on the known indicators of c display: ANY.RUN TI Feed name: ANY.RUN TI Feed script: - dockerimage: demisto/anyrun-sdk:1.0.0.10120494 + dockerimage: demisto/anyrun-sdk:1.0.0.12844876 subtype: python3 runonce: false script: '-' @@ -119,3 +126,6 @@ script: tests: - No tests fromversion: 6.0.0 +supportedModules: +- agentix +- xsiam diff --git a/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed_test.py b/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed_test.py index 002564f345ae..35bb2e719468 100644 --- a/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed_test.py +++ b/Packs/ANYRUN/Integrations/AnyRunFeed/AnyRunFeed_test.py @@ -33,6 +33,40 @@ def test_convert_indicators_returns_valid_demisto_indicators(indicators: list[di "last_seen": "2025-05-17T20:15:39.130Z", "vendor": "ANY.RUN", "source": "ANY.RUN TI Feed", + "tags": [], + "publications": [], }, } ] + + +def test_convert_indicators_populates_communitynotes_for_domain_indicators(): + indicators = [ + { + "created": "2022-02-23T07:34:19.000Z", + "modified": "2025-05-17T20:15:39.130Z", + "pattern": "[domain-name:value = 'some_domain.com']", + "labels": ["malware"], + "external_references": [ + {"source_name": "ANY.RUN", "url": "https://app.any.run/tasks/some-task-id"}, + ], + } + ] + + converted = convert_indicators(indicators) + + assert converted[0]["fields"]["tags"] == ["malware"] + assert converted[0]["fields"]["publications"] == [ + { + "title": "ANY.RUN", + "link": "https://app.any.run/tasks/some-task-id", + "source": "ANY.RUN TI Feed", + "timestamp": "2022-02-23T07:34:19.000Z", + } + ] + assert converted[0]["fields"]["communitynotes"] == [ + { + "notes": "https://app.any.run/tasks/some-task-id", + "timestamp": "2022-02-23T07:34:19.000Z", + } + ] diff --git a/Packs/ANYRUN/Integrations/AnyRunLookup/AnyRunLookup.py b/Packs/ANYRUN/Integrations/AnyRunLookup/AnyRunLookup.py index f8acb1b5d301..3d0caffeb7f4 100644 --- a/Packs/ANYRUN/Integrations/AnyRunLookup/AnyRunLookup.py +++ b/Packs/ANYRUN/Integrations/AnyRunLookup/AnyRunLookup.py @@ -1,11 +1,12 @@ import json -import demistomock as demisto -from CommonServerPython import * -from anyrun.connectors import LookupConnector from anyrun import RunTimeException +from anyrun.connectors import LookupConnector + +import demistomock as demisto +from CommonServerPython import * -VERSION = "PA-XSOAR:2.3.0" +VERSION = "PA-XSOAR:2.5.0" DBOT_SCORE_TYPE_RESOLVER = { "destination_ip": DBotScoreType.IP, diff --git a/Packs/ANYRUN/Integrations/AnyRunLookup/AnyRunLookup.yml b/Packs/ANYRUN/Integrations/AnyRunLookup/AnyRunLookup.yml index b99bf30b103c..e707623d5abe 100644 --- a/Packs/ANYRUN/Integrations/AnyRunLookup/AnyRunLookup.yml +++ b/Packs/ANYRUN/Integrations/AnyRunLookup/AnyRunLookup.yml @@ -942,7 +942,7 @@ script: - contextPath: DBotScore.Vendor description: The vendor used to calculate the score. type: String - dockerimage: demisto/anyrun-sdk:1.0.0.10120494 + dockerimage: demisto/anyrun-sdk:1.0.0.12844876 subtype: python3 runonce: false script: '-' diff --git a/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox.py b/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox.py index 4c369d34779d..394004e075dc 100644 --- a/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox.py +++ b/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox.py @@ -1,32 +1,68 @@ -import demistomock as demisto -from CommonServerPython import * +import json +from datetime import UTC, datetime, timedelta from anyrun import RunTimeException from anyrun.connectors import SandboxConnector from anyrun.connectors.sandbox.base_connector import BaseSandboxConnector from anyrun.connectors.sandbox.operation_systems import ( - WindowsConnector, - LinuxConnector, AndroidConnector, + LinuxConnector, + WindowsConnector, ) +import demistomock as demisto +from CommonServerPython import * -VERSION = "PA-XSOAR:2.3.0" +VERSION = "PA-XSOAR:2.5.0" SCORE_TO_VERDICT = {0: "Unknown", 1: "Suspicious", 2: "Malicious"} +XDR_REPUTATION = {0: "UNKNOWN", 1: "SUSPICIOUS", 2: "BAD"} +XDR_ALLOWED_IOC_REPUTATIONS = (1, 2) # Only Suspicious and Malicious IOCs are pushed to Cortex XDR +XDR_SEVERITY = {0: "INFO", 1: "MEDIUM", 2: "HIGH"} + +SCORE_BY_REPUTATION = { + 0: Common.DBotScore.NONE, + 1: Common.DBotScore.SUSPICIOUS, + 2: Common.DBotScore.BAD, +} ANYRUN_TO_SOAR_INDICATOR = { "ip": "IP", "url": "URL", "domain": "Domain", + "md5": "File MD5", + "sha1": "File SHA-1", "sha256": "File SHA-256", } +ANYRUN_TO_XDR_INDICATOR = { + "ip": "IP", + "url": "URL", + "domain": "DOMAIN_NAME", + "md5": "HASH", + "sha1": "HASH", + "sha256": "HASH", +} + +XDR_INDICATOR_CLASS = { + "IP": "IP", + "URL": "URL", + "DOMAIN_NAME": "Domain", + "HASH": "Malware", +} + +DEFAULT_ROOT_URL = "any.run" +IOC_EXPIRATION_DAYS = 30 + def test_module(params: dict) -> str: # pragma: no cover """Performs ANY.RUN API call to verify integration is operational""" try: - with BaseSandboxConnector(get_authentication(params), trust_env=argToBoolean(params.get("proxy", False))) as connector: + with BaseSandboxConnector( + get_authentication(params), + trust_env=argToBoolean(params.get("proxy", False)), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, + ) as connector: connector.check_authorization() return "ok" except RunTimeException as exception: @@ -76,6 +112,7 @@ def build_context_path(analysis_type: str, connector: WindowsConnector | LinuxCo def start_analyse( + params: dict, args: dict, analysis_type: str, connector: WindowsConnector | LinuxConnector | AndroidConnector, @@ -83,6 +120,7 @@ def start_analyse( """ Process Sandbox analysis + :param params: Demisto params :param args: Demisto args :param analysis_type: ANY.RUN Sandbox submission type :param connector: ANY.RUN connector instance @@ -94,20 +132,21 @@ def start_analyse( else: task_uuid = connector.run_url_analysis(**args) - return_results( - CommandResults( - outputs_prefix="ANYRUN.SandboxURL", - outputs=f"Link to the interactive analysis: https://app.any.run/tasks/{task_uuid}", - ignore_auto_extract=True, - ) - ) + root_url = params.get("root_url") or DEFAULT_ROOT_URL return_results( - CommandResults( - outputs_prefix=build_context_path(analysis_type, connector), - outputs=task_uuid, - ignore_auto_extract=True, - ) + [ + CommandResults( + outputs_prefix="ANYRUN.SandboxURL", + outputs=f"Link to the interactive analysis: https://app.{root_url}/tasks/{task_uuid}", + ignore_auto_extract=True, + ), + CommandResults( + outputs_prefix=build_context_path(analysis_type, connector), + outputs=task_uuid, + ignore_auto_extract=True, + ), + ] ) @@ -116,9 +155,10 @@ def detonate_entity_windows(params: dict, args: dict, analysis_type: str) -> Non get_authentication(params), integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), - verify_ssl=not params.get("insecure"), + verify_ssl=not argToBoolean(params.get("insecure", False)), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, ) as connector: - start_analyse(args, analysis_type, connector) + start_analyse(params, args, analysis_type, connector) def detonate_entity_linux(params: dict, args: dict, analysis_type: str) -> None: # pragma: no cover @@ -126,9 +166,10 @@ def detonate_entity_linux(params: dict, args: dict, analysis_type: str) -> None: get_authentication(params), integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), - verify_ssl=not params.get("insecure"), + verify_ssl=not argToBoolean(params.get("insecure", False)), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, ) as connector: - start_analyse(args, analysis_type, connector) + start_analyse(params, args, analysis_type, connector) def detonate_entity_android(params: dict, args: dict, analysis_type: str) -> None: # pragma: no cover @@ -136,16 +177,17 @@ def detonate_entity_android(params: dict, args: dict, analysis_type: str) -> Non get_authentication(params), integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), - verify_ssl=not params.get("insecure"), + verify_ssl=not argToBoolean(params.get("insecure", False)), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, ) as connector: - start_analyse(args, analysis_type, connector) + start_analyse(params, args, analysis_type, connector) -def detonate_file_widows(params: dict, args: dict) -> None: # pragma: no cover +def detonate_file_windows(params: dict, args: dict) -> None: # pragma: no cover detonate_entity_windows(params, args, "file") -def detonate_url_widows(params: dict, args: dict) -> None: # pragma: no cover +def detonate_url_windows(params: dict, args: dict) -> None: # pragma: no cover detonate_entity_windows(params, args, "url") @@ -173,6 +215,7 @@ def delete_task(params: dict, args: dict) -> None: # pragma: no cover integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), verify_ssl=not params.get("insecure"), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, ) as connector: connector.delete_task(task_uuid) @@ -187,10 +230,12 @@ def download_analysis_sample(params: dict, args: dict, download_type: str) -> No integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), verify_ssl=not params.get("insecure"), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, ) as connector: if download_type == "pcap": return_results(fileResult(f"{task_uuid}_traffic_dump.pcap", connector.download_pcap(task_uuid))) - return_results(fileResult(f"{task_uuid}_sample.zip", connector.download_file_sample(task_uuid))) + else: + return_results(fileResult(f"{task_uuid}_sample.zip", connector.download_file_sample(task_uuid))) def get_analysis_verdict(params: dict, args: dict) -> None: # pragma: no cover @@ -201,6 +246,7 @@ def get_analysis_verdict(params: dict, args: dict) -> None: # pragma: no cover integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), verify_ssl=not params.get("insecure"), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, ) as connector: for _ in connector.get_task_status(task_uuid): pass @@ -222,6 +268,7 @@ def get_user_limits(params: dict) -> None: # pragma: no cover integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), verify_ssl=not params.get("insecure"), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, ) as connector: user_limits = connector.get_user_limits() @@ -240,6 +287,7 @@ def get_analysis_history(params: dict, args: dict) -> None: # pragma: no cover integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), verify_ssl=not params.get("insecure"), + root_url=params.get("root_url") or DEFAULT_ROOT_URL, ) as connector: analysis_history = connector.get_analysis_history(**args) @@ -252,98 +300,167 @@ def get_analysis_history(params: dict, args: dict) -> None: # pragma: no cover ) -def create_indicators(report: dict, task_uuid: str) -> None: # pragma: no cover +def create_indicators( + report: list, task_uuid: str, root_url: str, create_xdr_iocs: bool = False +) -> tuple[list[dict], str, str]: # pragma: no cover """ Excludes IOCs from the analysis report. Sends them to Threat Intel :param report: Analysis report + :param task_uuid: ANY.RUN task UUID + :param create_xdr_iocs: Whether to also build XDR IOC objects (needed by the XDR playbooks only). + Unknown reputation IOCs are never included. + :return XDR IOCDetails, readable output, comma-separated IOC values """ output: list[dict[str, str]] = [] indicators: list[dict] = [] + ioc_details: list[dict] = [] + ioc_values: list[str] = [] + + expiration_date = int((datetime.now(UTC) + timedelta(days=IOC_EXPIRATION_DAYS)).timestamp() * 1000) for indicator in report: reputation = indicator.get("reputation") - - if reputation == 0: - score = Common.DBotScore.NONE - elif reputation == 1: - score = Common.DBotScore.SUSPICIOUS - elif reputation == 2: - score = Common.DBotScore.BAD - else: + indicator_value = indicator.get("ioc") + anyrun_indicator_type = indicator.get("type") + xsoar_indicator_type = ANYRUN_TO_SOAR_INDICATOR.get(anyrun_indicator_type) + verdict = SCORE_TO_VERDICT.get(reputation) + score = SCORE_BY_REPUTATION.get(reputation) + + xdr_indicator_type = ANYRUN_TO_XDR_INDICATOR.get(anyrun_indicator_type, "") + xdr_severity = XDR_SEVERITY.get(reputation, "HIGH") + xdr_reputation = XDR_REPUTATION.get(reputation, "UNKNOWN") + _xdr_indicator_class = XDR_INDICATOR_CLASS.get(xdr_indicator_type) + + if reputation is None or not indicator_value or not xsoar_indicator_type or score is None or verdict is None: continue - indicator_type = ANYRUN_TO_SOAR_INDICATOR.get(indicator.get("type"), "") - indicators.append( { - "type": indicator_type, - "value": indicator.get("ioc"), + "type": xsoar_indicator_type, + "value": indicator_value, "score": score, "fields": { "vendor": "ANY.RUN", "service": "ANY.RUN Cloud Sandbox", - "description": f"https://app.any.run/tasks/{task_uuid}", + "description": f"https://app.{root_url}/tasks/{task_uuid}", }, } ) output.append( { - "type": indicator_type, - "value": indicator.get("ioc"), - "verdict": SCORE_TO_VERDICT.get(reputation, ""), + "type": xsoar_indicator_type, + "value": indicator_value, + "verdict": verdict, } ) - demisto.createIndicators(indicators) + ioc_values.append(indicator_value) - return_results( - CommandResults( - readable_output=tableToMarkdown( - "Indicators from ANY.RUN Cloud Sandbox", - output, - headers=["type", "value", "verdict"], - headerTransform=string_to_table_header, - ), - ignore_auto_extract=True, + if ( + not create_xdr_iocs + or not xdr_indicator_type + or not _xdr_indicator_class + or reputation not in XDR_ALLOWED_IOC_REPUTATIONS + ): + continue + + ioc_object = { + "indicator": indicator_value, + "type": xdr_indicator_type, + "severity": xdr_severity, + "expiration_date": expiration_date, + "comment": f"ANY.RUN Cloud Sandbox | https://app.{root_url}/tasks/{task_uuid}", + "reputation": xdr_reputation, + "reliability": "A", + "class": _xdr_indicator_class, + "vendors": [ + { + "vendor_name": "ANY.RUN", + "reputation": xdr_reputation, + "reliability": "A", + } + ], + } + + ioc_details.append( + { + "ioc_object": json.dumps(ioc_object), + } ) + + if indicators: + demisto.createIndicators(indicators) + + readable_output = tableToMarkdown( + "Indicators from ANY.RUN Cloud Sandbox", + output, + headers=["type", "value", "verdict"], + headerTransform=string_to_table_header, ) + ioc_values = ",".join(ioc_values) + + return ioc_details, readable_output, ioc_values + def get_analysis_report(params: dict, args: dict) -> None: # pragma: no cover task_uuid = args.get("task_uuid", "") - report_format = args.get("report_format") + report_format = args.get("report_format", "summary") + root_url = params.get("root_url") or DEFAULT_ROOT_URL + create_xdr_iocs = argToBoolean(args.get("create_xdr_iocs", False)) with SandboxConnector.windows( get_authentication(params), integration=VERSION, trust_env=argToBoolean(params.get("proxy", False)), verify_ssl=not params.get("insecure"), + root_url=root_url, ) as connector: + if report_format == "summary": + report_format = "json" + report = connector.get_analysis_report(task_uuid, report_format=report_format) if report_format == "html": return_results(fileResult(f"anyrun_report_{task_uuid}.html", report)) - elif report_format == "summary": - return_results( - CommandResults( - outputs_prefix="ANYRUN.SandboxAnalysis", - outputs=report, - ignore_auto_extract=True, + elif report_format == "json": + return_results(fileResult(f"anyrun_report_{task_uuid}.json", json.dumps(report))) + elif report_format == "ioc": + ioc_details, readable_output, ioc_values = create_indicators(report or [], task_uuid, root_url, create_xdr_iocs) + + if not ioc_values: + return_results( + CommandResults( + readable_output="No IOCs found", + outputs_prefix="ANYRUN", + ignore_auto_extract=True, + ) ) - ) - elif report_format == "ioc" and report: - create_indicators(report, task_uuid) + return return_results( CommandResults( + readable_output=ioc_values, outputs_prefix="ANYRUN.IOCs", - outputs=",".join(indicator.get("ioc") for indicator in report) if report else "", + outputs=ioc_values, ignore_auto_extract=True, ) ) + if create_xdr_iocs: + return_results( + CommandResults( + readable_output=readable_output, + outputs_prefix="ANYRUN", + outputs={ + "IOCDetails": ioc_details, + }, + ignore_auto_extract=True, + ) + ) + def main(): # pragma: no cover """Main Execution block""" @@ -367,9 +484,9 @@ def main(): # pragma: no cover elif demisto.command() == "anyrun-get-analysis-history": get_analysis_history(params, args) elif demisto.command() == "anyrun-detonate-file-windows": - detonate_file_widows(params, args) + detonate_file_windows(params, args) elif demisto.command() == "anyrun-detonate-url-windows": - detonate_url_widows(params, args) + detonate_url_windows(params, args) elif demisto.command() == "anyrun-detonate-file-linux": detonate_file_linux(params, args) elif demisto.command() == "anyrun-detonate-url-linux": @@ -377,6 +494,7 @@ def main(): # pragma: no cover elif demisto.command() == "anyrun-detonate-file-android": detonate_file_android(params, args) elif demisto.command() == "anyrun-detonate-url-android": + args.pop("obj_ext_browser", None) detonate_url_android(params, args) elif demisto.command() == "anyrun-get-analysis-report": get_analysis_report(params, args) @@ -387,6 +505,11 @@ def main(): # pragma: no cover raise NotImplementedError(f"Command {demisto.command()} is not implemented") except RunTimeException as exception: return_error(exception.description, error=str(exception.json)) + except Exception as e: + return_error( + f"Failed to execute {demisto.command()} command.\nError:\n{str(e)}", + error=traceback.format_exc(), + ) if __name__ in ["__main__", "builtin", "builtins"]: diff --git a/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox.yml b/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox.yml index 3ee9eed39b94..9d870d95828c 100644 --- a/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox.yml +++ b/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox.yml @@ -21,6 +21,13 @@ configuration: type: 8 section: Connect required: false +- display: Base domain + name: root_url + type: 0 + required: false + section: Connect + defaultvalue: any.run + additionalinfo: The base domain for the ANY.RUN connection. Enter the domain only (for example, any.run), without http://, https://, or a trailing slash. Using an alternative domain requires special access. description: ANY.RUN Sandbox is an online interactive sandbox for malware analysis, a tool for detection, monitoring, and research of cyber threats in real time. display: ANY.RUN Cloud Sandbox name: ANY.RUN Cloud Sandbox @@ -64,37 +71,57 @@ script: defaultValue: 'en-US' required: false type: String - - description: 'Network connection state.' + - auto: PREDEFINED + description: 'Whether to enable network connection.' name: opt_network_connect - defaultValue: 'True' + defaultValue: true required: false - type: String - - description: 'FakeNet feature status.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to enable the FakeNet feature.' name: opt_network_fakenet - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'TOR using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use TOR.' name: opt_network_tor - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Tor geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The TOR geolocation option, for example, US or AU.' name: opt_network_geo defaultValue: 'fastest' required: false type: String - - description: 'HTTPS MITM proxy option.' + - auto: PREDEFINED + description: 'Whether to use the HTTPS MITM proxy.' name: opt_network_mitm - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use a residential proxy.' name: opt_network_residential_proxy - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The residential proxy geolocation option, for example, US or AU.' name: opt_network_residential_proxy_geo defaultValue: 'fastest' required: false @@ -116,7 +143,16 @@ script: required: false type: String - auto: PREDEFINED - description: 'Start file analysis from the specified directory.' + description: 'Whether to enable automated interactivity.' + name: opt_automated_interactivity + defaultValue: true + required: false + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'The directory from which to start the file analysis.' name: obj_ext_startfolder defaultValue: 'temp' required: false @@ -133,18 +169,26 @@ script: name: obj_ext_cmd required: false type: String - - description: 'Forces the file to execute with elevated privileges and an elevated token (for PE32, PE32+, PE64 files only).' + - auto: PREDEFINED + description: 'Whether to force the file to execute with elevated privileges and an elevated token (for PE32, PE32+, PE64 files only).' name: obj_force_elevation - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Change extension to valid.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to change the extension to a valid one.' name: obj_ext_extension - defaultValue: 'True' + defaultValue: true required: false - type: String + type: boolean + predefined: + - true + - false name: anyrun-detonate-file-windows - description: 'Perform File analysis using Windows VM.' + description: 'Performs file analysis using a Windows VM.' outputs: - contextPath: ANYRUN_DetonateFileWindows.TaskID description: 'Task UUID.' @@ -187,37 +231,57 @@ script: defaultValue: 'en-US' required: false type: String - - description: 'Network connection state.' + - auto: PREDEFINED + description: 'Whether to enable network connection.' name: opt_network_connect - defaultValue: 'True' + defaultValue: true required: false - type: String - - description: 'FakeNet feature status.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to enable the FakeNet feature.' name: opt_network_fakenet - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'TOR using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use TOR.' name: opt_network_tor - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Tor geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The TOR geolocation option, for example, US or AU.' name: opt_network_geo defaultValue: 'fastest' required: false type: String - - description: 'HTTPS MITM proxy option.' + - auto: PREDEFINED + description: 'Whether to use the HTTPS MITM proxy.' name: opt_network_mitm - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use a residential proxy.' name: opt_network_residential_proxy - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The residential proxy geolocation option, for example, US or AU.' name: opt_network_residential_proxy_geo defaultValue: 'fastest' required: false @@ -238,6 +302,15 @@ script: defaultValue: '240' required: false type: String + - auto: PREDEFINED + description: 'Whether to enable automated interactivity.' + name: opt_automated_interactivity + defaultValue: true + required: false + type: boolean + predefined: + - true + - false - auto: PREDEFINED description: 'Browser name.' name: obj_ext_browser @@ -249,13 +322,17 @@ script: - 'Mozilla Firefox' - 'Internet Explorer' - 'Microsoft Edge' - - description: 'Change extension to valid.' + - auto: PREDEFINED + description: 'Whether to change the extension to a valid one.' name: obj_ext_extension - defaultValue: 'True' + defaultValue: true required: false - type: String + type: boolean + predefined: + - true + - false name: anyrun-detonate-url-windows - description: 'Perform URL analysis using Windows VM.' + description: 'Performs URL analysis using a Windows VM.' outputs: - contextPath: ANYRUN_DetonateUrlWindows.TaskID description: 'Task UUID.' @@ -279,37 +356,57 @@ script: predefined: - 'ubuntu' - 'debian' - - description: 'Network connection state.' + - auto: PREDEFINED + description: 'Whether to enable network connection.' name: opt_network_connect - defaultValue: 'True' + defaultValue: true required: false - type: String - - description: 'FakeNet feature status.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to enable the FakeNet feature.' name: opt_network_fakenet - defaultValue: 'False' + defaultValue: true required: false - type: String - - description: 'TOR using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use TOR.' name: opt_network_tor - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Tor geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The TOR geolocation option, for example, US or AU.' name: opt_network_geo defaultValue: 'fastest' required: false type: String - - description: 'HTTPS MITM proxy option.' + - auto: PREDEFINED + description: 'Whether to use the HTTPS MITM proxy.' name: opt_network_mitm - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use a residential proxy.' name: opt_network_residential_proxy - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The residential proxy geolocation option, for example, US or AU.' name: opt_network_residential_proxy_geo defaultValue: 'fastest' required: false @@ -331,7 +428,17 @@ script: required: false type: String - auto: PREDEFINED - description: 'Start file analysis from the specified directory.' + description: 'Whether to enable automated interactivity.' + name: opt_automated_interactivity + defaultValue: false + required: false + type: boolean + hidden: true + predefined: + - true + - false + - auto: PREDEFINED + description: 'The directory from which to start the file analysis.' name: obj_ext_startfolder defaultValue: 'temp' required: false @@ -348,18 +455,26 @@ script: name: obj_ext_cmd required: false type: String - - description: 'Run file with superuser privileges.' + - auto: PREDEFINED + description: 'Whether to run the file with superuser privileges.' name: run_as_root - defaultValue: 'True' + defaultValue: true required: false - type: String - - description: 'Change extension to valid.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to change the extension to a valid one.' name: obj_ext_extension - defaultValue: 'True' + defaultValue: true required: false - type: String + type: boolean + predefined: + - true + - false name: anyrun-detonate-file-linux - description: 'Perform File analysis using Linux VM.' + description: 'Performs file analysis using a Linux VM.' outputs: - contextPath: ANYRUN_DetonateFileLinux.TaskID description: 'Task UUID.' @@ -383,37 +498,57 @@ script: predefined: - 'ubuntu' - 'debian' - - description: 'Network connection state.' + - auto: PREDEFINED + description: 'Whether to enable network connection.' name: opt_network_connect - defaultValue: 'True' + defaultValue: true required: false - type: String - - description: 'FakeNet feature status.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to enable the FakeNet feature.' name: opt_network_fakenet - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'TOR using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use TOR.' name: opt_network_tor - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Tor geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The TOR geolocation option, for example, US or AU.' name: opt_network_geo defaultValue: 'fastest' required: false type: String - - description: 'HTTPS MITM proxy option.' + - auto: PREDEFINED + description: 'Whether to use the HTTPS MITM proxy.' name: opt_network_mitm - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use a residential proxy.' name: opt_network_residential_proxy - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The residential proxy geolocation option, for example, US or AU.' name: opt_network_residential_proxy_geo defaultValue: 'fastest' required: false @@ -434,6 +569,16 @@ script: defaultValue: '120' required: false type: String + - auto: PREDEFINED + description: 'Whether to enable automated interactivity.' + name: opt_automated_interactivity + defaultValue: false + required: false + type: boolean + hidden: true + predefined: + - true + - false - auto: PREDEFINED description: 'Browser name.' name: obj_ext_browser @@ -443,13 +588,17 @@ script: predefined: - 'Google Chrome' - 'Mozilla Firefox' - - description: 'Change extension to valid.' + - auto: PREDEFINED + description: 'Whether to change the extension to a valid one.' name: obj_ext_extension - defaultValue: 'True' + defaultValue: true required: false - type: String + type: boolean + predefined: + - true + - false name: anyrun-detonate-url-linux - description: 'Perform URL analysis using Linux VM.' + description: 'Performs URL analysis using a Linux VM.' outputs: - contextPath: ANYRUN_DetonateUrlLinux.TaskID description: 'Task UUID.' @@ -464,37 +613,57 @@ script: defaultValue: 'en-US' required: false type: String - - description: 'Network connection state.' + - auto: PREDEFINED + description: 'Whether to enable network connection.' name: opt_network_connect - defaultValue: 'True' + defaultValue: true required: false - type: String - - description: 'FakeNet feature status.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to enable the FakeNet feature.' name: opt_network_fakenet - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'TOR using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use TOR.' name: opt_network_tor - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Tor geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The TOR geolocation option, for example, US or AU.' name: opt_network_geo defaultValue: 'fastest' required: false type: String - - description: 'HTTPS MITM proxy option.' + - auto: PREDEFINED + description: 'Whether to use the HTTPS MITM proxy.' name: opt_network_mitm - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use a residential proxy.' name: opt_network_residential_proxy - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The residential proxy geolocation option, for example, US or AU.' name: opt_network_residential_proxy_geo defaultValue: 'fastest' required: false @@ -515,12 +684,22 @@ script: defaultValue: '120' required: false type: String + - auto: PREDEFINED + description: 'Whether to enable automated interactivity.' + name: opt_automated_interactivity + defaultValue: false + required: false + type: boolean + hidden: true + predefined: + - true + - false - description: 'Optional command line.' name: obj_ext_cmd required: false type: String name: anyrun-detonate-file-android - description: 'Perform File analysis using Android VM.' + description: 'Performs file analysis using an Android VM.' outputs: - contextPath: ANYRUN_DetonateFileAndroid.TaskID description: 'Task UUID.' @@ -535,37 +714,57 @@ script: defaultValue: 'en-US' required: false type: String - - description: 'Network connection state.' + - auto: PREDEFINED + description: 'Whether to enable network connection.' name: opt_network_connect - defaultValue: 'True' + defaultValue: true required: false - type: String - - description: 'FakeNet feature status.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to enable the FakeNet feature.' name: opt_network_fakenet - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'TOR using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use TOR.' name: opt_network_tor - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Tor geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The TOR geolocation option, for example, US or AU.' name: opt_network_geo defaultValue: 'fastest' required: false type: String - - description: 'HTTPS MITM proxy option.' + - auto: PREDEFINED + description: 'Whether to use the HTTPS MITM proxy.' name: opt_network_mitm - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy using.' + type: boolean + predefined: + - true + - false + - auto: PREDEFINED + description: 'Whether to use a residential proxy.' name: opt_network_residential_proxy - defaultValue: 'False' + defaultValue: false required: false - type: String - - description: 'Residential proxy geo location option. Example: US, AU.' + type: boolean + predefined: + - true + - false + - description: 'The residential proxy geolocation option, for example, US or AU.' name: opt_network_residential_proxy_geo defaultValue: 'fastest' required: false @@ -586,17 +785,28 @@ script: defaultValue: '120' required: false type: String + - auto: PREDEFINED + description: 'Whether to enable automated interactivity.' + name: opt_automated_interactivity + defaultValue: false + required: false + type: boolean + hidden: true + predefined: + - true + - false - auto: PREDEFINED description: 'Browser name.' name: obj_ext_browser defaultValue: 'Google Chrome' required: false type: String + hidden: true predefined: - 'Google Chrome' - 'Mozilla Firefox' name: anyrun-detonate-url-android - description: 'Perform URL analysis using Android VM.' + description: 'Performs a URL analysis using an Android VM.' outputs: - contextPath: ANYRUN_DetonateUrlAndroid.TaskID description: 'Task UUID.' @@ -635,12 +845,16 @@ script: description: 'Defines limits for parallel runs.' type: String - arguments: - - description: 'Leave this field blank to get your history or specify to get team history.' + - auto: PREDEFINED + description: 'Whether to get team history (true) or your own history (false).' name: team - defaultValue: 'False.' + defaultValue: false required: false - type: String - - description: 'Skip the specified number of tasks.' + type: boolean + predefined: + - true + - false + - description: 'The number of tasks to skip.' name: skip defaultValue: '0' required: false @@ -717,6 +931,15 @@ script: - 'summary' - 'html' - 'ioc' + - auto: PREDEFINED + description: 'Whether to also build Cortex XDR IOC objects when report_format is ioc.' + name: create_xdr_iocs + defaultValue: false + required: false + type: boolean + predefined: + - true + - false name: anyrun-get-analysis-report description: 'Returns the analysis report summary.' outputs: @@ -1450,7 +1673,7 @@ script: - contextPath: ANYRUN.SandboxAnalysisReportVerdict description: "The analysis verdict." type: String - dockerimage: demisto/anyrun-sdk:1.0.0.10120494 + dockerimage: demisto/anyrun-sdk:1.0.0.12844876 subtype: python3 runonce: false script: '-' diff --git a/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox_description.md b/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox_description.md index 5807f455869e..8bf248ecd89a 100644 --- a/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox_description.md +++ b/Packs/ANYRUN/Integrations/AnyRunSandbox/AnyRunSandbox_description.md @@ -13,9 +13,17 @@ API-KEY from your ANY.RUN account is required. 3. Click **Add instance** to create and configure a new integration instance. 4. Insert ANY.RUN API-KEY into the **Password** parameter. 5. Please use "ANY.RUN" as username. -6Click **Test** to validate the URLs, token, and connection. +6. Click **Test** to validate the URLs, token, and connection. -| **Parameter** | **Description** | **Required** | -|------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------| --- | -| Password | ANY.RUN API-KEY without prefix | True | | True | +| **Parameter** | **Description** +|------------------|--------------------------------------------------------------------------| +| Password | ANY.RUN API-KEY without prefix +| Base domain | The base domain for the ANY.RUN connection. By default, the domain is any.run. + +### Base domain + +| **Base domain** | **Usage** +|-----------------|------------------------| +| any.run | Default +| anyrun.us | Special access required \ No newline at end of file diff --git a/Packs/ANYRUN/Integrations/AnyRunSandbox/README.md b/Packs/ANYRUN/Integrations/AnyRunSandbox/README.md index 6eab5c733e07..d3f4eed332cd 100644 --- a/Packs/ANYRUN/Integrations/AnyRunSandbox/README.md +++ b/Packs/ANYRUN/Integrations/AnyRunSandbox/README.md @@ -39,6 +39,7 @@ Integration is officially supported from XSOAR 8.x | **Parameter** | **Description** | **Required** | |-----------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------| --- | | Password | ANY.RUN API-KEY without prefix. | True | +| Base domain | Base domain for the ANY.RUN connection. By default, the domain is any.run. | False | ## Commands @@ -246,7 +247,6 @@ Perform URL analysis using Android VM. | opt_network_residential_proxy_geo | Residential proxy geo location option. Example: US, AU. Default is fastest. | Optional | | opt_privacy_type | Privacy settings. Possible values are: public, bylink, owner, byteam. Default is bylink. | Optional | | opt_timeout | Timeout option. Size range: 10-660. Default is 120. | Optional | -| obj_ext_browser | Browser name. Possible values are: Google Chrome, Mozilla Firefox. Default is Google Chrome. | Optional | #### Context Output diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android.yml index 50edb10b900e..de4446c01717 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android.yml +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android.yml @@ -20,7 +20,7 @@ tasks: description: "" nexttasks: '#none#': - - "7" + - "20" separatecontext: false continueonerrortype: "" view: |- @@ -37,6 +37,168 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + "20": + id: "20" + taskid: bcb7ce0f-60a4-4797-b492-2d83595993d1 + type: condition + task: + id: bcb7ce0f-60a4-4797-b492-2d83595993d1 + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "22" + "yes": + - "21" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 185 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "21": + id: "21" + taskid: a15b376c-8a9c-4ef3-9809-816ed237f8e2 + type: regular + task: + id: a15b376c-8a9c-4ef3-9809-816ed237f8e2 + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "22": + id: "22" + taskid: 699f4715-1d13-4e0e-8c67-c6cebcdbe4ce + type: regular + task: + id: 699f4715-1d13-4e0e-8c67-c6cebcdbe4ce + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "23" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "23": + id: "23" + taskid: 38f45aa6-533f-4810-81b3-fb595877813f + type: regular + task: + id: 38f45aa6-533f-4810-81b3-fb595877813f + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 495 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false "2": id: "2" taskid: 29050f73-a833-4d28-b383-de0e91bf45d5 @@ -54,6 +216,8 @@ tasks: '#none#': - "9" scriptarguments: + using: + simple: ${ANYRUNInstance} env_locale: simple: ${inputs.env_locale} file: @@ -84,7 +248,7 @@ tasks: { "position": { "x": 275, - "y": 560 + "y": 1000 } } note: false @@ -111,6 +275,8 @@ tasks: '#none#': - "10" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: html task_uuid: @@ -121,7 +287,7 @@ tasks: { "position": { "x": 387.5, - "y": 1085 + "y": 1525 } } note: false @@ -149,7 +315,7 @@ tasks: { "position": { "x": 50, - "y": 1785 + "y": 2225 } } note: false @@ -192,7 +358,7 @@ tasks: { "position": { "x": 162.5, - "y": 385 + "y": 825 } } note: false @@ -230,7 +396,7 @@ tasks: { "position": { "x": 50, - "y": 210 + "y": 650 } } note: false @@ -275,7 +441,7 @@ tasks: { "position": { "x": 275, - "y": 910 + "y": 1350 } } note: false @@ -303,6 +469,8 @@ tasks: '#none#': - "8" scriptarguments: + using: + simple: ${ANYRUNInstance} task_uuid: simple: ${ANYRUN_DetonateFileAndroid.TaskID} separatecontext: false @@ -311,7 +479,7 @@ tasks: { "position": { "x": 275, - "y": 735 + "y": 1175 } } note: false @@ -338,6 +506,8 @@ tasks: '#none#': - "11" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: ioc task_uuid: @@ -348,7 +518,7 @@ tasks: { "position": { "x": 387.5, - "y": 1260 + "y": 1700 } } note: false @@ -394,7 +564,7 @@ tasks: { "position": { "x": 387.5, - "y": 1435 + "y": 1875 } } note: false @@ -427,13 +597,17 @@ tasks: accessor: id indicatorsValues: simple: ${ANYRUN.IOCs} + retry-count: + simple: "5" + retry-interval: + simple: "3" separatecontext: false continueonerrortype: "" view: |- { "position": { "x": 630, - "y": 1610 + "y": 2050 } } note: false @@ -451,89 +625,96 @@ view: |- }, "paper": { "dimensions": { - "height": 1795, - "width": 960, + "height": 2235, + "width": 1260, "x": 50, "y": 50 } } } inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: - key: file value: simple: ${File.EntryID} required: false - description: XSOAR Entry ID + description: The existing War Room file EntryID to submit for analysis. playbookInputQuery: - key: env_locale value: simple: en-US required: false - description: 'Operation system language. Use locale identifier or country name (Ex: - "en-US" or "Brazil"). Case insensitive.' + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' playbookInputQuery: - key: opt_network_connect value: - simple: "True" + simple: "true" required: false - description: Network connection state. + description: Whether to enable network connection. playbookInputQuery: - key: opt_network_fakenet value: - simple: "False" + simple: "false" required: false - description: FakeNet feature status. + description: Whether to enable the FakeNet feature. playbookInputQuery: - key: opt_network_tor value: - simple: "False" + simple: "false" required: false - description: TOR using. + description: Whether to use TOR. playbookInputQuery: - key: opt_network_geo value: simple: fastest required: false - description: 'Tor geo location option. Example: US, AU.' + description: 'The TOR geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_network_mitm value: - simple: "False" + simple: "false" required: false - description: HTTPS MITM proxy option. + description: Whether to use the HTTPS MITM proxy. playbookInputQuery: - key: opt_network_residential_proxy value: - simple: "False" + simple: "false" required: false - description: Residential proxy using. + description: Whether to use a residential proxy. playbookInputQuery: - key: opt_network_residential_proxy_geo value: simple: fastest required: false - description: 'Residential proxy geo location option. Example: US, AU.' + description: 'The residential proxy geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_privacy_type value: simple: bylink required: false - description: 'Privacy settings. Supports: public, bylink, owner, byteam.' + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' playbookInputQuery: - key: opt_timeout value: simple: "240" required: false - description: 'Timeout option. Size range: 10-660.' + description: 'The timeout value. Size range: 10-660.' playbookInputQuery: - key: obj_ext_cmd value: {} required: false - description: Optional command line + description: The optional command line. playbookInputQuery: inputSections: - inputs: - file + - Using name: General (Inputs group) description: Generic group for inputs - inputs: @@ -560,7 +741,28 @@ outputSections: - outputs: [] name: General (Outputs group) description: Generic group for outputs -outputs: [] +outputs: +- contextPath: ANYRUN_DetonateFileAndroid.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown quiet: true tests: - No tests (auto formatted) +marketplaces: +- xsoar +- platform +- xpanse +- xsoar_saas +- marketplacev2 +supportedModules: +- agentix +- xsiam +- edr +- cloud +- cloud_runtime_security diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_README.md index 525e32636f33..46edf532894d 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_README.md +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_README.md @@ -14,7 +14,9 @@ This playbook does not use any sub-playbooks. ### Scripts +* GetInstances * IsIntegrationAvailable +* Set * associateIndicatorsToIncident ### Commands @@ -29,23 +31,28 @@ This playbook does not use any sub-playbooks. | **Name** | **Description** | **Default Value** | **Required** | | --- | --- | --- | --- | -| file | XSOAR Entry ID | ${File.EntryID} | Optional | -| env_locale | Operation system language. Use locale identifier or country name \(Ex: "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | -| opt_network_connect | Network connection state. | True | Optional | -| opt_network_fakenet | FakeNet feature status. | False | Optional | -| opt_network_tor | TOR using. | False | Optional | -| opt_network_geo | Tor geo location option. Example: US, AU. | fastest | Optional | -| opt_network_mitm | HTTPS MITM proxy option. | False | Optional | -| opt_network_residential_proxy | Residential proxy using. | False | Optional | -| opt_network_residential_proxy_geo | Residential proxy geo location option. Example: US, AU. | fastest | Optional | -| opt_privacy_type | Privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | -| opt_timeout | Timeout option. Size range: 10-660. | 240 | Optional | -| obj_ext_cmd | Optional command line | | Optional | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| file | The existing War Room file EntryID to submit for analysis. | ${File.EntryID} | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 240 | Optional | +| obj_ext_cmd | The optional command line. | | Optional | ## Playbook Outputs --- -There are no outputs for this playbook. +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateFileAndroid.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | ## Playbook Image diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_for_XDR.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_for_XDR.yml new file mode 100644 index 000000000000..b0f44fdf6f88 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_for_XDR.yml @@ -0,0 +1,809 @@ +id: ANYRUN Detonate File Android for XDR +version: -1 +fromversion: 6.0.0 +name: ANYRUN Detonate File Android for XDR +description: Submits a file from a Cortex XDR issue to the ANY.RUN Cloud Sandbox for dynamic analysis in an Android environment and automates malware detonation and behavior observation on Android OS. +starttaskid: "0" +tasks: + "0": + id: "0" + taskid: 3fe739c2-23a1-42fb-8396-8731168bb188 + type: start + task: + id: 3fe739c2-23a1-42fb-8396-8731168bb188 + version: -1 + name: "" + iscommand: false + brand: "" + description: "" + nexttasks: + '#none#': + - "40" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 50 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "40": + id: "40" + taskid: 65da9c41-4517-43e7-bfdd-c9a6f5a98c1f + type: condition + task: + id: 65da9c41-4517-43e7-bfdd-c9a6f5a98c1f + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "42" + "yes": + - "41" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 220 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "41": + id: "41" + taskid: a236f1c0-e778-4264-9be0-a850b47c365a + type: regular + task: + id: a236f1c0-e778-4264-9be0-a850b47c365a + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "42": + id: "42" + taskid: ec526ae5-134e-421e-8320-c47257f4083f + type: regular + task: + id: ec526ae5-134e-421e-8320-c47257f4083f + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "43" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "43": + id: "43" + taskid: 11f737eb-4bf7-40bf-88a6-dc074886a093 + type: regular + task: + id: 11f737eb-4bf7-40bf-88a6-dc074886a093 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 590 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "2": + id: "2" + taskid: 6b554c8f-a691-4cb4-8192-9c298f0c1fc8 + type: regular + task: + id: 6b554c8f-a691-4cb4-8192-9c298f0c1fc8 + version: -1 + name: '[ANY.RUN] Detonate File Android' + description: Performs file analysis using an Android VM in ANY.RUN Cloud Sandbox. + script: '|||anyrun-detonate-file-android' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "9" + scriptarguments: + using: + simple: ${ANYRUNInstance} + env_locale: + simple: ${inputs.env_locale} + file: + simple: ${inputs.file_entry_id} + obj_ext_cmd: + simple: ${inputs.obj_ext_cmd} + opt_network_connect: + simple: ${inputs.opt_network_connect} + opt_network_fakenet: + simple: ${inputs.opt_network_fakenet} + opt_network_geo: + simple: ${inputs.opt_network_geo} + opt_network_mitm: + simple: ${inputs.opt_network_mitm} + opt_network_residential_proxy: + simple: ${inputs.opt_network_residential_proxy} + opt_network_residential_proxy_geo: + simple: ${inputs.opt_network_residential_proxy_geo} + opt_network_tor: + simple: ${inputs.opt_network_tor} + opt_privacy_type: + simple: ${inputs.opt_privacy_type} + opt_timeout: + simple: ${inputs.opt_timeout} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1100 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "3": + id: "3" + taskid: f21f8d77-be2f-4dc4-80f5-dddbb0720d6c + type: regular + task: + id: f21f8d77-be2f-4dc4-80f5-dddbb0720d6c + version: -1 + name: '[ANY.RUN] Get analysis summary' + description: Returns the analysis report summary. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "10" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: html + task_uuid: + simple: ${ANYRUN_DetonateFileAndroid.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 1655 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "5": + id: "5" + taskid: a6d40dc3-ccce-4d4d-8f30-6393afc17935 + type: title + task: + id: a6d40dc3-ccce-4d4d-8f30-6393afc17935 + version: -1 + name: Done + type: title + iscommand: false + brand: "" + description: "" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 80, + "y": 2580 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "6": + id: "6" + taskid: a5d3003b-562a-4e95-8858-8d82dcbda09d + type: condition + task: + id: a5d3003b-562a-4e95-8858-8d82dcbda09d + version: -1 + name: Is there any File EntryID to detonate? + description: Checks whether a War Room file EntryID was provided. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "2" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.file_entry_id + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 162.5, + "y": 915 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "7": + id: "7" + taskid: ec34746f-8722-40fe-878c-58c7ec28cd0a + type: condition + task: + id: ec34746f-8722-40fe-878c-58c7ec28cd0a + version: -1 + name: Is there an active ANY.RUN instance? + description: Returns yes if the integration brand is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "6" + scriptarguments: + brandname: + simple: ANY.RUN Cloud Sandbox + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 730 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "8": + id: "8" + taskid: 96c2eacd-6290-4d96-8f21-97cbad7ac85b + type: condition + task: + id: 96c2eacd-6290-4d96-8f21-97cbad7ac85b + version: -1 + name: Was a malicious or suspicious verdict received? + description: Checks whether the argument values are equal. If either of the arguments is missing a value, no is returned. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "3" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEqualString + left: + value: + simple: ANYRUN.SandboxAnalysisReportVerdict + iscontext: true + right: + value: + simple: No threats detected + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1470 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "9": + id: "9" + taskid: 944ac489-9f60-4881-89ec-5da70f167ab1 + type: regular + task: + id: 944ac489-9f60-4881-89ec-5da70f167ab1 + version: -1 + name: '[ANY.RUN] Get analysis verdict' + description: 'Returns a threat level text. Possible values are No threats detected, Suspicious activity, or Malicious activity.' + script: '|||anyrun-get-analysis-verdict' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "8" + scriptarguments: + using: + simple: ${ANYRUNInstance} + task_uuid: + simple: ${ANYRUN_DetonateFileAndroid.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1285 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "10": + id: "10" + taskid: 718ee57c-0c82-421a-8ea0-256e3890ba59 + type: regular + task: + id: 718ee57c-0c82-421a-8ea0-256e3890ba59 + version: -1 + name: '[ANY.RUN] Load IOCs to Threat Intel' + description: Returns the analysis report summary. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "14" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: ioc + create_xdr_iocs: + simple: ${inputs.AutoAddIOCsToXDR} + task_uuid: + simple: ${ANYRUN_DetonateFileAndroid.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 1840 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "13": + id: "13" + taskid: ffa693e3-dc2c-4452-8705-4b74ddd3cf5d + type: regular + task: + id: ffa693e3-dc2c-4452-8705-4b74ddd3cf5d + version: -1 + name: '[XDR] Add ANY.RUN IOCs to XDR IOC rules' + description: Uploads the IOC rules to Cortex XDR. When the ioc_object is defined, + disregard any other provided arguments, as the ioc_object takes precedence. + Validates the indicator parameters when ioc_object is used. If `vendor_name`, + `vendor_reputation`, and `vendor_reliability` are used, only a single vendor + is supported. For multiple vendors, utilize an `ioc_object` in JSON format. + Adding a rule with the same indicator, but with different parameters, will + update the existing rule. + script: Cortex Core - IR|||core-add-indicator-rule + type: regular + iscommand: true + brand: Cortex Core - IR + nexttasks: + '#none#': + - "5" + scriptarguments: + input_format: + simple: JSON + ioc_object: + simple: ${ANYRUN.IOCDetails.ioc_object} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 612.5, + "y": 2395 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "14": + id: "14" + taskid: b7be7c2b-ad87-46bc-8d84-5bede2a22632 + type: condition + task: + id: b7be7c2b-ad87-46bc-8d84-5bede2a22632 + version: -1 + name: Are there ANY.RUN IOC details to add to XDR? + description: Checks whether the ANY.RUN IOC report returned an ioc_object that + can be uploaded to Cortex XDR IOC rules. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "15" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: ANYRUN.IOCDetails.ioc_object + iscontext: true + right: + value: {} + - - operator: isEqualString + left: + value: + simple: inputs.AutoAddIOCsToXDR + iscontext: true + right: + value: + simple: "true" + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 2025 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "15": + id: "15" + taskid: 041bcbfa-99e9-4be4-8cc3-e7c4bfa88b58 + type: condition + task: + id: 041bcbfa-99e9-4be4-8cc3-e7c4bfa88b58 + version: -1 + name: Is Cortex Core - IR available? + description: Returns yes if Cortex Core - IR is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "13" + scriptarguments: + brandname: + simple: Cortex Core - IR + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 500, + "y": 2210 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false +view: |- + { + "linkLabelsPosition": { + "8_5_#default#": 0.33 + }, + "paper": { + "dimensions": { + "height": 2590, + "width": 942.5, + "x": 50, + "y": 50 + } + } + } +inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: null +- key: file_entry_id + value: {} + required: false + description: The existing War Room file EntryID to submit to ANY.RUN Android analysis. + playbookInputQuery: null +- key: env_locale + value: + simple: en-US + required: false + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' + playbookInputQuery: null +- key: opt_network_connect + value: + simple: "true" + required: false + description: Whether to enable network connection. + playbookInputQuery: null +- key: opt_network_fakenet + value: + simple: "false" + required: false + description: Whether to enable the FakeNet feature. + playbookInputQuery: null +- key: opt_network_tor + value: + simple: "false" + required: false + description: Whether to use TOR. + playbookInputQuery: null +- key: opt_network_geo + value: + simple: fastest + required: false + description: 'The TOR geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_network_mitm + value: + simple: "false" + required: false + description: Whether to use the HTTPS MITM proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy + value: + simple: "false" + required: false + description: Whether to use a residential proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy_geo + value: + simple: fastest + required: false + description: 'The residential proxy geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_privacy_type + value: + simple: bylink + required: false + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' + playbookInputQuery: null +- key: opt_timeout + value: + simple: "240" + required: false + description: 'The timeout value. Size range: 10-660.' + playbookInputQuery: null +- key: obj_ext_cmd + value: {} + required: false + description: The optional command line. + playbookInputQuery: null +- key: AutoAddIOCsToXDR + value: + simple: "true" + required: false + description: 'Whether to automatically add malicious and suspicious ANY.RUN IOCs to Cortex XDR IOC rules. IOCs with an Unknown reputation are always excluded. Set to false to disable.' + playbookInputQuery: null +inputSections: +- inputs: + - AutoAddIOCsToXDR + - Using + - file_entry_id + name: Sample and Instance Selection + description: Optionally select the ANY.RUN instance with the Using input and provide an existing War Room file EntryID. Cortex endpoint file retrieval is not used for Android analysis. +- inputs: + - env_locale + name: ANY.RUN Environment + description: "" +- inputs: + - opt_network_connect + - opt_network_fakenet + - opt_network_tor + - opt_network_geo + - opt_network_mitm + - opt_network_residential_proxy + - opt_network_residential_proxy_geo + - opt_privacy_type + - opt_timeout + name: ANY.RUN Options + description: "" +- inputs: + - obj_ext_cmd + name: ANY.RUN Object + description: "" +outputSections: +- outputs: [] + name: General (Outputs group) + description: The generic group for outputs. +outputs: +- contextPath: ANYRUN_DetonateFileAndroid.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown +- contextPath: ANYRUN.IOCDetails + description: The detailed IOC objects prepared for Cortex XDR indicator rules. + type: Unknown +quiet: true +adopted: true +marketplaces: +- platform +- marketplacev2 +supportedModules: +- edr +- xsiam diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_for_XDR_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_for_XDR_README.md new file mode 100644 index 000000000000..15ef39e6b873 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Android_for_XDR_README.md @@ -0,0 +1,60 @@ +This playbook submits a file from a Cortex XDR issue to the ANY.RUN Cloud Sandbox for dynamic analysis in an Android environment and automates malware detonation and behavior observation on Android OS. + +## Dependencies + +This playbook uses the following sub-playbooks, integrations, and scripts. + +### Sub-playbooks + +This playbook does not use any sub-playbooks. + +### Integrations + +* AnyRunSandbox +* Cortex Core - IR + +### Scripts + +* GetInstances +* IsIntegrationAvailable +* Set + +### Commands + +* anyrun-detonate-file-android +* anyrun-get-analysis-report +* anyrun-get-analysis-verdict +* core-add-indicator-rule + +## Playbook Inputs + +--- + +| **Name** | **Description** | **Default Value** | **Required** | +| --- | --- | --- | --- | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| AutoAddIOCsToXDR | Whether to automatically add malicious and suspicious ANY.RUN IOCs (Unknown reputation IOCs are always excluded) to Cortex XDR IOC rules. Set to "false" to disable. | true | Optional | +| file_entry_id | The existing War Room file EntryID to submit to ANY.RUN Android analysis. | | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 240 | Optional | +| obj_ext_cmd | The optional command line. | | Optional | + +## Playbook Outputs + +--- +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateFileAndroid.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | +| ANYRUN.IOCDetails | The detailed IOC objects prepared for Cortex XDR indicator rules. | Unknown | + +![ANYRUN Detonate File Android for XDR](../doc_files/ANYRUN_Detonate_File_Android_for_XDR.png) diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux.yml index e83263bfe974..75612f81dd43 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux.yml +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux.yml @@ -20,7 +20,7 @@ tasks: description: "" nexttasks: '#none#': - - "7" + - "20" separatecontext: false continueonerrortype: "" view: |- @@ -37,6 +37,168 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + "20": + id: "20" + taskid: c7498dac-f323-4751-b8e9-976330367056 + type: condition + task: + id: c7498dac-f323-4751-b8e9-976330367056 + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "22" + "yes": + - "21" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 185 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "21": + id: "21" + taskid: 9a982c26-c18a-4a10-95fe-a479417c6392 + type: regular + task: + id: 9a982c26-c18a-4a10-95fe-a479417c6392 + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "22": + id: "22" + taskid: ceea7cf4-b5dc-45e7-930a-cc14ce21f48a + type: regular + task: + id: ceea7cf4-b5dc-45e7-930a-cc14ce21f48a + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "23" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "23": + id: "23" + taskid: b08b5d0e-dc14-45cc-81df-43218a5d96b6 + type: regular + task: + id: b08b5d0e-dc14-45cc-81df-43218a5d96b6 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 495 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false "2": id: "2" taskid: 7a25c747-44e0-470d-bf56-94cea07030c7 @@ -54,6 +216,8 @@ tasks: '#none#': - "9" scriptarguments: + using: + simple: ${ANYRUNInstance} env_locale: simple: ${inputs.env_locale} env_os: @@ -92,7 +256,7 @@ tasks: { "position": { "x": 275, - "y": 560 + "y": 1000 } } note: false @@ -119,6 +283,8 @@ tasks: '#none#': - "10" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: html task_uuid: @@ -129,7 +295,7 @@ tasks: { "position": { "x": 387.5, - "y": 1085 + "y": 1525 } } note: false @@ -157,7 +323,7 @@ tasks: { "position": { "x": 50, - "y": 1785 + "y": 2225 } } note: false @@ -200,7 +366,7 @@ tasks: { "position": { "x": 162.5, - "y": 385 + "y": 825 } } note: false @@ -238,7 +404,7 @@ tasks: { "position": { "x": 50, - "y": 210 + "y": 650 } } note: false @@ -283,7 +449,7 @@ tasks: { "position": { "x": 275, - "y": 910 + "y": 1350 } } note: false @@ -311,6 +477,8 @@ tasks: '#none#': - "8" scriptarguments: + using: + simple: ${ANYRUNInstance} task_uuid: simple: ${ANYRUN_DetonateFileLinux.TaskID} separatecontext: false @@ -319,7 +487,7 @@ tasks: { "position": { "x": 275, - "y": 735 + "y": 1175 } } note: false @@ -346,6 +514,8 @@ tasks: '#none#': - "11" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: ioc task_uuid: @@ -356,7 +526,7 @@ tasks: { "position": { "x": 387.5, - "y": 1260 + "y": 1700 } } note: false @@ -402,7 +572,7 @@ tasks: { "position": { "x": 387.5, - "y": 1435 + "y": 1875 } } note: false @@ -435,13 +605,17 @@ tasks: accessor: id indicatorsValues: simple: ${ANYRUN.IOCs} + retry-count: + simple: "5" + retry-interval: + simple: "3" separatecontext: false continueonerrortype: "" view: |- { "position": { "x": 700, - "y": 1610 + "y": 2050 } } note: false @@ -459,115 +633,121 @@ view: |- }, "paper": { "dimensions": { - "height": 1795, - "width": 1030, + "height": 2235, + "width": 1330, "x": 50, "y": 50 } } } inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: - key: file value: simple: ${File.EntryID} required: false - description: XSOAR Entry ID + description: The existing War Room file EntryID to submit for analysis. playbookInputQuery: - key: run_as_root value: - simple: "False" + simple: "false" required: false - description: Run file with superuser privileges + description: Whether to run the file with superuser privileges. playbookInputQuery: - key: env_locale value: simple: en-US required: false - description: 'Operation system language. Use locale identifier or country name (Ex: - "en-US" or "Brazil"). Case insensitive.' + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' playbookInputQuery: - key: env_os value: simple: ubuntu required: false - description: Operation system. Possible values - ubuntu, debian + description: 'The operating system. Possible values are ubuntu or debian.' playbookInputQuery: - key: opt_network_connect value: - simple: "True" + simple: "true" required: false - description: Network connection state. + description: Whether to enable network connection. playbookInputQuery: - key: opt_network_fakenet value: - simple: "False" + simple: "false" required: false - description: FakeNet feature status. + description: Whether to enable the FakeNet feature. playbookInputQuery: - key: opt_network_tor value: - simple: "False" + simple: "false" required: false - description: TOR using. + description: Whether to use TOR. playbookInputQuery: - key: opt_network_geo value: simple: fastest required: false - description: 'Tor geo location option. Example: US, AU.' + description: 'The TOR geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_network_mitm value: - simple: "False" + simple: "false" required: false - description: HTTPS MITM proxy option. + description: Whether to use the HTTPS MITM proxy. playbookInputQuery: - key: opt_network_residential_proxy value: - simple: "False" + simple: "false" required: false - description: Residential proxy using. + description: Whether to use a residential proxy. playbookInputQuery: - key: opt_network_residential_proxy_geo value: simple: fastest required: false - description: 'Residential proxy geo location option. Example: US, AU.' + description: 'The residential proxy geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_privacy_type value: simple: bylink required: false - description: 'Privacy settings. Supports: public, bylink, owner, byteam.' + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' playbookInputQuery: - key: opt_timeout value: simple: "240" required: false - description: 'Timeout option. Size range: 10-660.' + description: 'The timeout value. Size range: 10-660.' playbookInputQuery: - key: obj_ext_cmd value: {} required: false - description: Optional command line + description: The optional command line. playbookInputQuery: - key: obj_ext_startfolder value: simple: temp required: false - description: 'Start object from. Supports: desktop, home, downloads, appdata, temp, - windows, root' + description: 'The directory from which to start the file analysis. Possible values are desktop, home, downloads, appdata, temp, windows, or root.' playbookInputQuery: - key: obj_ext_extension value: - simple: "True" + simple: "true" required: false - description: Change extension to valid + description: Whether to change the extension to a valid one. playbookInputQuery: inputSections: - inputs: - file - run_as_root + - Using name: General (Inputs group) description: Generic group for inputs - inputs: @@ -597,7 +777,28 @@ outputSections: - outputs: [] name: General (Outputs group) description: Generic group for outputs -outputs: [] +outputs: +- contextPath: ANYRUN_DetonateFileLinux.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown quiet: true tests: - No tests (auto formatted) +marketplaces: +- xsoar +- platform +- xpanse +- xsoar_saas +- marketplacev2 +supportedModules: +- agentix +- xsiam +- edr +- cloud +- cloud_runtime_security diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_README.md index bf8a0b0c2cbb..b4f900229fef 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_README.md +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_README.md @@ -14,7 +14,9 @@ This playbook does not use any sub-playbooks. ### Scripts +* GetInstances * IsIntegrationAvailable +* Set * associateIndicatorsToIncident ### Commands @@ -29,27 +31,32 @@ This playbook does not use any sub-playbooks. | **Name** | **Description** | **Default Value** | **Required** | | --- | --- | --- | --- | -| file | XSOAR Entry ID | ${File.EntryID} | Optional | -| run_as_root | Run file with superuser privileges | False | Optional | -| env_locale | Operation system language. Use locale identifier or country name \(Ex: "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | -| env_os | Operation system | ubuntu | Optional | -| opt_network_connect | Network connection state. | True | Optional | -| opt_network_fakenet | FakeNet feature status. | False | Optional | -| opt_network_tor | TOR using. | False | Optional | -| opt_network_geo | Tor geo location option. Example: US, AU. | fastest | Optional | -| opt_network_mitm | HTTPS MITM proxy option. | False | Optional | -| opt_network_residential_proxy | Residential proxy using. | False | Optional | -| opt_network_residential_proxy_geo | Residential proxy geo location option. Example: US, AU. | fastest | Optional | -| opt_privacy_type | Privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | -| opt_timeout | Timeout option. Size range: 10-660. | 240 | Optional | -| obj_ext_cmd | Optional command line | | Optional | -| obj_ext_startfolder | Start object from. Supports: desktop, home, downloads, appdata, temp, windows, root | temp | Optional | -| obj_ext_extension | Change extension to valid | True | Optional | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| file | The existing War Room file EntryID to submit for analysis. | ${File.EntryID} | Optional | +| run_as_root | Whether to run the file with superuser privileges. | False | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| env_os | The operating system. Possible values: ubuntu, debian. | ubuntu | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 240 | Optional | +| obj_ext_cmd | The optional command line. | | Optional | +| obj_ext_startfolder | The directory from which to start the file analysis. Supports: desktop, home, downloads, appdata, temp, windows, root. | temp | Optional | +| obj_ext_extension | Whether to change the extension to a valid one. | True | Optional | ## Playbook Outputs --- -There are no outputs for this playbook. +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateFileLinux.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | ## Playbook Image diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_for_XDR.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_for_XDR.yml new file mode 100644 index 000000000000..f84996b3bc80 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_for_XDR.yml @@ -0,0 +1,1359 @@ +id: ANYRUN Detonate File Linux for XDR +version: -1 +fromversion: 6.0.0 +name: ANYRUN Detonate File Linux for XDR +description: Submits a file retrieved from a Cortex XDR endpoint to the ANY.RUN Cloud Sandbox for dynamic analysis in a Linux environment and automates malware detonation and behavior observation on Linux OS. +starttaskid: "0" +tasks: + "0": + id: "0" + taskid: e9b17277-f0de-4fec-8db5-cd0c1163e61a + type: start + task: + id: e9b17277-f0de-4fec-8db5-cd0c1163e61a + version: -1 + name: "" + iscommand: false + brand: "" + description: "" + nexttasks: + '#none#': + - "40" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 152.5, + "y": 50 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "40": + id: "40" + taskid: 73c38156-9191-4ecd-9273-bf9f12651c90 + type: condition + task: + id: 73c38156-9191-4ecd-9273-bf9f12651c90 + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "42" + "yes": + - "41" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 220 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "41": + id: "41" + taskid: c0c8106b-3844-4b52-b6fb-507550dbffc9 + type: regular + task: + id: c0c8106b-3844-4b52-b6fb-507550dbffc9 + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "42": + id: "42" + taskid: 4c23db59-2b68-4731-9b3d-98d8a686175a + type: regular + task: + id: 4c23db59-2b68-4731-9b3d-98d8a686175a + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "43" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "43": + id: "43" + taskid: 6c83b3a1-73ec-4ff2-b9c5-ac99462680b1 + type: regular + task: + id: 6c83b3a1-73ec-4ff2-b9c5-ac99462680b1 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 590 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "2": + id: "2" + taskid: 65de8af9-f2b1-4d0f-8c5d-f5f20aa54fda + type: regular + task: + id: 65de8af9-f2b1-4d0f-8c5d-f5f20aa54fda + version: -1 + name: '[ANY.RUN] Detonate File Linux' + description: Performs file analysis using a Linux VM in ANY.RUN Cloud Sandbox. + script: '|||anyrun-detonate-file-linux' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "9" + scriptarguments: + using: + simple: ${ANYRUNInstance} + env_locale: + simple: ${inputs.env_locale} + env_os: + simple: ${inputs.env_os} + file: + simple: ${SelectedOriginalFile.EntryID} + obj_ext_cmd: + simple: ${inputs.obj_ext_cmd} + obj_ext_extension: + simple: ${inputs.obj_ext_extension} + obj_ext_startfolder: + simple: ${inputs.obj_ext_startfolder} + opt_network_connect: + simple: ${inputs.opt_network_connect} + opt_network_fakenet: + simple: ${inputs.opt_network_fakenet} + opt_network_geo: + simple: ${inputs.opt_network_geo} + opt_network_mitm: + simple: ${inputs.opt_network_mitm} + opt_network_residential_proxy: + simple: ${inputs.opt_network_residential_proxy} + opt_network_residential_proxy_geo: + simple: ${inputs.opt_network_residential_proxy_geo} + opt_network_tor: + simple: ${inputs.opt_network_tor} + opt_privacy_type: + simple: ${inputs.opt_privacy_type} + opt_timeout: + simple: ${inputs.opt_timeout} + run_as_root: + simple: ${inputs.run_as_root} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 602.5, + "y": 2950 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "3": + id: "3" + taskid: a1b84003-ab1b-48af-86b3-d15b240dcd28 + type: regular + task: + id: a1b84003-ab1b-48af-86b3-d15b240dcd28 + version: -1 + name: '[ANY.RUN] Get analysis summary' + description: Returns the ANY.RUN analysis report in HTML format. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "10" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: html + task_uuid: + simple: ${ANYRUN_DetonateFileLinux.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 715, + "y": 3505 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "5": + id: "5" + taskid: 407e75d8-69ff-4a35-83f5-49b06c30f716 + type: title + task: + id: 407e75d8-69ff-4a35-83f5-49b06c30f716 + version: -1 + name: Done + type: title + iscommand: false + brand: "" + description: "" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 182.5, + "y": 4430 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "7": + id: "7" + taskid: 2725d55e-a7cf-4dda-8dac-386d8956cde1 + type: condition + task: + id: 2725d55e-a7cf-4dda-8dac-386d8956cde1 + version: -1 + name: Is there an active ANY.RUN instance? + description: Returns yes if the ANY.RUN Cloud Sandbox integration is available. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "36" + scriptarguments: + brandname: + simple: ANY.RUN Cloud Sandbox + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 152.5, + "y": 730 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "8": + id: "8" + taskid: b920a4c2-a219-4641-89aa-2a188f870831 + type: condition + task: + id: b920a4c2-a219-4641-89aa-2a188f870831 + version: -1 + name: Was a malicious or suspicious verdict received? + description: Checks whether the ANY.RUN verdict is not equal to No threats detected. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "3" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEqualString + left: + value: + simple: ANYRUN.SandboxAnalysisReportVerdict + iscontext: true + right: + value: + simple: No threats detected + continueonerrortype: "" + view: |- + { + "position": { + "x": 602.5, + "y": 3320 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "9": + id: "9" + taskid: e36d01ac-20cd-439b-84a3-1ec9650384f9 + type: regular + task: + id: e36d01ac-20cd-439b-84a3-1ec9650384f9 + version: -1 + name: '[ANY.RUN] Get analysis verdict' + description: 'Returns a threat level text. Possible values are No threats detected, Suspicious activity, or Malicious activity.' + script: '|||anyrun-get-analysis-verdict' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "8" + scriptarguments: + using: + simple: ${ANYRUNInstance} + task_uuid: + simple: ${ANYRUN_DetonateFileLinux.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 602.5, + "y": 3135 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "10": + id: "10" + taskid: 1950e00e-5296-46ec-8ce3-1f6327458982 + type: regular + task: + id: 1950e00e-5296-46ec-8ce3-1f6327458982 + version: -1 + name: '[ANY.RUN] Load IOCs to Threat Intel' + description: Returns the ANY.RUN analysis report in IOC format and loads the IOCs into the context. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "25" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: ioc + create_xdr_iocs: + simple: ${inputs.AutoAddIOCsToXDR} + task_uuid: + simple: ${ANYRUN_DetonateFileLinux.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 715, + "y": 3690 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "13": + id: "13" + taskid: 3ffbc2a9-2fe8-4233-8325-9615f56264e4 + type: condition + task: + id: 3ffbc2a9-2fe8-4233-8325-9615f56264e4 + version: -1 + name: Are endpoint ID and file path provided for endpoint retrieval? + description: Checks whether the minimum inputs are available to retrieve the + file from a Cortex XDR endpoint when no explicit file EntryID was provided. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "38" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.endpoint_id + iscontext: true + right: + value: {} + - - operator: isNotEmpty + left: + value: + simple: inputs.file_path + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 1100 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "14": + id: "14" + taskid: cc61dc69-ce9f-4b80-8528-a418508ceba1 + type: regular + task: + id: cc61dc69-ce9f-4b80-8528-a418508ceba1 + version: -1 + name: '[XDR] Retrieve file from endpoint' + description: Starts a Cortex Core IR file retrieval action for the requested + Linux file path. The task uses built-in polling arguments, so a separate core-action-status-get + task is not required. + script: '|||core-retrieve-files' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "15" + scriptarguments: + endpoint_ids: + simple: ${inputs.endpoint_id} + interval_in_seconds: + simple: ${inputs.retrieve_poll_interval_seconds} + linux_file_paths: + simple: ${inputs.file_path} + timeout_in_seconds: + simple: ${inputs.retrieve_timeout_seconds} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1470 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "15": + id: "15" + taskid: 34529cc7-27a4-48be-89a4-ab07a8568ccd + type: regular + task: + id: 34529cc7-27a4-48be-89a4-ab07a8568ccd + version: -1 + name: '[XDR] Get retrieved file details' + description: Creates a War Room file entry for the archive retrieved from Cortex XDR, using the action ID returned by core-retrieve-files. + script: '|||core-retrieve-file-details' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "26" + scriptarguments: + action_id: + simple: ${Core.RetrievedFiles.action_id} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1655 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "16": + id: "16" + taskid: b55fd135-5e05-44df-85e3-415eb914d202 + type: regular + task: + id: b55fd135-5e05-44df-85e3-415eb914d202 + version: -1 + name: '[XDR] Unzip retrieved archive' + description: Unzips the retrieved archive and uploads extracted files to the War Room as file entries. + scriptName: UnzipFile + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "17" + scriptarguments: + entryID: + simple: ${ANYRUN.RetrievedArchive.EntryID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 2025 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "17": + id: "17" + taskid: b8a32590-4653-410d-8419-cf1545e173df + type: regular + task: + id: b8a32590-4653-410d-8419-cf1545e173df + version: -1 + name: '[XDR] Extract requested file name from path' + description: Extracts the basename from the file_path playbook input and saves it to the ANYRUN.Input.FileName context key. + The playbook uses the filename to select the extracted original sample. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "18" + scriptarguments: + append: + simple: "false" + key: + simple: ANYRUN.Input.FileName + value: + complex: + root: inputs.file_path + transformers: + - operator: FirstArrayElement + - operator: RegexExtractAll + args: + regex: + value: + simple: '[^\\/]+$' + - operator: FirstArrayElement + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 2210 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "18": + id: "18" + taskid: f259122f-cc3c-4c96-812c-d0ece79be17d + type: regular + task: + id: f259122f-cc3c-4c96-812c-d0ece79be17d + version: -1 + name: '[XDR] Select original file EntryID by file name' + description: Selects the extracted original sample by matching File.Name to + the basename extracted from the file_path playbook input, then stores its EntryID in SelectedOriginalFile.EntryID. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "19" + scriptarguments: + append: + simple: "false" + key: + simple: SelectedOriginalFile.EntryID + value: + complex: + root: File + filters: + - - operator: isEqualString + left: + value: + simple: File.Name + iscontext: true + right: + value: + simple: ANYRUN.Input.FileName + iscontext: true + accessor: EntryID + transformers: + - operator: LastArrayElement + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 2395 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "19": + id: "19" + taskid: a8c73f4c-3c33-4415-8afb-8087a3cf8f9a + type: regular + task: + id: a8c73f4c-3c33-4415-8afb-8087a3cf8f9a + version: -1 + name: '[XDR] Store selected file name' + description: Stores the selected file name for visibility and downstream reporting. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "20" + scriptarguments: + append: + simple: "false" + key: + simple: SelectedOriginalFile.Name + value: + complex: + root: File + filters: + - - operator: isEqualString + left: + value: + simple: File.Name + iscontext: true + right: + value: + simple: ANYRUN.Input.FileName + iscontext: true + accessor: Name + transformers: + - operator: LastArrayElement + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 2580 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "20": + id: "20" + taskid: 546a99ae-d374-4765-870a-4ccc71256f83 + type: condition + task: + id: 546a99ae-d374-4765-870a-4ccc71256f83 + version: -1 + name: Was the original file selected? + description: Checks whether SelectedOriginalFile.EntryID exists before submitting the file to ANY.RUN. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "2" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: SelectedOriginalFile.EntryID + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 490, + "y": 2765 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "22": + id: "22" + taskid: 40a5759c-e6af-43d1-8474-bd1bb3680796 + type: regular + task: + id: 40a5759c-e6af-43d1-8474-bd1bb3680796 + version: -1 + name: '[XDR] Add ANY.RUN IOCs to XDR IOC rules' + description: Uploads the IOC rules to Cortex XDR. When the ioc_object is defined, disregard any other provided arguments, as the ioc_object takes precedence. Validates the indicator parameters when ioc_object is used. If `vendor_name`, `vendor_reputation`, and `vendor_reliability` are used, only a single vendor is supported. For multiple vendors, utilize an `ioc_object` in JSON format. Adding a rule with the same indicator, but with different parameters, will update the existing rule. + script: Cortex Core - IR|||core-add-indicator-rule + type: regular + iscommand: true + brand: Cortex Core - IR + nexttasks: + '#none#': + - "5" + scriptarguments: + input_format: + simple: JSON + ioc_object: + simple: ${ANYRUN.IOCDetails.ioc_object} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 940, + "y": 4245 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "25": + id: "25" + taskid: ecc1f907-b493-495c-80ea-4210cf5225c8 + type: condition + task: + id: ecc1f907-b493-495c-80ea-4210cf5225c8 + version: -1 + name: Are there ANY.RUN IOC details to add to XDR? + description: Checks whether ANY.RUN returned IOCDetails.ioc_object before calling + core-add-indicator-rule. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "39" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: ANYRUN.IOCDetails.ioc_object + iscontext: true + right: + value: {} + - - operator: isEqualString + left: + value: + simple: inputs.AutoAddIOCsToXDR + iscontext: true + right: + value: + simple: "true" + continueonerrortype: "" + view: |- + { + "position": { + "x": 715, + "y": 3875 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "26": + id: "26" + taskid: 567802c5-ccee-471e-8059-08de2fa57f19 + type: regular + task: + id: 567802c5-ccee-471e-8059-08de2fa57f19 + version: -1 + name: '[XDR] Store retrieved archive EntryID' + description: Stores only the EntryID of the newly retrieved archive. This prevents UnzipFile from iterating over every File.EntryID already present in the incident context. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "16" + scriptarguments: + append: + simple: "false" + key: + simple: ANYRUN.RetrievedArchive.EntryID + value: + complex: + root: File + accessor: EntryID + transformers: + - operator: LastArrayElement + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1840 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "36": + id: "36" + taskid: effa26e4-7a94-4876-8505-89fcdc49001f + type: condition + task: + id: effa26e4-7a94-4876-8505-89fcdc49001f + version: -1 + name: Was an existing file EntryID provided? + description: Checks whether an explicit War Room file EntryID was provided. If yes, endpoint retrieval is skipped. If not, the playbook attempts Cortex XDR endpoint retrieval using the endpoint_id and file_path inputs. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "13" + "yes": + - "37" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.file_entry_id + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 265, + "y": 915 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "37": + id: "37" + taskid: 720c49ec-d837-4281-8756-be83cc4bd1c3 + type: regular + task: + id: 720c49ec-d837-4281-8756-be83cc4bd1c3 + version: -1 + name: '[Context] Use provided file EntryID' + description: Stores the explicitly provided War Room file EntryID as SelectedOriginalFile.EntryID. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "20" + scriptarguments: + append: + simple: "false" + key: + simple: SelectedOriginalFile.EntryID + value: + simple: ${inputs.file_entry_id} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 705, + "y": 2580 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "38": + id: "38" + taskid: 289ed81a-2210-425e-8d6f-1158ffab281c + type: condition + task: + id: 289ed81a-2210-425e-8d6f-1158ffab281c + version: -1 + name: Is Cortex Core - IR available? + description: Returns yes if Cortex Core - IR is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "14" + scriptarguments: + brandname: + simple: Cortex Core - IR + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 162.5, + "y": 1285 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "39": + id: "39" + taskid: d2097d35-ff7c-403b-8875-f073c8829a34 + type: condition + task: + id: d2097d35-ff7c-403b-8875-f073c8829a34 + version: -1 + name: Is Cortex Core - IR available? + description: Returns yes if Cortex Core - IR is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "22" + scriptarguments: + brandname: + simple: Cortex Core - IR + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 827.5, + "y": 4060 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false +view: |- + { + "linkLabelsPosition": {}, + "paper": { + "dimensions": { + "height": 4440, + "width": 1270, + "x": 50, + "y": 50 + } + } + } +inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: null +- key: file_entry_id + value: {} + required: false + description: The existing War Room file EntryID. If provided, endpoint retrieval is skipped and this file is submitted to ANY.RUN. + playbookInputQuery: null +- key: endpoint_id + value: + simple: ${issue.xdmsourceagentidentifier} + required: false + description: The ID of the Cortex XDR endpoint or agent. Used only when file_entry_id is not provided. + playbookInputQuery: null +- key: file_path + value: + simple: ${issue.initiatorpath} + required: false + description: The full Linux path to the file on the endpoint. Used for retrieval and filename matching when file_entry_id is not provided. + playbookInputQuery: null +- key: retrieve_timeout_seconds + value: + simple: "240" + required: false + description: The time, in seconds, after which polling for the Cortex XDR file retrieval stops. + playbookInputQuery: null +- key: retrieve_poll_interval_seconds + value: + simple: "30" + required: false + description: The interval, in seconds, between status checks of the Cortex XDR file retrieval action. + playbookInputQuery: null +- key: env_locale + value: + simple: en-US + required: false + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' + playbookInputQuery: null +- key: env_os + value: + simple: ubuntu + required: false + description: 'The operating system. Possible values are ubuntu or debian.' + playbookInputQuery: null +- key: run_as_root + value: + simple: "false" + required: false + description: Whether to run the file with superuser privileges. + playbookInputQuery: null +- key: opt_network_connect + value: + simple: "true" + required: false + description: Whether to enable network connection. + playbookInputQuery: null +- key: opt_network_fakenet + value: + simple: "false" + required: false + description: Whether to enable the FakeNet feature. + playbookInputQuery: null +- key: opt_network_tor + value: + simple: "false" + required: false + description: Whether to use TOR. + playbookInputQuery: null +- key: opt_network_geo + value: + simple: fastest + required: false + description: 'The TOR geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_network_mitm + value: + simple: "false" + required: false + description: Whether to use the HTTPS MITM proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy + value: + simple: "false" + required: false + description: Whether to use a residential proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy_geo + value: + simple: fastest + required: false + description: 'The residential proxy geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_privacy_type + value: + simple: bylink + required: false + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' + playbookInputQuery: null +- key: opt_timeout + value: + simple: "240" + required: false + description: 'The timeout value. Size range: 10-660.' + playbookInputQuery: null +- key: obj_ext_cmd + value: {} + required: false + description: The optional command line. + playbookInputQuery: null +- key: obj_ext_startfolder + value: + simple: temp + required: false + description: 'The directory from which to start the file analysis. Possible values are desktop, home, downloads, appdata, temp, windows, or root.' + playbookInputQuery: null +- key: obj_ext_extension + value: + simple: "true" + required: false + description: Whether to change the extension to a valid one. + playbookInputQuery: null +- key: AutoAddIOCsToXDR + value: + simple: "true" + required: false + description: 'Whether to automatically add malicious and suspicious ANY.RUN IOCs to Cortex XDR IOC rules. IOCs with an Unknown reputation are always excluded. Set to false to disable.' + playbookInputQuery: null +inputSections: +- inputs: + - AutoAddIOCsToXDR + - Using + - file_entry_id + - endpoint_id + - file_path + - retrieve_timeout_seconds + - retrieve_poll_interval_seconds + name: Sample and Instance Selection + description: Optionally select the ANY.RUN instance with the Using input. If file_entry_id is provided, the playbook uses it directly. Otherwise, it retrieves the file from the Cortex XDR endpoint using endpoint_id and file_path, then selects the extracted file by filename. +- inputs: + - env_locale + - env_os + - run_as_root + name: ANY.RUN Environment + description: "" +- inputs: + - opt_network_connect + - opt_network_fakenet + - opt_network_tor + - opt_network_geo + - opt_network_mitm + - opt_network_residential_proxy + - opt_network_residential_proxy_geo + - opt_privacy_type + - opt_timeout + name: ANY.RUN Options + description: "" +- inputs: + - obj_ext_cmd + - obj_ext_startfolder + - obj_ext_extension + name: ANY.RUN Object + description: "" +outputSections: +- outputs: + - SelectedOriginalFile.EntryID + - SelectedOriginalFile.Name + - ANYRUN_DetonateFileLinux.TaskID + - ANYRUN.SandboxAnalysisReportVerdict + - ANYRUN.IOCs + - ANYRUN.IOCDetails + name: General + description: The main outputs. +outputs: +- contextPath: SelectedOriginalFile.EntryID + description: The War Room EntryID of the selected original file sent to ANY.RUN. + type: String +- contextPath: SelectedOriginalFile.Name + description: The name of the selected original file. + type: String +- contextPath: ANYRUN_DetonateFileLinux.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown +- contextPath: ANYRUN.IOCDetails + description: The detailed IOC objects prepared for Cortex XDR indicator rules. + type: Unknown +quiet: true +adopted: true +marketplaces: +- platform +- marketplacev2 +supportedModules: +- edr +- xsiam diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_for_XDR_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_for_XDR_README.md new file mode 100644 index 000000000000..8950dc8d7b55 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Linux_for_XDR_README.md @@ -0,0 +1,74 @@ +This playbook submits a file retrieved from a Cortex XDR endpoint to the ANY.RUN Cloud Sandbox for dynamic analysis in a Linux environment and automates malware detonation and behavior observation on Linux OS. + +## Dependencies + +This playbook uses the following sub-playbooks, integrations, and scripts. + +### Sub-playbooks + +This playbook does not use any sub-playbooks. + +### Integrations + +* AnyRunSandbox +* Cortex Core - IR + +### Scripts + +* GetInstances +* IsIntegrationAvailable +* Set +* SetAndHandleEmpty +* UnzipFile + +### Commands + +* anyrun-detonate-file-linux +* anyrun-get-analysis-report +* anyrun-get-analysis-verdict +* core-add-indicator-rule +* core-retrieve-file-details +* core-retrieve-files + +## Playbook Inputs + +--- + +| **Name** | **Description** | **Default Value** | **Required** | +| --- | --- | --- | --- | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| AutoAddIOCsToXDR | Whether to automatically add malicious and suspicious ANY.RUN IOCs (Unknown reputation IOCs are always excluded) to Cortex XDR IOC rules. Set to "false" to disable. | true | Optional | +| file_entry_id | The existing War Room file EntryID. If provided, endpoint retrieval is skipped and this file is submitted to ANY.RUN. | | Optional | +| endpoint_id | The Cortex XDR endpoint/agent ID. Used only when file_entry_id is not provided. | ${issue.xdmsourceagentidentifier} | Optional | +| file_path | The full Linux path to the file on the endpoint. Used for retrieval and filename matching when file_entry_id is not provided. | ${issue.initiatorpath} | Optional | +| retrieve_timeout_seconds | The timeout for Cortex XDR file retrieve polling. | 240 | Optional | +| retrieve_poll_interval_seconds | The polling interval for Cortex XDR file retrieve action status checks. | 30 | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| env_os | The operating system. Possible values: ubuntu, debian. | ubuntu | Optional | +| run_as_root | Whether to run the file with superuser privileges. | False | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 240 | Optional | +| obj_ext_cmd | The optional command line. | | Optional | +| obj_ext_startfolder | The directory from which to start the file analysis. Supports: desktop, home, downloads, appdata, temp, windows, root. | temp | Optional | +| obj_ext_extension | Whether to change the extension to a valid one. | True | Optional | + +## Playbook Outputs + +--- +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| SelectedOriginalFile.EntryID | The War Room EntryID of the selected original file sent to ANY.RUN. | String | +| SelectedOriginalFile.Name | The name of the selected original file. | String | +| ANYRUN_DetonateFileLinux.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | +| ANYRUN.IOCDetails | The detailed IOC objects prepared for Cortex XDR indicator rules. | Unknown | + +![ANYRUN Detonate File Linux for XDR](../doc_files/ANYRUN_Detonate_File_Linux_for_XDR.png) diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows.yml index 311347719edd..2917ac168cdd 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows.yml +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows.yml @@ -20,7 +20,7 @@ tasks: description: "" nexttasks: '#none#': - - "7" + - "20" separatecontext: false continueonerrortype: "" view: |- @@ -37,6 +37,168 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + "20": + id: "20" + taskid: fe196060-84e5-406c-bccf-1b25d6000758 + type: condition + task: + id: fe196060-84e5-406c-bccf-1b25d6000758 + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "22" + "yes": + - "21" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 185 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "21": + id: "21" + taskid: 472f8493-b890-4bb6-8cce-71b9d705cc14 + type: regular + task: + id: 472f8493-b890-4bb6-8cce-71b9d705cc14 + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "22": + id: "22" + taskid: 3f0e0342-5c72-4fd5-8b7d-82e864cce987 + type: regular + task: + id: 3f0e0342-5c72-4fd5-8b7d-82e864cce987 + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "23" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "23": + id: "23" + taskid: b09072ea-858a-4cc4-9dae-674b062ce603 + type: regular + task: + id: b09072ea-858a-4cc4-9dae-674b062ce603 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 495 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false "2": id: "2" taskid: 4e720c3e-e7cc-48ab-ae0e-1185dc7de62c @@ -54,6 +216,8 @@ tasks: '#none#': - "9" scriptarguments: + using: + simple: ${ANYRUNInstance} env_bitness: simple: ${inputs.env_bitness} env_locale: @@ -72,6 +236,8 @@ tasks: simple: ${inputs.obj_ext_startfolder} obj_force_elevation: simple: ${inputs.obj_force_elevation} + opt_automated_interactivity: + simple: ${inputs.opt_automated_interactivity} opt_network_connect: simple: ${inputs.opt_network_connect} opt_network_fakenet: @@ -96,7 +262,7 @@ tasks: { "position": { "x": 275, - "y": 560 + "y": 1000 } } note: false @@ -123,6 +289,8 @@ tasks: '#none#': - "10" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: html task_uuid: @@ -133,7 +301,7 @@ tasks: { "position": { "x": 387.5, - "y": 1085 + "y": 1525 } } note: false @@ -161,7 +329,7 @@ tasks: { "position": { "x": 50, - "y": 1785 + "y": 2225 } } note: false @@ -204,7 +372,7 @@ tasks: { "position": { "x": 162.5, - "y": 385 + "y": 825 } } note: false @@ -242,7 +410,7 @@ tasks: { "position": { "x": 50, - "y": 210 + "y": 650 } } note: false @@ -287,7 +455,7 @@ tasks: { "position": { "x": 275, - "y": 910 + "y": 1350 } } note: false @@ -315,6 +483,8 @@ tasks: '#none#': - "8" scriptarguments: + using: + simple: ${ANYRUNInstance} task_uuid: simple: ${ANYRUN_DetonateFileWindows.TaskID} separatecontext: false @@ -323,7 +493,7 @@ tasks: { "position": { "x": 275, - "y": 735 + "y": 1175 } } note: false @@ -350,6 +520,8 @@ tasks: '#none#': - "12" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: ioc task_uuid: @@ -360,7 +532,7 @@ tasks: { "position": { "x": 387.5, - "y": 1260 + "y": 1700 } } note: false @@ -393,13 +565,17 @@ tasks: accessor: id indicatorsValues: simple: ${ANYRUN.IOCs} + retry-count: + simple: "5" + retry-interval: + simple: "3" separatecontext: false continueonerrortype: "" view: |- { "position": { "x": 542.5, - "y": 1610 + "y": 2050 } } note: false @@ -445,7 +621,7 @@ tasks: { "position": { "x": 387.5, - "y": 1435 + "y": 1875 } } note: false @@ -463,128 +639,139 @@ view: |- }, "paper": { "dimensions": { - "height": 1795, - "width": 872.5, + "height": 2235, + "width": 1172.5, "x": 50, "y": 50 } } } inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: - key: file value: simple: ${File.EntryID} required: false - description: XSOAR Entry ID + description: The existing War Room file EntryID to submit for analysis. playbookInputQuery: - key: env_locale value: simple: en-US required: false - description: 'Operation system language. Use locale identifier or country name (Ex: - "en-US" or "Brazil"). Case insensitive.' + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' playbookInputQuery: - key: env_bitness value: simple: "64" required: false - description: Bitness of Operation System. Supports 32, 64 + description: 'The bitness of the operating system. Possible values are 32 or 64.' playbookInputQuery: - key: env_version value: simple: "10" required: false - description: 'Version of OS. Supports: 7, 10, 11' + description: 'The version of the operating system. Possible values are 7, 10, or 11.' playbookInputQuery: - key: env_type value: simple: complete required: false - description: Environment preset type. You can select **development** env for OS - Windows 10 x64. For all other cases, **complete** env is required + description: The environment preset type. You can select the **development** environment for Windows 10 x64. For all other cases, the **complete** environment is required. playbookInputQuery: - key: opt_network_connect value: - simple: "True" + simple: "true" required: false - description: Network connection state. + description: Whether to enable network connection. playbookInputQuery: - key: opt_network_fakenet value: - simple: "False" + simple: "false" required: false - description: FakeNet feature status. + description: Whether to enable the FakeNet feature. playbookInputQuery: - key: opt_network_tor value: - simple: "False" + simple: "false" required: false - description: TOR using. + description: Whether to use TOR. playbookInputQuery: - key: opt_network_geo value: simple: fastest required: false - description: 'Tor geo location option. Example: US, AU.' + description: 'The TOR geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_network_mitm value: - simple: "False" + simple: "false" required: false - description: HTTPS MITM proxy option. + description: Whether to use the HTTPS MITM proxy. playbookInputQuery: - key: opt_network_residential_proxy value: - simple: "False" + simple: "false" required: false - description: Residential proxy using. + description: Whether to use a residential proxy. playbookInputQuery: - key: opt_network_residential_proxy_geo value: simple: fastest required: false - description: 'Residential proxy geo location option. Example: US, AU.' + description: 'The residential proxy geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_privacy_type value: simple: bylink required: false - description: 'Privacy settings. Supports: public, bylink, owner, byteam.' + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' playbookInputQuery: - key: opt_timeout value: simple: "240" required: false - description: 'Timeout option. Size range: 10-660.' + description: 'The timeout value. Size range: 10-660.' + playbookInputQuery: +- key: opt_automated_interactivity + value: + simple: "true" + required: false + description: Whether to enable automated interactivity. playbookInputQuery: - key: obj_ext_cmd value: {} required: false - description: Optional command line + description: The optional command line. playbookInputQuery: - key: obj_ext_startfolder value: simple: temp required: false - description: 'Start object from. Supports: desktop, home, downloads, appdata, temp, windows, root' + description: 'The directory from which to start the file analysis. Possible values are desktop, home, downloads, appdata, temp, windows, or root.' playbookInputQuery: - key: obj_force_elevation value: - simple: "False" + simple: "false" required: false description: |- - Forces the file to execute with elevated privileges and an elevated token - (for PE32, PE32+, PE64 files only) + Whether to force the file to execute with elevated privileges and an elevated token (for PE32, PE32+, PE64 files only). playbookInputQuery: - key: obj_ext_extension value: - simple: "True" + simple: "true" required: false - description: Change extension to valid + description: Whether to change the extension to a valid one. playbookInputQuery: inputSections: - inputs: - file + - Using name: General (Inputs group) description: Generic group for inputs - inputs: @@ -604,6 +791,7 @@ inputSections: - opt_network_residential_proxy_geo - opt_privacy_type - opt_timeout + - opt_automated_interactivity name: Options description: "" - inputs: @@ -617,7 +805,28 @@ outputSections: - outputs: [] name: General (Outputs group) description: Generic group for outputs -outputs: [] +outputs: +- contextPath: ANYRUN_DetonateFileWindows.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown quiet: true tests: - No tests (auto formatted) +marketplaces: +- xsoar +- platform +- xpanse +- xsoar_saas +- marketplacev2 +supportedModules: +- agentix +- xsiam +- edr +- cloud +- cloud_runtime_security diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_README.md index 770b9e5a073c..383b4aafcb1f 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_README.md +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_README.md @@ -14,7 +14,9 @@ This playbook does not use any sub-playbooks. ### Scripts +* GetInstances * IsIntegrationAvailable +* Set * associateIndicatorsToIncident ### Commands @@ -29,29 +31,35 @@ This playbook does not use any sub-playbooks. | **Name** | **Description** | **Default Value** | **Required** | | --- | --- | --- | --- | -| file | XSOAR Entry ID | ${File.EntryID} | Optional | -| env_locale | Operation system language. Use locale identifier or country name \(Ex: "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | -| env_bitness | Bitness of Operation System. Supports 32, 64 | 64 | Optional | -| env_version | Version of OS. Supports: 7, 10, 11 | 10 | Optional | -| env_type | Environment preset type. You can select \*\*development\*\* env for OS Windows 10 x64. For all other cases, \*\*complete\*\* env is required | complete | Optional | -| opt_network_connect | Network connection state. | True | Optional | -| opt_network_fakenet | FakeNet feature status. | False | Optional | -| opt_network_tor | TOR using. | False | Optional | -| opt_network_geo | Tor geo location option. Example: US, AU. | fastest | Optional | -| opt_network_mitm | HTTPS MITM proxy option. | False | Optional | -| opt_network_residential_proxy | Residential proxy using. | False | Optional | -| opt_network_residential_proxy_geo | Residential proxy geo location option. Example: US, AU. | fastest | Optional | -| opt_privacy_type | Privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | -| opt_timeout | Timeout option. Size range: 10-660. | 240 | Optional | -| obj_ext_cmd | Optional command line | | Optional | -| obj_ext_startfolder | Start object from. Supports: desktop, home, downloads, appdata, temp, windows, root | temp | Optional | -| obj_force_elevation | Forces the file to execute with elevated privileges and an elevated token
\(for PE32, PE32\+, PE64 files only\) | False | Optional | -| obj_ext_extension | Change extension to valid | True | Optional | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| file | The existing War Room file EntryID to submit for analysis. | ${File.EntryID} | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| env_bitness | The bitness of the operating system. Supports: 32, 64. | 64 | Optional | +| env_version | The version of the OS. Supports: 7, 10, 11. | 10 | Optional | +| env_type | The environment preset type. You can select \*\*development\*\* env for OS Windows 10 x64. For all other cases, \*\*complete\*\* env is required. | complete | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 240 | Optional | +| opt_automated_interactivity | Whether to enable automated interactivity. | True | Optional | +| obj_ext_cmd | The optional command line. | | Optional | +| obj_ext_startfolder | The directory from which to start the file analysis. Supports: desktop, home, downloads, appdata, temp, windows, root. | temp | Optional | +| obj_force_elevation | Whether to force the file to execute with elevated privileges and an elevated token \(for PE32, PE32\+, PE64 files only\). | False | Optional | +| obj_ext_extension | Whether to change the extension to a valid one. | True | Optional | ## Playbook Outputs --- -There are no outputs for this playbook. +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateFileWindows.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | ## Playbook Image diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_for_XDR.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_for_XDR.yml new file mode 100644 index 000000000000..f636a6083029 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_for_XDR.yml @@ -0,0 +1,1385 @@ +id: ANYRUN Detonate File Windows for XDR +version: -1 +fromversion: 6.0.0 +name: ANYRUN Detonate File Windows for XDR +description: Submits a file retrieved from a Cortex XDR endpoint to the ANY.RUN Cloud Sandbox for dynamic analysis in a Windows environment. It automates malware detonation and behavior observation on Windows OS. +starttaskid: "0" +tasks: + "0": + id: "0" + taskid: 19175291-9d35-4fcb-8f53-1ccea76e04e4 + type: start + task: + id: 19175291-9d35-4fcb-8f53-1ccea76e04e4 + version: -1 + name: "" + iscommand: false + brand: "" + description: "" + nexttasks: + '#none#': + - "40" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 152.5, + "y": 50 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "40": + id: "40" + taskid: 810b40c6-d3b4-4344-a572-f74b3d66acb0 + type: condition + task: + id: 810b40c6-d3b4-4344-a572-f74b3d66acb0 + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "42" + "yes": + - "41" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 220 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "41": + id: "41" + taskid: 25cec696-ea84-47b4-bc60-95e186fa5aff + type: regular + task: + id: 25cec696-ea84-47b4-bc60-95e186fa5aff + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "42": + id: "42" + taskid: 4229b854-5603-4e05-a70d-3762c2fbeb4f + type: regular + task: + id: 4229b854-5603-4e05-a70d-3762c2fbeb4f + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "43" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "43": + id: "43" + taskid: 1a50a5ac-c495-44e4-a76c-73b8532d009b + type: regular + task: + id: 1a50a5ac-c495-44e4-a76c-73b8532d009b + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 590 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "2": + id: "2" + taskid: 0520382c-17b8-4840-8241-cfc472414156 + type: regular + task: + id: 0520382c-17b8-4840-8241-cfc472414156 + version: -1 + name: '[ANY.RUN] Detonate File Windows' + description: Performs file analysis using a Windows VM in ANY.RUN Cloud Sandbox. + script: '|||anyrun-detonate-file-windows' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "9" + scriptarguments: + using: + simple: ${ANYRUNInstance} + env_bitness: + simple: ${inputs.env_bitness} + env_locale: + simple: ${inputs.env_locale} + env_type: + simple: ${inputs.env_type} + env_version: + simple: ${inputs.env_version} + file: + simple: ${SelectedOriginalFile.EntryID} + obj_ext_cmd: + simple: ${inputs.obj_ext_cmd} + obj_ext_extension: + simple: ${inputs.obj_ext_extension} + obj_ext_startfolder: + simple: ${inputs.obj_ext_startfolder} + obj_force_elevation: + simple: ${inputs.obj_force_elevation} + opt_automated_interactivity: + simple: ${inputs.opt_automated_interactivity} + opt_network_connect: + simple: ${inputs.opt_network_connect} + opt_network_fakenet: + simple: ${inputs.opt_network_fakenet} + opt_network_geo: + simple: ${inputs.opt_network_geo} + opt_network_mitm: + simple: ${inputs.opt_network_mitm} + opt_network_residential_proxy: + simple: ${inputs.opt_network_residential_proxy} + opt_network_residential_proxy_geo: + simple: ${inputs.opt_network_residential_proxy_geo} + opt_network_tor: + simple: ${inputs.opt_network_tor} + opt_privacy_type: + simple: ${inputs.opt_privacy_type} + opt_timeout: + simple: ${inputs.opt_timeout} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 602.5, + "y": 2950 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "3": + id: "3" + taskid: 24d2bcfa-f20a-40a0-8c0f-1f9c46b85909 + type: regular + task: + id: 24d2bcfa-f20a-40a0-8c0f-1f9c46b85909 + version: -1 + name: '[ANY.RUN] Get analysis summary' + description: Returns the ANY.RUN analysis report in HTML format. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "10" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: html + task_uuid: + simple: ${ANYRUN_DetonateFileWindows.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 715, + "y": 3505 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "5": + id: "5" + taskid: 9fbf7c5f-582c-45b5-8e81-bc4d5e67855a + type: title + task: + id: 9fbf7c5f-582c-45b5-8e81-bc4d5e67855a + version: -1 + name: Done + type: title + iscommand: false + brand: "" + description: "" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 182.5, + "y": 4430 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "7": + id: "7" + taskid: 37b37e01-5cea-4839-853f-0e532d462666 + type: condition + task: + id: 37b37e01-5cea-4839-853f-0e532d462666 + version: -1 + name: Is there an active ANY.RUN instance? + description: Returns yes if the ANY.RUN Cloud Sandbox integration is available. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "36" + scriptarguments: + brandname: + simple: ANY.RUN Cloud Sandbox + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 152.5, + "y": 730 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "8": + id: "8" + taskid: e0db5ea8-371a-4a58-89d8-2f668af0e4a7 + type: condition + task: + id: e0db5ea8-371a-4a58-89d8-2f668af0e4a7 + version: -1 + name: Was a malicious or suspicious verdict received? + description: Checks whether the ANY.RUN verdict is not equal to No threats detected. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "3" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEqualString + left: + value: + simple: ANYRUN.SandboxAnalysisReportVerdict + iscontext: true + right: + value: + simple: No threats detected + continueonerrortype: "" + view: |- + { + "position": { + "x": 602.5, + "y": 3320 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "9": + id: "9" + taskid: d718e583-c390-44cb-84cf-1ec1a66e9645 + type: regular + task: + id: d718e583-c390-44cb-84cf-1ec1a66e9645 + version: -1 + name: '[ANY.RUN] Get analysis verdict' + description: 'Returns a threat level text. Possible values are No threats detected, Suspicious activity, or Malicious activity.' + script: '|||anyrun-get-analysis-verdict' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "8" + scriptarguments: + using: + simple: ${ANYRUNInstance} + task_uuid: + simple: ${ANYRUN_DetonateFileWindows.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 602.5, + "y": 3135 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "10": + id: "10" + taskid: a39e913f-83bb-4d80-8a02-821aca09fbf8 + type: regular + task: + id: a39e913f-83bb-4d80-8a02-821aca09fbf8 + version: -1 + name: '[ANY.RUN] Load IOCs to Threat Intel' + description: Returns the ANY.RUN analysis report in IOC format and loads the IOCs into the context. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "25" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: ioc + create_xdr_iocs: + simple: ${inputs.AutoAddIOCsToXDR} + task_uuid: + simple: ${ANYRUN_DetonateFileWindows.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 715, + "y": 3690 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "13": + id: "13" + taskid: dc6964fa-f3ea-4505-8d62-c3f2d0d54f6f + type: condition + task: + id: dc6964fa-f3ea-4505-8d62-c3f2d0d54f6f + version: -1 + name: Are endpoint ID and file path provided for endpoint retrieval? + description: Checks whether the minimum inputs are available to retrieve the + file from a Cortex XDR endpoint when no explicit file EntryID was provided. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "38" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.endpoint_id + iscontext: true + right: + value: {} + - - operator: isNotEmpty + left: + value: + simple: inputs.file_path + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 1100 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "14": + id: "14" + taskid: 803fca7c-4575-4230-89aa-f391384863fd + type: regular + task: + id: 803fca7c-4575-4230-89aa-f391384863fd + version: -1 + name: '[XDR] Retrieve file from endpoint' + description: Starts a Cortex Core IR file retrieval action for the requested + Windows path. The task uses built-in polling arguments, so a separate core-action-status-get + task is not required. + script: '|||core-retrieve-files' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "15" + scriptarguments: + endpoint_ids: + simple: ${inputs.endpoint_id} + interval_in_seconds: + simple: ${inputs.retrieve_poll_interval_seconds} + timeout_in_seconds: + simple: ${inputs.retrieve_timeout_seconds} + windows_file_paths: + simple: ${inputs.file_path} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1470 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "15": + id: "15" + taskid: 2bba64ba-83e7-40ce-8788-89abba9cb2a9 + type: regular + task: + id: 2bba64ba-83e7-40ce-8788-89abba9cb2a9 + version: -1 + name: '[XDR] Get retrieved file details' + description: Creates a War Room file entry for the archive retrieved from Cortex XDR, using the action ID returned by core-retrieve-files. + script: '|||core-retrieve-file-details' + type: regular + iscommand: true + brand: "" + nexttasks: + '#none#': + - "26" + scriptarguments: + action_id: + simple: ${Core.RetrievedFiles.action_id} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1655 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "16": + id: "16" + taskid: 8440830b-e3f6-453f-8c1e-fc25ae787b87 + type: regular + task: + id: 8440830b-e3f6-453f-8c1e-fc25ae787b87 + version: -1 + name: '[XDR] Unzip retrieved archive' + description: Unzips the retrieved archive and uploads extracted files to the War Room as file entries. + scriptName: UnzipFile + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "17" + scriptarguments: + entryID: + simple: ${ANYRUN.RetrievedArchive.EntryID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 2025 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "17": + id: "17" + taskid: 79e7aa8b-341f-4020-88dd-308c526c3535 + type: regular + task: + id: 79e7aa8b-341f-4020-88dd-308c526c3535 + version: -1 + name: '[XDR] Extract requested file name from path' + description: Extracts the basename from the file_path playbook input and saves it to the ANYRUN.Input.FileName context key. The playbook uses the filename to select the extracted original sample. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "18" + scriptarguments: + append: + simple: "false" + key: + simple: ANYRUN.Input.FileName + value: + complex: + root: inputs.file_path + transformers: + - operator: FirstArrayElement + - operator: RegexExtractAll + args: + regex: + value: + simple: '[^\\/]+$' + - operator: FirstArrayElement + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 2210 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "18": + id: "18" + taskid: fdfe7d07-bd04-4224-8100-9cfb211fdf61 + type: regular + task: + id: fdfe7d07-bd04-4224-8100-9cfb211fdf61 + version: -1 + name: '[XDR] Select original file EntryID by file name' + description: Selects the extracted original sample by matching File.Name to + the basename extracted from the file_path playbook input, then stores its EntryID in SelectedOriginalFile.EntryID. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "19" + scriptarguments: + append: + simple: "false" + key: + simple: SelectedOriginalFile.EntryID + value: + complex: + root: File + filters: + - - operator: isEqualString + left: + value: + simple: File.Name + iscontext: true + right: + value: + simple: ANYRUN.Input.FileName + iscontext: true + accessor: EntryID + transformers: + - operator: LastArrayElement + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 2395 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "19": + id: "19" + taskid: f523d6a5-1f66-4899-8e2f-ab4b097390bd + type: regular + task: + id: f523d6a5-1f66-4899-8e2f-ab4b097390bd + version: -1 + name: '[XDR] Store selected file name' + description: Stores the selected file name for visibility and downstream reporting. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "20" + scriptarguments: + append: + simple: "false" + key: + simple: SelectedOriginalFile.Name + value: + complex: + root: File + filters: + - - operator: isEqualString + left: + value: + simple: File.Name + iscontext: true + right: + value: + simple: ANYRUN.Input.FileName + iscontext: true + accessor: Name + transformers: + - operator: FirstArrayElement + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 2580 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "20": + id: "20" + taskid: 3e8d28ca-fef6-4324-8ea0-2a2d7c47989a + type: condition + task: + id: 3e8d28ca-fef6-4324-8ea0-2a2d7c47989a + version: -1 + name: Was the original file selected? + description: Checks whether SelectedOriginalFile.EntryID exists before submitting the file to ANY.RUN. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "2" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: SelectedOriginalFile.EntryID + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 490, + "y": 2765 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "22": + id: "22" + taskid: 7d1e8b3a-439e-4593-8cb1-ea9994e27b89 + type: regular + task: + id: 7d1e8b3a-439e-4593-8cb1-ea9994e27b89 + version: -1 + name: '[XDR] Add ANY.RUN IOCs to XDR IOC rules' + description: Uploads the IOC rules to Cortex XDR. When the ioc_object is defined, disregard any other provided arguments, as the ioc_object takes precedence. Validates the indicator parameters when ioc_object is used. If `vendor_name`, `vendor_reputation`, and `vendor_reliability` are used, only a single vendor is supported. For multiple vendors, utilize an `ioc_object` in JSON format. Adding a rule with the same indicator, but with different parameters, will update the existing rule. + script: Cortex Core - IR|||core-add-indicator-rule + type: regular + iscommand: true + brand: Cortex Core - IR + nexttasks: + '#none#': + - "5" + scriptarguments: + input_format: + simple: JSON + ioc_object: + simple: ${ANYRUN.IOCDetails.ioc_object} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 940, + "y": 4245 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "25": + id: "25" + taskid: 1df3e3cf-14c1-44c9-81f5-b842c44fe688 + type: condition + task: + id: 1df3e3cf-14c1-44c9-81f5-b842c44fe688 + version: -1 + name: Are there ANY.RUN IOC details to add to XDR? + description: Checks whether ANY.RUN returned IOCDetails.ioc_object before calling + core-add-indicator-rule. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "39" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: ANYRUN.IOCDetails.ioc_object + iscontext: true + right: + value: {} + - - operator: isEqualString + left: + value: + simple: inputs.AutoAddIOCsToXDR + iscontext: true + right: + value: + simple: "true" + continueonerrortype: "" + view: |- + { + "position": { + "x": 715, + "y": 3875 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "26": + id: "26" + taskid: 6013bd56-9d48-473f-80c4-f6bf9c4cbba8 + type: regular + task: + id: 6013bd56-9d48-473f-80c4-f6bf9c4cbba8 + version: -1 + name: '[XDR] Store retrieved archive EntryID' + description: Stores only the EntryID of the newly retrieved archive. This prevents UnzipFile from iterating over every File.EntryID already present in the incident context. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "16" + scriptarguments: + append: + simple: "false" + key: + simple: ANYRUN.RetrievedArchive.EntryID + value: + complex: + root: File + accessor: EntryID + transformers: + - operator: LastArrayElement + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1840 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "36": + id: "36" + taskid: 0c5ef628-7920-4096-8c9b-36b630a5d988 + type: condition + task: + id: 0c5ef628-7920-4096-8c9b-36b630a5d988 + version: -1 + name: Was an existing file EntryID provided? + description: Checks whether an explicit War Room file EntryID was provided. If yes, endpoint retrieval is skipped. If not, the playbook attempts Cortex XDR endpoint retrieval using the endpoint_id and file_path inputs. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "13" + "yes": + - "37" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.file_entry_id + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 265, + "y": 915 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "37": + id: "37" + taskid: a22b96ef-d35e-4cdd-804b-c618b02b1da9 + type: regular + task: + id: a22b96ef-d35e-4cdd-804b-c618b02b1da9 + version: -1 + name: '[Context] Use provided file EntryID' + description: Stores the explicitly provided War Room file EntryID as SelectedOriginalFile.EntryID. + scriptName: SetAndHandleEmpty + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "20" + scriptarguments: + append: + simple: "false" + key: + simple: SelectedOriginalFile.EntryID + value: + simple: ${inputs.file_entry_id} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 705, + "y": 2580 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "38": + id: "38" + taskid: 545a9bf9-c17c-483c-8f48-135eaf6e7800 + type: condition + task: + id: 545a9bf9-c17c-483c-8f48-135eaf6e7800 + version: -1 + name: Is Cortex Core - IR available? + description: Returns yes if Cortex Core - IR is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "14" + scriptarguments: + brandname: + simple: Cortex Core - IR + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 162.5, + "y": 1285 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "39": + id: "39" + taskid: f66a4148-402c-442d-8042-f9e3b11d6796 + type: condition + task: + id: f66a4148-402c-442d-8042-f9e3b11d6796 + version: -1 + name: Is Cortex Core - IR available? + description: Returns yes if Cortex Core - IR is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "22" + scriptarguments: + brandname: + simple: Cortex Core - IR + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 827.5, + "y": 4060 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false +view: |- + { + "linkLabelsPosition": {}, + "paper": { + "dimensions": { + "height": 4440, + "width": 1270, + "x": 50, + "y": 50 + } + } + } +inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: null +- key: file_entry_id + value: {} + required: false + description: The existing War Room file EntryID. If provided, endpoint retrieval is skipped and this file is submitted to ANY.RUN. + playbookInputQuery: null +- key: endpoint_id + value: + simple: ${issue.xdmsourceagentidentifier} + required: false + description: The ID of the Cortex XDR endpoint or agent. Used only when file_entry_id is not provided. + playbookInputQuery: null +- key: file_path + value: + simple: ${issue.initiatorpath} + required: false + description: The full path to the file on the endpoint. Used to retrieve the file and to extract the file name that selects the original file after the archive is unzipped. + playbookInputQuery: null +- key: retrieve_timeout_seconds + value: + simple: "240" + required: false + description: The time, in seconds, after which polling for the Cortex XDR file retrieval stops. + playbookInputQuery: null +- key: retrieve_poll_interval_seconds + value: + simple: "30" + required: false + description: The interval, in seconds, between status checks of the Cortex XDR file retrieval action. + playbookInputQuery: null +- key: env_locale + value: + simple: en-US + required: false + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' + playbookInputQuery: null +- key: env_bitness + value: + simple: "64" + required: false + description: 'The bitness of the operating system. Possible values are 32 or 64.' + playbookInputQuery: null +- key: env_version + value: + simple: "10" + required: false + description: 'The version of the operating system. Possible values are 7, 10, or 11.' + playbookInputQuery: null +- key: env_type + value: + simple: complete + required: false + description: The environment preset type. You can select the **development** environment for Windows 10 x64. For all other cases, the **complete** environment is required. + playbookInputQuery: null +- key: opt_network_connect + value: + simple: "true" + required: false + description: Whether to enable network connection. + playbookInputQuery: null +- key: opt_network_fakenet + value: + simple: "false" + required: false + description: Whether to enable the FakeNet feature. + playbookInputQuery: null +- key: opt_network_tor + value: + simple: "false" + required: false + description: Whether to use TOR. + playbookInputQuery: null +- key: opt_network_geo + value: + simple: fastest + required: false + description: 'The TOR geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_network_mitm + value: + simple: "false" + required: false + description: Whether to use the HTTPS MITM proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy + value: + simple: "false" + required: false + description: Whether to use a residential proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy_geo + value: + simple: fastest + required: false + description: 'The residential proxy geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_privacy_type + value: + simple: bylink + required: false + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' + playbookInputQuery: null +- key: opt_timeout + value: + simple: "240" + required: false + description: 'The timeout value. Size range: 10-660.' + playbookInputQuery: null +- key: opt_automated_interactivity + value: + simple: "true" + required: false + description: Whether to enable automated interactivity. + playbookInputQuery: null +- key: obj_ext_cmd + value: {} + required: false + description: The optional command line. + playbookInputQuery: null +- key: obj_ext_startfolder + value: + simple: temp + required: false + description: 'The directory from which to start the file analysis. Possible values are desktop, home, downloads, appdata, temp, windows, or root.' + playbookInputQuery: null +- key: obj_force_elevation + value: + simple: "false" + required: false + description: Whether to force the file to execute with elevated privileges and an elevated token (for PE32, PE32+, PE64 files only). + playbookInputQuery: null +- key: obj_ext_extension + value: + simple: "true" + required: false + description: Whether to change the extension to a valid one. + playbookInputQuery: null +- key: AutoAddIOCsToXDR + value: + simple: "true" + required: false + description: 'Whether to automatically add malicious and suspicious ANY.RUN IOCs to Cortex XDR IOC rules. IOCs with an Unknown reputation are always excluded. Set to false to disable.' + playbookInputQuery: null +inputSections: +- inputs: + - AutoAddIOCsToXDR + - Using + - file_entry_id + - endpoint_id + - file_path + - retrieve_timeout_seconds + - retrieve_poll_interval_seconds + name: Sample and Instance Selection + description: Optionally select the ANY.RUN instance with the Using input. If file_entry_id is provided, the playbook uses it directly. Otherwise, it retrieves the file from the Cortex XDR endpoint using endpoint_id and file_path, then selects the extracted file by filename. +- inputs: + - env_locale + - env_bitness + - env_version + - env_type + name: ANY.RUN Environment + description: "" +- inputs: + - opt_network_connect + - opt_network_fakenet + - opt_network_tor + - opt_network_geo + - opt_network_mitm + - opt_network_residential_proxy + - opt_network_residential_proxy_geo + - opt_privacy_type + - opt_timeout + - opt_automated_interactivity + name: ANY.RUN Options + description: "" +- inputs: + - obj_ext_cmd + - obj_ext_startfolder + - obj_force_elevation + - obj_ext_extension + name: ANY.RUN Object + description: "" +outputSections: +- outputs: + - SelectedOriginalFile.EntryID + - SelectedOriginalFile.Name + - ANYRUN_DetonateFileWindows.TaskID + - ANYRUN.SandboxAnalysisReportVerdict + - ANYRUN.IOCs + - ANYRUN.IOCDetails + name: General + description: The main outputs. +outputs: +- contextPath: SelectedOriginalFile.EntryID + description: The War Room EntryID of the selected original file sent to ANY.RUN. + type: String +- contextPath: SelectedOriginalFile.Name + description: The name of the selected original file. + type: String +- contextPath: ANYRUN_DetonateFileWindows.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown +- contextPath: ANYRUN.IOCDetails + description: The detailed IOC objects prepared for Cortex XDR indicator rules. + type: Unknown +quiet: true +adopted: true +marketplaces: +- platform +- marketplacev2 +supportedModules: +- edr +- xsiam diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_for_XDR_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_for_XDR_README.md new file mode 100644 index 000000000000..5ffa4cef053a --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_File_Windows_for_XDR_README.md @@ -0,0 +1,77 @@ +This playbook submits a file retrieved from a Cortex XDR endpoint to the ANY.RUN Cloud Sandbox for dynamic analysis in a Windows environment. It automates malware detonation and behavior observation on Windows OS. + +## Dependencies + +This playbook uses the following sub-playbooks, integrations, and scripts. + +### Sub-playbooks + +This playbook does not use any sub-playbooks. + +### Integrations + +* AnyRunSandbox +* Cortex Core - IR + +### Scripts + +* GetInstances +* IsIntegrationAvailable +* Set +* SetAndHandleEmpty +* UnzipFile + +### Commands + +* anyrun-detonate-file-windows +* anyrun-get-analysis-report +* anyrun-get-analysis-verdict +* core-add-indicator-rule +* core-retrieve-file-details +* core-retrieve-files + +## Playbook Inputs + +--- + +| **Name** | **Description** | **Default Value** | **Required** | +| --- | --- | --- | --- | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| AutoAddIOCsToXDR | Whether to automatically add malicious and suspicious ANY.RUN IOCs (Unknown reputation IOCs are always excluded) to Cortex XDR IOC rules. Set to "false" to disable. | true | Optional | +| file_entry_id | The existing War Room file EntryID. If provided, endpoint retrieval is skipped and this file is submitted to ANY.RUN. | | Optional | +| endpoint_id | The Cortex XDR endpoint/agent ID. Used only when file_entry_id is not provided. | ${issue.xdmsourceagentidentifier} | Optional | +| file_path | The full path to the file on the endpoint. Used for retrieval and for extracting the filename used to select the original file after unzip. | ${issue.initiatorpath} | Optional | +| retrieve_timeout_seconds | The timeout for Cortex XDR file retrieve polling. | 240 | Optional | +| retrieve_poll_interval_seconds | The polling interval for Cortex XDR file retrieve action status checks. | 30 | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| env_bitness | The bitness of the operating system. Supports: 32, 64. | 64 | Optional | +| env_version | The version of the OS. Supports: 7, 10, 11. | 10 | Optional | +| env_type | The environment preset type. You can select \*\*development\*\* env for OS Windows 10 x64. For all other cases, \*\*complete\*\* env is required. | complete | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 240 | Optional | +| opt_automated_interactivity | Whether to enable automated interactivity. | True | Optional | +| obj_ext_cmd | The optional command line. | | Optional | +| obj_ext_startfolder | The directory from which to start the file analysis. Supports: desktop, home, downloads, appdata, temp, windows, root. | temp | Optional | +| obj_force_elevation | Whether to force the file to execute with elevated privileges and an elevated token \(for PE32, PE32\+, PE64 files only\). | False | Optional | +| obj_ext_extension | Whether to change the extension to a valid one. | True | Optional | + +## Playbook Outputs + +--- +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| SelectedOriginalFile.EntryID | The War Room EntryID of the selected original file sent to ANY.RUN. | String | +| SelectedOriginalFile.Name | The name of the selected original file. | String | +| ANYRUN_DetonateFileWindows.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | +| ANYRUN.IOCDetails | The detailed IOC objects prepared for Cortex XDR indicator rules. | Unknown | + +![ANYRUN Detonate File Windows for XDR](../doc_files/ANYRUN_Detonate_File_Windows_for_XDR.png) diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android.yml index 9c2a347b303d..888a5ba9ec4d 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android.yml +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android.yml @@ -20,7 +20,7 @@ tasks: description: "" nexttasks: '#none#': - - "7" + - "20" separatecontext: false continueonerrortype: "" view: |- @@ -37,6 +37,168 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + "20": + id: "20" + taskid: 2ce35103-7b64-4f0c-8131-2c0a0ec8fa97 + type: condition + task: + id: 2ce35103-7b64-4f0c-8131-2c0a0ec8fa97 + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "22" + "yes": + - "21" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 185 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "21": + id: "21" + taskid: e9e71303-7a33-4d84-b4a0-e078440e1537 + type: regular + task: + id: e9e71303-7a33-4d84-b4a0-e078440e1537 + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "22": + id: "22" + taskid: a268c589-48e0-462e-97bd-55a2dffe1577 + type: regular + task: + id: a268c589-48e0-462e-97bd-55a2dffe1577 + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "23" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "23": + id: "23" + taskid: 2adb296f-b132-4a8e-a2e0-b2a79e251565 + type: regular + task: + id: 2adb296f-b132-4a8e-a2e0-b2a79e251565 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 495 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false "2": id: "2" taskid: 2edd7958-99a4-46a8-bd49-03c1c824adc2 @@ -54,10 +216,10 @@ tasks: '#none#': - "9" scriptarguments: + using: + simple: ${ANYRUNInstance} env_locale: simple: ${inputs.env_locale} - obj_ext_browser: - simple: ${inputs.obj_ext_browser} obj_url: simple: ${inputs.obj_url} opt_network_connect: @@ -84,7 +246,7 @@ tasks: { "position": { "x": 275, - "y": 560 + "y": 1000 } } note: false @@ -111,6 +273,8 @@ tasks: '#none#': - "10" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: html task_uuid: @@ -121,7 +285,7 @@ tasks: { "position": { "x": 387.5, - "y": 1085 + "y": 1525 } } note: false @@ -149,7 +313,7 @@ tasks: { "position": { "x": 50, - "y": 1785 + "y": 2225 } } note: false @@ -193,7 +357,7 @@ tasks: { "position": { "x": 162.5, - "y": 385 + "y": 825 } } note: false @@ -231,7 +395,7 @@ tasks: { "position": { "x": 50, - "y": 210 + "y": 650 } } note: false @@ -276,7 +440,7 @@ tasks: { "position": { "x": 275, - "y": 910 + "y": 1350 } } note: false @@ -304,6 +468,8 @@ tasks: '#none#': - "8" scriptarguments: + using: + simple: ${ANYRUNInstance} task_uuid: simple: ${ANYRUN_DetonateUrlAndroid.TaskID} separatecontext: false @@ -312,7 +478,7 @@ tasks: { "position": { "x": 275, - "y": 735 + "y": 1175 } } note: false @@ -339,6 +505,8 @@ tasks: '#none#': - "11" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: ioc task_uuid: @@ -349,7 +517,7 @@ tasks: { "position": { "x": 387.5, - "y": 1260 + "y": 1700 } } note: false @@ -395,7 +563,7 @@ tasks: { "position": { "x": 387.5, - "y": 1435 + "y": 1875 } } note: false @@ -428,13 +596,17 @@ tasks: accessor: id indicatorsValues: simple: ${ANYRUN.IOCs} + retry-count: + simple: "5" + retry-interval: + simple: "3" separatecontext: false continueonerrortype: "" view: |- { "position": { "x": 630, - "y": 1620 + "y": 2060 } } note: false @@ -452,90 +624,91 @@ view: |- }, "paper": { "dimensions": { - "height": 1795, - "width": 960, + "height": 2235, + "width": 1260, "x": 50, "y": 50 } } } inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: - key: obj_url value: simple: ${URL.Data} required: false - description: 'Target URL. Size range 5-512. Example: (http/https)://(your-link).' + description: 'The target URL. Size range: 5-512. For example, (http/https)://(your-link).' playbookInputQuery: - key: env_locale value: simple: en-US required: false - description: 'Operation system language. Use locale identifier or country name (Ex: - "en-US" or "Brazil"). Case insensitive.' + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' playbookInputQuery: - key: opt_network_connect value: - simple: "True" + simple: "true" required: false - description: Network connection state. + description: Whether to enable network connection. playbookInputQuery: - key: opt_network_fakenet value: - simple: "False" + simple: "false" required: false - description: FakeNet feature status. + description: Whether to enable the FakeNet feature. playbookInputQuery: - key: opt_network_tor value: - simple: "False" + simple: "false" required: false - description: TOR using. + description: Whether to use TOR. playbookInputQuery: - key: opt_network_geo value: simple: fastest required: false - description: 'Tor geo location option. Example: US, AU.' + description: 'The TOR geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_network_mitm value: - simple: "False" + simple: "false" required: false - description: HTTPS MITM proxy option. + description: Whether to use the HTTPS MITM proxy. playbookInputQuery: - key: opt_network_residential_proxy value: - simple: "False" + simple: "false" required: false - description: Residential proxy using. + description: Whether to use a residential proxy. playbookInputQuery: - key: opt_network_residential_proxy_geo value: simple: fastest required: false - description: 'Residential proxy geo location option. Example: US, AU.' + description: 'The residential proxy geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_privacy_type value: simple: bylink required: false - description: 'Privacy settings. Supports: public, bylink, owner, byteam.' + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' playbookInputQuery: - key: opt_timeout value: - simple: "60" + simple: "120" required: false - description: 'Timeout option. Size range: 10-660.' - playbookInputQuery: -- key: obj_ext_browser - value: - simple: Google Chrome - required: false - description: 'Browser name. Supports: Google Chrome, Mozilla Firefox' + description: 'The timeout value. Size range: 10-660.' playbookInputQuery: inputSections: - inputs: - obj_url + - Using name: General (Inputs group) description: Generic group for inputs - inputs: @@ -554,15 +727,32 @@ inputSections: - opt_timeout name: Options description: "" -- inputs: - - obj_ext_browser - name: Object - description: "" outputSections: - outputs: [] name: General (Outputs group) description: Generic group for outputs -outputs: [] +outputs: +- contextPath: ANYRUN_DetonateUrlAndroid.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown quiet: true tests: - No tests (auto formatted) +marketplaces: +- xsoar +- platform +- xpanse +- xsoar_saas +- marketplacev2 +supportedModules: +- agentix +- xsiam +- edr +- cloud +- cloud_runtime_security diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_README.md index b49b0140dc91..b540258e022c 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_README.md +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_README.md @@ -14,7 +14,9 @@ This playbook does not use any sub-playbooks. ### Scripts +* GetInstances * IsIntegrationAvailable +* Set * associateIndicatorsToIncident ### Commands @@ -29,26 +31,30 @@ This playbook does not use any sub-playbooks. | **Name** | **Description** | **Default Value** | **Required** | | --- | --- | --- | --- | -| obj_url | Target URL. Size range 5-512. Example: \(http/https\)://\(your-link\). | ${URL.Data} | Optional | -| env_locale | Operation system language. Use locale identifier or country name \(Ex: "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | -| opt_network_connect | Network connection state. | True | Optional | -| opt_network_fakenet | FakeNet feature status. | False | Optional | -| opt_network_tor | TOR using. | False | Optional | -| opt_network_geo | Tor geo location option. Example: US, AU. | fastest | Optional | -| opt_network_mitm | HTTPS MITM proxy option. | False | Optional | -| opt_network_residential_proxy | Residential proxy using. | False | Optional | -| opt_network_residential_proxy_geo | Residential proxy geo location option. Example: US, AU. | fastest | Optional | -| opt_privacy_type | Privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | -| opt_timeout | Timeout option. Size range: 10-660. | 60 | Optional | -| obj_ext_browser | Browser name. Supports: Google Chrome, Mozilla Firefox | Google Chrome | Optional | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| obj_url | The target URL. Size range 5-512. Example: \(http/https\)://\(your-link\). | ${URL.Data} | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 120 | Optional | ## Playbook Outputs --- -There are no outputs for this playbook. +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateUrlAndroid.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | ## Playbook Image --- -![ANYRUN Detonate URL Android](../doc_files/ANYRUN_Detonate_URL_Android.png) \ No newline at end of file +![ANYRUN Detonate Url Android](../doc_files/ANYRUN_Detonate_URL_Android.png) diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_for_XDR.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_for_XDR.yml new file mode 100644 index 000000000000..65871b60a44c --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_for_XDR.yml @@ -0,0 +1,805 @@ +id: ANYRUN Detonate URL Android for XDR +version: -1 +fromversion: 6.0.0 +name: ANYRUN Detonate URL Android for XDR +description: Submits a URL from a Cortex XDR issue to the ANY.RUN Cloud Sandbox for dynamic analysis in an Android environment and automates malware detonation and behavior observation on Android OS. +starttaskid: "0" +tasks: + "0": + id: "0" + taskid: 8ab32239-de7a-4606-8ffa-71caee430c68 + type: start + task: + id: 8ab32239-de7a-4606-8ffa-71caee430c68 + version: -1 + name: "" + iscommand: false + brand: "" + description: "" + nexttasks: + '#none#': + - "40" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 50 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "40": + id: "40" + taskid: 26d1554f-95f7-495c-8b4d-c638171f0c8e + type: condition + task: + id: 26d1554f-95f7-495c-8b4d-c638171f0c8e + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "42" + "yes": + - "41" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 220 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "41": + id: "41" + taskid: 398f7e62-4d39-4e32-9f30-98eb1c78cd7d + type: regular + task: + id: 398f7e62-4d39-4e32-9f30-98eb1c78cd7d + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "42": + id: "42" + taskid: cfb98274-d8ef-4285-9131-f82cdb7c9af9 + type: regular + task: + id: cfb98274-d8ef-4285-9131-f82cdb7c9af9 + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "43" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "43": + id: "43" + taskid: 4213190a-0370-4b83-a351-f16fe1bbf6f5 + type: regular + task: + id: 4213190a-0370-4b83-a351-f16fe1bbf6f5 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 590 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "2": + id: "2" + taskid: 2f78f28f-d630-4a20-8a3a-deaf2ce30864 + type: regular + task: + id: 2f78f28f-d630-4a20-8a3a-deaf2ce30864 + version: -1 + name: '[ANY.RUN] Detonate URL Android' + description: Performs URL analysis using an Android VM in ANY.RUN Cloud Sandbox. + script: '|||anyrun-detonate-url-android' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "9" + scriptarguments: + using: + simple: ${ANYRUNInstance} + env_locale: + simple: ${inputs.env_locale} + obj_ext_browser: + simple: ${inputs.obj_ext_browser} + obj_url: + simple: ${inputs.obj_url} + opt_network_connect: + simple: ${inputs.opt_network_connect} + opt_network_fakenet: + simple: ${inputs.opt_network_fakenet} + opt_network_geo: + simple: ${inputs.opt_network_geo} + opt_network_mitm: + simple: ${inputs.opt_network_mitm} + opt_network_residential_proxy: + simple: ${inputs.opt_network_residential_proxy} + opt_network_residential_proxy_geo: + simple: ${inputs.opt_network_residential_proxy_geo} + opt_network_tor: + simple: ${inputs.opt_network_tor} + opt_privacy_type: + simple: ${inputs.opt_privacy_type} + opt_timeout: + simple: ${inputs.opt_timeout} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1100 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "3": + id: "3" + taskid: 6b20f194-7997-4678-8db1-a65703e5db15 + type: regular + task: + id: 6b20f194-7997-4678-8db1-a65703e5db15 + version: -1 + name: '[ANY.RUN] Get analysis summary' + description: Returns the analysis report summary. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "10" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: html + task_uuid: + simple: ${ANYRUN_DetonateUrlAndroid.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 1655 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "5": + id: "5" + taskid: bc79555a-7d01-44c6-8cbf-0c0c0ccad8ae + type: title + task: + id: bc79555a-7d01-44c6-8cbf-0c0c0ccad8ae + version: -1 + name: Done + type: title + iscommand: false + brand: "" + description: "" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 80, + "y": 2580 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "6": + id: "6" + taskid: b6091c12-e8a5-4434-8ad5-b279ce140850 + type: condition + task: + id: b6091c12-e8a5-4434-8ad5-b279ce140850 + version: -1 + name: Is there any URL to detonate? + description: Returns yes if the integration brand is available. Otherwise, returns no. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "2" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.obj_url + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 162.5, + "y": 915 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "7": + id: "7" + taskid: 29e232fc-0e51-42a3-8324-bc332829a6f5 + type: condition + task: + id: 29e232fc-0e51-42a3-8324-bc332829a6f5 + version: -1 + name: Is there an active ANY.RUN instance? + description: Returns yes if the integration brand is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "6" + scriptarguments: + brandname: + simple: ANY.RUN Cloud Sandbox + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 730 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "8": + id: "8" + taskid: c8ea8096-a8b3-4a3a-8eb7-c7c319795668 + type: condition + task: + id: c8ea8096-a8b3-4a3a-8eb7-c7c319795668 + version: -1 + name: Was a malicious or suspicious verdict received? + description: Checks whether the values provided in the arguments are equal. If either of the arguments is missing, no is returned. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "3" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEqualString + left: + value: + simple: ANYRUN.SandboxAnalysisReportVerdict + iscontext: true + right: + value: + simple: No threats detected + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1470 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "9": + id: "9" + taskid: e8e10878-3ee5-442a-8ebe-c9643741c9f5 + type: regular + task: + id: e8e10878-3ee5-442a-8ebe-c9643741c9f5 + version: -1 + name: '[ANY.RUN] Get analysis verdict' + description: 'Returns a threat level text. Possible values are No threats detected, Suspicious activity, or Malicious activity.' + script: '|||anyrun-get-analysis-verdict' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "8" + scriptarguments: + using: + simple: ${ANYRUNInstance} + task_uuid: + simple: ${ANYRUN_DetonateUrlAndroid.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1285 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "10": + id: "10" + taskid: 4d84e32b-2746-4af0-82de-d82ed2edc33f + type: regular + task: + id: 4d84e32b-2746-4af0-82de-d82ed2edc33f + version: -1 + name: '[ANY.RUN] Load IOCs to Threat Intel' + description: Returns the analysis report summary. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "14" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: ioc + create_xdr_iocs: + simple: ${inputs.AutoAddIOCsToXDR} + task_uuid: + simple: ${ANYRUN_DetonateUrlAndroid.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 1840 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "13": + id: "13" + taskid: ccd41bfc-68a1-40b2-8976-aa21568488a2 + type: regular + task: + id: ccd41bfc-68a1-40b2-8976-aa21568488a2 + version: -1 + name: '[XDR] Add ANY.RUN IOCs to XDR IOC rules' + description: Uploads the IOC rules to Cortex XDR. When the ioc_object is defined, disregard any other provided arguments, as the ioc_object takes precedence. Validates the indicator parameters when ioc_object is used. If `vendor_name`, `vendor_reputation`, and `vendor_reliability` are used, only a single vendor is supported. For multiple vendors, utilize an `ioc_object` in JSON format. Adding a rule with the same indicator, but with different parameters, will update the existing rule. + script: Cortex Core - IR|||core-add-indicator-rule + type: regular + iscommand: true + brand: Cortex Core - IR + nexttasks: + '#none#': + - "5" + scriptarguments: + input_format: + simple: JSON + ioc_object: + simple: ${ANYRUN.IOCDetails.ioc_object} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 612.5, + "y": 2395 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "14": + id: "14" + taskid: d17b88b4-e0e9-44e9-8bfb-2399d3e0277f + type: condition + task: + id: d17b88b4-e0e9-44e9-8bfb-2399d3e0277f + version: -1 + name: Are there ANY.RUN IOC details to add to XDR? + description: Checks whether the ANY.RUN IOC report returned an ioc_object that + can be uploaded to Cortex XDR IOC rules. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "15" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: ANYRUN.IOCDetails.ioc_object + iscontext: true + right: + value: {} + - - operator: isEqualString + left: + value: + simple: inputs.AutoAddIOCsToXDR + iscontext: true + right: + value: + simple: "true" + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 2025 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "15": + id: "15" + taskid: 8f3159e5-cea0-42f4-871c-9fadfd1f6a7f + type: condition + task: + id: 8f3159e5-cea0-42f4-871c-9fadfd1f6a7f + version: -1 + name: Is Cortex Core - IR available? + description: Returns yes if Cortex Core - IR is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "13" + scriptarguments: + brandname: + simple: Cortex Core - IR + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 500, + "y": 2210 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false +view: |- + { + "linkLabelsPosition": { + "8_5_#default#": 0.33 + }, + "paper": { + "dimensions": { + "height": 2590, + "width": 942.5, + "x": 50, + "y": 50 + } + } + } +inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: null +- key: obj_url + value: + simple: ${issue.xdmtargeturl} + required: false + description: 'The target URL to submit to ANY.RUN. By default, this playbook uses the Cortex XDR normalized target URL field xdm.target.url (${issue.xdmtargeturl}). If your issue uses another URL field, such as xdm.network.http.url, override this input accordingly. Size range: 5-512. For example, (http/https)://(your-link).' + playbookInputQuery: null +- key: env_locale + value: + simple: en-US + required: false + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' + playbookInputQuery: null +- key: opt_network_connect + value: + simple: "true" + required: false + description: Whether to enable network connection. + playbookInputQuery: null +- key: opt_network_fakenet + value: + simple: "false" + required: false + description: Whether to enable the FakeNet feature. + playbookInputQuery: null +- key: opt_network_tor + value: + simple: "false" + required: false + description: Whether to use TOR. + playbookInputQuery: null +- key: opt_network_geo + value: + simple: fastest + required: false + description: 'The TOR geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_network_mitm + value: + simple: "false" + required: false + description: Whether to use the HTTPS MITM proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy + value: + simple: "false" + required: false + description: Whether to use a residential proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy_geo + value: + simple: fastest + required: false + description: 'The residential proxy geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_privacy_type + value: + simple: bylink + required: false + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' + playbookInputQuery: null +- key: opt_timeout + value: + simple: "120" + required: false + description: 'The timeout value. Size range: 10-660.' + playbookInputQuery: null +- key: obj_ext_browser + value: + simple: Google Chrome + required: false + description: The browser name. + playbookInputQuery: null +- key: AutoAddIOCsToXDR + value: + simple: "true" + required: false + description: 'Whether to automatically add malicious and suspicious ANY.RUN IOCs to Cortex XDR IOC rules. IOCs with an Unknown reputation are always excluded. Set to false to disable.' + playbookInputQuery: null +inputSections: +- inputs: + - AutoAddIOCsToXDR + - Using + - obj_url + name: URL and Instance Selection + description: Provide the target URL and optionally select the ANY.RUN instance with the Using input. +- inputs: + - env_locale + name: Environment + description: "" +- inputs: + - opt_network_connect + - opt_network_fakenet + - opt_network_tor + - opt_network_geo + - opt_network_mitm + - opt_network_residential_proxy + - opt_network_residential_proxy_geo + - opt_privacy_type + - opt_timeout + name: Options + description: "" +- inputs: + - obj_ext_browser + name: Object + description: "" +outputSections: +- outputs: [] + name: General (Outputs group) + description: The generic group for outputs. +outputs: +- contextPath: ANYRUN_DetonateUrlAndroid.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown +- contextPath: ANYRUN.IOCDetails + description: The detailed IOC objects prepared for Cortex XDR indicator rules. + type: Unknown +quiet: true +adopted: true +marketplaces: +- platform +- marketplacev2 +supportedModules: +- edr +- xsiam diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_for_XDR_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_for_XDR_README.md new file mode 100644 index 000000000000..a488bb15fdf1 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Android_for_XDR_README.md @@ -0,0 +1,60 @@ +This playbook submits a URL from a Cortex XDR issue to the ANY.RUN Cloud Sandbox for dynamic analysis in an Android environment and automates malware detonation and behavior observation on an Android Environment. + +## Dependencies + +This playbook uses the following sub-playbooks, integrations, and scripts. + +### Sub-playbooks + +This playbook does not use any sub-playbooks. + +### Integrations + +* AnyRunSandbox +* Cortex Core - IR + +### Scripts + +* GetInstances +* IsIntegrationAvailable +* Set + +### Commands + +* anyrun-detonate-url-android +* anyrun-get-analysis-report +* anyrun-get-analysis-verdict +* core-add-indicator-rule + +## Playbook Inputs + +--- + +| **Name** | **Description** | **Default Value** | **Required** | +| --- | --- | --- | --- | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| AutoAddIOCsToXDR | Whether to automatically add malicious and suspicious ANY.RUN IOCs (Unknown reputation IOCs are always excluded) to Cortex XDR IOC rules. Set to "false" to disable. | true | Optional | +| obj_url | The target URL to submit to ANY.RUN. By default, this playbook uses the Cortex XDR normalized target URL field xdm.target.url \(${issue.xdmtargeturl}\). If your issue uses another URL field, such as xdm.network.http.url, override this input accordingly. Size range 5-512. Example: \(http/https\)://\(your-link\). | ${issue.xdmtargeturl} | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 120 | Optional | +| obj_ext_browser | The browser name. | Google Chrome | Optional | + +## Playbook Outputs + +--- +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateUrlAndroid.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | +| ANYRUN.IOCDetails | The detailed IOC objects prepared for Cortex XDR indicator rules. | Unknown | + +![ANYRUN Detonate URL Android for XDR](../doc_files/ANYRUN_Detonate_URL_Android_for_XDR.png) \ No newline at end of file diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux.yml index a31095c0199f..71515bb93704 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux.yml +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux.yml @@ -2,8 +2,8 @@ id: ANYRUN Detonate Url Linux version: -1 name: ANYRUN Detonate Url Linux fromversion: 6.0.0 -description: This playbook submits a URL extracted from an indicator to the - ANY.RUN cloud sandbox for dynamic analysis in an Linux environment. +description: This playbook submits a URL extracted from an indicator to the + ANY.RUN cloud sandbox for dynamic analysis in an Linux environment. It automates the analysis of potentially malicious URLs on Linux OS. starttaskid: "0" tasks: @@ -20,7 +20,7 @@ tasks: description: "" nexttasks: '#none#': - - "7" + - "20" separatecontext: false continueonerrortype: "" view: |- @@ -37,6 +37,168 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + "20": + id: "20" + taskid: 30b73618-1aba-4b28-8604-bc96dff76de0 + type: condition + task: + id: 30b73618-1aba-4b28-8604-bc96dff76de0 + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "22" + "yes": + - "21" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 185 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "21": + id: "21" + taskid: d9668c10-48d8-4d6d-bd43-e3864747b492 + type: regular + task: + id: d9668c10-48d8-4d6d-bd43-e3864747b492 + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "22": + id: "22" + taskid: 49f4fd2c-782d-4a82-8e6f-f714d2b7b041 + type: regular + task: + id: 49f4fd2c-782d-4a82-8e6f-f714d2b7b041 + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "23" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "23": + id: "23" + taskid: 5548601f-4ef6-40f4-944f-3c0ce1f9611f + type: regular + task: + id: 5548601f-4ef6-40f4-944f-3c0ce1f9611f + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 495 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false "2": id: "2" taskid: 68f15f73-9164-4f65-9544-c96cf90f9def @@ -54,6 +216,8 @@ tasks: '#none#': - "9" scriptarguments: + using: + simple: ${ANYRUNInstance} env_locale: simple: ${inputs.env_locale} env_os: @@ -88,7 +252,7 @@ tasks: { "position": { "x": 275, - "y": 560 + "y": 1000 } } note: false @@ -115,6 +279,8 @@ tasks: '#none#': - "10" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: html task_uuid: @@ -125,7 +291,7 @@ tasks: { "position": { "x": 387.5, - "y": 1085 + "y": 1525 } } note: false @@ -153,7 +319,7 @@ tasks: { "position": { "x": 50, - "y": 1785 + "y": 2225 } } note: false @@ -197,7 +363,7 @@ tasks: { "position": { "x": 162.5, - "y": 385 + "y": 825 } } note: false @@ -228,14 +394,14 @@ tasks: - "6" scriptarguments: brandname: - simple: ANY.RUN Sandbox + simple: ANY.RUN Cloud Sandbox separatecontext: false continueonerrortype: "" view: |- { "position": { "x": 50, - "y": 210 + "y": 650 } } note: false @@ -280,7 +446,7 @@ tasks: { "position": { "x": 275, - "y": 910 + "y": 1350 } } note: false @@ -308,6 +474,8 @@ tasks: '#none#': - "8" scriptarguments: + using: + simple: ${ANYRUNInstance} task_uuid: simple: ${ANYRUN_DetonateUrlLinux.TaskID} separatecontext: false @@ -316,7 +484,7 @@ tasks: { "position": { "x": 275, - "y": 735 + "y": 1175 } } note: false @@ -343,6 +511,8 @@ tasks: '#none#': - "11" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: ioc task_uuid: @@ -353,7 +523,7 @@ tasks: { "position": { "x": 387.5, - "y": 1260 + "y": 1700 } } note: false @@ -399,7 +569,7 @@ tasks: { "position": { "x": 387.5, - "y": 1435 + "y": 1875 } } note: false @@ -432,13 +602,17 @@ tasks: accessor: id indicatorsValues: simple: ${ANYRUN.IOCs} + retry-count: + simple: "5" + retry-interval: + simple: "3" separatecontext: false continueonerrortype: "" view: |- { "position": { "x": 620, - "y": 1610 + "y": 2050 } } note: false @@ -456,102 +630,109 @@ view: |- }, "paper": { "dimensions": { - "height": 1795, - "width": 950, + "height": 2235, + "width": 1250, "x": 50, "y": 50 } } } inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: - key: obj_url value: simple: ${URL.Data} required: false - description: 'Target URL. Size range 5-512. Example: (http/https)://(your-link)' + description: 'The target URL. Size range: 5-512. For example, (http/https)://(your-link).' playbookInputQuery: null - key: env_locale value: simple: en-US required: false - description: 'Operation system language. Use locale identifier or country name (Ex: - "en-US" or "Brazil"). Case insensitive.' + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' playbookInputQuery: null - key: env_os value: simple: ubuntu required: false - description: Operation system. Possible values - ubuntu, debian + description: 'The operating system. Possible values are ubuntu or debian.' playbookInputQuery: null - key: opt_network_connect value: - simple: "True" + simple: "true" required: false - description: Network connection state. + description: Whether to enable network connection. playbookInputQuery: null - key: opt_network_fakenet value: - simple: "False" + simple: "false" required: false - description: FakeNet feature status. + description: Whether to enable the FakeNet feature. playbookInputQuery: null - key: opt_network_tor value: - simple: "False" + simple: "false" required: false - description: TOR using. + description: Whether to use TOR. playbookInputQuery: null - key: opt_network_geo value: simple: fastest required: false - description: 'Tor geo location option. Example: US, AU.' + description: 'The TOR geolocation option, for example, US or AU.' playbookInputQuery: null - key: opt_network_mitm value: - simple: "False" + simple: "false" required: false - description: HTTPS MITM proxy option. + description: Whether to use the HTTPS MITM proxy. playbookInputQuery: null - key: opt_network_residential_proxy value: - simple: "False" + simple: "false" required: false - description: Residential proxy using. + description: Whether to use a residential proxy. playbookInputQuery: null - key: opt_network_residential_proxy_geo value: simple: fastest required: false - description: 'Residential proxy geo location option. Example: US, AU.' + description: 'The residential proxy geolocation option, for example, US or AU.' playbookInputQuery: null - key: opt_privacy_type value: simple: bylink required: false - description: 'Privacy settings. Supports: public, bylink, owner, byteam.' + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' playbookInputQuery: null - key: opt_timeout value: simple: "120" required: false - description: 'Timeout option. Size range: 10-660.' + description: 'The timeout value. Size range: 10-660.' playbookInputQuery: null - key: obj_ext_browser value: simple: Google Chrome required: false - description: Optional command line + description: 'The browser name. Possible values are Google Chrome or Mozilla Firefox.' playbookInputQuery: null - key: obj_ext_extension value: - simple: "True" + simple: "true" required: false - description: Change extension to valid + description: Whether to change the extension to a valid one. playbookInputQuery: null inputSections: - inputs: - obj_url + - Using name: General (Inputs group) description: Generic group for inputs - inputs: @@ -580,6 +761,27 @@ outputSections: - outputs: [] name: General (Outputs group) description: Generic group for outputs -outputs: [] +outputs: +- contextPath: ANYRUN_DetonateUrlLinux.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown sourceplaybookid: 52d278a0-2905-4a37-8379-4853a06f65d8 quiet: true +marketplaces: +- xsoar +- platform +- xpanse +- xsoar_saas +- marketplacev2 +supportedModules: +- agentix +- xsiam +- edr +- cloud +- cloud_runtime_security diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_README.md index 3826da11d864..fbe40944dd9b 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_README.md +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_README.md @@ -14,7 +14,9 @@ This playbook does not use any sub-playbooks. ### Scripts +* GetInstances * IsIntegrationAvailable +* Set * associateIndicatorsToIncident ### Commands @@ -29,28 +31,33 @@ This playbook does not use any sub-playbooks. | **Name** | **Description** | **Default Value** | **Required** | | --- | --- | --- | --- | -| obj_url | Target URL. Size range 5-512. Example: \(http/https\)://\(your-link\) | ${URL.Data} | Optional | -| env_locale | Operation system language. Use locale identifier or country name \(Ex: "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | -| env_os | Operation system | ubuntu | Optional | -| opt_network_connect | Network connection state. | True | Optional | -| opt_network_fakenet | FakeNet feature status. | False | Optional | -| opt_network_tor | TOR using. | False | Optional | -| opt_network_geo | Tor geo location option. Example: US, AU. | fastest | Optional | -| opt_network_mitm | HTTPS MITM proxy option. | False | Optional | -| opt_network_residential_proxy | Residential proxy using. | False | Optional | -| opt_network_residential_proxy_geo | Residential proxy geo location option. Example: US, AU. | fastest | Optional | -| opt_privacy_type | Privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | -| opt_timeout | Timeout option. Size range: 10-660. | 120 | Optional | -| obj_ext_browser | Optional command line | Google Chrome | Optional | -| obj_ext_extension | Change extension to valid | True | Optional | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| obj_url | The target URL. Size range 5-512. Example: \(http/https\)://\(your-link\). | ${URL.Data} | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| env_os | The operating system. Possible values: ubuntu, debian. | ubuntu | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 120 | Optional | +| obj_ext_browser | The browser name. Supports: Google Chrome, Mozilla Firefox. | Google Chrome | Optional | +| obj_ext_extension | Whether to change the extension to a valid one. | True | Optional | ## Playbook Outputs --- -There are no outputs for this playbook. +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateUrlLinux.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | ## Playbook Image --- -![ANYRUN Detonate URL Linux](../doc_files/ANYRUN_Detonate_URL_Linux.png) +![ANYRUN Detonate Url Linux](../doc_files/ANYRUN_Detonate_URL_Linux.png) diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_for_XDR.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_for_XDR.yml new file mode 100644 index 000000000000..6196993352b3 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_for_XDR.yml @@ -0,0 +1,823 @@ +id: ANYRUN Detonate URL Linux for XDR +version: -1 +fromversion: 6.0.0 +name: ANYRUN Detonate URL Linux for XDR +description: Submits a URL from a Cortex XDR issue to the ANY.RUN Cloud Sandbox for dynamic analysis in a Linux environment and automates malware detonation and behavior observation on Linux OS. +starttaskid: "0" +tasks: + "0": + id: "0" + taskid: dcb8fc97-7163-40aa-8c61-38ad1ee59db9 + type: start + task: + id: dcb8fc97-7163-40aa-8c61-38ad1ee59db9 + version: -1 + name: "" + iscommand: false + brand: "" + description: "" + nexttasks: + '#none#': + - "40" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 50 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "40": + id: "40" + taskid: 20b42369-91c9-4dc7-bda5-86a75cbb70a3 + type: condition + task: + id: 20b42369-91c9-4dc7-bda5-86a75cbb70a3 + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "42" + "yes": + - "41" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 220 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "41": + id: "41" + taskid: 1a590795-bf2f-4db5-8262-7d84ac99100a + type: regular + task: + id: 1a590795-bf2f-4db5-8262-7d84ac99100a + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "42": + id: "42" + taskid: e89b96d2-c515-46b3-8b86-762601eacccb + type: regular + task: + id: e89b96d2-c515-46b3-8b86-762601eacccb + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "43" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "43": + id: "43" + taskid: 023b3105-f9f6-428a-babd-0b4378c88b24 + type: regular + task: + id: 023b3105-f9f6-428a-babd-0b4378c88b24 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 590 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "2": + id: "2" + taskid: f99a0135-524f-431f-8e35-3566c31943e7 + type: regular + task: + id: f99a0135-524f-431f-8e35-3566c31943e7 + version: -1 + name: '[ANY.RUN] Detonate URL Linux' + description: Performs URL analysis using a Linux VM in ANY.RUN Cloud Sandbox. + script: '|||anyrun-detonate-url-linux' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "9" + scriptarguments: + using: + simple: ${ANYRUNInstance} + env_locale: + simple: ${inputs.env_locale} + env_os: + simple: ${inputs.env_os} + obj_ext_browser: + simple: ${inputs.obj_ext_browser} + obj_ext_extension: + simple: ${inputs.obj_ext_extension} + obj_url: + simple: ${inputs.obj_url} + opt_network_connect: + simple: ${inputs.opt_network_connect} + opt_network_fakenet: + simple: ${inputs.opt_network_fakenet} + opt_network_geo: + simple: ${inputs.opt_network_geo} + opt_network_mitm: + simple: ${inputs.opt_network_mitm} + opt_network_residential_proxy: + simple: ${inputs.opt_network_residential_proxy} + opt_network_residential_proxy_geo: + simple: ${inputs.opt_network_residential_proxy_geo} + opt_network_tor: + simple: ${inputs.opt_network_tor} + opt_privacy_type: + simple: ${inputs.opt_privacy_type} + opt_timeout: + simple: ${inputs.opt_timeout} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1100 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "3": + id: "3" + taskid: 43c5ab5b-c736-4541-8ccf-c227014f70d9 + type: regular + task: + id: 43c5ab5b-c736-4541-8ccf-c227014f70d9 + version: -1 + name: '[ANY.RUN] Get analysis summary' + description: Returns the analysis report summary. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "10" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: html + task_uuid: + simple: ${ANYRUN_DetonateUrlLinux.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 1655 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "5": + id: "5" + taskid: bfc16810-0c3c-4521-8306-701cd919b061 + type: title + task: + id: bfc16810-0c3c-4521-8306-701cd919b061 + version: -1 + name: Done + type: title + iscommand: false + brand: "" + description: "" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 80, + "y": 2580 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "6": + id: "6" + taskid: 5838f61a-6fbc-4cd5-89ce-a7acd2307933 + type: condition + task: + id: 5838f61a-6fbc-4cd5-89ce-a7acd2307933 + version: -1 + name: Is there any URL to detonate? + description: Returns yes if the integration brand is available. Otherwise, returns no. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "2" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.obj_url + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 162.5, + "y": 915 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "7": + id: "7" + taskid: 84e9536d-ff9f-488d-8ab4-33d1a866c0a7 + type: condition + task: + id: 84e9536d-ff9f-488d-8ab4-33d1a866c0a7 + version: -1 + name: Is there an active ANY.RUN instance? + description: Returns yes if the integration brand is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "6" + scriptarguments: + brandname: + simple: ANY.RUN Cloud Sandbox + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 730 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "8": + id: "8" + taskid: c7811551-1a86-4b37-8870-f8b4f6a844d7 + type: condition + task: + id: c7811551-1a86-4b37-8870-f8b4f6a844d7 + version: -1 + name: Was a malicious or suspicious verdict received? + description: Checks whether the values provided in the arguments are equal. If either of the arguments is missing, no is returned. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "3" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEqualString + left: + value: + simple: ANYRUN.SandboxAnalysisReportVerdict + iscontext: true + right: + value: + simple: No threats detected + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1470 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "9": + id: "9" + taskid: 477c6c46-c168-4479-8b38-3c9c20f1f9f6 + type: regular + task: + id: 477c6c46-c168-4479-8b38-3c9c20f1f9f6 + version: -1 + name: '[ANY.RUN] Get analysis verdict' + description: 'Returns a threat level text. Possible values are No threats detected, Suspicious activity, or Malicious activity.' + script: '|||anyrun-get-analysis-verdict' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "8" + scriptarguments: + using: + simple: ${ANYRUNInstance} + task_uuid: + simple: ${ANYRUN_DetonateUrlLinux.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1285 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "10": + id: "10" + taskid: f002d3a8-ab08-4b3e-854c-05ffe9963c6c + type: regular + task: + id: f002d3a8-ab08-4b3e-854c-05ffe9963c6c + version: -1 + name: '[ANY.RUN] Load IOCs to Threat Intel' + description: Returns the analysis report summary. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "14" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: ioc + create_xdr_iocs: + simple: ${inputs.AutoAddIOCsToXDR} + task_uuid: + simple: ${ANYRUN_DetonateUrlLinux.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 1840 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "13": + id: "13" + taskid: 6afed61e-6414-417a-8c49-897f6fe0bc9d + type: regular + task: + id: 6afed61e-6414-417a-8c49-897f6fe0bc9d + version: -1 + name: '[XDR] Add ANY.RUN IOCs to XDR IOC rules' + description: Uploads the IOC rules to Cortex XDR. When the ioc_object is defined, disregard any other provided arguments, as the ioc_object takes precedence. Validates the indicator parameters when ioc_object is used. If `vendor_name`, `vendor_reputation`, and `vendor_reliability` are used, only a single vendor is supported. For multiple vendors, utilize an `ioc_object` in JSON format. Adding a rule with the same indicator, but with different parameters, will update the existing rule. + script: Cortex Core - IR|||core-add-indicator-rule + type: regular + iscommand: true + brand: Cortex Core - IR + nexttasks: + '#none#': + - "5" + scriptarguments: + input_format: + simple: JSON + ioc_object: + simple: ${ANYRUN.IOCDetails.ioc_object} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 612.5, + "y": 2395 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "14": + id: "14" + taskid: 7d30195f-0c63-433d-846d-cb0496a74660 + type: condition + task: + id: 7d30195f-0c63-433d-846d-cb0496a74660 + version: -1 + name: Are there ANY.RUN IOC details to add to XDR? + description: Checks whether the ANY.RUN IOC report returned an ioc_object that + can be uploaded to Cortex XDR IOC rules. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "15" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: ANYRUN.IOCDetails.ioc_object + iscontext: true + right: + value: {} + - - operator: isEqualString + left: + value: + simple: inputs.AutoAddIOCsToXDR + iscontext: true + right: + value: + simple: "true" + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 2025 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "15": + id: "15" + taskid: dae25d70-2c13-479c-8e12-06a2a2357ebf + type: condition + task: + id: dae25d70-2c13-479c-8e12-06a2a2357ebf + version: -1 + name: Is Cortex Core - IR available? + description: Returns yes if Cortex Core - IR is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "13" + scriptarguments: + brandname: + simple: Cortex Core - IR + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 500, + "y": 2210 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false +view: |- + { + "linkLabelsPosition": { + "8_5_#default#": 0.33 + }, + "paper": { + "dimensions": { + "height": 2590, + "width": 942.5, + "x": 50, + "y": 50 + } + } + } +inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: null +- key: obj_url + value: + simple: ${issue.xdmtargeturl} + required: false + description: 'The target URL to submit to ANY.RUN. By default, this playbook uses the Cortex XDR normalized target URL field xdm.target.url (${issue.xdmtargeturl}). If your issue uses another URL field, such as xdm.network.http.url, override this input accordingly. Size range: 5-512. For example, (http/https)://(your-link).' + playbookInputQuery: null +- key: env_locale + value: + simple: en-US + required: false + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' + playbookInputQuery: null +- key: env_os + value: + simple: ubuntu + required: false + description: 'The operating system. Possible values are ubuntu or debian.' + playbookInputQuery: null +- key: opt_network_connect + value: + simple: "true" + required: false + description: Whether to enable network connection. + playbookInputQuery: null +- key: opt_network_fakenet + value: + simple: "false" + required: false + description: Whether to enable the FakeNet feature. + playbookInputQuery: null +- key: opt_network_tor + value: + simple: "false" + required: false + description: Whether to use TOR. + playbookInputQuery: null +- key: opt_network_geo + value: + simple: fastest + required: false + description: 'The TOR geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_network_mitm + value: + simple: "false" + required: false + description: Whether to use the HTTPS MITM proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy + value: + simple: "false" + required: false + description: Whether to use a residential proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy_geo + value: + simple: fastest + required: false + description: 'The residential proxy geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_privacy_type + value: + simple: bylink + required: false + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' + playbookInputQuery: null +- key: opt_timeout + value: + simple: "120" + required: false + description: 'The timeout value. Size range: 10-660.' + playbookInputQuery: null +- key: obj_ext_browser + value: + simple: Google Chrome + required: false + description: 'The browser name. Possible values are Google Chrome or Mozilla Firefox.' + playbookInputQuery: null +- key: obj_ext_extension + value: + simple: "true" + required: false + description: Whether to change the extension to a valid one. + playbookInputQuery: null +- key: AutoAddIOCsToXDR + value: + simple: "true" + required: false + description: 'Whether to automatically add malicious and suspicious ANY.RUN IOCs to Cortex XDR IOC rules. IOCs with an Unknown reputation are always excluded. Set to false to disable.' + playbookInputQuery: null +inputSections: +- inputs: + - AutoAddIOCsToXDR + - Using + - obj_url + name: URL and Instance Selection + description: Provide the target URL and optionally select the ANY.RUN instance with the Using input. +- inputs: + - env_locale + - env_os + name: Environment + description: "" +- inputs: + - opt_network_connect + - opt_network_fakenet + - opt_network_tor + - opt_network_geo + - opt_network_mitm + - opt_network_residential_proxy + - opt_network_residential_proxy_geo + - opt_privacy_type + - opt_timeout + name: Options + description: "" +- inputs: + - obj_ext_browser + - obj_ext_extension + name: Object + description: "" +outputSections: +- outputs: [] + name: General (Outputs group) + description: The generic group for outputs. +outputs: +- contextPath: ANYRUN_DetonateUrlLinux.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown +- contextPath: ANYRUN.IOCDetails + description: The detailed IOC objects prepared for Cortex XDR indicator rules. + type: Unknown +quiet: true +adopted: true +marketplaces: +- platform +- marketplacev2 +supportedModules: +- edr +- xsiam diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_for_XDR_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_for_XDR_README.md new file mode 100644 index 000000000000..dded4e4e98e6 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Linux_for_XDR_README.md @@ -0,0 +1,62 @@ +This playbook submits a URL from a Cortex XDR issue to the ANY.RUN Cloud Sandbox for dynamic analysis in a Linux environment. It helps to automate malware detonation and behavior observation on Linux OS. + +## Dependencies + +This playbook uses the following sub-playbooks, integrations, and scripts. + +### Sub-playbooks + +This playbook does not use any sub-playbooks. + +### Integrations + +* AnyRunSandbox +* Cortex Core - IR + +### Scripts + +* GetInstances +* IsIntegrationAvailable +* Set + +### Commands + +* anyrun-detonate-url-linux +* anyrun-get-analysis-report +* anyrun-get-analysis-verdict +* core-add-indicator-rule + +## Playbook Inputs + +--- + +| **Name** | **Description** | **Default Value** | **Required** | +| --- | --- | --- | --- | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| AutoAddIOCsToXDR | Whether to automatically add malicious and suspicious ANY.RUN IOCs (Unknown reputation IOCs are always excluded) to Cortex XDR IOC rules. Set to "false" to disable. | true | Optional | +| obj_url | The target URL to submit to ANY.RUN. By default, this playbook uses the Cortex XDR normalized target URL field xdm.target.url \(${issue.xdmtargeturl}\). If your issue uses another URL field, such as xdm.network.http.url, override this input accordingly. Size range 5-512. Example: \(http/https\)://\(your-link\). | ${issue.xdmtargeturl} | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| env_os | The operating system. Possible values: ubuntu, debian. | ubuntu | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 120 | Optional | +| obj_ext_browser | The browser name. Supports: Google Chrome, Mozilla Firefox. | Google Chrome | Optional | +| obj_ext_extension | Whether to change the extension to a valid one. | True | Optional | + +## Playbook Outputs + +--- +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateUrlLinux.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | +| ANYRUN.IOCDetails | The detailed IOC objects prepared for Cortex XDR indicator rules. | Unknown | + +![ANYRUN Detonate URL Linux for XDR](../doc_files/ANYRUN_Detonate_URL_Linux_for_XDR.png) diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows.yml index 7cd2f790d4df..60ee4e1c5a69 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows.yml +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows.yml @@ -20,7 +20,7 @@ tasks: description: "" nexttasks: '#none#': - - "7" + - "20" separatecontext: false continueonerrortype: "" view: |- @@ -37,6 +37,168 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + "20": + id: "20" + taskid: 54071cbd-1cad-409e-b496-7a1f2405edcc + type: condition + task: + id: 54071cbd-1cad-409e-b496-7a1f2405edcc + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "22" + "yes": + - "21" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 185 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "21": + id: "21" + taskid: 602f0b17-54fb-40ef-afba-9fa2fc16b865 + type: regular + task: + id: 602f0b17-54fb-40ef-afba-9fa2fc16b865 + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "22": + id: "22" + taskid: c3e460e4-00aa-41b6-b84d-30d96d330cb8 + type: regular + task: + id: c3e460e4-00aa-41b6-b84d-30d96d330cb8 + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "23" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 340 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "23": + id: "23" + taskid: 3a4b0750-632d-40b8-878c-6cfe1ad64754 + type: regular + task: + id: 3a4b0750-632d-40b8-878c-6cfe1ad64754 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 495 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false "2": id: "2" taskid: ba727b14-8224-4cc4-a7a5-07eb0088482f @@ -54,6 +216,8 @@ tasks: '#none#': - "9" scriptarguments: + using: + simple: ${ANYRUNInstance} env_bitness: simple: ${inputs.env_bitness} env_locale: @@ -68,6 +232,8 @@ tasks: simple: ${inputs.obj_ext_extension} obj_url: simple: ${inputs.obj_url} + opt_automated_interactivity: + simple: ${inputs.opt_automated_interactivity} opt_network_connect: simple: ${inputs.opt_network_connect} opt_network_fakenet: @@ -92,7 +258,7 @@ tasks: { "position": { "x": 275, - "y": 560 + "y": 1000 } } note: false @@ -119,6 +285,8 @@ tasks: '#none#': - "10" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: html task_uuid: @@ -129,7 +297,7 @@ tasks: { "position": { "x": 387.5, - "y": 1085 + "y": 1525 } } note: false @@ -157,7 +325,7 @@ tasks: { "position": { "x": 70, - "y": 1785 + "y": 2225 } } note: false @@ -201,7 +369,7 @@ tasks: { "position": { "x": 162.5, - "y": 385 + "y": 825 } } note: false @@ -239,7 +407,7 @@ tasks: { "position": { "x": 50, - "y": 210 + "y": 650 } } note: false @@ -284,7 +452,7 @@ tasks: { "position": { "x": 275, - "y": 910 + "y": 1350 } } note: false @@ -312,6 +480,8 @@ tasks: '#none#': - "8" scriptarguments: + using: + simple: ${ANYRUNInstance} task_uuid: simple: ${ANYRUN_DetonateUrlWindows.TaskID} separatecontext: false @@ -320,7 +490,7 @@ tasks: { "position": { "x": 275, - "y": 735 + "y": 1175 } } note: false @@ -347,6 +517,8 @@ tasks: '#none#': - "12" scriptarguments: + using: + simple: ${ANYRUNInstance} report_format: simple: ioc task_uuid: @@ -357,7 +529,7 @@ tasks: { "position": { "x": 387.5, - "y": 1260 + "y": 1700 } } note: false @@ -390,13 +562,17 @@ tasks: accessor: id indicatorsValues: simple: ${ANYRUN.IOCs} + retry-count: + simple: "5" + retry-interval: + simple: "3" separatecontext: false continueonerrortype: "" view: |- { "position": { "x": 500, - "y": 1610 + "y": 2050 } } note: false @@ -442,7 +618,7 @@ tasks: { "position": { "x": 387.5, - "y": 1435 + "y": 1875 } } note: false @@ -460,116 +636,127 @@ view: |- }, "paper": { "dimensions": { - "height": 1795, - "width": 830, + "height": 2235, + "width": 1130, "x": 50, "y": 50 } } } inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: - key: obj_url value: simple: ${URL.Data} required: false - description: 'Target URL. Size range 5-512. Example: (http/https)://(your-link)' + description: 'The target URL. Size range: 5-512. For example, (http/https)://(your-link).' playbookInputQuery: - key: env_locale value: simple: en-US required: false - description: 'Operation system language. Use locale identifier or country name (Ex: - "en-US" or "Brazil"). Case insensitive.' + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' playbookInputQuery: - key: env_bitness value: simple: "64" required: false - description: Bitness of Operation System. Supports 32, 64 + description: 'The bitness of the operating system. Possible values are 32 or 64.' playbookInputQuery: - key: env_version value: simple: "10" required: false - description: 'Version of OS. Supports: 7, 10, 11' + description: 'The version of the operating system. Possible values are 7, 10, or 11.' playbookInputQuery: - key: env_type value: simple: complete required: false - description: Environment preset type. You can select **development** env for OS - Windows 10 x64. For all other cases, **complete** env is required + description: The environment preset type. You can select the **development** environment for Windows 10 x64. For all other cases, the **complete** environment is required. playbookInputQuery: - key: opt_network_connect value: - simple: "True" + simple: "true" required: false - description: Network connection state. + description: Whether to enable network connection. playbookInputQuery: - key: opt_network_fakenet value: - simple: "False" + simple: "false" required: false - description: FakeNet feature status. + description: Whether to enable the FakeNet feature. playbookInputQuery: - key: opt_network_tor value: - simple: "False" + simple: "false" required: false - description: TOR using. + description: Whether to use TOR. playbookInputQuery: - key: opt_network_geo value: simple: fastest required: false - description: 'Tor geo location option. Example: US, AU.' + description: 'The TOR geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_network_mitm value: - simple: "False" + simple: "false" required: false - description: HTTPS MITM proxy option. + description: Whether to use the HTTPS MITM proxy. playbookInputQuery: - key: opt_network_residential_proxy value: - simple: "False" + simple: "false" required: false - description: Residential proxy using. + description: Whether to use a residential proxy. playbookInputQuery: - key: opt_network_residential_proxy_geo value: simple: fastest required: false - description: 'Residential proxy geo location option. Example: US, AU.' + description: 'The residential proxy geolocation option, for example, US or AU.' playbookInputQuery: - key: opt_privacy_type value: simple: bylink required: false - description: 'Privacy settings. Supports: public, bylink, owner, byteam.' + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' playbookInputQuery: - key: opt_timeout value: - simple: "60" + simple: "120" + required: false + description: 'The timeout value. Size range: 10-660.' + playbookInputQuery: +- key: opt_automated_interactivity + value: + simple: "true" required: false - description: 'Timeout option. Size range: 10-660.' + description: Whether to enable automated interactivity. playbookInputQuery: - key: obj_ext_browser value: simple: Google Chrome required: false - description: 'Browser name. Supports: Google Chrome, Mozilla Firefox, Internet Explorer, - Microsoft Edge' + description: 'The browser name. Possible values are Google Chrome, Mozilla Firefox, Internet Explorer, or Microsoft Edge.' playbookInputQuery: - key: obj_ext_extension value: - simple: "True" + simple: "true" required: false - description: Change extension to valid + description: Whether to change the extension to a valid one. playbookInputQuery: inputSections: - inputs: - obj_url + - Using name: General (Inputs group) description: Generic group for inputs - inputs: @@ -589,6 +776,7 @@ inputSections: - opt_network_residential_proxy_geo - opt_privacy_type - opt_timeout + - opt_automated_interactivity name: Options description: "" - inputs: @@ -600,7 +788,28 @@ outputSections: - outputs: [] name: General (Outputs group) description: Generic group for outputs -outputs: [] +outputs: +- contextPath: ANYRUN_DetonateUrlWindows.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown quiet: true tests: - No tests (auto formatted) +marketplaces: +- xsoar +- platform +- xpanse +- xsoar_saas +- marketplacev2 +supportedModules: +- agentix +- xsiam +- edr +- cloud +- cloud_runtime_security diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_README.md index 032d016f576e..a6ee4841f2b0 100644 --- a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_README.md +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_README.md @@ -14,7 +14,9 @@ This playbook does not use any sub-playbooks. ### Scripts +* GetInstances * IsIntegrationAvailable +* Set * associateIndicatorsToIncident ### Commands @@ -29,30 +31,36 @@ This playbook does not use any sub-playbooks. | **Name** | **Description** | **Default Value** | **Required** | | --- | --- | --- | --- | -| obj_url | Target URL. Size range 5-512. Example: \(http/https\)://\(your-link\) | ${URL.Data} | Optional | -| env_locale | Operation system language. Use locale identifier or country name \(Ex: "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | -| env_bitness | Bitness of Operation System. Supports 32, 64 | 64 | Optional | -| env_version | Version of OS. Supports: 7, 10, 11 | 10 | Optional | -| env_type | Environment preset type. You can select \*\*development\*\* env for OS Windows 10 x64. For all other cases, \*\*complete\*\* env is required | complete | Optional | -| opt_network_connect | Network connection state. | True | Optional | -| opt_network_fakenet | FakeNet feature status. | False | Optional | -| opt_network_tor | TOR using. | False | Optional | -| opt_network_geo | Tor geo location option. Example: US, AU. | fastest | Optional | -| opt_network_mitm | HTTPS MITM proxy option. | False | Optional | -| opt_network_residential_proxy | Residential proxy using. | False | Optional | -| opt_network_residential_proxy_geo | Residential proxy geo location option. Example: US, AU. | fastest | Optional | -| opt_privacy_type | Privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | -| opt_timeout | Timeout option. Size range: 10-660. | 60 | Optional | -| obj_ext_browser | Browser name. Supports: Google Chrome, Mozilla Firefox, Internet Explorer, Microsoft Edge | Google Chrome | Optional | -| obj_ext_extension | Change extension to valid | True | Optional | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| obj_url | The target URL. Size range 5-512. Example: \(http/https\)://\(your-link\). | ${URL.Data} | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| env_bitness | The bitness of the operating system. Supports: 32, 64. | 64 | Optional | +| env_version | The version of the OS. Supports: 7, 10, 11. | 10 | Optional | +| env_type | The environment preset type. You can select \*\*development\*\* env for OS Windows 10 x64. For all other cases, \*\*complete\*\* env is required. | complete | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 120 | Optional | +| opt_automated_interactivity | Whether to enable automated interactivity. | True | Optional | +| obj_ext_browser | The browser name. Supports: Google Chrome, Mozilla Firefox, Internet Explorer, Microsoft Edge. | Google Chrome | Optional | +| obj_ext_extension | Whether to change the extension to a valid one. | True | Optional | ## Playbook Outputs --- -There are no outputs for this playbook. +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateUrlWindows.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | ## Playbook Image --- -![ANYRUN Detonate URL Windows](../doc_files/ANYRUN_Detonate_URL_Windows.png) +![ANYRUN Detonate Url Windows](../doc_files/ANYRUN_Detonate_URL_Windows.png) diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_for_XDR.yml b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_for_XDR.yml new file mode 100644 index 000000000000..841be2bb8d09 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_for_XDR.yml @@ -0,0 +1,850 @@ +id: ANYRUN Detonate URL Windows for XDR +version: -1 +fromversion: 6.0.0 +name: ANYRUN Detonate URL Windows for XDR +description: Submits a URL from a Cortex XDR issue to the ANY.RUN Cloud Sandbox for dynamic analysis in a Windows environment and automates malware detonation and behavior observation on Windows OS. +starttaskid: "0" +tasks: + "0": + id: "0" + taskid: 083e190e-3310-44f0-8d62-670e32ebae1e + type: start + task: + id: 083e190e-3310-44f0-8d62-670e32ebae1e + version: -1 + name: "" + iscommand: false + brand: "" + description: "" + nexttasks: + '#none#': + - "40" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 50 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "40": + id: "40" + taskid: cfc5da1c-1fdf-4951-a66f-4a85d958841f + type: condition + task: + id: cfc5da1c-1fdf-4951-a66f-4a85d958841f + version: -1 + name: Is a specific ANY.RUN instance provided? + description: Checks whether the Using playbook input was provided to determine which ANY.RUN Cloud Sandbox instance to run commands against. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "42" + "yes": + - "41" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.Using + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 220 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "41": + id: "41" + taskid: 025eb243-6077-4dcd-9c4d-b5ad9fabe735 + type: regular + task: + id: 025eb243-6077-4dcd-9c4d-b5ad9fabe735 + version: -1 + name: Set instance to use (from input) + description: Sets the selected ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + simple: ${inputs.Using} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "42": + id: "42" + taskid: 2d5f38ad-641c-4f44-a6c4-901077d4d9dc + type: regular + task: + id: 2d5f38ad-641c-4f44-a6c4-901077d4d9dc + version: -1 + name: Get active ANY.RUN Cloud Sandbox instances + description: Returns active ANY.RUN Cloud Sandbox integration instances. + scriptName: GetInstances + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "43" + scriptarguments: + brand: + simple: ANY.RUN Cloud Sandbox + instance_status: + simple: active + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 405 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "43": + id: "43" + taskid: 4593ef8f-65dc-453e-935e-ec32233af197 + type: regular + task: + id: 4593ef8f-65dc-453e-935e-ec32233af197 + version: -1 + name: Set instance to use (first active instance) + description: Sets the first active ANY.RUN integration instance in the playbook context. + scriptName: Set + type: regular + iscommand: false + brand: "" + nexttasks: + '#none#': + - "7" + scriptarguments: + key: + simple: ANYRUNInstance + value: + complex: + root: Modules + accessor: name + transformers: + - operator: atIndex + args: + index: + value: + simple: "0" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 600, + "y": 590 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "2": + id: "2" + taskid: 2c989c3b-28fc-4ff7-8869-a8f3c1390c71 + type: regular + task: + id: 2c989c3b-28fc-4ff7-8869-a8f3c1390c71 + version: -1 + name: '[ANY.RUN] Detonate URL Windows' + description: Performs URL analysis using a Windows VM in ANY.RUN Cloud Sandbox. + script: '|||anyrun-detonate-url-windows' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "9" + scriptarguments: + using: + simple: ${ANYRUNInstance} + env_bitness: + simple: ${inputs.env_bitness} + env_locale: + simple: ${inputs.env_locale} + env_type: + simple: ${inputs.env_type} + env_version: + simple: ${inputs.env_version} + obj_ext_browser: + simple: ${inputs.obj_ext_browser} + obj_ext_extension: + simple: ${inputs.obj_ext_extension} + obj_url: + simple: ${inputs.obj_url} + opt_automated_interactivity: + simple: ${inputs.opt_automated_interactivity} + opt_network_connect: + simple: ${inputs.opt_network_connect} + opt_network_fakenet: + simple: ${inputs.opt_network_fakenet} + opt_network_geo: + simple: ${inputs.opt_network_geo} + opt_network_mitm: + simple: ${inputs.opt_network_mitm} + opt_network_residential_proxy: + simple: ${inputs.opt_network_residential_proxy} + opt_network_residential_proxy_geo: + simple: ${inputs.opt_network_residential_proxy_geo} + opt_network_tor: + simple: ${inputs.opt_network_tor} + opt_privacy_type: + simple: ${inputs.opt_privacy_type} + opt_timeout: + simple: ${inputs.opt_timeout} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1100 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "3": + id: "3" + taskid: 4fc5d4eb-4702-49de-813c-27ccb0203790 + type: regular + task: + id: 4fc5d4eb-4702-49de-813c-27ccb0203790 + version: -1 + name: '[ANY.RUN] Get analysis summary' + description: Returns the analysis report summary. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "10" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: html + task_uuid: + simple: ${ANYRUN_DetonateUrlWindows.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 1655 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "5": + id: "5" + taskid: 57076a54-7a35-47ff-8ce6-832181be34dd + type: title + task: + id: 57076a54-7a35-47ff-8ce6-832181be34dd + version: -1 + name: Done + type: title + iscommand: false + brand: "" + description: "" + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 80, + "y": 2580 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "6": + id: "6" + taskid: 65536f05-f21b-40e5-800d-c8975c09e610 + type: condition + task: + id: 65536f05-f21b-40e5-800d-c8975c09e610 + version: -1 + name: Is there any URL to detonate? + description: Returns yes if the integration brand is available. Otherwise, returns no. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "2" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: inputs.obj_url + iscontext: true + right: + value: {} + continueonerrortype: "" + view: |- + { + "position": { + "x": 162.5, + "y": 915 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "7": + id: "7" + taskid: ad23a924-ee29-488b-825a-e499616b83d4 + type: condition + task: + id: ad23a924-ee29-488b-825a-e499616b83d4 + version: -1 + name: Is there an active ANY.RUN instance? + description: Returns yes if the integration brand is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "6" + scriptarguments: + brandname: + simple: ANY.RUN Cloud Sandbox + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 50, + "y": 730 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "8": + id: "8" + taskid: 0ab6b87b-c02e-408a-8715-54ffee5af727 + type: condition + task: + id: 0ab6b87b-c02e-408a-8715-54ffee5af727 + version: -1 + name: Was a malicious or suspicious verdict received? + description: Checks whether the values provided in the arguments are equal. If either of the arguments is missing, no is returned. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "3" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEqualString + left: + value: + simple: ANYRUN.SandboxAnalysisReportVerdict + iscontext: true + right: + value: + simple: No threats detected + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1470 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "9": + id: "9" + taskid: 912fa76a-940d-416b-8bbe-efef545cc163 + type: regular + task: + id: 912fa76a-940d-416b-8bbe-efef545cc163 + version: -1 + name: '[ANY.RUN] Get analysis verdict' + description: 'Returns a threat level text. Possible values are No threats detected, Suspicious activity, or Malicious activity.' + script: '|||anyrun-get-analysis-verdict' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "8" + scriptarguments: + using: + simple: ${ANYRUNInstance} + task_uuid: + simple: ${ANYRUN_DetonateUrlWindows.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 275, + "y": 1285 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "10": + id: "10" + taskid: ab0c6d57-28e4-4715-8939-f4b0be9979a2 + type: regular + task: + id: ab0c6d57-28e4-4715-8939-f4b0be9979a2 + version: -1 + name: '[ANY.RUN] Load IOCs to Threat Intel' + description: Returns the analysis report summary. + script: '|||anyrun-get-analysis-report' + type: regular + iscommand: true + brand: ANY.RUN Cloud Sandbox + nexttasks: + '#none#': + - "14" + scriptarguments: + using: + simple: ${ANYRUNInstance} + report_format: + simple: ioc + create_xdr_iocs: + simple: ${inputs.AutoAddIOCsToXDR} + task_uuid: + simple: ${ANYRUN_DetonateUrlWindows.TaskID} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 1840 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 2 + isoversize: false + isautoswitchedtoquietmode: false + "13": + id: "13" + taskid: 7efe4856-610c-4f1b-8585-aeb3fe3c1d2a + type: regular + task: + id: 7efe4856-610c-4f1b-8585-aeb3fe3c1d2a + version: -1 + name: '[XDR] Add ANY.RUN IOCs to XDR IOC rules' + description: Uploads the IOC rules to Cortex XDR. When the ioc_object is defined, disregard any other provided arguments, as the ioc_object takes precedence. Validates the indicator parameters when ioc_object is used. If `vendor_name`, `vendor_reputation`, and `vendor_reliability` are used, only a single vendor is supported. For multiple vendors, utilize an `ioc_object` in JSON format. Adding a rule with the same indicator, but with different parameters, will update the existing rule. + script: Cortex Core - IR|||core-add-indicator-rule + type: regular + iscommand: true + brand: Cortex Core - IR + nexttasks: + '#none#': + - "5" + scriptarguments: + input_format: + simple: JSON + ioc_object: + simple: ${ANYRUN.IOCDetails.ioc_object} + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 612.5, + "y": 2395 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 0 + isoversize: false + isautoswitchedtoquietmode: false + "14": + id: "14" + taskid: 5c592474-964a-40ec-82af-53f1a11f6b9d + type: condition + task: + id: 5c592474-964a-40ec-82af-53f1a11f6b9d + version: -1 + name: Are there ANY.RUN IOC details to add to XDR? + description: Checks whether the ANY.RUN IOC report returned an ioc_object that + can be uploaded to Cortex XDR IOC rules. + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "15" + separatecontext: false + conditions: + - label: "yes" + condition: + - - operator: isNotEmpty + left: + value: + simple: ANYRUN.IOCDetails.ioc_object + iscontext: true + right: + value: {} + - - operator: isEqualString + left: + value: + simple: inputs.AutoAddIOCsToXDR + iscontext: true + right: + value: + simple: "true" + continueonerrortype: "" + view: |- + { + "position": { + "x": 387.5, + "y": 2025 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false + "15": + id: "15" + taskid: 9da2dd71-d5d3-4a7d-8628-a194cae06c48 + type: condition + task: + id: 9da2dd71-d5d3-4a7d-8628-a194cae06c48 + version: -1 + name: Is Cortex Core - IR available? + description: Returns yes if Cortex Core - IR is available. Otherwise, returns no. + scriptName: IsIntegrationAvailable + type: condition + iscommand: false + brand: "" + nexttasks: + '#default#': + - "5" + "yes": + - "13" + scriptarguments: + brandname: + simple: Cortex Core - IR + separatecontext: false + continueonerrortype: "" + view: |- + { + "position": { + "x": 500, + "y": 2210 + } + } + note: false + timertriggers: [] + ignoreworker: false + skipunavailable: false + quietmode: 1 + isoversize: false + isautoswitchedtoquietmode: false +view: |- + { + "linkLabelsPosition": { + "8_5_#default#": 0.33 + }, + "paper": { + "dimensions": { + "height": 2590, + "width": 942.5, + "x": 50, + "y": 50 + } + } + } +inputs: +- key: Using + value: {} + required: false + description: |- + The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. + If left empty and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. + playbookInputQuery: null +- key: obj_url + value: + simple: ${issue.xdmtargeturl} + required: false + description: 'The target URL to submit to ANY.RUN. By default, this playbook uses the Cortex XDR normalized target URL field xdm.target.url (${issue.xdmtargeturl}). If your issue uses another URL field, such as xdm.network.http.url, override this input accordingly. Size range: 5-512. For example, (http/https)://(your-link).' + playbookInputQuery: null +- key: env_locale + value: + simple: en-US + required: false + description: 'The operating system language. Use a locale identifier or a country name, for example, en-US or Brazil. Case insensitive.' + playbookInputQuery: null +- key: env_bitness + value: + simple: "64" + required: false + description: 'The bitness of the operating system. Possible values are 32 or 64.' + playbookInputQuery: null +- key: env_version + value: + simple: "10" + required: false + description: 'The version of the operating system. Possible values are 7, 10, or 11.' + playbookInputQuery: null +- key: env_type + value: + simple: complete + required: false + description: The environment preset type. You can select the **development** environment for Windows 10 x64. For all other cases, the **complete** environment is required. + playbookInputQuery: null +- key: opt_network_connect + value: + simple: "true" + required: false + description: Whether to enable network connection. + playbookInputQuery: null +- key: opt_network_fakenet + value: + simple: "false" + required: false + description: Whether to enable the FakeNet feature. + playbookInputQuery: null +- key: opt_network_tor + value: + simple: "false" + required: false + description: Whether to use TOR. + playbookInputQuery: null +- key: opt_network_geo + value: + simple: fastest + required: false + description: 'The TOR geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_network_mitm + value: + simple: "false" + required: false + description: Whether to use the HTTPS MITM proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy + value: + simple: "false" + required: false + description: Whether to use a residential proxy. + playbookInputQuery: null +- key: opt_network_residential_proxy_geo + value: + simple: fastest + required: false + description: 'The residential proxy geolocation option, for example, US or AU.' + playbookInputQuery: null +- key: opt_privacy_type + value: + simple: bylink + required: false + description: 'The privacy settings. Possible values are public, bylink, owner, or byteam.' + playbookInputQuery: null +- key: opt_timeout + value: + simple: "120" + required: false + description: 'The timeout value. Size range: 10-660.' + playbookInputQuery: null +- key: opt_automated_interactivity + value: + simple: "true" + required: false + description: Whether to enable automated interactivity. + playbookInputQuery: null +- key: obj_ext_browser + value: + simple: Google Chrome + required: false + description: 'The browser name. Possible values are Google Chrome, Mozilla Firefox, Internet Explorer, or Microsoft Edge.' + playbookInputQuery: null +- key: obj_ext_extension + value: + simple: "true" + required: false + description: Whether to change the extension to a valid one. + playbookInputQuery: null +- key: AutoAddIOCsToXDR + value: + simple: "true" + required: false + description: 'Whether to automatically add malicious and suspicious ANY.RUN IOCs to Cortex XDR IOC rules. IOCs with an Unknown reputation are always excluded. Set to false to disable.' + playbookInputQuery: null +inputSections: +- inputs: + - AutoAddIOCsToXDR + - Using + - obj_url + name: URL and Instance Selection + description: Provide the target URL and optionally select the ANY.RUN instance with the Using input. +- inputs: + - env_locale + - env_bitness + - env_version + - env_type + name: Environment + description: "" +- inputs: + - opt_network_connect + - opt_network_fakenet + - opt_network_tor + - opt_network_geo + - opt_network_mitm + - opt_network_residential_proxy + - opt_network_residential_proxy_geo + - opt_privacy_type + - opt_timeout + - opt_automated_interactivity + name: Options + description: "" +- inputs: + - obj_ext_browser + - obj_ext_extension + name: Object + description: "" +outputSections: +- outputs: [] + name: General (Outputs group) + description: The generic group for outputs. +outputs: +- contextPath: ANYRUN_DetonateUrlWindows.TaskID + description: The ANY.RUN task UUID. + type: String +- contextPath: ANYRUN.SandboxAnalysisReportVerdict + description: The ANY.RUN verdict. + type: String +- contextPath: ANYRUN.IOCs + description: The IOCs extracted from the ANY.RUN report. + type: Unknown +- contextPath: ANYRUN.IOCDetails + description: The detailed IOC objects prepared for Cortex XDR indicator rules. + type: Unknown +quiet: true +adopted: true +marketplaces: +- platform +- marketplacev2 +supportedModules: +- edr +- xsiam diff --git a/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_for_XDR_README.md b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_for_XDR_README.md new file mode 100644 index 000000000000..168d6eb65022 --- /dev/null +++ b/Packs/ANYRUN/Playbooks/ANYRUN_Detonate_URL_Windows_for_XDR_README.md @@ -0,0 +1,65 @@ +This playbook submits a URL from a Cortex XDR issue to the ANY.RUN Cloud Sandbox for dynamic analysis in a Windows environment. It helps to automate malware detonation and behavior observation on Windows OS. + +## Dependencies + +This playbook uses the following sub-playbooks, integrations, and scripts. + +### Sub-playbooks + +This playbook does not use any sub-playbooks. + +### Integrations + +* AnyRunSandbox +* Cortex Core - IR + +### Scripts + +* GetInstances +* IsIntegrationAvailable +* Set + +### Commands + +* anyrun-detonate-url-windows +* anyrun-get-analysis-report +* anyrun-get-analysis-verdict +* core-add-indicator-rule + +## Playbook Inputs + +--- + +| **Name** | **Description** | **Default Value** | **Required** | +| --- | --- | --- | --- | +| Using | The name of the ANY.RUN Cloud Sandbox integration instance to use for running commands in this playbook. If left empty, and more than one instance is enabled, the playbook automatically selects the first active instance instead of running commands on every enabled instance. | | Optional | +| AutoAddIOCsToXDR | Whether to automatically add malicious and suspicious ANY.RUN IOCs (Unknown reputation IOCs are always excluded) to Cortex XDR IOC rules. Set to "false" to disable. | true | Optional | +| obj_url | The target URL to submit to ANY.RUN. By default, this playbook uses the Cortex XDR normalized target URL field xdm.target.url \(${issue.xdmtargeturl}\). If your issue uses another URL field, such as xdm.network.http.url, override this input accordingly. Size range 5-512. Example: \(http/https\)://\(your-link\). | ${issue.xdmtargeturl} | Optional | +| env_locale | The operating system language. Use locale identifier or country name \(for example, "en-US" or "Brazil"\). Case insensitive. | en-US | Optional | +| env_bitness | The bitness of the operating system. Supports: 32, 64. | 64 | Optional | +| env_version | The version of the OS. Supports: 7, 10, 11. | 10 | Optional | +| env_type | The environment preset type. You can select \*\*development\*\* env for OS Windows 10 x64. For all other cases, \*\*complete\*\* env is required. | complete | Optional | +| opt_network_connect | Whether to enable network connection. | True | Optional | +| opt_network_fakenet | Whether to enable the FakeNet feature. | False | Optional | +| opt_network_tor | Whether to use TOR. | False | Optional | +| opt_network_geo | The TOR geo location option, for example US, AU. | fastest | Optional | +| opt_network_mitm | Whether to use the HTTPS MITM proxy. | False | Optional | +| opt_network_residential_proxy | Whether to use a residential proxy. | False | Optional | +| opt_network_residential_proxy_geo | The residential proxy geo location option, for example US, AU. | fastest | Optional | +| opt_privacy_type | The privacy settings. Supports: public, bylink, owner, byteam. | bylink | Optional | +| opt_timeout | The timeout value. Size range: 10-660. | 120 | Optional | +| opt_automated_interactivity | Whether to enable automated interactivity. | True | Optional | +| obj_ext_browser | The browser name. Supports: Google Chrome, Mozilla Firefox, Internet Explorer, Microsoft Edge. | Google Chrome | Optional | +| obj_ext_extension | Whether to change the extension to a valid one. | True | Optional | + +## Playbook Outputs + +--- +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| ANYRUN_DetonateUrlWindows.TaskID | The ANY.RUN task UUID. | String | +| ANYRUN.SandboxAnalysisReportVerdict | The ANY.RUN verdict. | String | +| ANYRUN.IOCs | The IOCs extracted from the ANY.RUN report. | Unknown | +| ANYRUN.IOCDetails | The detailed IOC objects prepared for Cortex XDR indicator rules. | Unknown | + +![ANYRUN Detonate URL Windows for XDR](../doc_files/ANYRUN_Detonate_URL_Windows_for_XDR.png) diff --git a/Packs/ANYRUN/README.md b/Packs/ANYRUN/README.md index 656ec4d82f75..995adc0a9e6a 100644 --- a/Packs/ANYRUN/README.md +++ b/Packs/ANYRUN/README.md @@ -1,13 +1,13 @@ Note: Support for this Pack was moved to Partner on June 17, 2025. -### The Challenge +## The Challenge Modern SOCs are overwhelmed with incident alerts and lack fast, reliable ways to assess and prioritize them effectively. This leaves critical infrastructures vulnerable to attacks like ransomware and data theft, which might slip through security gaps and cause major damage and downtime. -### ANY.RUN Content Pack for Cortex XSOAR +## ANY.RUN Content Pack for Cortex XSOAR The Palo Alto Networks Cortex XSOAR and ANY.RUN integration enables SOCs to automate triage, expand threat coverage, and increase the detection rate, including hidden and missed malware. @@ -21,7 +21,7 @@ organization can: * Query threat details for IoCs, indicators of attack (IoAs), and indicators of behavior (IoBs) to enrich incident investigations. * Automate workflows using Cortex XSOAR playbooks to reduce manual workload. -### Palo Alto Networks and ANY.RUN Integrations +## Palo Alto Networks and ANY.RUN Integrations Product integrations between Palo Alto Networks and ANY.RUN include: @@ -29,7 +29,7 @@ Product integrations between Palo Alto Networks and ANY.RUN include: * [Threat Intelligence Lookup](https://any.run/threat-intelligence-lookup/?utm_source=cortex_marketplace&utm_medium=description&utm_campaign=palo_alto_networks_integration&utm_term=281025&utm_content=linktotilookuplanding) * [Threat Intelligence Feeds](https://any.run/threat-intelligence-feeds/?utm_source=cortex_marketplace&utm_medium=description&utm_campaign=palo_alto_networks_integration&utm_term=281025&utm_content=linktotifeedslanding) -### Key Benefits +## Key Benefits * Slash incident response time with automated, secure malware analysis in Cortex XSOAR® @@ -39,7 +39,15 @@ malware analysis in Cortex XSOAR® * Reduce SOC workload by automating threat triage and response tasks. * Help ensure compliance with SOC 2/GDPR via a secure, private analysis mode. -### Support +## Support This is an ANY.RUN’s supported connector. You can write to us for help with integration via [techsupport@any.run](mailto:techsupport@any.run). Contact us for a quote or demo via [this form](https://app.any.run/contact-us/?utm_source=cortex_marketplace&utm_medium=description&utm_campaign=palo_alto_networks_integration&utm_term=281025&utm_content=linktocontactus) + +## Pack Contributors + +--- + +* ANY.RUN + +Contributions are welcome and appreciated. For more info, visit our [Contribution Guide](https://xsoar.pan.dev/docs/contributing/contributing). diff --git a/Packs/ANYRUN/ReleaseNotes/2_4_0.json b/Packs/ANYRUN/ReleaseNotes/2_4_0.json new file mode 100644 index 000000000000..1ca6c1d25e67 --- /dev/null +++ b/Packs/ANYRUN/ReleaseNotes/2_4_0.json @@ -0,0 +1,4 @@ +{ + "breakingChanges": true, + "breakingChangesNotes": "Removed the obj_ext_browser argument from the anyrun-detonate-url-android command and from the ANYRUN Detonate Url Android playbook. Android detonation does not support browser selection via the API. Custom playbooks or automations that pass this argument to anyrun-detonate-url-android or use it as an input to the ANYRUN Detonate Url Android playbook must be updated to remove it." +} diff --git a/Packs/ANYRUN/ReleaseNotes/2_4_0.md b/Packs/ANYRUN/ReleaseNotes/2_4_0.md new file mode 100644 index 000000000000..869fa1237aa2 --- /dev/null +++ b/Packs/ANYRUN/ReleaseNotes/2_4_0.md @@ -0,0 +1,36 @@ + +#### Integrations + +##### ANY.RUN Cloud Sandbox + +- New: Added the *Root URL* configuration parameter (default: *any.run*), used to build the interactive analysis link and IOC descriptions instead of a hardcoded *app.any.run* host. Falls back safely to the default when left blank. +- Updated the ***anyrun-detonate-url-android*** command by removing the *obj_ext_browser* argument. This option is not supported by ANY.RUN's Android sandbox API and caused detonation to fail. +- Updated the ***anyrun-download-analysis-sample*** command so it no longer returns both the pcap and the file sample when *download_type* is *pcap*; the branches are now mutually exclusive. +- Updated the ***anyrun-get-analysis-report*** command with *report_format=summary* to now return the report as a downloaded JSON file (*anyrun_report_\.json*) instead of a JSON context output. +- Updated the Docker image to: *demisto/anyrun-sdk:1.0.0.10440656*. +- Fixed an issue with a typo in the setup instructions. + +##### ANY.RUN TI Feed + +- New: Indicators now include *tags* (from feed labels) and a *publications* field populated from the feed's external references. Domain indicators additionally populate *communitynotes* with the same reference links. +- New: Added the *Incremental Feed* configuration parameter (enabled by default) so that indicators absent from a given delta fetch are no longer incorrectly flagged as "Removed from feed". +- Updated the Docker image to: *demisto/anyrun-sdk:1.0.0.10440656*. + +##### ANY.RUN TI Lookup + +- Updated the Docker image to: *demisto/anyrun-sdk:1.0.0.10440656*. + +#### Playbooks + +##### ANYRUN Detonate Url Android + +- Updated the playbook by removing the *obj_ext_browser* input, task argument, and input section. Android detonation does not support browser selection via the API, and passing this argument caused Android URL detonation to fail. +- Updated the playbook inputs: *opt_timeout* default changed from *60* to *120*. + +##### ANYRUN Detonate Url Linux + +- Fixed an issue with the *brandname* value used by the "Is there an active ANY.RUN instance?" condition task, from *ANY.RUN Sandbox* to *ANY.RUN Cloud Sandbox*, matching the integration's actual brand name. + +##### ANYRUN Detonate Url Windows + +- Updated the playbook inputs: *opt_timeout* default changed from *60* to *120*. diff --git a/Packs/ANYRUN/ReleaseNotes/2_4_1.md b/Packs/ANYRUN/ReleaseNotes/2_4_1.md new file mode 100644 index 000000000000..fcb38c440f0a --- /dev/null +++ b/Packs/ANYRUN/ReleaseNotes/2_4_1.md @@ -0,0 +1,11 @@ + +#### Integrations + +##### ANY.RUN TI Feed + +- Updated the Docker image to: *demisto/anyrun-sdk:1.0.0.12844876*. + +##### ANY.RUN Cloud Sandbox + +- Updated the Docker image to: *demisto/anyrun-sdk:1.0.0.12844876*. + diff --git a/Packs/ANYRUN/ReleaseNotes/2_5_0.md b/Packs/ANYRUN/ReleaseNotes/2_5_0.md new file mode 100644 index 000000000000..3e307ba28100 --- /dev/null +++ b/Packs/ANYRUN/ReleaseNotes/2_5_0.md @@ -0,0 +1,74 @@ + +#### Integrations + +##### ANY.RUN Cloud Sandbox +- Added the *opt_automated_interactivity* argument (default: **True**) to the ***anyrun-detonate-file-windows*** and ***anyrun-detonate-url-windows*** commands. This argument controls whether the sandbox automatically interacts with the sample during analysis. This option is not supported on Linux or Android, so it remains hidden and defaults to **False** for the ***anyrun-detonate-file-linux***, ***anyrun-detonate-url-linux***, ***anyrun-detonate-file-android***, and ***anyrun-detonate-url-android*** commands. +- Updated the ***anyrun-get-analysis-report*** command with *report_format* set to **ioc** to return indicator details formatted for ingestion as Cortex XDR IOCs (*ANYRUN.IOCDetails*), in addition to the existing *ANYRUN.IOCs* output. Building and returning *ANYRUN.IOCDetails* now requires the new *create_xdr_iocs* argument (default: **False**), because returning it unconditionally could delay the availability of newly created indicators for immediate association with the incident. +- Excluded IOCs with an **Unknown** reputation from *ANYRUN.IOCDetails*, so only malicious and suspicious IOCs can be pushed to Cortex XDR IOC rules. +- Fixed an issue where the ***anyrun-get-analysis-report*** command failed when the *report_format* argument was not provided. It now defaults to **summary**. +- Fixed an issue where the ***anyrun-get-analysis-report*** command with *report_format* set to **ioc** returned no result when the analysis report contained no IOCs. It now returns a *"No IOCs found"* message. +- Fixed an issue where the *ANYRUN.IOCDetails* output of the ***anyrun-get-analysis-report*** command sent an invalid reliability code and a hardcoded vendor reputation. It now sends the correct reliability code and the actual reputation of each indicator. + +##### ANY.RUN TI Feed + +##### ANY.RUN TI Lookup +- Updated the Docker image to: *demisto/anyrun-sdk:1.0.0.12844876*. + + +#### Playbooks + +##### ANYRUN Detonate File Android + +- Added the *Using* playbook input to select which *ANY.RUN Cloud Sandbox* instance to run the playbook against. If left empty, the first active instance is automatically used. +- Fixed an issue where the *Associate IOCs to incident* task could fail with an *"Indicators do not exist"* error due to a race condition. The task now automatically retries a few times before failing. + +##### ANYRUN Detonate File Android for XDR + +- New: This playbook submits a file from a Cortex XDR issue to the *ANY.RUN Cloud Sandbox* for dynamic analysis in an Android environment. It automates malware detonation and behavior observation on Android OS. It also adds malicious and suspicious IOCs to Cortex XDR IOC rules. The *AutoAddIOCsToXDR* playbook input (default: **True**) controls this behavior. + +##### ANYRUN Detonate File Linux + +- Added the *Using* playbook input to select which *ANY.RUN Cloud Sandbox* instance to run the playbook against. If left empty, the first active instance is automatically used. +- Fixed an issue where the *Associate IOCs to incident* task could fail with an *"Indicators do not exist"* error due to a race condition. The task now automatically retries a few times before failing. + +##### ANYRUN Detonate File Linux for XDR + +- New: This playbook submits a file retrieved from a Cortex XDR endpoint to the *ANY.RUN Cloud Sandbox* for dynamic analysis in a Linux environment. It automates malware detonation and behavior observation on Linux OS. It also adds malicious and suspicious IOCs to Cortex XDR IOC rules. The *AutoAddIOCsToXDR* playbook input (default: **True**) controls this behavior. + +##### ANYRUN Detonate File Windows + +- Added the *Using* playbook input to select which *ANY.RUN Cloud Sandbox* instance to run the playbook against. If left empty, the first active instance is automatically used. +- Added the *opt_automated_interactivity* playbook input (default: **True**) to control whether the sandbox automatically interacts with the sample during analysis. +- Fixed an issue where the *Associate IOCS to Incident* task could fail with an *"Indicators do not exist"* error due to a race condition. The task now automatically retries a few times before failing. + +##### ANYRUN Detonate File Windows for XDR + +- New: This playbook submits a file retrieved from a Cortex XDR endpoint to the *ANY.RUN Cloud Sandbox* for dynamic analysis in a Windows environment. It automates malware detonation and behavior observation on Windows OS. It also adds malicious and suspicious IOCs to Cortex XDR IOC rules. The *AutoAddIOCsToXDR* playbook input (default: **True**) controls this behavior. + +##### ANYRUN Detonate Url Android + +- Added the *Using* playbook input to select which *ANY.RUN Cloud Sandbox* instance to run the playbook against. If left empty, the first active instance is automatically used. +- Fixed an issue where the *Associate IOCs to incident* task could fail with an *"Indicators do not exist"* error due to a race condition. The task now automatically retries a few times before failing. + +##### ANYRUN Detonate URL Android for XDR + +- New: This playbook submits a URL from a Cortex XDR issue to the *ANY.RUN Cloud Sandbox* for dynamic analysis in an Android environment. It automates malware detonation and behavior observation on Android OS. It also adds malicious and suspicious IOCs to Cortex XDR IOC rules. The *AutoAddIOCsToXDR* playbook input (default: **True**) controls this behavior. + +##### ANYRUN Detonate Url Linux + +- Added the *Using* playbook input to select which *ANY.RUN Cloud Sandbox* instance to run the playbook against. If left empty, the first active instance is automatically used. +- Fixed an issue where the *Associate IOCs to incident* task could fail with an *"Indicators do not exist"* error due to a race condition. The task now automatically retries a few times before failing. + +##### ANYRUN Detonate URL Linux for XDR + +- New: This playbook submits a URL from a Cortex XDR issue to the *ANY.RUN Cloud Sandbox* for dynamic analysis in a Linux environment. It automates malware detonation and behavior observation on Linux OS. It also adds malicious and suspicious IOCs to Cortex XDR IOC rules. The *AutoAddIOCsToXDR* playbook input (default: **True**) controls this behavior. + +##### ANYRUN Detonate Url Windows + +- Added the *Using* playbook input to select which *ANY.RUN Cloud Sandbox* instance to run the playbook against. If left empty, the first active instance is automatically used. +- Added the *opt_automated_interactivity* playbook input (default: **True**) to control whether the sandbox automatically interacts with the sample during analysis. +- Fixed an issue where the *Associate IOCs to incident* task could fail with an *"Indicators do not exist"* error due to a race condition. The task now automatically retries a few times before failing. + +##### ANYRUN Detonate URL Windows for XDR + +- New: This playbook submits a URL from a Cortex XDR issue to the *ANY.RUN Cloud Sandbox* for dynamic analysis in a Windows environment. It automates malware detonation and behavior observation on Windows OS. It also adds malicious and suspicious IOCs to Cortex XDR IOC rules. The *AutoAddIOCsToXDR* playbook input (default: **True**) controls this behavior. diff --git a/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Android_for_XDR.png b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Android_for_XDR.png new file mode 100644 index 000000000000..58ef4a31170c Binary files /dev/null and b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Android_for_XDR.png differ diff --git a/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Linux_for_XDR.png b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Linux_for_XDR.png new file mode 100644 index 000000000000..078bcd09001e Binary files /dev/null and b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Linux_for_XDR.png differ diff --git a/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Windows_for_XDR.png b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Windows_for_XDR.png new file mode 100644 index 000000000000..2182cf1ddd1c Binary files /dev/null and b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_File_Windows_for_XDR.png differ diff --git a/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Android_for_XDR.png b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Android_for_XDR.png new file mode 100644 index 000000000000..3592f90022cd Binary files /dev/null and b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Android_for_XDR.png differ diff --git a/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Linux_for_XDR.png b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Linux_for_XDR.png new file mode 100644 index 000000000000..10b7e08a6fbb Binary files /dev/null and b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Linux_for_XDR.png differ diff --git a/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Windows_for_XDR.png b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Windows_for_XDR.png new file mode 100644 index 000000000000..4e78de9bf75b Binary files /dev/null and b/Packs/ANYRUN/doc_files/ANYRUN_Detonate_URL_Windows_for_XDR.png differ diff --git a/Packs/ANYRUN/pack_metadata.json b/Packs/ANYRUN/pack_metadata.json index fee1316e0a22..01b19296ee0d 100644 --- a/Packs/ANYRUN/pack_metadata.json +++ b/Packs/ANYRUN/pack_metadata.json @@ -2,7 +2,7 @@ "name": "ANY.RUN", "description": "Empowers SOC teams with a Cloud Sandbox for real-time malware analysis, Threat Intelligence Lookup, and high-quality feeds to enhance detection and threat coverage.", "support": "partner", - "currentVersion": "2.3.5", + "currentVersion": "2.5.0", "author": "ANY.RUN", "url": "https://any.run/", "email": "techsupport@any.run", diff --git a/Packs/AWS-AccessAnalyzer/Integrations/AWS-AccessAnalyzer/AWS-AccessAnalyzer.yml b/Packs/AWS-AccessAnalyzer/Integrations/AWS-AccessAnalyzer/AWS-AccessAnalyzer.yml index 37c98dbd8fbe..0e59cd8bf2c8 100644 --- a/Packs/AWS-AccessAnalyzer/Integrations/AWS-AccessAnalyzer/AWS-AccessAnalyzer.yml +++ b/Packs/AWS-AccessAnalyzer/Integrations/AWS-AccessAnalyzer/AWS-AccessAnalyzer.yml @@ -61,6 +61,13 @@ configuration: type: 13 section: Collect required: false +- display: Incidents Fetch Interval + name: incidentFetchInterval + defaultvalue: '1' + required: false + type: 19 + section: Collect + advanced: true - display: Access Key name: credentials type: 9 @@ -330,7 +337,7 @@ script: name: roleSessionDuration description: Updates findings with the new values provided in the request. name: aws-access-analyzer-update-findings - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 isfetch: true runonce: false script: '-' diff --git a/Packs/AWS-AccessAnalyzer/ReleaseNotes/1_1_45.md b/Packs/AWS-AccessAnalyzer/ReleaseNotes/1_1_45.md new file mode 100644 index 000000000000..4058c993f053 --- /dev/null +++ b/Packs/AWS-AccessAnalyzer/ReleaseNotes/1_1_45.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### AWS - AccessAnalyzer + +- Updated the Fetch Incidents configuration parameters. diff --git a/Packs/AWS-AccessAnalyzer/ReleaseNotes/1_1_46.md b/Packs/AWS-AccessAnalyzer/ReleaseNotes/1_1_46.md new file mode 100644 index 000000000000..f0722f241b57 --- /dev/null +++ b/Packs/AWS-AccessAnalyzer/ReleaseNotes/1_1_46.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - AccessAnalyzer + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-AccessAnalyzer/pack_metadata.json b/Packs/AWS-AccessAnalyzer/pack_metadata.json index f00316f04445..3a8f4bf99e03 100644 --- a/Packs/AWS-AccessAnalyzer/pack_metadata.json +++ b/Packs/AWS-AccessAnalyzer/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - AccessAnalyzer", "description": "Amazon Web Services IAM Access Analyzer", "support": "xsoar", - "currentVersion": "1.1.44", + "currentVersion": "1.1.46", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-Athena/Integrations/AWS-Athena/AWS-Athena.yml b/Packs/AWS-Athena/Integrations/AWS-Athena/AWS-Athena.yml index 7c981120e02f..bcc3a5b6f0e8 100644 --- a/Packs/AWS-Athena/Integrations/AWS-Athena/AWS-Athena.yml +++ b/Packs/AWS-Athena/Integrations/AWS-Athena/AWS-Athena.yml @@ -431,7 +431,7 @@ script: - contextPath: AWS.Athena.QueryResults description: List of query results. type: List - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 subtype: python3 tests: - No tests diff --git a/Packs/AWS-Athena/ReleaseNotes/2_0_13.md b/Packs/AWS-Athena/ReleaseNotes/2_0_13.md new file mode 100644 index 000000000000..2677693ef9c7 --- /dev/null +++ b/Packs/AWS-Athena/ReleaseNotes/2_0_13.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - Athena + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-Athena/pack_metadata.json b/Packs/AWS-Athena/pack_metadata.json index 531f5c061518..7526b9b4182c 100644 --- a/Packs/AWS-Athena/pack_metadata.json +++ b/Packs/AWS-Athena/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - Athena", "description": "Amazon Web Services Athena", "support": "xsoar", - "currentVersion": "2.0.12", + "currentVersion": "2.0.13", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-CloudTrail/Integrations/AWS-CloudTrail/AWS-CloudTrail.yml b/Packs/AWS-CloudTrail/Integrations/AWS-CloudTrail/AWS-CloudTrail.yml index 57af66be9671..860cfcba0f6b 100644 --- a/Packs/AWS-CloudTrail/Integrations/AWS-CloudTrail/AWS-CloudTrail.yml +++ b/Packs/AWS-CloudTrail/Integrations/AWS-CloudTrail/AWS-CloudTrail.yml @@ -432,7 +432,7 @@ script: - contextPath: AWS.CloudTrail.TrailStatus.LatestDigestDeliveryError description: Displays any Amazon S3 error that CloudTrail encountered when attempting to deliver a digest file to the designated bucket. type: string - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 runonce: false script: '' subtype: python3 diff --git a/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail/AWSCloudTrail.yml b/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail/AWSCloudTrail.yml index d6bb68ef0043..16adf1577ebe 100644 --- a/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail/AWSCloudTrail.yml +++ b/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail/AWSCloudTrail.yml @@ -1,4 +1,5 @@ fromversion: 8.7.0 +toversion: 8.14.0 id: Amazon_AWS_CloudTrail_ModelingRule name: Amazon AWS CloudTrail Modeling Rule rules: '' diff --git a/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11.xif b/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11.xif new file mode 100644 index 000000000000..96662e813858 --- /dev/null +++ b/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11.xif @@ -0,0 +1,55 @@ +[MODEL: dataset = amazon_aws_raw] +/* + CloudTrail Audit Logs + AWS event schema version: 1.11 + https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-record-contents.html +*/ +filter _log_type = "Cloud Audit Log" +| alter // initializations + resource_arn = arraystring(arraymap(resources -> [], "@element" -> ARN), ","), + resource_account_id = arraystring(arraymap(resources -> [], "@element" -> accountId), ","), + resource_name = arraystring(arraymap(resources -> [], arrayindex(regextract("@element" -> ARN, "/([^\/]+)$"), -1)), ","), // extract last segment from ARN after last / + resource_type = arraystring(arraymap(resources -> [], "@element" -> type), ","), + user_identity_arn = userIdentity -> arn, + user_identity_type = userIdentity -> type, + source_ipv4 = if(sourceIPAddress ~= "(?:\d{1,3}\.){3}\d{1,3}", sourceIPAddress), // The IP address that the request was made from. For actions that originate from the service console, the address reported is for the underlying customer resource, not the console web server. + source_ipv6 = if(sourceIPAddress ~= "(?:[a-fA-F\d]{0,4}\:){2,7}[a-fA-F\d]{0,4}", sourceIPAddress) +| alter // XDM Mappings + xdm.event.description = object_create("requestParameters", requestParameters, "additionalEventData", additionalEventData, "responseElements", responseElements), + xdm.event.id = eventId, + xdm.event.operation_sub_type = eventName, + xdm.event.original_event_type = eventType, + xdm.event.outcome = if(errorCode = null, XDM_CONST.OUTCOME_SUCCESS, XDM_CONST.OUTCOME_FAILED), + xdm.event.outcome_reason = if(errorMessage != null and errorMessage != "", concat(errorCode, ": ", errorMessage), errorCode), + xdm.event.type = eventName, + xdm.network.session_id = requestID, + xdm.network.tls.cipher = tlsDetails -> cipherSuite, + xdm.network.tls.protocol_version = tlsDetails -> tlsVersion, + xdm.observer.content_version = eventVersion, + xdm.observer.name = eventSource, + xdm.observer.type = eventCategory, + xdm.session_context_id = sharedEventID, + xdm.source.cloud.project_id = userIdentity -> accountId, + xdm.source.cloud.provider = XDM_CONST.CLOUD_PROVIDER_AWS, + xdm.source.host.device_id = vpcEndpointId, + xdm.source.host.ipv4_public_addresses = arraymap(split(source_ipv4, ", "), if(not incidr(trim("@element"), "10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 127.0.0.0/8, 169.254.0.0/16, 100.64.0.0/10"),trim("@element"))), + xdm.source.host.fqdn = if(source_ipv4 = null and source_ipv6 = null and sourceIPAddress !~= "AWS Internal", sourceIPAddress), // For services in AWS, only the DNS name is displayed. + xdm.source.ipv4 = source_ipv4, + xdm.source.ipv6 = source_ipv6, + xdm.source.user_agent = userAgent, + xdm.source.user.groups = if(user_identity_type in ("Role", "AssumedRole"), arraycreate(userIdentity -> sessionContext.sessionIssuer.userName)), + xdm.source.identity.groups = if(user_identity_type in ("Role", "AssumedRole"), arraycreate(userIdentity -> sessionContext.sessionIssuer.userName)), + xdm.source.user.identifier = coalesce(userIdentity -> principalId, userIdentity -> onBehalfOf.userId, user_identity_arn), + xdm.source.identity.identifier = coalesce(userIdentity -> principalId, userIdentity -> onBehalfOf.userId, user_identity_arn), + xdm.source.user.username = coalesce(userIdentity -> userName, userIdentity -> sessionContext.sessionIssuer.userName), + xdm.source.identity.username = coalesce(userIdentity -> userName, userIdentity -> sessionContext.sessionIssuer.userName), + xdm.source.user.user_type = if(user_identity_type = "Unknown", XDM_CONST.IDENTITY_TYPE_UNKNOWN, user_identity_type), + xdm.source.identity.user_type = if(user_identity_type = "Unknown", XDM_CONST.IDENTITY_TYPE_UNKNOWN, user_identity_type), + xdm.target.cloud.project_id = if(resource_account_id != null and resource_account_id != "", resource_account_id, coalesce(recipientAccountId, vpcEndpointAccountId)), + xdm.target.cloud.provider = XDM_CONST.CLOUD_PROVIDER_AWS, + xdm.target.cloud.region = awsRegion, + xdm.target.host.fqdn = coalesce(requestParameters -> Host, tlsDetails -> clientProvidedHostHeader), + xdm.target.host.hostname = coalesce(requestParameters -> Host, tlsDetails -> clientProvidedHostHeader), + xdm.target.resource.id = if(resource_arn != "", resource_arn), + xdm.target.resource.name = if(resource_name != null and resource_name != "", resource_name, resource_arn), + xdm.target.resource.type = if(resource_type != "", resource_type); \ No newline at end of file diff --git a/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11.yml b/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11.yml new file mode 100644 index 000000000000..6294afbf0e1e --- /dev/null +++ b/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: Amazon_AWS_CloudTrail_ModelingRule +name: Amazon AWS CloudTrail Modeling Rule +rules: '' +schema: '' +tags: +supportedModules: +- xsiam diff --git a/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11_schema.json b/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11_schema.json new file mode 100644 index 000000000000..ec814b8e9e3c --- /dev/null +++ b/Packs/AWS-CloudTrail/ModelingRules/AWSCloudTrail_2_11/AWSCloudTrail_2_11_schema.json @@ -0,0 +1,104 @@ +{ + "amazon_aws_raw": { + "_log_type": { + "type": "string", + "is_array": false + }, + "additionalEventData": { + "type": "string", + "is_array": false + }, + "awsRegion": { + "type": "string", + "is_array": false + }, + "errorCode": { + "type": "string", + "is_array": false + }, + "errorMessage": { + "type": "string", + "is_array": false + }, + "eventCategory": { + "type": "string", + "is_array": false + }, + "eventID": { + "type": "string", + "is_array": false + }, + "eventName": { + "type": "string", + "is_array": false + }, + "eventSource": { + "type": "string", + "is_array": false + }, + "eventType": { + "type": "string", + "is_array": false + }, + "eventVersion": { + "type": "string", + "is_array": false + }, + "recipientAccountId": { + "type": "string", + "is_array": false + }, + "requestID": { + "type": "string", + "is_array": false + }, + "requestParameters": { + "type": "string", + "is_array": false + }, + "responseElements": { + "type": "string", + "is_array": false + }, + "resources": { + "type": "string", + "is_array": false + }, + "sharedEventID": { + "type": "string", + "is_array": false + }, + "sourceIPAddress": { + "type": "string", + "is_array": false + }, + "tlsDetails": { + "type": "string", + "is_array": false + }, + "userAgent": { + "type": "string", + "is_array": false + }, + "userIdentity": { + "type": "string", + "is_array": false + }, + "vpcEndpointAccountId": { + "type": "string", + "is_array": false + }, + "vpcEndpointId": { + "type": "string", + "is_array": false + }, + "eventTime": { + "type": "datetime", + "is_array": false + }, + "start": { + "type": "string", + "is_array": false + } + } +} \ No newline at end of file diff --git a/Packs/AWS-CloudTrail/ReleaseNotes/1_1_17.md b/Packs/AWS-CloudTrail/ReleaseNotes/1_1_17.md new file mode 100644 index 000000000000..7b1fb360d423 --- /dev/null +++ b/Packs/AWS-CloudTrail/ReleaseNotes/1_1_17.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: AWS-CloudTrail Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/AWS-CloudTrail/ReleaseNotes/1_1_18.md b/Packs/AWS-CloudTrail/ReleaseNotes/1_1_18.md new file mode 100644 index 000000000000..79198b55ef14 --- /dev/null +++ b/Packs/AWS-CloudTrail/ReleaseNotes/1_1_18.md @@ -0,0 +1,4 @@ + +## AWS-CloudTrail + +- Documentation and metadata improvements. diff --git a/Packs/AWS-CloudTrail/ReleaseNotes/1_1_19.md b/Packs/AWS-CloudTrail/ReleaseNotes/1_1_19.md new file mode 100644 index 000000000000..334530766499 --- /dev/null +++ b/Packs/AWS-CloudTrail/ReleaseNotes/1_1_19.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - CloudTrail + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-CloudTrail/ReleaseNotes/1_1_20.md b/Packs/AWS-CloudTrail/ReleaseNotes/1_1_20.md new file mode 100644 index 000000000000..f619bfbc7153 --- /dev/null +++ b/Packs/AWS-CloudTrail/ReleaseNotes/1_1_20.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: Amazon AWS CloudTrail Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/AWS-CloudTrail/pack_metadata.json b/Packs/AWS-CloudTrail/pack_metadata.json index 93eb62302da6..c0fbc2c60c18 100644 --- a/Packs/AWS-CloudTrail/pack_metadata.json +++ b/Packs/AWS-CloudTrail/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - CloudTrail", "description": "Amazon Web Services CloudTrail.", "support": "xsoar", - "currentVersion": "1.1.16", + "currentVersion": "1.1.20", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-EC2/ReleaseNotes/1_4_33.md b/Packs/AWS-EC2/ReleaseNotes/1_4_33.md new file mode 100644 index 000000000000..1d9109a0b698 --- /dev/null +++ b/Packs/AWS-EC2/ReleaseNotes/1_4_33.md @@ -0,0 +1,11 @@ + +#### Scripts + +##### AwsEC2GetPublicSGRules + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### AwsEC2SyncAccounts + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + diff --git a/Packs/AWS-EC2/ReleaseNotes/1_4_34.md b/Packs/AWS-EC2/ReleaseNotes/1_4_34.md new file mode 100644 index 000000000000..97148ac2fd0f --- /dev/null +++ b/Packs/AWS-EC2/ReleaseNotes/1_4_34.md @@ -0,0 +1,3 @@ +## AWS - EC2 + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AWS-EC2/Scripts/AwsEC2GetPublicSGRules/AwsEC2GetPublicSGRules.yml b/Packs/AWS-EC2/Scripts/AwsEC2GetPublicSGRules/AwsEC2GetPublicSGRules.yml index ce88c41cc48f..386eb9eed41e 100644 --- a/Packs/AWS-EC2/Scripts/AwsEC2GetPublicSGRules/AwsEC2GetPublicSGRules.yml +++ b/Packs/AWS-EC2/Scripts/AwsEC2GetPublicSGRules/AwsEC2GetPublicSGRules.yml @@ -62,6 +62,6 @@ tags: timeout: '0' type: python subtype: python3 -dockerimage: demisto/python3:3.12.13.10116658 +dockerimage: demisto/python3:3.12.14.13053055 runas: DBotWeakRole fromversion: 5.0.0 diff --git a/Packs/AWS-EC2/Scripts/AwsEC2SyncAccounts/AwsEC2SyncAccounts.yml b/Packs/AWS-EC2/Scripts/AwsEC2SyncAccounts/AwsEC2SyncAccounts.yml index 6a98354da6dd..66f69f29c1de 100644 --- a/Packs/AWS-EC2/Scripts/AwsEC2SyncAccounts/AwsEC2SyncAccounts.yml +++ b/Packs/AWS-EC2/Scripts/AwsEC2SyncAccounts/AwsEC2SyncAccounts.yml @@ -23,7 +23,7 @@ tags: timeout: '0' type: python subtype: python3 -dockerimage: demisto/python3:3.12.13.10116658 +dockerimage: demisto/python3:3.12.14.13053055 runas: DBotWeakRole fromversion: 6.10.0 tests: diff --git a/Packs/AWS-EC2/pack_metadata.json b/Packs/AWS-EC2/pack_metadata.json index 5890dec01302..62628dfc3229 100644 --- a/Packs/AWS-EC2/pack_metadata.json +++ b/Packs/AWS-EC2/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - EC2", "description": "Amazon Web Services Elastic Compute Cloud (EC2)", "support": "xsoar", - "currentVersion": "1.4.32", + "currentVersion": "1.4.34", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-EKS/Integrations/AWSEKS/AWSEKS.yml b/Packs/AWS-EKS/Integrations/AWSEKS/AWSEKS.yml index 6de449ba2fc1..ae9f9eb971e5 100644 --- a/Packs/AWS-EKS/Integrations/AWSEKS/AWSEKS.yml +++ b/Packs/AWS-EKS/Integrations/AWSEKS/AWSEKS.yml @@ -539,7 +539,7 @@ script: script: '-' type: python subtype: python3 - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 fromversion: 6.9.0 tests: - AWSEKS-Test diff --git a/Packs/AWS-EKS/ReleaseNotes/1_0_15.md b/Packs/AWS-EKS/ReleaseNotes/1_0_15.md new file mode 100644 index 000000000000..712bcf8c6f7d --- /dev/null +++ b/Packs/AWS-EKS/ReleaseNotes/1_0_15.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS-EKS + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-EKS/pack_metadata.json b/Packs/AWS-EKS/pack_metadata.json index d58a128c775a..05663c978449 100644 --- a/Packs/AWS-EKS/pack_metadata.json +++ b/Packs/AWS-EKS/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - EKS", "description": "The AWS EKS integration allows for the management and operation of Amazon Elastic Kubernetes Service (EKS) clusters.", "support": "xsoar", - "currentVersion": "1.0.14", + "currentVersion": "1.0.15", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-Enrichment-Remediation/ReleaseNotes/1_1_34.md b/Packs/AWS-Enrichment-Remediation/ReleaseNotes/1_1_34.md new file mode 100644 index 000000000000..d14d93d4bf65 --- /dev/null +++ b/Packs/AWS-Enrichment-Remediation/ReleaseNotes/1_1_34.md @@ -0,0 +1,11 @@ + +#### Scripts + +##### AWSRecreateSG + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### AWSAccountHierarchy + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + diff --git a/Packs/AWS-Enrichment-Remediation/ReleaseNotes/1_1_35.md b/Packs/AWS-Enrichment-Remediation/ReleaseNotes/1_1_35.md new file mode 100644 index 000000000000..ea470956433d --- /dev/null +++ b/Packs/AWS-Enrichment-Remediation/ReleaseNotes/1_1_35.md @@ -0,0 +1,3 @@ +## AWS Enrichment and Remediation + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AWS-Enrichment-Remediation/Scripts/AWSAccountHierarchy/AWSAccountHierarchy.yml b/Packs/AWS-Enrichment-Remediation/Scripts/AWSAccountHierarchy/AWSAccountHierarchy.yml index 31f74028222d..3a39de16d53f 100644 --- a/Packs/AWS-Enrichment-Remediation/Scripts/AWSAccountHierarchy/AWSAccountHierarchy.yml +++ b/Packs/AWS-Enrichment-Remediation/Scripts/AWSAccountHierarchy/AWSAccountHierarchy.yml @@ -6,7 +6,7 @@ comment: Determine AWS account hierarchy by looking up parent objects until the commonfields: id: AWSAccountHierarchy version: -1 -dockerimage: demisto/python3:3.12.13.10116658 +dockerimage: demisto/python3:3.12.14.13053055 enabled: true engineinfo: {} name: AWSAccountHierarchy diff --git a/Packs/AWS-Enrichment-Remediation/Scripts/AWSRecreateSG/AWSRecreateSG.yml b/Packs/AWS-Enrichment-Remediation/Scripts/AWSRecreateSG/AWSRecreateSG.yml index 559b45a43ae9..a184cdb134bf 100644 --- a/Packs/AWS-Enrichment-Remediation/Scripts/AWSRecreateSG/AWSRecreateSG.yml +++ b/Packs/AWS-Enrichment-Remediation/Scripts/AWSRecreateSG/AWSRecreateSG.yml @@ -51,7 +51,7 @@ dependson: - AWS - EC2|||aws-ec2-authorize-security-group-egress-rule - AWS - EC2|||aws-ec2-revoke-security-group-ingress-rule - AWS - EC2|||aws-ec2-revoke-security-group-egress-rule -dockerimage: demisto/python3:3.12.13.10116658 +dockerimage: demisto/python3:3.12.14.13053055 enabled: true name: AWSRecreateSG runas: DBotWeakRole diff --git a/Packs/AWS-Enrichment-Remediation/pack_metadata.json b/Packs/AWS-Enrichment-Remediation/pack_metadata.json index cf8d3f65fded..d3bac92fba9c 100644 --- a/Packs/AWS-Enrichment-Remediation/pack_metadata.json +++ b/Packs/AWS-Enrichment-Remediation/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS Enrichment and Remediation", "description": "Playbooks using multiple AWS content packs for enrichment and remediation purposes", "support": "xsoar", - "currentVersion": "1.1.33", + "currentVersion": "1.1.35", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-GuardDuty/.pack-ignore b/Packs/AWS-GuardDuty/.pack-ignore index 459381d1be37..6da161a77844 100644 --- a/Packs/AWS-GuardDuty/.pack-ignore +++ b/Packs/AWS-GuardDuty/.pack-ignore @@ -1,5 +1,5 @@ [file:AWSGuardDuty.yml] -ignore=IN126,BA108,BA109 +ignore=IN126,BA108,BA109,BA129 [known_words] gd diff --git a/Packs/AWS-GuardDuty/.secrets-ignore b/Packs/AWS-GuardDuty/.secrets-ignore index e093c0f393c6..bcfe50567e4a 100644 --- a/Packs/AWS-GuardDuty/.secrets-ignore +++ b/Packs/AWS-GuardDuty/.secrets-ignore @@ -1 +1,4 @@ -https://docs.aws.amazon.com/guardduty \ No newline at end of file +https://docs.aws.amazon.com/guardduty +https://s3.eu-central-1.amazonaws.com +https://s3.us-west-2.amazonaws.com +https://s3.amazonaws.com diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.py b/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.py index c83ef8a8de31..20f443a5b919 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.py +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.py @@ -22,6 +22,7 @@ DATE_FORMAT = "%Y-%m-%dT%H:%M:%S.%fZ" MAX_RESULTS_RESPONSE = 50 +MAX_TAGS = 50 class DatetimeEncoder(json.JSONEncoder): @@ -34,6 +35,39 @@ def default(self, obj: Any): # pylint: disable=E0202 return json.JSONEncoder.default(self, obj) +def parse_tag_field_to_dict(tags_string: str) -> dict: + """ + Parses a list representation of key and value with the form of 'key=,value=' into a + string-to-string map suitable for the GuardDuty API Tags parameter. You can specify up to 50 tags + per resource. The value may contain commas; only the ";" character separates pairs. + + Args: + tags_string: The name and value list, e.g. "key=Environment,value=prod;key=Team,value=sec". + + Returns: + A dict of the form {"": ""}, e.g. {"Environment": "prod", "Team": "sec"}. + """ + tags: dict[str, str] = {} + list_tags = argToList(tags_string, separator=";") + if len(list_tags) > MAX_TAGS: + demisto.debug( + f"[AWSGuardDuty] parse_tag_field_to_dict: The current number of tags is {len(list_tags)}. Number of tags exceeds " + f"{MAX_TAGS}, parsing only the first {MAX_TAGS} tags." + ) + list_tags = list_tags[0:MAX_TAGS] + # According to the AWS Tag restrictions docs. + regex = re.compile(r"^key=([a-zA-Z0-9\s+\-=._:/@]{1,128}),value=(.{0,256})$", flags=re.UNICODE) + for tag in list_tags: + match_tag = regex.match(tag) + if match_tag is None: + raise ValueError( + f"Could not parse field: {tag}. Please make sure you provided like so: key=abc,value=123;key=fed,value=456" + ) + tags[match_tag.group(1)] = match_tag.group(2) + demisto.debug(f"[AWSGuardDuty] parse_tag_field_to_dict: parsed {len(tags)} tag(s) with keys {list(tags.keys())}.") + return tags + + def create_detector(client: "GuardDutyClient", args: dict) -> CommandResults: """ Creates a single Amazon GuardDuty detector. @@ -356,6 +390,189 @@ def update_threat_intel_set(client: "GuardDutyClient", args: dict): raise Exception(f"Failed updating ThreatIntel set {args.get('threatIntelSetId')}. Response was: {response}") +def create_threat_entity_set(client: "GuardDutyClient", args: dict) -> CommandResults: + detector_id = args.get("detectorId") + demisto.debug( + f"[AWSGuardDuty] create_threat_entity_set: started for Detector {detector_id}, " + f"name={args.get('name')}, format={args.get('format')}, activate={args.get('activate')}, " + f"location_provided={bool(args.get('location'))}, " + f"expectedBucketOwner_provided={bool(args.get('expectedBucketOwner'))}, " + f"tags_provided={bool(args.get('tags'))}." + ) + tags = args.get("tags") + kwargs = remove_empty_elements( + { + "DetectorId": detector_id, + "Activate": arg_to_bool_or_none(args.get("activate")), + "Format": args.get("format"), + "Location": args.get("location"), + "Name": args.get("name"), + "ExpectedBucketOwner": args.get("expectedBucketOwner"), + "Tags": parse_tag_field_to_dict(tags) if tags else None, + } + ) + + demisto.debug( + f"[AWSGuardDuty] create_threat_entity_set: calling create_threat_entity_set for Detector " + f"{detector_id} with request fields: {list(kwargs.keys())}." + ) + response = client.create_threat_entity_set(**kwargs) + demisto.debug(f"[AWSGuardDuty] create_threat_entity_set: API response keys: {list(response.keys())}.") + + data = {"DetectorId": detector_id, "ThreatEntitySetId": response.get("ThreatEntitySetId")} + demisto.debug( + f"[AWSGuardDuty] create_threat_entity_set: created Threat Entity Set {data['ThreatEntitySetId']} " + f"for Detector {detector_id}. Returning CommandResults." + ) + + readable_output = tableToMarkdown("AWS GuardDuty ThreatEntity Set", data) + return CommandResults( + readable_output=readable_output, + outputs=data, + outputs_prefix="AWS.GuardDuty.ThreatEntitySet", + outputs_key_field="ThreatEntitySetId", + ) + + +def update_threat_entity_set(client: "GuardDutyClient", args: dict) -> CommandResults: + threat_entity_set_id = args.get("threatEntitySetId") + demisto.debug( + f"[AWSGuardDuty] update_threat_entity_set: started for Threat Entity Set {threat_entity_set_id} " + f"of Detector {args.get('detectorId')}, activate={args.get('activate')}, " + f"location_provided={bool(args.get('location'))}, name={args.get('name')}, " + f"expectedBucketOwner_provided={bool(args.get('expectedBucketOwner'))}." + ) + kwargs = remove_empty_elements( + { + "DetectorId": args.get("detectorId"), + "ThreatEntitySetId": threat_entity_set_id, + "Activate": arg_to_bool_or_none(args.get("activate")), + "Location": args.get("location"), + "Name": args.get("name"), + "ExpectedBucketOwner": args.get("expectedBucketOwner"), + } + ) + + demisto.debug( + f"[AWSGuardDuty] update_threat_entity_set: calling update_threat_entity_set for Threat Entity Set " + f"{threat_entity_set_id} of Detector {args.get('detectorId')} with request fields: {list(kwargs.keys())}." + ) + response = client.update_threat_entity_set(**kwargs) + demisto.debug( + f"[AWSGuardDuty] update_threat_entity_set: {threat_entity_set_id=} API response HTTPStatusCode=" + f"{response.get('ResponseMetadata', {}).get('HTTPStatusCode')}." + ) + + if response == {} or response.get("ResponseMetadata", {}).get("HTTPStatusCode") == 200: + return CommandResults(readable_output=f"Threat entity set {threat_entity_set_id} was updated successfully") + else: + status_code = response.get("ResponseMetadata", {}).get("HTTPStatusCode") + raise DemistoException(f"Failed updating Threat Entity set {threat_entity_set_id}. HTTPStatusCode was: {status_code}") + + +def get_threat_entity_set(client: "GuardDutyClient", args: dict) -> CommandResults: + detector_id = args.get("detectorId") + threat_entity_set_id = args.get("threatEntitySetId") + demisto.debug( + f"[AWSGuardDuty] get_threat_entity_set: started for Threat Entity Set {threat_entity_set_id} " + f"of Detector {detector_id}." + ) + response = client.get_threat_entity_set(DetectorId=detector_id, ThreatEntitySetId=threat_entity_set_id) + demisto.debug( + f"[AWSGuardDuty] get_threat_entity_set: API response keys: {list(response.keys())}, " f"Status={response.get('Status')}." + ) + data = remove_empty_elements( + { + "DetectorId": detector_id, + "ThreatEntitySetId": threat_entity_set_id, + "Name": response.get("Name"), + "Format": response.get("Format"), + "Location": response.get("Location"), + "Status": response.get("Status"), + "ExpectedBucketOwner": response.get("ExpectedBucketOwner"), + "CreatedAt": response.get("CreatedAt"), + "UpdatedAt": response.get("UpdatedAt"), + "Tags": response.get("Tags"), + } + ) + demisto.debug( + f"[AWSGuardDuty] get_threat_entity_set: returning {len(data)} non-empty field(s) for Threat Entity Set " + f"{threat_entity_set_id}." + ) + + return CommandResults( + readable_output=tableToMarkdown("AWS GuardDuty ThreatEntity Set", data), + outputs=data, + outputs_prefix="AWS.GuardDuty.ThreatEntitySet", + outputs_key_field="ThreatEntitySetId", + ) + + +def delete_threat_entity_set(client: "GuardDutyClient", args: dict) -> CommandResults: + threat_entity_set_id = args.get("threatEntitySetId") + detector_id = args.get("detectorId") + demisto.debug( + f"[AWSGuardDuty] delete_threat_entity_set: started for Threat Entity Set {threat_entity_set_id} " + f"from Detector {detector_id}." + ) + response = client.delete_threat_entity_set(DetectorId=detector_id, ThreatEntitySetId=threat_entity_set_id) + demisto.debug( + f"[AWSGuardDuty] delete_threat_entity_set: {threat_entity_set_id=} API response HTTPStatusCode=" + f"{response.get('ResponseMetadata', {}).get('HTTPStatusCode')}." + ) + if response == {} or response.get("ResponseMetadata", {}).get("HTTPStatusCode") == 200: + return CommandResults( + readable_output=f"Threat Entity Set {threat_entity_set_id} was deleted from Detector " f"{detector_id} successfully" + ) + else: + status_code = response.get("ResponseMetadata", {}).get("HTTPStatusCode") + raise DemistoException(f"Failed to delete Threat Entity set {threat_entity_set_id}. HTTPStatusCode was: {status_code}") + + +def list_threat_entity_sets(client: "GuardDutyClient", args: dict) -> CommandResults: + detector_id = args.get("detectorId") + limit, page_size, page = get_pagination_args(args) + demisto.debug( + f"[AWSGuardDuty] list_threat_entity_sets: started for Detector {detector_id} " + f"with limit={limit}, page_size={page_size}, page={page}." + ) + + paginator = client.get_paginator("list_threat_entity_sets") + response_iterator = paginator.paginate( + DetectorId=detector_id, + PaginationConfig={ + "MaxItems": limit, + "PageSize": page_size, + }, + ) + demisto.debug("[AWSGuardDuty] list_threat_entity_sets: paginator created, iterating pages.") + + data = [] + for i, page_response in enumerate(response_iterator): + if page is None or (page - 1) == i: + page_ids = page_response.get("ThreatEntitySetIds", []) + demisto.debug(f"[AWSGuardDuty] list_threat_entity_sets: page index {i} returned {len(page_ids)} ID(s).") + for threat_entity_set in page_ids: + data.append({"DetectorId": detector_id, "ThreatEntitySetId": threat_entity_set}) + if page: + break + + demisto.debug( + f"[AWSGuardDuty] list_threat_entity_sets: found {len(data)} Threat Entity Set(s) for Detector " + f"{detector_id}. Returning CommandResults." + ) + + readable_output = tableToMarkdown( + "AWS GuardDuty ThreatEntity Sets", data, headers=["DetectorId", "ThreatEntitySetId"], removeNull=True + ) + return CommandResults( + readable_output=readable_output, + outputs=data, + outputs_prefix="AWS.GuardDuty.ThreatEntitySet", + outputs_key_field="ThreatEntitySetId", + ) + + def severity_mapping(severity: Optional[float]) -> Optional[int]: demisto_severity = None if severity: @@ -836,6 +1053,21 @@ def main(): # pragma: no cover elif demisto.command() == "aws-gd-update-threatintel-set": result = update_threat_intel_set(client, demisto.args()) + elif demisto.command() == "aws-gd-create-threat-entity-set": + result = create_threat_entity_set(client, args) + + elif demisto.command() == "aws-gd-update-threat-entity-set": + result = update_threat_entity_set(client, args) + + elif demisto.command() == "aws-gd-get-threat-entity-set": + result = get_threat_entity_set(client, args) + + elif demisto.command() == "aws-gd-delete-threat-entity-set": + result = delete_threat_entity_set(client, args) + + elif demisto.command() == "aws-gd-list-threat-entity-sets": + result = list_threat_entity_sets(client, args) + elif demisto.command() == "aws-gd-list-findings": result = list_findings(client, demisto.args()) diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.yml b/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.yml index 66060bbff980..04c5298714e9 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.yml +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.yml @@ -621,6 +621,181 @@ script: name: roleSessionDuration description: Updates the ThreatIntelSet specified by ThreatIntelSet ID. name: aws-gd-update-threatintel-set + - name: aws-gd-create-threat-entity-set + description: Creates a new threat entity set. In a threat entity set, you can provide known malicious threat entities for your AWS environment. + execution: true + arguments: + - name: detectorId + description: The unique ID of the detector that will own the threat entity set. + required: true + - name: name + description: 'The user-friendly name of the threat entity set. Length 1-64.' + required: true + - name: format + description: The format of the entity-list file. + auto: PREDEFINED + predefined: + - TXT + - STIX + - OTX_CSV + - ALIEN_VAULT + - PROOF_POINT + - FIRE_EYE + required: true + - name: location + description: 'The S3 URI of the entity-list file. For example, https://s3.us-east-1.amazonaws.com/example-bucket/threat-entities.txt.' + required: true + - name: activate + description: Whether GuardDuty should start using the uploaded set immediately. + required: true + auto: PREDEFINED + predefined: + - 'True' + - 'False' + - name: expectedBucketOwner + description: The AWS account ID that owns the S3 bucket in the location parameter. Prevents cross-account spoofing. + - name: tags + description: 'The tags to add to the new threat entity set. Format: key=,value=. Separate multiple tags with a semicolon (;). For example, key=Environment,value=prod;key=Team,value=sec.' + - name: region + description: The AWS Region. If not specified, the default region is used. + - name: roleArn + description: The Amazon Resource Name (ARN) of the role to assume. + - name: roleSessionName + description: The identifier for the assumed role session. + - name: roleSessionDuration + description: 'The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role.' + outputs: + - contextPath: AWS.GuardDuty.ThreatEntitySet.DetectorId + description: The unique ID of the detector. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.ThreatEntitySetId + description: The ID of the created threat entity set resource. + type: string + - name: aws-gd-update-threat-entity-set + description: Updates the threat entity set that is specified by the threat entity set ID. + execution: true + arguments: + - name: detectorId + description: The ID of the detector that owns the set. + required: true + - name: threatEntitySetId + description: The ID of the threat entity set to update. + required: true + - name: activate + description: Whether GuardDuty should use the updated set. + auto: PREDEFINED + predefined: + - 'True' + - 'False' + - name: location + description: 'The updated S3 URI of the entity-list file. For example, https://s3.us-east-1.amazonaws.com/example-bucket/threat-entities-v2.txt.' + - name: name + description: 'The updated user-friendly name. Length 1-64.' + - name: expectedBucketOwner + description: The AWS account ID that owns the S3 bucket in the location parameter. + - name: region + description: The AWS Region. If not specified, the default region is used. + - name: roleArn + description: The Amazon Resource Name (ARN) of the role to assume. + - name: roleSessionName + description: The identifier for the assumed role session. + - name: roleSessionDuration + description: 'The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role.' + - name: aws-gd-get-threat-entity-set + description: Retrieves the threat entity set that is specified by the threat entity set ID. + arguments: + - name: detectorId + description: The unique ID of the detector that owns the threat entity set. + required: true + - name: threatEntitySetId + description: The ID of the threat entity set to retrieve. + required: true + - name: region + description: The AWS Region. If not specified, the default region is used. + - name: roleArn + description: The Amazon Resource Name (ARN) of the role to assume. + - name: roleSessionName + description: The identifier for the assumed role session. + - name: roleSessionDuration + description: 'The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role.' + outputs: + - contextPath: AWS.GuardDuty.ThreatEntitySet.DetectorId + description: The unique ID of the detector. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.ThreatEntitySetId + description: The unique identifier for the threat entity set. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.Name + description: The user-friendly name of the threat entity set. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.Format + description: The format of the threat entity set file. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.Location + description: The S3 URI of the threat entity set file. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.Status + description: The status of the threat entity set. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.ExpectedBucketOwner + description: The AWS account ID that owns the S3 bucket in the location parameter. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.CreatedAt + description: The timestamp when the threat entity set was created. For example, 2020-01-01T00:00:00.000Z. + type: date + - contextPath: AWS.GuardDuty.ThreatEntitySet.UpdatedAt + description: The timestamp when the threat entity set was last updated. For example, 2020-01-01T00:00:00.000Z. + type: date + - contextPath: AWS.GuardDuty.ThreatEntitySet.Tags + description: The tags associated with the threat entity set. + type: Unknown + - name: aws-gd-delete-threat-entity-set + description: Deletes the threat entity set that is specified by the threat entity set ID. + execution: true + arguments: + - name: detectorId + description: The ID of the detector that owns the set. + required: true + - name: threatEntitySetId + description: The ID of the threat entity set to delete. + required: true + - name: region + description: The AWS Region. If not specified, the default region is used. + - name: roleArn + description: The Amazon Resource Name (ARN) of the role to assume. + - name: roleSessionName + description: The identifier for the assumed role session. + - name: roleSessionDuration + description: 'The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role.' + - name: aws-gd-list-threat-entity-sets + description: Lists the threat entity sets of the GuardDuty service specified by the detector ID. + arguments: + - name: detectorId + description: The ID of the detector whose threat entity sets you want to list. + required: true + - name: limit + description: The maximum number of results across all pages. + defaultValue: '50' + - name: page_size + description: The page size for the underlying paginator. Valid range is 1-50. + defaultValue: '50' + - name: page + description: The 1-based page index. When set, only that page is returned. + - name: region + description: The AWS Region. If not specified, the default region is used. + - name: roleArn + description: The Amazon Resource Name (ARN) of the role to assume. + - name: roleSessionName + description: The identifier for the assumed role session. + - name: roleSessionDuration + description: 'The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role.' + outputs: + - contextPath: AWS.GuardDuty.ThreatEntitySet.DetectorId + description: The unique ID of the detector. + type: string + - contextPath: AWS.GuardDuty.ThreatEntitySet.ThreatEntitySetId + description: The unique identifier for a threat entity set. + type: string - arguments: - description: The ID of the detector that specifies the GuardDuty service whose findings you want to list. name: detectorId @@ -633,14 +808,14 @@ script: name: roleSessionName - description: The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. name: roleSessionDuration - - defaultValue: '50' - description: Number of total results to query. + - description: The number of total results to query. name: limit + defaultValue: '50' - description: Specific page to query. name: page - - defaultValue: '50' - description: Number of total results in each page. + - description: Number of total results in each page. name: page_size + defaultValue: '50' description: Lists Amazon GuardDuty findings for the specified detector ID. name: aws-gd-list-findings outputs: @@ -662,8 +837,8 @@ script: name: roleSessionName - description: The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. name: roleSessionDuration - - name: returnRawResponse - description: Select 'true' to save all fields from the response to the context. Otherwise, complex fields will be stored in JSON format. Default value is False. + - description: Select 'true' to save all fields from the response to the context. Otherwise, complex fields will be stored in JSON format. Default value is False. + name: returnRawResponse auto: PREDEFINED predefined: - 'true' @@ -888,7 +1063,7 @@ script: - contextPath: AWS.GuardDuty.Members.UpdatedAt description: The time a member was last updated. type: string - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 isfetch: true runonce: false script: '-' diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty_test.py b/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty_test.py index d9b4ceaf4d51..a5185469d494 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty_test.py +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty_test.py @@ -4,31 +4,38 @@ import demistomock as demisto # noqa: F401 import pytest from AWSGuardDuty import ( + DemistoException, archive_findings, connection_test, create_detector, create_ip_set, create_sample_findings, + create_threat_entity_set, create_threat_intel_set, delete_detector, delete_ip_set, + delete_threat_entity_set, delete_threat_intel_set, fetch_incidents, get_detector, get_findings, get_ip_set, get_members, + get_threat_entity_set, get_threat_intel_set, list_detectors, list_findings, list_ip_sets, list_members, + list_threat_entity_sets, list_threat_intel_sets, parse_incident_from_finding, + parse_tag_field_to_dict, unarchive_findings, update_detector, update_findings_feedback, update_ip_set, + update_threat_entity_set, update_threat_intel_set, ) from test_data.api_responses_for_test import ( @@ -114,7 +121,22 @@ def list_threat_intel_sets(self, **kwargs): def update_threat_intel_set(self, **kwargs): pass - def get_paginator(self, **kwargs): + def create_threat_entity_set(self, **kwargs): + pass + + def update_threat_entity_set(self, **kwargs): + pass + + def get_threat_entity_set(self, **kwargs): + pass + + def delete_threat_entity_set(self, **kwargs): + pass + + def list_threat_entity_sets(self, **kwargs): + pass + + def get_paginator(self, operation_name=None, **kwargs): pass @@ -580,6 +602,523 @@ def test_create_threat_intel_set(mocker): assert command_results.outputs == {"DetectorId": "some_id", "ThreatIntelSetId": "threat1"} +@pytest.mark.parametrize( + "tags_str, expected", + [ + pytest.param("key=Environment,value=prod", {"Environment": "prod"}, id="single pair"), + pytest.param( + "key=Environment,value=prod;key=Team,value=sec", + {"Environment": "prod", "Team": "sec"}, + id="multiple pairs", + ), + pytest.param("key=Owner,value=team-a,team-b", {"Owner": "team-a,team-b"}, id="value with commas"), + ], +) +def test_parse_tag_field_to_dict(tags_str, expected): + """ + Given: + A tags string in the "key=,value=" format (values may contain commas) + + When: + Running parse_tag_field_to_dict + + Then: + assert the string is parsed into the expected key-to-value mapping. + """ + assert parse_tag_field_to_dict(tags_str) == expected + + +@pytest.mark.parametrize( + "tags_str", + [ + pytest.param("Environment=prod", id="missing key/value fields"), + pytest.param("key=Environment", id="missing value"), + ], +) +def test_parse_tag_field_to_dict_invalid(tags_str): + """ + Given: + A malformed tags string + + When: + Running parse_tag_field_to_dict + + Then: + assert a ValueError is raised describing the expected format. + """ + with pytest.raises(ValueError, match="Could not parse field"): + parse_tag_field_to_dict(tags_str) + + +def test_parse_tag_field_to_dict_exceeds_max_tags(mocker): + """ + Given: + A tags string with more than 50 tag pairs + + When: + Running parse_tag_field_to_dict + + Then: + assert only the first 50 tags are parsed. + """ + tags_str = ";".join(f"key=k{i},value=v{i}" for i in range(60)) + + result = parse_tag_field_to_dict(tags_str) + + assert len(result) == 50 + assert "k0" in result + assert "k49" in result + assert "k50" not in result + + +def test_create_threat_entity_set(mocker): + """ + Given: + AWSClient session + create_threat_entity_set valid response + + When: + Running create_threat_entity_set command + + Then: + assert api calls are called exactly once and as expected. + """ + mocked_client = MockedBoto3Client() + create_threat_entity_set_mock = mocker.patch.object( + MockedBoto3Client, "create_threat_entity_set", side_effect=[{"ThreatEntitySetId": "entity1"}] + ) + + command_results = create_threat_entity_set( + mocked_client, + { + "detectorId": "some_id", + "activate": "True", + "format": "some_format", + "location": "some_location", + "name": "some_name", + }, + ) + + create_threat_entity_set_mock.assert_called_with( + DetectorId="some_id", Activate=True, Format="some_format", Location="some_location", Name="some_name" + ) + assert command_results.outputs == {"DetectorId": "some_id", "ThreatEntitySetId": "entity1"} + + +def test_create_threat_entity_set_with_optional_args(mocker): + """ + Given: + AWSClient session + create_threat_entity_set valid response and the optional expectedBucketOwner and tags arguments + + When: + Running create_threat_entity_set command + + Then: + assert the optional arguments are passed to the api call as expected and tags are parsed to a map. + """ + mocked_client = MockedBoto3Client() + create_threat_entity_set_mock = mocker.patch.object( + MockedBoto3Client, "create_threat_entity_set", side_effect=[{"ThreatEntitySetId": "entity1"}] + ) + + command_results = create_threat_entity_set( + mocked_client, + { + "detectorId": "some_id", + "activate": "True", + "format": "some_format", + "location": "some_location", + "name": "some_name", + "expectedBucketOwner": "123456789012", + "tags": "key=Environment,value=prod;key=Team,value=sec", + }, + ) + + create_threat_entity_set_mock.assert_called_with( + DetectorId="some_id", + Activate=True, + Format="some_format", + Location="some_location", + Name="some_name", + ExpectedBucketOwner="123456789012", + Tags={"Environment": "prod", "Team": "sec"}, + ) + assert command_results.outputs == {"DetectorId": "some_id", "ThreatEntitySetId": "entity1"} + + +def test_create_threat_entity_set_command_results(mocker): + """ + Given: + AWSClient session + create_threat_entity_set valid response + + When: + Running create_threat_entity_set command + + Then: + assert the CommandResults outputs_prefix, outputs_key_field and readable_output are as expected. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object(MockedBoto3Client, "create_threat_entity_set", side_effect=[{"ThreatEntitySetId": "entity1"}]) + + command_results = create_threat_entity_set( + mocked_client, + { + "detectorId": "some_id", + "activate": "False", + "format": "TXT", + "location": "some_location", + "name": "some_name", + }, + ) + + assert command_results.outputs_prefix == "AWS.GuardDuty.ThreatEntitySet" + assert command_results.outputs_key_field == "ThreatEntitySetId" + assert "AWS GuardDuty ThreatEntity Set" in command_results.readable_output + assert "entity1" in command_results.readable_output + + +@pytest.mark.parametrize( + "response, raises", + [ + pytest.param({}, does_not_raise(), id="Success"), + pytest.param(RESPONSE_METADATA, does_not_raise(), id="Success with Metadata"), + pytest.param({"response": "bad"}, pytest.raises(Exception), id="Failure"), + ], +) +def test_update_threat_entity_set(mocker, response, raises): + """ + Given: + AWSClient session + update_threat_entity_set various responses + + When: + Running update_threat_entity_set command + + Then: + assert exceptions are being raised according to api response + assert api calls are called exactly once and as expected. + """ + mocked_client = MockedBoto3Client() + update_threat_entity_set_mock = mocker.patch.object(MockedBoto3Client, "update_threat_entity_set", side_effect=[response]) + + with raises: + update_threat_entity_set( + mocked_client, + { + "detectorId": "some_id", + "threatEntitySetId": "ThreatEntitySetId1", + "activate": "True", + "location": "here", + "name": "some_name", + }, + ) + + update_threat_entity_set_mock.assert_called_with( + DetectorId="some_id", ThreatEntitySetId="ThreatEntitySetId1", Activate=True, Location="here", Name="some_name" + ) + + +def test_update_threat_entity_set_readable_output(mocker): + """ + Given: + AWSClient session + update_threat_entity_set successful response + + When: + Running update_threat_entity_set command + + Then: + assert the returned confirmation message contains the ThreatEntitySet ID. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object(MockedBoto3Client, "update_threat_entity_set", side_effect=[{}]) + + result = update_threat_entity_set(mocked_client, {"detectorId": "some_id", "threatEntitySetId": "ThreatEntitySetId1"}) + + assert result.readable_output == "Threat entity set ThreatEntitySetId1 was updated successfully" + + +def test_update_threat_entity_set_failure_message(mocker): + """ + Given: + AWSClient session + update_threat_entity_set response indicating a failure + + When: + Running update_threat_entity_set command + + Then: + assert an exception is raised with a descriptive failure message. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object(MockedBoto3Client, "update_threat_entity_set", side_effect=[{"response": "bad"}]) + + with pytest.raises(Exception, match="Failed updating Threat Entity set ThreatEntitySetId1"): + update_threat_entity_set(mocked_client, {"detectorId": "some_id", "threatEntitySetId": "ThreatEntitySetId1"}) + + +def test_get_threat_entity_set(mocker): + """ + Given: + AWSClient session + get_threat_entity_set valid response + + When: + Running get_threat_entity_set command + + Then: + assert the API is called as expected and the outputs contain all the entity set fields. + """ + mocked_client = MockedBoto3Client() + get_mock = mocker.patch.object( + MockedBoto3Client, + "get_threat_entity_set", + side_effect=[ + { + "Name": "some_name", + "Format": "TXT", + "Location": "some_location", + "Status": "ACTIVE", + "ExpectedBucketOwner": "123456789012", + "CreatedAt": "2024-01-01T00:00:00Z", + "UpdatedAt": "2024-01-02T00:00:00Z", + "Tags": {"Environment": "prod"}, + } + ], + ) + + command_results = get_threat_entity_set(mocked_client, {"detectorId": "some_id", "threatEntitySetId": "ThreatEntitySetId1"}) + + get_mock.assert_called_with(DetectorId="some_id", ThreatEntitySetId="ThreatEntitySetId1") + assert command_results.outputs == { + "DetectorId": "some_id", + "ThreatEntitySetId": "ThreatEntitySetId1", + "Name": "some_name", + "Format": "TXT", + "Location": "some_location", + "Status": "ACTIVE", + "ExpectedBucketOwner": "123456789012", + "CreatedAt": "2024-01-01T00:00:00Z", + "UpdatedAt": "2024-01-02T00:00:00Z", + "Tags": {"Environment": "prod"}, + } + + +def test_get_threat_entity_set_command_results(mocker): + """ + Given: + AWSClient session + get_threat_entity_set valid response + + When: + Running get_threat_entity_set command + + Then: + assert the CommandResults outputs_prefix, outputs_key_field and readable_output are as expected. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object(MockedBoto3Client, "get_threat_entity_set", side_effect=[{"Name": "some_name", "Status": "ACTIVE"}]) + + command_results = get_threat_entity_set(mocked_client, {"detectorId": "some_id", "threatEntitySetId": "ThreatEntitySetId1"}) + + assert command_results.outputs_prefix == "AWS.GuardDuty.ThreatEntitySet" + assert command_results.outputs_key_field == "ThreatEntitySetId" + assert "AWS GuardDuty ThreatEntity Set" in command_results.readable_output + assert "some_name" in command_results.readable_output + + +def test_get_threat_entity_set_empty_fields(mocker): + """ + Given: + AWSClient session + get_threat_entity_set response with empty optional fields + + When: + Running get_threat_entity_set command + + Then: + assert empty fields are removed from the outputs and the identifying fields remain. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object( + MockedBoto3Client, "get_threat_entity_set", side_effect=[{"Name": "some_name", "Tags": {}, "Location": None}] + ) + + command_results = get_threat_entity_set(mocked_client, {"detectorId": "some_id", "threatEntitySetId": "ThreatEntitySetId1"}) + + assert command_results.outputs == { + "DetectorId": "some_id", + "ThreatEntitySetId": "ThreatEntitySetId1", + "Name": "some_name", + } + + +@pytest.mark.parametrize( + "response, raises", + [ + pytest.param({}, does_not_raise(), id="Success"), + pytest.param(RESPONSE_METADATA, does_not_raise(), id="Success with Metadata"), + pytest.param({"response": "bad"}, pytest.raises(Exception), id="Failure"), + ], +) +def test_delete_threat_entity_set(mocker, response, raises): + """ + Given: + AWSClient session + delete_threat_entity_set various responses + + When: + Running delete_threat_entity_set command + + Then: + assert exceptions are being raised according to api response + assert api calls are called exactly once and as expected. + """ + mocked_client = MockedBoto3Client() + delete_threat_entity_set_mock = mocker.patch.object(MockedBoto3Client, "delete_threat_entity_set", side_effect=[response]) + + with raises: + delete_threat_entity_set(mocked_client, {"detectorId": "some_id", "threatEntitySetId": "ThreatEntitySetId1"}) + + delete_threat_entity_set_mock.assert_called_with(DetectorId="some_id", ThreatEntitySetId="ThreatEntitySetId1") + + +def test_delete_threat_entity_set_readable_output(mocker): + """ + Given: + AWSClient session + delete_threat_entity_set successful response + + When: + Running delete_threat_entity_set command + + Then: + assert the returned confirmation message contains the ThreatEntitySet ID and Detector ID. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object(MockedBoto3Client, "delete_threat_entity_set", side_effect=[{}]) + + result = delete_threat_entity_set(mocked_client, {"detectorId": "some_id", "threatEntitySetId": "ThreatEntitySetId1"}) + + assert result.readable_output == "Threat Entity Set ThreatEntitySetId1 was deleted from Detector some_id successfully" + + +def test_delete_threat_entity_set_failure_message(mocker): + """ + Given: + AWSClient session + delete_threat_entity_set response indicating a failure + + When: + Running delete_threat_entity_set command + + Then: + assert an exception is raised with a descriptive failure message. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object(MockedBoto3Client, "delete_threat_entity_set", side_effect=[{"response": "bad"}]) + + with pytest.raises(Exception, match="Failed to delete Threat Entity set ThreatEntitySetId1"): + delete_threat_entity_set(mocked_client, {"detectorId": "some_id", "threatEntitySetId": "ThreatEntitySetId1"}) + + +def test_list_threat_entity_sets(mocker): + """ + Given: + AWSClient session + list_threat_entity_sets valid paginated response + + When: + Running list_threat_entity_sets command + + Then: + assert the paginator is called as expected and the outputs contain the detector and entity set IDs. + """ + mocked_client = MockedBoto3Client() + get_paginator_mock = mocker.patch.object(MockedBoto3Client, "get_paginator", side_effect=[MockedPaginator()]) + paginate_mock = mocker.patch.object( + MockedPaginator, "paginate", side_effect=[[{"ThreatEntitySetIds": ["entity1", "entity2"]}]] + ) + + command_results = list_threat_entity_sets(mocked_client, {"detectorId": "some_id"}) + + get_paginator_mock.assert_called_with("list_threat_entity_sets") + paginate_mock.assert_called_with(DetectorId="some_id", PaginationConfig={"MaxItems": 50, "PageSize": 50}) + assert command_results.outputs == [ + {"DetectorId": "some_id", "ThreatEntitySetId": "entity1"}, + {"DetectorId": "some_id", "ThreatEntitySetId": "entity2"}, + ] + assert command_results.outputs_prefix == "AWS.GuardDuty.ThreatEntitySet" + + +def test_list_threat_entity_sets_specific_page(mocker): + """ + Given: + AWSClient session + list_threat_entity_sets paginated response spanning multiple pages and the page argument + + When: + Running list_threat_entity_sets command + + Then: + assert only the requested page's entity sets are returned. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object(MockedBoto3Client, "get_paginator", side_effect=[MockedPaginator()]) + mocker.patch.object( + MockedPaginator, + "paginate", + side_effect=[[{"ThreatEntitySetIds": ["entity1"]}, {"ThreatEntitySetIds": ["entity2"]}]], + ) + + command_results = list_threat_entity_sets(mocked_client, {"detectorId": "some_id", "page": "2", "page_size": "1"}) + + assert command_results.outputs == [ + {"DetectorId": "some_id", "ThreatEntitySetId": "entity2"}, + ] + + +def test_list_threat_entity_sets_empty(mocker): + """ + Given: + AWSClient session + list_threat_entity_sets paginated response with no ThreatEntitySet IDs + + When: + Running list_threat_entity_sets command + + Then: + assert an empty list is returned when there are no threat entity sets. + """ + mocked_client = MockedBoto3Client() + mocker.patch.object(MockedBoto3Client, "get_paginator", side_effect=[MockedPaginator()]) + mocker.patch.object(MockedPaginator, "paginate", side_effect=[[{"ThreatEntitySetIds": []}]]) + + command_results = list_threat_entity_sets(mocked_client, {"detectorId": "some_id"}) + + assert command_results.outputs == [] + + +def test_list_threat_entity_sets_invalid_page_size(mocker): + """ + Given: + AWSClient session + list_threat_entity_sets called with a page_size above the allowed maximum + + When: + Running list_threat_entity_sets command + + Then: + assert a DemistoException is raised for the out-of-range page_size. + """ + mocked_client = MockedBoto3Client() + + with pytest.raises(DemistoException, match="page_size argument must be between 1 to 50"): + list_threat_entity_sets(mocked_client, {"detectorId": "some_id", "page_size": "51"}) + + EXPECTED_IP_SET_RESULT = { "DetectorId": "some_id", "Format": "TXT", diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/README.md b/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/README.md index e92902c9dba1..0f7576b4e0c2 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/README.md +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/README.md @@ -630,9 +630,9 @@ Action: _guardduty:ListFindings_ | roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional | | roleSessionName | An identifier for the assumed role session. | Optional | | roleSessionDuration | The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role. | Optional | -| limit | Number of total results to query. Default is `50`. | Optional | +| limit | The number of total results to query. Default is 50. | Optional | | page | Specific page to query. | Optional | -| page_size | Number of total results in each page. Default is `50`. | Optional | +| page_size | Number of total results in each page. Default is 50. | Optional | #### Context Output @@ -917,6 +917,174 @@ Action: _guardduty:GetMembers_ ```!aws-gd-get-members detectorIds=4f1fc7cd7dsg26sdf4328d8dc813 accountIds=1f3fc2cd1dag26sdf4338d8aa813``` +### aws-gd-create-threat-entity-set + +*** +Creates a new threat entity set. In a threat entity set, you can provide known malicious threat entities for your AWS environment. + +#### Base Command + +`aws-gd-create-threat-entity-set` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| detectorId | The unique ID of the detector that will own the threat entity set. | Required | +| name | The user-friendly name of the threat entity set. Length 1-64. | Required | +| format | The format of the entity-list file. Possible values are: TXT, STIX, OTX_CSV, ALIEN_VAULT, PROOF_POINT, FIRE_EYE. | Required | +| location | The S3 URI of the entity-list file. For example, https://s3.us-east-1.amazonaws.com/example-bucket/threat-entities.txt. | Required | +| activate | Whether GuardDuty should start using the uploaded set immediately. Possible values are: True, False. | Required | +| expectedBucketOwner | The AWS account ID that owns the S3 bucket in the location parameter. Prevents cross-account spoofing. | Optional | +| tags | The tags to add to the new threat entity set. Format: key=<KEY>,value=<VALUE>. Separate multiple tags with a semicolon (;). For example, key=Environment,value=prod;key=Team,value=sec. | Optional | +| region | The AWS Region. If not specified, the default region is used. | Optional | +| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional | +| roleSessionName | The identifier for the assumed role session. | Optional | +| roleSessionDuration | The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| AWS.GuardDuty.ThreatEntitySet.DetectorId | string | The unique ID of the detector. | +| AWS.GuardDuty.ThreatEntitySet.ThreatEntitySetId | string | The ID of the created threat entity set resource. | + +#### Command Example + +```!aws-gd-create-threat-entity-set detectorId=12abc34d567e8fa901bc2d34e56789f0 name=malicious-entities format=TXT location=https://s3.us-east-1.amazonaws.com/example-bucket/threat-entities.txt activate=True tags=key=Environment,value=prod;key=Team,value=sec``` + +### aws-gd-update-threat-entity-set + +*** +Updates the threat entity set that is specified by the threat entity set ID. + +#### Base Command + +`aws-gd-update-threat-entity-set` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| detectorId | The ID of the detector that owns the set. | Required | +| threatEntitySetId | The ID of the threat entity set to update. | Required | +| activate | Whether GuardDuty should use the updated set. Possible values are: True, False. | Optional | +| location | The updated S3 URI of the entity-list file. For example, https://s3.us-east-1.amazonaws.com/example-bucket/threat-entities-v2.txt. | Optional | +| name | The updated user-friendly name. Length 1-64. | Optional | +| expectedBucketOwner | The AWS account ID that owns the S3 bucket in the location parameter. | Optional | +| region | The AWS Region. If not specified, the default region is used. | Optional | +| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional | +| roleSessionName | The identifier for the assumed role session. | Optional | +| roleSessionDuration | The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role. | Optional | + +#### Context Output + +There is no context output for this command. + +#### Command Example + +```!aws-gd-update-threat-entity-set detectorId=12abc34d567e8fa901bc2d34e56789f0 threatEntitySetId=ab12cd34ef56gh78ij90 activate=True name=malicious-entities-v2``` + +### aws-gd-get-threat-entity-set + +*** +Retrieves the threat entity set that is specified by the threat entity set ID. + +#### Base Command + +`aws-gd-get-threat-entity-set` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| detectorId | The unique ID of the detector that owns the threat entity set. | Required | +| threatEntitySetId | The ID of the threat entity set to retrieve. | Required | +| region | The AWS Region. If not specified, the default region is used. | Optional | +| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional | +| roleSessionName | The identifier for the assumed role session. | Optional | +| roleSessionDuration | The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| AWS.GuardDuty.ThreatEntitySet.DetectorId | string | The unique ID of the detector. | +| AWS.GuardDuty.ThreatEntitySet.ThreatEntitySetId | string | The unique identifier for the threat entity set. | +| AWS.GuardDuty.ThreatEntitySet.Name | string | The user-friendly name of the threat entity set. | +| AWS.GuardDuty.ThreatEntitySet.Format | string | The format of the threat entity set file. | +| AWS.GuardDuty.ThreatEntitySet.Location | string | The S3 URI of the threat entity set file. | +| AWS.GuardDuty.ThreatEntitySet.Status | string | The status of the threat entity set. | +| AWS.GuardDuty.ThreatEntitySet.ExpectedBucketOwner | string | The AWS account ID that owns the S3 bucket in the location parameter. | +| AWS.GuardDuty.ThreatEntitySet.CreatedAt | date | The timestamp when the threat entity set was created. For example, 2020-01-01T00:00:00.000Z. | +| AWS.GuardDuty.ThreatEntitySet.UpdatedAt | date | The timestamp when the threat entity set was last updated. For example, 2020-01-01T00:00:00.000Z. | +| AWS.GuardDuty.ThreatEntitySet.Tags | Unknown | The tags associated with the threat entity set. | + +#### Command Example + +```!aws-gd-get-threat-entity-set detectorId=12abc34d567e8fa901bc2d34e56789f0 threatEntitySetId=ab12cd34ef56gh78ij90``` + +### aws-gd-delete-threat-entity-set + +*** +Deletes the threat entity set that is specified by the threat entity set ID. + +#### Base Command + +`aws-gd-delete-threat-entity-set` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| detectorId | The ID of the detector that owns the set. | Required | +| threatEntitySetId | The ID of the threat entity set to delete. | Required | +| region | The AWS Region. If not specified, the default region is used. | Optional | +| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional | +| roleSessionName | The identifier for the assumed role session. | Optional | +| roleSessionDuration | The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role. | Optional | + +#### Context Output + +There is no context output for this command. + +#### Command Example + +```!aws-gd-delete-threat-entity-set detectorId=12abc34d567e8fa901bc2d34e56789f0 threatEntitySetId=ab12cd34ef56gh78ij90``` + +### aws-gd-list-threat-entity-sets + +*** +Lists the threat entity sets of the GuardDuty service specified by the detector ID. + +#### Base Command + +`aws-gd-list-threat-entity-sets` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| detectorId | The ID of the detector whose threat entity sets you want to list. | Required | +| limit | The maximum number of results across all pages. Default is 50. | Optional | +| page_size | The page size for the underlying paginator. Valid range is 1-50. Default is 50. | Optional | +| page | The 1-based page index. When set, only that page is returned. | Optional | +| region | The AWS Region. If not specified, the default region is used. | Optional | +| roleArn | The Amazon Resource Name (ARN) of the role to assume. | Optional | +| roleSessionName | The identifier for the assumed role session. | Optional | +| roleSessionDuration | The duration, in seconds, of the role session. Range: 900 seconds (15 minutes) to the maximum session duration setting for the role. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| AWS.GuardDuty.ThreatEntitySet.DetectorId | string | The unique ID of the detector. | +| AWS.GuardDuty.ThreatEntitySet.ThreatEntitySetId | string | The unique identifier for a threat entity set. | + +#### Command Example + +```!aws-gd-list-threat-entity-sets detectorId=12abc34d567e8fa901bc2d34e56789f0``` + ## Breaking changes from the previous version of this integration - AWS-GuardDuty The following sections list the changes in this version. diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.py b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.py index 1e9fe30a0c59..b4001ea7edc7 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.py +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.py @@ -1,5 +1,5 @@ import json -from datetime import date, datetime +from datetime import date, datetime, timedelta, UTC from typing import TYPE_CHECKING import demistomock as demisto # noqa: F401 @@ -17,6 +17,12 @@ MAX_RESULTS = 50 GD_SEVERITY_DICT = {"Low": 1, "Medium": 4, "High": 7} +# A fully-deduped cursor boundary second is only escaped once it is +# at least this far in the past relative to the fetch start time. This keeps a +# still-current second pinned (so a genuine later same-second update is still +# catchable) while breaking a boundary that has clearly gone stale. +STALE_BOUNDARY_MARGIN = timedelta(minutes=2) + PRODUCT = "guardduty" VENDOR = "aws" @@ -52,7 +58,7 @@ def _normalize_last_ids_entry(value) -> set[str]: """Coerce a stored ``last_ids`` value into a set of ids. The integration historically stored ``last_ids[detector_id]`` as a single - string (the last finding id seen). To fix XSUP-67097 we now track every + string (the last finding id seen). We now track every finding id sharing the cursor's ``UpdatedAt`` second, which means the value is conceptually a set. ``demisto.setLastRun`` serializes as JSON, so the on-disk representation must be a ``list``. This helper normalizes @@ -83,7 +89,7 @@ def _build_finding_criterion(updated_at: Optional[datetime], severity: str, excl updated_at: Inclusive lower bound on ``updatedAt``. severity: Minimum severity label (Low/Medium/High). exclude_archived: When ``True``, adds ``service.archived = false`` so suppressed/archived - findings (XSUP-67097 / XSUP-71079 complaint #2) are not re-fetched. + findings are not re-fetched. Returns: The criterion dict. @@ -103,6 +109,49 @@ def _event_updated_at(event: dict) -> Any: return event.get("UpdatedAt", event.get("CreatedAt")) +def _cursor_second(value: Any) -> datetime: + """Return the whole-second (microseconds truncated), naive-UTC datetime for a timestamp. + + GuardDuty findings carry MILLISECOND-precision ``UpdatedAt`` + values, and several findings routinely share the same whole second while + differing in their milliseconds. The fetch cursor and the same-second + sibling dedup are defined at SECOND resolution, + so all comparisons and grouping must truncate sub-second precision. Doing so + keeps the cursor deterministic (independent of AWS ``get_findings`` ordering) + and lets every finding sharing the cursor second be recognized as an + already-seen sibling on the inclusive ``Gte`` re-query. + + Timezone normalization: values reach this function from two sources with + different awareness. ``first_fetch`` (via ``arg_to_datetime``) is tz-AWARE for + absolute dates, while finding timestamps go through ``parse_date_string`` + (``strptime`` with no ``%z``) and are always NAIVE. Comparing the two (dedup + equality, and the stale-boundary ``<`` check against ``datetime.utcnow()``, + which is naive) would either never match (silent re-ingestion) or raise + ``TypeError: can't compare offset-naive and offset-aware datetimes``. To make + every cursor value consistently comparable, a tz-aware input is first + CONVERTED to UTC and THEN stripped of ``tzinfo`` (order matters — dropping + tzinfo before converting would shift, e.g., a ``+05:00`` cursor by 5 hours). + + Args: + value: A timestamp string or a ``datetime`` (naive or tz-aware). Every + GuardDuty finding carries a real ``UpdatedAt``/``CreatedAt``, so a + value is always present. + + Returns: + The corresponding naive-UTC ``datetime`` truncated to whole seconds. + + Raises: + ValueError: If ``value`` is not a parseable timestamp. This is not + expected from real GuardDuty data and surfaces loudly rather than + silently corrupting the fetch cursor. + """ + dt = value if isinstance(value, datetime) else parse_date_string(value) + if dt.tzinfo is not None: + # Convert to UTC first, then drop tzinfo so the result is naive-UTC. + dt = dt.astimezone(UTC).replace(tzinfo=None) + return dt.replace(microsecond=0) + + def get_events( aws_client: "GuardDutyClient", collect_from: dict, @@ -137,7 +186,7 @@ def get_events( Each value is a ``list[str]`` (JSON-serializable for setLastRun). new_collect_from (dict): The new collect_from dict, expected to receive as collect_from input in the next run. - Note (XSUP-71079): The fetch cursor is second-resolution and the ``updatedAt`` filter is inclusive + Note: The fetch cursor is second-resolution and the ``updatedAt`` filter is inclusive (``Gte``). To avoid silently skipping findings, the cursor is NEVER advanced into a second that was only partially consumed because ``limit`` was reached. When a fetch is truncated mid-second the cursor is rolled back to the last fully-drained second (and its sibling ids are persisted) so the next run @@ -153,16 +202,31 @@ def get_events( demisto.debug(f"AWSGuardDutyEventCollector Starting get_events. {collect_from=}, {collect_from_default=}, {last_ids=}") # List all detectors + list_detectors_page = 0 while next_token: + list_detectors_page += 1 list_detectors_args: dict = {"MaxResults": detectors_num} if next_token != "starting_token": list_detectors_args.update({"NextToken": next_token}) response = aws_client.list_detectors(**list_detectors_args) - detector_ids += response.get("DetectorIds", []) - next_token = response.get("NextToken", "") + page_detector_ids = response.get("DetectorIds", []) + raw_next_token = response.get("NextToken", "") + detector_ids += page_detector_ids + # Diagnostic: log per-page enumeration so a truncated/short-circuited pagination is visible. + demisto.debug( + f"AWSGuardDutyEventCollector - list_detectors page {list_detectors_page}: " + f"MaxResults={detectors_num}, returned={len(page_detector_ids)}, " + f"running_total={len(detector_ids)}, has_next_token={bool(raw_next_token)}, " + f"next_token_len={len(raw_next_token) if raw_next_token else 0}, " + f"page_detector_ids={page_detector_ids}" + ) + next_token = raw_next_token - demisto.debug(f"AWSGuardDutyEventCollector - Found detector ids: {detector_ids}") + demisto.debug( + f"AWSGuardDutyEventCollector - Finished list_detectors after {list_detectors_page} page(s). " + f"Total detectors found: {len(detector_ids)}. Found detector ids: {detector_ids}" + ) for detector_id in detector_ids: demisto.debug( @@ -173,16 +237,18 @@ def get_events( finding_ids: list = [] detector_events: list = [] updated_at = parse_date_string(collect_from.get(detector_id)) if collect_from.get(detector_id) else collect_from_default - # XSUP-67097: dedup against ALL ids seen at the cursor second, not just one. + # Dedup against all ids seen at the cursor second. seen_ids = _normalize_last_ids_entry(last_ids.get(detector_id)) - # List all finding ids - while next_token and len(events) + len(finding_ids) < limit: + # List all finding ids. The ``limit`` budget is applied PER DETECTOR so that a busy detector + # earlier in the loop can never starve detectors processed later (each detector gets its own + # full ``limit`` per cycle instead of sharing one global budget across all detectors). + while next_token and len(set(finding_ids)) < limit: demisto.debug(f"AWSGuardDutyEventCollector - Getting more finding ids with {next_token=}, {updated_at=}") list_finding_args = { "DetectorId": detector_id, "FindingCriteria": {"Criterion": _build_finding_criterion(updated_at, severity, exclude_archived)}, "SortCriteria": {"AttributeName": "updatedAt", "OrderBy": "ASC"}, - "MaxResults": min(limit - (len(events) + len(set(finding_ids))), MAX_RESULTS), + "MaxResults": min(limit - len(set(finding_ids)), MAX_RESULTS), } if next_token != "starting_token": list_finding_args.update({"NextToken": next_token}) @@ -202,87 +268,118 @@ def get_events( findings_response = aws_client.get_findings(DetectorId=detector_id, FindingIds=chunk_of_finding_ids) detector_events += findings_response.get("Findings", []) - # Dedup already-seen findings — but only at the cursor second. - # - # XSUP-67097: drop every finding we already ingested that still shares the cursor's UpdatedAt - # second (same-second siblings re-returned by the inclusive Gte query). - # - # XSUP-72455: do NOT drop a finding whose UpdatedAt has advanced past the cursor second. GuardDuty - # findings are long-lived and update in place; when a recurring finding gets a new occurrence its - # UpdatedAt moves forward and AWS returns it again. That is a legitimate new event and must be - # ingested. The previous ID-only dedup dropped it because its id was in last_ids, which produced an - # empty result and, because the cursor only advances when events are ingested, pinned the fetch - # behind that finding indefinitely. + # Drop already-seen findings only when their UpdatedAt still shares the cursor second. + # Findings whose UpdatedAt advanced past the cursor second are new events and are kept. + # Compared at whole-second resolution to catch millisecond siblings of the same second. + cursor_second = _cursor_second(updated_at) if seen_ids: before_ids = [ev.get("Id") for ev in detector_events] detector_events = [ ev for ev in detector_events - if ev.get("Id") not in seen_ids or parse_date_string(_event_updated_at(ev)) != updated_at + if ev.get("Id") not in seen_ids or _cursor_second(_event_updated_at(ev)) != cursor_second ] after_ids = [ev.get("Id") for ev in detector_events] if before_ids != after_ids: demisto.debug( f"AWSGuardDutyEventCollector - Dedup removed already-seen same-second findings " f"for {detector_id=}. Before: {before_ids}, after: {after_ids}, removed via {seen_ids=} " - f"at cursor second {updated_at=}." + f"at cursor second {cursor_second=}." + ) + else: + demisto.debug( + f"AWSGuardDutyEventCollector - Dedup ran for {detector_id=} at cursor second " + f"{cursor_second=} but removed nothing ({len(before_ids)} findings kept, " + f"{len(seen_ids)} seen ids carried in)." ) demisto.debug(f"AWSGuardDutyEventCollector - {detector_id=} findings found ({len(detector_events)}): {detector_events}") events += detector_events demisto.debug(f"AWSGuardDutyEventCollector - Number of events is {len(events)}") - # XSUP-71079: advance the cursor safely. - # - # The cursor is second-resolution and the updatedAt query is inclusive (Gte). Two failure modes - # are guarded here: - # 1. Same-second siblings (XSUP-67097): persist EVERY finding id whose UpdatedAt equals the - # cursor second so the next run can dedup them all (not just one). - # 2. Mid-second truncation (XSUP-71079): if this fetch stopped because it hit `limit` while - # there were still un-fetched findings (next_token is truthy) AND the last second is only - # partially consumed, advancing the cursor to that last second would skip the remaining - # siblings of that second (they fall on the same inclusive boundary but AWS may order them - # after the truncation point). To guarantee no loss we roll the cursor back to the last - # FULLY-drained second and persist its sibling ids, so the next run re-queries the - # truncated second from its start and makes forward progress. - truncated_by_limit = bool(next_token) # loop exited with a pending token => stopped due to limit + # Advance the cursor. The cursor is second-resolution and the updatedAt query is inclusive (Gte), + # so persist every finding id at the cursor second for next-run dedup, and roll the cursor back to + # the last fully-drained second when a limit-truncated page leaves a partial second. + truncated_by_limit = bool(next_token) # pending token => stopped due to limit if detector_events: - last_cursor_ts = _event_updated_at(detector_events[-1]) - cursor_ts = last_cursor_ts + # Pick the cursor at whole-second resolution: latest ingested second, max UpdatedAt within it. + event_seconds = [(ev, _cursor_second(_event_updated_at(ev))) for ev in detector_events] + latest_second = max(sec for _, sec in event_seconds) + cursor_second_target = latest_second if truncated_by_limit: - # Find the latest second strictly older than the last (partial) second. - distinct_seconds = {_event_updated_at(ev) for ev in detector_events} - fully_drained = sorted((s for s in distinct_seconds if s != last_cursor_ts), key=parse_date_string) + # Find the latest whole-second strictly older than the last (partial) second. + distinct_seconds = {sec for _, sec in event_seconds} + fully_drained = sorted(s for s in distinct_seconds if s != latest_second) if fully_drained: - cursor_ts = fully_drained[-1] + cursor_second_target = fully_drained[-1] demisto.debug( f"AWSGuardDutyEventCollector - Fetch truncated by limit for {detector_id=}. " - f"Rolling cursor back from partial second {last_cursor_ts} to last fully-drained " - f"second {cursor_ts} to avoid skipping same-second siblings." + f"Rolling cursor back from partial second {latest_second} to last fully-drained " + f"second {cursor_second_target} to avoid skipping same-second siblings." ) else: # The entire page is a single second that we could not fully drain. Keep the cursor # on that second and accumulate seen ids so progress happens via dedup next run. demisto.debug( f"AWSGuardDutyEventCollector - Fetch truncated by limit for {detector_id=} within a " - f"single second {last_cursor_ts}; keeping cursor and accumulating seen ids." + f"single second {latest_second}; keeping cursor and accumulating seen ids." ) + # Same-second siblings of the target second. + cursor_second_events = [ev for ev, sec in event_seconds if sec == cursor_second_target] + cursor_ts = max(_event_updated_at(ev) for ev in cursor_second_events) new_collect_from[detector_id] = cursor_ts - cursor_sibling_ids = {ev.get("Id") for ev in detector_events if _event_updated_at(ev) == cursor_ts} + cursor_sibling_ids = {ev.get("Id") for ev in cursor_second_events} cursor_sibling_ids.discard(None) - # Carry forward previously-seen ids when the cursor second did not advance past them, - # so we never forget same-second siblings across runs. - if seen_ids and parse_date_string(cursor_ts) == updated_at: + # Carry forward prior seen ids when the cursor second did not advance past them. + carried_forward = seen_ids and cursor_second_target == cursor_second + if carried_forward: cursor_sibling_ids |= seen_ids + demisto.debug( + f"AWSGuardDutyEventCollector - Cursor advance for {detector_id=}: " + f"boundary_second={cursor_second_target}, cursor_ts={cursor_ts!r}, " + f"same_second_sibling_count={len(cursor_sibling_ids)}, {carried_forward=}." + ) # Stored as list so demisto.setLastRun can JSON-serialize it; round-trips via # _normalize_last_ids_entry on the next call. new_last_ids[detector_id] = sorted(cursor_sibling_ids) elif finding_ids: - # No detector_events but we did see ids — keep the prior seen_ids as-is so - # we don't forget about them on the next fetch. - new_last_ids[detector_id] = sorted(seen_ids) if seen_ids else [] + # Everything listed deduped away as already-seen same-second siblings. If the fetch was not + # limit-truncated and the boundary second is safely in the past, step the cursor one second + # past it and clear seen ids to avoid a permanent stall; a recent boundary is left pinned. + now = datetime.utcnow() + boundary_is_stale = cursor_second < (now - STALE_BOUNDARY_MARGIN) + if not truncated_by_limit and boundary_is_stale: + next_second = cursor_second + timedelta(seconds=1) + new_collect_from[detector_id] = next_second.isoformat() + new_last_ids[detector_id] = [] + demisto.debug( + f"AWSGuardDutyEventCollector - Fully-deduped STALE boundary for {detector_id=}: " + f"{len(finding_ids)} finding id(s) listed but all deduped, nothing newer, and the " + f"boundary second {cursor_second} is older than {STALE_BOUNDARY_MARGIN} before " + f"now ({now}). Advancing cursor to {next_second.isoformat()} and clearing seen ids " + f"to break the stall." + ) + else: + # Bounded pause: keep prior seen ids for a still-current or limit-truncated boundary. + new_last_ids[detector_id] = sorted(seen_ids) if seen_ids else [] + demisto.debug( + f"AWSGuardDutyEventCollector - No new events for {detector_id=}: {len(finding_ids)} finding " + f"id(s) were listed but all were deduped as already-seen. Cursor left unchanged " + f"({truncated_by_limit=}, {boundary_is_stale=}); {len(seen_ids)} seen id(s) carried forward." + ) + else: + # Nothing matched the query this cycle; cursor unchanged. + demisto.debug( + f"AWSGuardDutyEventCollector - No findings returned for {detector_id=} this cycle. " f"Cursor left unchanged." + ) demisto.debug(f"AWSGuardDutyEventCollector - Total number of events is {len(events)}") + # Log the outgoing cursor value/type and per-detector last_ids counts before persisting. + demisto.debug( + "AWSGuardDutyEventCollector - Persisting state. " + f"new_collect_from={ {k: f'{v!r} ({type(v).__name__})' for k, v in new_collect_from.items()} }, " + f"new_last_ids_counts={ {k: len(v) for k, v in new_last_ids.items()} }" + ) events = convert_events_with_datetime_to_str(events) return events, new_last_ids, new_collect_from diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.yml b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.yml index 7e46f0a062b5..ad1277ebdba2 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.yml +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector.yml @@ -119,6 +119,22 @@ configuration: defaultvalue: 'false' additionalinfo: When selected, findings that have been archived in GuardDuty (including those archived by suppression rules) are not fetched. section: Collect +- display: Fetch Events + name: isFetchEvents + type: 8 + required: false + section: Collect + hidden: + - xsoar +- display: Events Fetch Interval + name: eventFetchInterval + type: 19 + required: false + defaultvalue: '1' + section: Collect + advanced: true + hidden: + - xsoar description: Amazon Web Services Guard Duty Service (gd) event collector integration for Cortex XSIAM. script: script: "" @@ -148,7 +164,7 @@ script: name: limit description: Manual command used to fetch events and display them. name: aws-gd-get-events - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 isfetchevents: true subtype: python3 marketplaces: diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector_test.py b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector_test.py index c24e9d27828e..7f0e089d7336 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector_test.py +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/AWSGuardDutyEventCollector_test.py @@ -1,10 +1,18 @@ +import copy from contextlib import nullcontext as does_not_raise -from datetime import datetime +from datetime import datetime, timedelta, timezone from unittest.mock import call import demistomock as demisto import pytest -from AWSGuardDutyEventCollector import _event_updated_at, _normalize_last_ids_entry, get_events +from AWSGuardDutyEventCollector import ( + STALE_BOUNDARY_MARGIN, + _cursor_second, + _event_updated_at, + _normalize_last_ids_entry, + get_events, +) +from CommonServerPython import arg_to_datetime from test_data.finding_for_test import FINDING, FINDING_OUTPUT, MOST_GENERAL_FINDING, MOST_GENERAL_FINDING_STR LIST_DETECTORS_RESPONSE = {"DetectorIds": ["detector_id1"]} @@ -18,6 +26,28 @@ FINDINGS = {"Findings": [FINDING]} +@pytest.fixture(autouse=True) +def isolate_shared_findings(): + """Protect shared module-level test data from cross-test mutation bleed. + + Many tests build inputs via ``FINDING.copy()`` / ``FINDING_OUTPUT.copy()``, + which is a SHALLOW copy — nested dicts/lists are shared with the module-level + original, so an in-place mutation in one test could leak into another and + cause order-dependent flakiness. This autouse fixture snapshots the shared + mutable objects with ``copy.deepcopy`` before each test and restores their + contents afterwards, guaranteeing every test starts from a pristine copy + regardless of what a previous test mutated. + """ + shared = [FINDING, FINDING_OUTPUT, MOST_GENERAL_FINDING] + snapshots = [copy.deepcopy(obj) for obj in shared] + try: + yield + finally: + for obj, snapshot in zip(shared, snapshots): + obj.clear() + obj.update(snapshot) + + def get_expected_list_finding_args( detector_id: str, updated_at_ts: int, gd_severity: int, max_results: int | None, next_token: str | None ): @@ -139,7 +169,7 @@ def test_test_module(mocker): ), call( **get_expected_list_finding_args( - detector_id="detector_id2", updated_at_ts=1661681559000, gd_severity=1, max_results=9, next_token=None + detector_id="detector_id2", updated_at_ts=1661681559000, gd_severity=1, max_results=10, next_token=None ) ), ], @@ -460,7 +490,7 @@ def test_get_events_returns_datetime_as_str(mocker, list_detectors_res, list_fin [call(DetectorId="detector_id1", FindingIds=["finding_id1"])], [update_finding_id(FINDING_OUTPUT.copy(), "finding_id1", updated_at="2022-09-28T10:12:39.923854")], {"detector_id1": "2022-09-28T10:12:39.923854"}, - # XSUP-67097: last_ids is now stored as list[str] of all ids sharing + # last_ids is now stored as list[str] of all ids sharing # the cursor's UpdatedAt second, not just one id. {"detector_id1": ["finding_id1"]}, id="1 detector, 1 new finding", @@ -470,7 +500,7 @@ def test_get_events_returns_datetime_as_str(mocker, list_detectors_res, list_fin {"detector_id1": "finding_id0"}, [{"DetectorIds": ["detector_id1"]}], [{"FindingIds": ["finding_id0", "finding_id1"]}], - # XSUP-72455: both ids are fetched so the collector can inspect each finding's + # Both ids are fetched so the collector can inspect each finding's # UpdatedAt. finding_id0 still shares the cursor second (already ingested) and is # deduped after the fetch; finding_id1 is new and is ingested. [ @@ -495,29 +525,6 @@ def test_get_events_returns_datetime_as_str(mocker, list_detectors_res, list_fin {"detector_id1": ["finding_id1"]}, id="1 detector, 1 new finding, 1 old finding", ), - pytest.param( - {"detector_id1": "2022-08-28T10:12:39.923854"}, - {"detector_id1": "finding_id0"}, - [{"DetectorIds": ["detector_id1"]}], - [{"FindingIds": ["finding_id0"]}], - # XSUP-72455: the id is fetched so its UpdatedAt can be inspected. It still shares the - # cursor second (already ingested), so it is deduped after the fetch — no events, cursor unchanged. - [{"Findings": [update_finding_id(FINDING.copy(), "finding_id0", updated_at="2022-08-28T10:12:39.923854")]}], - [call(MaxResults=50)], - [ - call( - **get_expected_list_finding_args( - detector_id="detector_id1", updated_at_ts=1661681559000, gd_severity=1, max_results=10, next_token=None - ) - ) - ], - [call(DetectorId="detector_id1", FindingIds=["finding_id0"])], - [], - {"detector_id1": "2022-08-28T10:12:39.923854"}, - # last_ids preserves the already-seen id at the unchanged cursor second. - {"detector_id1": ["finding_id0"]}, - id="1 detector, 1 old finding", - ), pytest.param( {"detector_id1": "2022-08-28T10:12:39.923854"}, {"detector_id1": "finding_id0"}, @@ -556,7 +563,7 @@ def test_get_events_returns_datetime_as_str(mocker, list_detectors_res, list_fin ), call( **get_expected_list_finding_args( - detector_id="detector_id2", updated_at_ts=1659003099000, gd_severity=1, max_results=9, next_token=None + detector_id="detector_id2", updated_at_ts=1659003099000, gd_severity=1, max_results=10, next_token=None ) ), ], @@ -626,8 +633,53 @@ def test_fetch_events( assert new_last_ids == expected_new_last_ids +def test_fetch_events_single_recent_old_finding_stays_pinned(mocker): + """ + Given: + One detector whose cursor sits at a RECENT second, and the only finding + returned is the already-ingested one sharing that cursor second (no + NextToken). Extracted from the former parametrized "1 detector, 1 old + finding" case; the boundary is recent so it is within + STALE_BOUNDARY_MARGIN. + + When: + get_events runs. + + Then: + The already-seen finding is deduped (no events), and because the boundary + is still current the cursor stays pinned with last_ids preserved. (A stale + boundary would instead advance via the freeze-breaker.) + + Reference: + AWSGuardDutyEventCollector.get_events — same-second dedup pins a current + boundary; the stale-boundary advance is time-gated. + """ + cursor = datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%S.%f") + + mocked_client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["detector_id1"]}], + list_finding_ids_res=[{"FindingIds": ["finding_id0"]}], + get_findings_res=[{"Findings": [update_finding_id(FINDING.copy(), "finding_id0", updated_at=cursor)]}], + ) + + events, new_last_ids, new_collect_from = get_events( + aws_client=mocked_client, + collect_from={"detector_id1": cursor}, + collect_from_default=datetime(2022, 7, 28, 10, 11, 39, 923854), + last_ids={"detector_id1": "finding_id0"}, + severity="Low", + limit=10, + ) + + assert events == [] + # Current boundary: cursor pinned and the already-seen id preserved. + assert new_collect_from == {"detector_id1": cursor} + assert new_last_ids == {"detector_id1": ["finding_id0"]} + + # --------------------------------------------------------------------------- -# Regression test for XSUP-67097 / XSUP-67552 — same-second sibling-loss bug. +# Regression test for the same-second sibling-loss bug. # # Scenario: three findings (A, B, C) all share the same UpdatedAt timestamp. # Run 1: AWS pagination returns [A, B] (limit=2 cuts off page 2). @@ -689,7 +741,7 @@ def test_same_second_sibling_loss_xsup_67097(mocker): run1_ids = sorted(e["Id"] for e in events_run1) assert run1_ids == ["finding_A", "finding_B"], f"Sanity check failed: run 1 should ingest A and B, got {run1_ids}" - # Cursor state after run 1: with the XSUP-67097 fix, last_ids stores ALL ids + # Cursor state after run 1: last_ids stores ALL ids # whose UpdatedAt equals the cursor (the same-second siblings), not just one. # Both A and B share the cursor second so both must be remembered. assert last_ids_after_run1 == {"det1": ["finding_A", "finding_B"]} @@ -726,7 +778,7 @@ def test_same_second_sibling_loss_xsup_67097(mocker): # once. Today's code drops C — this assertion fails as proof of the bug. all_ingested_ids = sorted(e["Id"] for e in (events_run1 + events_run2)) assert all_ingested_ids == ["finding_A", "finding_B", "finding_C"], ( - f"XSUP-67097 same-second sibling loss: expected all three findings " + f"Same-second sibling loss: expected all three findings " f"to be ingested across the two fetch cycles, but got {all_ingested_ids}. " f"Finding 'finding_C' was silently dropped because the dedup at " f"AWSGuardDutyEventCollector.py:135 slices after the single stored " @@ -789,7 +841,7 @@ def test_legacy_last_ids_str_shape_still_works(mocker): # --------------------------------------------------------------------------- -# Regression test for XSUP-71079 — mid-second cursor truncation. +# Regression test for mid-second cursor truncation. # # Scenario: a fetch is truncated by `limit` in the MIDDLE of a second that has # more siblings than were fetched. The cursor must NOT advance into that @@ -865,7 +917,7 @@ def test_exclude_archived_adds_criterion_xsup_71079(mocker): (default) the criterion does NOT include the archived filter. Reference: - AWSGuardDutyEventCollector._build_finding_criterion — XSUP-71079 #2. + AWSGuardDutyEventCollector._build_finding_criterion. """ finding = update_finding_id(FINDING.copy(), "finding_1", updated_at="2026-04-10T01:35:09.000000") @@ -910,7 +962,7 @@ def test_exclude_archived_adds_criterion_xsup_71079(mocker): # --------------------------------------------------------------------------- -# Regression test for XSUP-72455 — a recurring finding that is RE-UPDATED after +# Regression test for a recurring finding that is RE-UPDATED after # being stored in last_ids must not be suppressed forever. # # Scenario: finding_X is ingested at cursor second T1 and stored in last_ids. @@ -968,13 +1020,12 @@ def test_reupdated_finding_is_not_suppressed_xsup_72455(mocker): # The re-updated finding is a legitimate new event and must be ingested. assert [e["Id"] for e in events] == ["finding_X"], ( - "XSUP-72455: a finding whose UpdatedAt advanced past the stored cursor " + "A finding whose UpdatedAt advanced past the stored cursor " "second was dropped by ID-only dedup, so no events were ingested." ) # The cursor must advance to the finding's new second (T2), not stay pinned at T1. assert new_collect_from == {"det1": t2}, ( - "XSUP-72455: cursor stayed pinned at the old second because the " - "re-updated finding was suppressed, blocking all forward progress." + "Cursor stayed pinned at the old second because the " "re-updated finding was suppressed, blocking all forward progress." ) # last_ids now reflects the new cursor second (T2), holding finding_X so a # same-second re-query does not re-ingest it. @@ -993,13 +1044,16 @@ def test_same_id_same_second_still_deduped_xsup_72455(mocker): Then: finding_X must be deduped (not re-ingested) and the cursor stays at T1. - This guards that the XSUP-72455 fix does not regress the original - same-second dedup (XSUP-67097) behavior. + This guards that the re-update fix does not regress the original + same-second dedup behavior. Reference: AWSGuardDutyEventCollector.get_events — same-second re-reads are still deduped. """ - t1 = "2026-07-03T15:48:55.563000" + # A RECENT cursor second: the re-read is deduped and the cursor stays pinned + # (a real later same-second update could still arrive). Using a fixed past + # second would trip the stale-boundary advance instead. + t1 = datetime.utcnow().replace(microsecond=563000).strftime("%Y-%m-%dT%H:%M:%S.%f") same_second_again = update_finding_id(FINDING.copy(), "finding_X", updated_at=t1) client, _, _, _ = create_mocked_client( @@ -1100,7 +1154,7 @@ def test_multi_detector_truncation_advances_cursors_independently(mocker): Reference: AWSGuardDutyEventCollector.get_events — per-detector cursor bookkeeping - (XSUP-71079 truncation rollback applied per detector). + (truncation rollback applied per detector). """ a_t1 = "2026-04-10T01:35:08.000000" a_t2 = "2026-04-10T01:35:09.000000" @@ -1108,23 +1162,25 @@ def test_multi_detector_truncation_advances_cursors_independently(mocker): finding_a1 = update_finding_id(FINDING.copy(), "finding_A1", updated_at=a_t1) finding_a2 = update_finding_id(FINDING.copy(), "finding_A2", updated_at=a_t2) + finding_a3 = update_finding_id(FINDING.copy(), "finding_A3", updated_at=a_t2) finding_b1 = update_finding_id(FINDING.copy(), "finding_B1", updated_at=b_t1) - # limit=3: det_A returns 3 finding ids on one page (reaching the limit so its - # loop exits by limit) but one id is a duplicate, so it ingests only 2 events - # (A1, A2) and leaves budget for det_B to drain its single finding. + # limit=3 is applied PER DETECTOR. det_A returns 3 unique finding ids on one + # page, filling its own limit so its loop exits by limit with a pending + # NextToken (truncated mid-boundary at the partial second A_t2). det_B then + # gets its own fresh limit and drains its single finding. client, _, _, _ = create_mocked_client( mocker=mocker, list_detectors_res=[{"DetectorIds": ["det_A", "det_B"]}], list_finding_ids_res=[ - # det_A: 3 ids fill the limit => loop exits by limit; NextToken still - # pending => truncated mid-boundary at the partial second A_t2. - {"FindingIds": ["finding_A1", "finding_A2", "finding_A2"], "NextToken": "more"}, - # det_B: no NextToken => fully drained. + # det_A: 3 unique ids fill the per-detector limit => loop exits by + # limit; NextToken still pending => truncated mid-boundary at A_t2. + {"FindingIds": ["finding_A1", "finding_A2", "finding_A3"], "NextToken": "more"}, + # det_B: no NextToken => fully drained (its own fresh limit budget). {"FindingIds": ["finding_B1"]}, ], get_findings_res=[ - {"Findings": [finding_a1, finding_a2]}, + {"Findings": [finding_a1, finding_a2, finding_a3]}, {"Findings": [finding_b1]}, ], ) @@ -1139,7 +1195,7 @@ def test_multi_detector_truncation_advances_cursors_independently(mocker): ) # All distinct fetched findings are still returned this run (nothing dropped). - assert sorted(e["Id"] for e in events) == ["finding_A1", "finding_A2", "finding_B1"] + assert sorted(e["Id"] for e in events) == ["finding_A1", "finding_A2", "finding_A3", "finding_B1"] # detector_A: truncated mid-boundary => cursor rolls back to the last # fully-drained second (A_t1), NOT the partial A_t2. last_ids holds A_t1's @@ -1153,6 +1209,76 @@ def test_multi_detector_truncation_advances_cursors_independently(mocker): assert new_last_ids["det_B"] == ["finding_B1"] +def test_busy_detector_does_not_starve_later_detector(mocker): + """ + Given: + Two detectors processed in order [det_busy, det_late]. det_busy alone + produces exactly `limit` findings on its first page (filling the budget), + while det_late has its own findings to return. + + When: + get_events processes both detectors in the same run with limit=2. + + Then: + The `limit` budget is applied PER DETECTOR, so det_busy consuming the + whole limit must NOT prevent det_late from being queried. Both detectors' + findings are ingested and both cursors advance. This guards against a + global-budget starvation defect where a busy earlier detector left later + detectors un-queried with their cursor frozen. + + Reference: + AWSGuardDutyEventCollector.get_events — per-detector `limit` budget. + """ + busy_t = "2026-04-10T03:00:00.000000" + late_t = "2026-04-10T04:00:00.000000" + + finding_busy1 = update_finding_id(FINDING.copy(), "finding_busy1", updated_at=busy_t) + finding_busy2 = update_finding_id(FINDING.copy(), "finding_busy2", updated_at=busy_t) + finding_late1 = update_finding_id(FINDING.copy(), "finding_late1", updated_at=late_t) + finding_late2 = update_finding_id(FINDING.copy(), "finding_late2", updated_at=late_t) + + # limit=2: det_busy returns 2 ids on its first (and only) page => it alone + # fills the entire limit. Under a global budget, det_late's finding-ids loop + # guard would be False on entry, so list_findings would NEVER be called for + # det_late and it would be starved. Per-detector budget prevents that. + client, _, list_findings_mock, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det_busy", "det_late"]}], + list_finding_ids_res=[ + {"FindingIds": ["finding_busy1", "finding_busy2"]}, # det_busy: fills limit, fully drained + {"FindingIds": ["finding_late1", "finding_late2"]}, # det_late: must still be queried + ], + get_findings_res=[ + {"Findings": [finding_busy1, finding_busy2]}, + {"Findings": [finding_late1, finding_late2]}, + ], + ) + + events, new_last_ids, new_collect_from = get_events( + aws_client=client, + collect_from={}, + collect_from_default=datetime(2026, 4, 10, 2, 0, 0), + last_ids={}, + severity="Low", + limit=2, + ) + + # det_late was queried (not starved): list_findings called for BOTH detectors. + assert list_findings_mock.call_count == 2 + # Both detectors' findings are ingested — the later detector is not dropped. + assert sorted(e["Id"] for e in events) == [ + "finding_busy1", + "finding_busy2", + "finding_late1", + "finding_late2", + ] + # Both cursors advance independently to their respective seconds. + assert new_collect_from["det_busy"] == busy_t + assert new_collect_from["det_late"] == late_t + assert sorted(new_last_ids["det_busy"]) == ["finding_busy1", "finding_busy2"] + assert sorted(new_last_ids["det_late"]) == ["finding_late1", "finding_late2"] + + # --------------------------------------------------------------------------- # Single-boundary-cannot-drain — an entire truncated page shares one UpdatedAt. # @@ -1242,8 +1368,11 @@ def test_same_boundary_reupdate_does_not_freeze_fetch_forever(mocker): AWSGuardDutyEventCollector.get_events — dedup is scoped to the cursor second, so a later-boundary update always resumes forward progress. """ - t1 = "2026-07-03T15:48:55.563000" - t2 = "2026-07-04T08:04:55.843000" + # T1 is a RECENT second so the same-second re-reads stay pinned (not tripped by + # the stale-boundary advance). T2 is a strictly-later second. + now = datetime.utcnow() + t1 = now.replace(microsecond=563000).strftime("%Y-%m-%dT%H:%M:%S.%f") + t2 = (now + timedelta(seconds=17)).replace(microsecond=843000).strftime("%Y-%m-%dT%H:%M:%S.%f") # ---- Runs that keep re-reading finding_X at the SAME second T1 ---------- for run_number in range(3): @@ -1353,3 +1482,766 @@ def test_emitted_events_expose_updated_at_for_time_mapping(mocker): # Each event's _time source is the expected, distinct UpdatedAt value. assert sorted(_event_updated_at(e) for e in events) == [ts_1, ts_2] + + +# --------------------------------------------------------------------------- +# Regression test for sub-second (millisecond) cursor churn. +# +# Production GuardDuty findings carry MILLISECOND-precision UpdatedAt values +# (e.g. "...:23.311Z", "...:23.936Z"). Several findings routinely share the +# same whole SECOND while differing in their milliseconds. The fetch cursor is +# meant to be a per-second boundary, and every +# finding sharing the cursor's second must be remembered so the inclusive Gte +# re-query can dedup them. +# +# Two defects combined to re-ingest the whole second on every run (observed in +# live telemetry: the same (Id, UpdatedAt) pair ingested up to 1440x/day): +# 1. The cursor was set from detector_events[-1] (LIST order), not the max +# timestamp. AWS get_findings does NOT preserve the requested id order, so +# the cursor oscillated between the earliest/latest ms of the same second. +# 2. The remembered sibling set was scoped to the EXACT-microsecond cursor +# value, so only the one finding at that precise ms was remembered — the +# other same-second siblings were forgotten and re-ingested next run. +# +# This test reproduces the loop: three findings in one second at different ms, +# returned by get_findings in a NON-sorted order, run across two fetch cycles +# with the state round-tripped as a list (as setLastRun would persist it). +# --------------------------------------------------------------------------- + + +def test_millisecond_siblings_not_reingested_xsup_73410(mocker): + """ + Given: + Three findings (A, B, C) on one detector, all sharing the same whole + SECOND 13:18:23 but with different MILLISECONDS (.311 / .627 / .936). + get_findings returns them in a NON-sorted order — valid AWS behavior, + since get_findings does not preserve FindingIds order. + + When: + Run 1 fetches from before the second and ingests all three. + Run 2 re-queries with the persisted cursor (state round-tripped as a + list, as setLastRun serializes it). The inclusive Gte re-returns the + same three findings, again in a non-sorted order. + + Then: + Run 1 ingests A, B, C exactly once. + Run 2 ingests NOTHING — all three are recognized as the same + already-seen same-second findings and deduped. + The persisted cursor is deterministic (the max, .936) and stable across + runs (no oscillation), and last_ids remembers ALL three siblings. + + With the pre-fix code, run 2 re-ingests all three and the cursor + oscillates within the second, so these assertions fail — proving the + millisecond-churn re-ingestion bug. + + Reference: + AWSGuardDutyEventCollector.get_events — cursor advance + same-second + sibling dedup must operate at whole-second resolution. + """ + # Use a RECENT whole-second (within STALE_BOUNDARY_MARGIN of now) so the + # fully-deduped re-query in run 2 correctly stays pinned rather than tripping + # the stale-boundary freeze-breaker. A real later same-second update could + # still arrive at a current second, so pinning is the correct behavior here. + base_second = datetime.utcnow().replace(microsecond=0) + ts_a = base_second.replace(microsecond=311000).strftime("%Y-%m-%dT%H:%M:%S.%f") + ts_b = base_second.replace(microsecond=627000).strftime("%Y-%m-%dT%H:%M:%S.%f") + ts_c = base_second.replace(microsecond=936000).strftime("%Y-%m-%dT%H:%M:%S.%f") + + def make_findings(): + # Fresh copies each run so mutations never leak between cycles. + finding_a = update_finding_id(FINDING.copy(), "finding_A", updated_at=ts_a) + finding_b = update_finding_id(FINDING.copy(), "finding_B", updated_at=ts_b) + finding_c = update_finding_id(FINDING.copy(), "finding_C", updated_at=ts_c) + return finding_a, finding_b, finding_c + + # ------------------------------------------------------------------ Run 1 + fa1, fb1, fc1 = make_findings() + run1_client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + list_finding_ids_res=[{"FindingIds": ["finding_A", "finding_B", "finding_C"]}], + # get_findings returns them in a DIFFERENT order than requested. + get_findings_res=[{"Findings": [fc1, fa1, fb1]}], + ) + + events_run1, last_ids_after_run1, collect_from_after_run1 = get_events( + aws_client=run1_client, + collect_from={}, + collect_from_default=datetime(2026, 8, 1, 13, 18, 0), + last_ids={}, + severity="Low", + limit=10, + ) + + assert sorted(e["Id"] for e in events_run1) == [ + "finding_A", + "finding_B", + "finding_C", + ], "Sanity: run 1 must ingest all three same-second findings." + # The cursor must be the MAX timestamp of the second — deterministic, not + # whichever finding happened to be last in get_findings order. + assert collect_from_after_run1 == { + "det1": ts_c + }, f"Cursor must advance to the max same-second timestamp, got {collect_from_after_run1}." + # Every finding sharing the cursor second must be remembered for dedup. + assert last_ids_after_run1 == { + "det1": ["finding_A", "finding_B", "finding_C"] + }, f"All same-second siblings must be remembered, got {last_ids_after_run1}." + + # ------------------------------------------------------------------ Run 2 + # The inclusive Gte re-query returns the same three findings again, in yet + # another order. State is round-tripped as a list (setLastRun serialization). + mocker.resetall() + fa2, fb2, fc2 = make_findings() + run2_client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + list_finding_ids_res=[{"FindingIds": ["finding_A", "finding_B", "finding_C"]}], + get_findings_res=[{"Findings": [fb2, fc2, fa2]}], + ) + + events_run2, last_ids_after_run2, collect_from_after_run2 = get_events( + aws_client=run2_client, + collect_from=collect_from_after_run1, + collect_from_default=datetime(2026, 8, 1, 13, 18, 0), + last_ids=last_ids_after_run1, + severity="Low", + limit=10, + ) + + # ------------------------------------------------------------ Assertion + # Nothing new happened — the same second's findings must NOT be re-ingested. + assert events_run2 == [], ( + f"Millisecond re-ingest: the same same-second findings were " + f"re-ingested on run 2, got {[e['Id'] for e in events_run2]}. The inclusive " + f"Gte re-query returns them and they must all be deduped as already-seen." + ) + # The cursor must be STABLE — no oscillation within the second across runs. + assert collect_from_after_run2 == {"det1": ts_c}, ( + f"Cursor must remain stable at the max same-second timestamp across runs, " + f"got {collect_from_after_run2} (oscillation indicates the churn bug)." + ) + assert last_ids_after_run2 == {"det1": ["finding_A", "finding_B", "finding_C"]} + + +# --------------------------------------------------------------------------- +# Regression test for the FULLY-DEDUPED STALE BOUNDARY +# dead-end. +# +# The cursor is second-resolution and the updatedAt query is inclusive (Gte). +# On a quiet detector the fetch can reach a state where the ONLY finding at/after +# the cursor second is one already in last_ids. Every cycle then: +# 1. list_findings returns just that one already-seen id (nothing newer), +# 2. dedup drops it as a same-second sibling -> 0 events, +# 3. the cursor only advances when events are ingested -> it never moves. +# The fetch is wedged forever, even though the boundary second is long in the +# past. Live evidence: the cursor sat frozen at a single second for 9+ hours, +# re-querying one finding, ingesting nothing. +# +# The fix is time-gated: when a fetch is NOT truncated by limit (AWS returned +# everything at/after the cursor) and everything deduped away, AND the boundary +# second is safely in the PAST relative to collect_from_default (so no new +# same-second finding can still arrive), advance the cursor one second past the +# boundary and drop the stale seen_ids. A RECENT boundary must still stay pinned +# (that bounded pause is required by test_same_boundary_reupdate_...). +# --------------------------------------------------------------------------- + + +def test_stale_fully_deduped_boundary_advances_cursor_xsup_71079(mocker): + """ + Given: + A quiet detector whose cursor sits at a second T0 that is well in the + PAST (hours before collect_from_default/"now"). The only finding the + inclusive Gte query returns is finding_X, which is already in last_ids + (an already-ingested same-second finding). The fetch is NOT truncated by + limit — AWS returned everything at/after T0 and there is nothing newer. + + When: + get_events runs this cycle. + + Then: + Nothing is ingested (finding_X is correctly deduped), but because the + boundary second is stale and fully drained, the cursor MUST advance one + second past T0 so the next run queries strictly after the dead boundary, + and the stale seen_ids for that detector MUST be cleared. This breaks the + perpetual freeze. + + With the pre-fix code the elif-finding_ids branch leaves the cursor and + seen_ids unchanged, so the cursor stays frozen at T0 forever — these + assertions fail, proving the dead-end. + + Reference: + AWSGuardDutyEventCollector.get_events — the fully-deduped-boundary + advance guard (time-gated so a recent boundary still pins). + """ + # A fixed past boundary is stale against real wall-clock now — no mocking needed. + # collect_from_default is the first_fetch lookback FLOOR and is intentionally + # unrelated to the boundary; the guard must not depend on it. + collect_from_default_floor = datetime(2020, 1, 1, 0, 0, 0) + t0 = "2020-01-01T08:19:28.960000" # years in the past — clearly stale + t0_plus_1s = "2020-01-01T08:19:29" # cursor must step to the next second + + stale_only = update_finding_id(FINDING.copy(), "finding_X", updated_at=t0) + client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + # Only the already-seen boundary finding is returned; NO NextToken -> not + # truncated by limit, so AWS has nothing newer at/after T0. + list_finding_ids_res=[{"FindingIds": ["finding_X"]}], + get_findings_res=[{"Findings": [stale_only]}], + ) + + events, new_last_ids, new_collect_from = get_events( + aws_client=client, + collect_from={"det1": t0}, + collect_from_default=collect_from_default_floor, + last_ids={"det1": ["finding_X"]}, + severity="Low", + limit=10, + ) + + # Nothing new happened this cycle — the one boundary finding is deduped. + assert events == [], "The already-seen boundary finding must not be re-ingested." + # The wedge must break: the cursor steps one whole second past the stale + # boundary so the next inclusive Gte query starts strictly after it. + assert new_collect_from == {"det1": t0_plus_1s}, ( + f"Stale fully-deduped boundary must advance the cursor past T0, got " + f"{new_collect_from} (unchanged == the permanent freeze)." + ) + # The stale seen_ids belonged to the dead boundary second; the new second has + # no known siblings, so they must be cleared. + assert new_last_ids == {"det1": []}, f"Seen ids for the abandoned boundary second must be cleared, got {new_last_ids}." + + +def test_recent_fully_deduped_boundary_stays_pinned_xsup_71079(mocker): + """ + Given: + The same fully-deduped single-finding situation as the stale-boundary + test, but the cursor second is RECENT — only moments before + collect_from_default/"now". A genuine later-same-second update could + still arrive, so the cursor must not jump past it yet. + + When: + get_events runs this cycle. + + Then: + Nothing is ingested and the cursor stays PINNED at the boundary second + with its seen_ids preserved (the bounded, correct pause). This guards the + time-gate: the freeze-breaking advance must fire ONLY for stale + boundaries, never for a still-current second. + + Reference: + AWSGuardDutyEventCollector.get_events — the fully-deduped-boundary + advance is time-gated; a recent boundary keeps the existing pin behavior. + """ + # Boundary is only a few seconds before real wall-clock now — still current, + # so it falls inside STALE_BOUNDARY_MARGIN and must stay pinned. No mocking. + now = datetime.utcnow() + t_recent = (now - timedelta(seconds=3)).strftime("%Y-%m-%dT%H:%M:%S.%f") + + recent_only = update_finding_id(FINDING.copy(), "finding_X", updated_at=t_recent) + client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + list_finding_ids_res=[{"FindingIds": ["finding_X"]}], + get_findings_res=[{"Findings": [recent_only]}], + ) + + events, new_last_ids, new_collect_from = get_events( + aws_client=client, + collect_from={"det1": t_recent}, + collect_from_default=now, + last_ids={"det1": ["finding_X"]}, + severity="Low", + limit=10, + ) + + assert events == [], "The already-seen finding must not be re-ingested." + # Recent boundary: keep the bounded pause so a later same-second update is + # still catchable — cursor and seen_ids unchanged. + assert new_collect_from == {"det1": t_recent}, f"A recent fully-deduped boundary must stay pinned, got {new_collect_from}." + assert new_last_ids == { + "det1": ["finding_X"] + }, f"Seen ids for a still-current boundary must be preserved, got {new_last_ids}." + + +def test_stale_boundary_with_a_genuinely_new_finding_ingests_and_does_not_skip_xsup_71079(mocker): + """ + Given: + A stale cursor second T0. This cycle the inclusive Gte query returns the + already-seen boundary finding (finding_old, in last_ids at T0) AND a + genuinely new finding (finding_new) whose UpdatedAt is a strictly-later + second T1 — a real occurrence that must be ingested. + + When: + get_events runs. + + Then: + finding_new is ingested (it is NOT deduped — its second is past the + cursor) and the cursor advances to T1 via the NORMAL ingest path. The + stale-boundary freeze-breaker must NOT fire here (there are events), and + crucially nothing is skipped: a real update at a stale boundary is never + lost by the freeze-breaking logic. + + Reference: + AWSGuardDutyEventCollector.get_events — the stale-boundary advance lives + in the elif-finding_ids (zero-events) branch only; any real ingestion + takes the normal path and is unaffected. + """ + t0 = "2020-01-01T08:19:28.960000" # stale boundary (already seen) + t1 = "2020-01-01T08:19:45.100000" # a genuinely newer second + + finding_old = update_finding_id(FINDING.copy(), "finding_old", updated_at=t0) + finding_new = update_finding_id(FINDING.copy(), "finding_new", updated_at=t1) + client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + list_finding_ids_res=[{"FindingIds": ["finding_old", "finding_new"]}], + get_findings_res=[{"Findings": [finding_old, finding_new]}], + ) + + events, new_last_ids, new_collect_from = get_events( + aws_client=client, + collect_from={"det1": t0}, + collect_from_default=datetime(2020, 1, 1, 0, 0, 0), + last_ids={"det1": ["finding_old"]}, + severity="Low", + limit=10, + ) + + # The genuinely new finding must be ingested (the old boundary one is deduped). + assert [e["Id"] for e in events] == [ + "finding_new" + ], f"A real new finding at a stale boundary must be ingested, got {[e['Id'] for e in events]}." + # Cursor advances to the new finding's second via the normal ingest path — + # NOT via the freeze-breaker (+1s), proving the guard did not fire. + assert new_collect_from == {"det1": t1}, f"Cursor must advance to the ingested finding's timestamp, got {new_collect_from}." + assert new_last_ids == { + "det1": ["finding_new"] + }, f"Only the new cursor-second sibling should be remembered, got {new_last_ids}." + + +def test_truncated_stale_fully_deduped_boundary_does_not_advance_xsup_71079(mocker): + """ + Given: + A stale cursor second T0 whose only returned finding is already-seen (so + it dedups to zero events), BUT the finding-id listing was TRUNCATED by + limit — list_findings returned a NextToken, meaning AWS still has + un-fetched findings that may sit at this same boundary second. + + When: + get_events runs. + + Then: + The cursor must NOT advance and seen_ids must be preserved. Even though + the boundary is stale, advancing past a limit-truncated second could skip + the un-fetched findings still sitting there. The freeze-breaker is gated + on `not truncated_by_limit` precisely to prevent that data loss. + + Reference: + AWSGuardDutyEventCollector.get_events — the stale-boundary advance + requires a fully-drained (non-truncated) page. + """ + t0 = "2020-01-01T08:19:28.960000" # stale boundary + + stale_seen = update_finding_id(FINDING.copy(), "finding_X", updated_at=t0) + client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + # NextToken present => the id-listing loop stops on limit with more to + # fetch, so truncated_by_limit is True. + list_finding_ids_res=[{"FindingIds": ["finding_X"], "NextToken": "more"}], + get_findings_res=[{"Findings": [stale_seen]}], + ) + + events, new_last_ids, new_collect_from = get_events( + aws_client=client, + collect_from={"det1": t0}, + collect_from_default=datetime(2020, 1, 1, 0, 0, 0), + last_ids={"det1": ["finding_X"]}, + severity="Low", + limit=1, # force the listing loop to stop with the NextToken still pending + ) + + assert events == [], "Nothing new should be ingested." + # Truncated page => must NOT advance past the boundary (un-fetched siblings + # may still be there); cursor and seen_ids stay put. + assert new_collect_from == { + "det1": t0 + }, f"A limit-truncated stale boundary must NOT advance the cursor, got {new_collect_from}." + assert new_last_ids == { + "det1": ["finding_X"] + }, f"Seen ids must be preserved when the boundary page was truncated, got {new_last_ids}." + + +def test_stale_wedged_detector_advances_without_affecting_healthy_detector_xsup_71079(mocker): + """ + Given: + Two detectors on one instance: + - det_stuck: wedged at a stale fully-deduped boundary (only its + already-seen finding is returned). + - det_ok: healthy, returning a brand-new finding to ingest. + + When: + get_events runs over both detectors in one cycle. + + Then: + det_stuck's cursor is unwedged (advances one second past its stale + boundary, seen_ids cleared) while det_ok ingests its new finding and + advances normally. The freeze-breaker is per-detector and must not + disturb a healthy detector processed in the same run. + + Reference: + AWSGuardDutyEventCollector.get_events — per-detector cursor/seen-ids + handling; the stale-boundary advance is scoped to the affected detector. + """ + t_stale = "2020-01-01T08:19:28.960000" + t_stale_plus_1s = "2020-01-01T08:19:29" + t_new = "2020-01-01T17:59:10.500000" # healthy detector's new finding (ingested via normal path) + + stuck_seen = update_finding_id(FINDING.copy(), "finding_stuck", updated_at=t_stale) + ok_new = update_finding_id(FINDING.copy(), "finding_ok", updated_at=t_new) + + # Detectors are processed in listing order: det_stuck first, then det_ok. + client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det_stuck", "det_ok"]}], + list_finding_ids_res=[ + {"FindingIds": ["finding_stuck"]}, # det_stuck listing + {"FindingIds": ["finding_ok"]}, # det_ok listing + ], + get_findings_res=[ + {"Findings": [stuck_seen]}, # det_stuck get_findings + {"Findings": [ok_new]}, # det_ok get_findings + ], + ) + + events, new_last_ids, new_collect_from = get_events( + aws_client=client, + collect_from={"det_stuck": t_stale}, # det_ok has no prior cursor + collect_from_default=datetime(2020, 1, 1, 0, 0, 0), + last_ids={"det_stuck": ["finding_stuck"]}, + severity="Low", + limit=10, + ) + + # Only the healthy detector's new finding is ingested. + assert [e["Id"] for e in events] == [ + "finding_ok" + ], f"Healthy detector's new finding must be ingested, got {[e['Id'] for e in events]}." + # Stuck detector unwedged; healthy detector advanced normally to its finding. + assert new_collect_from == {"det_stuck": t_stale_plus_1s, "det_ok": t_new}, ( + f"Stuck detector must advance past its stale boundary and healthy detector " + f"must advance to its ingested finding, got {new_collect_from}." + ) + # Stuck detector's stale seen_ids cleared; healthy detector remembers its sibling. + assert new_last_ids == {"det_stuck": [], "det_ok": ["finding_ok"]}, f"Per-detector seen ids incorrect, got {new_last_ids}." + + +def test_z_suffixed_stale_cursor_is_comparable_to_wallclock_now_xsup_71079(mocker): + """ + Given: + The cursor is a real GuardDuty timestamp string in UTC "Z" form + (e.g. "2020-01-01T08:19:28.960Z"), matching what GuardDuty emits, and the + only finding returned is already-seen (fully deduped, not truncated). + + When: + get_events evaluates the stale-boundary guard, which compares the parsed + cursor second against datetime.utcnow(). + + Then: + The comparison must not raise (parse_date_string yields a NAIVE UTC + datetime for "Z" input, and utcnow() is also naive UTC), and the stale + boundary must advance normally. This pins the naive-UTC assumption so a + future change to timestamp parsing can't silently break the comparison + (e.g. by introducing an offset-aware datetime, which would raise + "can't compare offset-naive and offset-aware datetimes"). + + Reference: + AWSGuardDutyEventCollector.get_events — staleness compares the parsed + cursor second to naive-UTC now. + """ + t0_z = "2020-01-01T08:19:28.960Z" # real GuardDuty UTC "Z" form, years past + t0_plus_1s = "2020-01-01T08:19:29" + + stale_only = update_finding_id(FINDING.copy(), "finding_X", updated_at=t0_z) + client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + list_finding_ids_res=[{"FindingIds": ["finding_X"]}], + get_findings_res=[{"Findings": [stale_only]}], + ) + + with does_not_raise(): + events, new_last_ids, new_collect_from = get_events( + aws_client=client, + collect_from={"det1": t0_z}, + collect_from_default=datetime(2020, 1, 1, 0, 0, 0), + last_ids={"det1": ["finding_X"]}, + severity="Low", + limit=10, + ) + + assert events == [], "The already-seen boundary finding must not be re-ingested." + assert new_collect_from == {"det1": t0_plus_1s}, ( + f"A Z-suffixed stale boundary must compare cleanly to naive-UTC now and advance, " f"got {new_collect_from}." + ) + assert new_last_ids == {"det1": []}, f"Seen ids for the abandoned boundary second must be cleared, got {new_last_ids}." + + +# --------------------------------------------------------------------------- +# Regression tests for timezone normalization in _cursor_second. +# +# collect_from_default (the first_fetch floor) reaches get_events tz-AWARE when +# parsed by arg_to_datetime for an absolute date (e.g. "2026-08-01T00:00:00Z"), +# while finding UpdatedAt values are parsed by parse_date_string (no %z) and are +# always NAIVE. Before normalization, comparing the two either never matched +# (dedup silently failed -> every finding re-ingested) or raised +# "TypeError: can't compare offset-naive and offset-aware datetimes" against the +# naive datetime.utcnow() in the stale-boundary check. _cursor_second must render +# every value naive-UTC so all comparisons are clean. +# --------------------------------------------------------------------------- + + +def test_cursor_second_normalizes_tz_aware_to_naive_utc(): + """ + Given: + A tz-aware timestamp (both a "Z"/UTC value and a "+05:00" offset value) + and a naive value. + + When: + _cursor_second is called on each. + + Then: + The result is always naive (tzinfo is None), truncated to whole seconds, + and the offset value is converted to UTC BEFORE tzinfo is dropped (so a + +05:00 time is shifted back 5 hours, not merely stripped). + """ + # tz-aware UTC ("Z") input -> naive-UTC, seconds truncated. + aware_utc = arg_to_datetime("2026-08-01T00:00:00Z") + result_utc = _cursor_second(aware_utc) + assert result_utc.tzinfo is None + assert result_utc == datetime(2026, 8, 1, 0, 0, 0) + + # tz-aware +05:00 input -> converted to UTC (05:00 -> 00:00) THEN made naive. + aware_offset = datetime(2026, 8, 1, 5, 0, 0, tzinfo=timezone(timedelta(hours=5))) + assert _cursor_second(aware_offset) == datetime(2026, 8, 1, 0, 0, 0) + + # Naive input is unchanged apart from microsecond truncation. + assert _cursor_second(datetime(2026, 8, 1, 0, 0, 0, 123456)) == datetime(2026, 8, 1, 0, 0, 0) + + # A naive string (as produced by parse_date_string) truncates to the second. + assert _cursor_second("2026-08-01T00:00:00.936Z") == datetime(2026, 8, 1, 0, 0, 0) + + +def test_tz_aware_collect_from_default_dedups_without_typeerror(mocker): + """ + Given: + A RECENT cursor whose collect_from_default is tz-AWARE (as arg_to_datetime + returns for an absolute first_fetch), and the only finding the inclusive + Gte query returns is one already in last_ids at that same whole second + (naive UpdatedAt from parse_date_string). + + When: + get_events runs. + + Then: + The tz-aware cursor and the naive finding timestamp are both normalized to + naive-UTC, so the same-second dedup matches: no TypeError is raised and the + already-seen finding is NOT re-ingested (events == []). A recent boundary + stays pinned (not tripped by the stale-boundary advance). + + Before the normalization fix, the naive/aware mismatch made dedup never + match (re-ingesting the finding) or raised a TypeError in the stale-boundary + comparison — this test guards both. + + Reference: + AWSGuardDutyEventCollector._cursor_second — tz normalization at the single + comparison choke point. + """ + # A recent whole-second, expressed as a tz-aware "Z" string, matching how an + # absolute first_fetch is stored/parsed. The finding shares this exact second. + base_second = datetime.utcnow().replace(microsecond=0) + cursor_str = base_second.strftime("%Y-%m-%dT%H:%M:%S") + "Z" + collect_from_default_aware = arg_to_datetime(cursor_str) + finding_ts = base_second.strftime("%Y-%m-%dT%H:%M:%S.%f") + + mocked_client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + list_finding_ids_res=[{"FindingIds": ["finding_seen"]}], + get_findings_res=[{"Findings": [update_finding_id(FINDING.copy(), "finding_seen", updated_at=finding_ts)]}], + ) + + with does_not_raise(): + events, new_last_ids, new_collect_from = get_events( + aws_client=mocked_client, + collect_from={"det1": cursor_str}, + collect_from_default=collect_from_default_aware, + last_ids={"det1": ["finding_seen"]}, + severity="Low", + limit=10, + ) + + assert events == [], ( + "A tz-aware cursor and a naive finding timestamp at the same second must " + "dedup cleanly; the already-seen finding must not be re-ingested." + ) + # Recent boundary: cursor stays pinned and the seen id is preserved for next run. + assert new_collect_from == {"det1": cursor_str} + assert new_last_ids == {"det1": ["finding_seen"]} + + +def test_cursor_second_raises_on_unparseable_value(): + """ + Given: + A value that is not a parseable timestamp. + + When: + _cursor_second is called on it. + + Then: + It raises (surfacing loudly rather than silently corrupting the cursor), + as promised by the docstring. + """ + with pytest.raises((ValueError, TypeError)): + _cursor_second("not-a-timestamp") + + +def test_event_updated_at_falls_back_to_created_at(): + """ + Given: + A finding with no ``UpdatedAt`` but with a ``CreatedAt``, and a finding + that has both. + + When: + _event_updated_at is called. + + Then: + It returns ``CreatedAt`` when ``UpdatedAt`` is absent, and prefers + ``UpdatedAt`` when present. (Guards the CreatedAt fallback.) + """ + assert _event_updated_at({"CreatedAt": "2026-08-01T00:00:00.000Z"}) == "2026-08-01T00:00:00.000Z" + assert ( + _event_updated_at({"UpdatedAt": "2026-08-02T00:00:00.000Z", "CreatedAt": "2026-08-01T00:00:00.000Z"}) + == "2026-08-02T00:00:00.000Z" + ) + + +def test_finding_without_id_does_not_leak_none_into_last_ids(mocker): + """ + Given: + A single NEW finding at a recent second that is missing its ``Id`` field + (defensive: real GuardDuty findings always carry an Id). + + When: + get_events ingests it and builds the same-second sibling id set. + + Then: + The event is still ingested, but ``None`` must never appear in the + persisted ``last_ids`` (guards the ``discard(None)`` in cursor bookkeeping), + and the persisted list must remain JSON-serializable/sortable. + """ + recent = datetime.utcnow().replace(microsecond=0).strftime("%Y-%m-%dT%H:%M:%S.%f") + finding_no_id = update_finding_id(FINDING.copy(), "placeholder", updated_at=recent) + finding_no_id.pop("Id", None) + + mocked_client, _, _, _ = create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + list_finding_ids_res=[{"FindingIds": ["some_id"]}], + get_findings_res=[{"Findings": [finding_no_id]}], + ) + + _events, new_last_ids, _new_collect_from = get_events( + aws_client=mocked_client, + collect_from={"det1": recent}, + collect_from_default=datetime(2026, 8, 1, 0, 0, 0), + last_ids={}, + severity="Low", + limit=10, + ) + + assert None not in new_last_ids.get("det1", []), "None must never be persisted as a same-second sibling id." + + +def _fully_deduped_boundary_client(mocker, cursor_str): + """Build a client whose only finding is the already-seen boundary sibling.""" + return create_mocked_client( + mocker=mocker, + list_detectors_res=[{"DetectorIds": ["det1"]}], + list_finding_ids_res=[{"FindingIds": ["finding_seen"]}], + get_findings_res=[{"Findings": [update_finding_id(FINDING.copy(), "finding_seen", updated_at=cursor_str)]}], + )[0] + + +def test_stale_boundary_well_past_margin_advances_cursor(mocker): + """ + Given: + A fully-deduped boundary whose second is one minute older than + STALE_BOUNDARY_MARGIN. The only finding returned is already in last_ids, + nothing is newer, and the page is not limit-truncated. + + When: + get_events runs. + + Then: + The stale boundary advances exactly one second past the drained second and the + now-abandoned seen ids are cleared, breaking the stall. + + Reference: + AWSGuardDutyEventCollector.get_events — the time-gated fully-deduped + stale-boundary advance (STALE_BOUNDARY_MARGIN). + """ + boundary_second = datetime.utcnow() - (STALE_BOUNDARY_MARGIN + timedelta(minutes=1)) + cursor_str = boundary_second.strftime("%Y-%m-%dT%H:%M:%S.%f") + mocked_client = _fully_deduped_boundary_client(mocker, cursor_str) + + events, new_last_ids, new_collect_from = get_events( + aws_client=mocked_client, + collect_from={"det1": cursor_str}, + collect_from_default=datetime(2026, 8, 1, 0, 0, 0), + last_ids={"det1": ["finding_seen"]}, + severity="Low", + limit=10, + ) + + expected_next = (boundary_second.replace(microsecond=0) + timedelta(seconds=1)).isoformat() + assert events == [], "The already-seen boundary finding must never be re-ingested." + assert new_collect_from == {"det1": expected_next}, "A stale boundary must advance one second past the drained second." + assert new_last_ids == {"det1": []}, "Stale-boundary advance must clear the now-abandoned seen ids." + + +def test_current_boundary_within_margin_stays_pinned(mocker): + """ + Given: + A fully-deduped boundary whose second is one minute inside + STALE_BOUNDARY_MARGIN. The only finding returned is already in last_ids, + nothing is newer, and the page is not limit-truncated. + + When: + get_events runs. + + Then: + The cursor is left pinned (a genuine later same-second update could still + arrive) and the seen ids are preserved. + + Reference: + AWSGuardDutyEventCollector.get_events — a recent boundary stays pinned. + """ + boundary_second = datetime.utcnow() - (STALE_BOUNDARY_MARGIN - timedelta(minutes=1)) + cursor_str = boundary_second.strftime("%Y-%m-%dT%H:%M:%S.%f") + mocked_client = _fully_deduped_boundary_client(mocker, cursor_str) + + events, new_last_ids, new_collect_from = get_events( + aws_client=mocked_client, + collect_from={"det1": cursor_str}, + collect_from_default=datetime(2026, 8, 1, 0, 0, 0), + last_ids={"det1": ["finding_seen"]}, + severity="Low", + limit=10, + ) + + assert events == [], "The already-seen boundary finding must never be re-ingested." + assert new_collect_from == {"det1": cursor_str}, "A current boundary must stay pinned." + assert new_last_ids == {"det1": ["finding_seen"]}, "A pinned boundary must preserve its seen ids." diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/README.md b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/README.md index c3d46d8b10dc..8718e532b6cb 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/README.md +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/README.md @@ -1,7 +1,5 @@ Amazon Web Services Guard Duty Service Event Collector integration for Cortex XSIAM. -This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM. - ## Configure AWS - GuardDuty Event Collector in Cortex | **Parameter** | **Description** | **Required** | diff --git a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/test_data/finding_for_test.py b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/test_data/finding_for_test.py index 20b7be90d487..4d15dbeec93e 100644 --- a/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/test_data/finding_for_test.py +++ b/Packs/AWS-GuardDuty/Integrations/AWSGuardDutyEventCollector/test_data/finding_for_test.py @@ -581,7 +581,7 @@ "Severity": 123.0, "Title": "string", "Type": "string", - "UpdatedAt": "string", + "UpdatedAt": "2015-01-01T00:00:00.000000", } MOST_GENERAL_FINDING_STR = { @@ -911,5 +911,5 @@ "Severity": 123.0, "Title": "string", "Type": "string", - "UpdatedAt": "string", + "UpdatedAt": "2015-01-01T00:00:00.000000", } diff --git a/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_1_3/AWSGuardDutyModelingRules_1_3.yml b/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_1_3/AWSGuardDutyModelingRules_1_3.yml index 7e10096a0fc7..b718c4071813 100644 --- a/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_1_3/AWSGuardDutyModelingRules_1_3.yml +++ b/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_1_3/AWSGuardDutyModelingRules_1_3.yml @@ -1,4 +1,5 @@ fromversion: 6.10.0 +toversion: 8.14.0 id: aws_guard_duty_modeling_rule name: AWSGuardDuty Modeling Rule rules: '' diff --git a/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11.xif b/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11.xif new file mode 100644 index 000000000000..0fa070526c9a --- /dev/null +++ b/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11.xif @@ -0,0 +1,237 @@ +[MODEL: dataset="aws_guardduty_raw"] +alter // init aliases for root level & common objects for supporting both camelCase and PascalCase field naming convention extractions + finding_resource = coalesce(Resource, resource), + finding_service = coalesce(Service, service) +| alter + finding_process = coalesce(finding_service -> RuntimeDetails.Process{}, service -> runtimeDetails.process{}), + finding_network = coalesce(finding_service -> Action.NetworkConnectionAction{}, service -> action.networkConnectionAction{}), + finding_aws_api = coalesce(finding_service -> Action.AwsApiCallAction{}, service -> action.awsApiCallAction{}), + finding_k8s_api = coalesce(finding_service -> Action.KubernetesApiCallAction{}, service -> action.kubernetesApiCallAction{}), + finding_rds_login = coalesce(finding_service -> Action.RdsLoginAttemptAction{}, service -> action.rdsLoginAttemptAction{}) + +| alter // extract common finding fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_Finding.html + finding_account_id = coalesce(AccountId, accountId), // The ID of the account in which the finding was generated. + finding_arn = coalesce(Arn, arn), // The ARN of the finding. + finding_description = coalesce(Description, description), // The description of the finding. + finding_id = coalesce(Id, id), // The ID of the finding. + finding_region = coalesce(Region, region), // The Region where the finding was generated. + finding_severity = to_float(coalesce(Severity, severity)), // The severity of the finding: https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings.html#guardduty_findings-severity + finding_title = coalesce(Title, title), // The title of the finding. + finding_type = coalesce(Type, type), // The type of finding. + + // extract resource fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_Resource.html + resource_type = coalesce(finding_resource -> ResourceType, resource -> resourceType), // The type of AWS resource. + resource_instance_id = coalesce(finding_resource -> InstanceDetails.InstanceId, resource -> instanceDetails.instanceId), // The ID of the EC2 instance. + resource_instance_type = coalesce(finding_resource -> InstanceDetails.InstanceType, resource -> instanceDetails.instanceType), // The type of the EC2 instance. + resource_username = coalesce( + finding_resource -> RdsDbUserDetails.User, resource -> rdsDbUserDetails.user, // The user name used in the anomalous login attempt. + finding_resource -> AccessKeyDetails.UserName, resource -> accessKeyDetails.userName, // The name of the user. + finding_resource -> KubernetesDetails.KubernetesUserDetails.Username, resource -> kubernetesDetails.kubernetesUserDetails.username), // The username of the user who called the Kubernetes API. + resource_user_id = coalesce( + finding_resource -> AccessKeyDetails.PrincipalId, resource -> accessKeyDetails.principalId, // The principal ID of the user. + finding_resource -> KubernetesDetails.KubernetesUserDetails.Uid, resource -> kubernetesDetails.kubernetesUserDetails.uid), // The user ID of the user who called the Kubernetes API. + resource_user_type = coalesce(finding_resource -> AccessKeyDetails.UserType, resource -> accessKeyDetails.userType), // The type of the user. + resource_user_groups = coalesce(finding_resource -> KubernetesDetails.KubernetesUserDetails.Groups[], resource -> kubernetesDetails.kubernetesUserDetails.groups[]), // The groups that include the user who called the Kubernetes API. + resource_rdsDbUserDetails_application = coalesce(finding_resource -> RdsDbUserDetails.Application, resource -> rdsDbUserDetails.application), // The application name used in the anomalous login attempt. + resource_rdsDbUserDetails_auth_method = coalesce(finding_resource -> RdsDbUserDetails.AuthMethod, resource -> rdsDbUserDetails.authMethod), // The authentication method used by the user involved in the finding. + resource_availability_zone = coalesce( // The Availability Zone of the EC2 instance. + finding_resource -> Instance.InstanceDetails.AvailabilityZone, resource -> instance.instanceDetails.availabilityZone, + finding_resource -> EcsClusterDetails.InstanceDetails.AvailabilityZone, resource -> ecsClusterDetails.instanceDetails.availabilityZone, + finding_resource -> EksClusterDetails.InstanceDetails.AvailabilityZone, resource -> eksClusterDetails.instanceDetails.availabilityZone), + resource_name = coalesce( + finding_resource -> LambdaDetails.FunctionName, resource -> lambdaDetails.functionName, // Lambda + finding_resource -> ContainerDetails.Name, resource -> containerDetails.name, // Container + finding_resource -> RdsDbUserDetails.Database, resource -> rdsDbUserDetails.database, // RDSDBInstance + finding_resource -> EcsClusterDetails.InstanceDetails.EcsClusterDetails.Name, resource -> ecsClusterDetails.instanceDetails.ecsClusterDetails.name, // ECSCluster + finding_resource -> EksClusterDetails.InstanceDetails.EksClusterDetails.Name, resource -> eksClusterDetails.instanceDetails.eksClusterDetails.name, // EKSCluster + finding_resource -> InstanceDetails.InstanceId, resource -> instanceDetails.instanceId, // Instance + finding_resource -> AccessKeyDetails.UserName, resource -> accessKeyDetails.userName, // AccessKey + arraystring(arraymap(finding_resource -> S3BucketDetails[], "@element" -> Name), ","), arraystring(arraymap(resource -> s3BucketDetails[], "@element" -> name), ",")), // S3Object + + // extract common service fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_Service.html + service_action_type = coalesce(finding_service -> Action.ActionType, service -> action.actionType), // The GuardDuty finding activity type. + service_detector_id = coalesce(finding_service -> DetectorId, service -> detectorId), // The detector ID for the GuardDuty service. + service_evidence_threat_intelligence_details = coalesce(finding_service -> Evidence.ThreatIntelligenceDetails[], service -> evidence.threatIntelligenceDetails[]), // A list of threat intelligence details related to the evidence. + + // extract service runtime fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_ProcessDetails.html + service_runtime_details_process_pid = to_integer(coalesce(finding_process -> Pid, finding_process -> pid)), // The ID of the process. + service_runtime_details_process_uuid = coalesce(finding_process -> Uuid, finding_process -> uuid), // The unique ID assigned to the process by GuardDuty. + service_runtime_details_process_parent_uuid = coalesce(finding_process -> ParentUuid, finding_process -> parentUuid), // The unique ID of the parent process. This ID is assigned to the parent process by GuardDuty. + service_runtime_details_process_name = coalesce(finding_process -> Name, finding_process -> name), // The name of the process. + service_runtime_details_process_executable_path = coalesce(finding_process -> ExecutablePath, finding_process -> executablePath), // The absolute path of the process executable file. + service_runtime_details_process_executable_sha256 = coalesce(finding_process -> ExecutableSha256, finding_process -> executableSha256), // The SHA256 hash of the process executable. + service_runtime_details_process_user = coalesce(finding_process -> User, finding_process -> user), // The user that executed the process. + service_runtime_details_process_user_id = coalesce(finding_process -> UserId, finding_process -> userId), // The unique ID of the user that executed the process. + + // extract service network connection action fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_NetworkConnectionAction.html + service_action_network_connection_is_blocked = to_boolean(coalesce(finding_network -> Blocked, finding_network -> blocked)), // Indicates whether EC2 blocked the network connection to your instance. + service_action_network_connection_direction = coalesce(finding_network -> ConnectionDirection, finding_network -> connectionDirection), // The network connection direction. + service_action_network_connection_protocol = coalesce(finding_network -> Protocol, finding_network -> protocol), // The network connection protocol. + service_action_network_connection_local_ipv4 = coalesce(finding_network -> LocalIpDetails.IpAddressV4, finding_network -> localIpDetails.ipAddressV4), // The IPv4 local address of the connection. + service_action_network_connection_local_ipv6 = coalesce(finding_network -> LocalIpDetails.IpAddressV6, finding_network -> localIpDetails.ipAddressV6), // The IPv6 local address of the connection. + service_action_network_connection_local_port = coalesce(finding_network -> LocalPortDetails.Port, finding_network -> localPortDetails.port), // The port number of the local connection. + service_action_network_connection_local_port_name = coalesce(finding_network -> LocalPortDetails.PortName, finding_network -> localPortDetails.portName), // The port name of the local connection. + service_action_network_connection_remote_asn = coalesce(finding_network -> RemoteIpDetails.Organization.Asn, finding_network -> remoteIpDetails.organization.asn), //(ASN) of the internet provider of the remote IP address. + service_action_network_connection_remote_asn_org = coalesce(finding_network -> RemoteIpDetails.Organization.AsnOrg, finding_network -> remoteIpDetails.organization.asnOrg), // The organization that registered this ASN. + service_action_network_connection_remote_asn_isp = coalesce(finding_network -> RemoteIpDetails.Organization.Isp, finding_network -> remoteIpDetails.organization.isp), // The ISP information for the internet provider. + service_action_network_connection_remote_asn_isp_org = coalesce(finding_network -> RemoteIpDetails.Organization.Org, finding_network -> remoteIpDetails.organization.org), // The name of the internet provider. + service_action_network_connection_remote_ipv4 = coalesce(finding_network -> RemoteIpDetails.IpAddressV4, finding_network -> remoteIpDetails.ipAddressV4), // The IPv4 remote address of the connection. + service_action_network_connection_remote_ipv6 = coalesce(finding_network -> RemoteIpDetails.IpAddressV6, finding_network -> remoteIpDetails.ipAddressV6), // The IPv6 remote address of the connection. + service_action_network_connection_remote_port = coalesce(finding_network -> RemotePortDetails.Port, finding_network -> remotePortDetails.port), // The port number of the remote connection. + service_action_network_connection_remote_port_name = coalesce(finding_network -> RemotePortDetails.PortName, finding_network -> remotePortDetails.portName), // The port name of the remote connection. + + // extract service aws api call action (AWS_API_CALL) fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_AwsApiCallAction.html + service_action_api_call_user_agent = coalesce( // The agent through which the API request was made. + finding_aws_api -> UserAgent, finding_aws_api -> userAgent, + to_json_string(finding_service -> AdditionalInfo.Value) -> userAgent.fullUserAgent, + to_json_string(finding_service -> additionalInfo.value) -> userAgent.fullUserAgent), + service_action_api_call_remote_ipv4 = coalesce(finding_aws_api -> RemoteIpDetails.IpAddressV4, finding_aws_api -> remoteIpDetails.ipAddressV4), // The IPv4 remote address of the connection. + service_action_api_call_remote_ipv6 = coalesce(finding_aws_api -> RemoteIpDetails.IpAddressV6, finding_aws_api -> remoteIpDetails.ipAddressV6), // The IPv6 remote address of the connection. + service_action_api_call_remote_asn = coalesce(finding_aws_api -> RemoteIpDetails.Organization.Asn, finding_aws_api -> remoteIpDetails.organization.asn), // The Autonomous System Number (ASN) of the internet provider of the remote IP address. + service_action_api_call_remote_asn_org = coalesce(finding_aws_api -> RemoteIpDetails.Organization.AsnOrg, finding_aws_api -> remoteIpDetails.organization.asnOrg), // The organization that registered this ASN. + service_action_api_call_remote_isp = coalesce(finding_aws_api -> RemoteIpDetails.Organization.Isp, finding_aws_api -> remoteIpDetails.organization.isp), // The ISP information for the internet provider. + service_action_api_call_remote_isp_org = coalesce(finding_aws_api -> RemoteIpDetails.Organization.Org, finding_aws_api -> remoteIpDetails.organization.org), // The name of the internet provider. + service_action_api_call_remote_account_id = coalesce(finding_aws_api -> RemoteAccountDetails.AccountId, finding_aws_api -> remoteAccountDetails.accountId), // The AWS account ID of the remote API caller. + service_action_api_call_error_code = coalesce(finding_aws_api -> ErrorCode, finding_aws_api -> errorCode), // The error code of the failed AWS API action. + service_action_api_call_api = coalesce(finding_aws_api -> api, finding_aws_api -> Api), + + // extract service dns request action (DNS_REQUEST) fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_DnsRequestAction.html + service_action_dns_request_action_domain = coalesce(finding_service -> Action.DnsRequestAction.Domain, service -> action.dnsRequestAction.domain), // The domain information for the DNS query. + service_action_dns_request_action_protocol = coalesce(finding_service -> Action.DnsRequestAction.Protocol, service -> action.dnsRequestAction.protocol), // The network connection protocol observed in the activity that prompted GuardDuty to generate the finding. + service_action_dns_request_action_blocked = to_boolean(coalesce(finding_service -> Action.DnsRequestAction.Blocked, service -> action.dnsRequestAction.blocked)), // Indicates whether the targeted port is blocked. + + // extract service k8s api call action (KUBERNETES_API_CALL) fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_KubernetesApiCallAction.html + service_action_k8s_api_call_verb = coalesce(finding_k8s_api -> Verb, finding_k8s_api -> verb), // The Kubernetes API request HTTP verb. + service_action_k8s_api_call_request_uri = coalesce(finding_k8s_api -> RequestUri, finding_k8s_api -> requestUri), // The Kubernetes API request URI. + service_action_k8s_api_call_status_code = coalesce(finding_k8s_api -> StatusCode, finding_k8s_api -> statusCode), // The resulting HTTP response code of the Kubernetes API call action. + service_action_k8s_api_call_user_agent = coalesce(finding_k8s_api -> UserAgent, finding_k8s_api -> userAgent), // The user agent of the caller of the Kubernetes API. + service_action_k8s_api_call_resource = coalesce(finding_k8s_api -> Resource, finding_k8s_api -> resource), // The resource component in the Kubernetes API call action. + service_action_k8s_api_call_resource_name = coalesce(finding_k8s_api -> ResourceName, finding_k8s_api -> resourceName), // The name of the resource in the Kubernetes API call action. + service_action_k8s_api_call_sourceIPs = coalesce(finding_k8s_api -> SourceIPs[], finding_k8s_api -> sourceIPs[]), // The IP of the Kubernetes API caller and the IPs of any proxies or load balancers between the caller and the API endpoint. + service_action_k8s_api_call_remote_ipv4 = coalesce(finding_k8s_api -> RemoteIpDetails.IpAddressV4, finding_k8s_api -> remoteIpDetails.ipAddressV4), //The IPv4 remote address of the connection. + service_action_k8s_api_call_remote_ipv6 = coalesce(finding_k8s_api -> RemoteIpDetails.IpAddressV6, finding_k8s_api -> remoteIpDetails.ipAddressV6), //The IPv6 remote address of the connection. + service_action_k8s_api_call_remote_asn = coalesce(finding_k8s_api -> RemoteIpDetails.Organization.Asn, finding_k8s_api -> remoteIpDetails.organization.asn), // The Autonomous System Number (ASN) of the internet provider of the remote IP address. + service_action_k8s_api_call_remote_asn_org = coalesce(finding_k8s_api -> RemoteIpDetails.Organization.AsnOrg, finding_k8s_api -> remoteIpDetails.organization.asnOrg), // The organization that registered this ASN. + service_action_k8s_api_call_remote_isp = coalesce(finding_k8s_api -> RemoteIpDetails.Organization.Isp, finding_k8s_api -> remoteIpDetails.organization.isp), // The ISP information for the internet provider. + service_action_k8s_api_call_remote_isp_org = coalesce(finding_k8s_api -> RemoteIpDetails.Organization.Org, finding_k8s_api -> remoteIpDetails.organization.org), // The name of the internet provider. + + // extract service port probe action (PORT_PROBE) fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_PortProbeAction.html + service_action_port_probing_remote_ipv4_addresses = coalesce( // The IPv4 remote address of the connection. + arraymap(finding_service -> Action.PortProbeAction.PortProbeDetails[], "@element" -> RemoteIpDetails.IpAddressV4), + arraymap(service -> action.portProbeAction.portProbeDetails[], "@element" -> remoteIpDetails.ipAddressV4)), + service_action_port_probing_remote_ipv6_addresses = coalesce( // The IPv6 remote address of the connection. + arraymap(finding_service -> Action.PortProbeAction.PortProbeDetails[], "@element" -> RemoteIpDetails.IpAddressV6), + arraymap(service -> action.portProbeAction.portProbeDetails[], "@element" -> remoteIpDetails.ipAddressV6)), + + // extract service k8s role fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_KubernetesRoleDetails.html + service_action_k8s_role_name = coalesce(finding_service -> Action.KubernetesRoleDetails.Name, service -> action.kubernetesRoleDetails.name), // The name of the Kubernetes role. + + // extract service k8s role binding fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_KubernetesRoleBindingDetails.html + service_action_k8s_role_binding_name = coalesce(finding_service -> Action.KubernetesRoleBindingDetails.Name, service -> action.kubernetesRoleBindingDetails.name), // The name of the RoleBinding. + + // extract service rds login attempt action fields: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_RdsLoginAttemptAction.html + service_action_rds_login_attempt_users = coalesce( // Indicates the user name which attempted to log in. + arraystring(arraymap(finding_rds_login -> LoginAttribute[], "@element" -> User), ","), + arraystring(arraymap(finding_rds_login -> loginAttribute[], "@element" -> user), ",")), + service_action_rds_login_attempt_applications = coalesce( // Indicates the application name used to attempt log in. + arraystring(arraymap(finding_rds_login -> LoginAttribute[], "@element" -> Application), ","), + arraystring(arraymap(finding_rds_login -> loginAttribute[], "@element" -> application), ",")), + service_action_rds_login_attempt_remote_ipv4 = coalesce(finding_rds_login -> RemoteIpDetails.IpAddressV4, finding_rds_login -> remoteIpDetails.ipAddressV4), // The IPv4 remote address of the connection. + service_action_rds_login_attempt_remote_ipv6 = coalesce(finding_rds_login -> RemoteIpDetails.IpAddressV6, finding_rds_login -> remoteIpDetails.ipAddressV6), // The IPv6 remote address of the connection. + service_action_rds_login_attempt_remote_asn = coalesce(finding_rds_login -> RemoteIpDetails.Organization.Asn, finding_rds_login -> remoteIpDetails.organization.asn), // The Autonomous System Number (ASN) of the internet provider of the remote IP address. + service_action_rds_login_attempt_remote_asn_org = coalesce(finding_rds_login -> RemoteIpDetails.Organization.AsnOrg, finding_rds_login -> remoteIpDetails.organization.asnOrg), // The organization that registered this ASN. + service_action_rds_login_attempt_remote_isp = coalesce(finding_rds_login -> RemoteIpDetails.Organization.Isp, finding_rds_login -> remoteIpDetails.organization.isp), // The ISP information for the internet provider. + service_action_rds_login_attempt_remote_isp_org = coalesce(finding_rds_login -> RemoteIpDetails.Organization.Org, finding_rds_login -> remoteIpDetails.organization.org) // The name of the internet provider. + +| alter // post extraction processing + is_connection_inbound = if(service_action_network_connection_direction != null and service_action_network_connection_direction = "INBOUND"), + is_connection_outbound = if(service_action_network_connection_direction != null and service_action_network_connection_direction = "OUTBOUND"), + http_code = to_integer(service_action_k8s_api_call_status_code), + http_verb = service_action_k8s_api_call_verb, + network_protocol = coalesce(service_action_network_connection_protocol, service_action_dns_request_action_protocol), + k8s_api_call_src_ipv4_addresses = arrayfilter(service_action_k8s_api_call_sourceIPs, "@element" ~= "(?:\d{1,3}\.){3}\d{1,3}"), + k8s_api_call_src_ipv6_addresses = arrayfilter(service_action_k8s_api_call_sourceIPs, "@element" ~= "(?:[a-fA-F\d]{0,4}\:){1,7}[a-fA-F\d]{0,4}") +| alter + network_application_protocol = if(is_connection_inbound, service_action_network_connection_local_port_name, is_connection_outbound, service_action_network_connection_remote_port_name), + source_ipv4 = if(is_connection_inbound, service_action_network_connection_remote_ipv4, is_connection_outbound, service_action_network_connection_local_ipv4, coalesce(service_action_api_call_remote_ipv4, service_action_k8s_api_call_remote_ipv4, service_action_rds_login_attempt_remote_ipv4)), + source_ipv6 = if(is_connection_inbound, service_action_network_connection_remote_ipv6, is_connection_outbound, service_action_network_connection_local_ipv6, coalesce(service_action_api_call_remote_ipv6, service_action_k8s_api_call_remote_ipv6, service_action_rds_login_attempt_remote_ipv6)), + target_ipv4 = if(is_connection_inbound, service_action_network_connection_local_ipv4, is_connection_outbound, service_action_network_connection_remote_ipv4), + target_ipv6 = if(is_connection_inbound, service_action_network_connection_local_ipv6, is_connection_outbound, service_action_network_connection_remote_ipv6) +| alter + source_ipv4_addresses = if(array_length(k8s_api_call_src_ipv4_addresses) > 0, k8s_api_call_src_ipv4_addresses, array_length(service_action_port_probing_remote_ipv4_addresses) > 0, service_action_port_probing_remote_ipv4_addresses), + source_ipv6_addresses = if(array_length(k8s_api_call_src_ipv6_addresses) > 0, k8s_api_call_src_ipv6_addresses, array_length(service_action_port_probing_remote_ipv6_addresses) > 0, service_action_port_probing_remote_ipv6_addresses) +| alter + all_source_ipv4_addresses = if(array_length(source_ipv4_addresses) > 0 and source_ipv4 != null, arrayconcat(arraycreate(source_ipv4), source_ipv4_addresses), source_ipv4 != null, arraycreate(source_ipv4), source_ipv4_addresses), + all_source_ipv6_addresses = if(array_length(source_ipv6_addresses) > 0 and source_ipv6 != null, arrayconcat(arraycreate(source_ipv6), source_ipv6_addresses), source_ipv6 != null, arraycreate(source_ipv6), source_ipv6_addresses) + +| alter // XDM mappings + xdm.alert.description = finding_description, + xdm.alert.name = finding_title, + xdm.alert.original_alert_id = finding_id, + xdm.alert.risks = if(array_length(service_evidence_threat_intelligence_details) > 0, service_evidence_threat_intelligence_details), + xdm.alert.severity = if( // https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings.html#guardduty_findings-severity + finding_severity >= 9, "Critical", + finding_severity >= 7 and finding_severity <= 8.9 , "High", + finding_severity >= 4 and finding_severity <= 6.9 , "Medium", + finding_severity >= 1 and finding_severity <= 3.9 , "Low", + to_string(finding_severity)), + xdm.alert.subcategory = finding_type, + xdm.auth.auth_method = resource_rdsDbUserDetails_auth_method, + xdm.event.operation = service_action_type, + xdm.event.operation_sub_type = service_action_api_call_api, + xdm.event.outcome = if(service_action_network_connection_is_blocked or service_action_dns_request_action_blocked or service_action_api_call_error_code != null, XDM_CONST.OUTCOME_FAILED), + xdm.event.outcome_reason = service_action_api_call_error_code, + xdm.network.application_protocol = if(network_application_protocol != "Unknown", network_application_protocol), + xdm.network.dns.dns_question.name = service_action_dns_request_action_domain, + xdm.network.http.method = if(http_verb = null, http_verb, http_verb = "GET", XDM_CONST.HTTP_METHOD_GET, http_verb = "POST", XDM_CONST.HTTP_METHOD_POST, http_verb = "PUT", XDM_CONST.HTTP_METHOD_PUT, http_verb = "PATCH", XDM_CONST.HTTP_METHOD_PATCH, http_verb = "OPTIONS", XDM_CONST.HTTP_METHOD_OPTIONS, http_verb = "HEAD", XDM_CONST.HTTP_METHOD_HEAD, http_verb = "ACL", XDM_CONST.HTTP_METHOD_ACL, http_verb = "BASELINE_CONTROL", XDM_CONST.HTTP_METHOD_BASELINE_CONTROL, http_verb = "BIND", XDM_CONST.HTTP_METHOD_BIND, http_verb = "CHECKIN", XDM_CONST.HTTP_METHOD_CHECKIN, http_verb = "CHECKOUT", XDM_CONST.HTTP_METHOD_CHECKOUT, http_verb = "CONNECT", XDM_CONST.HTTP_METHOD_CONNECT, http_verb = "COPY", XDM_CONST.HTTP_METHOD_COPY, http_verb = "DELETE", XDM_CONST.HTTP_METHOD_DELETE, http_verb = "LABEL", XDM_CONST.HTTP_METHOD_LABEL, http_verb = "LINK", XDM_CONST.HTTP_METHOD_LINK, http_verb = "LOCK", XDM_CONST.HTTP_METHOD_LOCK, http_verb = "MERGE", XDM_CONST.HTTP_METHOD_MERGE, http_verb = "MKACTIVITY", XDM_CONST.HTTP_METHOD_MKACTIVITY, http_verb = "MKCALENDAR", XDM_CONST.HTTP_METHOD_MKCALENDAR, http_verb = "MKCOL", XDM_CONST.HTTP_METHOD_MKCOL, http_verb = "MKREDIRECTREF", XDM_CONST.HTTP_METHOD_MKREDIRECTREF, http_verb = "MKWORKSPACE", XDM_CONST.HTTP_METHOD_MKWORKSPACE, http_verb = "MOVE", XDM_CONST.HTTP_METHOD_MOVE, http_verb = "ORDERPATCH", XDM_CONST.HTTP_METHOD_ORDERPATCH, http_verb = "PRI", XDM_CONST.HTTP_METHOD_PRI, http_verb = "PROPFIND", XDM_CONST.HTTP_METHOD_PROPFIND, http_verb = "PROPPATCH", XDM_CONST.HTTP_METHOD_PROPPATCH, http_verb = "REBIND", XDM_CONST.HTTP_METHOD_REBIND, http_verb = "REPORT", XDM_CONST.HTTP_METHOD_REPORT, http_verb = "SEARCH", XDM_CONST.HTTP_METHOD_SEARCH, http_verb = "TRACE", XDM_CONST.HTTP_METHOD_TRACE, http_verb = "UNBIND", XDM_CONST.HTTP_METHOD_UNBIND, http_verb = "UNCHECKOUT", XDM_CONST.HTTP_METHOD_UNCHECKOUT, http_verb = "UNLINK", XDM_CONST.HTTP_METHOD_UNLINK, http_verb = "UNLOCK", XDM_CONST.HTTP_METHOD_UNLOCK, http_verb = "UPDATE", XDM_CONST.HTTP_METHOD_UPDATE, http_verb = "UPDATEREDIRECTREF", XDM_CONST.HTTP_METHOD_UPDATEREDIRECTREF, http_verb = "VERSION_CONTROL", XDM_CONST.HTTP_METHOD_VERSION_CONTROL, uppercase(http_verb)), + xdm.network.http.response_code = if(http_code = null, null, http_code = 200, XDM_CONST.HTTP_RSP_CODE_OK, http_code = 201, XDM_CONST.HTTP_RSP_CODE_CREATED, http_code = 302, XDM_CONST.HTTP_RSP_CODE_FOUND, http_code = 401, XDM_CONST.HTTP_RSP_CODE_UNAUTHORIZED, http_code = 403, XDM_CONST.HTTP_RSP_CODE_FORBIDDEN, http_code = 404, XDM_CONST.HTTP_RSP_CODE_NOT_FOUND, http_code = 500, XDM_CONST.HTTP_RSP_CODE_INTERNAL_SERVER_ERROR, http_code = 501, XDM_CONST.HTTP_RSP_CODE_NOT_IMPLEMENTED, http_code = 502, XDM_CONST.HTTP_RSP_CODE_BAD_GATEWAY, http_code = 503, XDM_CONST.HTTP_RSP_CODE_SERVICE_UNAVAILABLE, http_code = 504, XDM_CONST.HTTP_RSP_CODE_GATEWAY_TIMEOUT, http_code = 505, XDM_CONST.HTTP_RSP_CODE_HTTP_VERSION_NOT_SUPPORTED, http_code = 506, XDM_CONST.HTTP_RSP_CODE_VARIANT_ALSO_NEGOTIATES, http_code = 507, XDM_CONST.HTTP_RSP_CODE_INSUFFICIENT_STORAGE, http_code = 508, XDM_CONST.HTTP_RSP_CODE_LOOP_DETECTED, http_code = 511, XDM_CONST.HTTP_RSP_CODE_NETWORK_AUTHENTICATION_REQUIRED, http_code = 100, XDM_CONST.HTTP_RSP_CODE_CONTINUE, http_code = 101, XDM_CONST.HTTP_RSP_CODE_SWITCHING_PROTOCOLS, http_code = 102, XDM_CONST.HTTP_RSP_CODE_PROCESSING, http_code = 103, XDM_CONST.HTTP_RSP_CODE_EARLY_HINTS, http_code = 202, XDM_CONST.HTTP_RSP_CODE_ACCEPTED, http_code = 203, XDM_CONST.HTTP_RSP_CODE_NON__AUTHORITATIVE_INFORMATION, http_code = 204, XDM_CONST.HTTP_RSP_CODE_NO_CONTENT, http_code = 205, XDM_CONST.HTTP_RSP_CODE_RESET_CONTENT, http_code = 206, XDM_CONST.HTTP_RSP_CODE_PARTIAL_CONTENT, http_code = 207, XDM_CONST.HTTP_RSP_CODE_MULTI__STATUS, http_code = 208, XDM_CONST.HTTP_RSP_CODE_ALREADY_REPORTED, http_code = 226, XDM_CONST.HTTP_RSP_CODE_IM_USED, http_code = 300, XDM_CONST.HTTP_RSP_CODE_MULTIPLE_CHOICES, http_code = 301, XDM_CONST.HTTP_RSP_CODE_MOVED_PERMANENTLY, http_code = 303, XDM_CONST.HTTP_RSP_CODE_SEE_OTHER, http_code = 304, XDM_CONST.HTTP_RSP_CODE_NOT_MODIFIED, http_code = 305, XDM_CONST.HTTP_RSP_CODE_USE_PROXY, http_code = 307, XDM_CONST.HTTP_RSP_CODE_TEMPORARY_REDIRECT, http_code = 308, XDM_CONST.HTTP_RSP_CODE_PERMANENT_REDIRECT, http_code = 400, XDM_CONST.HTTP_RSP_CODE_BAD_REQUEST, http_code = 402, XDM_CONST.HTTP_RSP_CODE_PAYMENT_REQUIRED, http_code = 405, XDM_CONST.HTTP_RSP_CODE_METHOD_NOT_ALLOWED, http_code = 406, XDM_CONST.HTTP_RSP_CODE_NOT_ACCEPTABLE, http_code = 407, XDM_CONST.HTTP_RSP_CODE_PROXY_AUTHENTICATION_REQUIRED, http_code = 408, XDM_CONST.HTTP_RSP_CODE_REQUEST_TIMEOUT, http_code = 409, XDM_CONST.HTTP_RSP_CODE_CONFLICT, http_code = 410, XDM_CONST.HTTP_RSP_CODE_GONE, http_code = 411, XDM_CONST.HTTP_RSP_CODE_LENGTH_REQUIRED, http_code = 412, XDM_CONST.HTTP_RSP_CODE_PRECONDITION_FAILED, http_code = 413, XDM_CONST.HTTP_RSP_CODE_CONTENT_TOO_LARGE, http_code = 414, XDM_CONST.HTTP_RSP_CODE_URI_TOO_LONG, http_code = 415, XDM_CONST.HTTP_RSP_CODE_UNSUPPORTED_MEDIA_TYPE, http_code = 416, XDM_CONST.HTTP_RSP_CODE_RANGE_NOT_SATISFIABLE, http_code = 417, XDM_CONST.HTTP_RSP_CODE_EXPECTATION_FAILED, http_code = 421, XDM_CONST.HTTP_RSP_CODE_MISDIRECTED_REQUEST, http_code = 422, XDM_CONST.HTTP_RSP_CODE_UNPROCESSABLE_CONTENT, http_code = 423, XDM_CONST.HTTP_RSP_CODE_LOCKED, http_code = 424, XDM_CONST.HTTP_RSP_CODE_FAILED_DEPENDENCY, http_code = 425, XDM_CONST.HTTP_RSP_CODE_TOO_EARLY, http_code = 426, XDM_CONST.HTTP_RSP_CODE_UPGRADE_REQUIRED, http_code = 428, XDM_CONST.HTTP_RSP_CODE_PRECONDITION_REQUIRED, http_code = 429, XDM_CONST.HTTP_RSP_CODE_TOO_MANY_REQUESTS, http_code = 431, XDM_CONST.HTTP_RSP_CODE_REQUEST_HEADER_FIELDS_TOO_LARGE, http_code = 451, XDM_CONST.HTTP_RSP_CODE_UNAVAILABLE_FOR_LEGAL_REASONS, service_action_k8s_api_call_status_code), + xdm.network.http.url = service_action_k8s_api_call_request_uri, + xdm.network.ip_protocol = if(network_protocol = "UDP", XDM_CONST.IP_PROTOCOL_UDP, network_protocol = "TCP", XDM_CONST.IP_PROTOCOL_TCP), + xdm.observer.unique_identifier = service_detector_id, + xdm.source.application.name = if(resource_rdsDbUserDetails_application != null or service_action_rds_login_attempt_applications != null, arraystring(arraycreate(resource_rdsDbUserDetails_application, service_action_rds_login_attempt_applications), ",")), + xdm.source.asn.as_name = if(is_connection_inbound, service_action_network_connection_remote_asn_org, coalesce(service_action_api_call_remote_asn_org, service_action_k8s_api_call_remote_asn_org, service_action_rds_login_attempt_remote_asn_org)), + xdm.source.asn.as_number = to_integer(if(is_connection_inbound, service_action_network_connection_remote_asn, coalesce(service_action_api_call_remote_asn, service_action_k8s_api_call_remote_asn, service_action_rds_login_attempt_remote_asn))), + xdm.source.asn.isp = if(is_connection_inbound, coalesce(service_action_network_connection_remote_asn_isp, service_action_network_connection_remote_asn_isp_org), coalesce(service_action_api_call_remote_isp, service_action_api_call_remote_isp_org, service_action_k8s_api_call_remote_isp, service_action_k8s_api_call_remote_isp_org, service_action_rds_login_attempt_remote_isp, service_action_rds_login_attempt_remote_isp_org)), + xdm.source.cloud.project_id = coalesce(service_action_api_call_remote_account_id, finding_account_id), + xdm.source.cloud.provider = XDM_CONST.CLOUD_PROVIDER_AWS, + xdm.source.cloud.region = finding_region, + xdm.source.cloud.zone = resource_availability_zone, + xdm.source.host.device_model = resource_instance_type, + xdm.source.host.ipv4_addresses = if(array_length(all_source_ipv4_addresses) > 0, all_source_ipv4_addresses), + xdm.source.host.ipv6_addresses = if(array_length(all_source_ipv6_addresses) > 0, all_source_ipv6_addresses), + xdm.source.host.ipv4_public_addresses = arrayfilter(all_source_ipv4_addresses, not incidr("@element", "10.0.0.0/8") and not incidr("@element", "172.16.0.0/12") and not incidr("@element", "192.168.0.0/16") and not incidr("@element", "127.0.0.0/8") and not incidr("@element", "169.254.0.0/16") and not incidr("@element", "100.64.0.0/10")), + xdm.source.host.hostname = resource_instance_id, + xdm.source.ipv4 = coalesce(source_ipv4, arrayindex(all_source_ipv4_addresses, 0)), + xdm.source.ipv6 = coalesce(source_ipv6, arrayindex(all_source_ipv6_addresses, 0)), + xdm.source.port = to_integer(if(is_connection_inbound, service_action_network_connection_remote_port, is_connection_outbound, service_action_network_connection_local_port)), + xdm.source.process.executable.path = service_runtime_details_process_executable_path, + xdm.source.process.executable.sha256 = service_runtime_details_process_executable_sha256, + xdm.source.process.identifier = service_runtime_details_process_uuid, + xdm.source.process.name = service_runtime_details_process_name, + xdm.source.process.parent_id = service_runtime_details_process_parent_uuid, + xdm.source.user_agent = coalesce(service_action_api_call_user_agent, service_action_k8s_api_call_user_agent), + xdm.source.user.groups = if(service_action_k8s_role_name != null, arraycreate(service_action_k8s_role_name), resource_user_groups), + xdm.source.identity.groups = if(service_action_k8s_role_name != null, arraycreate(service_action_k8s_role_name), resource_user_groups), + xdm.source.user.identifier = if(service_runtime_details_process_user_id != null or resource_user_id != null, arraystring(arraycreate(resource_user_id, service_runtime_details_process_user_id), ",")), + xdm.source.identity.identifier = if(service_runtime_details_process_user_id != null or resource_user_id != null, arraystring(arraycreate(resource_user_id, service_runtime_details_process_user_id), ",")), + xdm.source.user.user_type = resource_user_type, + xdm.source.identity.user_type = resource_user_type, + xdm.source.user.username = trim(coalesce(service_action_rds_login_attempt_users, service_runtime_details_process_user, resource_username), ","), + xdm.source.identity.username = trim(coalesce(service_action_rds_login_attempt_users, service_runtime_details_process_user, resource_username), ","), + xdm.target.asn.as_name = if(is_connection_outbound, service_action_api_call_remote_asn_org), + xdm.target.asn.as_number = to_integer(if(is_connection_outbound, service_action_api_call_remote_asn)), + xdm.target.asn.isp = if(is_connection_outbound, coalesce(service_action_api_call_remote_isp, service_action_api_call_remote_isp_org)), + xdm.target.cloud.project_id = finding_account_id, + xdm.target.cloud.provider = XDM_CONST.CLOUD_PROVIDER_AWS, + xdm.target.cloud.region = finding_region, + xdm.target.cloud.zone = resource_availability_zone, + xdm.target.host.ipv4_addresses = if(target_ipv4 != null, arraycreate(target_ipv4)), + xdm.target.host.ipv6_addresses = if(target_ipv6 != null, arraycreate(target_ipv6)), + xdm.target.host.ipv4_public_addresses = if(target_ipv4 != null and not incidr(target_ipv4, "10.0.0.0/8") and not incidr(target_ipv4, "172.16.0.0/12") and not incidr(target_ipv4, "192.168.0.0/16") and not incidr(target_ipv4, "127.0.0.0/8") and not incidr(target_ipv4, "169.254.0.0/16") and not incidr(target_ipv4, "100.64.0.0/10"), arraycreate(target_ipv4)), + xdm.target.ipv4 = target_ipv4, + xdm.target.ipv6 = target_ipv6, + xdm.target.port = to_integer(if(is_connection_inbound, service_action_network_connection_local_port, is_connection_outbound, service_action_network_connection_remote_port)), + xdm.target.resource.id = finding_arn, + xdm.target.resource.name = coalesce(resource_name, service_action_k8s_api_call_resource_name, service_action_k8s_role_binding_name), + xdm.target.resource.type = resource_type, + xdm.target.resource.value = service_action_k8s_api_call_resource; \ No newline at end of file diff --git a/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11.yml b/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11.yml new file mode 100644 index 000000000000..05404f1055a8 --- /dev/null +++ b/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: AWSGuardDuty_ModelingRule +name: AWSGuardDuty Modeling Rule +rules: '' +schema: '' +tags: AWSGuardDuty +supportedModules: +- xsiam diff --git a/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11_schema.json b/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11_schema.json new file mode 100644 index 000000000000..c4cc51b7a254 --- /dev/null +++ b/Packs/AWS-GuardDuty/ModelingRules/AWSGuardDutyModelingRules_2_11/AWSGuardDutyModelingRules_2_11_schema.json @@ -0,0 +1,44 @@ +{ + "aws_guardduty_raw": { + "accountId": { + "type": "string", + "is_array": false + }, + "arn": { + "type": "string", + "is_array": false + }, + "description": { + "type": "string", + "is_array": false + }, + "id": { + "type": "string", + "is_array": false + }, + "region": { + "type": "string", + "is_array": false + }, + "resource": { + "type": "string", + "is_array": false + }, + "service": { + "type": "string", + "is_array": false + }, + "severity": { + "type": "float", + "is_array": false + }, + "title": { + "type": "string", + "is_array": false + }, + "type": { + "type": "string", + "is_array": false + } + } +} \ No newline at end of file diff --git a/Packs/AWS-GuardDuty/ReleaseNotes/1_4_1.md b/Packs/AWS-GuardDuty/ReleaseNotes/1_4_1.md new file mode 100644 index 000000000000..3ca05f6a3620 --- /dev/null +++ b/Packs/AWS-GuardDuty/ReleaseNotes/1_4_1.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: AWSGuardDuty Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/AWS-GuardDuty/ReleaseNotes/1_4_2.md b/Packs/AWS-GuardDuty/ReleaseNotes/1_4_2.md new file mode 100644 index 000000000000..5dccc0780898 --- /dev/null +++ b/Packs/AWS-GuardDuty/ReleaseNotes/1_4_2.md @@ -0,0 +1,8 @@ + +#### Integrations + +##### AWS - GuardDuty Event Collector + +- Fixed an issue where findings sharing the same second could be re-fetched every cycle, causing duplicate events. +- Fixed an issue where fetching could stall on a detector when the only findings returned had already been collected, preventing newer findings from being fetched. +- Added debug logging to simplify troubleshooting. diff --git a/Packs/AWS-GuardDuty/ReleaseNotes/1_5_0.md b/Packs/AWS-GuardDuty/ReleaseNotes/1_5_0.md new file mode 100644 index 000000000000..45fcfc1ecd61 --- /dev/null +++ b/Packs/AWS-GuardDuty/ReleaseNotes/1_5_0.md @@ -0,0 +1,10 @@ + +#### Integrations + +##### AWS - GuardDuty + +- Added the ***aws-gd-create-threat-entity-set*** command to create a GuardDuty Threat Entity Set. +- Added the ***aws-gd-update-threat-entity-set*** command to update a GuardDuty Threat Entity Set. +- Added the ***aws-gd-get-threat-entity-set*** command to retrieve a GuardDuty Threat Entity Set. +- Added the ***aws-gd-list-threat-entity-sets*** command to list the GuardDuty Threat Entity Sets of a detector. +- Added the ***aws-gd-delete-threat-entity-set*** command to delete a GuardDuty Threat Entity Set. diff --git a/Packs/AWS-GuardDuty/ReleaseNotes/1_5_1.md b/Packs/AWS-GuardDuty/ReleaseNotes/1_5_1.md new file mode 100644 index 000000000000..c3a53d1699a4 --- /dev/null +++ b/Packs/AWS-GuardDuty/ReleaseNotes/1_5_1.md @@ -0,0 +1,11 @@ + +#### Integrations + +##### AWS - GuardDuty + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + +##### AWS - GuardDuty Event Collector + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-GuardDuty/ReleaseNotes/1_5_2.md b/Packs/AWS-GuardDuty/ReleaseNotes/1_5_2.md new file mode 100644 index 000000000000..e28ec8dd0b16 --- /dev/null +++ b/Packs/AWS-GuardDuty/ReleaseNotes/1_5_2.md @@ -0,0 +1,3 @@ +## AWS - GuardDuty + +- Documentation and metadata improvements. diff --git a/Packs/AWS-GuardDuty/pack_metadata.json b/Packs/AWS-GuardDuty/pack_metadata.json index 202384ded98a..85484300467d 100644 --- a/Packs/AWS-GuardDuty/pack_metadata.json +++ b/Packs/AWS-GuardDuty/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - GuardDuty", "description": "Amazon Web Services Guard Duty Service (gd)", "support": "xsoar", - "currentVersion": "1.4.0", + "currentVersion": "1.5.2", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -29,7 +29,6 @@ "marketplacev2", "platform" ], - "defaultDataSource": "AWS - GuardDuty Event Collector", "supportedModules": [ "agentix", "xsiam" diff --git a/Packs/AWS-IAM/Integrations/AWS-IAM/AWS-IAM.yml b/Packs/AWS-IAM/Integrations/AWS-IAM/AWS-IAM.yml index 6e4312bd710f..c5cf6226077d 100644 --- a/Packs/AWS-IAM/Integrations/AWS-IAM/AWS-IAM.yml +++ b/Packs/AWS-IAM/Integrations/AWS-IAM/AWS-IAM.yml @@ -1754,7 +1754,7 @@ script: required: false description: Deletes a virtual MFA device. name: aws-iam-delete-mfa-devices - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 runonce: false script: '-' subtype: python3 diff --git a/Packs/AWS-IAM/ReleaseNotes/1_1_88.md b/Packs/AWS-IAM/ReleaseNotes/1_1_88.md new file mode 100644 index 000000000000..ec78fceca5e5 --- /dev/null +++ b/Packs/AWS-IAM/ReleaseNotes/1_1_88.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - Identity and Access Management + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-IAM/ReleaseNotes/1_1_89.md b/Packs/AWS-IAM/ReleaseNotes/1_1_89.md new file mode 100644 index 000000000000..2579c4c26952 --- /dev/null +++ b/Packs/AWS-IAM/ReleaseNotes/1_1_89.md @@ -0,0 +1,3 @@ +## AWS - IAM + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AWS-IAM/pack_metadata.json b/Packs/AWS-IAM/pack_metadata.json index 36dfce871731..53f67e8564ef 100644 --- a/Packs/AWS-IAM/pack_metadata.json +++ b/Packs/AWS-IAM/pack_metadata.json @@ -3,7 +3,7 @@ "description": "Amazon Web Services Identity and Access Management (IAM)", "support": "xsoar", "author": "Cortex XSOAR", - "currentVersion": "1.1.87", + "currentVersion": "1.1.89", "url": "https://www.paloaltonetworks.com/cortex", "email": "", "created": "2020-04-14T00:00:00Z", diff --git a/Packs/AWS-IAMIdentityCenter/Integrations/AWSIAMIdentityCenter/AWSIAMIdentityCenter.yml b/Packs/AWS-IAMIdentityCenter/Integrations/AWSIAMIdentityCenter/AWSIAMIdentityCenter.yml index c58cd6b77ec8..837ab672e07c 100644 --- a/Packs/AWS-IAMIdentityCenter/Integrations/AWSIAMIdentityCenter/AWSIAMIdentityCenter.yml +++ b/Packs/AWS-IAMIdentityCenter/Integrations/AWSIAMIdentityCenter/AWSIAMIdentityCenter.yml @@ -1174,7 +1174,7 @@ script: required: true description: Updates an IAM Identity Center group for your AWS account. name: aws-iam-identitycenter-update-group - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 runonce: false script: '' subtype: python3 diff --git a/Packs/AWS-IAMIdentityCenter/ReleaseNotes/1_0_15.md b/Packs/AWS-IAMIdentityCenter/ReleaseNotes/1_0_15.md new file mode 100644 index 000000000000..fd16c8aa6bab --- /dev/null +++ b/Packs/AWS-IAMIdentityCenter/ReleaseNotes/1_0_15.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - IAM Identity Center + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-IAMIdentityCenter/pack_metadata.json b/Packs/AWS-IAMIdentityCenter/pack_metadata.json index b8b52cc03358..502d8c4bbae9 100644 --- a/Packs/AWS-IAMIdentityCenter/pack_metadata.json +++ b/Packs/AWS-IAMIdentityCenter/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - IAM Identity Center", "description": "AWS IAM Identity Center\n\nWith AWS IAM Identity Center (successor to AWS Single Sign-On), you can manage sign-in security for your workforce identities, also known as workforce users. IAM Identity Center provides one place where you can create or connect workforce users and manage their access centrally across all their AWS accounts and applications. IAM Identity Center is the recommended approach for workforce authentication and authorization in AWS, for organizations of any size and type.", "support": "xsoar", - "currentVersion": "1.0.14", + "currentVersion": "1.0.15", "author": "Cortex XSOAR", "url": "", "email": "", diff --git a/Packs/AWS-Organizations/Integrations/AWSOrganizations/AWSOrganizations.yml b/Packs/AWS-Organizations/Integrations/AWSOrganizations/AWSOrganizations.yml index 43b319ea4914..230c6ec3acc3 100644 --- a/Packs/AWS-Organizations/Integrations/AWSOrganizations/AWSOrganizations.yml +++ b/Packs/AWS-Organizations/Integrations/AWSOrganizations/AWSOrganizations.yml @@ -691,7 +691,7 @@ script: - contextPath: AWS.Organizations.TagNextToken description: If not null, indicates that more output is available than is included in the current response. Use this value as the next_token argument in a subsequent call of the command to get the next part of the output. type: String - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 feed: false longRunning: false longRunningPort: false diff --git a/Packs/AWS-Organizations/ReleaseNotes/1_0_18.md b/Packs/AWS-Organizations/ReleaseNotes/1_0_18.md new file mode 100644 index 000000000000..ac66ba50ff48 --- /dev/null +++ b/Packs/AWS-Organizations/ReleaseNotes/1_0_18.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - Organizations + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-Organizations/ReleaseNotes/1_0_19.md b/Packs/AWS-Organizations/ReleaseNotes/1_0_19.md new file mode 100644 index 000000000000..eaf8473b0ee6 --- /dev/null +++ b/Packs/AWS-Organizations/ReleaseNotes/1_0_19.md @@ -0,0 +1,3 @@ +## AWS Organizations + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AWS-Organizations/pack_metadata.json b/Packs/AWS-Organizations/pack_metadata.json index c4b90a4444c6..a58676e337bc 100644 --- a/Packs/AWS-Organizations/pack_metadata.json +++ b/Packs/AWS-Organizations/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS Organizations", "description": "AWS Organizations offers policy-based management for multiple AWS accounts.", "support": "xsoar", - "currentVersion": "1.0.17", + "currentVersion": "1.0.19", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-Route53/Integrations/AWSRoute53/AWSRoute53.yml b/Packs/AWS-Route53/Integrations/AWSRoute53/AWSRoute53.yml index 6864307179af..c9f2c9472c93 100644 --- a/Packs/AWS-Route53/Integrations/AWSRoute53/AWSRoute53.yml +++ b/Packs/AWS-Route53/Integrations/AWSRoute53/AWSRoute53.yml @@ -411,7 +411,7 @@ script: - contextPath: AWS.Route53.RecordSetsChange.Comment description: A complex type that describes change information about changes made to your hosted zone. type: string - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 runonce: false script: '' subtype: python3 diff --git a/Packs/AWS-Route53/ReleaseNotes/1_1_51.md b/Packs/AWS-Route53/ReleaseNotes/1_1_51.md new file mode 100644 index 000000000000..41f4eae5b77b --- /dev/null +++ b/Packs/AWS-Route53/ReleaseNotes/1_1_51.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - Route53 + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-Route53/ReleaseNotes/1_1_52.md b/Packs/AWS-Route53/ReleaseNotes/1_1_52.md new file mode 100644 index 000000000000..847d3668bdca --- /dev/null +++ b/Packs/AWS-Route53/ReleaseNotes/1_1_52.md @@ -0,0 +1,3 @@ +## AWS - Route53 + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AWS-Route53/pack_metadata.json b/Packs/AWS-Route53/pack_metadata.json index 689ff4eae089..9c3bbd387aac 100644 --- a/Packs/AWS-Route53/pack_metadata.json +++ b/Packs/AWS-Route53/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - Route53", "description": "Amazon Web Services Managed Cloud DNS Service.", "support": "xsoar", - "currentVersion": "1.1.50", + "currentVersion": "1.1.52", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-S3/Integrations/AWS-S3/AWS-S3.yml b/Packs/AWS-S3/Integrations/AWS-S3/AWS-S3.yml index 4bbb51d6e265..b4d4a6ea29a3 100644 --- a/Packs/AWS-S3/Integrations/AWS-S3/AWS-S3.yml +++ b/Packs/AWS-S3/Integrations/AWS-S3/AWS-S3.yml @@ -405,7 +405,7 @@ script: required: true description: Creates or modifies the PublicAccessBlock configuration for an Amazon S3 bucket. name: aws-s3-put-public-access-block - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 runonce: false script: '' type: python diff --git a/Packs/AWS-S3/ReleaseNotes/1_2_43.md b/Packs/AWS-S3/ReleaseNotes/1_2_43.md new file mode 100644 index 000000000000..e518e3a7a94b --- /dev/null +++ b/Packs/AWS-S3/ReleaseNotes/1_2_43.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - S3 + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-S3/ReleaseNotes/1_2_44.md b/Packs/AWS-S3/ReleaseNotes/1_2_44.md new file mode 100644 index 000000000000..b530392bde3a --- /dev/null +++ b/Packs/AWS-S3/ReleaseNotes/1_2_44.md @@ -0,0 +1,3 @@ +## AWS - S3 + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AWS-S3/pack_metadata.json b/Packs/AWS-S3/pack_metadata.json index 87c126964ab9..f1444e3bf3e8 100644 --- a/Packs/AWS-S3/pack_metadata.json +++ b/Packs/AWS-S3/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - S3", "description": "Amazon Web Services Simple Storage Service (S3)", "support": "xsoar", - "currentVersion": "1.2.42", + "currentVersion": "1.2.44", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-SNS-Listener/Integrations/AWSSNSListener/AWSSNSListener.yml b/Packs/AWS-SNS-Listener/Integrations/AWSSNSListener/AWSSNSListener.yml index d58a17f28821..722f7a333ceb 100644 --- a/Packs/AWS-SNS-Listener/Integrations/AWSSNSListener/AWSSNSListener.yml +++ b/Packs/AWS-SNS-Listener/Integrations/AWSSNSListener/AWSSNSListener.yml @@ -62,7 +62,7 @@ display: AWS-SNS-Listener name: AWS-SNS-Listener script: commands: [] - dockerimage: demisto/fastapi:0.125.0.10158186 + dockerimage: demisto/fastapi:0.125.0.12689358 longRunning: true longRunningPort: true script: '-' @@ -83,3 +83,4 @@ triggers: action: hidden: true required: false +excludefromtightlycoupled: true diff --git a/Packs/AWS-SNS-Listener/ReleaseNotes/1_0_17.md b/Packs/AWS-SNS-Listener/ReleaseNotes/1_0_17.md new file mode 100644 index 000000000000..a3f67d1bac18 --- /dev/null +++ b/Packs/AWS-SNS-Listener/ReleaseNotes/1_0_17.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS-SNS-Listener + +- Updated the Docker image to: *demisto/fastapi:0.125.0.12689358*. + diff --git a/Packs/AWS-SNS-Listener/ReleaseNotes/1_0_18.md b/Packs/AWS-SNS-Listener/ReleaseNotes/1_0_18.md new file mode 100644 index 000000000000..c304fef25965 --- /dev/null +++ b/Packs/AWS-SNS-Listener/ReleaseNotes/1_0_18.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### AWS-SNS-Listener + +- Documentation and metadata improvements. diff --git a/Packs/AWS-SNS-Listener/pack_metadata.json b/Packs/AWS-SNS-Listener/pack_metadata.json index 9ab985f59132..5659c2594a69 100644 --- a/Packs/AWS-SNS-Listener/pack_metadata.json +++ b/Packs/AWS-SNS-Listener/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS-SNS-Listener", "description": "A long running AWS SNS Listener service that can subscribe to an SNS topic and create incidents from the messages received.", "support": "xsoar", - "currentVersion": "1.0.16", + "currentVersion": "1.0.18", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-SNS/Integrations/AWSSNS/AWSSNS.yml b/Packs/AWS-SNS/Integrations/AWSSNS/AWSSNS.yml index ad1589cd2118..5596a710ea39 100644 --- a/Packs/AWS-SNS/Integrations/AWSSNS/AWSSNS.yml +++ b/Packs/AWS-SNS/Integrations/AWSSNS/AWSSNS.yml @@ -232,7 +232,7 @@ script: outputs: - contextPath: AWS.SNS.Subscriptions.SubscriptionArn description: The Subscription Arn. - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 script: '' subtype: python3 type: python diff --git a/Packs/AWS-SNS/ReleaseNotes/1_0_27.md b/Packs/AWS-SNS/ReleaseNotes/1_0_27.md new file mode 100644 index 000000000000..98de0a4e088f --- /dev/null +++ b/Packs/AWS-SNS/ReleaseNotes/1_0_27.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - SNS + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-SNS/pack_metadata.json b/Packs/AWS-SNS/pack_metadata.json index d3006ab27dd2..54dfd5cff4f4 100644 --- a/Packs/AWS-SNS/pack_metadata.json +++ b/Packs/AWS-SNS/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - SNS", "description": "This is the integration content pack which can create or delete topic/subscription on AWS Simple Notification System and send the message via SNS as well.", "support": "xsoar", - "currentVersion": "1.0.26", + "currentVersion": "1.0.27", "author": "Cortex XSOAR", "url": "", "email": "", diff --git a/Packs/AWS-SQS/Integrations/AWS-SQS/AWS-SQS.yml b/Packs/AWS-SQS/Integrations/AWS-SQS/AWS-SQS.yml index 0b47d9192a72..d2e329f53657 100644 --- a/Packs/AWS-SQS/Integrations/AWS-SQS/AWS-SQS.yml +++ b/Packs/AWS-SQS/Integrations/AWS-SQS/AWS-SQS.yml @@ -102,6 +102,13 @@ configuration: type: 13 section: Connect required: false +- display: Incidents Fetch Interval + name: incidentFetchInterval + defaultvalue: '1' + required: false + type: 19 + section: Collect + advanced: true - display: AWS STS Regional Endpoints additionalinfo: Sets the AWS_STS_REGIONAL_ENDPOINTS environment variable to specify the AWS STS endpoint resolution logic. By default, this option is set to “legacy” in AWS. Leave empty if the environment variable is already set using server configuration. name: sts_regional_endpoint diff --git a/Packs/AWS-SQS/ReleaseNotes/1_2_37.md b/Packs/AWS-SQS/ReleaseNotes/1_2_37.md new file mode 100644 index 000000000000..aa438c7cba06 --- /dev/null +++ b/Packs/AWS-SQS/ReleaseNotes/1_2_37.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### AWS - SQS + +- Updated the Fetch Incidents configuration parameters. diff --git a/Packs/AWS-SQS/pack_metadata.json b/Packs/AWS-SQS/pack_metadata.json index 9fa7738c2878..8436cad15225 100644 --- a/Packs/AWS-SQS/pack_metadata.json +++ b/Packs/AWS-SQS/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - SQS", "description": "Amazon Web Services Simple Queuing Service (SQS)", "support": "xsoar", - "currentVersion": "1.2.36", + "currentVersion": "1.2.37", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.py b/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.py index 6907314e0b1a..8fec86762437 100644 --- a/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.py +++ b/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.py @@ -26,26 +26,39 @@ def parse_aws_timestamp(timestamp_str: str) -> dt.datetime: """ - Parse AWS timestamp with flexible format support. - The AWS API returns timestamps in two formats: - - With milliseconds: "2023-01-01T00:00:00.000Z" - - Without milliseconds: "2023-01-01T00:00:00Z" + Parse an AWS timestamp into a naive UTC datetime. + + AWS returns 'CreatedAt' in several ISO-8601 shapes, e.g.: + - With milliseconds and 'Z': "2023-01-01T00:00:00.000Z" + - Without milliseconds and 'Z': "2023-01-01T00:00:00Z" + - With a UTC/other offset: "2026-08-08T19:55:21.905451+00:00" + + The returned value feeds strftime() when building the AWS findings filter, which + expects a naive UTC value. A timezone-aware value would shift the fetch window, + so any offset is normalized to UTC and then dropped, yielding a naive datetime. Args: - timestamp_str (str): Timestamp string from AWS API + timestamp_str (str): Timestamp string from AWS API. Returns: - datetime: Parsed datetime object + datetime: Parsed naive UTC datetime object. Raises: - ValueError: If timestamp format is not supported + ValueError: If the timestamp cannot be parsed as ISO-8601. """ - try: - # Try parsing with milliseconds first - return dt.datetime.strptime(timestamp_str, DATETIME_FORMAT) - except ValueError: - # Fallback to parsing without milliseconds - return dt.datetime.strptime(timestamp_str, DATETIME_FORMAT_NO_MS) + for fmt in (DATETIME_FORMAT, DATETIME_FORMAT_NO_MS): + try: + return dt.datetime.strptime(timestamp_str, fmt) + except ValueError: + continue + + # Fallback: accept any ISO-8601 form (e.g. an offset-style "+00:00"). + # 'Z' is normalized to "+00:00" for fromisoformat compatibility on older Python versions. + parsed = dt.datetime.fromisoformat(timestamp_str.replace("Z", "+00:00")) + if parsed.tzinfo is not None: + # Normalize to UTC, then drop tzinfo so the value stays naive for strftime(). + parsed = parsed.astimezone(dt.UTC).replace(tzinfo=None) + return parsed def generate_last_run(events: list["AwsSecurityFindingTypeDef"], previous_last_run: dict | None = None) -> dict[str, Any]: @@ -239,8 +252,9 @@ def fetch_events( """ demisto.debug(f"Fetching events with last_run: {last_run}") if last_run.get("last_update_date"): + # If the stored cursor cannot be parsed, let the ValueError propagate and fail the + # fetch loudly rather than silently resetting the fetch window. start_time = parse_aws_timestamp(last_run["last_update_date"]) - else: start_time = cast(dt.datetime, first_fetch_time) diff --git a/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.yml b/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.yml index e9f455937b3c..9d88a4210346 100644 --- a/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.yml +++ b/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector.yml @@ -132,7 +132,7 @@ script: name: end_time description: Fetch events from AWS Security Hub. name: aws-securityhub-get-events - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 isfetchevents: true script: '-' subtype: python3 diff --git a/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector_test.py b/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector_test.py index 1749a0074f12..d068c73365d8 100644 --- a/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector_test.py +++ b/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/AWSSecurityHubEventCollector_test.py @@ -128,16 +128,29 @@ class TestParseAwsTimestamp: ("2023-01-01T12:30:45.123Z", dt.datetime(2023, 1, 1, 12, 30, 45, 123000)), # Valid timestamp without milliseconds ("2023-01-01T12:30:45Z", dt.datetime(2023, 1, 1, 12, 30, 45)), + # ISO-8601 with UTC offset (+00:00) - returned verbatim by AWS CreatedAt for some sources + ("2026-08-08T19:55:21.905451+00:00", dt.datetime(2026, 8, 8, 19, 55, 21, 905451)), + # ISO-8601 with UTC offset, no microseconds + ("2026-08-08T19:55:21+00:00", dt.datetime(2026, 8, 8, 19, 55, 21)), + # ISO-8601 with a non-zero offset should be normalized to naive UTC + ("2026-08-08T21:55:21+02:00", dt.datetime(2026, 8, 8, 19, 55, 21)), + # ISO-8601 without 'Z' and without offset: fails both strptime formats and is + # parsed by fromisoformat as an already-naive datetime (no tzinfo normalization). + ("2026-08-08T19:55:21.905451", dt.datetime(2026, 8, 8, 19, 55, 21, 905451)), + ("2026-08-08T19:55:21", dt.datetime(2026, 8, 8, 19, 55, 21)), ], ) def test_parse_timestamp_valid_formats(self, timestamp_str, expected_result): """ - Given: A valid timestamp string from AWS API (with or without milliseconds). + Given: A valid timestamp string from AWS API (Z-suffixed or ISO-8601 offset form). When: Parsing the timestamp using parse_aws_timestamp. - Then: Should return the correct datetime object. + Then: Should return the correct naive UTC datetime object. """ result = parse_aws_timestamp(timestamp_str) assert result == expected_result + # The value feeds strftime() to build the AWS filter, so it must be naive (no tzinfo) + # to avoid shifting the fetch window. + assert result.tzinfo is None def test_parse_timestamp_invalid_format(self): """ @@ -145,7 +158,7 @@ def test_parse_timestamp_invalid_format(self): When: Parsing the timestamp using parse_aws_timestamp. Then: Should raise ValueError. """ - invalid_timestamp = "2023/01/01 12:30:45" + invalid_timestamp = "not-a-timestamp" with pytest.raises(ValueError): parse_aws_timestamp(invalid_timestamp) @@ -662,3 +675,33 @@ def test_fetch_events_api_error_handling(fetch_events_mocks): # Verify error logging error_calls = [call.args[0] for call in mocks["error"].call_args_list] assert any("Error while fetching events" in msg for msg in error_calls) + + +def test_fetch_events_unparseable_last_run_raises(fetch_events_mocks): + """ + Test fetch_events fails loudly when last_run holds a value that parse_aws_timestamp + cannot parse, rather than silently resetting the fetch window. + + Given: last_run["last_update_date"] cannot be parsed + When: Calling fetch_events + Then: The ValueError from parse_aws_timestamp propagates and get_events is never called. + """ + mocks = fetch_events_mocks + + # Simulate parse_aws_timestamp rejecting the stored cursor. That a truly malformed value + # actually raises here is covered by TestParseAwsTimestamp; this test isolates propagation. + parse_error = ValueError("Invalid isoformat string: 'not-a-parseable-timestamp'") + mocks["parse_timestamp"].side_effect = parse_error + + with pytest.raises(ValueError) as exc_info: + fetch_events( + client=mocks["client"], + last_run={"last_update_date": "not-a-parseable-timestamp", "last_update_date_finding_ids": ["old-1"]}, + first_fetch_time=FIRST_FETCH_TIME, + page_size=50, + limit=0, + ) + + assert exc_info.value is parse_error + # The fetch fails before any events are requested. + mocks["get_events"].assert_not_called() diff --git a/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/README.md b/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/README.md index 4c252118addf..666310349450 100644 --- a/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/README.md +++ b/Packs/AWS-SecurityHub/Integrations/AWSSecurityHubEventCollector/README.md @@ -1,7 +1,5 @@ An XSIAM event collector for AWS Security Hub. -This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM. - ## Configure AWS Security Hub Event Collector in Cortex | **Parameter** | **Description** | **Required** | diff --git a/Packs/AWS-SecurityHub/Integrations/AWS_SecurityHub/AWS_SecurityHub.yml b/Packs/AWS-SecurityHub/Integrations/AWS_SecurityHub/AWS_SecurityHub.yml index d9158b623e51..a8ca11f3b3ba 100644 --- a/Packs/AWS-SecurityHub/Integrations/AWS_SecurityHub/AWS_SecurityHub.yml +++ b/Packs/AWS-SecurityHub/Integrations/AWS_SecurityHub/AWS_SecurityHub.yml @@ -122,6 +122,13 @@ configuration: type: 13 section: Connect required: false +- display: Incidents Fetch Interval + name: incidentFetchInterval + defaultvalue: '1' + required: false + type: 19 + section: Collect + advanced: true - display: Security Hub Incidents Severity Level name: sh_severity type: 15 diff --git a/Packs/AWS-SecurityHub/Integrations/AWS_SecurityHub_V2/AWS_SecurityHub_V2.yml b/Packs/AWS-SecurityHub/Integrations/AWS_SecurityHub_V2/AWS_SecurityHub_V2.yml index 4c7c06d03daa..00362209541c 100644 --- a/Packs/AWS-SecurityHub/Integrations/AWS_SecurityHub_V2/AWS_SecurityHub_V2.yml +++ b/Packs/AWS-SecurityHub/Integrations/AWS_SecurityHub_V2/AWS_SecurityHub_V2.yml @@ -1,7 +1,7 @@ name: AWS - Security Hub v2 display: AWS - Security Hub v2 category: IT Services -provider: Amazon Web Services +provider: Amazon sectionorder: - Connect - Collect @@ -320,7 +320,7 @@ script: - name: remoteId description: The remote finding metadata UID to update. required: false - dockerimage: demisto/boto3py3:1.0.0.11314142 + dockerimage: demisto/boto3py3:1.0.0.12887184 isfetch: true ismappable: true isremotesyncin: true diff --git a/Packs/AWS-SecurityHub/ReleaseNotes/2_0_1.md b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_1.md new file mode 100644 index 000000000000..6d4c15bae254 --- /dev/null +++ b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_1.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### AWS - Security Hub v2 + +Updated the integration provider name to **Amazon**. diff --git a/Packs/AWS-SecurityHub/ReleaseNotes/2_0_2.md b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_2.md new file mode 100644 index 000000000000..718ad57f1b44 --- /dev/null +++ b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_2.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### AWS - Security Hub + +- Updated the Fetch Incidents configuration parameters. diff --git a/Packs/AWS-SecurityHub/ReleaseNotes/2_0_3.md b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_3.md new file mode 100644 index 000000000000..de592cfab333 --- /dev/null +++ b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_3.md @@ -0,0 +1,11 @@ + +#### Integrations + +##### AWS Security Hub Event Collector + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + +##### AWS - Security Hub v2 + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-SecurityHub/ReleaseNotes/2_0_4.md b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_4.md new file mode 100644 index 000000000000..d6bc88f31818 --- /dev/null +++ b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_4.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### AWS Security Hub Event Collector + +- Fixed an issue where the AWS Security Hub Event Collector fetch events failed due to an unsupported timestamp format returned by AWS. diff --git a/Packs/AWS-SecurityHub/ReleaseNotes/2_0_5.md b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_5.md new file mode 100644 index 000000000000..c6e483415ba4 --- /dev/null +++ b/Packs/AWS-SecurityHub/ReleaseNotes/2_0_5.md @@ -0,0 +1,3 @@ +## AWS - Security Hub + +- Documentation and metadata improvements. diff --git a/Packs/AWS-SecurityHub/pack_metadata.json b/Packs/AWS-SecurityHub/pack_metadata.json index f882613e2c0e..87fd5fdbaca7 100644 --- a/Packs/AWS-SecurityHub/pack_metadata.json +++ b/Packs/AWS-SecurityHub/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS - Security Hub", "description": "Amazon Web Services Security Hub Service.", "support": "xsoar", - "currentVersion": "2.0.0", + "currentVersion": "2.0.5", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -26,7 +26,6 @@ "itemPrefix": [ "AWS Security Hub" ], - "defaultDataSource": "AWS Security Hub Event Collector", "supportedModules": [ "agentix", "xsiam" diff --git a/Packs/AWS-SecurityLake/Integrations/AWSSecurityLake/AWSSecurityLake.yml b/Packs/AWS-SecurityLake/Integrations/AWSSecurityLake/AWSSecurityLake.yml index fdcc669b2249..eeb9b0b8006a 100644 --- a/Packs/AWS-SecurityLake/Integrations/AWSSecurityLake/AWSSecurityLake.yml +++ b/Packs/AWS-SecurityLake/Integrations/AWSSecurityLake/AWSSecurityLake.yml @@ -1212,7 +1212,7 @@ script: script: "-" type: python subtype: python3 - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 feed: false isfetch: false fromversion: 6.10.0 diff --git a/Packs/AWS-SecurityLake/ReleaseNotes/1_0_20.md b/Packs/AWS-SecurityLake/ReleaseNotes/1_0_20.md new file mode 100644 index 000000000000..e5ec28c99b1c --- /dev/null +++ b/Packs/AWS-SecurityLake/ReleaseNotes/1_0_20.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### Amazon Security Lake + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS-SecurityLake/pack_metadata.json b/Packs/AWS-SecurityLake/pack_metadata.json index 5595f503fde6..c51a18bda7be 100644 --- a/Packs/AWS-SecurityLake/pack_metadata.json +++ b/Packs/AWS-SecurityLake/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Amazon - Security Lake", "description": "Amazon Security Lake is a fully managed security data lake service.", "support": "xsoar", - "currentVersion": "1.0.19", + "currentVersion": "1.0.20", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS/Integrations/AWS/AWS.py b/Packs/AWS/Integrations/AWS/AWS.py index 3b6d020f5a4f..f19e71ff2129 100644 --- a/Packs/AWS/Integrations/AWS/AWS.py +++ b/Packs/AWS/Integrations/AWS/AWS.py @@ -649,6 +649,8 @@ def aws_ec2_block_device_mapping_args_builder(args: Dict[str, Any]) -> List[Dict "VolumeSize": arg_to_number(args.get("ebs_volume_size")), "VolumeType": args.get("ebs_volume_type"), "Throughput": arg_to_number(args.get("ebs_throughput")), + "EbsCardIndex": arg_to_number(args.get("ebs_card_index")), + "VolumeInitializationRate": arg_to_number(args.get("ebs_initialization_rate")), }, "NoDevice": args.get("block_device_mappings_no_device"), "VirtualName": args.get("block_device_mappings_virtual_name"), @@ -1682,7 +1684,7 @@ def get_public_access_block_command(client: BotoClient, args: Dict[str, Any]) -> if response.get("ResponseMetadata", {}).get("HTTPStatusCode") == HTTPStatus.OK: return CommandResults( - outputs_prefix="AWS.S3-Buckets", + outputs_prefix="AWS.S3.Buckets", outputs_key_field="BucketName", outputs={"BucketName": bucket_name, "PublicAccessBlock": response.get("PublicAccessBlockConfiguration", {})}, readable_output=tableToMarkdown( @@ -1723,7 +1725,7 @@ def get_bucket_encryption_command(client: BotoClient, args: Dict[str, Any]) -> C "ServerSideEncryptionConfiguration": response.get("ServerSideEncryptionConfiguration", {}), } return CommandResults( - outputs_prefix="AWS.S3-Buckets", + outputs_prefix="AWS.S3.Buckets", outputs_key_field="BucketName", outputs=outputs, readable_output=tableToMarkdown( @@ -1761,7 +1763,7 @@ def get_bucket_policy_command(client: BotoClient, args: Dict[str, Any]) -> Comma json_response = json.loads(response.get("Policy", "{}")) json_statement = json_response.get("Statement", []) return CommandResults( - outputs_prefix="AWS.S3-Buckets", + outputs_prefix="AWS.S3.Buckets", outputs_key_field="BucketName", outputs={ "BucketName": bucket_name, @@ -1866,7 +1868,7 @@ def file_download_command(client: BotoClient, args: Dict[str, Any]): filename = key.rsplit("/", 1)[-1] return fileResult(filename, data) except ClientError as err: - AWSErrorHandler.handle_client_error(err) + return AWSErrorHandler.handle_client_error(err) except Exception as e: raise DemistoException(f"Error: {str(e)}") @@ -1901,7 +1903,7 @@ def file_upload_command(client: BotoClient, args: Dict[str, Any]): return CommandResults(readable_output="Failed to upload file") @staticmethod - def get_bucket_website_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def get_bucket_website_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Retrieves the website configuration for a specified Amazon S3 bucket. The function calls the AWS S3 'get_bucket_website' API to check if the bucket @@ -1921,7 +1923,7 @@ def get_bucket_website_command(client: BotoClient, args: Dict[str, Any]) -> Comm response = client.get_bucket_website(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response["WebsiteConfiguration"] = { "ErrorDocument": response.get("ErrorDocument"), @@ -1939,13 +1941,13 @@ def get_bucket_website_command(client: BotoClient, args: Dict[str, Any]) -> Comm ) return CommandResults( readable_output=readable_output, - outputs_prefix="AWS.S3-Buckets.BucketWebsite", + outputs_prefix="AWS.S3.Buckets.BucketWebsite", outputs=response.get("WebsiteConfiguration", {}), raw_response=response.get("WebsiteConfiguration", {}), ) @staticmethod - def get_bucket_acl_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def get_bucket_acl_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Retrieves the Access Control List (ACL) of a specified Amazon S3 bucket. @@ -1966,7 +1968,7 @@ def get_bucket_acl_command(client: BotoClient, args: Dict[str, Any]) -> CommandR response = client.get_bucket_acl(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response["AccessControlPolicy"] = { "Grants": response.get("Grants"), @@ -1981,13 +1983,13 @@ def get_bucket_acl_command(client: BotoClient, args: Dict[str, Any]) -> CommandR ) return CommandResults( readable_output=readable_output, - outputs_prefix="AWS.S3-Buckets.BucketAcl", + outputs_prefix="AWS.S3.Buckets.BucketAcl", outputs=response.get("AccessControlPolicy", {}), raw_response=response.get("AccessControlPolicy", {}), ) @staticmethod - def bucket_create_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def bucket_create_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates a new S3 bucket. @@ -2001,6 +2003,7 @@ def bucket_create_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe bucket_name = args.get("bucket_name") location = args.get("location_constraint") or args.get("region", "") kwargs = { + "ACL": args.get("acl"), "Bucket": bucket_name, "GrantFullControl": args.get("grant_full_control"), "GrantRead": args.get("grant_read"), @@ -2019,7 +2022,7 @@ def bucket_create_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe demisto.debug(f"{response=}") if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) output = {"Location": response.get("Location"), "BucketArn": response.get("BucketArn"), "BucketName": bucket_name} @@ -2032,7 +2035,7 @@ def bucket_create_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe ) @staticmethod - def buckets_list_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def buckets_list_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Returns a list of all buckets owned by the authenticated sender of the request. @@ -2052,7 +2055,7 @@ def buckets_list_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes response = client.list_buckets(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) buckets = response.get("Buckets", []) for bucket in buckets: @@ -2693,7 +2696,7 @@ def modify_snapshot_permission_command(client: BotoClient, args: Dict[str, Any]) return CommandResults(readable_output=f"Snapshot {args.get('snapshot_id')} permissions were successfully updated.") @staticmethod - def describe_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Retrieves detailed information about EC2 instances including status, configuration, and metadata. @@ -2723,7 +2726,7 @@ def describe_instances_command(client: BotoClient, args: Dict[str, Any]) -> Comm remove_nulls_from_dictionary(kwargs) response = client.describe_instances(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) # Extract instances from reservations reservations = response.get("Reservations", []) @@ -2749,7 +2752,7 @@ def describe_instances_command(client: BotoClient, args: Dict[str, Any]) -> Comm ) @staticmethod - def run_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def run_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Runs one or more Amazon EC2 instances. @@ -2853,7 +2856,7 @@ def run_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.run_instances(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) instances = response.get("Instances", []) if not instances: @@ -3260,7 +3263,7 @@ def describe_ipam_resource_discovery_associations_command(client: BotoClient, ar return command_results @staticmethod - def get_latest_ami_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def get_latest_ami_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Retrieves information about the latest Amazon Machine Image (AMI) based on provided filters. The function calls the AWS EC2 'describe_images' API, sorts the results by @@ -3287,7 +3290,7 @@ def get_latest_ami_command(client: BotoClient, args: Dict[str, Any]) -> CommandR remove_nulls_from_dictionary(kwargs) response = client.describe_images(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) amis = response.get("Images", []) iterates = 1 @@ -3296,7 +3299,7 @@ def get_latest_ami_command(client: BotoClient, args: Dict[str, Any]) -> CommandR kwargs["NextToken"] = response.get("nextToken") response = client.describe_images(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) amis.extend(response.get("Images", [])) iterates += 1 @@ -3327,7 +3330,7 @@ def get_latest_ami_command(client: BotoClient, args: Dict[str, Any]) -> CommandR ) @staticmethod - def create_network_acl_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_network_acl_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates a Network Access Control List (Network ACL) for the specified VPC. The function calls the AWS EC2 'create_network_acl' API. It requires the ID @@ -3356,7 +3359,7 @@ def create_network_acl_command(client: BotoClient, args: Dict[str, Any]) -> Comm response = client.create_network_acl(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) network_acl = response.get("NetworkAcl") readable_data = { @@ -3387,7 +3390,7 @@ def create_network_acl_command(client: BotoClient, args: Dict[str, Any]) -> Comm ) @staticmethod - def get_ipam_discovered_public_addresses_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def get_ipam_discovered_public_addresses_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ aws-ec2-get-ipam-discovered-public-addresses: Gets the public IP addresses that have been discovered by IPAM. @@ -3410,7 +3413,7 @@ def get_ipam_discovered_public_addresses_command(client: BotoClient, args: Dict[ remove_nulls_from_dictionary(kwargs) response = client.get_ipam_discovered_public_addresses(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) if not response.get("IpamDiscoveredPublicAddresses"): return CommandResults(readable_output="No Ipam Discovered Public Addresses were found.") @@ -3430,7 +3433,7 @@ def get_ipam_discovered_public_addresses_command(client: BotoClient, args: Dict[ ) @staticmethod - def create_tags_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_tags_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Adds or overwrites one or more tags for the specified AWS resources. @@ -3454,12 +3457,12 @@ def create_tags_command(client: BotoClient, args: Dict[str, Any]) -> CommandResu response = client.create_tags(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults(readable_output="The resources where tagged successfully") @staticmethod - def network_interface_attribute_modify_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def network_interface_attribute_modify_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Modifies the specified network interface attribute. Args: @@ -3475,7 +3478,7 @@ def network_interface_attribute_modify_command(client: BotoClient, args: Dict[st response = client.modify_network_interface_attribute(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) demisto.debug(f"{response=}") outputs = { @@ -3494,7 +3497,7 @@ def network_interface_attribute_modify_command(client: BotoClient, args: Dict[st ) @staticmethod - def regions_describe_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def regions_describe_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes the Regions that are enabled for your account, or all Regions. @@ -3522,7 +3525,7 @@ def regions_describe_command(client: BotoClient, args: Dict[str, Any]) -> Comman demisto.debug(f"{kwargs=}") response = client.describe_regions(**kwargs) if response["ResponseMetadata"]["HTTPStatusCode"] not in [HTTPStatus.OK, HTTPStatus.NO_CONTENT]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) regions = response.get("Regions") readable_output = tableToMarkdown( @@ -3670,7 +3673,7 @@ def describe_security_groups_command(client: BotoClient, args: Dict[str, Any]) - return AWSErrorHandler.handle_response_error(response) @staticmethod - def describe_addresses_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_addresses_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes one or more Elastic IP addresses. @@ -3704,7 +3707,7 @@ def describe_addresses_command(client: BotoClient, args: Dict[str, Any]) -> Comm response = client.describe_addresses(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) addresses = response.get("Addresses", []) if not addresses: @@ -3737,7 +3740,7 @@ def describe_addresses_command(client: BotoClient, args: Dict[str, Any]) -> Comm ) @staticmethod - def allocate_address_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def allocate_address_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Allocates an Elastic IP address to your AWS account. @@ -3767,7 +3770,7 @@ def allocate_address_command(client: BotoClient, args: Dict[str, Any]) -> Comman print_debug_logs(client, f"Allocating address with parameters: {kwargs}") response = client.allocate_address(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response to handle datetime objects response = serialize_response_with_datetime_encoding(response) @@ -3788,7 +3791,7 @@ def allocate_address_command(client: BotoClient, args: Dict[str, Any]) -> Comman ) @staticmethod - def associate_address_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def associate_address_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Associates an Elastic IP address with an instance or a network interface. @@ -3817,7 +3820,7 @@ def associate_address_command(client: BotoClient, args: Dict[str, Any]) -> Comma response = client.associate_address(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Build output data output_data = { @@ -3840,7 +3843,7 @@ def associate_address_command(client: BotoClient, args: Dict[str, Any]) -> Comma ) @staticmethod - def disassociate_address_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def disassociate_address_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Disassociates an Elastic IP address from the instance or network interface it's associated with. @@ -3858,7 +3861,7 @@ def disassociate_address_command(client: BotoClient, args: Dict[str, Any]) -> Co response = client.disassociate_address(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults( readable_output=f"Successfully disassociated Elastic IP address (Association ID: {args.get('association_id')})", @@ -3866,7 +3869,7 @@ def disassociate_address_command(client: BotoClient, args: Dict[str, Any]) -> Co ) @staticmethod - def release_address_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def release_address_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Releases the specified Elastic IP address. @@ -3890,7 +3893,7 @@ def release_address_command(client: BotoClient, args: Dict[str, Any]) -> Command response = client.release_address(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults( readable_output=f"Successfully released Elastic IP address (Allocation ID: {args.get('allocation_id')})", @@ -3940,7 +3943,7 @@ def authorize_security_group_egress_command(client: BotoClient, args: Dict[str, return AWSErrorHandler.handle_response_error(response) @staticmethod - def describe_images_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_images_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes one or more Amazon Machine Images (AMIs) available to you. @@ -3995,7 +3998,7 @@ def describe_images_command(client: BotoClient, args: Dict[str, Any]) -> Command response = client.describe_images(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) images = response.get("Images", []) if not images: @@ -4029,7 +4032,7 @@ def describe_images_command(client: BotoClient, args: Dict[str, Any]) -> Command ) @staticmethod - def create_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates an Amazon EBS-backed AMI from an Amazon EBS-backed instance. @@ -4072,7 +4075,7 @@ def create_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes response = client.create_image(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response to handle datetime objects response = serialize_response_with_datetime_encoding(response) @@ -4100,7 +4103,7 @@ def create_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes ) @staticmethod - def deregister_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def deregister_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deregisters the specified Amazon Machine Image (AMI). @@ -4120,7 +4123,7 @@ def deregister_image_command(client: BotoClient, args: Dict[str, Any]) -> Comman response = client.deregister_image(ImageId=image_id) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults( readable_output=f"Successfully deregistered AMI: {image_id}", @@ -4128,7 +4131,7 @@ def deregister_image_command(client: BotoClient, args: Dict[str, Any]) -> Comman ) @staticmethod - def copy_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def copy_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Copy an Amazon Machine Image (AMI) from a source region to the current region. @@ -4170,7 +4173,7 @@ def copy_image_command(client: BotoClient, args: Dict[str, Any]) -> CommandResul response = client.copy_image(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) region = args.get("region", "") @@ -4283,7 +4286,7 @@ def monitor_instances_command(client: BotoClient, args: Dict[str, Any]) -> Comma response = client.monitor_instances(InstanceIds=instance_ids) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) instance_monitorings = response.get("InstanceMonitorings", []) @@ -4331,7 +4334,7 @@ def unmonitor_instances_command(client: BotoClient, args: Dict[str, Any]) -> Com response = client.unmonitor_instances(InstanceIds=instance_ids) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) instance_monitorings = response.get("InstanceMonitorings", []) @@ -4379,7 +4382,7 @@ def reboot_instances_command(client: BotoClient, args: Dict[str, Any]) -> Comman response = client.reboot_instances(InstanceIds=instance_ids) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults( readable_output=f"Successfully initiated reboot for instances: {', '.join(instance_ids)}", raw_response=response @@ -4567,7 +4570,7 @@ def describe_iam_instance_profile_associations_command(client: BotoClient, args: response = client.describe_iam_instance_profile_associations(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) associations = response.get("IamInstanceProfileAssociations", []) @@ -4609,7 +4612,7 @@ def get_password_data_command(client: BotoClient, args: Dict[str, Any]) -> Comma response = client.get_password_data(InstanceId=instance_id) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize datetime response = serialize_response_with_datetime_encoding(response) @@ -4669,7 +4672,7 @@ def describe_reserved_instances_command(client: BotoClient, args: Dict[str, Any] response = client.describe_reserved_instances(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize datetime objects serialized_response = serialize_response_with_datetime_encoding(response) @@ -4706,7 +4709,7 @@ def describe_reserved_instances_command(client: BotoClient, args: Dict[str, Any] ) @staticmethod - def describe_volumes_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_volumes_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes the specified EBS volumes or all of your EBS volumes. @@ -4736,7 +4739,7 @@ def describe_volumes_command(client: BotoClient, args: Dict[str, Any]) -> Comman response = client.describe_volumes(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) volumes = response.get("Volumes", []) @@ -4763,7 +4766,7 @@ def describe_volumes_command(client: BotoClient, args: Dict[str, Any]) -> Comman ) @staticmethod - def modify_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def modify_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Modifies several parameters of an existing EBS volume, including volume size, volume type, and IOPS capacity. @@ -4788,7 +4791,7 @@ def modify_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.modify_volume(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) volume_modification = response.get("VolumeModification", {}) @@ -4826,7 +4829,7 @@ def modify_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe ) @staticmethod - def create_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates an EBS volume that can be attached to an instance in the same Availability Zone. @@ -4859,7 +4862,7 @@ def create_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.create_volume(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) @@ -4880,7 +4883,7 @@ def create_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe ) @staticmethod - def attach_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def attach_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Attaches an EBS volume to a running or stopped instance and exposes it to the instance with the specified device name. @@ -4901,7 +4904,7 @@ def attach_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.attach_volume(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) outputs = {k: v for k, v in response.items() if k != "ResponseMetadata"} @@ -4922,7 +4925,7 @@ def attach_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe ) @staticmethod - def detach_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def detach_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Detaches an EBS volume from an instance. @@ -4945,7 +4948,7 @@ def detach_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.detach_volume(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) @@ -4967,7 +4970,7 @@ def detach_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe ) @staticmethod - def delete_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def delete_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes the specified EBS volume. The volume must be in the available state (not attached to an instance). @@ -4983,12 +4986,12 @@ def delete_volume_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.delete_volume(VolumeId=volume_id) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults(readable_output=f"Successfully deleted volume {volume_id}") @staticmethod - def describe_snapshots_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_snapshots_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes one or more Amazon EBS snapshots available to you. @@ -5026,7 +5029,7 @@ def describe_snapshots_command(client: BotoClient, args: Dict[str, Any]) -> Comm response = client.describe_snapshots(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) snapshots = response.get("Snapshots", []) @@ -5064,7 +5067,7 @@ def describe_snapshots_command(client: BotoClient, args: Dict[str, Any]) -> Comm ) @staticmethod - def delete_snapshot_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def delete_snapshot_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes the specified Amazon EBS snapshot. @@ -5081,12 +5084,12 @@ def delete_snapshot_command(client: BotoClient, args: Dict[str, Any]) -> Command response = client.delete_snapshot(SnapshotId=snapshot_id) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults(readable_output=f"Successfully deleted snapshot {snapshot_id}", raw_response=response) @staticmethod - def copy_snapshot_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def copy_snapshot_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Copies a point-in-time snapshot of an Amazon EBS volume and stores it in Amazon S3. @@ -5123,7 +5126,7 @@ def copy_snapshot_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.copy_snapshot(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) outputs = {k: v for k, v in response.items() if k != "ResponseMetadata"} readable_output = tableToMarkdown( @@ -5192,7 +5195,7 @@ def snapshot_completed_waiter_command(client: BotoClient, args: Dict[str, Any]) raise DemistoException(f"Waiter error: {str(e)}") @staticmethod - def describe_launch_templates_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_launch_templates_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes one or more launch templates. @@ -5226,7 +5229,7 @@ def describe_launch_templates_command(client: BotoClient, args: Dict[str, Any]) response = client.describe_launch_templates(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response to handle datetime objects response = serialize_response_with_datetime_encoding(response) @@ -5259,7 +5262,7 @@ def describe_launch_templates_command(client: BotoClient, args: Dict[str, Any]) ) @staticmethod - def create_launch_template_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_launch_template_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates a launch template. A launch template contains the parameters to launch an instance. @@ -5276,7 +5279,7 @@ def create_launch_template_command(client: BotoClient, args: Dict[str, Any]) -> response = client.create_launch_template(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response to handle datetime objects response = serialize_response_with_datetime_encoding(response) @@ -5304,7 +5307,7 @@ def create_launch_template_command(client: BotoClient, args: Dict[str, Any]) -> ) @staticmethod - def delete_launch_template_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def delete_launch_template_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes a launch template. Deleting a launch template deletes all of its versions. @@ -5331,7 +5334,7 @@ def delete_launch_template_command(client: BotoClient, args: Dict[str, Any]) -> response = client.delete_launch_template(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response to handle datetime objects response = serialize_response_with_datetime_encoding(response) @@ -5359,7 +5362,7 @@ def delete_launch_template_command(client: BotoClient, args: Dict[str, Any]) -> ) @staticmethod - def create_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Launches an EC2 Fleet. @@ -5380,7 +5383,7 @@ def create_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes response = client.create_fleet(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) outputs = {k: v for k, v in response.items() if k != "ResponseMetadata"} @@ -5400,7 +5403,7 @@ def create_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes ) @staticmethod - def delete_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def delete_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes the specified EC2 Fleet. @@ -5420,7 +5423,7 @@ def delete_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes response = client.delete_fleets(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) successful = response.get("SuccessfulFleetDeletions", []) @@ -5466,7 +5469,7 @@ def delete_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes ) @staticmethod - def describe_fleets_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_fleets_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes one or more of your EC2 Fleets. @@ -5493,7 +5496,7 @@ def describe_fleets_command(client: BotoClient, args: Dict[str, Any]) -> Command response = client.describe_fleets(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) fleets = response.get("Fleets", []) @@ -5519,7 +5522,7 @@ def describe_fleets_command(client: BotoClient, args: Dict[str, Any]) -> Command ) @staticmethod - def describe_fleet_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_fleet_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes the running instances for the specified EC2 Fleet. @@ -5543,7 +5546,7 @@ def describe_fleet_instances_command(client: BotoClient, args: Dict[str, Any]) - response = client.describe_fleet_instances(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) active_instances = response.get("ActiveInstances", []) @@ -5555,6 +5558,7 @@ def describe_fleet_instances_command(client: BotoClient, args: Dict[str, Any]) - response_data = {k: v for k, v in response.items() if k != "ResponseMetadata"} return CommandResults( + outputs_prefix="AWS.EC2.Fleets", outputs=response_data, outputs_key_field="FleetId", readable_output=tableToMarkdown( @@ -5568,7 +5572,7 @@ def describe_fleet_instances_command(client: BotoClient, args: Dict[str, Any]) - ) @staticmethod - def modify_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def modify_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Modifies the specified EC2 Fleet. @@ -5603,7 +5607,7 @@ def modify_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes response = client.modify_fleet(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults( outputs_prefix="AWS.EC2.Fleets", @@ -5614,7 +5618,7 @@ def modify_fleet_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes ) @staticmethod - def delete_vpc_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def delete_vpc_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes the specified VPC. @@ -5630,12 +5634,12 @@ def delete_vpc_command(client: BotoClient, args: Dict[str, Any]) -> CommandResul response = client.delete_vpc(VpcId=vpc_id) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults(readable_output=f"Successfully deleted VPC {vpc_id}") @staticmethod - def create_vpc_endpoint_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_vpc_endpoint_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates a VPC endpoint for a specified service. An endpoint enables you to create a private connection between your VPC and the service. @@ -5683,7 +5687,7 @@ def create_vpc_endpoint_command(client: BotoClient, args: Dict[str, Any]) -> Com response = client.create_vpc_endpoint(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) vpc_endpoint = response.get("VpcEndpoint", {}) @@ -5703,7 +5707,7 @@ def create_vpc_endpoint_command(client: BotoClient, args: Dict[str, Any]) -> Com ) @staticmethod - def describe_internet_gateways_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_internet_gateways_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes one or more of your internet gateways. @@ -5727,7 +5731,7 @@ def describe_internet_gateways_command(client: BotoClient, args: Dict[str, Any]) response = client.describe_internet_gateways(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) internet_gateways = response.get("InternetGateways", []) @@ -5767,7 +5771,7 @@ def describe_internet_gateways_command(client: BotoClient, args: Dict[str, Any]) ) @staticmethod - def detach_internet_gateway_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def detach_internet_gateway_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Detaches an internet gateway from a VPC, disabling connectivity between the internet and the VPC. @@ -5785,12 +5789,12 @@ def detach_internet_gateway_command(client: BotoClient, args: Dict[str, Any]) -> response = client.detach_internet_gateway(InternetGatewayId=internet_gateway_id, VpcId=vpc_id) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults(readable_output=f"Successfully detached internet gateway {internet_gateway_id} from VPC {vpc_id}") @staticmethod - def delete_internet_gateway_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def delete_internet_gateway_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes the specified internet gateway. @@ -5806,12 +5810,12 @@ def delete_internet_gateway_command(client: BotoClient, args: Dict[str, Any]) -> response = client.delete_internet_gateway(InternetGatewayId=internet_gateway_id) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults(readable_output=f"Successfully deleted internet gateway {internet_gateway_id}") @staticmethod - def delete_subnet_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def delete_subnet_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes the specified subnet. @@ -5827,12 +5831,12 @@ def delete_subnet_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.delete_subnet(SubnetId=subnet_id) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults(readable_output=f"Successfully deleted subnet {subnet_id}") @staticmethod - def create_network_acl_entry_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_network_acl_entry_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates an entry (a rule) in a network ACL with the specified rule number. @@ -5871,12 +5875,12 @@ def create_network_acl_entry_command(client: BotoClient, args: Dict[str, Any]) - response = client.create_network_acl_entry(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) return CommandResults(readable_output=f"Successfully created network ACL entry for {args.get('network_acl_id')}") @staticmethod - def describe_key_pairs_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_key_pairs_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes the specified key pairs or all of your key pairs. Args: @@ -5897,7 +5901,7 @@ def describe_key_pairs_command(client: BotoClient, args: Dict[str, Any]) -> Comm print_debug_logs(client, f"Describing key pairs with parameters: {kwargs}") response = client.describe_key_pairs(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) key_pairs = response.get("KeyPairs", []) @@ -5920,7 +5924,7 @@ def describe_key_pairs_command(client: BotoClient, args: Dict[str, Any]) -> Comm ) @staticmethod - def allocate_hosts_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def allocate_hosts_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Allocates a Dedicated Host to your account. Args: @@ -5950,7 +5954,7 @@ def allocate_hosts_command(client: BotoClient, args: Dict[str, Any]) -> CommandR print_debug_logs(client, f"Allocating Dedicated Hosts with parameters: {kwargs}") response = client.allocate_hosts(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) host_ids = response.get("HostIds", []) return CommandResults( @@ -5961,7 +5965,7 @@ def allocate_hosts_command(client: BotoClient, args: Dict[str, Any]) -> CommandR ) @staticmethod - def release_hosts_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def release_hosts_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Releases the specified Dedicated Hosts. Args: @@ -5975,7 +5979,7 @@ def release_hosts_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.release_hosts(HostIds=host_ids) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) outputs = remove_empty_elements( { @@ -5998,7 +6002,7 @@ def release_hosts_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe ) @staticmethod - def create_traffic_mirror_session_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_traffic_mirror_session_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates a Traffic Mirror session. Args: @@ -6023,7 +6027,7 @@ def create_traffic_mirror_session_command(client: BotoClient, args: Dict[str, An print_debug_logs(client, f"Creating Traffic Mirror session with parameters: {kwargs}") response = client.create_traffic_mirror_session(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) session = response.get("TrafficMirrorSession", {}) @@ -6208,7 +6212,7 @@ def associate_access_policy_command(client: BotoClient, args: Dict[str, Any]) -> ) @staticmethod - def update_access_entry_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def update_access_entry_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Updates an existing Access Entry for an Amazon EKS cluster. @@ -6240,7 +6244,7 @@ def update_access_entry_command(client: BotoClient, args: Dict[str, Any]) -> Com response = client.update_access_entry(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) access_entry = serialize_response_with_datetime_encoding(response.get("accessEntry", {})) @@ -6262,7 +6266,7 @@ def update_access_entry_command(client: BotoClient, args: Dict[str, Any]) -> Com ) @staticmethod - def create_access_entry_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def create_access_entry_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates a new Access Entry for an Amazon EKS cluster. @@ -6293,7 +6297,7 @@ def create_access_entry_command(client: BotoClient, args: Dict[str, Any]) -> Com response = client.create_access_entry(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) access_entry = serialize_response_with_datetime_encoding(response.get("accessEntry", {})) @@ -6315,7 +6319,7 @@ def create_access_entry_command(client: BotoClient, args: Dict[str, Any]) -> Com ) @staticmethod - def list_clusters_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def list_clusters_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Returns a list of EKS clusters owned by the authenticated sender of the request. @@ -6335,7 +6339,7 @@ def list_clusters_command(client: BotoClient, args: Dict[str, Any]) -> CommandRe response = client.list_clusters(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) clusters = response.get("clusters", []) next_token = response.get("nextToken") @@ -6639,7 +6643,7 @@ def modify_event_subscription_command(client: BotoClient, args: Dict[str, Any]) raise DemistoException(f"Error: {str(e)}") @staticmethod - def describe_db_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def describe_db_instances_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Describes provisioned RDS instances. @@ -6667,7 +6671,7 @@ def describe_db_instances_command(client: BotoClient, args: Dict[str, Any]) -> C response = serialize_response_with_datetime_encoding(response) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) db_instances = response.get("DBInstances", []) if not db_instances: @@ -6702,7 +6706,7 @@ class Redshift: service = AWSServices.Redshift @staticmethod - def modify_cluster_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def modify_cluster_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Modifies the settings of a cluster. @@ -6747,7 +6751,7 @@ def modify_cluster_command(client: BotoClient, args: Dict[str, Any]) -> CommandR response = client.modify_cluster(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) cluster_data = response.get("Cluster", {}) @@ -7450,7 +7454,7 @@ class Lambda: service = AWSServices.LAMBDA @staticmethod - def get_function_configuration_command(client: BotoClient, args: Dict[str, Any]): + def get_function_configuration_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: """ Retrieves the configuration information for a Lambda function. @@ -7501,7 +7505,7 @@ def get_function_configuration_command(client: BotoClient, args: Dict[str, Any]) ) @staticmethod - def get_function_url_configuration_command(client: BotoClient, args: Dict[str, Any]): + def get_function_url_configuration_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: """ Retrieves the configuration for a Lambda function URL. @@ -7546,7 +7550,7 @@ def get_function_url_configuration_command(client: BotoClient, args: Dict[str, A ) @staticmethod - def update_function_url_configuration_command(client: BotoClient, args: Dict[str, Any]): + def update_function_url_configuration_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: """ Updates the configuration for a Lambda function URL. @@ -7648,7 +7652,7 @@ def _parse_policy_response(data: dict[str, Any]) -> tuple[dict, list | None]: return policy_table, statements_table @staticmethod - def get_policy_command(client: BotoClient, args: Dict[str, Any]): + def get_policy_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Retrieves the policy for a Lambda function from AWS and parses it into a dictionary. @@ -7666,7 +7670,7 @@ def get_policy_command(client: BotoClient, args: Dict[str, Any]): response = client.get_policy(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) fixed_response = {} fixed_response["AccountId"] = args.get("account_id", "") @@ -7746,7 +7750,7 @@ def invoke_command(client: BotoClient, args: Dict[str, Any]): ) @staticmethod - def get_function_command(client: BotoClient, args: Dict[str, Any]): + def get_function_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Retrieves information about a Lambda function including configuration, code location, and metadata. @@ -7771,7 +7775,7 @@ def get_function_command(client: BotoClient, args: Dict[str, Any]): response = client.get_function(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response with datetime encoding response = serialize_response_with_datetime_encoding(response) @@ -7803,7 +7807,7 @@ def get_function_command(client: BotoClient, args: Dict[str, Any]): ) @staticmethod - def list_functions_command(client: BotoClient, args: Dict[str, Any]): + def list_functions_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Lists Lambda functions in the specified region. @@ -7829,7 +7833,7 @@ def list_functions_command(client: BotoClient, args: Dict[str, Any]): response = client.list_functions(**pagination_kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response with datetime encoding serialized_response = serialize_response_with_datetime_encoding(response) @@ -7864,7 +7868,7 @@ def list_functions_command(client: BotoClient, args: Dict[str, Any]): ) @staticmethod - def list_aliases_command(client: BotoClient, args: Dict[str, Any]): + def list_aliases_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Lists aliases for a Lambda function. @@ -7896,7 +7900,7 @@ def list_aliases_command(client: BotoClient, args: Dict[str, Any]): response = client.list_aliases(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response with datetime encoding serialized_response = serialize_response_with_datetime_encoding(response) @@ -7928,7 +7932,7 @@ def list_aliases_command(client: BotoClient, args: Dict[str, Any]): ) @staticmethod - def get_account_settings_command(client: BotoClient, args: Dict[str, Any]): + def get_account_settings_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Retrieves account settings for AWS Lambda. @@ -7946,7 +7950,7 @@ def get_account_settings_command(client: BotoClient, args: Dict[str, Any]): response = client.get_account_settings() if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response with datetime encoding serialized_response = serialize_response_with_datetime_encoding(response) @@ -7993,7 +7997,7 @@ def get_account_settings_command(client: BotoClient, args: Dict[str, Any]): ) @staticmethod - def list_versions_by_function_command(client: BotoClient, args: Dict[str, Any]): + def list_versions_by_function_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Lists the versions of a Lambda function and returns the results. @@ -8022,7 +8026,7 @@ def list_versions_by_function_command(client: BotoClient, args: Dict[str, Any]): response = client.list_versions_by_function(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response with datetime encoding serialized_response = serialize_response_with_datetime_encoding(response) @@ -8090,7 +8094,7 @@ def delete_function_url_config_command(client: BotoClient, args: Dict[str, Any]) return AWSErrorHandler.handle_response_error(response) @staticmethod - def create_function_command(client: BotoClient, args: Dict[str, Any]): + def create_function_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates a Lambda function from AWS. @@ -8126,7 +8130,7 @@ def create_function_command(client: BotoClient, args: Dict[str, Any]): response = client.create_function(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.CREATED: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response with datetime encoding response = serialize_response_with_datetime_encoding(response) @@ -8156,7 +8160,7 @@ def create_function_command(client: BotoClient, args: Dict[str, Any]): ) @staticmethod - def list_layer_versions_command(client: BotoClient, args: Dict[str, Any]): + def list_layer_versions_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Lists the versions of an Lambda layer. @@ -8193,7 +8197,7 @@ def list_layer_versions_command(client: BotoClient, args: Dict[str, Any]): response = client.list_layer_versions(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response with datetime encoding serialized_response = serialize_response_with_datetime_encoding(response) @@ -8224,7 +8228,7 @@ def list_layer_versions_command(client: BotoClient, args: Dict[str, Any]): ) @staticmethod - def delete_function_command(client: BotoClient, args: Dict[str, Any]): + def delete_function_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes a Lambda function from AWS. @@ -8252,7 +8256,7 @@ def delete_function_command(client: BotoClient, args: Dict[str, Any]): return AWSErrorHandler.handle_response_error(response) @staticmethod - def delete_layer_version_command(client: BotoClient, args: Dict[str, Any]): + def delete_layer_version_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Deletes a version of a Lambda layer. @@ -8279,7 +8283,7 @@ def delete_layer_version_command(client: BotoClient, args: Dict[str, Any]): return AWSErrorHandler.handle_response_error(response) @staticmethod - def publish_layer_version_command(client: BotoClient, args: Dict[str, Any]): + def publish_layer_version_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Creates a Lambda layer from a ZIP archive. @@ -8334,7 +8338,7 @@ def publish_layer_version_command(client: BotoClient, args: Dict[str, Any]): response = client.publish_layer_version(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") not in [HTTPStatus.OK, HTTPStatus.CREATED]: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) # Serialize response with datetime encoding outputs = serialize_response_with_datetime_encoding(response) @@ -8366,7 +8370,7 @@ def publish_layer_version_command(client: BotoClient, args: Dict[str, Any]): ) @staticmethod - def update_function_configuration_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults: + def update_function_configuration_command(client: BotoClient, args: Dict[str, Any]) -> CommandResults | None: """ Updates the configuration of a Lambda function. @@ -8385,7 +8389,7 @@ def update_function_configuration_command(client: BotoClient, args: Dict[str, An response = client.update_function_configuration(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) outputs = copy.deepcopy(response) if outputs.get("ResponseMetadata", {}): @@ -9352,7 +9356,7 @@ def command_list_command(client: BotoClient, args: Dict[str, Any]) -> CommandRes response = client.list_commands(**kwargs) if response.get("ResponseMetadata", {}).get("HTTPStatusCode") != HTTPStatus.OK: - AWSErrorHandler.handle_response_error(response, args.get("account_id")) + return AWSErrorHandler.handle_response_error(response, args.get("account_id")) response = serialize_response_with_datetime_encoding(response) print_debug_logs(client, f"Retry {response=}") diff --git a/Packs/AWS/Integrations/AWS/AWS.yml b/Packs/AWS/Integrations/AWS/AWS.yml index 232538080044..2b412f2a21df 100644 --- a/Packs/AWS/Integrations/AWS/AWS.yml +++ b/Packs/AWS/Integrations/AWS/AWS.yml @@ -1041,17 +1041,17 @@ script: description: The bucket name for which to get the website configuration. required: true outputs: - - contextPath: AWS.S3-Buckets.BucketWebsite.ErrorDocument + - contextPath: AWS.S3.Buckets.BucketWebsite.ErrorDocument description: The object key name of the website error document to use for 4XX class errors. type: Object - - contextPath: AWS.S3-Buckets.BucketWebsite.IndexDocument + - contextPath: AWS.S3.Buckets.BucketWebsite.IndexDocument description: The name of the index document for the website (for example index.html). type: Object - - contextPath: AWS.S3-Buckets.BucketWebsite.RedirectAllRequestsTo - description: Specifies the redirect behavior of all requests to a website endpoint of an Amazon S3 bucket. + - contextPath: AWS.S3.Buckets.BucketWebsite.RedirectAllRequestsTo + description: The redirect behavior of all requests to a website endpoint of an Amazon S3 bucket. type: Object - - contextPath: AWS.S3-Buckets.BucketWebsite.RoutingRules - description: Rules that define when a redirect is applied and the redirect behavior. + - contextPath: AWS.S3.Buckets.BucketWebsite.RoutingRules + description: The rules that define when a redirect is applied and the redirect behavior. type: Array - name: aws-s3-bucket-acl-get @@ -1105,11 +1105,11 @@ script: description: Specifies the S3 bucket whose ACL is being requested. required: true outputs: - - contextPath: AWS.S3-Buckets.BucketAcl.Grants - description: A list of grants. + - contextPath: AWS.S3.Buckets.BucketAcl.Grants + description: The list of grants. type: Array - - contextPath: AWS.S3-Buckets.BucketAcl.Owner - description: Container for the bucket owner's display name and ID. + - contextPath: AWS.S3.Buckets.BucketAcl.Owner + description: The container for the bucket owner's display name and ID. type: Object - name: aws-iam-account-password-policy-get @@ -5167,7 +5167,9 @@ script: description: The ID of the AWS account with which the EC2 instance is associated. This key is only present when the parameter "AWS organization accounts" is provided. type: string - - arguments: + - name: aws-ec2-ipam-resource-discoveries-describe + description: "Returns details for IPAM resource discoveries. A resource discovery is an IPAM component that enables IPAM to manage and monitor resources owned by the account. Required IAM permission: ec2:DescribeIpamResourceDiscoveries." + arguments: - description: A comma-separated list of the IPAM resource discovery IDs. name: ipam_resource_discovery_ids - description: One or more filters separated by ';' (for example, name=,values=;name=,values=). See AWS documentation for details & filter options. @@ -5177,12 +5179,13 @@ script: defaultValue: 50 - description: The token for the next set of results. name: next_token - - description: The Amazon Web Services region for the IP address. - name: address_region - description: The AWS account ID. Required for Cortex Platform (which includes Cortex XSIAM version >=3.0 and Cortex Cloud). Optional for Cortex XSOAR and Cortex XSIAM version < 3.0, where it can be retrieved from the integration configuration. name: account_id required: false required:platform: true + - description: The Amazon Web Services region for the IP address. + name: address_region + hidden: true - description: The AWS region. Required for Cortex Platform (which includes Cortex XSIAM version >=3.0 and Cortex Cloud). Optional for Cortex XSOAR and Cortex XSIAM version < 3.0, where it can be retrieved from the integration configuration. name: region required: false @@ -5236,10 +5239,10 @@ script: - contextPath: AWS.EC2.IpamResourceDiscoveries.AccountId description: The ID of the AWS account associated with the EC2 instance. This key is only present when the parameter "AWS organization accounts" is provided. type: string - description: "Returns details for IPAM resource discoveries. A resource discovery is an IPAM component that enables IPAM to manage and monitor resources owned by the account. Required IAM permission: ec2:DescribeIpamResourceDiscoveries." - name: aws-ec2-ipam-resource-discoveries-describe - - arguments: + - name: aws-ec2-ipam-resource-discovery-associations-describe + description: "Returns details for a resource discovery associated with an Amazon VPC IPAM. Required IAM permission: ec2:DescribeIpamResourceDiscoveryAssociations." + arguments: - description: A comma-separated list of the resource discovery association IDs. name: ipam_resource_discovery_association_ids - description: One or more filters separated by ';' (for example, name=,values=;name=,values=). See AWS documentation for details & filter options. @@ -5249,12 +5252,13 @@ script: defaultValue: 50 - description: The token for the next set of results. name: next_token - - description: The Amazon Web Services region for the IP address. - name: address_region - description: The AWS account ID. Required for Cortex Platform (which includes Cortex XSIAM version >=3.0 and Cortex Cloud). Optional for Cortex XSOAR and Cortex XSIAM version < 3.0, where it can be retrieved from the integration configuration. name: account_id required: false required:platform: true + - description: The Amazon Web Services region for the IP address. + name: address_region + hidden: true - description: The AWS region. Required for Cortex Platform (which includes Cortex XSIAM version >=3.0 and Cortex Cloud). Optional for Cortex XSOAR and Cortex XSIAM version < 3.0, where it can be retrieved from the integration configuration. name: region required: false @@ -5308,8 +5312,6 @@ script: - contextPath: AWS.EC2.IpamResourceDiscoveryAssociations.AccountId description: The ID of the AWS account associated with the EC2 instance. This key is only present when the parameter "AWS organization accounts" is provided. type: string - description: "Returns details for a resource discovery associated with an Amazon VPC IPAM. Required IAM permission: ec2:DescribeIpamResourceDiscoveryAssociations." - name: aws-ec2-ipam-resource-discovery-associations-describe - name: aws-cloudtrail-trail-update description: 'Updates trail settings for event logging and log file handling. Designates an existing bucket for log delivery without requiring a service restart. Note: This must be executed from the region where the trail was created. Required IAM permission: cloudtrail:UpdateTrail.' @@ -7090,16 +7092,16 @@ script: - contextPath: AWS.CloudTrail.Trails.CloudWatchLogsRoleArn description: The role assumed by CloudTrail to write logs to the CloudWatch log group. type: string - - contextPath: AWS.CloudTrail.KmsKeyId + - contextPath: AWS.CloudTrail.Trails.KmsKeyId description: The KMS key ID that encrypts logs delivered by CloudTrail. type: string - - contextPath: AWS.CloudTrail.HasCustomEventSelectors + - contextPath: AWS.CloudTrail.Trails.HasCustomEventSelectors description: Specifies if the trail has custom event selectors. type: boolean - - contextPath: AWS.CloudTrail.HasInsightSelectors + - contextPath: AWS.CloudTrail.Trails.HasInsightSelectors description: Specifies whether a trail has insight types specified in an InsightSelector list. type: boolean - - contextPath: AWS.CloudTrail.IsOrganizationTrail + - contextPath: AWS.CloudTrail.Trails.IsOrganizationTrail description: Specifies whether the trail is an organization trail. type: boolean - name: aws-s3-bucket-policy-delete @@ -7207,19 +7209,19 @@ script: - description: The account ID of the expected bucket owner. If the account ID that you provide does not match the actual owner of the bucket, the request fails. name: expected_bucket_owner outputs: - - contextPath: AWS.S3-Buckets.BucketName - description: Name of the S3 bucket. + - contextPath: AWS.S3.Buckets.BucketName + description: The name of the S3 bucket. type: string - - contextPath: AWS.S3-Buckets.PublicAccessBlock.BlockPublicAcls + - contextPath: AWS.S3.Buckets.PublicAccessBlock.BlockPublicAcls description: Whether Amazon S3 blocks public access control lists (ACLs) for this bucket and objects in this bucket. type: boolean - - contextPath: AWS.S3-Buckets.PublicAccessBlock.IgnorePublicAcls + - contextPath: AWS.S3.Buckets.PublicAccessBlock.IgnorePublicAcls description: Whether Amazon S3 ignores public ACLs for this bucket and objects in this bucket. type: boolean - - contextPath: AWS.S3-Buckets.PublicAccessBlock.BlockPublicPolicy + - contextPath: AWS.S3.Buckets.PublicAccessBlock.BlockPublicPolicy description: Whether Amazon S3 blocks public bucket policies for this bucket. type: boolean - - contextPath: AWS.S3-Buckets.PublicAccessBlock.RestrictPublicBuckets + - contextPath: AWS.S3.Buckets.PublicAccessBlock.RestrictPublicBuckets description: Whether Amazon S3 restricts public bucket policies for this bucket. type: boolean @@ -8103,11 +8105,11 @@ script: name: bucket required: true outputs: - - contextPath: AWS.S3-Buckets.BucketName - description: Name of the S3 bucket. + - contextPath: AWS.S3.Buckets.BucketName + description: The name of the S3 bucket. type: string - - contextPath: AWS.S3-Buckets.ServerSideEncryptionConfiguration.Rules - description: Container for information about a particular server-side encryption configuration rule. + - contextPath: AWS.S3.Buckets.ServerSideEncryptionConfiguration.Rules + description: The container for information about a particular server-side encryption configuration rule. type: array - name: aws-s3-bucket-policy-get @@ -8163,29 +8165,32 @@ script: - description: The account ID of the expected bucket owner. If the account ID that you provide does not match the actual owner of the bucket, the request fails. name: expected_bucket_owner outputs: - - contextPath: AWS.S3-Buckets.BucketName - description: Name of the S3 bucket. + - contextPath: AWS.S3.Buckets.BucketName + description: The name of the S3 bucket. type: string - - contextPath: AWS.S3-Buckets.Policy.Version + - contextPath: AWS.S3.Buckets.Policy.Version description: The version of the policy. type: string - - contextPath: AWS.S3-Buckets.Policy.Id + - contextPath: AWS.S3.Buckets.Policy.Id description: The id of the policy. type: string - - contextPath: AWS.S3-Buckets.Policy.Statement.Sid - description: Identifier of the policy statement. + - contextPath: AWS.S3.Buckets.Policy.Statement.Sid + description: The identifier of the policy statement. type: string - - contextPath: AWS.S3-Buckets.Policy.Statement.Effect - description: Specifies whether the statement results in an allow or an explicit deny. + - contextPath: AWS.S3.Buckets.Policy.Statement.Effect + description: The specification of whether the statement results in an allow or an explicit deny. type: string - - contextPath: AWS.S3-Buckets.Policy.Statement.Principal - description: Specify the principal that is allowed or denied access to a resource. - - contextPath: AWS.S3-Buckets.Policy.Statement.Action - description: Describes the specific action or actions that will be allowed or denied. - - contextPath: AWS.S3-Buckets.Policy.Statement.Resource - description: Defines the object or objects that the statement applies to. - - contextPath: AWS.S3-Buckets.Policy.Statement.Condition - description: Specify conditions for when a policy is in effect. + - contextPath: AWS.S3.Buckets.Policy.Statement.Principal + description: The principal that is allowed or denied access to a resource. + type: Unknown + - contextPath: AWS.S3.Buckets.Policy.Statement.Action + description: The specific action or actions that will be allowed or denied. + type: Unknown + - contextPath: AWS.S3.Buckets.Policy.Statement.Resource + description: The object or objects that the statement applies to. + type: Unknown + - contextPath: AWS.S3.Buckets.Policy.Statement.Condition + description: The conditions for when a policy is in effect. type: string - name: aws-lambda-function-configuration-get @@ -21114,7 +21119,7 @@ script: script: '-' type: python subtype: python3 - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 supportsquickactions: true fromversion: 5.5.0 source: 'cooc' diff --git a/Packs/AWS/Integrations/AWS/AWS_test.py b/Packs/AWS/Integrations/AWS/AWS_test.py index dbab17933940..e9c95340e51f 100644 --- a/Packs/AWS/Integrations/AWS/AWS_test.py +++ b/Packs/AWS/Integrations/AWS/AWS_test.py @@ -6,6 +6,38 @@ import pytest from CommonServerPython import CommandResults, DemistoException +import ast +import os +import re + +QUICK_ACTION_SUFFIX = "-quick-action" +_INTEGRATION_DIR = os.path.dirname(os.path.abspath(__file__)) +_YML_PATH = os.path.join(_INTEGRATION_DIR, "AWS.yml") +_PY_PATH = os.path.join(_INTEGRATION_DIR, "AWS.py") + +# Platform-standard arguments that are resolved centrally rather than read with +# args.get(...) inside each command handler: +# - region / account_id -> consumed by get_service_client() when building the +# boto3 client and by the multi-account fan-out in execute_aws_command(). +# - limit / next_token -> consumed by the shared build_pagination_kwargs(). +# - polling_timeout / interval_in_seconds / hide_polling_output -> consumed by +# the @polling_function decorator, not by the handler body. +PLATFORM_STANDARD_ARGS = { + "region", + "account_id", + "limit", + "next_token", + "polling_timeout", + "interval_in_seconds", + "hide_polling_output", +} + +# Output roots produced by platform helpers rather than by an explicit +# outputs_prefix in the handler. `File.*` entries come from fileResult(). +PLATFORM_STANDARD_OUTPUT_ROOTS = ("File.",) + +# An ``ast`` node that declares a function (sync or async). +FunctionNode = ast.FunctionDef | ast.AsyncFunctionDef def test_parse_resource_ids_with_valid_input(): @@ -2119,7 +2151,7 @@ def test_s3_get_bucket_policy_command_success(mocker): result = S3.get_bucket_policy_command(mock_client, args) assert isinstance(result, CommandResults) - assert result.outputs_prefix == "AWS.S3-Buckets" + assert result.outputs_prefix == "AWS.S3.Buckets" assert result.outputs_key_field == "BucketName" assert result.outputs["BucketName"] == "test-bucket" assert result.outputs["Policy"] == policy_document @@ -2319,7 +2351,7 @@ def test_s3_get_bucket_encryption_command_success(mocker): result = S3.get_bucket_encryption_command(mock_client, args) assert isinstance(result, CommandResults) - assert result.outputs_prefix == "AWS.S3-Buckets" + assert result.outputs_prefix == "AWS.S3.Buckets" assert result.outputs_key_field == "BucketName" assert result.outputs["BucketName"] == "test-bucket" assert "ServerSideEncryptionConfiguration" in result.outputs @@ -2484,7 +2516,7 @@ def test_s3_get_public_access_block_command_success(mocker): result = S3.get_public_access_block_command(mock_client, args) assert isinstance(result, CommandResults) - assert result.outputs_prefix == "AWS.S3-Buckets" + assert result.outputs_prefix == "AWS.S3.Buckets" assert result.outputs_key_field == "BucketName" assert result.outputs["BucketName"] == "test-bucket" assert result.outputs["PublicAccessBlock"] == public_access_block_config @@ -6198,6 +6230,46 @@ def test_get_bucket_website_command_failure(mocker): mock_error_handler.assert_called_once() +def test_get_bucket_website_command_context_output(mocker): + """ + Given: + - A mocked boto3 S3 client returning a full website configuration + (IndexDocument, ErrorDocument, RedirectAllRequestsTo, RoutingRules). + - A valid bucket name. + When: + - get_bucket_website_command is called. + Then: + - The CommandResults context output prefix is "AWS.S3.Buckets.BucketWebsite". + - The outputs match the expected website configuration exactly. + """ + from AWS import S3 + + mock_client = mocker.Mock() + index_document = {"Suffix": "index.html"} + error_document = {"Key": "error.html"} + redirect_all_requests_to = {"HostName": "example.com", "Protocol": "https"} + routing_rules = [{"Redirect": {"ReplaceKeyPrefixWith": "documents/"}}] + mock_client.get_bucket_website.return_value = { + "ResponseMetadata": {"HTTPStatusCode": HTTPStatus.OK}, + "IndexDocument": index_document, + "ErrorDocument": error_document, + "RedirectAllRequestsTo": redirect_all_requests_to, + "RoutingRules": routing_rules, + } + args = {"bucket": "mock_bucket_name"} + + result = S3.get_bucket_website_command(mock_client, args) + + assert isinstance(result, CommandResults) + assert result.outputs_prefix == "AWS.S3.Buckets.BucketWebsite" + assert result.outputs == { + "ErrorDocument": error_document, + "IndexDocument": index_document, + "RedirectAllRequestsTo": redirect_all_requests_to, + "RoutingRules": routing_rules, + } + + def test_get_bucket_acl_command_success(mocker): """ Given: A mocked boto3 S3 client and a valid bucket name. @@ -6230,6 +6302,41 @@ def test_get_bucket_acl_command_failure(mocker): mock_error_handler.assert_called_once() +def test_get_bucket_acl_command_context_output(mocker): + """ + Given: + - A mocked boto3 S3 client returning a full access control policy (Grants and Owner). + - A valid bucket name. + When: + - get_bucket_acl_command is called. + Then: + - The CommandResults context output prefix is "AWS.S3.Buckets.BucketAcl". + - The outputs match the expected access control policy exactly. + """ + from AWS import S3 + + mock_client = mocker.Mock() + owner = {"DisplayName": "owner-display-name", "ID": "owner-id"} + grants = [ + { + "Grantee": {"Type": "CanonicalUser", "DisplayName": "owner-display-name", "ID": "owner-id"}, + "Permission": "FULL_CONTROL", + } + ] + mock_client.get_bucket_acl.return_value = { + "ResponseMetadata": {"HTTPStatusCode": HTTPStatus.OK}, + "Owner": owner, + "Grants": grants, + } + args = {"bucket": "mock_bucket_name"} + + result = S3.get_bucket_acl_command(mock_client, args) + + assert isinstance(result, CommandResults) + assert result.outputs_prefix == "AWS.S3.Buckets.BucketAcl" + assert result.outputs == {"Grants": grants, "Owner": owner} + + def test_create_network_acl_command_success(mocker): """ Given: A mocked boto3 EC2 client and a valid VPC ID. @@ -22155,3 +22262,438 @@ def test_update_logging_configuration_command_api_error(mocker): NetworkFirewall.update_logging_configuration_command(mock_client, args) mock_error_handler.assert_called_once() + + +# --------------------------------------------------------------------------- +# YML <-> PY wiring assertion tests +# +# These tests read the integration's .yml and .py from disk and assert that +# every command, its arguments, and its output prefixes declared in the YML are +# actually wired up in the Python code. This proves the command, its args, and +# its outputs are implemented as declared. +# +# The comparison is PER-COMMAND: each YML command is resolved to its handler +# function via COMMANDS_MAPPING in AWS.py. Unlike the GCP integration (whose +# handlers are plain module-level functions), AWS handlers are `@staticmethod`s +# on service classes (e.g. ``S3.buckets_list_command``), and they frequently +# delegate argument parsing to shared builders (e.g. +# ``aws_ec2_fleet_create_args_builder(args)``). The analysis is therefore done +# with the ``ast`` module and follows those delegations transitively, so an +# argument read inside a helper still counts as wired. +# +# Quick-action commands (names ending in "-quick-action") and the built-in +# "test-module" command are intentionally excluded. +# +# NOTE: these tests are pure static analysis - they only parse the .yml/.py +# text. They perform no network/API calls, read no environment variables, and +# are independent of execution order and of the system clock. +# --------------------------------------------------------------------------- + + +def _is_included_command(command_name: str) -> bool: + """Return whether the command should be checked by the YML <-> PY assertions. + + Args: + command_name: The command name as declared in the YML / COMMANDS_MAPPING. + + Returns: + True if the command is neither ``test-module`` nor a quick action, False otherwise. + """ + return command_name != "test-module" and not command_name.endswith(QUICK_ACTION_SUFFIX) + + +def _load_yml_spec() -> dict: + """Parse the integration YML and return its command specifications. + + Arguments marked ``hidden: true`` in the YML are excluded: they are not + user-facing (they carry internal polling state or a fixed defaultValue for a + quick action) and are frequently consumed by shared/platform code rather + than read directly in the command handler. + + Returns: + A dict mapping command_name -> {"args": [arg_names], "outputs": [contextPaths]} + for every non-quick-action, non-test-module command. + """ + import yaml + + with open(_YML_PATH, encoding="utf-8") as f: + yml = yaml.safe_load(f) + + spec: dict[str, dict[str, list[str]]] = {} + for command in yml.get("script", {}).get("commands", []): + name = command.get("name", "") + if not _is_included_command(name): + continue + arg_names = [arg["name"] for arg in (command.get("arguments") or []) if arg.get("name") and not arg.get("hidden")] + context_paths = [out["contextPath"] for out in (command.get("outputs") or []) if out.get("contextPath")] + spec[name] = {"args": arg_names, "outputs": context_paths} + return spec + + +def _load_py_tree() -> ast.Module: + """Parse the integration .py file into an AST. + + Returns: + The parsed AWS.py module as an ``ast.Module``. + """ + with open(_PY_PATH, encoding="utf-8") as f: + return ast.parse(f.read()) + + +@pytest.fixture(scope="module") +def yml_spec() -> dict: + """Provide the AWS.yml command specifications, parsed once for the whole module. + + Returns: + The value of :func:`_load_yml_spec`, shared by every test in this module. + """ + return _load_yml_spec() + + +@pytest.fixture(scope="module") +def py_tree() -> ast.Module: + """Provide the AWS.py AST, parsed once for the whole module. + + Returns: + The value of :func:`_load_py_tree`, shared by every test in this module. + """ + return _load_py_tree() + + +def _parse_commands_mapping(tree: ast.Module) -> dict[str, str]: + """Parse ``COMMANDS_MAPPING`` in AWS.py, mapping command name -> handler name. + + ``COMMANDS_MAPPING`` is an annotated assignment (``: dict[str, Callable]``), + so both ``Assign`` and ``AnnAssign`` nodes are considered. + + Args: + tree: The parsed AWS.py module. + + Returns: + A dict mapping command_name -> dotted handler name (e.g. "S3.buckets_list_command"), + excluding test-module and quick-action commands. + """ + mapping: dict[str, str] = {} + for node in ast.walk(tree): + if isinstance(node, ast.Assign): + targets = [t.id for t in node.targets if isinstance(t, ast.Name)] + elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name): + targets = [node.target.id] + else: + continue + if "COMMANDS_MAPPING" not in targets or not isinstance(node.value, ast.Dict): + continue + for key, value in zip(node.value.keys, node.value.values): + if isinstance(key, ast.Constant) and isinstance(key.value, str) and _is_included_command(key.value): + mapping[key.value] = ast.unparse(value) + return mapping + + +def _index_functions(tree: ast.Module) -> tuple: + """Index every function in the module by qualified name and by short name. + + Args: + tree: The parsed AWS.py module. + + Returns: + A ``(qualified_index, short_index)`` tuple of dicts mapping a function name to its + definition node, where qualified names look like "S3.buckets_list_command" and short + names look like "buckets_list_command" (first definition wins on a short-name clash). + """ + qualified: dict[str, FunctionNode] = {} + short: dict[str, FunctionNode] = {} + + def walk(node: ast.AST, prefix: str = "") -> None: + """Recursively index the functions declared under ``node``. + + Args: + node: The AST node whose ``body`` is scanned (module, class, or function). + prefix: The dotted name prefix accumulated from the enclosing scopes. + + Returns: + None. ``qualified`` and ``short`` are populated in place. + """ + for child in node.body: + if isinstance(child, ast.ClassDef): + walk(child, f"{prefix}{child.name}.") + elif isinstance(child, ast.FunctionDef | ast.AsyncFunctionDef): + name = f"{prefix}{child.name}" + qualified[name] = child + short.setdefault(child.name, child) + walk(child, f"{name}.") + + walk(tree) + return qualified, short + + +def _resolve(name: str, qualified: dict, short: dict) -> FunctionNode | None: + """Resolve a (possibly dotted) callee name to its function definition node. + + Args: + name: The callee name as written in the source (e.g. "S3.buckets_list_command"). + qualified: Index of functions by qualified (dotted) name. + short: Index of functions by bare function name. + + Returns: + The matching function definition node, or None if the name is not defined in AWS.py. + """ + return qualified.get(name) or short.get(name.split(".")[-1]) + + +def _camel_to_snake(value: str) -> str: + """Convert a CamelCase string to its snake_case form. + + Mirrors ``CommonServerPython.camel_case_to_underscore``, which AWS.py uses at + runtime to derive argument names from AWS API keys, so consecutive uppercase + letters (acronyms) are kept together instead of being split letter by letter. + + Args: + value: The string to convert (e.g. "KMSKeyId"). + + Returns: + The snake_case form of the string (e.g. "kms_key_id"). + """ + partially_split = re.sub(r"(.)([A-Z][a-z]+)", r"\1_\2", value) + return re.sub(r"([a-z0-9])([A-Z])", r"\1_\2", partially_split).lower() + + +def _string_literals(node: ast.AST) -> set[str]: + """Collect all string literals in a function, plus their snake_case forms. + + Some handlers derive the argument name from a CamelCase AWS API key, e.g. + ``camel_case_to_underscore("Runtime")`` -> ``"runtime"``, so the snake_case + variants are included to avoid false positives. + + Args: + node: The AST node (typically a function definition) to scan. + + Returns: + A set of every string literal found under the node, unioned with the snake_case + form of each of those literals. + """ + literals = {n.value for n in ast.walk(node) if isinstance(n, ast.Constant) and isinstance(n.value, str)} + return literals | {_camel_to_snake(v) for v in literals} + + +def _callees_receiving_args(node: ast.AST) -> list[str]: + """Find the functions a handler calls while forwarding its ``args`` dict. + + Args: + node: The AST node (typically a handler's function definition) to scan. + + Returns: + A list of callee names (as written in the source) that receive ``args`` either + positionally or as a keyword argument. + """ + callees: list[str] = [] + for child in ast.walk(node): + if not isinstance(child, ast.Call): + continue + forwards_args = any(isinstance(a, ast.Name) and a.id == "args" for a in child.args) or any( + isinstance(kw.value, ast.Name) and kw.value.id == "args" for kw in child.keywords + ) + if forwards_args: + callees.append(ast.unparse(child.func)) + return callees + + +def _collect_arg_names( + node: ast.AST, + qualified: dict, + short: dict, + depth: int = 6, + seen: set[int] | None = None, +) -> set[str]: + """Collect argument names read by a handler, following ``args`` delegation. + + Args: + node: The handler function definition to scan. + qualified: Index of functions by qualified (dotted) name. + short: Index of functions by bare function name. + depth: Maximum delegation depth to follow into callees that receive ``args``. + seen: Ids of already-visited function nodes, used to avoid infinite recursion. + + Returns: + A set of candidate argument names (string literals, plus their snake_case forms) + read by the handler and by the helpers it forwards ``args`` to. + """ + seen = seen if seen is not None else set() + found = _string_literals(node) + if depth <= 0: + return found + for callee in _callees_receiving_args(node): + target = _resolve(callee, qualified, short) + if target is not None and id(target) not in seen: + seen.add(id(target)) + found |= _collect_arg_names(target, qualified, short, depth - 1, seen) + return found + + +def _collect_output_prefixes( + node: ast.AST, + qualified: dict, + short: dict, + depth: int = 4, + seen: set[int] | None = None, +) -> set[str]: + """Collect context-path-like literals produced by a handler and its helpers. + + Args: + node: The handler function definition to scan. + qualified: Index of functions by qualified (dotted) name. + short: Index of functions by bare function name. + depth: Maximum call depth to follow into helper functions. + seen: Ids of already-visited function nodes, used to avoid infinite recursion. + + Returns: + A set of dotted string literals (with any trailing ``(...)`` DT filter stripped) + that look like context output prefixes. + """ + seen = seen if seen is not None else set() + found = {re.sub(r"\(.*\)$", "", literal) for literal in _string_literals(node) if "." in literal} + if depth <= 0: + return found + for child in ast.walk(node): + if not isinstance(child, ast.Call): + continue + target = _resolve(ast.unparse(child.func), qualified, short) + if target is not None and id(target) not in seen: + seen.add(id(target)) + found |= _collect_output_prefixes(target, qualified, short, depth - 1, seen) + return found + + +@pytest.mark.parametrize( + "value, expected", + [ + ("Runtime", "runtime"), + ("RuntimeVersion", "runtime_version"), + ("bucketName", "bucket_name"), + ("KMSKeyId", "kms_key_id"), + ("S3Bucket", "s3_bucket"), + ("already_snake", "already_snake"), + ], +) +def test_camel_to_snake(value: str, expected: str) -> None: + """ + Given: A CamelCase string, either a plain one or one containing an acronym. + When: Converting it with _camel_to_snake. + Then: The result matches what CommonServerPython.camel_case_to_underscore produces + at runtime, keeping consecutive uppercase letters together instead of + splitting them letter by letter. + + Args: + value: The string to convert. + expected: The expected snake_case form. + """ + assert _camel_to_snake(value) == expected + + +def test_yml_commands_are_wired_in_py(yml_spec: dict, py_tree: ast.Module) -> None: + """ + Given: The integration YML declaring command names. + When: Comparing against the COMMANDS_MAPPING wired in AWS.py. + Then: Every non-quick-action YML command must be wired in the .py, and every + wired handler must resolve to a real function. + + Args: + yml_spec: The parsed AWS.yml command specifications (module-scoped fixture). + py_tree: The parsed AWS.py AST (module-scoped fixture). + """ + # When + command_map = _parse_commands_mapping(py_tree) + qualified, short = _index_functions(py_tree) + + # Then + missing = sorted(name for name in yml_spec if name not in command_map) + assert not missing, ( + "The following commands are declared in AWS.yml but are NOT wired in " f"COMMANDS_MAPPING in AWS.py: {missing}" + ) + + unresolved = sorted( + f"{command} -> {handler}" for command, handler in command_map.items() if _resolve(handler, qualified, short) is None + ) + assert not unresolved, ( + "The following COMMANDS_MAPPING handlers do not resolve to a function " f"defined in AWS.py: {unresolved}" + ) + + +def test_yml_args_match_py_handler_verbatim(yml_spec: dict, py_tree: ast.Module) -> None: + """ + Given: The arguments declared per command in the integration YML. + When: Comparing (verbatim) against the argument names read in that command's + resolved handler in AWS.py, following delegation into shared builders. + Then: Each YML argument name must appear exactly as-is in the handler. + Any naming difference (snake_case vs camelCase, casing) fails, except + for platform-standard args. + + Args: + yml_spec: The parsed AWS.yml command specifications (module-scoped fixture). + py_tree: The parsed AWS.py AST (module-scoped fixture). + """ + # Given + command_map = _parse_commands_mapping(py_tree) + qualified, short = _index_functions(py_tree) + + # When + mismatches: list[str] = [] + for command_name in sorted(yml_spec): + handler = command_map.get(command_name) + node = _resolve(handler, qualified, short) if handler else None + if node is None: + # Missing wiring is reported by test_yml_commands_are_wired_in_py. + continue + handler_args = _collect_arg_names(node, qualified, short) + for arg_name in yml_spec[command_name]["args"]: + if arg_name in PLATFORM_STANDARD_ARGS: + continue + if arg_name not in handler_args: + mismatches.append(f'{command_name} (handler {handler}) -> args.get("{arg_name}")') + + # Then + assert not mismatches, ( + "The following YML arguments are NOT read verbatim from args in their " + "command's handler in AWS.py (a naming difference such as snake_case vs " + "camelCase means the YML and PY are out of sync):\n" + "\n".join(mismatches) + ) + + +def test_yml_output_prefixes_match_py_handler(yml_spec: dict[str, dict[str, list[str]]], py_tree: ast.Module) -> None: + """ + Given: The output contextPaths declared per command in the integration YML. + When: Comparing against the output prefixes declared in that command's + resolved handler in AWS.py. + Then: Every YML output contextPath must be covered by an output prefix + declared in the handler (the prefix must be a leading segment of the + contextPath), except for platform-produced outputs. + + Args: + yml_spec: The parsed AWS.yml command specifications (module-scoped fixture). + py_tree: The parsed AWS.py AST (module-scoped fixture). + """ + # Given + command_map = _parse_commands_mapping(py_tree) + qualified, short = _index_functions(py_tree) + + def _is_covered(context_path: str, prefixes: set) -> bool: + return any(context_path == prefix or context_path.startswith(prefix + ".") for prefix in prefixes) + + # When + uncovered: list = [] + for command_name in sorted(yml_spec): + handler = command_map.get(command_name) + node = _resolve(handler, qualified, short) if handler else None + if node is None: + continue + handler_prefixes = _collect_output_prefixes(node, qualified, short) + for context_path in yml_spec[command_name]["outputs"]: + if context_path.startswith(PLATFORM_STANDARD_OUTPUT_ROOTS): + continue + if not _is_covered(context_path, handler_prefixes): + uncovered.append(f"{command_name} (handler {handler}) -> {context_path}") + + # Then + assert not uncovered, ( + "The following YML output contextPaths are NOT covered by any output " + "prefix declared in their command's handler in AWS.py:\n" + "\n".join(uncovered) + ) diff --git a/Packs/AWS/Integrations/AWS/README.md b/Packs/AWS/Integrations/AWS/README.md index 2811e2b1f933..48b479652f70 100644 --- a/Packs/AWS/Integrations/AWS/README.md +++ b/Packs/AWS/Integrations/AWS/README.md @@ -297,10 +297,10 @@ Retrieves the website configuration for a bucket. Required IAM permission: s3:Ge | **Path** | **Type** | **Description** | | --- | --- | --- | -| AWS.S3-Buckets.BucketWebsite.ErrorDocument | Object | The object key name of the website error document to use for 4XX class errors. | -| AWS.S3-Buckets.BucketWebsite.IndexDocument | Object | The name of the index document for the website \(for example index.html\). | -| AWS.S3-Buckets.BucketWebsite.RedirectAllRequestsTo | Object | Specifies the redirect behavior of all requests to a website endpoint of an Amazon S3 bucket. | -| AWS.S3-Buckets.BucketWebsite.RoutingRules | Array | Rules that define when a redirect is applied and the redirect behavior. | +| AWS.S3.Buckets.BucketWebsite.ErrorDocument | Object | The object key name of the website error document to use for 4XX class errors. | +| AWS.S3.Buckets.BucketWebsite.IndexDocument | Object | The name of the index document for the website \(for example index.html\). | +| AWS.S3.Buckets.BucketWebsite.RedirectAllRequestsTo | Object | The redirect behavior of all requests to a website endpoint of an Amazon S3 bucket. | +| AWS.S3.Buckets.BucketWebsite.RoutingRules | Array | The rules that define when a redirect is applied and the redirect behavior. | ### aws-s3-bucket-acl-get @@ -323,8 +323,8 @@ Retrieves the access control list (ACL) of a bucket. Required IAM permission: s3 | **Path** | **Type** | **Description** | | --- | --- | --- | -| AWS.S3-Buckets.BucketAcl.Grants | Array | A list of grants. | -| AWS.S3-Buckets.BucketAcl.Owner | Object | Container for the bucket owner's display name and ID. | +| AWS.S3.Buckets.BucketAcl.Grants | Array | The list of grants. | +| AWS.S3.Buckets.BucketAcl.Owner | Object | The container for the bucket owner's display name and ID. | ### aws-iam-account-password-policy-get @@ -2225,10 +2225,10 @@ Retrieves settings for a specific trail or returns information about all trails | AWS.CloudTrail.Trails.LogFileValidationEnabled | boolean | Whether log file validation is enabled. | | AWS.CloudTrail.Trails.CloudWatchLogsLogGroupArn | string | The ARN of the CloudWatch log group to which CloudTrail logs are delivered. | | AWS.CloudTrail.Trails.CloudWatchLogsRoleArn | string | The role assumed by CloudTrail to write logs to the CloudWatch log group. | -| AWS.CloudTrail.KmsKeyId | string | The KMS key ID that encrypts logs delivered by CloudTrail. | -| AWS.CloudTrail.HasCustomEventSelectors | boolean | Specifies if the trail has custom event selectors. | -| AWS.CloudTrail.HasInsightSelectors | boolean | Specifies whether a trail has insight types specified in an InsightSelector list. | -| AWS.CloudTrail.IsOrganizationTrail | boolean | Specifies whether the trail is an organization trail. | +| AWS.CloudTrail.Trails.KmsKeyId | string | The KMS key ID that encrypts logs delivered by CloudTrail. | +| AWS.CloudTrail.Trails.HasCustomEventSelectors | boolean | Specifies if the trail has custom event selectors. | +| AWS.CloudTrail.Trails.HasInsightSelectors | boolean | Specifies whether a trail has insight types specified in an InsightSelector list. | +| AWS.CloudTrail.Trails.IsOrganizationTrail | boolean | Specifies whether the trail is an organization trail. | ### aws-s3-bucket-policy-delete @@ -2273,11 +2273,11 @@ Retrieves the public access block configuration for an Amazon S3 bucket. Shows t | **Path** | **Type** | **Description** | | --- | --- | --- | -| AWS.S3-Buckets.BucketName | string | Name of the S3 bucket. | -| AWS.S3-Buckets.PublicAccessBlock.BlockPublicAcls | boolean | Whether Amazon S3 blocks public access control lists \(ACLs\) for this bucket and objects in this bucket. | -| AWS.S3-Buckets.PublicAccessBlock.IgnorePublicAcls | boolean | Whether Amazon S3 ignores public ACLs for this bucket and objects in this bucket. | -| AWS.S3-Buckets.PublicAccessBlock.BlockPublicPolicy | boolean | Whether Amazon S3 blocks public bucket policies for this bucket. | -| AWS.S3-Buckets.PublicAccessBlock.RestrictPublicBuckets | boolean | Whether Amazon S3 restricts public bucket policies for this bucket. | +| AWS.S3.Buckets.BucketName | string | The name of the S3 bucket. | +| AWS.S3.Buckets.PublicAccessBlock.BlockPublicAcls | boolean | Whether Amazon S3 blocks public access control lists \(ACLs\) for this bucket and objects in this bucket. | +| AWS.S3.Buckets.PublicAccessBlock.IgnorePublicAcls | boolean | Whether Amazon S3 ignores public ACLs for this bucket and objects in this bucket. | +| AWS.S3.Buckets.PublicAccessBlock.BlockPublicPolicy | boolean | Whether Amazon S3 blocks public bucket policies for this bucket. | +| AWS.S3.Buckets.PublicAccessBlock.RestrictPublicBuckets | boolean | Whether Amazon S3 restricts public bucket policies for this bucket. | ### aws-ec2-images-describe @@ -2608,8 +2608,8 @@ Retrieves the default encryption configuration for an Amazon S3 bucket. Shows th | **Path** | **Type** | **Description** | | --- | --- | --- | -| AWS.S3-Buckets.BucketName | string | Name of the S3 bucket. | -| AWS.S3-Buckets.ServerSideEncryptionConfiguration.Rules | array | Container for information about a particular server-side encryption configuration rule. | +| AWS.S3.Buckets.BucketName | string | The name of the S3 bucket. | +| AWS.S3.Buckets.ServerSideEncryptionConfiguration.Rules | array | The container for information about a particular server-side encryption configuration rule. | ### aws-s3-bucket-policy-get @@ -2633,15 +2633,15 @@ Retrieves the bucket policy for an Amazon S3 bucket. Returns the policy document | **Path** | **Type** | **Description** | | --- | --- | --- | -| AWS.S3-Buckets.BucketName | string | Name of the S3 bucket. | -| AWS.S3-Buckets.Policy.Version | string | The version of the policy. | -| AWS.S3-Buckets.Policy.Id | string | The id of the policy. | -| AWS.S3-Buckets.Policy.Statement.Sid | string | Identifier of the policy statement. | -| AWS.S3-Buckets.Policy.Statement.Effect | string | Specifies whether the statement results in an allow or an explicit deny. | -| AWS.S3-Buckets.Policy.Statement.Principal | unknown | Specify the principal that is allowed or denied access to a resource. | -| AWS.S3-Buckets.Policy.Statement.Action | unknown | Describes the specific action or actions that will be allowed or denied. | -| AWS.S3-Buckets.Policy.Statement.Resource | unknown | Defines the object or objects that the statement applies to. | -| AWS.S3-Buckets.Policy.Statement.Condition | string | Specify conditions for when a policy is in effect. | +| AWS.S3.Buckets.BucketName | string | The name of the S3 bucket. | +| AWS.S3.Buckets.Policy.Version | string | The version of the policy. | +| AWS.S3.Buckets.Policy.Id | string | The id of the policy. | +| AWS.S3.Buckets.Policy.Statement.Sid | string | The identifier of the policy statement. | +| AWS.S3.Buckets.Policy.Statement.Effect | string | The specification of whether the statement results in an allow or an explicit deny. | +| AWS.S3.Buckets.Policy.Statement.Principal | Unknown | The principal that is allowed or denied access to a resource. | +| AWS.S3.Buckets.Policy.Statement.Action | Unknown | The specific action or actions that will be allowed or denied. | +| AWS.S3.Buckets.Policy.Statement.Resource | Unknown | The object or objects that the statement applies to. | +| AWS.S3.Buckets.Policy.Statement.Condition | string | The conditions for when a policy is in effect. | ### aws-lambda-function-configuration-get diff --git a/Packs/AWS/ReleaseNotes/2_10_0.md b/Packs/AWS/ReleaseNotes/2_10_0.md new file mode 100644 index 000000000000..2cb5875afbdb --- /dev/null +++ b/Packs/AWS/ReleaseNotes/2_10_0.md @@ -0,0 +1,3 @@ +## AWS + +- Documentation and metadata improvements. diff --git a/Packs/AWS/ReleaseNotes/2_11_0.md b/Packs/AWS/ReleaseNotes/2_11_0.md new file mode 100644 index 000000000000..d02b62f9e109 --- /dev/null +++ b/Packs/AWS/ReleaseNotes/2_11_0.md @@ -0,0 +1,6 @@ +## AWS + +- Documentation and metadata improvements. + diff --git a/Packs/AWS/ReleaseNotes/2_7_0.json b/Packs/AWS/ReleaseNotes/2_7_0.json new file mode 100644 index 000000000000..214438515893 --- /dev/null +++ b/Packs/AWS/ReleaseNotes/2_7_0.json @@ -0,0 +1,4 @@ +{ + "breakingChanges": true, + "breakingChangesNotes": "The context outputs paths of the following commands were updated: aws-s3-bucket-website-get, aws-s3-bucket-acl-get, aws-s3-public-access-block-get, aws-s3-bucket-encryption-get, aws-s3-bucket-policy-get." +} \ No newline at end of file diff --git a/Packs/AWS/ReleaseNotes/2_7_0.md b/Packs/AWS/ReleaseNotes/2_7_0.md new file mode 100644 index 000000000000..4de8758527ee --- /dev/null +++ b/Packs/AWS/ReleaseNotes/2_7_0.md @@ -0,0 +1,8 @@ +#### Integrations + +##### Amazon Web Services +- Updated the context output path of the command ***aws-s3-bucket-website-get*** to start with the prefix *AWS.S3.Buckets*. +- Updated the context output path of the command ***aws-s3-bucket-acl-get*** to start with the prefix *AWS.S3.Buckets*. +- Updated the context output path of the command ***aws-s3-public-access-block-get*** to start with the prefix *AWS.S3.Buckets*. +- Updated the context output path of the command ***aws-s3-bucket-encryption-get*** to start with the prefix *AWS.S3.Buckets*. +- Updated the context output path of the command ***aws-s3-bucket-policy-get*** to start with the prefix *AWS.S3.Buckets*. \ No newline at end of file diff --git a/Packs/AWS/ReleaseNotes/2_8_0.json b/Packs/AWS/ReleaseNotes/2_8_0.json new file mode 100644 index 000000000000..1ce22454000e --- /dev/null +++ b/Packs/AWS/ReleaseNotes/2_8_0.json @@ -0,0 +1,4 @@ +{ + "breakingChanges": true, + "breakingChangesNotes": "The context outputs paths of the following command was updated: aws-cloudtrail-trails-describe. Now all of the command outputs have the same prefix AWS.CloudTrail.Trails." +} \ No newline at end of file diff --git a/Packs/AWS/ReleaseNotes/2_8_0.md b/Packs/AWS/ReleaseNotes/2_8_0.md new file mode 100644 index 000000000000..298c41732976 --- /dev/null +++ b/Packs/AWS/ReleaseNotes/2_8_0.md @@ -0,0 +1,9 @@ +#### Integrations + +##### Amazon Web Services +- Fixed an issue where the arguments **ebs_card_index** and **ebs_initialization_rate** in the command ***aws-ec2-launch-template-create*** weren't forwarded to AWS. +- Fixed an issue where the argument *acl* in the command ***aws-s3-bucket-create*** wasn't forwarded to AWS. +- Fixed an issue where the command ***aws-ec2-fleet-instances-describe*** was lacking an output prefix. +- Updated the ***aws-ec2-ipam-resource-discoveries-describe*** command by removing the irrelevant argument *address_region*. +- Updated the ***aws-ec2-ipam-resource-discovery-associations-describe*** command by removing the irrelevant argument *address_region*. +- Updated the ***aws-cloudtrail-trails-describe*** command context outputs so all of them will have the prefix *AWS.CloudTrail.Trails*. diff --git a/Packs/AWS/ReleaseNotes/2_9_0.md b/Packs/AWS/ReleaseNotes/2_9_0.md new file mode 100644 index 000000000000..f4ef641567c9 --- /dev/null +++ b/Packs/AWS/ReleaseNotes/2_9_0.md @@ -0,0 +1,6 @@ +<~PLATFORM> +#### Integrations + +##### Amazon Web Services +Fixed an issue where the *account_id* was processed as an integer instead of a string. + \ No newline at end of file diff --git a/Packs/AWS/ReleaseNotes/2_9_1.md b/Packs/AWS/ReleaseNotes/2_9_1.md new file mode 100644 index 000000000000..7182246e49fb --- /dev/null +++ b/Packs/AWS/ReleaseNotes/2_9_1.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### Amazon Web Services + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS/pack_metadata.json b/Packs/AWS/pack_metadata.json index 970e60b67c00..17f3907b2f24 100644 --- a/Packs/AWS/pack_metadata.json +++ b/Packs/AWS/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS", "description": "Amazon Web Services", "support": "xsoar", - "currentVersion": "2.6.0", + "currentVersion": "2.11.0", "author": "Cortex", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -25,6 +25,5 @@ "marketplacev2", "platform" ], - "source": "cooc", - "managed": true + "derivedSource": "cooc" } \ No newline at end of file diff --git a/Packs/AWS_SystemManager/Integrations/AWSSystemManager/AWSSystemManager.yml b/Packs/AWS_SystemManager/Integrations/AWSSystemManager/AWSSystemManager.yml index 9b4d5bec2685..11715b6f7952 100644 --- a/Packs/AWS_SystemManager/Integrations/AWSSystemManager/AWSSystemManager.yml +++ b/Packs/AWS_SystemManager/Integrations/AWSSystemManager/AWSSystemManager.yml @@ -2398,7 +2398,7 @@ script: script: '-' type: python subtype: python3 - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 fromversion: 6.9.0 marketplaces: - xsoar diff --git a/Packs/AWS_SystemManager/ReleaseNotes/1_0_25.md b/Packs/AWS_SystemManager/ReleaseNotes/1_0_25.md new file mode 100644 index 000000000000..935e22b3f6e9 --- /dev/null +++ b/Packs/AWS_SystemManager/ReleaseNotes/1_0_25.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS - System Manager + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS_SystemManager/ReleaseNotes/1_0_26.md b/Packs/AWS_SystemManager/ReleaseNotes/1_0_26.md new file mode 100644 index 000000000000..dfdf78025600 --- /dev/null +++ b/Packs/AWS_SystemManager/ReleaseNotes/1_0_26.md @@ -0,0 +1,7 @@ + +#### Scripts + +##### AWSPackageUpgrade + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + diff --git a/Packs/AWS_SystemManager/ReleaseNotes/1_0_27.md b/Packs/AWS_SystemManager/ReleaseNotes/1_0_27.md new file mode 100644 index 000000000000..80e6e8b4ba88 --- /dev/null +++ b/Packs/AWS_SystemManager/ReleaseNotes/1_0_27.md @@ -0,0 +1,3 @@ +## AWS Systems Manager + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AWS_SystemManager/Scripts/AWSPackageUpgrade/AWSPackageUpgrade.yml b/Packs/AWS_SystemManager/Scripts/AWSPackageUpgrade/AWSPackageUpgrade.yml index 704bcdab664a..7c7a0b29908f 100644 --- a/Packs/AWS_SystemManager/Scripts/AWSPackageUpgrade/AWSPackageUpgrade.yml +++ b/Packs/AWS_SystemManager/Scripts/AWSPackageUpgrade/AWSPackageUpgrade.yml @@ -36,7 +36,7 @@ comment: This is an AWS script that upgrades a package on the AWS EC2 instance u commonfields: id: AWSPackageUpgrade version: -1 -dockerimage: demisto/python3:3.12.13.10116658 +dockerimage: demisto/python3:3.12.14.13053055 enabled: true engineinfo: {} name: AWSPackageUpgrade diff --git a/Packs/AWS_SystemManager/pack_metadata.json b/Packs/AWS_SystemManager/pack_metadata.json index d788fd6fab58..1bcbc15ec7ee 100644 --- a/Packs/AWS_SystemManager/pack_metadata.json +++ b/Packs/AWS_SystemManager/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS Systems Manager", "description": "AWS Systems Manager is the operations hub for your AWS applications and resources and a secure end-to-end management solution for hybrid cloud environments that enables safe and secure operations at scale.", "support": "xsoar", - "currentVersion": "1.0.24", + "currentVersion": "1.0.27", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF.py b/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF.py index b1dc2a807c48..84db00b44c11 100644 --- a/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF.py +++ b/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF.py @@ -1009,8 +1009,8 @@ def main() -> None: # pragma: no cover aws_role_arn = params.get("roleArn") aws_role_session_name = params.get("roleSessionName") aws_role_session_duration = params.get("sessionDuration") - aws_access_key_id = params.get("access_key", {}).get("password") or params.get("access_key") - aws_secret_access_key = params.get("secret_key", {}).get("password") or params.get("secret_key") + aws_access_key_id = params.get("access_key", {}).get("password") + aws_secret_access_key = params.get("secret_key", {}).get("password") verify_certificate = not params.get("insecure", True) timeout = params.get("timeout") or 1 retries = params.get("retries") or 5 diff --git a/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF.yml b/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF.yml index af944676b42c..8cfbe7dabca8 100644 --- a/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF.yml +++ b/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF.yml @@ -1746,7 +1746,7 @@ script: script: '-' type: python subtype: python3 - dockerimage: demisto/boto3py3:1.0.0.10221838 + dockerimage: demisto/boto3py3:1.0.0.12887184 fromversion: 6.5.0 tests: - No tests (auto formatted) diff --git a/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF_test.py b/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF_test.py index b3015e0653f6..6c4f5cb8ba92 100644 --- a/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF_test.py +++ b/Packs/AWS_WAF/Integrations/AWSWAF/AWSWAF_test.py @@ -1,6 +1,7 @@ import json import pytest +import AWSWAF from AWSWAF import BYTE_MATCH_STATEMENT, OPERATOR_TO_STATEMENT_OPERATOR, REGEX_MATCH_STATEMENT from CommonServerPython import * @@ -648,3 +649,80 @@ def test_delete_rule_group_command(mocker): delete_rule_group_mock = mocker.patch.object(client, "delete_rule_group") delete_rule_group_command(client=client, args=delete_rule_group_args) delete_rule_group_mock.assert_called_with(Name="name", Id="id", Scope="REGIONAL", LockToken="lockToken") + + +def build_credentials_param(password: str) -> dict: + """Builds a type-9 credentials parameter as sent by the platform.""" + return { + "credential": "", + "credentials": { + "cacheVersn": 0, + "id": "", + "locked": False, + "modified": "0001-01-01T00:00:00Z", + "name": "", + "password": "", + "sizeInBytes": 0, + "sshkey": "", + "sshkeyPass": "", + "user": "", + "vaultInstanceId": "", + "version": 0, + "workgroup": "", + }, + "identifier": "", + "password": password, + "passwordChanged": False, + } + + +@pytest.mark.parametrize( + "access_key_password, secret_key_password", + [ + pytest.param("", "", id="empty credentials"), + pytest.param("test_access_key", "test_secret_key", id="populated credentials"), + ], +) +def test_main_credentials_params_retrieval(mocker, access_key_password, secret_key_password): + """ + Given: + - Integration params where access_key and secret_key are credentials objects, + either left empty or populated with a password. + When: + - Running main() with the test-module command. + Then: + - The access key and secret key passed to validate_params and AWSClient are the + password strings, and never the raw credentials dict. + """ + + # Given + params = { + "defaultRegion": "us-east-1", + "roleArn": "arn:aws:iam::123456789012:role/test", + "roleSessionName": "test_session", + "access_key": build_credentials_param(access_key_password), + "secret_key": build_credentials_param(secret_key_password), + } + mocker.patch.object(demisto, "params", return_value=params) + mocker.patch.object(demisto, "args", return_value={}) + mocker.patch.object(demisto, "command", return_value="test-module") + validate_params_mock = mocker.patch.object(AWSWAF, "validate_params") + aws_client_mock = mocker.patch.object(AWSWAF, "AWSClient") + mocker.patch.object(AWSWAF, "connection_test", return_value="ok") + return_results_mock = mocker.patch.object(AWSWAF, "return_results") + return_error_mock = mocker.patch.object(AWSWAF, "return_error") + + # When + AWSWAF.main() + + # Then + return_error_mock.assert_not_called() + return_results_mock.assert_any_call("ok") + validate_params_mock.assert_called_once_with( + "us-east-1", "arn:aws:iam::123456789012:role/test", "test_session", access_key_password, secret_key_password + ) + client_kwargs = aws_client_mock.call_args.kwargs + assert client_kwargs["aws_access_key_id"] == access_key_password + assert client_kwargs["aws_secret_access_key"] == secret_key_password + assert not isinstance(client_kwargs["aws_access_key_id"], dict) + assert not isinstance(client_kwargs["aws_secret_access_key"], dict) diff --git a/Packs/AWS_WAF/ReleaseNotes/1_0_23.md b/Packs/AWS_WAF/ReleaseNotes/1_0_23.md new file mode 100644 index 000000000000..0248a8b1f83f --- /dev/null +++ b/Packs/AWS_WAF/ReleaseNotes/1_0_23.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### AWS-WAF + +- Updated the Docker image to: *demisto/boto3py3:1.0.0.12887184*. + diff --git a/Packs/AWS_WAF/ReleaseNotes/1_0_24.md b/Packs/AWS_WAF/ReleaseNotes/1_0_24.md new file mode 100644 index 000000000000..683e750c93c8 --- /dev/null +++ b/Packs/AWS_WAF/ReleaseNotes/1_0_24.md @@ -0,0 +1,5 @@ + +#### Integrations + +##### AWS-WAF +Fixed an issue where the *Access Key* and *Secret Key* were incorrectly populated when left empty. \ No newline at end of file diff --git a/Packs/AWS_WAF/pack_metadata.json b/Packs/AWS_WAF/pack_metadata.json index 1d5a531775b5..d041a9d74940 100644 --- a/Packs/AWS_WAF/pack_metadata.json +++ b/Packs/AWS_WAF/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AWS WAF", "description": "Amazon Web Services Web Application Firewall (WAF)", "support": "xsoar", - "currentVersion": "1.0.22", + "currentVersion": "1.0.24", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -15,7 +15,7 @@ "useCases": [ "Network Security" ], - "keywords": [], + "keywords": ["AWS"], "marketplaces": [ "xsoar", "marketplacev2", diff --git a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.py b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.py index cd8e121beaa4..ccb6f2030460 100644 --- a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.py +++ b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.py @@ -1,10 +1,17 @@ -import logging +import asyncio +import sys +import threading +import time +from collections.abc import Callable, Generator +from concurrent.futures import ThreadPoolExecutor +from dataclasses import dataclass, field from datetime import datetime, timedelta from typing import Any import demistomock as demisto # noqa: F401 import urllib3 from CommonServerPython import * # noqa: F401 +from ContentClientApiModule import * # noqa: F401 urllib3.disable_warnings() @@ -28,6 +35,26 @@ # 4xx status codes that indicate systemic issues and should NOT be skipped NON_SKIPPABLE_STATUS_CODES = {401, 403, 429} +LAST_RUN_VERSION = 2 +# Each fetch window is listed as one page of this many IDs. A window with more items is halved +# instead of paginated, because list endpoints page by offset over live data and are not sorted by +# the field the window filters on, so paginating can skip an item that changes mid-pagination. +LIST_PAGE_SIZE = 500 +DEFAULT_FETCH_TIME_BUDGET_SECONDS = 150 +DEFAULT_MAX_WINDOW_MINUTES = 1440 +DEFAULT_DETAIL_CONCURRENCY = 4 +# Well under the SOAR API's limit of 300 requests a minute per customer, which other clients share. +DEFAULT_DETAIL_RATE_PER_SECOND = 2.0 +CASE_MODE_CREATED = "created" +CASE_MODE_MODIFIED = "modified" +CASE_FETCH_MODES = {"Last modified time": CASE_MODE_MODIFIED, "Created time": CASE_MODE_CREATED} +MIN_REQUEST_SECONDS = 1.0 +MAX_ITEM_FAILURES = 3 +MAX_CONSECUTIVE_FAILURES = 5 +# 50,000 items in one window. Only a window that can't be halved further, or that's already partly +# emitted, is paginated, so this guards against a page number that never runs out. +MAX_LIST_PAGES = 100 + def _is_skippable_error(e: DemistoException) -> bool: """Check if a DemistoException from an API call is a 4xx error that can be safely skipped. @@ -42,12 +69,12 @@ def _is_skippable_error(e: DemistoException) -> bool: Returns: True if the error can be safely skipped, False otherwise. """ - status_code = None - if e.res is not None and hasattr(e.res, "status_code"): - status_code = e.res.status_code - if status_code is None: - return False - return 400 <= status_code < 500 and status_code not in NON_SKIPPABLE_STATUS_CODES + status_code = _status_code(e) + return status_code is not None and 400 <= status_code < 500 and status_code not in NON_SKIPPABLE_STATUS_CODES + + +def _status_code(e: Exception) -> int | None: + return getattr(getattr(e, "response", None), "status_code", None) def try_str_to_datetime(time: str) -> datetime: @@ -65,17 +92,87 @@ def get_current_datetime() -> datetime: return datetime.utcnow().astimezone(timezone.utc) -class FetchIncidentsError(Exception): - """Raised when there's an error in fetching incidents.""" +class AuthError(Exception): + """Raised on a 401 or 403, which no retry can fix, so the fetch run commits nothing.""" + + +class RateLimitedError(Exception): + """Raised on a 429. The API asks for a ~60s wait, longer than is worth spending in one run.""" + + +class BudgetExhaustedError(Exception): + """Raised when the fetch time budget has run out before or during an HTTP call.""" + + +class Deadline: + """Wall-clock budget shared by every HTTP call in one fetch run. + + XSOAR kills a fetch-incidents run after 3 minutes and discards its results, so the run has to stop + on its own in time to save its progress. + """ + + def __init__(self, seconds: float, clock: Callable[[], float] = time.monotonic): + self._clock = clock + self._expires_at = clock() + seconds + + def remaining(self) -> float: + return self._expires_at - self._clock() + + def expired(self) -> bool: + return self.remaining() < MIN_REQUEST_SECONDS + + def request_timeout(self) -> float: + remaining = self.remaining() + if remaining < MIN_REQUEST_SECONDS: + raise BudgetExhaustedError + return remaining -class Client(BaseClient): +class Client(ContentClient): CASES = "cases" ABUSE_CAMPAIGNS = "abusecampaigns" THREATS = "threats" def __init__(self, server_url, verify, proxy, headers, auth): - super().__init__(base_url=server_url, verify=verify, proxy=proxy, headers=headers, auth=auth, timeout=2400) + super().__init__( + base_url=server_url, + verify=verify, + proxy=proxy, + headers=headers, + auth=auth, + timeout=2400, + # No retries, as in earlier versions; fetch retries a failed call on its next run instead. + retry_policy=RetryPolicy(max_attempts=1), # type: ignore[call-arg] + # Fetch stops one incident type after MAX_CONSECUTIVE_FAILURES. A breaker shared by the + # whole client would also stop the other types, and would count skippable 404s. + circuit_breaker=CircuitBreakerPolicy(failure_threshold=sys.maxsize), # type: ignore[call-arg] + client_name="AbnormalSecurity", + ) + + def limit_rate(self, rate_per_second: float) -> None: + """Caps the requests this client sends, from all threads together, at `rate_per_second`.""" + self._rate_limiter = TokenBucketRateLimiter(RateLimitPolicy(rate_per_second=rate_per_second)) # type: ignore[call-arg] + + def _http_request(self, *args, deadline: Deadline | None = None, **kwargs): + """Fetch passes a `deadline`, which caps the whole call, from waiting for a rate token to + reading the body, at the time left. + + With a deadline, a 401 or 403 raises `AuthError` and a 429 raises `RateLimitedError`. + """ + if deadline is None: + return super()._http_request(*args, **kwargs) + try: + response = asyncio.run(asyncio.wait_for(self._request(*args, **kwargs), deadline.request_timeout())) + except TimeoutError as e: + raise BudgetExhaustedError from e + except ContentClientAuthenticationError as e: + raise AuthError(str(e)) from e + except ContentClientRateLimitError as e: + raise RateLimitedError(str(e)) from e + try: + return response.json() + except ValueError as e: + raise DemistoException(f"The response from {response.url} isn't valid JSON: {e}", e) from e def check_the_status_of_an_action_requested_on_a_case_request(self, case_id, action_id, subtenant): params = assign_params(subtenant) @@ -102,13 +199,13 @@ def download_data_from_threat_log_in_csv_format_request(self, filter_, source, s return response def get_a_list_of_abnormal_cases_identified_by_abnormal_security_request( - self, filter_="", page_size=None, page_number=None, subtenant=None + self, filter_="", page_size=None, page_number=None, subtenant=None, deadline=None ): params = assign_params(filter=filter_, pageSize=page_size, pageNumber=page_number, subtenant=subtenant) headers = self._headers - response = self._http_request("get", "cases", params=params, headers=headers) + response = self._http_request("get", "cases", params=params, headers=headers, deadline=deadline) return response @@ -124,6 +221,7 @@ def get_a_list_of_campaigns_submitted_to_abuse_mailbox_request( reporter=None, attackType=None, threatType=None, + deadline=None, ): params = assign_params( filter=filter_, @@ -140,7 +238,7 @@ def get_a_list_of_campaigns_submitted_to_abuse_mailbox_request( headers = self._headers - response = self._http_request("get", "abusecampaigns", params=params, headers=headers) + response = self._http_request("get", "abusecampaigns", params=params, headers=headers, deadline=deadline) return response @@ -157,6 +255,7 @@ def get_a_list_of_threats_request( topic=None, attackType=None, attackVector=None, + deadline=None, ): params = assign_params( filter=filter_, @@ -174,71 +273,11 @@ def get_a_list_of_threats_request( headers = self._headers - response = self._http_request("get", "threats", params=params, headers=headers) + response = self._http_request("get", "threats", params=params, headers=headers, deadline=deadline) return response - def get_page_number_and_max_iterations(self, max_incidents_to_fetch): - page_size = min(max_incidents_to_fetch, 100) - max_iterations = (max_incidents_to_fetch // page_size) + 1 - return page_size, max_iterations - - def get_paginated_cases_list(self, filter_="", max_incidents_to_fetch=FETCH_LIMIT): - cases_response: dict[str, list[dict]] = {"cases": []} - if max_incidents_to_fetch < 1: - return cases_response - - page_number, current_iteration = 1, 1 - page_size, max_iterations = self.get_page_number_and_max_iterations(max_incidents_to_fetch) - - while page_number is not None: - response = self.get_a_list_of_abnormal_cases_identified_by_abnormal_security_request( - filter_=filter_, page_size=page_size, page_number=page_number - ) - cases_response["cases"].extend(response.get("cases", [])) - page_number = response.get("nextPageNumber", None) - current_iteration += 1 - if current_iteration > max_iterations: - break - return cases_response - - def get_paginated_threats_list(self, filter_="", max_incidents_to_fetch=FETCH_LIMIT): - threats_response: dict[str, list[dict]] = {"threats": []} - if max_incidents_to_fetch < 1: - return threats_response - - page_number, current_iteration = 1, 1 - page_size, max_iterations = self.get_page_number_and_max_iterations(max_incidents_to_fetch) - - while page_number is not None: - response = self.get_a_list_of_threats_request(filter_=filter_, page_size=page_size, page_number=page_number) - threats_response["threats"].extend(response.get("threats", [])) - page_number = response.get("nextPageNumber", None) - current_iteration += 1 - if current_iteration > max_iterations: - break - return threats_response - - def get_paginated_abusecampaigns_list(self, filter_="", max_incidents_to_fetch=FETCH_LIMIT): - campaigns_response: dict[str, list[dict]] = {"campaigns": []} - if max_incidents_to_fetch < 1: - return campaigns_response - - page_number, current_iteration = 1, 1 - page_size, max_iterations = self.get_page_number_and_max_iterations(max_incidents_to_fetch) - - while page_number is not None: - response = self.get_a_list_of_campaigns_submitted_to_abuse_mailbox_request( - filter_=filter_, page_size=page_size, page_number=page_number - ) - campaigns_response["campaigns"].extend(response.get("campaigns", [])) - page_number = response.get("nextPageNumber", None) - current_iteration += 1 - if current_iteration > max_iterations: - break - return campaigns_response - - def get_details_of_a_threat_request(self, threat_id, subtenant=None, page_size=None, page_number=None): + def get_details_of_a_threat_request(self, threat_id, subtenant=None, page_size=None, page_number=None, deadline=None): """ Get details of a specific threat with pagination support. @@ -254,23 +293,23 @@ def get_details_of_a_threat_request(self, threat_id, subtenant=None, page_size=N headers = self._headers params = assign_params(subtenant=subtenant, pageSize=page_size, pageNumber=page_number) - response = self._http_request("get", f"threats/{threat_id}", params=params, headers=headers) + response = self._http_request("get", f"threats/{threat_id}", params=params, headers=headers, deadline=deadline) return response - def get_details_of_an_abnormal_case_request(self, case_id, subtenant=None): + def get_details_of_an_abnormal_case_request(self, case_id, subtenant=None, deadline=None): headers = self._headers params = assign_params(subtenant=subtenant) - response = self._http_request("get", f"cases/{case_id}", params=params, headers=headers) + response = self._http_request("get", f"cases/{case_id}", params=params, headers=headers, deadline=deadline) return response - def get_details_of_an_abuse_mailbox_campaign_request(self, campaign_id, subtenant=None): + def get_details_of_an_abuse_mailbox_campaign_request(self, campaign_id, subtenant=None, deadline=None): headers = self._headers params = assign_params(subtenant=subtenant) - response = self._http_request("get", f"abusecampaigns/{campaign_id}", params=params, headers=headers) + response = self._http_request("get", f"abusecampaigns/{campaign_id}", params=params, headers=headers, deadline=deadline) return response @@ -1306,93 +1345,484 @@ def download_message_eml_command(client, args): return results -def generate_threat_incidents(client, threats, max_page_number, start_datetime, end_datetime): - incidents = [] - for threat in threats: - page_number = 1 - all_messages, all_filtered_messages = [], [] - threat_details = None - try: - while page_number is not None: - threat_details = client.get_details_of_a_threat_request(threat["threatId"], page_number=page_number) - for message in threat_details["messages"]: - all_messages.append(message) - remediation_datetime = try_str_to_datetime(message.get("remediationTimestamp")) - if remediation_datetime and start_datetime <= remediation_datetime <= end_datetime: - all_filtered_messages.append(message) - if remediation_datetime and remediation_datetime < start_datetime: - break - page_number = threat_details.get("nextPageNumber", None) - if page_number is not None and page_number > max_page_number: - break - except DemistoException as e: - if _is_skippable_error(e): - demisto.debug(f"Threat {threat['threatId']} returned a skippable error, skipping: {e}") - continue - raise +def floor_to_second(dt: datetime) -> datetime: + return dt.replace(microsecond=0) - # Skip if we didn't get any threat details (shouldn't happen but defensive) - if threat_details is None: - continue - received_time = "" - threat_details["messages"] = all_filtered_messages or all_messages - if threat_details.get("messages", []): - received_time = threat_details["messages"][0].get("receivedTime") - - incident = { - "dbotMirrorId": str(threat["threatId"]), - "name": "Threat", - "occurred": received_time[:26] if len(received_time) > 26 else received_time, - "details": "Threat", - "rawJSON": json.dumps(threat_details) if threat_details else {}, +def format_timestamp(dt: datetime) -> str: + return dt.strftime(ISO_8601_FORMAT) + + +def parse_timestamp(value: str) -> datetime: + return datetime.strptime(value, ISO_8601_FORMAT).replace(tzinfo=timezone.utc) + + +@dataclass +class FetchWindow: + """Progress through one incident type's time window, persisted in `last_run`. + + The window only moves forward once every item listed in it has become an incident or been + skipped, so a run that stops early is picked up by the next run without losing anything. + """ + + window_start: datetime + window_end: datetime + emitted_ids: list[str] = field(default_factory=list) + failed: dict[str, int] = field(default_factory=dict) + # Which timestamp the Account Takeover case window filters on; None for the other types. + mode: str | None = None + + @classmethod + def new(cls, start: datetime, upper: datetime, max_window: timedelta, mode: str | None = None) -> "FetchWindow": + return cls(window_start=start, window_end=_next_window_end(start, upper, max_window), mode=mode) + + @classmethod + def from_dict(cls, data: dict[str, Any]) -> "FetchWindow": + return cls( + window_start=parse_timestamp(data["window_start"]), + window_end=parse_timestamp(data["window_end"]), + emitted_ids=list(data.get("emitted_ids") or []), + failed=dict(data.get("failed") or {}), + mode=data.get("mode"), + ) + + def to_dict(self) -> dict[str, Any]: + data: dict[str, Any] = { + "window_start": format_timestamp(self.window_start), + "window_end": format_timestamp(self.window_end), + "emitted_ids": self.emitted_ids, + "failed": self.failed, } - incidents.append(incident) - return incidents + if self.mode is not None: + data["mode"] = self.mode + return data + + def is_empty(self) -> bool: + return self.window_end <= self.window_start + + def is_untouched(self) -> bool: + return not self.emitted_ids and not self.failed + + def advance(self, upper: datetime, max_window: timedelta) -> None: + self.window_start = self.window_end + self.window_end = _next_window_end(self.window_start, upper, max_window) + self.emitted_ids = [] + self.failed = {} + + def restart(self, upper: datetime, max_window: timedelta) -> None: + """Re-lists the window from its current start, e.g. after it starts filtering on another field.""" + self.window_end = _next_window_end(self.window_start, upper, max_window) + self.emitted_ids = [] + self.failed = {} + + def extend_if_empty(self, upper: datetime, max_window: timedelta) -> None: + if self.is_empty(): + self.window_end = _next_window_end(self.window_start, upper, max_window) + + +def _next_window_end(start: datetime, upper: datetime, max_window: timedelta) -> datetime: + return max(start, min(start + max_window, upper)) + + +@dataclass(frozen=True) +class FetchTypeSpec: + """How one incident type is listed and turned into incidents.""" + + key: str + label: str + list_key: str + id_key: str + filter_field: str + # `/threats` treats `lte` as exclusive; `/cases` and `/abusecampaigns` treat it as inclusive. + end_inclusive: bool + list_method: str + builder: Callable[..., dict] + # Filter field by `FetchWindow.mode`, for types whose window can filter on more than one field. + mode_filter_fields: dict[str, str] | None = None + + +def build_threat_incident( + client: Client, threat: dict, window: FetchWindow, deadline: Deadline | None = None, max_page_number: int = 8 +) -> dict: + """Builds the incident for one threat, keeping the messages remediated inside the window.""" + page_number: int | None = 1 + all_messages, all_filtered_messages = [], [] + threat_details: dict = {} + while page_number is not None: + threat_details = client.get_details_of_a_threat_request(threat["threatId"], page_number=page_number, deadline=deadline) + for message in threat_details.get("messages", []): + all_messages.append(message) + timestamp = message.get("remediationTimestamp") + if not timestamp: + continue + remediation_datetime = try_str_to_datetime(timestamp) + if window.window_start <= remediation_datetime < window.window_end: + all_filtered_messages.append(message) + if remediation_datetime < window.window_start: + break + page_number = threat_details.get("nextPageNumber") + if page_number is not None and page_number > max_page_number: + break + + received_time = "" + threat_details["messages"] = all_filtered_messages or all_messages + if threat_details["messages"]: + received_time = threat_details["messages"][0].get("receivedTime") or "" + + return { + "dbotMirrorId": str(threat["threatId"]), + "name": "Threat", + "occurred": received_time[:26], + "details": "Threat", + "rawJSON": json.dumps(threat_details), + } + + +def build_abuse_campaign_incident(client: Client, campaign: dict, window: FetchWindow, deadline: Deadline | None = None, **_): + campaign_details = client.get_details_of_an_abuse_mailbox_campaign_request(campaign["campaignId"], deadline=deadline) + first_reported = campaign_details.get("firstReported", "") + return { + "dbotMirrorId": str(campaign.get("campaignId", "")), + "name": "Abuse Campaign", + "occurred": first_reported[:26], + "details": "Abuse Campaign", + "rawJSON": json.dumps(campaign_details), + } + + +def build_account_takeover_case_incident(client: Client, case: dict, window: FetchWindow, deadline: Deadline | None = None, **_): + case_details = client.get_details_of_an_abnormal_case_request(case["caseId"], deadline=deadline) + return { + "dbotMirrorId": str(case["caseId"]), + "name": "Account Takeover Case", + "occurred": case_details["firstObserved"], + "details": case["description"], + "genaiSummary": case_details["genai_summary"], + "rawJSON": json.dumps(case_details), + } + + +THREATS_SPEC = FetchTypeSpec( + key="threats", + label="threats", + list_key="threats", + id_key="threatId", + filter_field="latestTimeRemediated", + end_inclusive=False, + list_method="get_a_list_of_threats_request", + builder=build_threat_incident, +) +ABUSE_CAMPAIGNS_SPEC = FetchTypeSpec( + key="abuse_campaigns", + label="abuse campaigns", + list_key="campaigns", + id_key="campaignId", + filter_field="lastReportedTime", + end_inclusive=True, + list_method="get_a_list_of_campaigns_submitted_to_abuse_mailbox_request", + builder=build_abuse_campaign_incident, +) +ACCOUNT_TAKEOVER_SPEC = FetchTypeSpec( + key="account_takeover", + label="account takeover cases", + list_key="cases", + id_key="caseId", + filter_field="lastModifiedTime", + end_inclusive=True, + list_method="get_a_list_of_abnormal_cases_identified_by_abnormal_security_request", + builder=build_account_takeover_case_incident, + # Filtering on lastModifiedTime re-creates a case as a new incident every time it's modified, + # resolved or reopened; createdTime emits each case once. + mode_filter_fields={CASE_MODE_CREATED: "createdTime", CASE_MODE_MODIFIED: "lastModifiedTime"}, +) +FETCH_TYPE_SPECS = (THREATS_SPEC, ABUSE_CAMPAIGNS_SPEC, ACCOUNT_TAKEOVER_SPEC) + + +def build_list_filter(spec: FetchTypeSpec, window: FetchWindow) -> str: + field_name = spec.filter_field + if spec.mode_filter_fields and window.mode: + field_name = spec.mode_filter_fields[window.mode] + start = format_timestamp(window.window_start) + if spec.end_inclusive: + end = (window.window_end - timedelta(microseconds=1)).strftime(TIME_FORMAT_WITHMS) + else: + end = format_timestamp(window.window_end) + return f"{field_name} gte {start} and {field_name} lte {end}" + +def migrate_last_run( + last_run: dict[str, Any], + first_fetch: datetime, + now: datetime, + polling_lag: timedelta, + max_window: timedelta, + enabled: dict[str, bool], + case_mode: str = CASE_MODE_MODIFIED, +) -> tuple[dict[str, FetchWindow], int]: + """Loads the per-type windows from `last_run`, upgrading the 2.4.9 `{"last_fetch": ...}` format. + + Returns: + The windows by type key, and the rotation offset for the type order. + """ + upper = floor_to_second(now - polling_lag) + windows: dict[str, FetchWindow] = {} + offset = 0 + if last_run.get("version") == LAST_RUN_VERSION: + offset = int(last_run.get("type_order_offset", 0)) + windows = {spec.key: FetchWindow.from_dict(last_run[spec.key]) for spec in FETCH_TYPE_SPECS if spec.key in last_run} + else: + # 2.4.9's next run would have started at `last_fetch - polling_lag`, so starting there leaves no gap. + if last_run.get("last_fetch"): + start = floor_to_second(parse_timestamp(last_run["last_fetch"]) - polling_lag) + else: + start = floor_to_second(first_fetch) + windows = {spec.key: FetchWindow.new(start, upper, max_window) for spec in FETCH_TYPE_SPECS if enabled[spec.key]} + + for spec in FETCH_TYPE_SPECS: + mode = case_mode if spec.mode_filter_fields else None + window = windows.get(spec.key) + if not enabled[spec.key]: + windows.pop(spec.key, None) + elif window is None: + # A newly enabled type starts at the current time, as 2.4.9 did, instead of replaying a backlog. + windows[spec.key] = FetchWindow.new(upper, upper, max_window, mode=mode) + elif mode is not None and (window.mode or CASE_MODE_MODIFIED) != mode: + window.mode = mode + window.restart(upper, max_window) + else: + window.mode = mode + window.extend_if_empty(upper, max_window) + return windows, offset + + +@dataclass +class FetchRun: + """What every incident type shares during one fetch run.""" + + client: Client + deadline: Deadline + upper: datetime + max_window: timedelta + concurrency: int + max_page_number: int + warnings: list[str] = field(default_factory=list) + + +def _list_page(run: FetchRun, spec: FetchTypeSpec, window: FetchWindow, page_number: int) -> dict: + request = getattr(run.client, spec.list_method) + return request( + filter_=build_list_filter(spec, window), page_size=LIST_PAGE_SIZE, page_number=page_number, deadline=run.deadline + ) + + +def list_window_snapshot(run: FetchRun, spec: FetchTypeSpec, window: FetchWindow) -> list[dict]: + """Lists every item in the window, halving the window until it fits in one page. -def generate_abuse_campaign_incidents(client, campaigns): - incidents = [] - for campaign in campaigns: + A window that already has emitted or failed IDs is paginated instead of halved, since halving + it would drop the record of what the next window has already emitted. + """ + response = _list_page(run, spec, window, 1) + while response.get("nextPageNumber") and window.is_untouched() and _span_seconds(window) >= 2: + window.window_end = window.window_start + timedelta(seconds=_span_seconds(window) // 2) + demisto.debug(f"Halved the {spec.label} window to end at {format_timestamp(window.window_end)}") + response = _list_page(run, spec, window, 1) + items = list(response.get(spec.list_key) or []) + next_page = response.get("nextPageNumber") + while next_page: + if next_page > MAX_LIST_PAGES: + raise DemistoException(f"Listing {spec.label} returned more than {MAX_LIST_PAGES} pages") + response = _list_page(run, spec, window, next_page) + items.extend(response.get(spec.list_key) or []) + next_page = response.get("nextPageNumber") + return items + + +def _span_seconds(window: FetchWindow) -> int: + return int((window.window_end - window.window_start).total_seconds()) + + +def _is_retention_error(e: Exception) -> bool: + # The error message includes the response body, which names the retention range. + return _status_code(e) == 400 and "retention" in str(e).lower() + + +def _run_stopping_error(e: Exception, deadline: Deadline) -> Exception | None: + """Returns the error to stop the whole run with, or None if only this item or list call failed.""" + if isinstance(e, AuthError | RateLimitedError | BudgetExhaustedError): + return e + if deadline.expired(): + return BudgetExhaustedError() + return None + + +def build_incidents( + run: FetchRun, spec: FetchTypeSpec, items: list[dict], window: FetchWindow +) -> Generator[tuple[dict, dict | Exception | None], None, None]: + """Yields `(item, incident_or_exception)` in list order, so the output doesn't depend on timing. + + Items not started because an earlier one stopped the run yield None. Workers share one `Client`, + which is safe to use from several threads; only the caller updates the window. + """ + stop = threading.Event() + + def build(item: dict) -> dict | Exception | None: + if stop.is_set(): + return None + if run.deadline.expired(): + stop.set() + return BudgetExhaustedError() try: - campaign_details = client.get_details_of_an_abuse_mailbox_campaign_request(campaign["campaignId"]) - except DemistoException as e: - if _is_skippable_error(e): - demisto.debug(f"Campaign {campaign['campaignId']} returned a skippable error, skipping: {e}") - continue - raise - first_reported = campaign_details.get("firstReported", "") - incident = { - "dbotMirrorId": str(campaign.get("campaignId", "")), - "name": "Abuse Campaign", - "occurred": first_reported[:26] if len(first_reported) > 26 else first_reported, - "details": "Abuse Campaign", - "rawJSON": json.dumps(campaign_details) if campaign_details else {}, - } - incidents.append(incident) - return incidents + return spec.builder(run.client, item, window, deadline=run.deadline, max_page_number=run.max_page_number) + except Exception as e: + if _run_stopping_error(e, run.deadline) is not None: + stop.set() + return e + + executor = ThreadPoolExecutor(max_workers=run.concurrency) + try: + futures = [(item, executor.submit(build, item)) for item in items] + for item, future in futures: + yield item, future.result() + finally: + stop.set() + executor.shutdown(wait=True, cancel_futures=True) + + +def _list_error_reason(run: FetchRun, spec: FetchTypeSpec, window: FetchWindow, e: Exception) -> str | None: + """Handles a failed list call. Returns why the type stops, or None if the window was skipped instead.""" + if spec is THREATS_SPEC and _is_retention_error(e): + run.warnings.append( + f"Skipped {spec.label} from {format_timestamp(window.window_start)} to " + f"{format_timestamp(window.window_end)}: outside the data retention period." + ) + window.advance(run.upper, run.max_window) + return None + if _status_code(e) == 402: + # Starting over at now each run means licensing the tenant later doesn't replay a backlog. + window.window_start = run.upper + window.restart(run.upper, run.max_window) + run.warnings.append(f"Skipped {spec.label}: the tenant isn't licensed for them.") + return "not_licensed" + run.warnings.append(f"Listing {spec.label} failed; will retry next run: {e}") + return "list_error" + +def _build_batch(run: FetchRun, spec: FetchTypeSpec, window: FetchWindow, batch: list[dict], incidents: list[dict]) -> bool: + """Turns `batch` into incidents and records each outcome in the window. -def generate_account_takeover_cases_incidents(client, cases): - incidents = [] - for case in cases: + Returns: + False if the type stopped after too many failures in a row. A run-stopping error is raised + after the incidents other workers already completed are kept. + """ + consecutive_failures = 0 + stop_error: Exception | None = None + results = build_incidents(run, spec, batch, window) + for item, result in results: + item_id = str(item[spec.id_key]) + if isinstance(result, dict): + incidents.append(result) + window.emitted_ids.append(item_id) + window.failed.pop(item_id, None) + consecutive_failures = 0 + continue + if result is None: + continue + stop_error = stop_error or _run_stopping_error(result, run.deadline) + if stop_error is not None: + continue + if isinstance(result, DemistoException) and _is_skippable_error(result): + demisto.debug(f"{spec.label} item {item_id} returned a skippable error, skipping: {result}") + window.emitted_ids.append(item_id) + consecutive_failures = 0 + continue + failures = window.failed.pop(item_id, 0) + 1 + if failures >= MAX_ITEM_FAILURES: + run.warnings.append(f"Skipped {spec.label} item {item_id} after it failed on {failures} runs: {result}") + window.emitted_ids.append(item_id) + else: + demisto.debug(f"{spec.label} item {item_id} failed (attempt {failures}), will retry: {result}") + window.failed[item_id] = failures + consecutive_failures += 1 + if consecutive_failures >= MAX_CONSECUTIVE_FAILURES: + results.close() + run.warnings.append(f"Stopped fetching {spec.label} after {consecutive_failures} failures in a row: {result}") + return False + if stop_error is not None: + raise stop_error + return True + + +def fetch_type(run: FetchRun, spec: FetchTypeSpec, window: FetchWindow, quota: int) -> tuple[list[dict], str]: + """Turns the type's items into incidents, window by window, until something stops it. + + Returns: + The incidents, and why the type stopped: `caught_up`, `max_fetch`, `budget`, `rate_limited`, + `item_errors`, `errors`, `list_error` or `not_licensed`. An `AuthError` propagates. + """ + incidents: list[dict] = [] + try: + while not window.is_empty(): + if len(incidents) >= quota: + return incidents, "max_fetch" + try: + items = list_window_snapshot(run, spec, window) + except Exception as e: + stop_error = _run_stopping_error(e, run.deadline) + if stop_error is not None: + raise stop_error from e + reason = _list_error_reason(run, spec, window, e) + if reason is None: + continue + return incidents, reason + + done = set(window.emitted_ids) + pending = list({str(item[spec.id_key]): item for item in items if str(item[spec.id_key]) not in done}.values()) + batch = pending[: quota - len(incidents)] + if not _build_batch(run, spec, window, batch, incidents): + return incidents, "errors" + if len(batch) < len(pending): + return incidents, "max_fetch" + if window.failed: + return incidents, "item_errors" + window.advance(run.upper, run.max_window) + except BudgetExhaustedError: + return incidents, "budget" + except RateLimitedError as e: + run.warnings.append(f"Rate limited while fetching {spec.label}; will resume next run: {e}") + return incidents, "rate_limited" + return incidents, "caught_up" + + +RUN_STOPPING_REASONS = {"budget", "rate_limited"} + + +def fetch_settings_from_params(params: dict[str, Any]) -> dict[str, Any]: + """Reads the advanced fetch params. XSOAR doesn't add yml defaults to instances created before + the params existed, so an unset param falls back to the code default, which matches the yml + default.""" + + def positive(name: str, default: float, cast: Callable[[Any], float]) -> Any: + value = params.get(name) + if value in (None, ""): + return default try: - case_details = client.get_details_of_an_abnormal_case_request(case["caseId"]) - except DemistoException as e: - if _is_skippable_error(e): - demisto.debug(f"Case {case['caseId']} returned a skippable error, skipping: {e}") - continue - raise - incident = { - "dbotMirrorId": str(case["caseId"]), - "name": "Account Takeover Case", - "occurred": case_details["firstObserved"], - "details": case["description"], - "genaiSummary": case_details["genai_summary"], - "rawJSON": json.dumps(case_details) if case_details else {}, - } - incidents.append(incident) - return incidents + number = cast(value) + except ValueError as e: + raise DemistoException(f"{name} must be a number, got {value}") from e + if number <= 0: + raise DemistoException(f"{name} must be greater than 0, got {value}") + return number + + case_fetch_mode = params.get("case_fetch_mode") or "Last modified time" + if case_fetch_mode not in CASE_FETCH_MODES: + raise DemistoException(f"Unknown case fetch mode: {case_fetch_mode}. Use one of: {', '.join(CASE_FETCH_MODES)}") + return { + "fetch_time_budget": positive("fetch_time_budget", DEFAULT_FETCH_TIME_BUDGET_SECONDS, float), + "max_window_minutes": positive("max_window_minutes", DEFAULT_MAX_WINDOW_MINUTES, int), + "detail_concurrency": positive("detail_concurrency", DEFAULT_DETAIL_CONCURRENCY, int), + "detail_rate_per_second": positive("detail_rate_per_second", DEFAULT_DETAIL_RATE_PER_SECOND, float), + "case_fetch_mode": CASE_FETCH_MODES[case_fetch_mode], + } def fetch_incidents( @@ -1405,75 +1835,67 @@ def fetch_incidents( max_page_number: int = 8, max_incidents_to_fetch: int = FETCH_LIMIT, polling_lag: timedelta = timedelta(minutes=0), -): + fetch_time_budget: float = DEFAULT_FETCH_TIME_BUDGET_SECONDS, + max_window_minutes: int = DEFAULT_MAX_WINDOW_MINUTES, + detail_concurrency: int = DEFAULT_DETAIL_CONCURRENCY, + detail_rate_per_second: float = DEFAULT_DETAIL_RATE_PER_SECOND, + case_fetch_mode: str = CASE_MODE_MODIFIED, + deadline: Deadline | None = None, +) -> tuple[dict[str, Any], list[dict], list[str]]: """ - Fetch incidents from various sources (threats, abuse campaigns, and account takeovers). + Fetch incidents from threats, abuse campaigns and account takeover cases. - Parameters: - - client (Client): Client object to interact with the API. - - last_run (Dict[str, Any]): Dictionary containing details about the last time incidents were fetched. - - first_fetch_time (str): ISO formatted string indicating the first time from which to start fetching incidents. - - max_page_number (int): Maximum number of pages to fetch for incidents. - - max_incidents_to_fetch (int, optional): Maximum number of incidents to fetch. Defaults to FETCH_LIMIT. - - polling_lag (int, optional): Time in minutes to subtract from polling time window for data consistency. Defaults to 0. + Each type keeps its own resumable window in `last_run`, so a run that stops early (at + `max_incidents_to_fetch`, at the time budget, or on errors) loses nothing. A 401 or 403 raises + `AuthError`, and the caller commits nothing. Returns: - - Tuple[Dict[str, str], List[Dict]]: Tuple containing a dictionary with the `last_fetch` time and a list of fetched incidents. + The next `last_run`, the incidents, and warnings to show in the instance's health. """ - try: - last_fetch = last_run.get("last_fetch", first_fetch_time) - last_fetch = datetime.fromisoformat(last_fetch[:-1]).astimezone(timezone.utc) - - current_datetime = get_current_datetime() - start_time = last_fetch + timedelta(milliseconds=1) # Not to overlap with previous polling window - end_time = get_current_datetime() - - if polling_lag is not None: - start_time = start_time - polling_lag - end_time = end_time - polling_lag - - start_timestamp = start_time.strftime(ISO_8601_FORMAT) - end_timestamp = end_time.strftime(ISO_8601_FORMAT) - - all_incidents = [] - current_pending_incidents_to_fetch = max_incidents_to_fetch - threat_incidents, abuse_campaign_incidents, account_takeover_cases_incidents = [], [], [] - - if fetch_threats and current_pending_incidents_to_fetch > 0: - threats_filter = f"latestTimeRemediated gte {start_timestamp} and latestTimeRemediated lte {end_timestamp}" - threats_response = client.get_paginated_threats_list( - filter_=threats_filter, max_incidents_to_fetch=current_pending_incidents_to_fetch - ) - threat_incidents = generate_threat_incidents( - client, threats_response.get("threats", []), max_page_number, start_time, end_time - ) - current_pending_incidents_to_fetch -= len(threat_incidents) - - if fetch_abuse_campaigns and current_pending_incidents_to_fetch > 0: - abuse_campaigns_filter = f"lastReportedTime gte {start_timestamp} and lastReportedTime lte {end_timestamp}" - abuse_campaigns_response = client.get_paginated_abusecampaigns_list( - filter_=abuse_campaigns_filter, max_incidents_to_fetch=current_pending_incidents_to_fetch - ) - abuse_campaign_incidents = generate_abuse_campaign_incidents(client, abuse_campaigns_response.get("campaigns", [])) - current_pending_incidents_to_fetch -= len(abuse_campaign_incidents) - - if fetch_account_takeover_cases and current_pending_incidents_to_fetch > 0: - account_takeover_cases_filter = f"lastModifiedTime gte {start_timestamp} and lastModifiedTime lte {end_timestamp}" - account_takeover_cases_response = client.get_paginated_cases_list( - filter_=account_takeover_cases_filter, max_incidents_to_fetch=current_pending_incidents_to_fetch - ) - account_takeover_cases_incidents = generate_account_takeover_cases_incidents( - client, account_takeover_cases_response.get("cases", []) - ) - - all_incidents = threat_incidents + abuse_campaign_incidents + account_takeover_cases_incidents - except Exception as e: - logging.error(f"Failed fetching incidents: {e}") - raise FetchIncidentsError(f"Error while fetching incidents: {e}") - - next_run = {"last_fetch": current_datetime.strftime(ISO_8601_FORMAT)} - - return next_run, all_incidents[:max_incidents_to_fetch] + deadline = deadline or Deadline(fetch_time_budget) + client.limit_rate(detail_rate_per_second) + now = get_current_datetime() + polling_lag = polling_lag or timedelta(0) + max_window = timedelta(minutes=max_window_minutes) + upper = floor_to_second(now - polling_lag) + enabled = { + THREATS_SPEC.key: bool(fetch_threats), + ABUSE_CAMPAIGNS_SPEC.key: bool(fetch_abuse_campaigns), + ACCOUNT_TAKEOVER_SPEC.key: bool(fetch_account_takeover_cases), + } + first_fetch = arg_to_datetime(first_fetch_time) or now + if first_fetch.tzinfo is None: + first_fetch = first_fetch.replace(tzinfo=timezone.utc) + windows, offset = migrate_last_run(last_run, first_fetch, now, polling_lag, max_window, enabled, case_fetch_mode) + + specs = [spec for spec in FETCH_TYPE_SPECS if enabled[spec.key]] + if specs: + rotation = offset % len(specs) + specs = specs[rotation:] + specs[:rotation] + + run = FetchRun(client, deadline, upper, max_window, detail_concurrency, max_page_number) + incidents: list[dict] = [] + summary: dict[str, Any] = {} + for spec in specs: + quota = max_incidents_to_fetch - len(incidents) + if quota <= 0: + break + type_incidents, reason = fetch_type(run, spec, windows[spec.key], quota) + incidents.extend(type_incidents) + summary[spec.key] = {"emitted": len(type_incidents), "stop_reason": reason} + if reason in RUN_STOPPING_REASONS: + break + demisto.debug(f"AbnormalSecurity fetch summary: {json.dumps(summary)}") + + next_run: dict[str, Any] = { + "version": LAST_RUN_VERSION, + "type_order_offset": offset + 1, + **{key: window.to_dict() for key, window in windows.items()}, + } + starts = [window.window_start for window in windows.values()] + # 2.4.9 only reads `last_fetch`, so writing the oldest window start lets a rollback resume without a gap. + next_run["last_fetch"] = format_timestamp(min(starts) if starts else upper) + return next_run, incidents, run.warnings def test_module(client): @@ -1564,7 +1986,8 @@ def main(): # pragma: nocover first_fetch_time = first_fetch_datetime.strftime(ISO_8601_FORMAT) else: first_fetch_time = datetime.now().strftime(ISO_8601_FORMAT) - next_run, incidents = fetch_incidents( + # An AuthError reaches return_error below, so the instance shows as errored and nothing is committed. + next_run, incidents, warnings = fetch_incidents( client=client, last_run=demisto.getLastRun(), first_fetch_time=first_fetch_time, @@ -1574,9 +1997,12 @@ def main(): # pragma: nocover fetch_account_takeover_cases=fetch_account_takeover_cases, max_page_number=max_page_number, polling_lag=polling_lag_delta, + **fetch_settings_from_params(params), ) demisto.setLastRun(next_run) demisto.incidents(incidents) + if warnings: + demisto.updateModuleHealth("; ".join(warnings)) elif command in commands: return_results(commands[command](client, args)) # type: ignore else: diff --git a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.yml b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.yml index 2a2d9200496e..4154e955fa01 100644 --- a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.yml +++ b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.yml @@ -33,7 +33,7 @@ configuration: required: false section: Collect type: 8 -- additionalinfo: Maximum number of incidents per fetch. The default value is 200. +- additionalinfo: The maximum number of incidents per fetch run, across all incident types. defaultvalue: '200' display: Maximum incidents to fetch. name: max_fetch @@ -87,6 +87,49 @@ configuration: required: false additionalinfo: Maximum number of pages to fetch for incidents section: Collect +- display: Case fetch mode + name: case_fetch_mode + defaultvalue: Last modified time + options: + - Last modified time + - Created time + type: 15 + required: false + advanced: true + additionalinfo: 'The timestamp used to fetch Account Takeover cases. "Last modified time" creates a new incident each time a case is modified, resolved or reopened, as in earlier versions. "Created time" creates one incident per case.' + section: Collect +- display: Fetch time budget (in seconds) + name: fetch_time_budget + defaultvalue: "150" + type: 0 + required: false + advanced: true + additionalinfo: The maximum time a fetch run can spend calling the API before it saves its progress and stops. Keep this below the 3-minute fetch timeout. + section: Collect +- display: Maximum fetch window (in minutes) + name: max_window_minutes + defaultvalue: "1440" + type: 0 + required: false + advanced: true + additionalinfo: The longest time range listed in one request. Windows with too many items to list at once are split automatically. + section: Collect +- display: Concurrent detail requests + name: detail_concurrency + defaultvalue: "4" + type: 0 + required: false + advanced: true + additionalinfo: The number of incident detail requests that run concurrently. + section: Collect +- display: Maximum requests per second + name: detail_rate_per_second + defaultvalue: "2" + type: 0 + required: false + advanced: true + additionalinfo: The maximum number of API requests per second during a fetch run, shared by all concurrent requests. + section: Collect description: Abnormal Security detects the whole spectrum of email attacks, from vendor email compromise and spear-phishing to unwanted email spam and graymail. To stop these advanced attacks, Abnormal leverages the industry’s most advanced behavioral data science to baseline known good behavior and detects anomalies. display: Abnormal Security name: Abnormal Security @@ -1126,7 +1169,7 @@ script: description: Download a message in RFC822 EML format for forensic analysis. For quarantine messages, both quarantine_identity and recipient_mailbox parameters are required. name: abnormal-security-download-message-eml - dockerimage: demisto/python3:3.12.13.10116658 + dockerimage: demisto/fastapi:0.125.0.12689358 script: "" subtype: python3 type: python diff --git a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity_test.py b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity_test.py index ac136363d6ae..b27dcebf2bb8 100644 --- a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity_test.py +++ b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity_test.py @@ -1,6 +1,11 @@ import json +import threading +import time from datetime import datetime, timedelta, UTC +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + import demistomock as demisto +import httpx import pytest from AbnormalSecurity import ( @@ -28,17 +33,25 @@ get_employee_information_command, get_employee_login_information_for_last_30_days_in_csv_format_command, download_data_from_threat_log_in_csv_format_command, - generate_threat_incidents, - generate_abuse_campaign_incidents, - generate_account_takeover_cases_incidents, _is_skippable_error, get_a_list_of_unanalyzed_abuse_mailbox_campaigns_command, fetch_incidents, + build_threat_incident, + build_account_takeover_case_incident, + build_list_filter, + FetchWindow, + Deadline, + AuthError, + fetch_settings_from_params, + THREATS_SPEC, + ABUSE_CAMPAIGNS_SPEC, + ACCOUNT_TAKEOVER_SPEC, ISO_8601_FORMAT, ) from CommonServerPython import DemistoException +from ContentClientApiModule import ContentClient, ContentClientError, TokenBucketRateLimiter from test_data.fixtures import BASE_URL, apikey -from test_data.mock_paginated_response import create_mock_paginator_side_effect, create_mock_detail_side_effect +from test_data.fake_soar_api import FakeSoarApi headers = { @@ -488,676 +501,96 @@ def test_provides_the_analysis_and_timeline_details_of_a_case_command(mocker): assert results.outputs_prefix == "AbnormalSecurity.CaseAnalysis" -def test_fetch_threat_incidents(mocker, mock_get_a_list_of_threats_request): - client = mock_client(mocker, util_load_json("test_data/test_get_details_of_a_threat_page2.json")) - first_fetch_time = datetime.now().strftime(ISO_8601_FORMAT) - _, incidents = fetch_incidents( - client=client, - last_run={"last_fetch": "2023-09-17T14:43:09Z"}, - first_fetch_time=first_fetch_time, - max_incidents_to_fetch=200, - fetch_account_takeover_cases=False, - fetch_abuse_campaigns=False, - fetch_threats=True, - ) - assert len(incidents) == 1 - - -def test_fetch_cases_incidents(mocker, mock_get_a_list_of_abnormal_cases_identified_by_abnormal_security_request): - client = mock_client(mocker, util_load_json("test_data/test_get_details_of_an_abnormal_case.json")) - first_fetch_time = datetime.now().strftime(ISO_8601_FORMAT) - _, incidents = fetch_incidents( - client=client, - last_run={"last_fetch": "2023-09-17T14:43:09Z"}, - first_fetch_time=first_fetch_time, - max_incidents_to_fetch=200, - fetch_account_takeover_cases=True, - fetch_abuse_campaigns=False, - fetch_threats=False, - ) - assert len(incidents) == 1 - assert incidents[0].get("genaiSummary") == "genai_summary" - - -def test_fetch_abuse_campaign_incidents(mocker, mock_get_a_list_of_campaigns_submitted_to_abuse_mailbox_request): - client = mock_client(mocker, util_load_json("test_data/test_get_details_of_abuse_campaign.json")) - first_fetch_time = datetime.now().strftime(ISO_8601_FORMAT) - _, incidents = fetch_incidents( - client=client, - last_run={"last_fetch": "2023-09-17T14:43:09Z"}, - first_fetch_time=first_fetch_time, - max_incidents_to_fetch=200, - fetch_account_takeover_cases=False, - fetch_abuse_campaigns=True, - fetch_threats=False, - ) - assert len(incidents) == 1 +def threat_window(start, end): + return FetchWindow(window_start=start, window_end=end) -def test_get_details_of_a_threat_request_two_pages(mocker): +def test_build_threat_incident_two_pages(mocker): return_val = util_load_json("test_data/test_get_details_of_a_threat.json") return_val["messages"][0]["remediationTimestamp"] = "2023-09-17T15:43:09Z" page_2 = util_load_json("test_data/test_get_details_of_a_threat_page2.json") page_2["messages"][0]["remediationTimestamp"] = "2023-09-17T16:43:09Z" - client = mock_client(mocker, side_effect=[return_val, page_2]) - # Create datetime objects instead of using strings - start_datetime = datetime(2023, 9, 17, 14, 43, 9, tzinfo=UTC) - end_datetime = datetime(2023, 9, 18, 14, 43, 9, tzinfo=UTC) - - incidents = generate_threat_incidents(client, [{"threatId": "asdf097sdf907"}], 2, start_datetime, end_datetime) - assert len(incidents) == 1 - assert len(json.loads(incidents[0].get("rawJSON")).get("messages")) == 2 - - -def test_get_details_of_a_threat_request_single_page(mocker): - return_val = util_load_json("test_data/test_get_details_of_a_threat_page2.json") - return_val["messages"][0]["remediationTimestamp"] = "2023-09-17T15:43:09Z" - client = mock_client(mocker, response=return_val) - # Create datetime objects instead of using strings - start_datetime = datetime(2023, 9, 17, 14, 43, 9, tzinfo=UTC) - end_datetime = datetime(2023, 9, 18, 14, 43, 9, tzinfo=UTC) - - incidents = generate_threat_incidents(client, [{"threatId": "asdf097sdf907"}], 1, start_datetime, end_datetime) - assert len(incidents) == 1 - - -def test_get_details_of_a_threat_request_nanosecond_timestamp(mocker, mock_get_details_of_a_threat_request): - client = mock_client(mocker, response=util_load_json("test_data/test_get_list_of_abnormal_threats.json")) - last_run = {"last_fetch": "2023-09-17T14:43:09Z"} - first_fetch_time = "3 days" - max_incidents = 200 - # Call fetch_incidents with the polling lag - _, incidents = fetch_incidents( - client=client, - last_run=last_run, - first_fetch_time=first_fetch_time, - max_incidents_to_fetch=max_incidents, - fetch_account_takeover_cases=False, - fetch_abuse_campaigns=False, - fetch_threats=True, - ) - assert len(incidents) == 1 - assert incidents[0].get("occurred") == "2023-12-03T19:26:36.123456" - - -def test_polling_lag(mocker, mock_get_details_of_a_threat_request): - """Test that polling lag is correctly applied when fetching incidents.""" - # Mock the client and its get_a_list_of_threats_request method - return_val = util_load_json("test_data/test_get_list_of_abnormal_threats.json") - client = mock_client(mocker, response=return_val) - - # Create a spy on the get_a_list_of_threats_request method to capture its calls - get_threats_spy = mocker.spy(client, "get_a_list_of_threats_request") - - # Define test parameters - last_run = {"last_fetch": "2023-09-17T14:43:09Z"} - first_fetch_time = "3 days" - max_incidents = 200 + window = threat_window(datetime(2023, 9, 17, 14, 43, 9, tzinfo=UTC), datetime(2023, 9, 18, 14, 43, 9, tzinfo=UTC)) - # Set up a 5-minute polling lag - polling_lag = timedelta(minutes=5) + incident = build_threat_incident(client, {"threatId": "asdf097sdf907"}, window, max_page_number=2) - # Calculate expected timestamps - original_timestamp = datetime.fromisoformat(last_run["last_fetch"][:-1]).replace(tzinfo=UTC) + timedelta(milliseconds=1) - adjusted_start_time = original_timestamp - polling_lag - expected_start_time = adjusted_start_time.strftime("%Y-%m-%dT%H:%M:%SZ") + assert len(json.loads(incident["rawJSON"])["messages"]) == 2 - # Mock the get_current_datetime function to return a fixed time - fixed_current_time = datetime(2023, 9, 18, 14, 43, 9, tzinfo=UTC) - mocker.patch("AbnormalSecurity.get_current_datetime", return_value=fixed_current_time) - # Calculate expected end time based on the fixed current time - adjusted_end_time = fixed_current_time - polling_lag - expected_end_time = adjusted_end_time.strftime("%Y-%m-%dT%H:%M:%SZ") - - expected_filter = f"latestTimeRemediated gte {expected_start_time} and latestTimeRemediated lte {expected_end_time}" - - # Call fetch_incidents with the polling lag - _, _ = fetch_incidents( - client=client, - last_run=last_run, - first_fetch_time=first_fetch_time, - max_incidents_to_fetch=max_incidents, - fetch_account_takeover_cases=False, - fetch_abuse_campaigns=False, - fetch_threats=True, - polling_lag=polling_lag, - ) +def test_build_threat_incident_nanosecond_timestamp(mocker): + threat_details = util_load_json("test_data/test_get_details_of_a_threat_page2.json") + threat_details["messages"][0]["remediationTimestamp"] = "2023-09-17T15:43:09Z" + client = mock_client(mocker, response=threat_details) + window = threat_window(datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC), datetime(2023, 9, 18, 0, 0, 0, tzinfo=UTC)) - # Check that the method was called with the expected filter - get_threats_spy.assert_called_once() - call_args = get_threats_spy.call_args[1] + incident = build_threat_incident(client, {"threatId": "asdf097sdf907"}, window, max_page_number=1) - # Assert that the filter matches our expected filter - assert call_args["filter_"] == expected_filter - assert call_args["page_size"] == 100 + assert incident["occurred"] == "2023-12-03T19:26:36.123456" -def test_get_details_of_a_threat_request_time_window_filtering(mocker): - """Test that messages outside the time window are filtered out.""" - # Create a mock response with 3 messages +def test_build_threat_incident_filters_messages_to_window(mocker): + """ + Given a threat whose messages were remediated at the window start, just before its end, at its end, + before it, and one with no remediation timestamp. + When the incident is built. + Then only the messages in [start, end) are kept, and the missing timestamp doesn't raise. + """ + start, end = datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC), datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC) mock_response = { "threatId": "test-threat-id", "messages": [ - { - "threatId": "test-threat-id", - "receivedTime": "2023-09-17T15:00:00Z", - "remediationTimestamp": "2023-09-17T15:30:00Z", # Inside window - }, - { - "threatId": "test-threat-id", - "receivedTime": "2023-09-17T16:00:00Z", - "remediationTimestamp": "2023-09-17T16:30:00Z", # Inside window - }, - { - "threatId": "test-threat-id", - "receivedTime": "2023-09-17T12:00:00Z", - "remediationTimestamp": "2023-09-17T12:30:00Z", # Outside window (before start_time) - }, + {"receivedTime": "2023-09-17T17:00:00Z", "remediationTimestamp": "2023-09-17T17:00:00Z"}, + {"receivedTime": "2023-09-17T16:59:59Z", "remediationTimestamp": "2023-09-17T16:59:59.999999Z"}, + {"receivedTime": "2023-09-17T15:00:00Z"}, + {"receivedTime": "2023-09-17T14:00:00Z", "remediationTimestamp": "2023-09-17T14:00:00Z"}, + {"receivedTime": "2023-09-17T12:00:00Z", "remediationTimestamp": "2023-09-17T12:30:00Z"}, ], } - client = mock_client(mocker, response=mock_response) - # Define time window that includes only the first two messages - start_datetime = datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC) - end_datetime = datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC) - - incidents = generate_threat_incidents(client, [{"threatId": "test-threat-id"}], 1, start_datetime, end_datetime) - - # Verify we get one incident - assert len(incidents) == 1 + incident = build_threat_incident(client, {"threatId": "test-threat-id"}, threat_window(start, end), max_page_number=1) - # Verify the incident contains only the messages within the time window - incident_data = json.loads(incidents[0]["rawJSON"]) - assert len(incident_data["messages"]) == 2 + remediation_times = [m["remediationTimestamp"] for m in json.loads(incident["rawJSON"])["messages"]] + assert remediation_times == ["2023-09-17T16:59:59.999999Z", "2023-09-17T14:00:00Z"] + assert incident["occurred"] == "2023-09-17T16:59:59Z" - # Verify the filtered messages are the ones we expect - remediation_times = [msg["remediationTimestamp"] for msg in incident_data["messages"]] - assert "2023-09-17T15:30:00Z" in remediation_times - assert "2023-09-17T16:30:00Z" in remediation_times - assert "2023-09-17T12:30:00Z" not in remediation_times - -def test_get_details_of_a_threat_request_early_exit(mocker): - """Test that processing stops early when encountering messages outside the time window.""" - # Create mock responses for two pages - # Page 1 with 2 messages (both inside time window) +def test_build_threat_incident_stops_at_messages_before_window(mocker): page_1 = { "threatId": "test-threat-id", "messages": [ - { - "threatId": "test-threat-id", - "receivedTime": "2023-09-17T16:00:00Z", - "remediationTimestamp": "2023-09-17T16:30:00Z", # Inside window (latest) - }, - { - "threatId": "test-threat-id", - "receivedTime": "2023-09-17T15:00:00Z", - "remediationTimestamp": "2023-09-17T15:30:00Z", # Inside window - }, + {"receivedTime": "2023-09-17T16:00:00Z", "remediationTimestamp": "2023-09-17T16:30:00Z"}, + {"receivedTime": "2023-09-17T15:00:00Z", "remediationTimestamp": "2023-09-17T15:30:00Z"}, ], - "nextPageNumber": 2, # Indicate there's a second page + "nextPageNumber": 2, } - - # Page 2 with 2 messages (both outside time window) page_2 = { "threatId": "test-threat-id", "messages": [ - { - "threatId": "test-threat-id", - "receivedTime": "2023-09-17T13:00:00Z", - "remediationTimestamp": "2023-09-17T13:30:00Z", # Outside window - }, - { - "threatId": "test-threat-id", - "receivedTime": "2023-09-17T12:00:00Z", - "remediationTimestamp": "2023-09-17T12:30:00Z", # Outside window (earliest) - }, + {"receivedTime": "2023-09-17T13:00:00Z", "remediationTimestamp": "2023-09-17T13:30:00Z"}, + {"receivedTime": "2023-09-17T12:00:00Z", "remediationTimestamp": "2023-09-17T12:30:00Z"}, ], - "nextPageNumber": None, # No more pages } - - # Create a spy for the get_details_of_a_threat_request method client = mock_client(mocker, side_effect=[page_1, page_2]) get_details_spy = mocker.spy(client, "get_details_of_a_threat_request") + window = threat_window(datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC), datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC)) - # Define time window that includes only the first two messages - start_datetime = datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC) - end_datetime = datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC) - - incidents = generate_threat_incidents(client, [{"threatId": "test-threat-id"}], 3, start_datetime, end_datetime) - - # Verify we get one incident - assert len(incidents) == 1 - - # Verify the incident contains only the messages within the time window - incident_data = json.loads(incidents[0]["rawJSON"]) - assert len(incident_data["messages"]) == 2 - - # Verify the filtered messages are the ones we expect (from page 1 only) - remediation_times = [msg["remediationTimestamp"] for msg in incident_data["messages"]] - assert "2023-09-17T16:30:00Z" in remediation_times - assert "2023-09-17T15:30:00Z" in remediation_times - assert "2023-09-17T13:30:00Z" not in remediation_times - assert "2023-09-17T12:30:00Z" not in remediation_times - - # Verify that get_details_of_a_threat_request was called exactly twice - # (once for page 1, once for page 2 where we encounter messages outside the time window and exit early) - assert get_details_spy.call_count == 2 - - # Verify the calls were made with the correct page numbers - first_call_args = get_details_spy.call_args_list[0][1] - second_call_args = get_details_spy.call_args_list[1][1] - assert first_call_args["page_number"] == 1 - assert second_call_args["page_number"] == 2 - - -def test_pagination_methods_in_fetch_incidents(mocker): - """ - Test that the pagination methods are called correctly from fetch_incidents. - This test verifies: - 1. The methods are called with the correct parameters - 2. The pagination logic is executed as expected - 3. The returned incidents are correctly processed - """ - # Create mock pagination side effects for threats, cases, and campaigns - threat_list_side_effect = create_mock_paginator_side_effect("threat") - case_list_side_effect = create_mock_paginator_side_effect("case") - campaign_list_side_effect = create_mock_paginator_side_effect("campaign") - - # Create mock detail side effects - threat_detail_side_effect = create_mock_detail_side_effect("threat") - case_detail_side_effect = create_mock_detail_side_effect("case") - campaign_detail_side_effect = create_mock_detail_side_effect("campaign") - - # Create client - client = Client(server_url=BASE_URL, verify=False, proxy=False, auth=None, headers=headers) - - # Get threat response samples for the mock - threat_page1 = threat_list_side_effect(page_number=1, page_size=2) - threat_page2 = threat_list_side_effect(page_number=2, page_size=2) - - # Get case response samples for the mock - case_page1 = case_list_side_effect(page_number=1, page_size=2) - case_page2 = case_list_side_effect(page_number=2, page_size=2) - - # Get campaign response samples for the mock - campaign_page1 = campaign_list_side_effect(page_number=1, page_size=2) - campaign_page2 = campaign_list_side_effect(page_number=2, page_size=2) - - # Extract threat IDs for detail responses - for each page we'll get exactly page_size items - threat_ids = [threat["threatId"] for threat in threat_page1.get("threats")[:2] + threat_page2.get("threats")[:2]] - case_ids = [case["caseId"] for case in case_page1.get("cases")[:2] + case_page2.get("cases")[:2]] - campaign_ids = [ - campaign["campaignId"] for campaign in campaign_page1.get("campaigns")[:2] + campaign_page2.get("campaigns")[:2] - ] - - # Combine responses for each type - threats_combined = {"threats": threat_page1.get("threats") + threat_page2.get("threats")} - - cases_combined = {"cases": case_page1.get("cases") + case_page2.get("cases")} - - campaigns_combined = {"campaigns": campaign_page1.get("campaigns") + campaign_page2.get("campaigns")} - - # Set up test parameters - last_run = {"last_fetch": "2023-09-17T14:43:09Z"} - first_fetch_time = "3 days" - max_incidents = 200 - polling_lag = timedelta(minutes=5) - - # Mock the three pagination methods - get_paginated_threats_spy = mocker.patch.object(client, "get_paginated_threats_list", return_value=threats_combined) - - get_paginated_cases_spy = mocker.patch.object(client, "get_paginated_cases_list", return_value=cases_combined) - - get_paginated_campaigns_spy = mocker.patch.object( - client, "get_paginated_abusecampaigns_list", return_value=campaigns_combined - ) - - # Mock the get_details methods to return appropriate data for incident generation - mocker.patch.object( - client, "get_details_of_a_threat_request", side_effect=lambda threat_id, **kwargs: threat_detail_side_effect(threat_id) - ) - - mocker.patch.object( - client, "get_details_of_an_abnormal_case_request", side_effect=lambda case_id, **kwargs: case_detail_side_effect(case_id) - ) - - mocker.patch.object( - client, - "get_details_of_an_abuse_mailbox_campaign_request", - side_effect=lambda campaign_id, **kwargs: campaign_detail_side_effect(campaign_id), - ) - - # Mock the get_current_datetime function to return a fixed time - mocker.patch("AbnormalSecurity.get_current_datetime", return_value=datetime(2023, 9, 18, 14, 43, 9, tzinfo=UTC)) - - # Call fetch_incidents with all three fetch options enabled - next_run, incidents = fetch_incidents( - client=client, - last_run=last_run, - first_fetch_time=first_fetch_time, - max_incidents_to_fetch=max_incidents, - fetch_account_takeover_cases=True, - fetch_abuse_campaigns=True, - fetch_threats=True, - polling_lag=polling_lag, - ) - - # Verify the pagination methods were called with the correct filters - - # 1. Verify threats pagination - get_paginated_threats_spy.assert_called_once() - threats_call_kwargs = get_paginated_threats_spy.call_args.kwargs - - # Verify the filter contains latestTimeRemediated with adjusted time due to polling lag - assert "latestTimeRemediated gte" in threats_call_kwargs["filter_"] - assert "latestTimeRemediated lte" in threats_call_kwargs["filter_"] - assert threats_call_kwargs["max_incidents_to_fetch"] == max_incidents - - # 2. Verify abuse campaigns pagination (this is called next in the code) - get_paginated_campaigns_spy.assert_called_once() - campaigns_call_kwargs = get_paginated_campaigns_spy.call_args.kwargs - - # Verify the filter contains lastReportedTime - assert "lastReportedTime gte" in campaigns_call_kwargs["filter_"] - assert "lastReportedTime lte" in campaigns_call_kwargs["filter_"] - assert campaigns_call_kwargs["max_incidents_to_fetch"] == max_incidents - len(threat_ids) - - # 3. Verify cases pagination (this is called last in the code) - get_paginated_cases_spy.assert_called_once() - cases_call_kwargs = get_paginated_cases_spy.call_args.kwargs - # Verify the filter contains lastModifiedTime - assert "lastModifiedTime gte" in cases_call_kwargs["filter_"] - assert "lastModifiedTime lte" in cases_call_kwargs["filter_"] - assert cases_call_kwargs["max_incidents_to_fetch"] == max_incidents - len(threat_ids) - len(campaign_ids) - - # Verify we got the expected number of incidents - expected_incident_count = len(threat_ids) + len(case_ids) + len(campaign_ids) - assert len(incidents) == expected_incident_count - - # Verify the types of incidents - threat_incidents = [i for i in incidents if i.get("name") == "Threat"] - case_incidents = [i for i in incidents if i.get("name") == "Account Takeover Case"] - campaign_incidents = [i for i in incidents if i.get("name") == "Abuse Campaign"] - - assert len(threat_incidents) == len(threat_ids) - assert len(case_incidents) == len(case_ids) - assert len(campaign_incidents) == len(campaign_ids) - - # Verify next_run contains updated last_fetch timestamp - assert next_run.get("last_fetch", None) is not None - assert next_run.get("last_fetch") > last_run.get("last_fetch") - - -def test_get_paginated_threats_list(mocker): - """ - Test the get_paginated_threats_list method to verify: - 1. It correctly handles pagination - 2. It respects the max_incidents_to_fetch parameter - """ - # Create client - client = Client(server_url=BASE_URL, verify=False, proxy=False, auth=None, headers=headers) - - # Create a side effect function for threats - get_threats_side_effect = create_mock_paginator_side_effect("threat") - - # Mock the underlying get_a_list_of_threats_request method - get_threats_mock = mocker.patch.object(client, "get_a_list_of_threats_request", side_effect=get_threats_side_effect) - - # Test case 1: Get all threats with high limit (max_incidents_to_fetch > existing items) - # This should set page_size to the limit (10) but return only as many items as exist - result = client.get_paginated_threats_list(filter_="test filter", max_incidents_to_fetch=10) - - # Verify the result contains threats (the exact count depends on the mock function) - assert len(result["threats"]) > 0 - - # Verify the first call was made with correct parameters - assert get_threats_mock.call_count >= 1 - first_call_kwargs = get_threats_mock.call_args_list[0][1] - assert first_call_kwargs["filter_"] == "test filter" - assert first_call_kwargs["page_size"] == 10 - assert first_call_kwargs["page_number"] == 1 - - # Reset the mock for the next test - get_threats_mock.reset_mock() - - # Test case 2: Limited page size (max_incidents_to_fetch = 2) - # With many threats available and max_incidents_to_fetch=2, we expect page_size=2 - # This should result in multiple page calls since there are more threats than fit on one page - result = client.get_paginated_threats_list(filter_="test filter", max_incidents_to_fetch=2) - - # Verify we got threats - assert len(result["threats"]) > 0 - - # Verify each page was requested with the correct parameters - assert get_threats_mock.call_count >= 1 - - # Check first call parameters - first_call_kwargs = get_threats_mock.call_args_list[0][1] - assert first_call_kwargs["filter_"] == "test filter" - assert first_call_kwargs["page_size"] == 2 - assert first_call_kwargs["page_number"] == 1 - - # If there was a second call, check its parameters - if get_threats_mock.call_count > 1: - second_call_kwargs = get_threats_mock.call_args_list[1][1] - assert second_call_kwargs["page_size"] == 2 - assert second_call_kwargs["page_number"] == 2 - - # Reset the mock for the next test - get_threats_mock.reset_mock() - - # Test case 3: One threat per page (max_incidents_to_fetch = 1) - # With many threats available and max_incidents_to_fetch=1, we expect page_size=1 - # This should result in multiple page calls, one per threat - result = client.get_paginated_threats_list(filter_="test filter", max_incidents_to_fetch=1) - - # Verify we got threats - assert len(result["threats"]) > 0 - - # Verify multiple pages were requested - assert get_threats_mock.call_count >= 1 - - # Check that all calls have the correct page_size - for i in range(get_threats_mock.call_count): - call_kwargs = get_threats_mock.call_args_list[i][1] - assert call_kwargs["page_size"] == 1 - assert call_kwargs["page_number"] == i + 1 - - # Reset the mock for the next test - get_threats_mock.reset_mock() - - # Test case 4: No threats to fetch (max_incidents_to_fetch = 0) - result = client.get_paginated_threats_list(filter_="test filter", max_incidents_to_fetch=0) - - # Verify that no threats were fetched - assert len(result["threats"]) == 0 - - # Verify that the underlying method was not called - assert get_threats_mock.call_count == 0 - - -def test_get_paginated_cases_list(mocker): - """ - Test the get_paginated_cases_list method to verify: - 1. It correctly handles pagination - 2. It respects the max_incidents_to_fetch parameter - """ - # Create client - client = Client(server_url=BASE_URL, verify=False, proxy=False, auth=None, headers=headers) - - # Create a side effect function for cases - get_cases_side_effect = create_mock_paginator_side_effect("case") - - # Mock the underlying get_a_list_of_abnormal_cases_identified_by_abnormal_security_request method - get_cases_mock = mocker.patch.object( - client, "get_a_list_of_abnormal_cases_identified_by_abnormal_security_request", side_effect=get_cases_side_effect - ) - - # Test case 1: Get all cases with high limit (max_incidents_to_fetch > existing items) - # This should set page_size to the limit (10) but return only as many items as exist - result = client.get_paginated_cases_list(filter_="test filter", max_incidents_to_fetch=10) - - # Verify the result contains cases (the exact count depends on the mock function) - assert len(result["cases"]) > 0 - - # Verify the first call was made with correct parameters - assert get_cases_mock.call_count >= 1 - first_call_kwargs = get_cases_mock.call_args_list[0][1] - assert first_call_kwargs["filter_"] == "test filter" - assert first_call_kwargs["page_size"] == 10 - assert first_call_kwargs["page_number"] == 1 - - # Reset the mock for the next test - get_cases_mock.reset_mock() - - # Test case 2: Limited page size (max_incidents_to_fetch = 2) - # With many cases available and max_incidents_to_fetch=2, we expect page_size=2 - # This should result in multiple page calls since there are more cases than fit on one page - result = client.get_paginated_cases_list(filter_="test filter", max_incidents_to_fetch=2) - - # Verify we got cases - assert len(result["cases"]) > 0 - - # Verify each page was requested with the correct parameters - assert get_cases_mock.call_count >= 1 - - # Check first call parameters - first_call_kwargs = get_cases_mock.call_args_list[0][1] - assert first_call_kwargs["filter_"] == "test filter" - assert first_call_kwargs["page_size"] == 2 - assert first_call_kwargs["page_number"] == 1 - - # If there was a second call, check its parameters - if get_cases_mock.call_count > 1: - second_call_kwargs = get_cases_mock.call_args_list[1][1] - assert second_call_kwargs["page_size"] == 2 - assert second_call_kwargs["page_number"] == 2 - - # Reset the mock for the next test - get_cases_mock.reset_mock() - - # Test case 3: One case per page (max_incidents_to_fetch = 1) - # With many cases available and max_incidents_to_fetch=1, we expect page_size=1 - # This should result in multiple page calls, one per case - result = client.get_paginated_cases_list(filter_="test filter", max_incidents_to_fetch=1) + incident = build_threat_incident(client, {"threatId": "test-threat-id"}, window, max_page_number=3) - # Verify we got cases - assert len(result["cases"]) > 0 + remediation_times = [m["remediationTimestamp"] for m in json.loads(incident["rawJSON"])["messages"]] + assert remediation_times == ["2023-09-17T16:30:00Z", "2023-09-17T15:30:00Z"] + assert [c.kwargs["page_number"] for c in get_details_spy.call_args_list] == [1, 2] - # Verify multiple pages were requested - assert get_cases_mock.call_count >= 1 - # Check that all calls have the correct page_size - for i in range(get_cases_mock.call_count): - call_kwargs = get_cases_mock.call_args_list[i][1] - assert call_kwargs["page_size"] == 1 - assert call_kwargs["page_number"] == i + 1 - - # Reset the mock for the next test - get_cases_mock.reset_mock() - - # Test case 4: No cases to fetch (max_incidents_to_fetch = 0) - result = client.get_paginated_cases_list(filter_="test filter", max_incidents_to_fetch=0) - - # Verify that no cases were fetched - assert len(result["cases"]) == 0 - - # Verify that the underlying method was not called - assert get_cases_mock.call_count == 0 - - -def test_get_paginated_abusecampaigns_list(mocker): - """ - Test the get_paginated_abusecampaigns_list method to verify: - 1. It correctly handles pagination - 2. It respects the max_incidents_to_fetch parameter - """ - # Create client - client = Client(server_url=BASE_URL, verify=False, proxy=False, auth=None, headers=headers) - - # Create a side effect function for campaigns - get_campaigns_side_effect = create_mock_paginator_side_effect("campaign") - - # Mock the underlying get_a_list_of_campaigns_submitted_to_abuse_mailbox_request method - get_campaigns_mock = mocker.patch.object( - client, "get_a_list_of_campaigns_submitted_to_abuse_mailbox_request", side_effect=get_campaigns_side_effect - ) - - # Test case 1: Get all campaigns with high limit (max_incidents_to_fetch > existing items) - # This should set page_size to the limit (10) but return only as many items as exist - result = client.get_paginated_abusecampaigns_list(filter_="test filter", max_incidents_to_fetch=10) - - # Verify the result contains campaigns (the exact count depends on the mock function) - assert len(result["campaigns"]) > 0 - - # Verify the first call was made with correct parameters - assert get_campaigns_mock.call_count >= 1 - first_call_kwargs = get_campaigns_mock.call_args_list[0][1] - assert first_call_kwargs["filter_"] == "test filter" - assert first_call_kwargs["page_size"] == 10 - assert first_call_kwargs["page_number"] == 1 - - # Reset the mock for the next test - get_campaigns_mock.reset_mock() - - # Test case 2: Limited page size (max_incidents_to_fetch = 2) - # With many campaigns available and max_incidents_to_fetch=2, we expect page_size=2 - # This should result in multiple page calls since there are more campaigns than fit on one page - result = client.get_paginated_abusecampaigns_list(filter_="test filter", max_incidents_to_fetch=2) - - # Verify we got campaigns - assert len(result["campaigns"]) > 0 - - # Verify each page was requested with the correct parameters - assert get_campaigns_mock.call_count >= 1 - - # Check first call parameters - first_call_kwargs = get_campaigns_mock.call_args_list[0][1] - assert first_call_kwargs["filter_"] == "test filter" - assert first_call_kwargs["page_size"] == 2 - assert first_call_kwargs["page_number"] == 1 - - # If there was a second call, check its parameters - if get_campaigns_mock.call_count > 1: - second_call_kwargs = get_campaigns_mock.call_args_list[1][1] - assert second_call_kwargs["page_size"] == 2 - assert second_call_kwargs["page_number"] == 2 - - # Reset the mock for the next test - get_campaigns_mock.reset_mock() - - # Test case 3: One campaign per page (max_incidents_to_fetch = 1) - # With many campaigns available and max_incidents_to_fetch=1, we expect page_size=1 - # This should result in multiple page calls, one per campaign - result = client.get_paginated_abusecampaigns_list(filter_="test filter", max_incidents_to_fetch=1) - - # Verify we got campaigns - assert len(result["campaigns"]) > 0 - - # Verify multiple pages were requested - assert get_campaigns_mock.call_count >= 1 - - # Check that all calls have the correct page_size - for i in range(get_campaigns_mock.call_count): - call_kwargs = get_campaigns_mock.call_args_list[i][1] - assert call_kwargs["page_size"] == 1 - assert call_kwargs["page_number"] == i + 1 - - # Reset the mock for the next test - get_campaigns_mock.reset_mock() - - # Test case 4: No campaigns to fetch (max_incidents_to_fetch = 0) - result = client.get_paginated_abusecampaigns_list(filter_="test filter", max_incidents_to_fetch=0) +def test_build_account_takeover_case_incident(mocker): + client = mock_client(mocker, util_load_json("test_data/test_get_details_of_an_abnormal_case.json")) + window = threat_window(datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC), datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC)) - # Verify that no campaigns were fetched - assert len(result["campaigns"]) == 0 + incident = build_account_takeover_case_incident(client, {"caseId": "1234", "description": "d"}, window) - # Verify that the underlying method was not called - assert get_campaigns_mock.call_count == 0 + assert incident["genaiSummary"] == "genai_summary" + assert incident["details"] == "d" def test_search_messages_command(mocker): @@ -1521,7 +954,7 @@ def test_download_message_eml_command_with_quarantine(mocker): ) def test_is_skippable_error(status_code, expected): """Test that _is_skippable_error correctly categorizes errors by response status code.""" - exc = DemistoException(f"Error in API call [{status_code}]", res=MockResponse(None, status_code)) + exc = ContentClientError(f"Request failed [{status_code}]", response=httpx.Response(status_code)) assert _is_skippable_error(exc) == expected @@ -1532,232 +965,664 @@ def test_is_skippable_error_no_response(): """ - generate_threat_incidents Error Handling Tests + fetch-incidents Tests """ +FETCH_NOW = datetime(2026, 10, 2, 12, 0, 0, tzinfo=UTC) +FETCH_START = "2026-10-02T09:00:00Z" +ALL_TYPES = {"fetch_threats": True, "fetch_abuse_campaigns": True, "fetch_account_takeover_cases": True} -def test_generate_threat_incidents_skips_4xx_error(mocker): - """ - Test that skippable 4xx errors for one threat don't abort processing of other threats. - When: - - Fetching threat details for multiple threats - - One threat returns a 404 error (deleted/archived) - Then: - - The errored threat should be skipped - - Other threats should still be processed - """ - valid_threat_response = { - "threatId": "valid-threat-id", - "messages": [ - { - "threatId": "valid-threat-id", - "receivedTime": "2023-09-17T15:00:00Z", - "remediationTimestamp": "2023-09-17T15:30:00Z", - } - ], +@pytest.fixture +def api(mocker): + mocker.patch("AbnormalSecurity.get_current_datetime", return_value=FETCH_NOW) + # ContentClient logs every error response with demisto.error, which prints in tests. + mocker.patch.object(demisto, "error") + fake = FakeSoarApi(BASE_URL) + transport = httpx.MockTransport(fake.handle) + mocker.patch.object(ContentClient, "_get_async_client", lambda self: httpx.AsyncClient(transport=transport)) + return fake + + +def run_fetch(last_run=None, **kwargs): + options = { + "first_fetch_time": FETCH_START, + "fetch_threats": True, + "fetch_abuse_campaigns": False, + "fetch_account_takeover_cases": False, + "polling_lag": timedelta(0), + "max_incidents_to_fetch": 200, + # Hour-long windows let small fixtures span several windows. + "max_window_minutes": 60, + # Fast enough not to slow the tests; the limiter itself is tested on its own. + "detail_rate_per_second": 1000, + **kwargs, } + client = Client(server_url=BASE_URL, verify=False, proxy=False, auth=None, headers=headers) + return fetch_incidents(client=client, last_run=last_run or {}, **options) - def mock_get_details(threat_id, **kwargs): - if threat_id == "deleted-threat-id": - raise DemistoException("Error in API call [404] - Not Found", res=MockResponse(None, 404)) - return valid_threat_response - client = mock_client(mocker, response=None) - mocker.patch.object(client, "get_details_of_a_threat_request", side_effect=mock_get_details) +def fetch_until_caught_up(last_run=None, max_runs=30, **kwargs): + """Runs fetch the way XSOAR does, feeding each run's last_run into the next, until a run returns nothing.""" + last_run, emitted, all_warnings = last_run or {}, [], [] + for _ in range(max_runs): + last_run, incidents, warnings = run_fetch(last_run, **kwargs) + all_warnings.extend(warnings) + if not incidents and not any( + last_run[k]["failed"] for k in ("threats", "abuse_campaigns", "account_takeover") if k in last_run + ): + return last_run, emitted, all_warnings + emitted.extend(incident["dbotMirrorId"] for incident in incidents) + raise AssertionError("fetch never caught up") - start_datetime = datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC) - end_datetime = datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC) - threats = [ - {"threatId": "deleted-threat-id"}, - {"threatId": "valid-threat-id"}, - ] +def add_threats(api, count, start="2026-10-02T09:00:00Z", step_minutes=7): + base = datetime.strptime(start, ISO_8601_FORMAT).replace(tzinfo=UTC) + ids = [] + for i in range(count): + threat_id = f"t-{i:03d}" + api.add_threat(threat_id, (base + timedelta(minutes=step_minutes * i)).strftime(ISO_8601_FORMAT)) + ids.append(threat_id) + return ids - incidents = generate_threat_incidents(client, threats, 1, start_datetime, end_datetime) - assert len(incidents) == 1 - assert incidents[0]["dbotMirrorId"] == "valid-threat-id" +def test_fetch_resumes_after_max_fetch(api): + """ + Given 20 threats over 3 hours, more than one run's max_fetch. + When fetch runs repeatedly with max_fetch=3. + Then every threat becomes exactly one incident, which 2.4.9 lost by moving last_fetch to now. + """ + ids = add_threats(api, 20, step_minutes=9) + + last_run, emitted, warnings = fetch_until_caught_up(max_incidents_to_fetch=3) + + assert sorted(emitted) == ids + assert warnings == [] + assert last_run["threats"]["window_start"] == "2026-10-02T12:00:00Z" + assert last_run["last_fetch"] == "2026-10-02T12:00:00Z" -@pytest.mark.parametrize("status_code,reason", [(401, "Unauthorized"), (403, "Forbidden"), (429, "Too Many Requests")]) -def test_generate_threat_incidents_raises_non_skippable_errors(mocker, status_code, reason): +def test_fetch_resumes_after_budget_exhausted(api): """ - Test that non-skippable errors (401, 403, 429) are re-raised. + Given a fake clock that advances 10s per HTTP call and a 45s budget. + When fetch runs. + Then it stops early without raising, and the next runs pick up the rest with nothing lost. """ + ids = add_threats(api, 8) + clock = {"now": 0.0} - def mock_get_details(threat_id, **kwargs): - raise DemistoException(f"Error in API call [{status_code}] - {reason}", res=MockResponse(None, status_code)) + def tick(_path): + clock["now"] += 10 - client = mock_client(mocker, response=None) - mocker.patch.object(client, "get_details_of_a_threat_request", side_effect=mock_get_details) + api.on_call = tick + first_run, incidents, _ = run_fetch(deadline=Deadline(45, clock=lambda: clock["now"]), detail_concurrency=1) - start_datetime = datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC) - end_datetime = datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC) + assert 0 < len(incidents) < len(ids) + assert first_run["threats"]["window_start"] == "2026-10-02T09:00:00Z" + api.on_call = None + _, rest, _ = fetch_until_caught_up(first_run) + assert sorted([i["dbotMirrorId"] for i in incidents] + rest) == ids - threats = [{"threatId": "some-threat-id"}] - with pytest.raises(DemistoException) as exc_info: - generate_threat_incidents(client, threats, 1, start_datetime, end_datetime) +def test_fetch_relisted_window_in_new_order_deduplicates(api): + ids = add_threats(api, 6, step_minutes=5) + first_run, first, _ = run_fetch(max_incidents_to_fetch=2) + api.ascending = True - assert str(status_code) in str(exc_info.value) + _, rest, _ = fetch_until_caught_up(first_run, max_incidents_to_fetch=2) + emitted = [i["dbotMirrorId"] for i in first] + rest + assert sorted(emitted) == ids + assert len(emitted) == len(set(emitted)) -def test_generate_threat_incidents_raises_5xx_errors(mocker): - """Test that 5xx errors are re-raised.""" - def mock_get_details(threat_id, **kwargs): - raise DemistoException("Error in API call [500] - Internal Server Error", res=MockResponse(None, 500)) +def test_fetch_5xx_on_cases_still_lets_threats_advance(api): + add_threats(api, 3) + api.add_case("c-1", "2026-10-02T09:30:00Z") + api.fail(r"^/cases$", 500) - client = mock_client(mocker, response=None) - mocker.patch.object(client, "get_details_of_a_threat_request", side_effect=mock_get_details) + last_run, incidents, warnings = run_fetch(**ALL_TYPES) - start_datetime = datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC) - end_datetime = datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC) + assert {i["name"] for i in incidents} == {"Threat"} + assert last_run["threats"]["window_start"] == "2026-10-02T12:00:00Z" + assert last_run["account_takeover"]["window_start"] == "2026-10-02T09:00:00Z" + assert any("account takeover cases failed" in w for w in warnings) + _, rest, _ = fetch_until_caught_up(last_run, **ALL_TYPES) + assert rest == ["c-1"] - with pytest.raises(DemistoException) as exc_info: - generate_threat_incidents(client, [{"threatId": "id"}], 1, start_datetime, end_datetime) - assert "500" in str(exc_info.value) +@pytest.mark.parametrize("path, status", [(r"^/threats$", 401), (r"^/threats/", 403)]) +def test_fetch_auth_error_commits_nothing(api, path, status): + add_threats(api, 2) + api.fail(path, status) + with pytest.raises(AuthError): + run_fetch() -def test_generate_threat_incidents_handles_4xx_mid_pagination(mocker): + +def test_fetch_404_detail_counts_as_emitted(api): + add_threats(api, 2) + api.add_threat("t-gone", "2026-10-02T09:01:00Z") + api.fail(r"^/threats/t-gone$", 404, times=None) + + last_run, incidents, warnings = run_fetch() + + assert sorted(i["dbotMirrorId"] for i in incidents) == ["t-000", "t-001"] + assert last_run["threats"]["window_start"] == "2026-10-02T12:00:00Z" + assert warnings == [] + + +def test_fetch_retention_400_moves_window_forward_and_warns(api): + add_threats(api, 1, start="2026-10-02T10:30:00Z") + message = "Dates provided (2026-10-02 09:00:00 to 2026-10-02 10:00:00) are out of accepted data retention range" + api.fail(r"^/threats$", 400, body={"message": message}) + + _, incidents, warnings = run_fetch() + + assert [i["dbotMirrorId"] for i in incidents] == ["t-000"] + assert len(warnings) == 1 + assert "retention" in warnings[0] + + +def test_fetch_other_400_does_not_skip_window(api): + add_threats(api, 1) + api.fail(r"^/threats$", 400, body={"message": "bad filter"}) + + last_run, incidents, warnings = run_fetch() + + assert incidents == [] + assert last_run["threats"]["window_start"] == "2026-10-02T09:00:00Z" + assert len(warnings) == 1 + + +def test_fetch_402_on_cases_skips_account_takeover_cases_and_warns(api): + add_threats(api, 1) + api.add_case("c-1", "2026-10-02T09:30:00Z") + api.fail(r"^/cases$", 402, times=None) + + last_run, incidents, warnings = run_fetch(**ALL_TYPES) + + assert [i["name"] for i in incidents] == ["Threat"] + assert warnings == ["Skipped account takeover cases: the tenant isn't licensed for them."] + # The window starts over at now, so licensing the tenant later doesn't replay the backlog. + assert last_run["account_takeover"]["window_start"] == last_run["account_takeover"]["window_end"] == "2026-10-02T12:00:00Z" + assert last_run["last_fetch"] == "2026-10-02T12:00:00Z" + api.clear_failures() + _, incidents, warnings = run_fetch(last_run, **ALL_TYPES) + assert incidents == [] + assert warnings == [] + + +def test_fetch_item_failing_on_three_runs_is_skipped_with_warning(api): """ - Test that skippable 4xx errors during pagination are handled gracefully. + Given one threat whose detail call always returns 500. + When fetch runs three times. + Then the others become incidents on the first run, the window waits for the bad one, + and on the third failure it's skipped with a warning and the window advances. """ + add_threats(api, 3) + api.fail(r"^/threats/t-001$", 500, times=None) - def mock_get_details(threat_id, **kwargs): - if threat_id == "paginating-threat-id": - if kwargs.get("page_number", 1) == 1: - return { - "threatId": "paginating-threat-id", - "messages": [ - { - "threatId": "paginating-threat-id", - "receivedTime": "2023-09-17T15:00:00Z", - "remediationTimestamp": "2023-09-17T15:30:00Z", - } - ], - "nextPageNumber": 2, - } - else: - raise DemistoException("Error in API call [404] - Not Found", res=MockResponse(None, 404)) - return { - "threatId": "valid-threat-id", - "messages": [ - { - "threatId": "valid-threat-id", - "receivedTime": "2023-09-17T16:00:00Z", - "remediationTimestamp": "2023-09-17T16:30:00Z", - } - ], - } + run_1, incidents_1, warnings_1 = run_fetch() + run_2, incidents_2, _ = run_fetch(run_1) + run_3, incidents_3, warnings_3 = run_fetch(run_2) - client = mock_client(mocker, response=None) - mocker.patch.object(client, "get_details_of_a_threat_request", side_effect=mock_get_details) + assert sorted(i["dbotMirrorId"] for i in incidents_1) == ["t-000", "t-002"] + assert run_1["threats"]["failed"] == {"t-001": 1} + assert run_2["threats"]["failed"] == {"t-001": 2} + assert incidents_2 == incidents_3 == [] + assert warnings_1 == [] + assert any("t-001" in w and "3 runs" in w for w in warnings_3) + assert run_3["threats"]["window_start"] == "2026-10-02T12:00:00Z" + assert len(api.detail_calls("threats")) == 3 + 1 + 1 - start_datetime = datetime(2023, 9, 17, 14, 0, 0, tzinfo=UTC) - end_datetime = datetime(2023, 9, 17, 17, 0, 0, tzinfo=UTC) - threats = [ - {"threatId": "paginating-threat-id"}, - {"threatId": "valid-threat-id"}, - ] +def test_fetch_bad_item_does_not_stop_others(api): + api.add_case("c-1", "2026-10-02T09:10:00Z") + api.add_case("c-2", "2026-10-02T09:20:00Z", firstObserved=None) + api.add_case("c-3", "2026-10-02T09:30:00Z") - incidents = generate_threat_incidents(client, threats, 5, start_datetime, end_datetime) + last_run, incidents, _ = run_fetch(fetch_threats=False, fetch_account_takeover_cases=True) - assert len(incidents) == 1 - assert incidents[0]["dbotMirrorId"] == "valid-threat-id" + assert sorted(i["dbotMirrorId"] for i in incidents) == ["c-1", "c-3"] + assert last_run["account_takeover"]["failed"] == {"c-2": 1} -""" - generate_abuse_campaign_incidents Error Handling Tests -""" +def test_fetch_five_consecutive_failures_stop_the_type(api): + add_threats(api, 7) + api.add_campaign("a-1", "2026-10-02T09:30:00Z") + api.fail(r"^/threats/", 503, times=None) + + last_run, incidents, warnings = run_fetch(fetch_abuse_campaigns=True, detail_concurrency=1) + + assert [i["dbotMirrorId"] for i in incidents] == ["a-1"] + assert any("5 failures in a row" in w for w in warnings) + assert len(last_run["threats"]["failed"]) == 5 -def test_generate_abuse_campaign_incidents_skips_4xx_error(mocker): - """Test that skippable 4xx errors skip the campaign and continue.""" - valid_campaign_response = { - "campaignId": "valid-campaign-id", - "firstReported": "2023-09-17T15:00:00Z", +def test_fetch_overflow_halves_window_and_saves_new_end(api, mocker): + """ + Given 6 threats in the first hour and a list page size of 2. + When fetch runs with max_fetch=1. + Then the window is halved until it fits in one page, the halved end is saved, and later runs + still emit every threat once. + """ + mocker.patch("AbnormalSecurity.LIST_PAGE_SIZE", 2) + ids = add_threats(api, 6, step_minutes=9) + + first_run, incidents, _ = run_fetch(max_incidents_to_fetch=1) + + assert len(incidents) == 1 + assert first_run["threats"]["window_end"] < "2026-10-02T10:00:00Z" + assert all(int(q["pageSize"]) == 2 and q.get("pageNumber", "1") == "1" for q in api.list_calls("threats")) + _, rest, _ = fetch_until_caught_up(first_run, max_incidents_to_fetch=1) + assert sorted([incidents[0]["dbotMirrorId"]] + rest) == ids + + +def test_fetch_partly_drained_window_paginates_instead_of_halving(api, mocker): + mocker.patch("AbnormalSecurity.LIST_PAGE_SIZE", 2) + add_threats(api, 4, step_minutes=5) + last_run = { + "version": 2, + "threats": { + "window_start": "2026-10-02T09:00:00Z", + "window_end": "2026-10-02T10:00:00Z", + "emitted_ids": ["t-003"], + "failed": {}, + }, } - def mock_get_campaign(campaign_id, **kwargs): - if campaign_id == "deleted-campaign-id": - raise DemistoException("Error in API call [404] - Not Found", res=MockResponse(None, 404)) - return valid_campaign_response + next_run, incidents, _ = run_fetch(last_run) + + assert sorted(i["dbotMirrorId"] for i in incidents) == ["t-000", "t-001", "t-002"] + first_window_calls = api.list_calls("threats")[:2] + assert [q.get("pageNumber") for q in first_window_calls] == ["1", "2"] + assert all(q["filter"].endswith("lte 2026-10-02T10:00:00Z") for q in first_window_calls) + assert next_run["threats"]["window_start"] == "2026-10-02T12:00:00Z" + + +def test_build_list_filter_strings(): + window = FetchWindow( + window_start=datetime(2026, 10, 2, 9, 0, 0, tzinfo=UTC), + window_end=datetime(2026, 10, 2, 10, 0, 0, tzinfo=UTC), + ) + + assert ( + build_list_filter(THREATS_SPEC, window) + == "latestTimeRemediated gte 2026-10-02T09:00:00Z and latestTimeRemediated lte 2026-10-02T10:00:00Z" + ) + assert ( + build_list_filter(ABUSE_CAMPAIGNS_SPEC, window) + == "lastReportedTime gte 2026-10-02T09:00:00Z and lastReportedTime lte 2026-10-02T09:59:59.999999Z" + ) + assert ( + build_list_filter(ACCOUNT_TAKEOVER_SPEC, window) + == "lastModifiedTime gte 2026-10-02T09:00:00Z and lastModifiedTime lte 2026-10-02T09:59:59.999999Z" + ) + + +def test_fetch_window_boundary_items_land_in_exactly_one_window(api): + api.add_threat("t-edge", "2026-10-02T10:00:00Z") + api.add_campaign("a-edge", "2026-10-02T10:00:00Z") - client = mock_client(mocker, response=None) - mocker.patch.object(client, "get_details_of_an_abuse_mailbox_campaign_request", side_effect=mock_get_campaign) + _, emitted, _ = fetch_until_caught_up(fetch_abuse_campaigns=True) - campaigns = [ - {"campaignId": "deleted-campaign-id"}, - {"campaignId": "valid-campaign-id"}, - ] + assert sorted(emitted) == ["a-edge", "t-edge"] - incidents = generate_abuse_campaign_incidents(client, campaigns) + +def test_fetch_migrates_v1_last_run(api): + """ + Given a 2.4.9 last_run and a 2-minute polling lag. + When the new code runs for the first time. + Then each window starts where 2.4.9's next run would have started. + """ + api.add_threat("t-1", "2026-10-02T10:59:00Z") + + next_run, incidents, _ = run_fetch( + {"last_fetch": "2026-10-02T11:00:00Z"}, polling_lag=timedelta(minutes=2), max_incidents_to_fetch=0 + ) + + assert next_run["version"] == 2 + assert next_run["threats"]["window_start"] == "2026-10-02T10:58:00Z" + assert next_run["threats"]["window_end"] == "2026-10-02T11:58:00Z" + assert next_run["last_fetch"] == "2026-10-02T10:58:00Z" + assert "abuse_campaigns" not in next_run + assert incidents == [] + _, incidents, _ = run_fetch(next_run, polling_lag=timedelta(minutes=2)) + assert [i["dbotMirrorId"] for i in incidents] == ["t-1"] + + +def test_fetch_writes_oldest_window_start_as_last_fetch_for_rollback(api): + add_threats(api, 2) + api.add_case("c-1", "2026-10-02T09:30:00Z") + api.fail(r"^/cases$", 500) + + next_run, _, _ = run_fetch(**ALL_TYPES) + + assert next_run["last_fetch"] == next_run["account_takeover"]["window_start"] == "2026-10-02T09:00:00Z" + + +def test_fetch_newly_enabled_type_starts_at_now_minus_lag(api): + add_threats(api, 1) + api.add_campaign("a-old", "2026-10-02T09:30:00Z") + api.add_campaign("a-new", "2026-10-02T11:59:30Z") + first_run, _, _ = run_fetch() + first_run, _, _ = run_fetch(first_run, fetch_abuse_campaigns=False) + + next_run, incidents, _ = run_fetch(first_run, fetch_abuse_campaigns=True, polling_lag=timedelta(minutes=1)) + + assert next_run["abuse_campaigns"]["window_start"] == "2026-10-02T11:59:00Z" + assert incidents == [] + + +def test_fetch_disabled_type_drops_its_window(api): + first_run, _, _ = run_fetch(**ALL_TYPES) + + next_run, _, _ = run_fetch(first_run, fetch_abuse_campaigns=False) + + assert "abuse_campaigns" not in next_run + assert next_run["last_fetch"] == next_run["threats"]["window_start"] + + +def test_fetch_rotates_which_type_goes_first(api): + api.add_threat("t-1", "2026-10-02T09:10:00Z") + api.add_threat("t-2", "2026-10-02T09:20:00Z") + api.add_campaign("a-1", "2026-10-02T09:10:00Z") + api.add_campaign("a-2", "2026-10-02T09:20:00Z") + + run_1, incidents_1, _ = run_fetch(fetch_abuse_campaigns=True, max_incidents_to_fetch=1) + _, incidents_2, _ = run_fetch(run_1, fetch_abuse_campaigns=True, max_incidents_to_fetch=1) + + assert [i["name"] for i in incidents_1 + incidents_2] == ["Threat", "Abuse Campaign"] + + +def test_fetch_with_wide_max_window_uses_one_window(api): + add_threats(api, 1) + + next_run, incidents, _ = run_fetch(max_window_minutes=10**6) assert len(incidents) == 1 - assert incidents[0]["dbotMirrorId"] == "valid-campaign-id" + assert next_run["threats"]["window_start"] == next_run["threats"]["window_end"] == "2026-10-02T12:00:00Z" + + +class _StubHandler(BaseHTTPRequestHandler): + """Serves no response at all (`hang`), a body that trickles in a byte at a time (`drip`), a body + that trickles in and then stops mid-way (`stall`), or an error whose body trickles in (`error_drip`).""" + + # HTTP/1.1 keeps the connection open, as the SOAR API does, so the client can reach its socket. + protocol_version = "HTTP/1.1" + + def log_message(self, *args): + pass + + def do_GET(self): + try: + if self.server.mode == "hang": + self.server.release.wait(30) + return + self.send_response(503 if self.server.mode == "error_drip" else 200) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", "100000") + self.end_headers() + started = time.monotonic() + while not self.server.release.wait(0.2): + if self.server.mode == "stall" and time.monotonic() - started > 1.5: + continue + self.wfile.write(b" ") + self.wfile.flush() + except OSError: + pass + + +@pytest.fixture +def stub_server(): + server = ThreadingHTTPServer(("127.0.0.1", 0), _StubHandler) + server.daemon_threads = True + server.release = threading.Event() + threading.Thread(target=server.serve_forever, daemon=True).start() + yield server + server.release.set() + server.shutdown() + server.server_close() + + +@pytest.mark.parametrize("mode", ["hang", "drip", "stall", "error_drip"]) +def test_fetch_time_budget_bounds_hung_and_slow_responses(stub_server, mode, mocker): + """ + Given a server that never responds, sends its body a byte at a time, stops sending mid-body, or + sends an error response whose body arrives a byte at a time. + When fetch runs with a 4s budget. + Then the run stops within the budget and keeps its window, instead of waiting for the server. + """ + stub_server.mode = mode + mocker.patch("AbnormalSecurity.get_current_datetime", return_value=FETCH_NOW) + mocker.patch.object(demisto, "error") + client = Client( + server_url=f"http://127.0.0.1:{stub_server.server_port}", verify=False, proxy=False, auth=None, headers=headers + ) + started = time.monotonic() + + next_run, incidents, _ = fetch_incidents( + client=client, + last_run={}, + first_fetch_time=FETCH_START, + fetch_threats=True, + fetch_abuse_campaigns=False, + fetch_account_takeover_cases=False, + fetch_time_budget=4, + ) + # A stalled read that kept the socket timeout from the start of the call would end ~1.5s late. + assert time.monotonic() - started < 4 + 1 + assert incidents == [] + assert next_run["threats"]["window_start"] == "2026-10-02T09:00:00Z" -def test_generate_abuse_campaign_incidents_raises_non_skippable_errors(mocker): - """Test that non-skippable errors (401) are re-raised.""" - def mock_get_campaign(campaign_id, **kwargs): - raise DemistoException("Error in API call [401] - Unauthorized", res=MockResponse(None, 401)) +def test_fetch_rate_limit_spaces_http_calls(api): + """ + Given one list call and 4 detail calls, and a limit of 20 requests per second with a burst of 1. + When fetch runs with 4 workers. + Then the 5 calls take at least 4 intervals of 50ms, however many workers there are. + """ + add_threats(api, 4) + started = time.monotonic() - client = mock_client(mocker, response=None) - mocker.patch.object(client, "get_details_of_an_abuse_mailbox_campaign_request", side_effect=mock_get_campaign) + _, incidents, _ = run_fetch(detail_rate_per_second=20, max_window_minutes=10**6) - with pytest.raises(DemistoException) as exc_info: - generate_abuse_campaign_incidents(client, [{"campaignId": "id"}]) + assert len(incidents) == 4 + assert len(api.calls) == 5 + assert time.monotonic() - started >= 4 * 0.05 - assert "401" in str(exc_info.value) +def test_fetch_waiting_for_a_rate_token_counts_against_the_budget(api): + """ + Given a limit of one request every 2s and a 3s budget. + When fetch runs. + Then it stops at the budget while waiting for a token, instead of waiting past it. + """ + add_threats(api, 3) + started = time.monotonic() -""" - generate_account_takeover_cases_incidents Error Handling Tests -""" + next_run, _, _ = run_fetch(detail_rate_per_second=0.5, fetch_time_budget=3, max_window_minutes=10**6, detail_concurrency=1) + assert time.monotonic() - started < 3 + 1 + assert len(api.calls) == 2 + assert next_run["threats"]["window_start"] == "2026-10-02T09:00:00Z" -def test_generate_account_takeover_cases_incidents_skips_4xx_error(mocker): - """Test that skippable 4xx errors skip the case and continue.""" - valid_case_response = { - "caseId": "valid-case-id", - "firstObserved": "2023-09-17T15:00:00Z", - "genai_summary": "Test summary", + +def patch_threat_details(mocker, delay): + """Serves threat details after a per-threat delay, to control when each worker finishes.""" + + def get_details(self, threat_id, deadline=None, **kwargs): + time.sleep(delay(threat_id)) + remediated = "2026-10-02T09:30:00Z" + return {"threatId": threat_id, "messages": [{"receivedTime": remediated, "remediationTimestamp": remediated}]} + + return mocker.patch.object(Client, "get_details_of_a_threat_request", autospec=True, side_effect=get_details) + + +def test_fetch_detail_calls_stay_under_the_worker_cap(api, mocker): + add_threats(api, 12, step_minutes=4) + lock, in_flight = threading.Lock(), {"now": 0, "max": 0} + + def delay(_threat_id): + with lock: + in_flight["now"] += 1 + in_flight["max"] = max(in_flight["max"], in_flight["now"]) + time.sleep(0.05) + with lock: + in_flight["now"] -= 1 + return 0 + + patch_threat_details(mocker, delay) + _, incidents, _ = run_fetch(detail_concurrency=4, max_window_minutes=10**6) + + assert len(incidents) == 12 + assert in_flight["max"] == 4 + + +def test_fetch_takes_one_rate_token_per_http_call(api, mocker): + page = [{"receivedTime": "2026-10-02T09:10:00Z", "remediationTimestamp": "2026-10-02T09:10:00Z"}] + api.add_threat("t-paged", "2026-10-02T09:10:00Z", message_pages=[page, page, page]) + acquire = mocker.spy(TokenBucketRateLimiter, "acquire") + + _, incidents, _ = run_fetch(max_window_minutes=10**6) + + assert len(json.loads(incidents[0]["rawJSON"])["messages"]) == 3 + assert len(api.detail_calls("threats")) == 3 + assert acquire.call_count == len(api.calls) + + +def test_fetch_429_mid_batch_keeps_completed_incidents(api): + """ + Given 8 threats where the third one's detail call is rate limited. + When fetch runs with 2 workers. + Then incidents completed before the 429 are kept, the run stops, the window stays, and the + next run emits the rest with no duplicates. + """ + ids = add_threats(api, 8) + api.ascending = True + api.fail(r"^/threats/t-002$", 429) + + first_run, incidents, warnings = run_fetch(detail_concurrency=2, **ALL_TYPES) + + emitted = [i["dbotMirrorId"] for i in incidents] + assert {"t-000", "t-001"} <= set(emitted) + assert "t-002" not in emitted + assert any("Rate limited" in w for w in warnings) + assert first_run["threats"]["window_start"] == "2026-10-02T09:00:00Z" + assert api.list_calls("abusecampaigns") == [] + _, rest, _ = fetch_until_caught_up(first_run) + assert sorted(emitted + rest) == ids + + +def test_fetch_output_order_does_not_depend_on_worker_timing(api, mocker): + add_threats(api, 10, step_minutes=5) + # Earlier items finish last, so completion order is the reverse of list order. + patch_threat_details(mocker, lambda threat_id: 0.1 - int(threat_id[2:]) * 0.01) + + _, concurrent, _ = run_fetch(detail_concurrency=4, max_window_minutes=10**6) + _, serial, _ = run_fetch(detail_concurrency=1, max_window_minutes=10**6) + + assert [i["dbotMirrorId"] for i in concurrent] == [i["dbotMirrorId"] for i in serial] + + +@pytest.mark.parametrize("mode, expected_incidents", [("created", 1), ("modified", 4)]) +def test_fetch_case_mode_controls_duplicate_incidents_for_modified_cases(api, mocker, mode, expected_incidents): + """ + Given one Account Takeover case that is modified 3 times after it's first fetched. + When fetch runs after each modification. + Then created-time mode creates one incident, and last-modified mode creates a new one every time. + """ + clock = {"now": FETCH_NOW} + mocker.patch("AbnormalSecurity.get_current_datetime", side_effect=lambda: clock["now"]) + api.add_case("c-1", "2026-10-02T09:10:00Z") + options = {"fetch_threats": False, "fetch_account_takeover_cases": True, "case_fetch_mode": mode} + + last_run, incidents, _ = run_fetch(**options) + emitted = [i["dbotMirrorId"] for i in incidents] + for minutes in (10, 20, 30): + api.items["cases"][0]["lastModifiedTime"] = (FETCH_NOW + timedelta(minutes=minutes)).strftime(ISO_8601_FORMAT) + clock["now"] = FETCH_NOW + timedelta(minutes=minutes + 5) + last_run, incidents, _ = run_fetch(last_run, **options) + emitted += [i["dbotMirrorId"] for i in incidents] + + assert emitted == ["c-1"] * expected_incidents + assert last_run["account_takeover"]["mode"] == mode + field_name = "createdTime" if mode == "created" else "lastModifiedTime" + assert all(q["filter"].startswith(f"{field_name} gte") for q in api.list_calls("cases")) + + +def test_fetch_case_mode_change_restarts_window_from_its_start(api): + api.add_case("c-1", "2026-10-02T09:10:00Z") + last_run = { + "version": 2, + "account_takeover": { + "window_start": "2026-10-02T09:00:00Z", + "window_end": "2026-10-02T09:30:00Z", + "emitted_ids": ["c-1"], + "failed": {"c-2": 1}, + "mode": "modified", + }, } - def mock_get_case(case_id, **kwargs): - if case_id == "deleted-case-id": - raise DemistoException("Error in API call [410] - Gone", res=MockResponse(None, 410)) - return valid_case_response + next_run, incidents, _ = run_fetch( + last_run, fetch_threats=False, fetch_account_takeover_cases=True, case_fetch_mode="created", max_incidents_to_fetch=0 + ) + + assert next_run["account_takeover"] == { + "window_start": "2026-10-02T09:00:00Z", + "window_end": "2026-10-02T10:00:00Z", + "emitted_ids": [], + "failed": {}, + "mode": "created", + } + assert incidents == [] - client = mock_client(mocker, response=None) - mocker.patch.object(client, "get_details_of_an_abnormal_case_request", side_effect=mock_get_case) - cases = [ - {"caseId": "deleted-case-id", "description": "Deleted case"}, - {"caseId": "valid-case-id", "description": "Valid case"}, - ] +def test_fetch_settings_from_params_defaults_for_instances_without_the_new_params(): + assert fetch_settings_from_params({}) == { + "fetch_time_budget": 150, + "max_window_minutes": 1440, + "detail_concurrency": 4, + "detail_rate_per_second": 2.0, + "case_fetch_mode": "modified", + } - incidents = generate_account_takeover_cases_incidents(client, cases) - assert len(incidents) == 1 - assert incidents[0]["dbotMirrorId"] == "valid-case-id" +def test_fetch_settings_from_params_parses_configured_values(): + settings = fetch_settings_from_params( + { + "fetch_time_budget": "100", + "max_window_minutes": "120", + "detail_concurrency": "2", + "detail_rate_per_second": "0.5", + "case_fetch_mode": "Created time", + } + ) + assert settings == { + "fetch_time_budget": 100.0, + "max_window_minutes": 120, + "detail_concurrency": 2, + "detail_rate_per_second": 0.5, + "case_fetch_mode": "created", + } -def test_generate_account_takeover_cases_incidents_raises_non_skippable_errors(mocker): - """Test that non-skippable errors (429) are re-raised.""" - def mock_get_case(case_id, **kwargs): - raise DemistoException("Error in API call [429] - Too Many Requests", res=MockResponse(None, 429)) +@pytest.mark.parametrize("params", [{"detail_concurrency": "0"}, {"detail_rate_per_second": "-1"}, {"case_fetch_mode": "x"}]) +def test_fetch_settings_from_params_rejects_invalid_values(params): + with pytest.raises(DemistoException): + fetch_settings_from_params(params) - client = mock_client(mocker, response=None) - mocker.patch.object(client, "get_details_of_an_abnormal_case_request", side_effect=mock_get_case) - with pytest.raises(DemistoException) as exc_info: - generate_account_takeover_cases_incidents(client, [{"caseId": "id", "description": "test"}]) +def test_fetch_upgrade_from_v1_keeps_last_modified_case_mode(api): + next_run, _, _ = run_fetch( + {"last_fetch": "2026-10-02T11:00:00Z"}, + fetch_account_takeover_cases=True, + **{k: v for k, v in fetch_settings_from_params({}).items() if k != "detail_rate_per_second"}, + ) - assert "429" in str(exc_info.value) + assert next_run["account_takeover"]["mode"] == "modified" + assert next_run["account_takeover"]["window_start"] == "2026-10-02T12:00:00Z" + assert "mode" not in next_run["threats"] diff --git a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/README.md b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/README.md index 4c41eb17ce40..4818d929c5e9 100644 --- a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/README.md +++ b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/README.md @@ -10,7 +10,7 @@ This integration was integrated and tested with version 1.3.0 of Abnormal Securi | Trust any certificate (not secure) | | False | | Use system proxy settings | | False | | Fetch incidents | Retrieves incidents based on the customer's selection from three categories: Threats, Account Takeover Cases, and Abuse Campaigns | False | -| Maximum incidents to fetch. | Maximum number of incidents per fetch. The default value is 200. | False | +| Maximum incidents to fetch. | The maximum number of incidents per fetch run, across all incident types. | False | | Fetch Threats | | False | | Fetch Abuse Campaigns | | False | | Fetch Account Takeover Cases | | False | @@ -19,6 +19,11 @@ This integration was integrated and tested with version 1.3.0 of Abnormal Securi | Incidents Fetch Interval | | False | | Polling Lag Time (in minutes) | Time in minutes to subtract from polling time window for data consistency (Default : 2 mins) | False | | Maximum incidents pages to fetch | Maximum number of pages to fetch for incidents | False | +| Case fetch mode | The timestamp used to fetch Account Takeover cases. "Last modified time" creates a new incident each time a case is modified, resolved or reopened, as in earlier versions. "Created time" creates one incident per case. | False | +| Fetch time budget (in seconds) | The maximum time a fetch run can spend calling the API before it saves its progress and stops. Keep this below the 3-minute fetch timeout. | False | +| Maximum fetch window (in minutes) | The longest time range listed in one request. Windows with too many items to list at once are split automatically. | False | +| Concurrent detail requests | The number of incident detail requests that run concurrently. | False | +| Maximum requests per second | The maximum number of API requests per second during a fetch run, shared by all concurrent requests. | False | ## Commands diff --git a/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/test_data/fake_soar_api.py b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/test_data/fake_soar_api.py new file mode 100644 index 000000000000..17db22afd4ca --- /dev/null +++ b/Packs/AbnormalSecurity/Integrations/AbnormalSecurity/test_data/fake_soar_api.py @@ -0,0 +1,138 @@ +""" +In-memory stand-in for the SOAR API endpoints that fetch-incidents calls, served as an httpx transport. + +List endpoints filter on the window the integration sends, page by offset, and are sorted by ID rather +than by the filtered field, which is how the real API behaves. Failures can be injected per path so +the integration's real `ContentClient` status handling is exercised. +""" + +import re +from collections.abc import Callable +from datetime import datetime + +import httpx + +# list endpoint -> (response list key, ID field) +LIST_ENDPOINTS = { + "threats": ("threats", "threatId"), + "abusecampaigns": ("campaigns", "campaignId"), + "cases": ("cases", "caseId"), +} +# `/threats` floors both bounds to whole seconds and excludes the upper one; the others include both. +EXCLUSIVE_END_ENDPOINTS = {"threats"} +FILTER_RE = re.compile(r"(\w+) gte (\S+) and \1 lte (\S+)") +PATH_RE = re.compile(r"^/(threats|abusecampaigns|cases)(?:/([^/]+))?$") + + +class FakeSoarApi: + def __init__(self, base_url: str): + self.base_path = httpx.URL(base_url).path.rstrip("/") + self.items: dict[str, list[dict]] = {endpoint: [] for endpoint in LIST_ENDPOINTS} + self.calls: list[tuple[str, dict]] = [] + self.ascending = False + self.on_call: Callable[[str], None] | None = None + self._failures: list[dict] = [] + + def add_threat(self, threat_id: str, remediated: str, message_pages: list[list[dict]] | None = None) -> None: + self.items["threats"].append({"threatId": threat_id, "latestTimeRemediated": remediated, "pages": message_pages}) + + def add_campaign(self, campaign_id: str, last_reported: str) -> None: + self.items["abusecampaigns"].append({"campaignId": campaign_id, "lastReportedTime": last_reported}) + + def add_case(self, case_id: str, created: str, modified: str | None = None, **details) -> None: + self.items["cases"].append( + { + "caseId": case_id, + "createdTime": created, + "lastModifiedTime": modified or created, + "description": f"case {case_id}", + "details": details, + } + ) + + def fail(self, path_pattern: str, status: int, times: int | None = 1, body: dict | None = None) -> None: + """Makes requests whose path matches `path_pattern` fail with `status`, `times` times (None: always).""" + self._failures.append({"pattern": re.compile(path_pattern), "status": status, "times": times, "body": body}) + + def clear_failures(self) -> None: + self._failures.clear() + + def detail_calls(self, endpoint: str) -> list[str]: + return [path for path, _ in self.calls if path.startswith(f"/{endpoint}/")] + + def list_calls(self, endpoint: str) -> list[dict]: + return [query for path, query in self.calls if path == f"/{endpoint}"] + + def handle(self, request: httpx.Request) -> httpx.Response: + path = request.url.path.removeprefix(self.base_path) + query = dict(request.url.params) + self.calls.append((path, query)) + if self.on_call: + self.on_call(path) + for failure in self._failures: + if failure["times"] != 0 and failure["pattern"].search(path): + if failure["times"] is not None: + failure["times"] -= 1 + return httpx.Response(failure["status"], json=failure["body"] or {"message": f"injected {failure['status']}"}) + match = PATH_RE.match(path) + if match is None: + return httpx.Response(404, json={"message": f"no fake for {path}"}) + endpoint, item_id = match.groups() + if item_id is None: + return self._list(endpoint, query) + return self._detail(endpoint, item_id, query) + + def _list(self, endpoint: str, query: dict) -> httpx.Response: + list_key, id_key = LIST_ENDPOINTS[endpoint] + match = FILTER_RE.match(query.get("filter", "")) + if match is None: + return httpx.Response(400, json={"message": f"unsupported filter: {query.get('filter')}"}) + field_name, gte, lte = match.groups() + start, end = datetime.fromisoformat(gte), datetime.fromisoformat(lte) + if endpoint in EXCLUSIVE_END_ENDPOINTS: + start, end = start.replace(microsecond=0), end.replace(microsecond=0) + + def in_window(t): + return start <= t < end + else: + + def in_window(t): + return start <= t <= end + + page_size, page_number = query.get("pageSize", "100"), query.get("pageNumber", "1") + if not (page_size.isdigit() and page_number.isdigit()): + return httpx.Response(400, json={"message": "pageSize and pageNumber must be integers"}) + page_size, page_number = int(page_size), int(page_number) + matching = [item for item in self.items[endpoint] if in_window(datetime.fromisoformat(item[field_name]))] + matching.sort(key=lambda item: item[id_key], reverse=not self.ascending) + page = matching[(page_number - 1) * page_size : page_number * page_size] + response = {list_key: [self._summary(endpoint, item) for item in page], "pageNumber": page_number} + if page_number * page_size < len(matching): + response["nextPageNumber"] = page_number + 1 + return httpx.Response(200, json=response) + + def _summary(self, endpoint: str, item: dict) -> dict: + if endpoint == "cases": + return {"caseId": item["caseId"], "description": item["description"]} + _, id_key = LIST_ENDPOINTS[endpoint] + return {id_key: item[id_key]} + + def _detail(self, endpoint: str, item_id: str, query: dict) -> httpx.Response: + _, id_key = LIST_ENDPOINTS[endpoint] + item = next((i for i in self.items[endpoint] if i[id_key] == item_id), None) + if item is None: + return httpx.Response(404, json={"message": "not found"}) + if endpoint == "threats": + pages = item["pages"] or [ + [{"threatId": item_id, "receivedTime": item["latestTimeRemediated"], "remediationTimestamp": item["latestTimeRemediated"]}] + ] + page_number = int(query.get("pageNumber", "1")) + response = {"threatId": item_id, "messages": pages[page_number - 1], "pageNumber": page_number} + if page_number < len(pages): + response["nextPageNumber"] = page_number + 1 + return httpx.Response(200, json=response) + if endpoint == "abusecampaigns": + return httpx.Response(200, json={"campaignId": item_id, "firstReported": item["lastReportedTime"]}) + details = {"caseId": item_id, "firstObserved": item["createdTime"], "genai_summary": f"summary {item_id}"} + details.update(item["details"]) + return httpx.Response(200, json={key: value for key, value in details.items() if value is not None}) diff --git a/Packs/AbnormalSecurity/Integrations/AbnormalSecurityEventCollector/README.md b/Packs/AbnormalSecurity/Integrations/AbnormalSecurityEventCollector/README.md index 49ccce26edfb..edbaa2b72323 100644 --- a/Packs/AbnormalSecurity/Integrations/AbnormalSecurityEventCollector/README.md +++ b/Packs/AbnormalSecurity/Integrations/AbnormalSecurityEventCollector/README.md @@ -1,8 +1,6 @@ Abnormal Security Event Collector integration for XSIAM. This integration was integrated and tested with version 01 of Abnormal Security Event Collector -This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM. - ## Configure Abnormal Security Event Collector in Cortex | **Parameter** | **Required** | diff --git a/Packs/AbnormalSecurity/ReleaseNotes/2_4_10.md b/Packs/AbnormalSecurity/ReleaseNotes/2_4_10.md new file mode 100644 index 000000000000..61971044bc28 --- /dev/null +++ b/Packs/AbnormalSecurity/ReleaseNotes/2_4_10.md @@ -0,0 +1,3 @@ +## Abnormal Security + +- Documentation and metadata improvements. diff --git a/Packs/AbnormalSecurity/ReleaseNotes/2_5_0.md b/Packs/AbnormalSecurity/ReleaseNotes/2_5_0.md new file mode 100644 index 000000000000..d4b56c6480f6 --- /dev/null +++ b/Packs/AbnormalSecurity/ReleaseNotes/2_5_0.md @@ -0,0 +1,13 @@ + +#### Integrations + +##### Abnormal Security + +- Fixed an issue where incidents could be missed when a fetch run reached the *Maximum incidents to fetch* limit. Each incident type now resumes from where the previous run stopped. +- Fixed an issue where a single failing threat, abuse campaign or case could stop the whole fetch run. Failing items are retried on the next runs and skipped with a warning in the instance's health after failing on 3 runs. +- Added support for creating one incident per Account Takeover case, instead of a new incident each time a case is modified, resolved or reopened. To use it, set *Case fetch mode* to *Created time*. +- Improved implementation of ***fetch-incidents***. Incident details are now retrieved concurrently, at a request rate you can set, which by default stays well under the API's rate limit. +- Improved implementation of ***fetch-incidents***. Each run now stops within a time budget you can set and saves its progress, so it finishes before the fetch timeout. +- Added the *Case fetch mode*, *Fetch time budget (in seconds)*, *Maximum fetch window (in minutes)*, *Concurrent detail requests* and *Maximum requests per second* parameters. *Case fetch mode* defaults to *Last modified time*, which keeps the current Account Takeover case behavior. +- Updated the integration to use **ContentClient** for its API calls. +- Updated the Docker image to: *demisto/fastapi:0.125.0.12689358*. diff --git a/Packs/AbnormalSecurity/pack_metadata.json b/Packs/AbnormalSecurity/pack_metadata.json index 411fc8541cbe..7abe69142384 100644 --- a/Packs/AbnormalSecurity/pack_metadata.json +++ b/Packs/AbnormalSecurity/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Abnormal Security", "description": "Abnormal Security detects and protects against the whole spectrum of email attacks", "support": "partner", - "currentVersion": "2.4.9", + "currentVersion": "2.5.0", "created": "2021-07-11T00:00:00Z", "author": "Abnormal Security", "url": "", @@ -29,7 +29,6 @@ "marketplacev2", "platform" ], - "defaultDataSource": "Abnormal Security Event Collector", "supportedModules": [ "agentix", "xsiam" diff --git a/Packs/Absolute/Integrations/Absolute/Absolute.py b/Packs/Absolute/Integrations/Absolute/Absolute.py index 64f12a89c063..a36969c3a57d 100644 --- a/Packs/Absolute/Integrations/Absolute/Absolute.py +++ b/Packs/Absolute/Integrations/Absolute/Absolute.py @@ -350,7 +350,7 @@ def fetch_events_between_dates(self, fetch_limit: int, start_date: datetime, end return all_events def prepare_query_string_for_fetch_events( - self, start_date: datetime, end_date: datetime, page_size: int = None, next_page: str = None + self, start_date: datetime, end_date: datetime, page_size: Optional[int] = None, next_page: Optional[str] = None ) -> str: """ Prepares the query string for fetching events based on the provided parameters. @@ -1142,6 +1142,261 @@ def get_device_location_command(args, client) -> CommandResults: return CommandResults(readable_output=f"No device locations found in {INTEGRATION} for the given filters: {args}") +def prepare_wipe_request_query(args: dict, status: str, next_page: Optional[str] = None, page_size: Optional[int] = None) -> str: + created_from = args.get("created_from_date_time_utc") + created_to = args.get("created_to_date_time_utc") + request_status = args.get("request_status") or args.get("action_status") + + query_params = [] + if created_from: + query_params.append(f"createdFromDateTimeUtc={created_from}") + if created_to: + query_params.append(f"createdToDateTimeUtc={created_to}") + if request_status: + query_params.append(f"{status}={request_status}") + if next_page: + query_params.append(f"nextPage={next_page}") + if page_size: + query_params.append(f"pageSize={page_size}") + + return "&".join(query_params) + + +def wipe_actions_list_command(args, client) -> CommandResults: + """ + Get the status of actions by devices or by request. + + Args: + args (Dict[str, Any]): Command arguments. + client (ClientV3): Absolute Client. + + Returns: + CommandResults: The command results. + """ + prefix = "[absolute-wipe-actions-list]" + demisto.debug(f"{prefix} {args=}") + request_uid = args.get("request_uid") + device_uids = argToList(args.get("device_uids")) + next_page = args.get("page") + limit = arg_to_number(args.get("limit", DEFAULT_LIMIT)) + + payload: dict[str, Any] = {} + payload["deviceUids"] = device_uids + + query_string = prepare_wipe_request_query(args, "actionStatus", next_page=next_page, page_size=limit) + if request_uid: + demisto.debug(f"{prefix} request_uid provided: {request_uid}. Getting actions for the request.") + demisto.debug(f"{prefix} query_string: {query_string}.\nPayload: {payload}.") + res = client.api_request_absolute( + "POST", + f"/v3/actions/wipe/get-actions/{request_uid}", + query_string=query_string, + body=payload, + ) + else: + demisto.debug(f"{prefix} request_uid not provided. Getting actions for the devices.") + demisto.debug(f"{prefix} query_string: {query_string}.\nPayload: {payload}.") + res = client.api_request_absolute( + "POST", + "/v3/actions/wipe/get-actions", + query_string=query_string, + body=payload, + ) + + if res: + outputs = res + human_readable = tableToMarkdown( + f"{INTEGRATION} wipe actions list:", + outputs, + headers=["actionUid", "requestUid", "deviceUid", "deviceName", "esn", "actionStatus"], + headerTransform=string_to_table_header, + removeNull=True, + ) + return CommandResults( + outputs_prefix="Absolute.WipeAction", + outputs=outputs, + outputs_key_field="actionUid", + readable_output=human_readable, + raw_response=res, + ) + else: + return CommandResults(readable_output=f"No wipe actions found in {INTEGRATION} for the given filters: {args}") + + +def wipe_request_cancel_command(args, client) -> CommandResults: + """ + Cancel a wipe request based on provided arguments. + + Args: + args (Dict[str, Any]): Command arguments. + client (ClientV3): Absolute Client. + + Returns: + CommandResults: The command results. + """ + prefix = "[absolute-wipe-request-cancel]" + demisto.debug(f"{prefix} {args=}") + request_uid = args.get("request_uid") + action_uids = argToList(args.get("action_uids")) + cancel_all_actions = args.get("cancel_all_actions", False) + + payload: dict[str, Any] = {} + if action_uids: + payload["actionUids"] = action_uids + payload["cancelAllActions"] = argToBoolean(cancel_all_actions) + + demisto.debug(f"{prefix} Canceling wipe actions for the request {request_uid}.\nPayload: {payload}.") + res = client.api_request_absolute( + "POST", + f"/v3/actions/wipe/cancel-actions/{request_uid}", + body=payload, + success_status_code=[202], + resp_type="response", + ) + + return CommandResults( + readable_output=f"Wipe actions for the request {request_uid} have been successfully canceled.", + raw_response=res, + ) + + +def wipe_request_create_command(args, client) -> CommandResults: + """ + Create a Wipe request for the devices based on provided arguments. + + Args: + args (Dict[str, Any]): Command arguments. + client (ClientV3): Absolute Client. + + Returns: + CommandResults: The command results. + """ + prefix = "[absolute-wipe-request-create]" + demisto.debug(f"{prefix} {args=}") + device_uids = argToList(args.get("device_uids")) + mac_user_name = args.get("mac_user_name") + mac_pwd = args.get("mac_pwd") + name = args.get("name") + description = args.get("description") + secure_erase_count = arg_to_number(args.get("secure_erase_count"), required=False) + + payload: dict[str, Any] = { + "deviceUids": device_uids, + "unenrollDevicesAndFreeLicenses": argToBoolean(args.get("unenroll_devices_and_free_licenses", False)), + } + if name: + payload["name"] = name + if description: + payload["description"] = description + if mac_user_name: + payload["macUsername"] = mac_user_name + if mac_pwd: + payload["macPwd"] = mac_pwd + + firmware_wipe_requested = args.get("firmware_wipe_requested") + if firmware_wipe_requested is not None: + payload["firmwareWipeRequested"] = argToBoolean(firmware_wipe_requested) + + crypto_wipe_requested = args.get("crypto_wipe_requested") + if crypto_wipe_requested is not None: + payload["cryptoWipeRequested"] = argToBoolean(crypto_wipe_requested) + + wipe_used_space_only = args.get("wipe_used_space_only") + if wipe_used_space_only is not None: + payload["wipeUsedSpaceOnly"] = argToBoolean(wipe_used_space_only) + + delete_all_files_requested = args.get("delete_all_files_requested") + if delete_all_files_requested is not None: + payload["deleteAllFilesRequested"] = argToBoolean(delete_all_files_requested) + + disable_window_os = args.get("disable_window_os") + if disable_window_os is not None: + payload["disableWindowOS"] = argToBoolean(disable_window_os) + + if secure_erase_count is not None: + if secure_erase_count not in (1, 3, 7): + raise_demisto_exception("secure_erase_count must be one of: 1, 3, 7.") + payload["secureEraseCount"] = secure_erase_count + + demisto.debug(f"{prefix} Creating wipe request with payload: {payload}") + res = client.api_request_absolute( + "POST", + "/v3/actions/requests/wipe", + body=payload, + ) + if not res: + raise DemistoException(f"{INTEGRATION} error: Failed to create wipe request - received empty response from API.") + + return CommandResults( + readable_output=f"Wipe request {res.get('requestUid')} has been successfully created.", + raw_response=res, + ) + + +def wipe_request_list_command(args, client) -> CommandResults: + """ + Retrieves wipe requests based on provided arguments. + + Args: + args (Dict[str, Any]): Command arguments. + client (ClientV3): Absolute Client. + + Returns: + CommandResults: The command results. + """ + prefix = "[absolute-wipe-request-list]" + demisto.debug(f"{prefix} {args=}") + request_uid = args.get("request_uid") + next_page = args.get("page") + limit = arg_to_number(args.get("limit", DEFAULT_LIMIT)) + + hr_headers = [ + "requestId", + "requestUid", + "requestStatus", + "totalDevices", + "pending", + "processing", + "completed", + "canceled", + "failed", + ] + + if request_uid: + demisto.debug(f"{prefix} request_uid provided, fetching details for request_uid: {request_uid}") + response = client.send_request_to_api("GET", f"/v3/actions/requests/wipe/{request_uid}", "", ok_codes=(200,)) + res = response.get("data") if isinstance(response, dict) else response + else: + demisto.debug(f"{prefix} no request_uid provided, fetching wipe requests list with filters: {args}") + query_string = prepare_wipe_request_query(args, "requestStatus", next_page=next_page, page_size=limit) + query_string = client.prepare_query_string_for_canonical_request(query_string) + demisto.debug(f"{prefix} query_string for wipe requests list: {query_string}") + response = client.send_request_to_api("GET", "/v3/actions/requests/wipe", query_string, ok_codes=(200,)) + res = response.get("data") if isinstance(response, dict) else response + + if res: + human_readable = tableToMarkdown( + f"{INTEGRATION} wipe request details:" if request_uid else f"{INTEGRATION} wipe requests list:", + res, + headers=hr_headers, + headerTransform=string_to_table_header, + removeNull=True, + ) + return CommandResults( + outputs_prefix="Absolute.WipeRequest", + outputs=res, + outputs_key_field="requestUid", + readable_output=human_readable, + raw_response=res, + ) + else: + return CommandResults( + readable_output=f"No wipe request found in {INTEGRATION} for the given request_uid: {request_uid}" + if request_uid + else f"No wipe requests found in {INTEGRATION} for the given filters: {args}" + ) + + """ EVENT COLLECTOR """ @@ -1313,6 +1568,18 @@ def main() -> None: # pragma: no cover send_events_to_xsiam(events=events, vendor=VENDOR, product=PRODUCT) return_results(command_result) + elif demisto.command() == "absolute-wipe-request-list": + return_results(wipe_request_list_command(args=args, client=client_v3)) + + elif demisto.command() == "absolute-wipe-actions-list": + return_results(wipe_actions_list_command(args=args, client=client_v3)) + + elif demisto.command() == "absolute-wipe-request-create": + return_results(wipe_request_create_command(args=args, client=client_v3)) + + elif demisto.command() == "absolute-wipe-request-cancel": + return_results(wipe_request_cancel_command(args=args, client=client_v3)) + else: raise NotImplementedError(f"{demisto.command()} is not an existing {INTEGRATION} command.") diff --git a/Packs/Absolute/Integrations/Absolute/Absolute.yml b/Packs/Absolute/Integrations/Absolute/Absolute.yml index b9d6815076ba..348f0b383fc5 100644 --- a/Packs/Absolute/Integrations/Absolute/Absolute.yml +++ b/Packs/Absolute/Integrations/Absolute/Absolute.yml @@ -53,6 +53,7 @@ configuration: supportedModules: - xsiam - defaultvalue: 'false' + display: Fetch Events name: isFetchEvents type: 8 required: false @@ -813,6 +814,210 @@ script: description: The maximum number of records to return per page. Note, this may be restricted by fixed system limits. name: absolute-device-get-events description: Retrieves a list of events from the Absolute instance. + - arguments: + - name: request_uid + description: The system-defined unique identifier of the Device Action request. + - name: request_status + description: Filters results to include responses where requestStatus equals the given value. + auto: PREDEFINED + predefined: + - Init + - Canceled + - Completed + - InProgress + - Failed + - Scheduled + - PendingApproval + - name: created_from_date_time_utc + description: Filters results to include responses where createdDateTimeUtc is from the given date and time (inclusive). Format . Example 2011-12-03T10:15:30.000Z. + - name: created_to_date_time_utc + description: Filters results to include responses where createdDateTimeUtc is to the given date and time (inclusive). Format . Example 2011-12-03T10:15:30.000Z. + - name: page + description: The pagination token for the next page of results. + - name: limit + description: Pagination limit - maximum number of events to return. + defaultValue: '50' + name: absolute-wipe-request-list + description: Get information about the Wipe requests. + outputs: + - contextPath: Absolute.WipeRequest.requestId + description: The ID of the wipe request. + type: String + - contextPath: Absolute.WipeRequest.requestUid + description: The unique identifier of the wipe request. + type: String + - contextPath: Absolute.WipeRequest.requestStatus + description: The status of the wipe request. + type: String + - contextPath: Absolute.WipeRequest.totalDevices + description: The total number of devices in the wipe request. + type: Number + - contextPath: Absolute.WipeRequest.pending + description: The number of devices pending wipe. + type: Number + - contextPath: Absolute.WipeRequest.processing + description: The number of devices currently processing the wipe. + type: Number + - contextPath: Absolute.WipeRequest.completed + description: The number of devices that have completed the wipe. + type: Number + - contextPath: Absolute.WipeRequest.canceled + description: The number of devices that have canceled the wipe. + type: Number + - contextPath: Absolute.WipeRequest.failed + description: The number of devices that have failed the wipe. + type: Number + - contextPath: Absolute.WipeRequest.macUsername + description: The username of the user with administrative rights (applies to Mac devices only). + type: String + - contextPath: Absolute.WipeRequest.macPwd + description: The password of the user with administrative rights (applies to Mac devices only). + type: String + - contextPath: Absolute.WipeRequest.unenrollDevicesAndFreeLicenses + description: Indicates whether a Windows device should be unenrolled after the Wipe is complete. + type: Boolean + - contextPath: Absolute.WipeRequest.createdDateTimeUtc + description: The date and time when the wipe request was created. + type: Date + - contextPath: Absolute.WipeRequest.updatedDateTimeUtc + description: The date and time when the wipe request was last updated. + type: Date + - contextPath: Absolute.WipeRequest.requester + description: The requester of the wipe request. + type: String + - arguments: + - name: request_uid + description: The system-defined unique identifier of the Device Action request. + - name: device_uids + description: The system-defined unique identifiers of the devices included in the request. Passing an empty list will return results for all applicable devices. + isArray: true + required: true + - name: action_status + description: Filters results to include responses where action_status equals the given value. + auto: PREDEFINED + predefined: + - Failed + - Pending + - Canceled + - Processing + - Completed + - name: created_from_date_time_utc + description: Filters results to include responses where createdDateTimeUtc is from the given date and time (inclusive). Format . Example 2011-12-03T10:15:30.000Z. + - name: created_to_date_time_utc + description: Filters results to include responses where createdDateTimeUtc is to the given date and time (inclusive). Format . Example 2011-12-03T10:15:30.000Z. + - name: page + description: The pagination token for the next page of results. + - name: limit + description: Pagination limit - maximum number of events to return. + defaultValue: '50' + name: absolute-wipe-actions-list + description: Get the status of actions by devices or by request. + outputs: + - contextPath: Absolute.WipeAction.actionUid + description: The unique identifier of the wipe action. + type: String + - contextPath: Absolute.WipeAction.requestUid + description: The unique identifier of the wipe request. + type: String + - contextPath: Absolute.WipeAction.deviceUid + description: The unique identifier of the device. + type: String + - contextPath: Absolute.WipeAction.deviceName + description: The name of the device. + type: String + - contextPath: Absolute.WipeAction.esn + description: The Electronic Serial Number of the device. + type: String + - contextPath: Absolute.WipeAction.actionStatus + description: The status of the wipe action. + type: String + - contextPath: Absolute.WipeAction.updatedDateTimeUtc + description: The date and time when the wipe action was last updated. + type: Date + - contextPath: Absolute.WipeAction.createdDateTimeUtc + description: The date and time when the wipe action was created. + type: Date + - contextPath: Absolute.WipeAction.certificateUid + description: The unique identifier of the certificate. + type: String + - arguments: + - name: device_uids + description: The system-defined unique identifiers of the devices included in the request. Passing an empty list will return results for all applicable devices. + isArray: true + required: true + - name: name + description: The name of the Wipe request. + - name: description + description: The user-defined description of the Wipe request. + - name: firmware_wipe_requested + description: Indicates whether a Firmware Wipe is requested. + auto: PREDEFINED + predefined: + - 'true' + - 'false' + - name: crypto_wipe_requested + description: Indicates whether a Cryptographic Wipe is requested. Default is true if no other Wipe options are specified. + auto: PREDEFINED + predefined: + - 'true' + - 'false' + - name: wipe_used_space_only + description: Indicates whether only the device's used disk space is wiped. Applies to Cryptographic Wipe requests only. + auto: PREDEFINED + predefined: + - 'true' + - 'false' + - name: delete_all_files_requested + description: Indicates whether a Delete All Files wipe is requested. + auto: PREDEFINED + predefined: + - 'true' + - 'false' + - name: disable_window_os + description: Indicates whether all non-OS and user profile data is deleted and the Windows operating system is disabled after the Wipe is complete. Applies only to Delete All Files wipe requests on Windows devices. + auto: PREDEFINED + predefined: + - 'true' + - 'false' + - name: secure_erase_count + description: Indicates the number of times the file content is overwritten with non-sensitive data (series of zeros and ones). Applies to Delete All Files wipe requests only. + auto: PREDEFINED + predefined: + - '1' + - '3' + - '7' + - name: mac_user_name + description: The username of the user with administrative rights (applies to Mac devices only). + - name: mac_pwd + description: The password of the user with administrative rights (applies to Mac devices only). + secret: true + - name: unenroll_devices_and_free_licenses + description: Indicates whether a Windows device should be unenrolled after the Wipe is complete. + auto: PREDEFINED + predefined: + - 'true' + - 'false' + defaultValue: 'false' + name: absolute-wipe-request-create + description: Create a Wipe request for the devices. + outputs: [] + - arguments: + - name: request_uid + description: The system-defined unique identifier of the Device Action request. + required: true + - name: action_uids + description: The system-defined unique identifiers of the actions included in the request. + isArray: true + - name: cancel_all_actions + description: Indicates whether all actions in the request should be canceled. + auto: PREDEFINED + predefined: + - 'true' + - 'false' + defaultValue: 'false' + name: absolute-wipe-request-cancel + description: Attempt to cancel all Wipe actions with an actionStatus of Pending for the request specified by the requestUid path parameter. + outputs: [] runonce: false script: '-' type: python diff --git a/Packs/Absolute/Integrations/Absolute/Absolute_test.py b/Packs/Absolute/Integrations/Absolute/Absolute_test.py index acd3650dcce7..5da5d8117c7c 100644 --- a/Packs/Absolute/Integrations/Absolute/Absolute_test.py +++ b/Packs/Absolute/Integrations/Absolute/Absolute_test.py @@ -931,3 +931,276 @@ def test_last_run_hold_created_time_and_events_sorted_by_created_time_in_fetch_e assert last_run_object.get("latest_events_time") == events[-1].get("createdDateTimeUtc") for event in events: assert event.get("_time") == event.get("eventDateTimeUtc") + + +def test_wipe_request_list_command(mocker, absolute_client_v3): + """ + Given: + - All relevant arguments for the command that is executed + + When: + - wipe_request_list_command is executed + + Then: + - The http request is called with the right arguments + """ + from Absolute import wipe_request_list_command + + response = util_load_json("test_data/wipe_requests.json") + mocker.patch.object(absolute_client_v3, "send_request_to_api", return_value={"data": response}) + command_results = wipe_request_list_command( + args={"created_from_date_time_utc": "2018-1-03T10:15:30.000Z", "created_to_date_time_utc": "2020-1-03T10:15:30.000Z"}, + client=absolute_client_v3, + ) + assert isinstance(command_results.outputs, list) + assert command_results.outputs[0].get("requestUid") == "23dbd460-0547-466d-a655-99d9340ff598" + + +def test_wipe_request_list_command_with_request_uid(mocker, absolute_client_v3): + """ + Given: + - A request_uid argument is provided + + When: + - wipe_request_list_command is executed + + Then: + - The http request is called with the right endpoint and the single request data is returned + """ + from Absolute import wipe_request_list_command + + response = util_load_json("test_data/wipe_requests.json") + mock_api = mocker.patch.object(absolute_client_v3, "send_request_to_api", return_value={"data": response[0]}) + command_results = wipe_request_list_command( + args={"request_uid": "23dbd460-0547-466d-a655-99d9340ff598"}, + client=absolute_client_v3, + ) + assert command_results.outputs.get("requestUid") == "23dbd460-0547-466d-a655-99d9340ff598" + call_args = mock_api.call_args + assert "/v3/actions/requests/wipe/23dbd460-0547-466d-a655-99d9340ff598" in call_args[0][1] + + +def test_wipe_actions_list_command(mocker, absolute_client_v3): + """ + Given: + - All relevant arguments for the command that is executed + + When: + - wipe_actions_list_command is executed + + Then: + - The http request is called with the right arguments + """ + from Absolute import wipe_actions_list_command + + response = util_load_json("test_data/wipe_actions.json") + mocker.patch.object(absolute_client_v3, "api_request_absolute", return_value=response) + command_results = wipe_actions_list_command( + args={ + "request_uid": "1", + "device_uids": "2", + "created_from_date_time_utc": "2018-1-03T10:15:30.000Z", + "created_to_date_time_utc": "2020-1-03T10:15:30.000Z", + }, + client=absolute_client_v3, + ) + assert isinstance(command_results.outputs, list) + assert command_results.outputs[0].get("requestUid") == "1" + assert command_results.outputs[0].get("actionUid") == "7f0741c5-fbe9-4144-8c54-498db6559304" + + +def test_wipe_request_create_command(mocker, absolute_client_v3): + """ + Given: + - All relevant arguments for the command that is executed + + When: + - wipe_request_create_command is executed + + Then: + - The http request is called with the right arguments + """ + from Absolute import wipe_request_create_command + + response = {"requestUid": "23dbd460-0547-466d-a655-99d9340ff598"} + mocker.patch.object(absolute_client_v3, "api_request_absolute", return_value=response) + command_results = wipe_request_create_command( + args={"device_uids": "497f6eca-6276-4993-bfeb-53cbbbba6f08"}, client=absolute_client_v3 + ) + assert command_results.readable_output == "Wipe request 23dbd460-0547-466d-a655-99d9340ff598 has been successfully created." + + +def test_wipe_request_create_command_with_all_args(mocker, absolute_client_v3): + """ + Given: + - All optional arguments for the wipe request create command + + When: + - wipe_request_create_command is executed with all arguments + + Then: + - The http request is called with the right payload including all optional fields + """ + from Absolute import wipe_request_create_command + + response = {"requestUid": "23dbd460-0547-466d-a655-99d9340ff598"} + mock_api = mocker.patch.object(absolute_client_v3, "api_request_absolute", return_value=response) + command_results = wipe_request_create_command( + args={ + "device_uids": "497f6eca-6276-4993-bfeb-53cbbbba6f08", + "name": "Test Wipe", + "description": "Test wipe description", + "firmware_wipe_requested": "true", + "crypto_wipe_requested": "false", + "wipe_used_space_only": "true", + "delete_all_files_requested": "true", + "disable_window_os": "false", + "secure_erase_count": "3", + "mac_user_name": "admin", + "mac_pwd": "password123", + "unenroll_devices_and_free_licenses": "true", + }, + client=absolute_client_v3, + ) + assert command_results.readable_output == "Wipe request 23dbd460-0547-466d-a655-99d9340ff598 has been successfully created." + + call_kwargs = mock_api.call_args + payload = call_kwargs.kwargs.get("body") or call_kwargs[1].get("body") + assert payload["deviceUids"] == ["497f6eca-6276-4993-bfeb-53cbbbba6f08"] + assert payload["name"] == "Test Wipe" + assert payload["description"] == "Test wipe description" + assert payload["firmwareWipeRequested"] is True + assert payload["cryptoWipeRequested"] is False + assert payload["wipeUsedSpaceOnly"] is True + assert payload["deleteAllFilesRequested"] is True + assert payload["disableWindowOS"] is False + assert payload["secureEraseCount"] == 3 + assert payload["macUsername"] == "admin" + assert payload["macPwd"] == "password123" + assert payload["unenrollDevicesAndFreeLicenses"] is True + + +def test_wipe_request_create_command_invalid_secure_erase_count(mocker, absolute_client_v3): + """ + Given: + - An invalid secure_erase_count value (not 1, 3, or 7) + + When: + - wipe_request_create_command is executed + + Then: + - A DemistoException is raised with the appropriate error message + """ + from Absolute import wipe_request_create_command + + with pytest.raises(DemistoException, match="secure_erase_count must be one of: 1, 3, 7."): + wipe_request_create_command( + args={ + "device_uids": "497f6eca-6276-4993-bfeb-53cbbbba6f08", + "secure_erase_count": "5", + }, + client=absolute_client_v3, + ) + + +def test_wipe_request_cancel_command(mocker, absolute_client_v3): + """ + Given: + - A request_uid and default arguments + - The Absolute API returns HTTP 202 with an empty body (as documented for cancel-actions) + + When: + - wipe_request_cancel_command is executed + + Then: + - The command completes successfully without a JSON parse error (regression test for + the bug where resp_type defaulted to "json" and failed to parse the empty 202 body) + - api_request_absolute is called with resp_type="response" + - The readable_output confirms cancellation + """ + from Absolute import wipe_request_cancel_command + from requests.models import Response + + # Simulate a real 202 empty-body response coming back from the Absolute API. + empty_202 = Response() + empty_202.status_code = 202 + empty_202._content = b"" + http_mock = mocker.patch.object(absolute_client_v3, "_http_request", return_value=empty_202) + api_spy = mocker.spy(absolute_client_v3, "api_request_absolute") + + command_results = wipe_request_cancel_command(args={"request_uid": "1"}, client=absolute_client_v3) + + assert command_results.readable_output == "Wipe actions for the request 1 have been successfully canceled." + # Regression guard: resp_type MUST be "response" so BaseClient does not try to JSON-parse b''. + assert api_spy.call_args.kwargs.get("resp_type") == "response" + # Sanity: BaseClient was called with the JWS validation URL (real path exercised end-to-end). + assert http_mock.call_count == 1 + + +def test_wipe_request_cancel_command_cancel_all(mocker, absolute_client_v3): + """ + Given: + - cancel_all_actions is set to true and no action_uids are provided + - The Absolute API returns HTTP 202 with an empty body + + When: + - wipe_request_cancel_command is executed + + Then: + - The command completes successfully (no JSON parse error on empty 202) + - The payload includes cancelAllActions=true and does not include actionUids + - api_request_absolute is called with resp_type="response" + """ + from Absolute import wipe_request_cancel_command + from requests.models import Response + + empty_202 = Response() + empty_202.status_code = 202 + empty_202._content = b"" + mocker.patch.object(absolute_client_v3, "_http_request", return_value=empty_202) + api_spy = mocker.spy(absolute_client_v3, "api_request_absolute") + + command_results = wipe_request_cancel_command( + args={"request_uid": "1", "cancel_all_actions": "true"}, + client=absolute_client_v3, + ) + assert command_results.readable_output == "Wipe actions for the request 1 have been successfully canceled." + + payload = api_spy.call_args.kwargs.get("body") + assert payload["cancelAllActions"] is True + assert "actionUids" not in payload + assert api_spy.call_args.kwargs.get("resp_type") == "response" + + +def test_wipe_request_cancel_command_cancel_all_false(mocker, absolute_client_v3): + """ + Given: + - cancel_all_actions is explicitly set to false with an action_uid provided + - The Absolute API returns HTTP 202 with an empty body + + When: + - wipe_request_cancel_command is executed + + Then: + - The command completes successfully (no JSON parse error on empty 202) + - The payload includes cancelAllActions=false (not silently dropped) and actionUids + - api_request_absolute is called with resp_type="response" + """ + from Absolute import wipe_request_cancel_command + from requests.models import Response + + empty_202 = Response() + empty_202.status_code = 202 + empty_202._content = b"" + mocker.patch.object(absolute_client_v3, "_http_request", return_value=empty_202) + api_spy = mocker.spy(absolute_client_v3, "api_request_absolute") + + wipe_request_cancel_command( + args={"request_uid": "1", "action_uids": "uid1", "cancel_all_actions": "false"}, + client=absolute_client_v3, + ) + + payload = api_spy.call_args.kwargs.get("body") + assert payload["cancelAllActions"] is False + assert payload["actionUids"] == ["uid1"] + assert api_spy.call_args.kwargs.get("resp_type") == "response" diff --git a/Packs/Absolute/Integrations/Absolute/README.md b/Packs/Absolute/Integrations/Absolute/README.md index 07b7de727a18..62ccf30050ff 100644 --- a/Packs/Absolute/Integrations/Absolute/README.md +++ b/Packs/Absolute/Integrations/Absolute/README.md @@ -1100,3 +1100,139 @@ In the ***absolute-device-application-list*** command, the following argument wa - *Absolute.DeviceUnenroll.Serial* - *Absolute.DeviceUnenroll.SystemName* - *Absolute.DeviceUnenroll.Username* + +### absolute-wipe-request-list + +*** +Get information about the Wipe requests. + +#### Base Command + +`absolute-wipe-request-list` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| request_uid | The system-defined unique identifier of the Device Action request. | Optional | +| request_status | Filters results to include responses where requestStatus equals the given value. Possible values are: Init, Canceled, Completed, InProgress, Failed, Scheduled, PendingApproval. | Optional | +| created_from_date_time_utc | Filters results to include responses where createdDateTimeUtc is from the given date and time (inclusive). Format <yyyy-MM-dd'T'HH:mm:ss.SSS'Z'>. Example 2011-12-03T10:15:30.000Z. | Optional | +| created_to_date_time_utc | Filters results to include responses where createdDateTimeUtc is to the given date and time (inclusive). Format <yyyy-MM-dd'T'HH:mm:ss.SSS'Z'>. Example 2011-12-03T10:15:30.000Z. | Optional | +| page | The pagination token for the next page of results. | Optional | +| limit | Pagination limit - maximum number of events to return. Default is 50. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Absolute.WipeRequest.requestId | String | The ID of the wipe request. | +| Absolute.WipeRequest.requestUid | String | The unique identifier of the wipe request. | +| Absolute.WipeRequest.requestStatus | String | The status of the wipe request. | +| Absolute.WipeRequest.totalDevices | Number | The total number of devices in the wipe request. | +| Absolute.WipeRequest.pending | Number | The number of devices pending wipe. | +| Absolute.WipeRequest.processing | Number | The number of devices currently processing the wipe. | +| Absolute.WipeRequest.completed | Number | The number of devices that have completed the wipe. | +| Absolute.WipeRequest.canceled | Number | The number of devices that have canceled the wipe. | +| Absolute.WipeRequest.failed | Number | The number of devices that have failed the wipe. | +| Absolute.WipeRequest.macUsername | String | The username of the user with administrative rights \(applies to Mac devices only\). | +| Absolute.WipeRequest.macPwd | String | The password of the user with administrative rights \(applies to Mac devices only\). | +| Absolute.WipeRequest.unenrollDevicesAndFreeLicenses | Boolean | Indicates whether a Windows device should be unenrolled after the Wipe is complete. | +| Absolute.WipeRequest.createdDateTimeUtc | Date | The date and time when the wipe request was created. | +| Absolute.WipeRequest.updatedDateTimeUtc | Date | The date and time when the wipe request was last updated. | +| Absolute.WipeRequest.requester | String | The requester of the wipe request. | + +### absolute-wipe-actions-list + +*** +Get the status of actions by devices or by request. + +#### Base Command + +`absolute-wipe-actions-list` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| request_uid | The system-defined unique identifier of the Device Action request. | Optional | +| device_uids | The system-defined unique identifiers of the devices included in the request. Passing an empty list will return results for all applicable devices. | Required | +| action_status | Filters results to include responses where action_status equals the given value. Possible values are: Failed, Pending, Canceled, Processing, Completed. | Optional | +| created_from_date_time_utc | Filters results to include responses where createdDateTimeUtc is from the given date and time (inclusive). Format <yyyy-MM-dd'T'HH:mm:ss.SSS'Z'>. Example 2011-12-03T10:15:30.000Z. | Optional | +| created_to_date_time_utc | Filters results to include responses where createdDateTimeUtc is to the given date and time (inclusive). Format <yyyy-MM-dd'T'HH:mm:ss.SSS'Z'>. Example 2011-12-03T10:15:30.000Z. | Optional | +| page | The pagination token for the next page of results. | Optional | +| limit | Pagination limit - maximum number of events to return. Default is 50. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Absolute.WipeAction.actionUid | String | The unique identifier of the wipe action. | +| Absolute.WipeAction.requestUid | String | The unique identifier of the wipe request. | +| Absolute.WipeAction.deviceUid | String | The unique identifier of the device. | +| Absolute.WipeAction.deviceName | String | The name of the device. | +| Absolute.WipeAction.esn | String | The Electronic Serial Number of the device. | +| Absolute.WipeAction.actionStatus | String | The status of the wipe action. | +| Absolute.WipeAction.updatedDateTimeUtc | Date | The date and time when the wipe action was last updated. | +| Absolute.WipeAction.createdDateTimeUtc | Date | The date and time when the wipe action was created. | +| Absolute.WipeAction.certificateUid | String | The unique identifier of the certificate. | + +### absolute-wipe-request-create + +*** +Create a Wipe request for the devices. + +#### Base Command + +`absolute-wipe-request-create` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| device_uids | The system-defined unique identifiers of the devices included in the request. Passing an empty list will return results for all applicable devices. | Required | +| name | The name of the Wipe request. | Optional | +| description | The user-defined description of the Wipe request. | Optional | +| firmware_wipe_requested | Indicates whether a Firmware Wipe is requested. Possible values are: true, false. | Optional | +| crypto_wipe_requested | Indicates whether a Cryptographic Wipe is requested. Default is true if no other Wipe options are specified. Possible values are: true, false. | Optional | +| wipe_used_space_only | Indicates whether only the device's used disk space is wiped. Applies to Cryptographic Wipe requests only. Possible values are: true, false. | Optional | +| delete_all_files_requested | Indicates whether a Delete All Files wipe is requested. Possible values are: true, false. | Optional | +| disable_window_os | Indicates whether all non-OS and user profile data is deleted and the Windows operating system is disabled after the Wipe is complete. Applies only to Delete All Files wipe requests on Windows devices. Possible values are: true, false. | Optional | +| secure_erase_count | Indicates the number of times the file content is overwritten with non-sensitive data (series of zeros and ones). Applies to Delete All Files wipe requests only. Possible values are: 1, 3, 7. | Optional | +| mac_user_name | The username of the user with administrative rights (applies to Mac devices only). | Optional | +| mac_pwd | The password of the user with administrative rights (applies to Mac devices only). | Optional | +| unenroll_devices_and_free_licenses | Indicates whether a Windows device should be unenrolled after the Wipe is complete. Possible values are: true, false. Default is false. | Optional | + +#### Context Output + +There is no context output for this command. + +### absolute-wipe-request-cancel + +*** +Attempt to cancel all Wipe actions with an actionStatus of Pending for the request specified by the requestUid path parameter. + +#### Base Command + +`absolute-wipe-request-cancel` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| request_uid | The system-defined unique identifier of the Device Action request. | Required | +| action_uids | The system-defined unique identifiers of the actions included in the request. | Optional | +| cancel_all_actions | Indicates whether all actions in the request should be canceled. Possible values are: true, false. Default is false. | Optional | + +#### Context Output + +There is no context output for this command. + +<~PLATFORM> + +## License Requirements + +The following configuration parameters require the **Cortex XSIAM** license: + +- Fetch Events + + diff --git a/Packs/Absolute/Integrations/Absolute/test_data/cancel_wipe_request.json b/Packs/Absolute/Integrations/Absolute/test_data/cancel_wipe_request.json new file mode 100644 index 000000000000..2de14b2469f1 --- /dev/null +++ b/Packs/Absolute/Integrations/Absolute/test_data/cancel_wipe_request.json @@ -0,0 +1,11 @@ +{ + "code": "INVALID_REQUEST", + "message": "invalid request", + "details": [ + { + "message": "invalid json", + "code": "INVALID_JSON", + "field": "deviceUids" + } + ] +} \ No newline at end of file diff --git a/Packs/Absolute/Integrations/Absolute/test_data/create_wipe_request.json b/Packs/Absolute/Integrations/Absolute/test_data/create_wipe_request.json new file mode 100644 index 000000000000..891892503958 --- /dev/null +++ b/Packs/Absolute/Integrations/Absolute/test_data/create_wipe_request.json @@ -0,0 +1,10 @@ +{ + "data": { + "requestUid": "23dbd460-0547-466d-a655-99d9340ff598" + }, + "metadata": { + "pagination": { + "nextPage": "string" + } + } +} \ No newline at end of file diff --git a/Packs/Absolute/Integrations/Absolute/test_data/wipe_actions.json b/Packs/Absolute/Integrations/Absolute/test_data/wipe_actions.json new file mode 100644 index 000000000000..35491472a37e --- /dev/null +++ b/Packs/Absolute/Integrations/Absolute/test_data/wipe_actions.json @@ -0,0 +1,13 @@ +[ + { + "actionUid": "7f0741c5-fbe9-4144-8c54-498db6559304", + "requestUid": "1", + "deviceUid": "2", + "deviceName": "string", + "esn": "string", + "updatedDateTimeUtc": "2019-08-24T14:15:22Z", + "createdDateTimeUtc": "2019-08-24T14:15:22Z", + "actionStatus": "Pending", + "certificateUid": "string" + } +] \ No newline at end of file diff --git a/Packs/Absolute/Integrations/Absolute/test_data/wipe_requests.json b/Packs/Absolute/Integrations/Absolute/test_data/wipe_requests.json new file mode 100644 index 000000000000..1efc3fcef0c6 --- /dev/null +++ b/Packs/Absolute/Integrations/Absolute/test_data/wipe_requests.json @@ -0,0 +1,20 @@ + +[ + { + "totalDevices": 0, + "pending": 0, + "processing": 0, + "completed": 0, + "canceled": 0, + "failed": 0, + "requestId": "string", + "requestUid": "23dbd460-0547-466d-a655-99d9340ff598", + "requestStatus": "string", + "createdDateTimeUtc": "2019-08-24T14:15:22Z", + "updatedDateTimeUtc": "2019-08-24T14:15:22Z", + "requester": "string", + "macUsername": "string", + "macPwd": "string", + "unenrollDevicesAndFreeLicenses": true + } + ] diff --git a/Packs/Absolute/ModelingRules/Absolute/Absolute.yml b/Packs/Absolute/ModelingRules/Absolute/Absolute.yml index 9ae7613c700c..e85fef9ec9ef 100644 --- a/Packs/Absolute/ModelingRules/Absolute/Absolute.yml +++ b/Packs/Absolute/ModelingRules/Absolute/Absolute.yml @@ -1,4 +1,5 @@ fromversion: 8.4.0 +toversion: 8.14.0 id: Absolute_ModelingRule name: Absolute Modeling Rule rules: '' diff --git a/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11.xif b/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11.xif new file mode 100644 index 000000000000..cb45235510c8 --- /dev/null +++ b/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11.xif @@ -0,0 +1,52 @@ +[MODEL: dataset = absolute_secure_endpoint_raw] +alter + get_IpAddress_ipv4 = if(parsed_fields_NewValue -> IpAddress ~= "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}", parsed_fields_NewValue -> IpAddress), + get_IpAddress_ipv6 = if(parsed_fields_NewValue -> IpAddress ~= "\:", parsed_fields_NewValue -> IpAddress), + get_Longitude = if(objectDisplayName = "Device Location", parsed_fields_OldValue -> Longitude), + get_Latitude = if(objectDisplayName = "Device Location", parsed_fields_OldValue -> Latitude) +| alter +// General Fields + xdm.event.id = id, + xdm.event.type = eventType, + xdm.event.operation_sub_type = verb, + xdm.source.host.device_id = if(actorObjectType = "Device", actorDisplayId), + xdm.source.user.identifier = if(actorObjectType = "User", actorDisplayId), + xdm.source.identity.identifier = if(actorObjectType = "User", actorDisplayId), + xdm.source.application.version = if(actorObjectType = "System", actorDisplayId), + xdm.source.user.identity_type = if( + actorObjectType = "Device", XDM_CONST.IDENTITY_TYPE_MACHINE, + actorObjectType = "User", XDM_CONST.IDENTITY_TYPE_USER, + actorObjectType = "System", XDM_CONST.IDENTITY_TYPE_BUILTIN, + actorObjectType = "NA", XDM_CONST.IDENTITY_TYPE_UNKNOWN, + actorObjectType = null, null, to_string(actorObjectType)), + xdm.source.identity.identity_type = if( + actorObjectType = "Device", XDM_CONST.IDENTITY_TYPE_MACHINE, + actorObjectType = "User", XDM_CONST.IDENTITY_TYPE_USER, + actorObjectType = "System", XDM_CONST.IDENTITY_TYPE_BUILTIN, + actorObjectType = "NA", XDM_CONST.IDENTITY_TYPE_UNKNOWN, + actorObjectType = null, null, to_string(actorObjectType)), + xdm.source.host.hostname = if(actorObjectType = "Device", actorDisplayName), + xdm.source.user.username = if(actorObjectType = "User", actorDisplayName), + xdm.source.identity.username = if(actorObjectType = "User", actorDisplayName), + xdm.source.application.name = if(actorObjectType = "System", actorDisplayName), + xdm.target.resource.id = objectDisplayId, + xdm.target.resource.type = objectObjectType, + xdm.target.resource.name = objectDisplayName, + xdm.target.resource.sub_type = to_string(object_create( + "secondaryObjectDisplayId", secondaryObjectDisplayId, + "secondaryObjectDisplayName", secondaryObjectDisplayName, + "secondaryObjectType",secondaryObjectType)), +// Fields extracted from objectProperties + xdm.target.host.ipv6_addresses = arraycreate(parsed_fields_NewValue -> DeviceSystemInformationUpdated_LocalIpV6Addr), + xdm.target.host.ipv6_public_addresses = arraycreate(parsed_fields_NewValue -> DeviceSystemInformationUpdated_PublicIpV6Addr), + xdm.target.host.ipv4_public_addresses = arraycreate(parsed_fields_NewValue -> DeviceSystemInformationUpdated_PublicIpAddr), + xdm.target.host.ipv4_addresses = arraycreate(parsed_fields_NewValue -> DeviceSystemInformationUpdated_LocalIpAddr), + xdm.source.ipv4 = get_IpAddress_ipv4, + xdm.source.ipv6 = get_IpAddress_ipv6, + xdm.source.user_agent = parsed_fields_NewValue -> BrowserAgent, + xdm.source.location.longitude = to_float(get_Longitude), + xdm.source.location.latitude = to_float(get_Latitude), + xdm.source.location.city = if(objectDisplayName = "Device Location", parsed_fields_OldValue -> City), + xdm.source.location.country = if(objectDisplayName = "Device Location", parsed_fields_OldValue -> CountryName), + xdm.target.resource_before.value = to_string(parsed_fields_OldValue), + xdm.target.resource.value = to_string(parsed_fields_NewValue); \ No newline at end of file diff --git a/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11.yml b/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11.yml new file mode 100644 index 000000000000..99f6197b95a4 --- /dev/null +++ b/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: Absolute_ModelingRule +name: Absolute Modeling Rule +rules: '' +schema: '' +tags: '' +supportedModules: +- xsiam diff --git a/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11_schema.json b/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11_schema.json new file mode 100644 index 000000000000..4f55d8d92cf0 --- /dev/null +++ b/Packs/Absolute/ModelingRules/Absolute_2_11/Absolute_2_11_schema.json @@ -0,0 +1,64 @@ +{ + "absolute_secure_endpoint_raw": { + "parsed_fields_NewValue": { + "type": "string", + "is_array": false + }, + "parsed_fields_OldValue": { + "type": "string", + "is_array": false + }, + "id": { + "type": "string", + "is_array": false + }, + "verb": { + "type": "string", + "is_array": false + }, + "eventType": { + "type": "string", + "is_array": false + }, + "actorDisplayId": { + "type": "string", + "is_array": false + }, + "actorObjectType": { + "type": "string", + "is_array": false + }, + "objectDisplayId": { + "type": "string", + "is_array": false + }, + "actorDisplayName": { + "type": "string", + "is_array": false + }, + "objectObjectType": { + "type": "string", + "is_array": false + }, + "objectDisplayName": { + "type": "string", + "is_array": false + }, + "secondaryObjectType": { + "type": "string", + "is_array": false + }, + "secondaryObjectDisplayName": { + "type": "string", + "is_array": false + }, + "secondaryObjectDisplayId": { + "type": "string", + "is_array": false + }, + "objectProperties": { + "type": "string", + "is_array": false + } + } +} diff --git a/Packs/Absolute/ReleaseNotes/2_0_12.md b/Packs/Absolute/ReleaseNotes/2_0_12.md new file mode 100644 index 000000000000..c0a6628fa084 --- /dev/null +++ b/Packs/Absolute/ReleaseNotes/2_0_12.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: Absolute Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/Absolute/ReleaseNotes/2_0_13.md b/Packs/Absolute/ReleaseNotes/2_0_13.md new file mode 100644 index 000000000000..42540da35bfc --- /dev/null +++ b/Packs/Absolute/ReleaseNotes/2_0_13.md @@ -0,0 +1,9 @@ + +#### Integrations + +##### Absolute + +- Added support for the ***absolute-wipe-request-list*** command that get information about the wipe requests. +- Added support for the ***absolute-wipe-actions-list*** command that get the status of actions by devices or by request. +- Added support for the ***absolute-wipe-request-create*** command that create a wipe request for the devices. +- Added support for the ***absolute-wipe-request-cancel*** command that attempts to cancel all wipe actions with an action status of pending for the request specified by the *request_uid* path parameter. diff --git a/Packs/Absolute/pack_metadata.json b/Packs/Absolute/pack_metadata.json index cb0c5f08b2fd..e260722a62fe 100644 --- a/Packs/Absolute/pack_metadata.json +++ b/Packs/Absolute/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Absolute", "description": "Absolute is an adaptive endpoint security solution that delivers device security, data security and asset management of endpoints", "support": "xsoar", - "currentVersion": "2.0.11", + "currentVersion": "2.0.13", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AbuseDB/ReleaseNotes/1_1_7.md b/Packs/AbuseDB/ReleaseNotes/1_1_7.md new file mode 100644 index 000000000000..0d133b538f05 --- /dev/null +++ b/Packs/AbuseDB/ReleaseNotes/1_1_7.md @@ -0,0 +1,7 @@ + +#### Scripts + +##### AbuseIPDBPopulateIndicators + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + diff --git a/Packs/AbuseDB/ReleaseNotes/1_1_8.md b/Packs/AbuseDB/ReleaseNotes/1_1_8.md new file mode 100644 index 000000000000..7f06baea36f3 --- /dev/null +++ b/Packs/AbuseDB/ReleaseNotes/1_1_8.md @@ -0,0 +1,7 @@ + +#### Scripts + +##### AbuseIPDBPopulateIndicators + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + diff --git a/Packs/AbuseDB/Scripts/AbuseIPDBPopulateIndicators/AbuseIPDBPopulateIndicators.yml b/Packs/AbuseDB/Scripts/AbuseIPDBPopulateIndicators/AbuseIPDBPopulateIndicators.yml index 9cdc78436a99..3df275eefc5f 100644 --- a/Packs/AbuseDB/Scripts/AbuseIPDBPopulateIndicators/AbuseIPDBPopulateIndicators.yml +++ b/Packs/AbuseDB/Scripts/AbuseIPDBPopulateIndicators/AbuseIPDBPopulateIndicators.yml @@ -28,4 +28,4 @@ runas: DBotWeakRole tests: - AbuseIPDB PopulateIndicators Test fromversion: 5.0.0 -dockerimage: demisto/python3:3.12.13.10404775 +dockerimage: demisto/python3:3.12.14.13495286 diff --git a/Packs/AbuseDB/pack_metadata.json b/Packs/AbuseDB/pack_metadata.json index 23dcdbd6e439..369cf74eb29a 100644 --- a/Packs/AbuseDB/pack_metadata.json +++ b/Packs/AbuseDB/pack_metadata.json @@ -2,7 +2,7 @@ "name": "AbuseIPDB", "description": "Central repository to report and identify IP addresses that have been associated with malicious activity online. Check the Detailed Information section for more information on how to configure the integration.", "support": "xsoar", - "currentVersion": "1.1.6", + "currentVersion": "1.1.8", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AccessInvestigation/ReleaseNotes/1_2_20.md b/Packs/AccessInvestigation/ReleaseNotes/1_2_20.md new file mode 100644 index 000000000000..4a0c5ba98e57 --- /dev/null +++ b/Packs/AccessInvestigation/ReleaseNotes/1_2_20.md @@ -0,0 +1,3 @@ +## Access Investigation + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AccessInvestigation/pack_metadata.json b/Packs/AccessInvestigation/pack_metadata.json index 5b581313d239..fa57e09e5a59 100644 --- a/Packs/AccessInvestigation/pack_metadata.json +++ b/Packs/AccessInvestigation/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Access Investigation", "description": "This Content Pack automates response to unauthorised access incidents and contains customer access incident views and layouts to aid investigation.", "support": "xsoar", - "currentVersion": "1.2.19", + "currentVersion": "1.2.20", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/ActiveMQ/Integrations/ActiveMQ/ActiveMQ.yml b/Packs/ActiveMQ/Integrations/ActiveMQ/ActiveMQ.yml index 7cf1d713395a..2bb185069797 100644 --- a/Packs/ActiveMQ/Integrations/ActiveMQ/ActiveMQ.yml +++ b/Packs/ActiveMQ/Integrations/ActiveMQ/ActiveMQ.yml @@ -60,6 +60,13 @@ configuration: type: 13 required: false section: Collect +- display: Incidents Fetch Interval + name: incidentFetchInterval + defaultvalue: '1' + required: false + type: 19 + section: Collect + advanced: true - display: Topic Name (for subscription) name: topic-name type: 0 diff --git a/Packs/ActiveMQ/ReleaseNotes/1_1_21.md b/Packs/ActiveMQ/ReleaseNotes/1_1_21.md new file mode 100644 index 000000000000..d44db4d3b155 --- /dev/null +++ b/Packs/ActiveMQ/ReleaseNotes/1_1_21.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### ActiveMQ + +- Updated the Fetch Incidents configuration parameters. diff --git a/Packs/ActiveMQ/pack_metadata.json b/Packs/ActiveMQ/pack_metadata.json index 5955e10c9135..6206929ab9b8 100644 --- a/Packs/ActiveMQ/pack_metadata.json +++ b/Packs/ActiveMQ/pack_metadata.json @@ -2,7 +2,7 @@ "name": "ActiveMQ", "description": "Uses Durable Topic Subscribers to fetch messages and ingest them as incidents in Demisto.", "support": "xsoar", - "currentVersion": "1.1.20", + "currentVersion": "1.1.21", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/Active_Directory_Query/.secrets-ignore b/Packs/Active_Directory_Query/.secrets-ignore index c4ddce660f8d..c2d9c0ba9ba8 100644 --- a/Packs/Active_Directory_Query/.secrets-ignore +++ b/Packs/Active_Directory_Query/.secrets-ignore @@ -14,3 +14,4 @@ test@demisto.int user@test.local manager@test.local eric mann +local admin diff --git a/Packs/Active_Directory_Query/ReleaseNotes/1_8_1.md b/Packs/Active_Directory_Query/ReleaseNotes/1_8_1.md new file mode 100644 index 000000000000..3c0f4ad40bb5 --- /dev/null +++ b/Packs/Active_Directory_Query/ReleaseNotes/1_8_1.md @@ -0,0 +1,7 @@ + +#### Scripts + +##### IAMInitADUser + +- Updated the Docker image to: *demisto/py3-tools:1.0.0.11597437*. + diff --git a/Packs/Active_Directory_Query/ReleaseNotes/1_8_2.md b/Packs/Active_Directory_Query/ReleaseNotes/1_8_2.md new file mode 100644 index 000000000000..d78dbdae3139 --- /dev/null +++ b/Packs/Active_Directory_Query/ReleaseNotes/1_8_2.md @@ -0,0 +1,11 @@ + +#### Scripts + +##### IAMInitADUser + +- Updated the Docker image to: *demisto/py3-tools:1.0.0.13176850*. + +##### SendEmailToManager + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + diff --git a/Packs/Active_Directory_Query/ReleaseNotes/1_8_3.md b/Packs/Active_Directory_Query/ReleaseNotes/1_8_3.md new file mode 100644 index 000000000000..bfeb2caaf231 --- /dev/null +++ b/Packs/Active_Directory_Query/ReleaseNotes/1_8_3.md @@ -0,0 +1,3 @@ +## Active Directory Query + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/Active_Directory_Query/Scripts/IAMInitADUser/IAMInitADUser.yml b/Packs/Active_Directory_Query/Scripts/IAMInitADUser/IAMInitADUser.yml index b66729f29728..e7554ebce83c 100644 --- a/Packs/Active_Directory_Query/Scripts/IAMInitADUser/IAMInitADUser.yml +++ b/Packs/Active_Directory_Query/Scripts/IAMInitADUser/IAMInitADUser.yml @@ -80,7 +80,7 @@ tags: - active directory - Utility type: python -dockerimage: demisto/py3-tools:1.0.0.10120494 +dockerimage: demisto/py3-tools:1.0.0.13176850 runas: DBotWeakRole tests: - Active Directory Test diff --git a/Packs/Active_Directory_Query/Scripts/SendEmailToManager/SendEmailToManager.yml b/Packs/Active_Directory_Query/Scripts/SendEmailToManager/SendEmailToManager.yml index 1d47d2c81a3e..f2dc6439de39 100644 --- a/Packs/Active_Directory_Query/Scripts/SendEmailToManager/SendEmailToManager.yml +++ b/Packs/Active_Directory_Query/Scripts/SendEmailToManager/SendEmailToManager.yml @@ -42,7 +42,7 @@ dependson: timeout: 0s tests: - No test -dockerimage: demisto/python3:3.12.13.10116658 +dockerimage: demisto/python3:3.12.14.13053055 supportedModules: - agentix - xsiam diff --git a/Packs/Active_Directory_Query/pack_metadata.json b/Packs/Active_Directory_Query/pack_metadata.json index e733cbcbbbb0..36648de9846c 100644 --- a/Packs/Active_Directory_Query/pack_metadata.json +++ b/Packs/Active_Directory_Query/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Active Directory Query", "description": "Active Directory Query integration enables you to access and manage Active Directory objects (users, contacts, and computers).", "support": "xsoar", - "currentVersion": "1.8.0", + "currentVersion": "1.8.3", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/AdminByRequest/.secrets-ignore b/Packs/AdminByRequest/.secrets-ignore index e69de29bb2d1..f954b9cc4d6d 100644 --- a/Packs/AdminByRequest/.secrets-ignore +++ b/Packs/AdminByRequest/.secrets-ignore @@ -0,0 +1 @@ +local admin \ No newline at end of file diff --git a/Packs/AdminByRequest/Integrations/AdminByRequestEventCollector/README.md b/Packs/AdminByRequest/Integrations/AdminByRequestEventCollector/README.md index 52ac7bc41fa2..fc85bccbcf82 100644 --- a/Packs/AdminByRequest/Integrations/AdminByRequestEventCollector/README.md +++ b/Packs/AdminByRequest/Integrations/AdminByRequestEventCollector/README.md @@ -1,7 +1,5 @@ AdminByRequest is a Privileged Access Management (PAM) solution that enables secure, temporary elevation to local admin rights. -This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM. - ## Configure Admin By Request in Cortex | **Parameter** | **Description** | **Required** | diff --git a/Packs/AdminByRequest/ModelingRules/AdminByRequest/AdminByRequest.yml b/Packs/AdminByRequest/ModelingRules/AdminByRequest/AdminByRequest.yml index 73dd6ccc1ade..3351cd5f0444 100644 --- a/Packs/AdminByRequest/ModelingRules/AdminByRequest/AdminByRequest.yml +++ b/Packs/AdminByRequest/ModelingRules/AdminByRequest/AdminByRequest.yml @@ -1,4 +1,5 @@ fromversion: 6.10.0 +toversion: 8.14.0 id: Admin_By_Request_ModelingRule name: Admin By Request Modeling Rule rules: '' diff --git a/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11.xif b/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11.xif new file mode 100644 index 000000000000..d8f908be1993 --- /dev/null +++ b/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11.xif @@ -0,0 +1,47 @@ +[MODEL: dataset=admin_by_request_raw] +filter source_log_type = "auditlog" +| alter additional_data = object_create("Application_Name", application -> name, "Application_Vendor", application -> vendor, "Application_Threat" , application -> threat, "Application_VT_Link", application -> virustotalLink, "Request_Reason", reason, "Approved_By", approvedBy , "Approved_By_Email", approvedByEmail , "Denied_By", deniedBy , "Denied_By_Email", deniedByEmail , "Denied_Reason", deniedReason,"Audit_Log_URL", auditLogURL) +| alter + xdm.target.file.filename = application -> file, + xdm.target.file.path = application -> path, + xdm.target.file.sha256 = application -> ["sha256"], + xdm.target.host.os = computer -> platform, + xdm.event.outcome_reason = application -> scanResult, + xdm.event.description = additional_data, + xdm.target.host.hostname = computer -> name, + xdm.event.type = source_log_type, + xdm.event.operation = type, + xdm.source.user.username = user -> account, + xdm.source.identity.username = user -> account, + xdm.source.user.upn = user -> email, + xdm.source.identity.upn = user -> email; +filter source_log_type = "events" +| alter + event_log_level = to_string(eventLevel), + additional_data = object_create("Application_Name", application -> name, "Application_Vendor", application -> vendor, "Additional_Data", additionalData) +| alter + xdm.target.file.filename = application -> file, + xdm.target.file.path = application -> path, + xdm.target.host.hostname = computerName, + xdm.event.log_level = if(event_log_level = "0", XDM_CONST.LOG_LEVEL_INFORMATIONAL , event_log_level = "1", XDM_CONST.LOG_LEVEL_WARNING , event_log_level = "2", XDM_CONST.LOG_LEVEL_ALERT), + xdm.event.type = source_log_type, + xdm.event.description = additional_data, + xdm.source.user.username = coalesce(userAccount,userName), + xdm.source.identity.username = coalesce(userAccount,userName), + xdm.event.operation = eventText; +filter source_log_type = "request" +| alter additional_data = object_create("Application_Name", application -> name, "Application_Vendor", application -> vendor, "Application_Threat" , application -> threat, "Application_VT_Link", application -> virustotalLink, "Request_Reason", reason, "Approved_By" ,approvedBy , "Approved_By_Email", approvedByEmail , "Denied_By", deniedBy , "Denied_By_Email", deniedByEmail , "Denied_Reason", deniedReason, "Audit_Log_Link", auditlogLink) +| alter + xdm.target.file.filename = application -> file, + xdm.target.file.path = application -> path, + xdm.target.file.sha256 = application -> ["sha256"], + xdm.event.outcome_reason = application -> scanResult, + xdm.target.host.os = computer -> platform, + xdm.event.description = additional_data, + xdm.target.host.hostname = computer -> name, + xdm.event.type = source_log_type, + xdm.source.user.username = user -> account, + xdm.source.identity.username = user -> account, + xdm.source.user.upn = user -> email, + xdm.source.identity.upn = user -> email, + xdm.event.operation = type; \ No newline at end of file diff --git a/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11.yml b/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11.yml new file mode 100644 index 000000000000..01da725efb5f --- /dev/null +++ b/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: Admin_By_Request_ModelingRule +name: Admin By Request Modeling Rule +rules: '' +schema: '' +tags: 'Admin By Request' +supportedModules: +- xsiam diff --git a/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11_schema.json b/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11_schema.json new file mode 100644 index 000000000000..a45d3754e59a --- /dev/null +++ b/Packs/AdminByRequest/ModelingRules/AdminByRequest_2_11/AdminByRequest_2_11_schema.json @@ -0,0 +1,80 @@ +{ + "admin_by_request_raw": { + "application": { + "type": "string", + "is_array": false + }, + "computer": { + "type": "string", + "is_array": false + }, + "auditLogURL": { + "type": "string", + "is_array": false + }, + "source_log_type": { + "type": "string", + "is_array": false + }, + "type": { + "type": "string", + "is_array": false + }, + "user": { + "type": "string", + "is_array": false + }, + "computerName": { + "type": "string", + "is_array": false + }, + "eventLevel": { + "type": "string", + "is_array": false + }, + "additionalData": { + "type": "string", + "is_array": false + }, + "userAccount": { + "type": "string", + "is_array": false + }, + "userName": { + "type": "string", + "is_array": false + }, + "reason": { + "type": "string", + "is_array": false + }, + "eventText": { + "type": "string", + "is_array": false + }, + "auditlogLink": { + "type": "string", + "is_array": false + }, + "approvedBy": { + "type": "string", + "is_array": false + }, + "approvedByEmail": { + "type": "string", + "is_array": false + }, + "deniedBy": { + "type": "string", + "is_array": false + }, + "deniedByEmail": { + "type": "string", + "is_array": false + }, + "deniedReason": { + "type": "string", + "is_array": false + } + } +} \ No newline at end of file diff --git a/Packs/AdminByRequest/ReleaseNotes/1_0_10.md b/Packs/AdminByRequest/ReleaseNotes/1_0_10.md new file mode 100644 index 000000000000..6ad1bbba4a99 --- /dev/null +++ b/Packs/AdminByRequest/ReleaseNotes/1_0_10.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: AdminByRequest Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/AdminByRequest/ReleaseNotes/1_0_11.md b/Packs/AdminByRequest/ReleaseNotes/1_0_11.md new file mode 100644 index 000000000000..ac2edc379bf8 --- /dev/null +++ b/Packs/AdminByRequest/ReleaseNotes/1_0_11.md @@ -0,0 +1,4 @@ + +## AdminByRequest + +- Documentation and metadata improvements. diff --git a/Packs/AdminByRequest/ReleaseNotes/1_0_12.md b/Packs/AdminByRequest/ReleaseNotes/1_0_12.md new file mode 100644 index 000000000000..88eff6ff3db5 --- /dev/null +++ b/Packs/AdminByRequest/ReleaseNotes/1_0_12.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: Admin By Request Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/AdminByRequest/ReleaseNotes/1_0_13.md b/Packs/AdminByRequest/ReleaseNotes/1_0_13.md new file mode 100644 index 000000000000..4ce960839538 --- /dev/null +++ b/Packs/AdminByRequest/ReleaseNotes/1_0_13.md @@ -0,0 +1,3 @@ +## Admin By Request + +- Documentation and metadata improvements. diff --git a/Packs/AdminByRequest/pack_metadata.json b/Packs/AdminByRequest/pack_metadata.json index 4563d4f9d815..77b81be84588 100644 --- a/Packs/AdminByRequest/pack_metadata.json +++ b/Packs/AdminByRequest/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Admin By Request", "description": "AdminByRequest is a Privileged Access Management (PAM) solution that enables secure, temporary elevation to local admin rights.", "support": "xsoar", - "currentVersion": "1.0.9", + "currentVersion": "1.0.13", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -19,9 +19,8 @@ "marketplacev2", "xsoar" ], - "defaultDataSource": "AdminByRequest", "supportedModules": [ "agentix", "xsiam" ] -} \ No newline at end of file +} diff --git a/Packs/Aella_StarLight/Integrations/AellaStarLight/AellaStarLight.yml b/Packs/Aella_StarLight/Integrations/AellaStarLight/AellaStarLight.yml index 86ea047bb9c8..7c785962dae4 100644 --- a/Packs/Aella_StarLight/Integrations/AellaStarLight/AellaStarLight.yml +++ b/Packs/Aella_StarLight/Integrations/AellaStarLight/AellaStarLight.yml @@ -32,6 +32,13 @@ configuration: type: 13 required: false section: Collect +- display: Incidents Fetch Interval + name: incidentFetchInterval + defaultvalue: '1' + required: false + type: 19 + section: Collect + advanced: true - display: Fetching interval in minutes (default is 15, minimum is 15 ) name: fetch_interval section: Collect diff --git a/Packs/Aella_StarLight/ReleaseNotes/1_0_21.md b/Packs/Aella_StarLight/ReleaseNotes/1_0_21.md new file mode 100644 index 000000000000..bff89269999d --- /dev/null +++ b/Packs/Aella_StarLight/ReleaseNotes/1_0_21.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### Aella Star Light + +- Updated the Fetch Incidents configuration parameters. diff --git a/Packs/Aella_StarLight/pack_metadata.json b/Packs/Aella_StarLight/pack_metadata.json index f676baa5a343..9703214dab12 100644 --- a/Packs/Aella_StarLight/pack_metadata.json +++ b/Packs/Aella_StarLight/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Aella Star Light", "description": "Aella Star Light Integration", "support": "community", - "currentVersion": "1.0.20", + "currentVersion": "1.0.21", "author": "Aella Star Light", "url": "", "email": "", diff --git a/Packs/AgariPhishingDefense/Integrations/AgariPhishingDefense/AgariPhishingDefense.yml b/Packs/AgariPhishingDefense/Integrations/AgariPhishingDefense/AgariPhishingDefense.yml index 9288790db2fd..3bac221e592a 100644 --- a/Packs/AgariPhishingDefense/Integrations/AgariPhishingDefense/AgariPhishingDefense.yml +++ b/Packs/AgariPhishingDefense/Integrations/AgariPhishingDefense/AgariPhishingDefense.yml @@ -3,33 +3,44 @@ provider: Fortra commonfields: id: Agari Phishing Defense version: -1 +sectionorder: +- Connect +- Collect configuration: - defaultvalue: https://api.agari.com/v1/ep display: URL to connect to Agari name: url required: true type: 0 + section: Connect - additionalinfo: API Client ID created in the User management UI. display: API Key name: apikey required: true type: 4 + section: Connect - additionalinfo: API Client Secret created in the User management UI. display: Secret Key name: apisecret required: true type: 4 + section: Connect - defaultvalue: '50' display: Maximum number of incidents to fetch every time name: max_fetch required: true type: 0 + section: Collect + supportedModules: + - agentix + - xsiam - additionalinfo: The time range to consider for the initial data fetch. ( , e.g., 2 minutes, 2 hours, 2 days, 2 months, 2 years). Default is 3 days. defaultvalue: 3 days display: First fetch time interval name: first_fetch type: 0 required: false + section: Collect - additionalinfo: Apply the filter to fetch policy events having actions such as "deliver", "mark-spam", "move", "inbox", "delete", "none". Fetches all types of actions by default. display: Policy action taken name: fetch_policy_actions @@ -42,6 +53,7 @@ configuration: - none type: 15 required: false + section: Collect - additionalinfo: Exclude alerts while fetching policy events of type 'Message Alert' or 'System Alert'. defaultvalue: System Alert display: Exclude alerts @@ -51,28 +63,50 @@ configuration: - Message Alert type: 15 required: false + section: Collect - additionalinfo: "Syntax: field.operator(operand) {and/or field.operator(operand)} \n\nExample: created_at.after(2020-11-11T09:16:44Z) or id.eq(537557694)" display: Fetches policy events as per the filter to limit the amount of data. Can be applied to specific fields. name: policy_filter type: 12 required: false + section: Collect - defaultvalue: Agari Phishing Defense Policy Event display: Incident type name: incidentType type: 13 required: false + section: Collect + supportedModules: + - agentix + - xsiam +- display: Incidents Fetch Interval + name: incidentFetchInterval + defaultvalue: '1' + required: false + type: 19 + section: Collect + advanced: true + supportedModules: + - agentix + - xsiam - display: Fetch incidents name: isFetch type: 8 required: false + section: Collect + supportedModules: + - agentix + - xsiam - display: Trust any certificate (not secure) name: insecure type: 8 required: false + section: Connect - display: Use system proxy settings name: proxy type: 8 required: false + section: Connect description: Agari Phishing Defense stops phishing, BEC, and other identity deception attacks that trick employees into harming your business. display: Agari Phishing Defense name: Agari Phishing Defense diff --git a/Packs/AgariPhishingDefense/ReleaseNotes/1_1_28.md b/Packs/AgariPhishingDefense/ReleaseNotes/1_1_28.md new file mode 100644 index 000000000000..78398a3f8696 --- /dev/null +++ b/Packs/AgariPhishingDefense/ReleaseNotes/1_1_28.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### Agari Phishing Defense + +- Documentation and metadata improvements. diff --git a/Packs/AgariPhishingDefense/pack_metadata.json b/Packs/AgariPhishingDefense/pack_metadata.json index 0483ad9840f7..c43c3c1c4718 100644 --- a/Packs/AgariPhishingDefense/pack_metadata.json +++ b/Packs/AgariPhishingDefense/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Agari Phishing Defense", "description": "Use the Agari Phishing Defense integration to retrieve Policy Events as Incidents, retrieve messages and remediate suspected messages.", "support": "partner", - "currentVersion": "1.1.27", + "currentVersion": "1.1.28", "author": "Agari", "url": "https://www.agari.com/support/", "email": "support@agari.com", diff --git a/Packs/AggregatedScripts/.pack-ignore b/Packs/AggregatedScripts/.pack-ignore index 7b5bf9e92e79..0760cc5343e5 100644 --- a/Packs/AggregatedScripts/.pack-ignore +++ b/Packs/AggregatedScripts/.pack-ignore @@ -15,3 +15,6 @@ ignore=BA127 [file:IndicatorEnrichment.yml] ignore=BA127,SC101 + +[file:BlockDomain.yml] +ignore=BA129 diff --git a/Packs/AggregatedScripts/ReleaseNotes/1_3_53.md b/Packs/AggregatedScripts/ReleaseNotes/1_3_53.md new file mode 100644 index 000000000000..0f6f6b308ef8 --- /dev/null +++ b/Packs/AggregatedScripts/ReleaseNotes/1_3_53.md @@ -0,0 +1,7 @@ +<~PLATFORM> +#### Scripts + +##### quarantine-file + +- Updated the script to run the Cortex Core quarantine file action as a Cortex platform built-in command. On the platform, the executing brand is now reported as "Cortex Builtin". + diff --git a/Packs/AggregatedScripts/ReleaseNotes/1_4_0.md b/Packs/AggregatedScripts/ReleaseNotes/1_4_0.md new file mode 100644 index 000000000000..1a4e772ffc65 --- /dev/null +++ b/Packs/AggregatedScripts/ReleaseNotes/1_4_0.md @@ -0,0 +1,6 @@ + +#### Scripts + +##### New: block-domain + +- Added the **block-domain** script, which blocks one or more domains across your configured security products. Currently only supports Palo Alto Networks Panorama. diff --git a/Packs/AggregatedScripts/ReleaseNotes/1_4_1.md b/Packs/AggregatedScripts/ReleaseNotes/1_4_1.md new file mode 100644 index 000000000000..c288ef99d31f --- /dev/null +++ b/Packs/AggregatedScripts/ReleaseNotes/1_4_1.md @@ -0,0 +1,6 @@ + +#### Scripts + +##### block-url + +New: Added a script that blocks a list of URLs in supported integrations. diff --git a/Packs/AggregatedScripts/ReleaseNotes/1_4_2.md b/Packs/AggregatedScripts/ReleaseNotes/1_4_2.md new file mode 100644 index 000000000000..3a23f01957e7 --- /dev/null +++ b/Packs/AggregatedScripts/ReleaseNotes/1_4_2.md @@ -0,0 +1,3 @@ +## Aggregated Scripts + +- Locked dependencies of the pack to ensure stability for versioned core packs. No changes in this release. diff --git a/Packs/AggregatedScripts/ReleaseNotes/1_4_3.md b/Packs/AggregatedScripts/ReleaseNotes/1_4_3.md new file mode 100644 index 000000000000..abb525bdf688 --- /dev/null +++ b/Packs/AggregatedScripts/ReleaseNotes/1_4_3.md @@ -0,0 +1,63 @@ + +#### Scripts + +##### expire-password + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### url-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### quarantine-file + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### clear-user-session + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### file-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### get-user-data + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### domain-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### ip-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### isolate-endpoint + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### cve-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### indicator-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### disable-user + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### block-domain + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### get-endpoint-data + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + +##### block-external-ip + +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. + diff --git a/Packs/AggregatedScripts/ReleaseNotes/1_4_4.md b/Packs/AggregatedScripts/ReleaseNotes/1_4_4.md new file mode 100644 index 000000000000..e801215325b7 --- /dev/null +++ b/Packs/AggregatedScripts/ReleaseNotes/1_4_4.md @@ -0,0 +1,53 @@ + +#### Scripts + +##### isolate-endpoint + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### url-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### quarantine-file + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### block-url + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### file-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### get-user-data + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + + +##### cve-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### indicator-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### block-domain + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + + +##### clear-user-session + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### domain-enrichment + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + +##### block-external-ip + +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. + diff --git a/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain.py b/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain.py new file mode 100644 index 000000000000..b58259d9cdf9 --- /dev/null +++ b/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain.py @@ -0,0 +1,1152 @@ +import hashlib +import re +from typing import Any + +import demistomock as demisto # noqa: F401 +from CommonServerPython import * # noqa: F401 +from CommonServerUserPython import * # noqa: F401 + +""" CONSTANTS """ + +SUPPORTED_BRANDS = ["Panorama"] + +OBJECT_NAME_PREFIX = "Cortex-" +# PAN-OS object names are capped at 63 chars; reserve room for the prefix and hash suffix. +MAX_OBJECT_NAME_LENGTH = 63 +HASH_SUFFIX_LENGTH = 8 +OBJECT_NAME_SANITIZE_REGEX = re.compile(r"[^A-Za-z0-9.\-]") + +PRE_POST = "pre-rulebase" + +FQDN_REGEX = re.compile(r"^(?=.{1,253}$)(?!-)[A-Za-z0-9-]{1,63}(? bool: + """Check whether a domain is a wildcard (unsupported). + + Args: + domain (str): The domain to check. + Returns: + True if the domain contains an asterisk, False otherwise. + """ + return "*" in domain + + +def is_valid_fqdn(domain: str) -> bool: + """Check whether a value is a valid, non-wildcard FQDN. + + Args: + domain (str): The domain to validate. + Returns: + True if the value looks like a valid FQDN, False otherwise. + """ + return bool(FQDN_REGEX.match(domain)) + + +def derive_object_name(domain: str) -> str: + """Derive a deterministic PAN-OS address-object name from a domain. + + The name is a pure function of the domain so re-runs are idempotent. On overflow of the PAN-OS + max object-name length, the sanitised body is truncated and a short deterministic hash suffix is + appended to keep the name unique. + + Args: + domain (str): The domain to derive the object name from. + Returns: + The derived object name (for example, 'Cortex-evil.example.com'). + """ + sanitised = OBJECT_NAME_SANITIZE_REGEX.sub("-", domain).strip("-") + candidate = f"{OBJECT_NAME_PREFIX}{sanitised}" + if len(candidate) <= MAX_OBJECT_NAME_LENGTH: + return candidate + + digest = hashlib.sha256(domain.encode("utf-8")).hexdigest()[:HASH_SUFFIX_LENGTH] + keep = MAX_OBJECT_NAME_LENGTH - len(OBJECT_NAME_PREFIX) - 1 - HASH_SUFFIX_LENGTH # 1 for the '-' separator. + truncated = sanitised[:keep].strip("-") + return f"{OBJECT_NAME_PREFIX}{truncated}-{digest}" + + +def normalize_tags(raw_tags: Any) -> list[str]: + """Normalize a PAN-OS tags value into a clean list of tag names. + + PAN-OS may return an entity's tags as a single string, a list of strings, or None/absent. + This flattens all of those into a list (empty when there are no tags). + + Args: + raw_tags (Any): The raw 'Tags' value from a PAN-OS context entry. + Returns: + A list of tag-name strings (possibly empty). + """ + if not raw_tags: + return [] + if isinstance(raw_tags, list): + return [str(tag) for tag in raw_tags if tag] + return [str(raw_tags)] + + +def build_result_row( + domain: str, + brand: str | None, + status: str, + result: str, + message: str, + rule_name: str | None = None, +) -> dict: + """Assemble a single BlockDomain row. + + Any field with no meaningful value is stored as ``None`` (real JSON null) rather than an empty + string, so downstream context never carries empty-string placeholders (e.g. a failed-validation + row has ``Brand: null`` and ``RuleName: null`` instead of ``""``). + + Note: there is intentionally no ``Action`` field - the security team confirmed that ``Result`` + plus ``Message`` convey everything they need, so per-row Created/Modified/Unchanged is not + surfaced. Change-detection is still performed internally to gate the commit/push. + + Args: + domain (str): The processed domain. + brand (str | None): The brand used, or None when not applicable (e.g. failed validation). + status (str): The lifecycle status. + result (str): Success or Failed. + message (str): A human-readable message. + rule_name (str | None): The rule name used, or None when no rule was involved. + Returns: + A dict representing a single result row, with empty values normalised to None. + """ + return { + "Domain": domain or None, + "Brand": brand or None, + "Status": status or None, + "Result": result or None, + "RuleName": rule_name or None, + "Message": message or None, + } + + +def validate_domains(domain_list: list) -> tuple[list, list]: + """Split the input into valid domains and failed-validation rows. + + Wildcard and invalid entries fail validation and never reach a vendor; they produce a per-row + Failed result while the rest of the list continues. + + Args: + domain_list (list): The list of domains to validate. + Returns: + A tuple of (valid_domains, failed_rows). + """ + valid_domains: list = [] + failed_rows: list = [] + for domain in domain_list: + if is_wildcard(domain): + failed_rows.append( + build_result_row( + domain=domain, + brand=None, + status=STATUS_FAILED, + result=RESULT_FAILED, + message=f"Wildcard domain '{domain}' is not supported by this script; skipped.", + ) + ) + elif not is_valid_fqdn(domain): + failed_rows.append( + build_result_row( + domain=domain, + brand=None, + status=STATUS_FAILED, + result=RESULT_FAILED, + message=f"Invalid FQDN '{domain}' - skipped.", + ) + ) + else: + valid_domains.append(domain) + return valid_domains, failed_rows + + +def get_enabled_brands() -> set: + """Return the set of brands that have at least one active instance. + + Returns: + A set of enabled brand names. + """ + modules = demisto.getModules() + return {module.get("brand") for module in modules.values() if module.get("state") == "active"} + + +""" EXECUTE-COMMAND / CONTEXT HELPERS """ + + +def run_execute_command(command_name: str, args: dict[str, Any]) -> list[dict]: + """Execute a command and return its raw entries. + + Args: + command_name (str): The command to execute. + args (dict): The command arguments. + Returns: + The raw list of command entries. + Raises: + DemistoException: If the command returned no entries (callers index res[0] downstream). + """ + res = demisto.executeCommand(command_name, args) + if not res: + raise DemistoException(f"Command {command_name!r} returned an empty response.") + return res + + +def is_polling_in_progress(poll_response: Any) -> bool: + """Return whether a polling function's result indicates it should keep polling. + + The @polling_function decorator attaches a ScheduledCommand to the returned CommandResults only + while polling should continue; when the job finishes it returns the plain response with none. + + Args: + poll_response (Any): The value returned by a @polling_function-decorated call. + Returns: + True if polling should continue, False when the job has finished. + """ + return getattr(poll_response, "scheduled_command", None) is not None + + +def get_relevant_context(original_context: dict[str, Any], key: str) -> dict | list: + """Get the relevant context object from the execute_command response, tolerating suffixed keys. + + Args: + original_context (dict): The 'EntryContext' from the command response. + key (str): The key to extract. + Returns: + A dict or list that is the relevant command context. + """ + if not original_context: + return {} + if relevant_context := original_context.get(key, {}): + return relevant_context + for k in original_context: + if k.startswith(key): + return original_context.get(k, {}) + return {} + + +""" HUMAN-READABLE / FINAL-RESULT AGGREGATION """ + + +def build_verbose_human_readable(responses: list) -> str: + """Concatenate the per-command human-readable outputs into a single, blank-line-separated string. + + Args: + responses (list): The accumulated command responses (each a list of entries). + Returns: + A single markdown string with each command's human-readable output separated by a blank line, + or an empty string if no command produced human-readable output. + """ + human_readables: list = [] + for res in responses or []: + for entry in res or []: + command_hr = entry.get("HumanReadable") + if command_hr and command_hr != str(None): + human_readables.append(command_hr) + # A leading "" yields a blank line separating the summary table from the first verbose entry. + return "\n\n".join(["", *human_readables]) if human_readables else "" + + +def build_final_command_results(rows: list, verbose: bool, responses: list) -> CommandResults: + """Build the single final CommandResults for the run. + + The CommandResults carries the aggregated BlockDomain context and a markdown summary table. + When verbose is True, the per-command human-readable outputs are appended to the same readable + output (blank-line separated), mirroring the ExpirePassword aggregated script. + + When there is at least one row and every row failed, the whole run is reported as an error. + + Args: + rows (list): The aggregated BlockDomain rows. + verbose (bool): Whether to append per-command human-readable output. + responses (list): The accumulated command responses (used only when verbose). + Returns: + A single CommandResults to return from the script. + """ + all_failed = bool(rows) and all(row.get("Result") == RESULT_FAILED for row in rows) + table_title = "Block Domain: All runs failed." if all_failed else "Block Domain" + readable_output = tableToMarkdown( + table_title, + rows, + headers=["Domain", "Brand", "Status", "Result", "RuleName", "Message"], + removeNull=True, + ) + if all_failed: + readable_output += "\n\n**All runs failed.** Review the table above for the specific error messages." + if verbose: + readable_output += build_verbose_human_readable(responses) + command_results = CommandResults( + outputs_prefix="BlockDomain", + outputs_key_field=["Domain", "Brand"], + outputs=rows, + readable_output=readable_output, + ) + if all_failed: + # Surface the whole run as an error entry so the war room / playbook can branch on failure. + command_results.entry_type = EntryType.ERROR + return command_results + + +""" PAN-OS FLOW """ + + +class DynamicGroupError(Exception): + """Raised when the target address-group exists and is dynamic (customer-managed).""" + + +class PanOs: + """Implements the PAN-OS static-address-group domain-blocking flow. + + The address-group and the deny rule are singletons (their names are constant), so they are + ensured once per run. Each valid domain then gets an FQDN address-object that is added to the + group. Commit + optional push happen once after all domains are processed. + """ + + def __init__(self, args: dict) -> None: + """Initialize the PanOs flow. + + Args: + args (dict): The flow arguments (domains, rule_name, address_group, tag, etc.). + """ + self.args = args + self.brand = "Panorama" + self.rule_name = args["rule_name"] + self.address_group = args["address_group"] + self.tag = args.get("tag", "") + self.log_forwarding_name = args.get("log_forwarding_name", "") + # Key MUST match the YAML arg name; args_for_next_run re-passes it during polling re-entry. + self.domains: list = args.get("domain_list", []) + self.responses: list = [] + # True once the rule was created or edited this run + self._rule_changed: bool = False + # True once any address object was created, or added to the group, this run. + self._address_made_changes: bool = False + # The pre-existing rule's current tags (captured once per run by rule_destinations), used to + # merge in the configured tag without duplicating it. + self._rule_tags: list[str] = [] + # The pre-existing rule's current log-forwarding profile (single value, captured once per + # run by rule_destinations). + self._rule_log_forwarding: str = "" + + # ---- tag helper ---------------------------------------------------- + + def ensure_tag(self) -> None: + """Create the configured tag up front so the group and rule can reference it. + + Creating the tag here decouples it from address-object creation, which is important when + every object already exists and the create-address auto-create path is skipped. An error + from an already-existing tag is expected and tolerated (logged, not raised). + + Returns: + None. + """ + if not self.tag: + return + command_name = "pan-os-create-tag" + res = run_execute_command(command_name, {"name": self.tag}) + self.responses.append(res) + if is_error(res): + # An already-existing tag is the common, benign case; log and continue. + demisto.debug( + f"{LOG_TAG} pan-os-create-tag for '{self.tag}' returned an error " f"(likely already exists): {get_error(res)}" + ) + + def merged_tags_if_missing(self, existing_tags: list[str]) -> list[str] | None: + """Return the tag list to write when the configured tag is not already present. + + Args: + existing_tags (list[str]): The entity's current tags. + Returns: + The merged tag list to write, or None when nothing needs to change. + """ + if not self.tag or self.tag in existing_tags: + return None + return [*existing_tags, self.tag] + + # ---- execution helper ---------------------------------------------- + + def execute_or_raise(self, command_name: str, command_args: dict, error_prefix: str) -> list[dict]: + """Run a command, record its response, and raise on error. + + Args: + command_name (str): The command to execute. + command_args (dict): The command arguments. + error_prefix (str): A prefix for the raised error message. + Returns: + The raw command entries. + """ + demisto.debug(f"{LOG_TAG} executing {command_name!r} with args={command_args}") + res = run_execute_command(command_name, command_args) + self.responses.append(res) + if is_error(res): + raise DemistoException(f"{error_prefix}: {get_error(res)}") + return res + + # ---- context probes ------------------------------------------------- + + def address_object_exists(self, object_name: str) -> tuple[bool, list[str]]: + """Check whether an address-object already exists, and return its current tags. + + Args: + object_name (str): The address-object name to probe. + Returns: + A tuple of (exists, current_tags). current_tags is empty when the object is absent or + has no tags. + """ + command_name = "pan-os-get-address" + res = run_execute_command(command_name, {"name": object_name}) + # pan-os-get-address raises when the object is absent; treat that as 'does not exist'. + if is_error(res): + return False, [] + self.responses.append(res) + context = get_relevant_context(res[0].get("EntryContext", {}), "Panorama.Addresses") + items = context if isinstance(context, list) else [context] + for item in items: + if item.get("Name") == object_name: + return True, normalize_tags(item.get("Tags")) + return False, [] + + def get_address_group(self) -> dict | None: + """Return the target address-group context dict, or None if it does not exist. + + Returns: + The address-group context dict, or None. + """ + res = self.execute_or_raise("pan-os-list-address-groups", {}, "Failed to list address groups") + context = get_relevant_context(res[0].get("EntryContext", {}), "Panorama.AddressGroups") + items = context if isinstance(context, list) else [context] + for item in items: + if item.get("Name") == self.address_group: + return item + return None + + def rule_destinations(self) -> tuple[bool, list, list[str]]: + """Return whether the rule exists, its current destination list, and its current tags. + + Also records the rule's current log-forwarding profile on ``self._rule_log_forwarding`` + (single-value profile) so ensure_rule can add it only when the configured profile differs. + + Returns: + A tuple of (rule_exists, destination_list, current_tags). + """ + res = self.execute_or_raise("pan-os-list-rules", {"pre_post": PRE_POST}, "Failed to list rules") + context = get_relevant_context(res[0].get("EntryContext", {}), "Panorama.SecurityRule") + items = context if isinstance(context, list) else [context] + for item in items: + if item.get("Name") == self.rule_name: + destination = item.get("Destination") + destination_list = destination if isinstance(destination, list) else [destination] if destination else [] + self._rule_log_forwarding = item.get("LogForwardingProfile") or "" + return True, destination_list, normalize_tags(item.get("Tags")) + return False, [], [] + + # ---- single-run writes (group + rule are singletons) ---------------- + + def ensure_group(self, group_context: dict | None, object_names: list[str]) -> None: + """Create the static address-group with all members, or add this run's members to it. + + A static group references its members by name, so every address object must already exist on + the device before the group is created or edited (otherwise PAN-OS rejects the reference). + Callers must create all objects first, then call this once with the full list of names. + + pan-os-edit-address-group requires exactly one of element_to_add/element_to_remove for a + static group and applies tags only alongside it, so the configured tag is merged into the + same element_to_add call. Re-adding an existing member is a no-op (the command de-duplicates). + + Args: + group_context (dict | None): The existing group's context, or None if it does not exist. + object_names (list[str]): Every address-object name for this run (the full member list). + Returns: + None. + Raises: + DynamicGroupError: If the group exists but is a customer-managed dynamic group. + """ + if group_context is None: + create_args: dict = {"name": self.address_group, "type": "static", "addresses": object_names} + if self.tag: + create_args["tags"] = self.tag + self.execute_or_raise( + "pan-os-create-address-group", create_args, f"Failed to create address-group '{self.address_group}'" + ) + self._address_made_changes = True + return + + group_type = (group_context.get("Type") or "").lower() + if group_type == "dynamic": + raise DynamicGroupError( + f"Address-group '{self.address_group}' already exists as dynamic; " f"will not modify a managed dynamic group." + ) + + existing_members = normalize_tags(group_context.get("Addresses")) + members_to_add = [name for name in object_names if name not in existing_members] + merged_tags = self.merged_tags_if_missing(normalize_tags(group_context.get("Tags"))) + if not members_to_add and merged_tags is None: + return + # element_to_add is required by the edit; when only the tag changed, re-add existing members. + edit_args: dict = {"name": self.address_group, "type": "static", "element_to_add": members_to_add or object_names} + if merged_tags is not None: + edit_args["tags"] = merged_tags + self.execute_or_raise( + "pan-os-edit-address-group", + edit_args, + f"Failed to add object(s) to address-group '{self.address_group}'", + ) + self._address_made_changes = True + + def ensure_rule(self, rule_present: bool, rule_destinations: list) -> None: + """Ensure the deny rule exists, points at the group, carries the tag, and sits at the top. + + Args: + rule_present (bool): Whether the rule already exists. + rule_destinations (list): The rule's current destination list. + Returns: + None. + """ + if not rule_present: + create_rule_args: dict = { + "rulename": self.rule_name, + "action": "deny", + "source": "any", + "destination": self.address_group, + "application": "any", + "service": "any", + "pre_post": PRE_POST, + "where": "top", + } + if self.tag: + create_rule_args["tags"] = self.tag + if self.log_forwarding_name: + create_rule_args["log_forwarding"] = self.log_forwarding_name + self.execute_or_raise("pan-os-create-rule", create_rule_args, f"Failed to create rule '{self.rule_name}'") + self._rule_changed = True + else: + if self.address_group not in rule_destinations: + # Rule exists but doesn't reference our group - add without replacing existing destinations. + self.execute_or_raise( + "pan-os-edit-rule", + { + "rulename": self.rule_name, + "element_to_change": "destination", + "element_value": self.address_group, + "behaviour": "add", + "pre_post": PRE_POST, + }, + f"Failed to add group to rule '{self.rule_name}'", + ) + self._rule_changed = True + # Add the configured tag to the pre-existing rule if it isn't already present. Uses + # behaviour=add so it never clobbers or duplicates the rule's existing tags. + if self.tag and self.tag not in self._rule_tags: + self.execute_or_raise( + "pan-os-edit-rule", + { + "rulename": self.rule_name, + "element_to_change": "tag", + "element_value": self.tag, + "behaviour": "add", + "pre_post": PRE_POST, + }, + f"Failed to add tag to rule '{self.rule_name}'", + ) + self._rule_changed = True + # Add the configured log-forwarding profile to the pre-existing rule if it differs. + if self.log_forwarding_name and self.log_forwarding_name != self._rule_log_forwarding: + self.execute_or_raise( + "pan-os-edit-rule", + { + "rulename": self.rule_name, + "element_to_change": "log-forwarding", + "element_value": self.log_forwarding_name, + "pre_post": PRE_POST, + }, + f"Failed to set log-forwarding profile on rule '{self.rule_name}'", + ) + self._rule_changed = True + # Enforce top placement only when we created/changed the rule; a pre-existing, unchanged + # rule is left where it is to avoid a needless move (and candidate-config churn). + if self._rule_changed or not rule_present: + self.execute_or_raise( + "pan-os-move-rule", + {"rulename": self.rule_name, "where": "top", "pre_post": PRE_POST}, + f"Failed to move rule '{self.rule_name}' to top", + ) + + def ensure_address_object(self, domain: str) -> tuple[str, bool, str]: + """Ensure a single domain's FQDN address-object exists and carries the configured tag. + + Object-only: it does not touch the group or the rule. The group is built afterwards from the + full set of object names, so every member reference resolves at group-creation time. + + Args: + domain (str): The domain to block. + Returns: + A tuple of (object_name, changed, message). 'changed' is True when the object was created + or its tag was added. + """ + object_name = derive_object_name(domain) + object_exists, object_tags = self.address_object_exists(object_name) + if object_exists: + # Merge the configured tag onto the existing object only when it is missing. + merged = self.merged_tags_if_missing(object_tags) + if merged is not None: + self.execute_or_raise( + "pan-os-edit-address", + {"name": object_name, "element_to_change": "tag", "element_value": merged}, + f"Failed to add tag to address-object '{object_name}'", + ) + return object_name, True, f"Address-object '{object_name}' already existed; tag '{self.tag}' added." + return object_name, False, f"Address-object '{object_name}' already exists." + + create_args: dict = {"name": object_name, "fqdn": domain} + if self.tag: + # create_tag=true auto-creates the tag; pan-os-create-address fails otherwise. + create_args["tag"] = self.tag + create_args["create_tag"] = "true" + self.execute_or_raise("pan-os-create-address", create_args, f"Failed to create address-object '{object_name}'") + return object_name, True, f"Address-object '{object_name}' created for '{domain}'." + + # ---- orchestration -------------------------------------------------- + + def process_domains(self) -> list: + """Block all domains in three phases: objects, then the static group, then the rule. + + The phased order is required because a static address-group references its members by name, + so every address object must exist before the group is created or edited. + + Returns: + The list of BlockDomain rows for the processed domains. + """ + rows: list = [] + try: + # Create the tag up front so group/rule creation can reference it even when every + # address object already exists (and the create-address auto-create path is skipped). + self.ensure_tag() + + # Phase 1: create/verify EVERY address object first (each carrying the tag). + object_names: list[str] = [] + per_domain_messages: list[tuple[str, str]] = [] + for domain in self.domains: + object_name, changed, message = self.ensure_address_object(domain) + object_names.append(object_name) + per_domain_messages.append((domain, message)) + self._address_made_changes = self._address_made_changes or changed + + # Phase 2: create/edit the static group ONCE with the full member list (all objects now + # exist, so every reference resolves). Tag is merged in the same call. + group_context = self.get_address_group() + self.ensure_group(group_context, object_names) + + # Phase 3: ensure the rule once (group now exists, so its destination reference resolves). + rule_present, rule_destinations, self._rule_tags = self.rule_destinations() + self.ensure_rule(rule_present, rule_destinations) + + for domain, message in per_domain_messages: + rows.append( + build_result_row( + domain=domain, + brand=self.brand, + status=STATUS_DONE, + result=RESULT_SUCCESS, + rule_name=self.rule_name, + message=f"{message} Added to '{self.address_group}'. Rule '{self.rule_name}' enforced at top.", + ) + ) + except DynamicGroupError as dyn_err: + # Abort the whole brand for this run: the target group is dynamic, so we can't add static + # members. Every requested domain failed to be blocked, so report them as failures. + demisto.error(f"{LOG_TAG} process_domains aborted for domains {self.domains}: {dyn_err}") + for domain in self.domains: + rows.append( + build_result_row( + domain=domain, + brand=self.brand, + status=STATUS_FAILED, + result=RESULT_FAILED, + rule_name="", + message=str(dyn_err), + ) + ) + except Exception as ex: + demisto.error(f"{LOG_TAG} process_domains failed for domains {self.domains}: {traceback.format_exc()}") + for domain in self.domains: + rows.append( + build_result_row( + domain=domain, + brand=self.brand, + status=STATUS_FAILED, + result=RESULT_FAILED, + rule_name=self.rule_name, + message=f"Failed to block '{domain}' on Panorama: {ex!s}", + ) + ) + return rows + + def pan_os_is_panorama(self) -> bool: + """Check whether the instance is a Panorama (vs a single firewall). + + Returns: + True if the instance model is 'Panorama', False otherwise. + """ + command_name = "pan-os" + res = run_execute_command(command_name, {"cmd": "", "type": "op"}) + self.responses.append(res) + context = get_relevant_context(res[0].get("EntryContext", {}), "Panorama.Command") + model = context.get("response", {}).get("result", {}).get("system", {}).get("model", "") # type: ignore + return model == "Panorama" + + def reduce_responses(self) -> list: + """Reduce the accumulated responses to the parts needed across polling cycles. + + Returns: + A list of reduced response entries suitable for serialization to context. + """ + reduced = [] + for res in self.responses: + reduced.append( + [ + { + "HumanReadable": entry.get("HumanReadable"), + "Contents": entry.get("Contents"), + "Type": entry.get("Type"), + "Metadata": entry.get("Metadata"), + } + for entry in res + ] + ) + return reduced + + def restore_responses(self) -> None: + """Restore the accumulated responses that were serialized to context in a previous cycle.""" + stored = demisto.context().get("panorama_responses", "") or "" + self.responses = json.loads(stored) if stored else [] + + def save_responses(self) -> None: + """Serialize the accumulated responses to context for the next polling cycle.""" + demisto.setContext("panorama_responses", json.dumps(self.reduce_responses())) + + def manage_pan_os_flow(self) -> Any: # pragma: no cover + """Dispatch the PAN-OS flow to the correct state. + + On re-entry (a push or commit job is in flight) the flow jumps straight to the relevant + status poller. Otherwise it starts the object/group/rule flow. + + Returns: + A PollResult when a job is in flight, or the list of result rows when finished. + """ + # Polling re-entry rules: + # - commit_job_id is carried in self.args by args_for_next_run. + # - push_job_id is only written to demisto.context() by pan_os_push_to_device. + # - A fresh manual invocation has neither in args; any leftover context is stale and + # must be scrubbed so it can't hijack the fresh run. + incident_context = demisto.context() + commit_job_id = self.args.get("commit_job_id") + context_push_job_id = demisto.get(incident_context, "push_job_id") + context_commit_job_id = demisto.get(incident_context, "commit_job_id") + + is_polling_reentry = bool(commit_job_id) + if not is_polling_reentry and (context_commit_job_id or context_push_job_id): + demisto.debug( + f"{LOG_TAG} Stale polling context on fresh invocation " + f"(commit={context_commit_job_id!r}, push={context_push_job_id!r}); clearing." + ) + demisto.setContext("commit_job_id", "") + demisto.setContext("push_job_id", "") + demisto.setContext("panorama_responses", "") + demisto.setContext("block_domain_rows", "") + context_push_job_id = None + + push_job_id = context_push_job_id if is_polling_reentry else None + + demisto.debug(f"{LOG_TAG} dispatch: {commit_job_id=}, {push_job_id=}") + if push_job_id: + return self.handle_push_in_flight(push_job_id) + if commit_job_id: + return self.handle_commit_in_flight(commit_job_id) + return self.start_flow() + + def handle_push_in_flight(self, push_job_id: str) -> Any: # pragma: no cover + """Poll the status of an in-flight push-to-device-group job. + + Args: + push_job_id (str): The push job ID to poll. + Returns: + A PollResult while the push is running, or the final result rows when it finishes. + """ + self.restore_responses() + self.args["push_job_id"] = push_job_id + res_push_status = pan_os_push_status(self.args, self.responses) + if not is_polling_in_progress(res_push_status): + demisto.debug(f"{LOG_TAG} Push job {push_job_id} finished.") + return self.finish() + self.save_responses() + return res_push_status + + def handle_commit_in_flight(self, commit_job_id: str) -> Any: # pragma: no cover + """Poll the status of an in-flight commit job, then start the push if needed. + + Args: + commit_job_id (str): The commit job ID to poll. + Returns: + A PollResult while commit/push is running, or the final result rows when finished. + """ + self.args["commit_job_id"] = commit_job_id + self.restore_responses() + poll_commit_status = pan_os_commit_status(self.args, self.responses) + if is_polling_in_progress(poll_commit_status): + self.save_responses() + return poll_commit_status + demisto.debug(f"{LOG_TAG} Commit job {commit_job_id} finished.") + # Commit finished - push to the device group if this is Panorama. + if self.pan_os_is_panorama(): + poll_push = pan_os_push_to_device(self.args, self.responses) + if not is_polling_in_progress(poll_push): + return self.finish() + self.save_responses() + return poll_push + return self.finish() + + def start_flow(self) -> Any: # pragma: no cover + """Run the object/group/rule flow, then start the commit if there were changes. + + Returns: + A PollResult while the commit is running, or the final result rows when finished. + """ + rows = self.process_domains() + demisto.setContext("block_domain_rows", json.dumps(rows)) + # Commit/push only when the run mutated Panorama state, to avoid a needless multi-minute + # push poll on a pure "nothing changed" run. + made_changes = self._address_made_changes or self._rule_changed + auto_commit = argToBoolean(self.args.get("auto_commit", True)) + demisto.debug(f"{LOG_TAG} start_flow: {made_changes=}, {auto_commit=}, {len(rows)} row(s)") + if made_changes and auto_commit: + poll_commit = pan_os_commit(self.args, self.responses) + if not is_polling_in_progress(poll_commit): + return self.finish() + self.save_responses() + return poll_commit + # Nothing changed on the device: return the rows and skip the commit/push. + demisto.debug(f"{LOG_TAG} start_flow: no changes detected; skipping commit/push.") + return rows + + def finish(self) -> list: # pragma: no cover + """Clean up polling context and return the final result rows. + + The accumulated responses (restored from context across polling cycles) are kept on the + instance so the caller can build verbose output before they are cleared from context. + + Returns: + The list of BlockDomain rows accumulated for the run. + """ + rows_raw = demisto.context().get("block_domain_rows", "") or "" + # Preserve responses on the instance for verbose output before clearing context. + stored = demisto.context().get("panorama_responses", "") or "" + if stored: + try: + self.responses = json.loads(stored) + except (ValueError, TypeError) as err: + demisto.debug(f"{LOG_TAG} Could not parse stored responses from context; ignoring. Error: {err}") + demisto.setContext("commit_job_id", "") + demisto.setContext("push_job_id", "") + demisto.setContext("panorama_responses", "") + demisto.setContext("block_domain_rows", "") + try: + return json.loads(rows_raw) if rows_raw else [] + except (ValueError, TypeError): + return [] + + +""" POLLING FUNCTIONS (commit / push) """ + + +@polling_function(name="block-domain", interval=60, timeout=1200) +def pan_os_commit(args: dict, responses: list) -> PollResult: + """Execute pan-os-commit. + + Args: + args (dict): The arguments of the function. + responses (list): The responses of the commands executed so far. + Returns: + The PollResult object. + """ + command_name = "pan-os-commit" + res_commit = run_execute_command(command_name, {"polling": True}) + responses.append(res_commit) + # Surface a failed commit instead of silently treating it as "no changes to commit". + if is_error(res_commit): + failure_output = {"Status": "Failure"} + return PollResult( + response=CommandResults( + outputs=failure_output, + readable_output=tableToMarkdown("Commit Status:", failure_output, removeNull=True), + ), + args_for_next_run=args, + continue_to_poll=False, + ) + polling_args = res_commit[0].get("Metadata", {}).get("pollingArgs", {}) + job_id = polling_args.get("commit_job_id") + if job_id: + context_output = {"JobID": job_id, "Status": "Pending"} + continue_to_poll = True + commit_output: Any = CommandResults( + outputs=context_output, readable_output=tableToMarkdown("Commit Status:", context_output, removeNull=True) + ) + demisto.setContext("commit_job_id", job_id) + else: + commit_output = res_commit[0].get("Contents") or "There are no changes to commit." + continue_to_poll = False + + args_for_next_run = args | { + "commit_job_id": job_id, + "interval_in_seconds": arg_to_number(args.get("interval_in_seconds", 60)), + "timeout": arg_to_number(args.get("timeout", 1200)), + "polling": True, + } + return PollResult( + response=commit_output, + continue_to_poll=continue_to_poll, + args_for_next_run=args_for_next_run, + partial_result=CommandResults(readable_output=f"Waiting for commit job ID {job_id} to finish..."), + ) + + +@polling_function(name="block-domain", interval=60, timeout=1200) +def pan_os_commit_status(args: dict, responses: list) -> PollResult: + """Check the status of the commit job in pan-os. + + Args: + args (dict): The arguments of the function. + responses (list): The responses of the previous command. + Returns: + The PollResult object. + """ + commit_job_id = args["commit_job_id"] + command_name = "pan-os-commit-status" + res_commit_status = run_execute_command(command_name, {"job_id": commit_job_id}) + responses.append(res_commit_status) + # When pan-os-commit-status errors, Contents is a plain string instead of the nested dict. + # Treat as a terminal failure to avoid a `.get()` crash on a string. + raw_contents = res_commit_status[0].get("Contents", {}) if res_commit_status else {} + if not isinstance(raw_contents, dict): + commit_output = {"JobID": commit_job_id, "Status": "Failure"} + return PollResult( + response=CommandResults( + outputs=commit_output, + outputs_key_field="JobID", + readable_output=tableToMarkdown("Commit Status:", commit_output, removeNull=True), + ), + args_for_next_run=args, + continue_to_poll=False, + ) + result_commit_status = raw_contents.get("response", {}).get("result", {}).get("job", {}) + job_result = result_commit_status.get("result") + continue_to_poll = result_commit_status.get("status") != "FIN" + if continue_to_poll: + commit_output = {"JobID": commit_job_id, "Status": "Pending"} + else: + commit_output = {"JobID": commit_job_id, "Status": "Success" if job_result == "OK" else "Failure"} + return PollResult( + response=CommandResults( + outputs=commit_output, + outputs_key_field="JobID", + readable_output=tableToMarkdown("Commit Status:", commit_output, removeNull=True), + ), + args_for_next_run=args, + continue_to_poll=continue_to_poll, + ) + + +@polling_function(name="block-domain", interval=60, timeout=1200) +def pan_os_push_to_device(args: dict, responses: list) -> PollResult: + """Execute pan-os-push-to-device-group. + + Args: + args (dict): The arguments of the function. + responses (list): The responses of the previous command. + Returns: + The PollResult object. + """ + command_name = "pan-os-push-to-device-group" + res_push_to_device = run_execute_command(command_name, {"polling": True}) + responses.append(res_push_to_device) + # Surface a failed push instead of silently treating it as "no changes to push". + if is_error(res_push_to_device): + push_output = {"Status": "Failure"} + return PollResult( + response=CommandResults( + outputs=push_output, + readable_output=tableToMarkdown("Push to Device Group:", push_output, removeNull=True), + ), + args_for_next_run=args, + continue_to_poll=False, + ) + polling_args = res_push_to_device[0].get("Metadata", {}).get("pollingArgs", {}) + job_id = polling_args.get("push_job_id") + device_group = polling_args.get("device-group") + if job_id: + context_output = {"DeviceGroup": device_group, "JobID": job_id, "Status": "Pending"} + continue_to_poll = True + push_cr = CommandResults( + outputs_key_field="JobID", + outputs=context_output, + readable_output=tableToMarkdown("Push to Device Group:", context_output, removeNull=True), + ) + demisto.setContext("push_job_id", job_id) + else: + push_cr = CommandResults(readable_output=res_push_to_device[0].get("Contents") or "There are no changes to push.") + continue_to_poll = False + return PollResult( + response=push_cr, + continue_to_poll=continue_to_poll, + partial_result=CommandResults(readable_output=f"Waiting for Job-ID {job_id} to finish pushing the changes..."), + ) + + +@polling_function(name="block-domain", interval=60, timeout=1200) +def pan_os_push_status(args: dict, responses: list) -> PollResult: + """Check the status of the push job in pan-os. + + Args: + args (dict): The arguments of the function. + responses (list): The responses of the previous command. + Returns: + The PollResult object. + """ + push_job_id = args["push_job_id"] + command_name = "pan-os-push-status" + res_push_status = run_execute_command(command_name, {"job_id": push_job_id}) + responses.append(res_push_status) + # When pan-os-push-status errors, Contents is a plain string instead of the nested dict. + # Treat as a terminal failure to avoid a `.get()` crash on a string (mirrors pan_os_commit_status). + raw_contents = res_push_status[0].get("Contents", {}) if res_push_status else {} + if is_error(res_push_status) or not isinstance(raw_contents, dict): + push_output = {"JobID": push_job_id, "Status": "Failure"} + return PollResult( + response=CommandResults( + outputs=push_output, + outputs_key_field="JobID", + readable_output=tableToMarkdown("Push to Device Group:", push_output, ["JobID", "Status"], removeNull=True), + ), + args_for_next_run=args, + continue_to_poll=False, + ) + push_status = raw_contents.get("response", {}).get("result", {}).get("job", {}).get("status", "") + continue_to_poll = bool(push_status and push_status != "FIN") + context_output = {"Status": push_status, "JobID": push_job_id} + push_cr = CommandResults( + outputs_key_field="JobID", + outputs=context_output, + readable_output=tableToMarkdown("Push to Device Group:", context_output, ["JobID", "Status"], removeNull=True), + ) + return PollResult( + response=push_cr, + continue_to_poll=continue_to_poll, + partial_result=CommandResults(readable_output=f"Waiting for Job-ID {push_job_id} to finish pushing the changes..."), + ) + + +""" MAIN FUNCTION """ + + +def main() -> None: # pragma: no cover + try: + args = demisto.args() + demisto.debug(f"{LOG_TAG} block-domain invoked with {args=}") + + domain_list = argToList(args.get("domain_list", [])) + rule_name = args.get("rule_name", "Cortex - Block Domain") + log_forwarding_name = args.get("log_forwarding_name", "") + address_group = args.get("address_group", "Blocked Domains - Cortex") + tag = args.get("tag", "cortex-blocked-domains") + auto_commit = argToBoolean(args.get("auto_commit", True)) + verbose = argToBoolean(args.get("verbose", False)) + brands_to_run = argToList(args.get("brands", ",".join(SUPPORTED_BRANDS))) + + if not domain_list: + raise DemistoException("domain_list argument is required.") + + valid_domains, failed_rows = validate_domains(domain_list) + enabled_brands = get_enabled_brands() + brands_to_run = brands_to_run or list(SUPPORTED_BRANDS) + + runnable_brands = [b for b in brands_to_run if b in SUPPORTED_BRANDS and b in enabled_brands] + if not runnable_brands: + raise DemistoException( + f"No integrations were found for the brands {brands_to_run}. " + f"Please verify the brand instances' setup. Supported brands: {SUPPORTED_BRANDS}." + ) + + results: list = list(failed_rows) + command_responses: list = [] # accumulated per-command responses, used for verbose output. + + for brand in brands_to_run: + if brand not in SUPPORTED_BRANDS: + results.append( + build_result_row( + domain="", + brand=brand, + status=STATUS_FAILED, + result=RESULT_FAILED, + message=f"The brand {brand} is not supported by 'block-domain'. Supported: {SUPPORTED_BRANDS}.", + ) + ) + elif brand not in enabled_brands: + results.append( + build_result_row( + domain="", + brand=brand, + status=STATUS_FAILED, + result=RESULT_FAILED, + message=f"The brand {brand} isn't enabled.", + ) + ) + elif brand == "Panorama" and valid_domains: + pan_os = PanOs( + { + # Key MUST match the YAML arg name; polling re-invocation validates it. + "domain_list": valid_domains, + "rule_name": rule_name, + "log_forwarding_name": log_forwarding_name, + "address_group": address_group, + "tag": tag, + "auto_commit": auto_commit, + "verbose": verbose, + "commit_job_id": args.get("commit_job_id"), + "push_job_id": args.get("push_job_id"), + "polling": True, + } + ) + pan_os_result = pan_os.manage_pan_os_flow() + # A list means the run finished. Anything else (PollResult / bare CommandResults + # from a freshly-started poll) means a job is in flight. + if not isinstance(pan_os_result, list): + return_results(pan_os_result) + return + results.extend(pan_os_result) + command_responses.extend(pan_os.responses) + + return_results(build_final_command_results(results, verbose, command_responses)) + + except Exception as ex: + demisto.error(f"{LOG_TAG} block-domain failed: {traceback.format_exc()}") + return_error(f"Failed to execute block-domain. Error: {ex!s}") + + +""" ENTRY POINT """ + +if __name__ in ("__main__", "__builtin__", "builtins"): + main() diff --git a/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain.yml b/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain.yml new file mode 100644 index 000000000000..87e9c3981c2a --- /dev/null +++ b/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain.yml @@ -0,0 +1,93 @@ +args: +- description: A comma-separated list of domain FQDNs to block. Wildcard entries (e.g. *.evil.com) are not supported and are skipped. + isArray: true + name: domain_list + required: true +- defaultValue: 'Cortex - Block Domain' + description: The name of the rule which will be created in the relevant products. + isArray: false + name: rule_name + required: false +- description: The Panorama log forwarding object name that specifies the log Forwarding setting to apply to the PAN-OS custom rules. + isArray: false + name: log_forwarding_name + required: false +- description: The name of the PAN-OS Panorama or Firewall address group used to hold the blocked domain FQDN objects. + isArray: false + name: address_group + required: false + defaultValue: 'Blocked Domains - Cortex' +- description: Whether to commit the new rule and push to the device group at the end of the run. + isArray: false + name: auto_commit + required: false + defaultValue: 'true' + auto: PREDEFINED + predefined: + - 'true' + - 'false' +- description: The designated tag name for the domain FQDN object. Applied to every object the script creates. + isArray: false + name: tag + required: false + defaultValue: 'cortex-blocked-domains' +- description: |- + A comma-separated list of integration brands to run the command for. If not provided, the command runs for all available integrations. + isArray: true + name: brands + required: false + auto: PREDEFINED + predefined: + - 'Panorama' +- description: Whether to retrieve a human-readable entry for every command or only the final result. True retrieves a human-readable entry for every command. False retrieves a human-readable entry only for the final result. + name: verbose + defaultValue: 'false' + auto: PREDEFINED + predefined: + - 'true' + - 'false' +- description: The commit job ID to use in polling commands. Automatically filled by polling. + name: commit_job_id + hidden: true +- description: The push job ID to use in polling commands. Automatically filled by polling. + name: push_job_id + hidden: true +comment: Blocks a list of domain FQDNs across the configured security products. +commonfields: + id: block-domain + version: -1 +enabled: false +name: block-domain +outputs: +- contextPath: BlockDomain.Domain + description: The domain FQDN that was processed. + type: String +- contextPath: BlockDomain.Brand + description: The brand (integration) used to block the domain. + type: String +- contextPath: BlockDomain.Status + description: "The lifecycle status of the action. One of 'Done', 'Pending', 'Skipped', 'Failed'." + type: String +- contextPath: BlockDomain.Result + description: "The result of the action. 'Success' or 'Failed'." + type: String +- contextPath: BlockDomain.RuleName + description: The name of the rule used for this integration. Empty if no rule was used. + type: String +- contextPath: BlockDomain.Message + description: A message concerning the result of the action. + type: String +script: '-' +system: false +timeout: 20m0s +type: python +subtype: python3 +dockerimage: demisto/python3:3.12.14.13495286 +fromversion: 6.10.0 +marketplaces: +- xsoar +- marketplacev2 +- platform +polling: true +tests: +- No tests (auto formatted) diff --git a/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain_test.py b/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain_test.py new file mode 100644 index 000000000000..9c201335b0fa --- /dev/null +++ b/Packs/AggregatedScripts/Scripts/BlockDomain/BlockDomain_test.py @@ -0,0 +1,1662 @@ +import json + +import pytest + +import BlockDomain +from BlockDomain import ( + OBJECT_NAME_PREFIX, + MAX_OBJECT_NAME_LENGTH, + RESULT_FAILED, + RESULT_SUCCESS, + STATUS_DONE, + STATUS_FAILED, + PanOs, + build_final_command_results, + build_verbose_human_readable, + derive_object_name, + is_valid_fqdn, + is_wildcard, + normalize_tags, + pan_os_commit, + pan_os_commit_status, + pan_os_push_status, + pan_os_push_to_device, + run_execute_command, + validate_domains, +) + + +def ok_entry(entry_context=None, contents="ok", instance=None, brand="Panorama"): + """Build a minimal successful execute_command entry. + + When ``instance`` is provided, includes a ``Metadata`` block matching what the platform + actually returns (``Metadata.instance`` / ``Metadata.brand``) so tests can assert that the + aggregate script correctly captures the serving-instance name from response entries. + """ + entry: dict = {"Type": 1, "Contents": contents, "HumanReadable": "", "EntryContext": entry_context or {}} + if instance is not None: + entry["Metadata"] = {"instance": instance, "brand": brand} + return entry + + +def err_entry(contents="error"): + """Build a minimal error execute_command entry (Type 4 == entryTypes['error']).""" + return {"Type": 4, "Contents": contents, "HumanReadable": "", "EntryContext": {}} + + +def test_run_execute_command_returns_entries(monkeypatch): + """ + Given: + - executeCommand returns a non-empty list of entries. + When: + - Calling run_execute_command. + Then: + - The entries are returned unchanged. + """ + entries = [ok_entry()] + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", lambda *a, **k: entries) + + assert run_execute_command("pan-os-get-address", {"name": "x"}) == entries + + +@pytest.mark.parametrize("empty_response", [[], None]) +def test_run_execute_command_raises_on_empty_response(monkeypatch, empty_response): + """ + Given: + - executeCommand returns an empty response (empty list or None). + When: + - Calling run_execute_command. + Then: + - A DemistoException is raised instead of letting downstream res[0] indexing crash. + """ + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", lambda *a, **k: empty_response) + + with pytest.raises(BlockDomain.DemistoException, match="returned an empty response"): + run_execute_command("pan-os-get-address", {"name": "x"}) + + +@pytest.mark.parametrize( + "domain, expected", + [ + ("*.evil.com", True), + ("evil.*.com", True), + ("evil.example.com", False), + ("sub.domain.co.uk", False), + ], +) +def test_is_wildcard(domain, expected): + """ + Given: + - A domain string that may or may not contain a wildcard character. + When: + - Calling is_wildcard to detect wildcard patterns. + Then: + - Returns True for domains containing '*', False otherwise. + """ + assert is_wildcard(domain) is expected + + +@pytest.mark.parametrize( + "domain, expected", + [ + ("evil.example.com", True), + ("sub.domain.co.uk", True), + ("a.b", True), + ("no-dot", False), + ("*.evil.com", False), + ("-leading.example.com", False), + ("trailing-.example.com", False), + ("space in.example.com", False), + ("", False), + ], +) +def test_is_valid_fqdn(domain, expected): + """ + Given: + - A domain string that may or may not be a syntactically valid FQDN. + When: + - Calling is_valid_fqdn to validate the domain. + Then: + - Returns True for well-formed FQDNs (has dot, valid labels, no wildcard/illegal chars), + and False for anything else including empty strings and wildcards. + """ + assert is_valid_fqdn(domain) is expected + + +def test_derive_object_name_is_deterministic(): + """ + Given: + - The same FQDN passed to derive_object_name twice. + When: + - Comparing the two returned object names. + Then: + - Both invocations return the exact same string (deterministic mapping). + """ + assert derive_object_name("evil.example.com") == derive_object_name("evil.example.com") + + +def test_derive_object_name_illegal_chars(): + """ + Given: + - A domain containing an underscore, which is not a legal PAN-OS object-name character. + When: + - Calling derive_object_name. + Then: + - Underscores are normalized to hyphens so the resulting name is accepted by PAN-OS. + """ + assert derive_object_name("bad_domain.example.com") == "Cortex-bad-domain.example.com" + + +def test_derive_object_name_overflow_truncates_and_hashes(): + """ + Given: + - A domain long enough that the naive prefixed name would exceed the PAN-OS + MAX_OBJECT_NAME_LENGTH limit. + When: + - Calling derive_object_name. + Then: + - The returned name fits within MAX_OBJECT_NAME_LENGTH, still starts with the + "Cortex-" prefix, and remains deterministic across calls. + """ + long_domain = ("a" * 80) + ".example.com" + name = derive_object_name(long_domain) + assert len(name) <= MAX_OBJECT_NAME_LENGTH + assert name.startswith(OBJECT_NAME_PREFIX) + + +def test_derive_object_name_overflow_distinct_domains_are_unique(): + """ + Given: + - Two DIFFERENT over-length domains. + When: + - Deriving each object name (both take the truncate+hash overflow path). + Then: + - The two names are different, proving the hash suffix disambiguates over-length names + (the digest is a function of the full domain, not the truncated body). + """ + name_a = derive_object_name(("a" * 80) + ".example.com") + name_b = derive_object_name(("b" * 80) + ".example.com") + assert name_a != name_b + + +def test_derive_object_name_overflow_same_prefix_different_tail_are_unique(): + """ + Given: + - Two over-length domains that share an identical long leading segment (so their + truncated bodies are the same) but differ only in a tail beyond the truncation point. + When: + - Deriving each object name. + Then: + - The names are STILL unique - the digest is computed over the full original domain, so + differences past the truncation boundary are preserved in the hash suffix. This is the + key anti-collision guarantee for the prefix-hash naming scheme. + """ + shared_head = "a" * 80 + name_a = derive_object_name(f"{shared_head}.first-tail.example.com") + name_b = derive_object_name(f"{shared_head}.second-tail.example.com") + assert name_a != name_b + # Both are truncated to the same length budget, so equal length; only the digest differs. + assert len(name_a) == len(name_b) + + +def test_validate_domains_splits_valid_and_failed(): + """ + Given: + - A mixed list of domains containing valid FQDNs, a wildcard, and an invalid FQDN. + When: + - Calling validate_domains to partition the input. + Then: + - Valid FQDNs are returned in the first list; the wildcard and invalid entries are + returned as failed rows with STATUS_FAILED / RESULT_FAILED and appropriate messages. + """ + valid, failed = validate_domains(["evil.example.com", "*.evil.com", "no-dot", "phish.attacker.net"]) + + assert valid == ["evil.example.com", "phish.attacker.net"] + assert len(failed) == 2 + + wildcard_row = next(row for row in failed if row["Domain"] == "*.evil.com") + assert wildcard_row["Status"] == STATUS_FAILED + assert wildcard_row["Result"] == RESULT_FAILED + assert "Wildcard" in wildcard_row["Message"] + # No brand/rule applies to a validation-failed row - these must be real null, not "". + assert wildcard_row["Brand"] is None + assert wildcard_row["RuleName"] is None + + invalid_row = next(row for row in failed if row["Domain"] == "no-dot") + assert invalid_row["Status"] == STATUS_FAILED + assert invalid_row["Result"] == RESULT_FAILED + assert "Invalid FQDN" in invalid_row["Message"] + + +def _pan_os(domains): + return PanOs( + { + "domain_list": domains, + "rule_name": "Cortex - Block Domain", + "address_group": "Blocked Domains - Cortex", + "tag": "cortex-blocked-domains", + "log_forwarding_name": "", + "auto_commit": True, + } + ) + + +def _mock_execute(monkeypatch, responses_by_command): + """Patch BlockDomain.demisto.executeCommand to answer per command name (order-independent). + + ``responses_by_command`` maps a command name to the entry list it should return. Probe commands + (pan-os-get-address / -list-address-groups / -list-rules) supply the tenant state; write commands + default to a bare success entry when not specified. ``pan-os-create-tag`` is always auto-answered. + + Args: + monkeypatch: The pytest monkeypatch fixture. + responses_by_command (dict): Mapping of command name -> entry list to return. + Returns: + The ``calls`` list of (command_name, args) tuples recorded during the run. + """ + calls: list = [] + + def _dispatch(command_name, args=None, *a, **k): + calls.append((command_name, args or {})) + if command_name == "pan-os-create-tag": + return [ok_entry()] + return responses_by_command.get(command_name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _dispatch) + return calls + + +def test_process_domains_create_everything(monkeypatch): + """ + Given: + - A tenant where neither the address group nor the security rule exist yet, and the + address object for the domain also does not exist. + When: + - Calling process_domains for a single domain. + Then: + - The address is created first, then the group is seeded with that object, then the + rule is created against the now-existing group, then the rule is moved to the top. + The resulting row is Done / Success / Created and carries the expected rule name. + """ + _mock_execute( + monkeypatch, + { + "pan-os-get-address": [err_entry("not found")], + "pan-os-list-address-groups": [ok_entry({"Panorama.AddressGroups": []})], + "pan-os-list-rules": [ok_entry({"Panorama.SecurityRule": []})], + }, + ) + pan = _pan_os(["evil.example.com"]) + rows = pan.process_domains() + assert len(rows) == 1 + assert rows[0]["Status"] == STATUS_DONE + assert rows[0]["Result"] == RESULT_SUCCESS + assert rows[0]["RuleName"] == "Cortex - Block Domain" + # Action is no longer surfaced; change-detection is tracked internally to gate the commit. + assert pan._address_made_changes is True + assert "Action" not in rows[0] + + +def test_ensure_tag_creates_tag_up_front(monkeypatch): + """ + Given: + - A PanOs flow configured with a tag. + When: + - Calling ensure_tag. + Then: + - pan-os-create-tag is invoked exactly once with the configured tag name, so the tag exists + before any group/rule references it (independent of whether an address object is created). + """ + calls: list = [] + + def _capture(name, args): + calls.append((name, args)) + return [ok_entry()] + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + + _pan_os(["evil.example.com"]).ensure_tag() + + assert calls == [("pan-os-create-tag", {"name": "cortex-blocked-domains"})] + + +def test_ensure_tag_tolerates_already_exists_error(monkeypatch): + """ + Given: + - pan-os-create-tag returns an error (the tag already exists on the device). + When: + - Calling ensure_tag. + Then: + - ensure_tag does NOT raise - an already-existing tag is an expected, benign outcome. + """ + + def _err(name, args): + return [err_entry("tag already exists")] + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _err) + + # Must not raise. + _pan_os(["evil.example.com"]).ensure_tag() + + +def test_ensure_tag_noop_when_no_tag(monkeypatch): + """ + Given: + - A PanOs flow with an empty tag. + When: + - Calling ensure_tag. + Then: + - No pan-os-create-tag call is made (nothing to create). + """ + calls: list = [] + + def _capture(name, args): + calls.append(name) + return [ok_entry()] + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + + pan = _pan_os(["evil.example.com"]) + pan.tag = "" + pan.ensure_tag() + + assert calls == [] + + +# ---- tag helpers (pure) ------------------------------------------------ + + +@pytest.mark.parametrize( + "raw, expected", + [ + (None, []), + ("", []), + ("solo", ["solo"]), + (["a", "b"], ["a", "b"]), + (["a", "", None, "b"], ["a", "b"]), + ], +) +def test_normalize_tags(raw, expected): + """ + Given: + - A raw PAN-OS 'Tags' value that may be None, an empty string, a single string, or a list + possibly containing falsy entries. + When: + - Calling normalize_tags. + Then: + - It flattens the value into a clean list of tag-name strings, dropping falsy entries. + """ + assert normalize_tags(raw) == expected + + +@pytest.mark.parametrize( + "existing, expected", + [ + ([], ["cortex-blocked-domains"]), + (["other"], ["other", "cortex-blocked-domains"]), + (["cortex-blocked-domains"], None), + (["other", "cortex-blocked-domains"], None), + ], +) +def test_merged_tags_if_missing(existing, expected): + """ + Given: + - An entity's current tag list and a PanOs flow configured with 'cortex-blocked-domains'. + When: + - Calling merged_tags_if_missing. + Then: + - Returns the de-duplicated union (existing + configured tag) when the tag is absent, and + None (no edit needed) when the tag is already present. The existing tag is never duplicated. + """ + assert _pan_os(["evil.example.com"]).merged_tags_if_missing(existing) == expected + + +def test_start_flow_skips_commit_when_all_actions_unchanged(monkeypatch): + """ + Given: + - A tenant already fully configured for the requested domain (group exists, rule exists, + address object already a member) so every row is Unchanged. + When: + - Calling start_flow. + Then: + - pan_os_commit is never invoked; start_flow returns the Unchanged rows directly + (skipping the multi-minute commit + push polling cycle on Panorama). + """ + calls: list = [] + + def _capture(name, args): + calls.append((name, args)) + seq = { + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + { + "Name": "Blocked Domains - Cortex", + "Type": "static", + "Addresses": ["Cortex-evil.example.com"], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + "pan-os-list-rules": [ + ok_entry( + { + "Panorama.SecurityRule": [ + { + "Name": "Cortex - Block Domain", + "Destination": ["Blocked Domains - Cortex"], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + "pan-os-get-address": [ + ok_entry({"Panorama.Addresses": {"Name": "Cortex-evil.example.com", "Tags": ["cortex-blocked-domains"]}}) + ], + "pan-os-move-rule": [ok_entry()], + } + return seq.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + monkeypatch.setattr( + BlockDomain, "pan_os_commit", lambda *a, **k: pytest.fail("pan_os_commit must not be called when all rows are Unchanged") + ) + monkeypatch.setattr(BlockDomain.demisto, "setContext", lambda *a, **k: None) + + result = _pan_os(["evil.example.com"]).start_flow() + + assert isinstance(result, list) + assert len(result) == 1 + assert "Action" not in result[0] + assert "pan-os-commit" not in [c[0] for c in calls] + + +def test_start_flow_commits_when_at_least_one_row_modified(monkeypatch): + """ + Given: + - A tenant where the group and rule already exist but the address object for the + requested domain is missing, so at least one row will end up Created. + When: + - Calling start_flow. + Then: + - pan_os_commit is invoked (candidate config was modified and must be pushed). + """ + commit_called: list = [] + + def _fake_commit(args, responses): + commit_called.append(True) + return BlockDomain.CommandResults(readable_output="fake commit ok") + + def _capture(name, args): + seq = { + "pan-os-list-address-groups": [ + ok_entry({"Panorama.AddressGroups": [{"Name": "Blocked Domains - Cortex", "Type": "static", "Addresses": []}]}) + ], + "pan-os-list-rules": [ + ok_entry( + {"Panorama.SecurityRule": [{"Name": "Cortex - Block Domain", "Destination": ["Blocked Domains - Cortex"]}]} + ) + ], + "pan-os-get-address": [err_entry("not found")], + "pan-os-create-address": [ok_entry()], + "pan-os-edit-address-group": [ok_entry()], + "pan-os-move-rule": [ok_entry()], + } + return seq.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + monkeypatch.setattr(BlockDomain.demisto, "setContext", lambda *a, **k: None) + monkeypatch.setattr(BlockDomain, "pan_os_commit", _fake_commit) + monkeypatch.setattr(BlockDomain.demisto, "context", lambda: {"block_domain_rows": "[]"}) + + _pan_os(["evil.example.com"]).start_flow() + + assert commit_called, "pan_os_commit must be called when at least one row is Created/Modified" + + +def test_process_domains_rule_created_when_object_unchanged_sets_rule_changed(monkeypatch): + """ + Given: + - The address object and group already exist and the object is already a member (so the + per-domain Action is Unchanged), but the requested rule_name does NOT exist yet, so a + new rule must be created. + When: + - Calling process_domains. + Then: + - The row Action is Unchanged (object/membership did not change), but the instance's + _rule_changed flag is set True because pan-os-create-rule ran. This is what lets + start_flow still commit a rule-only change. + """ + obj = "Cortex-evil.example.com" + + def _capture(name, args): + seq = { + # Group exists, already contains the object, and already carries the tag -> Unchanged. + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + { + "Name": "Blocked Domains - Cortex", + "Type": "static", + "Addresses": [obj], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + # No rule with the requested name exists -> ensure_rule will create it. + "pan-os-list-rules": [ok_entry({"Panorama.SecurityRule": []})], + # Object exists and already carries the tag -> no object tag edit. + "pan-os-get-address": [ok_entry({"Panorama.Addresses": [{"Name": obj, "Tags": ["cortex-blocked-domains"]}]})], + "pan-os-create-rule": [ok_entry()], + "pan-os-move-rule": [ok_entry()], + } + return seq.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + + pan_os = _pan_os(["evil.example.com"]) + rows = pan_os.process_domains() + + # Object + membership did not change (so _made_changes stays False), but the rule was created, + # so _rule_changed is True - that alone must still trigger a commit in start_flow. + assert "Action" not in rows[0] + assert pan_os._address_made_changes is False + assert pan_os._rule_changed is True + + +def test_start_flow_commits_when_only_the_rule_changed(monkeypatch): + """ + Given: + - Every address object is already present and a member (all rows Unchanged), but a new + rule had to be created this run (rule-only change). + When: + - Calling start_flow. + Then: + - pan_os_commit is still invoked, so the newly created rule is actually committed/pushed + instead of silently sitting in the candidate config. + """ + obj = "Cortex-evil.example.com" + commit_called: list = [] + + def _fake_commit(args, responses): + commit_called.append(True) + return BlockDomain.CommandResults(readable_output="fake commit ok") + + def _capture(name, args): + seq = { + "pan-os-list-address-groups": [ + ok_entry({"Panorama.AddressGroups": [{"Name": "Blocked Domains - Cortex", "Type": "static", "Addresses": [obj]}]}) + ], + "pan-os-list-rules": [ok_entry({"Panorama.SecurityRule": []})], + "pan-os-get-address": [ok_entry({"Panorama.Addresses": [{"Name": obj}]})], + "pan-os-create-rule": [ok_entry()], + "pan-os-move-rule": [ok_entry()], + } + return seq.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + monkeypatch.setattr(BlockDomain.demisto, "setContext", lambda *a, **k: None) + monkeypatch.setattr(BlockDomain, "pan_os_commit", _fake_commit) + monkeypatch.setattr(BlockDomain.demisto, "context", lambda: {"block_domain_rows": "[]"}) + + _pan_os(["evil.example.com"]).start_flow() + + assert commit_called, "pan_os_commit must be called when only the rule changed (all objects Unchanged)" + + +def test_process_domains_missing_group_created_lazily_with_first_object(monkeypatch): + """ + Given: + - A tenant with neither group nor rule pre-configured, and no address object for the + requested domain. PAN-OS refuses to create a static group with no members, and + refuses to create a rule whose destination does not resolve to an existing object. + When: + - Calling process_domains for a single domain. + Then: + - The address is created first; then the group is created lazily seeded with that + first object (never as an empty static group); then the rule is created against the + now-existing group. No edit-address-group call is emitted for the seed domain. + """ + calls: list = [] + + def _capture(name, args): + calls.append((name, args)) + seq = { + "pan-os-list-address-groups": [ok_entry({"Panorama.AddressGroups": []})], + "pan-os-list-rules": [ok_entry({"Panorama.SecurityRule": []})], + "pan-os-create-rule": [ok_entry()], + "pan-os-move-rule": [ok_entry()], + "pan-os-get-address": [err_entry("not found")], + "pan-os-create-address": [ok_entry()], + "pan-os-create-address-group": [ok_entry()], + } + return seq.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + + _pan_os(["evil.example.com"]).process_domains() + + names = [c[0] for c in calls] + assert names.index("pan-os-create-address") < names.index("pan-os-create-address-group") + assert names.index("pan-os-create-address-group") < names.index("pan-os-create-rule") + group_create_args = next(args for name, args in calls if name == "pan-os-create-address-group") + assert group_create_args["type"] == "static" + # The group is created once with the full member list (all objects created first). + assert group_create_args["addresses"] == ["Cortex-evil.example.com"] + rule_create_args = next(args for name, args in calls if name == "pan-os-create-rule") + assert rule_create_args["destination"] == "Blocked Domains - Cortex" + assert "pan-os-edit-address-group" not in names + + +def test_process_domains_multi_domain_creates_all_objects_before_group(monkeypatch): + """ + Given: + - A tenant with no group, no rule, and neither address object existing, for TWO domains. + When: + - Calling process_domains for both domains. + Then: + - Both address objects are created BEFORE the single group-create, and the group is created + once with the full member list (this is the ordering that avoids the reference error). + """ + calls = _mock_execute( + monkeypatch, + { + "pan-os-get-address": [err_entry("not found")], + "pan-os-list-address-groups": [ok_entry({"Panorama.AddressGroups": []})], + "pan-os-list-rules": [ok_entry({"Panorama.SecurityRule": []})], + }, + ) + _pan_os(["evil.example.com", "phish.attacker.net"]).process_domains() + + names = [name for name, _ in calls] + create_addresses = [i for i, name in enumerate(names) if name == "pan-os-create-address"] + group_create_index = names.index("pan-os-create-address-group") + # Both objects are created before the single group-create. + assert len(create_addresses) == 2 + assert max(create_addresses) < group_create_index + assert names.count("pan-os-create-address-group") == 1 + group_args = next(args for name, args in calls if name == "pan-os-create-address-group") + assert group_args["addresses"] == ["Cortex-evil.example.com", "Cortex-phish.attacker.net"] + + +def test_process_domains_all_unchanged(monkeypatch): + """ + Given: + - A tenant where the group, rule, and address object all already exist and the + address object is already a member of the group. + When: + - Calling process_domains. + Then: + - The resulting row is Done / Unchanged (no create/modify happened). + """ + obj = "Cortex-evil.example.com" + calls = _mock_execute( + monkeypatch, + { + "pan-os-get-address": [ok_entry({"Panorama.Addresses": [{"Name": obj, "Tags": ["cortex-blocked-domains"]}]})], + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + { + "Name": "Blocked Domains - Cortex", + "Type": "static", + "Addresses": [obj], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + "pan-os-list-rules": [ + ok_entry( + { + "Panorama.SecurityRule": [ + { + "Name": "Cortex - Block Domain", + "Destination": ["Blocked Domains - Cortex"], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + }, + ) + pan = _pan_os(["evil.example.com"]) + rows = pan.process_domains() + assert rows[0]["Status"] == STATUS_DONE + # Everything already in place (incl. the tag) -> no change -> commit will be skipped. + assert pan._address_made_changes is False + assert pan._rule_changed is False + assert "Action" not in rows[0] + # A pre-existing, unchanged rule must NOT be moved (no needless candidate-config churn). + assert not any(name == "pan-os-move-rule" for name, _ in calls) + + +def test_process_domains_modified_when_added_to_existing_group(monkeypatch): + """ + Given: + - A tenant where the group and rule already exist but the group has no members yet + and the address object for the requested domain does not exist. + When: + - Calling process_domains. + Then: + - The address is created and added to the existing group; the resulting row is + Done / Created (most-significant action of the create-address step). + """ + _mock_execute( + monkeypatch, + { + "pan-os-get-address": [err_entry("not found")], + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + { + "Name": "Blocked Domains - Cortex", + "Type": "static", + "Addresses": [], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + "pan-os-list-rules": [ + ok_entry( + { + "Panorama.SecurityRule": [ + { + "Name": "Cortex - Block Domain", + "Destination": ["Blocked Domains - Cortex"], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + }, + ) + pan = _pan_os(["evil.example.com"]) + rows = pan.process_domains() + assert rows[0]["Status"] == STATUS_DONE + # A new object was created and added to the group -> a change occurred -> commit will run. + assert pan._address_made_changes is True + assert "Action" not in rows[0] + + +def test_process_domains_existing_rule_missing_group_is_edited(monkeypatch): + """ + Given: + - A tenant where the rule exists but its destination does not yet reference our + address group; the group and the address object already exist. + When: + - Calling process_domains. + Then: + - The rule is edited to include the group in its destination; the resulting row is + Done / Unchanged because the address object and its group membership were unchanged + (rule-level fix is a group-scope, not per-domain, mutation). + """ + obj = "Cortex-evil.example.com" + _mock_execute( + monkeypatch, + { + "pan-os-get-address": [ok_entry({"Panorama.Addresses": [{"Name": obj, "Tags": ["cortex-blocked-domains"]}]})], + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + { + "Name": "Blocked Domains - Cortex", + "Type": "static", + "Addresses": [obj], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + "pan-os-list-rules": [ + ok_entry( + { + "Panorama.SecurityRule": [ + { + "Name": "Cortex - Block Domain", + "Destination": ["something-else"], + "Tags": ["cortex-blocked-domains"], + } + ] + } + ) + ], + }, + ) + pan = _pan_os(["evil.example.com"]) + rows = pan.process_domains() + assert rows[0]["Status"] == STATUS_DONE + # Object + membership unchanged, but the rule was edited to add the group -> _rule_changed. + assert pan._address_made_changes is False + assert pan._rule_changed is True + assert "Action" not in rows[0] + + +def _tag_capture(monkeypatch): + """Patch executeCommand with a dict-based capture that tolerates call ordering. + + Returns the ``calls`` list of (command_name, args) tuples. Read/probe commands return + fixtures that model an already-existing object/group/rule WITHOUT the configured tag, so the + merge/edit paths are exercised. Every other command returns a bare success entry. + """ + obj = "Cortex-evil.example.com" + calls: list = [] + fixtures = { + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + {"Name": "Blocked Domains - Cortex", "Type": "static", "Addresses": [obj], "Tags": ["other"]} + ] + } + ) + ], + "pan-os-list-rules": [ + ok_entry( + { + "Panorama.SecurityRule": [ + {"Name": "Cortex - Block Domain", "Destination": ["Blocked Domains - Cortex"], "Tags": ["other"]} + ] + } + ) + ], + "pan-os-get-address": [ok_entry({"Panorama.Addresses": [{"Name": obj, "Tags": ["other"]}]})], + } + + def _capture(name, args): + calls.append((name, args)) + return fixtures.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + return calls + + +def test_process_domains_merges_tag_on_existing_object_group_and_rule(monkeypatch): + """ + Given: + - A tenant where the address object, address group, and rule all already exist, each + carrying a different pre-existing tag ('other') but NOT the configured tag. + When: + - Calling process_domains with tag='cortex-blocked-domains'. + Then: + - The configured tag is merged (never replacing) onto all three entities: + * pan-os-edit-address with element_to_change=tag and the merged list ['other', tag] + * pan-os-edit-address-group with tags=['other', tag] + * pan-os-edit-rule with element_to_change=tag, behaviour=add, element_value=tag + """ + calls = _tag_capture(monkeypatch) + pan = _pan_os(["evil.example.com"]) + rows = pan.process_domains() + + assert rows[0]["Status"] == STATUS_DONE + + edit_address = [args for name, args in calls if name == "pan-os-edit-address"] + assert edit_address == [ + { + "name": "Cortex-evil.example.com", + "element_to_change": "tag", + "element_value": ["other", "cortex-blocked-domains"], + } + ] + + # The group edit carries element_to_add (required by the static-group guard) together with the + # merged tags in one call. The object is already a member, so the full list is re-added (no-op). + edit_group = [args for name, args in calls if name == "pan-os-edit-address-group" and "tags" in args] + assert edit_group == [ + { + "name": "Blocked Domains - Cortex", + "type": "static", + "element_to_add": ["Cortex-evil.example.com"], + "tags": ["other", "cortex-blocked-domains"], + } + ] + + edit_rule_tag = [args for name, args in calls if name == "pan-os-edit-rule" and args.get("element_to_change") == "tag"] + assert edit_rule_tag == [ + { + "rulename": "Cortex - Block Domain", + "element_to_change": "tag", + "element_value": "cortex-blocked-domains", + "behaviour": "add", + "pre_post": "pre-rulebase", + } + ] + + +def test_process_domains_does_not_re_tag_when_tag_already_present(monkeypatch): + """ + Given: + - The demo scenario: an existing FQDN, an existing group, and an existing rule that ALL + already carry the configured tag; the user re-runs the command. + When: + - Calling process_domains with the same tag. + Then: + - No tag edit is issued on the object, group, or rule (the tag is never duplicated), and + nothing changes so the commit is skipped. + """ + obj = "Cortex-evil.example.com" + tag = "cortex-blocked-domains" + calls: list = [] + fixtures = { + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + {"Name": "Blocked Domains - Cortex", "Type": "static", "Addresses": [obj], "Tags": [tag]} + ] + } + ) + ], + "pan-os-list-rules": [ + ok_entry( + { + "Panorama.SecurityRule": [ + {"Name": "Cortex - Block Domain", "Destination": ["Blocked Domains - Cortex"], "Tags": [tag]} + ] + } + ) + ], + "pan-os-get-address": [ok_entry({"Panorama.Addresses": [{"Name": obj, "Tags": [tag]}]})], + } + + def _capture(name, args): + calls.append((name, args)) + return fixtures.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + + pan = _pan_os(["evil.example.com"]) + pan.process_domains() + + assert [args for name, args in calls if name == "pan-os-edit-address"] == [] + assert [args for name, args in calls if name == "pan-os-edit-address-group" and "tags" in args] == [] + assert [args for name, args in calls if name == "pan-os-edit-rule" and args.get("element_to_change") == "tag"] == [] + assert pan._address_made_changes is False + assert pan._rule_changed is False + + +def test_process_domains_new_rule_name_does_not_re_tag_existing_object_and_group(monkeypatch): + """ + Given: + - An existing FQDN and existing group that ALREADY carry the configured tag, but a brand-new + rule name that does not exist yet (the "just add a rule" case). + When: + - Calling process_domains. + Then: + - The object and group are NOT re-tagged (no duplication); only the new rule is created + (which carries the tag at creation time via its own create args). + """ + obj = "Cortex-evil.example.com" + tag = "cortex-blocked-domains" + calls: list = [] + fixtures = { + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + {"Name": "Blocked Domains - Cortex", "Type": "static", "Addresses": [obj], "Tags": [tag]} + ] + } + ) + ], + # Rule does not exist yet -> it will be created (with the tag in its create args). + "pan-os-list-rules": [ok_entry({"Panorama.SecurityRule": []})], + "pan-os-get-address": [ok_entry({"Panorama.Addresses": [{"Name": obj, "Tags": [tag]}]})], + } + + def _capture(name, args): + calls.append((name, args)) + return fixtures.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + + pan = _pan_os(["evil.example.com"]) + pan.process_domains() + + # Object and group already carry the tag -> no re-tag edits. + assert [args for name, args in calls if name == "pan-os-edit-address"] == [] + assert [args for name, args in calls if name == "pan-os-edit-address-group" and "tags" in args] == [] + # The new rule is created and carries the tag at creation. + create_rule = [args for name, args in calls if name == "pan-os-create-rule"] + assert len(create_rule) == 1 + assert create_rule[0]["tags"] == tag + assert pan._rule_changed is True + + +def _pan_os_with_log_forwarding(domains, profile): + """Build a PanOs flow configured with a log-forwarding profile name (rule-only).""" + pan = _pan_os(domains) + pan.log_forwarding_name = profile + return pan + + +def _lf_capture(monkeypatch, existing_profile): + """Patch executeCommand modelling an existing object/group/rule (everything already in place). + + The existing rule carries ``existing_profile`` as its LogForwardingProfile. Returns the + ``calls`` list of (command_name, args) tuples. Tags are pre-present so no tag edits fire. + """ + obj = "Cortex-evil.example.com" + tag = "cortex-blocked-domains" + calls: list = [] + fixtures = { + "pan-os-list-address-groups": [ + ok_entry( + { + "Panorama.AddressGroups": [ + {"Name": "Blocked Domains - Cortex", "Type": "static", "Addresses": [obj], "Tags": [tag]} + ] + } + ) + ], + "pan-os-list-rules": [ + ok_entry( + { + "Panorama.SecurityRule": [ + { + "Name": "Cortex - Block Domain", + "Destination": ["Blocked Domains - Cortex"], + "Tags": [tag], + "LogForwardingProfile": existing_profile, + } + ] + } + ) + ], + "pan-os-get-address": [ok_entry({"Panorama.Addresses": [{"Name": obj, "Tags": [tag]}]})], + } + + def _capture(name, args): + calls.append((name, args)) + return fixtures.get(name, [ok_entry()]) + + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", _capture) + return calls + + +def test_process_domains_sets_log_forwarding_on_existing_rule_when_absent(monkeypatch): + """ + Given: + - An existing rule with NO log-forwarding profile, and a flow configured with one. + When: + - Calling process_domains. + Then: + - pan-os-edit-rule is issued with element_to_change=log-forwarding and the configured + profile, and _rule_changed is set. + """ + calls = _lf_capture(monkeypatch, existing_profile="") + pan = _pan_os_with_log_forwarding(["evil.example.com"], "cortex-lfp") + pan.process_domains() + + edit_lf = [args for name, args in calls if name == "pan-os-edit-rule" and args.get("element_to_change") == "log-forwarding"] + assert edit_lf == [ + { + "rulename": "Cortex - Block Domain", + "element_to_change": "log-forwarding", + "element_value": "cortex-lfp", + "pre_post": "pre-rulebase", + } + ] + assert pan._rule_changed is True + + +def test_process_domains_sets_log_forwarding_on_existing_rule_when_different(monkeypatch): + """ + Given: + - An existing rule whose log-forwarding profile differs from the configured one. + When: + - Calling process_domains. + Then: + - pan-os-edit-rule replaces it with the configured profile (single-value, no dedup concept). + """ + calls = _lf_capture(monkeypatch, existing_profile="old-lfp") + pan = _pan_os_with_log_forwarding(["evil.example.com"], "cortex-lfp") + pan.process_domains() + + edit_lf = [args for name, args in calls if name == "pan-os-edit-rule" and args.get("element_to_change") == "log-forwarding"] + assert len(edit_lf) == 1 + assert edit_lf[0]["element_value"] == "cortex-lfp" + assert pan._rule_changed is True + + +def test_process_domains_does_not_set_log_forwarding_when_already_matches(monkeypatch): + """ + Given: + - An existing rule that ALREADY carries the configured log-forwarding profile. + When: + - Calling process_domains. + Then: + - No log-forwarding edit is issued (no unnecessary edit), and since nothing else changed + the rule is left untouched. + """ + calls = _lf_capture(monkeypatch, existing_profile="cortex-lfp") + pan = _pan_os_with_log_forwarding(["evil.example.com"], "cortex-lfp") + pan.process_domains() + + edit_lf = [args for name, args in calls if name == "pan-os-edit-rule" and args.get("element_to_change") == "log-forwarding"] + assert edit_lf == [] + assert pan._rule_changed is False + + +def test_process_domains_dynamic_group_is_failed(monkeypatch): + """ + Given: + - A tenant where the target group already exists but as a *dynamic* address group + (which cannot accept manually added static members). + When: + - Calling process_domains. + Then: + - The domain could not be blocked, so the row is marked Failed / Failed and the message + explains the group is dynamic. + """ + _mock_execute( + monkeypatch, + { + "pan-os-get-address": [err_entry("not found")], + "pan-os-list-address-groups": [ + ok_entry({"Panorama.AddressGroups": [{"Name": "Blocked Domains - Cortex", "Type": "dynamic", "Match": "x"}]}) + ], + }, + ) + # Capture the abort log so it does not leak to stdout (conftest fails on any stdout). + errors: list = [] + monkeypatch.setattr(BlockDomain.demisto, "error", lambda msg: errors.append(msg)) + + rows = _pan_os(["evil.example.com"]).process_domains() + assert rows[0]["Status"] == STATUS_FAILED + assert rows[0]["Result"] == RESULT_FAILED + assert "dynamic" in rows[0]["Message"] + # The aborted domains are logged for troubleshooting. + assert any("evil.example.com" in msg for msg in errors) + + +def test_process_domains_moves_rule_to_top_when_rule_created(monkeypatch): + """ + Given: + - A tenant where neither the rule nor the address object exist yet, so the rule is created. + When: + - Calling process_domains. + Then: + - pan-os-move-rule is issued to enforce top placement for the newly created rule. + """ + calls = _mock_execute( + monkeypatch, + { + "pan-os-get-address": [err_entry("not found")], + "pan-os-list-address-groups": [ + ok_entry({"Panorama.AddressGroups": [{"Name": "Blocked Domains - Cortex", "Type": "static", "Addresses": []}]}) + ], + "pan-os-list-rules": [ok_entry({"Panorama.SecurityRule": []})], + }, + ) + pan = _pan_os(["evil.example.com"]) + pan.process_domains() + assert pan._rule_changed is True + assert any(name == "pan-os-move-rule" for name, _ in calls) + + +def test_process_domains_failure_marks_row_failed(monkeypatch): + """ + Given: + - A tenant where the group and rule do not exist yet, and the pan-os-create-address + command fails (e.g. permission denied) before the group and rule are touched. + When: + - Calling process_domains. + Then: + - The resulting row is Failed / Failed and the error message from PAN-OS is surfaced, + and the full traceback is logged via demisto.error. + """ + _mock_execute( + monkeypatch, + { + "pan-os-get-address": [err_entry("not found")], + "pan-os-create-address": [err_entry("permission denied")], + "pan-os-list-address-groups": [ok_entry({"Panorama.AddressGroups": []})], + "pan-os-list-rules": [ok_entry({"Panorama.SecurityRule": []})], + }, + ) + # Capture the traceback log so it does not leak to stdout (conftest fails on any stdout). + errors: list = [] + monkeypatch.setattr(BlockDomain.demisto, "error", lambda msg: errors.append(msg)) + + rows = _pan_os(["evil.example.com"]).process_domains() + assert rows[0]["Status"] == STATUS_FAILED + assert rows[0]["Result"] == RESULT_FAILED + assert "permission denied" in rows[0]["Message"] + assert any("process_domains failed" in msg for msg in errors) + + +def test_build_verbose_human_readable_joins_with_blank_lines(): + """ + Given: + - A list of response entries where some carry a HumanReadable string and some do not. + When: + - Calling build_verbose_human_readable. + Then: + - Entries without HumanReadable are skipped; the rest are joined with a blank line + separator, prefixed by a leading blank line so the block detaches from the summary + table above it. + """ + responses = [ + [ok_entry(contents="c1")], + [{"Type": 1, "Contents": "c2", "HumanReadable": "HR-two", "EntryContext": {}}], + [{"Type": 1, "Contents": "c3", "HumanReadable": "HR-three", "EntryContext": {}}], + ] + verbose_hr = build_verbose_human_readable(responses) + assert verbose_hr == "\n\nHR-two\n\nHR-three" + + +def test_build_verbose_human_readable_empty_when_no_hr(): + """ + Given: + - A response list where no entry has a HumanReadable string, or an empty list. + When: + - Calling build_verbose_human_readable. + Then: + - Returns an empty string (nothing to append to the summary table). + """ + assert build_verbose_human_readable([[ok_entry(contents="c1")]]) == "" + assert build_verbose_human_readable([]) == "" + + +def test_build_final_command_results_non_verbose_is_table_only(): + """ + Given: + - A rows list and a set of responses that include HumanReadable content, with + verbose=False. + When: + - Calling build_final_command_results. + Then: + - The returned CommandResults uses the BlockDomain context prefix, exposes the + rows unchanged as outputs, renders the summary table containing the domain, and does + NOT append any of the per-command verbose HR blocks. + """ + rows = [ + { + "Domain": "a.com", + "Brand": "Panorama", + "Status": STATUS_DONE, + "Result": RESULT_SUCCESS, + "RuleName": "Cortex - Block Domain", + "Message": "ok", + } + ] + responses = [[{"Type": 1, "Contents": "c", "HumanReadable": "HR", "EntryContext": {}}]] + + result = build_final_command_results(rows, verbose=False, responses=responses) + assert result.outputs_prefix == "BlockDomain" + assert result.outputs == rows + assert "a.com" in result.readable_output + assert "HR" not in result.readable_output + + +def test_build_final_command_results_verbose_appends_command_hr(): + """ + Given: + - A rows list and responses with a HumanReadable block, with verbose=True. + When: + - Calling build_final_command_results. + Then: + - The returned CommandResults exposes the rows as outputs, renders the summary table + containing the domain, and appends the per-command HR block at the end of the + readable_output (so users can see exactly what each downstream call produced). + """ + rows = [ + { + "Domain": "a.com", + "Brand": "Panorama", + "Status": STATUS_DONE, + "Result": RESULT_SUCCESS, + "RuleName": "Cortex - Block Domain", + "Message": "ok", + } + ] + responses = [[{"Type": 1, "Contents": "c", "HumanReadable": "HR-one", "EntryContext": {}}]] + + result = build_final_command_results(rows, verbose=True, responses=responses) + assert result.outputs == rows + assert "a.com" in result.readable_output + assert result.readable_output.endswith("HR-one") + + +def test_build_final_command_results_all_failed_is_error_entry(): + """ + Given: + - A rows list where every row has Result == Failed. + When: + - Calling build_final_command_results. + Then: + - The returned CommandResults is marked as an ERROR entry, still exposes the rows as + outputs, and its readable_output carries the "All runs failed" header so the war room + and any downstream playbook can branch on the failure. + """ + rows = [ + { + "Domain": "a.com", + "Brand": "Panorama", + "Status": STATUS_FAILED, + "Result": RESULT_FAILED, + "RuleName": "", + "Message": "boom", + }, + { + "Domain": "b.com", + "Brand": "Panorama", + "Status": STATUS_FAILED, + "Result": RESULT_FAILED, + "RuleName": "", + "Message": "boom", + }, + ] + + result = build_final_command_results(rows, verbose=False, responses=[]) + assert result.entry_type == BlockDomain.EntryType.ERROR + assert result.outputs == rows + assert "All runs failed" in result.readable_output + + +def test_build_final_command_results_partial_failure_is_not_error_entry(): + """ + Given: + - A rows list with at least one successful row alongside a failed row. + When: + - Calling build_final_command_results. + Then: + - The returned CommandResults is NOT an ERROR entry (entry_type is unset / not ERROR), + so a partial success is still reported as a normal result. + """ + rows = [ + { + "Domain": "a.com", + "Brand": "Panorama", + "Status": STATUS_DONE, + "Result": RESULT_SUCCESS, + "RuleName": "Cortex - Block Domain", + "Message": "ok", + }, + { + "Domain": "b.com", + "Brand": "Panorama", + "Status": STATUS_FAILED, + "Result": RESULT_FAILED, + "RuleName": "", + "Message": "boom", + }, + ] + + result = build_final_command_results(rows, verbose=False, responses=[]) + assert result.entry_type != BlockDomain.EntryType.ERROR + assert "All runs failed" not in result.readable_output + + +def test_build_final_command_results_empty_rows_is_not_error_entry(): + """ + Given: + - An empty rows list (e.g. no domains reached a vendor). + When: + - Calling build_final_command_results. + Then: + - The run is NOT reported as an error (an empty run is not an all-failed run). + """ + result = build_final_command_results([], verbose=False, responses=[]) + assert result.entry_type != BlockDomain.EntryType.ERROR + assert "All runs failed" not in result.readable_output + + +def test_pan_os_commit_status_reports_pending_while_job_active(monkeypatch): + """ + Given: + - pan-os-commit-status returns a job with result 'OK' but status 'ACT' (still running). + When: + - Calling pan_os_commit_status. + Then: + - The war-room output must NOT show 'Success' while the job is still active; it reports + 'Pending' (job.result 'OK' and job.status 'ACT' are independent PAN-OS fields, and success + is only reported once the job status is 'FIN'). + """ + active_entry = { + "Type": 1, + "Contents": {"response": {"result": {"job": {"result": "OK", "status": "ACT"}}}}, + "HumanReadable": "", + "EntryContext": {}, + } + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", lambda *a, **k: [active_entry]) + + result = pan_os_commit_status({"commit_job_id": "123"}, []) + + assert result.outputs == {"JobID": "123", "Status": "Pending"} # type: ignore[attr-defined] + + +def test_pan_os_commit_status_reports_success_only_when_finished(monkeypatch): + """ + Given: + - pan-os-commit-status returns a finished job (status 'FIN') with result 'OK'. + When: + - Calling pan_os_commit_status. + Then: + - The output reports 'Success' and polling stops. + """ + fin_entry = { + "Type": 1, + "Contents": {"response": {"result": {"job": {"result": "OK", "status": "FIN"}}}}, + "HumanReadable": "", + "EntryContext": {}, + } + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", lambda *a, **k: [fin_entry]) + + result = pan_os_commit_status({"commit_job_id": "123"}, []) + + assert result.outputs == {"JobID": "123", "Status": "Success"} # type: ignore[attr-defined] + assert not BlockDomain.is_polling_in_progress(result) + + +def test_pan_os_commit_reports_failure_on_error(monkeypatch): + """ + Given: + - pan-os-commit returns an error entry. + When: + - Calling pan_os_commit. + Then: + - The failure is surfaced (Status 'Failure') and polling stops, instead of being silently + treated as 'no changes to commit'. + """ + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", lambda *a, **k: [err_entry("commit failed")]) + monkeypatch.setattr(BlockDomain.demisto, "setContext", lambda *a, **k: None) + + result = pan_os_commit({}, []) + + assert result.outputs == {"Status": "Failure"} # type: ignore[attr-defined] + assert not BlockDomain.is_polling_in_progress(result) + + +def test_pan_os_push_to_device_reports_failure_on_error(monkeypatch): + """ + Given: + - pan-os-push-to-device-group returns an error entry. + When: + - Calling pan_os_push_to_device. + Then: + - The failure is surfaced (Status 'Failure') and polling stops, instead of being silently + treated as 'no changes to push'. + """ + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", lambda *a, **k: [err_entry("push failed")]) + monkeypatch.setattr(BlockDomain.demisto, "setContext", lambda *a, **k: None) + + result = pan_os_push_to_device({}, []) + + assert result.outputs == {"Status": "Failure"} # type: ignore[attr-defined] + assert not BlockDomain.is_polling_in_progress(result) + + +def test_pan_os_push_status_error_contents_is_terminal_failure(monkeypatch): + """ + Given: + - pan-os-push-status returns an error entry whose Contents is a plain string + (PAN-OS surfaces errors as a bare string instead of the nested status dict). + When: + - Calling pan_os_push_status. + Then: + - The function does NOT crash with 'str object has no attribute get'; it stops polling + (no ScheduledCommand attached) and reports a Failure status for the job. + """ + monkeypatch.setattr( + BlockDomain.demisto, + "executeCommand", + lambda *a, **k: [err_entry("Failed to execute pan-os-push-status. Error: job not found")], + ) + + # The @polling_function decorator unwraps the PollResult and returns its CommandResults response + # at runtime, though the declared return type is still PollResult (hence the type: ignore below). + result = pan_os_push_status({"push_job_id": "123"}, []) + + assert not BlockDomain.is_polling_in_progress(result) + assert result.outputs == {"JobID": "123", "Status": "Failure"} # type: ignore[attr-defined] + + +def test_pan_os_push_status_fin_stops_polling(monkeypatch): + """ + Given: + - pan-os-push-status returns a well-formed nested dict whose job status is 'FIN'. + When: + - Calling pan_os_push_status. + Then: + - Polling stops (no ScheduledCommand attached) and the reported job status is 'FIN'. + """ + fin_entry = { + "Type": 1, + "Contents": {"response": {"result": {"job": {"status": "FIN"}}}}, + "HumanReadable": "", + "EntryContext": {}, + } + monkeypatch.setattr(BlockDomain.demisto, "executeCommand", lambda *a, **k: [fin_entry]) + + result = pan_os_push_status({"push_job_id": "456"}, []) + + assert not BlockDomain.is_polling_in_progress(result) + assert result.outputs == {"Status": "FIN", "JobID": "456"} # type: ignore[attr-defined] + + +def _install_fake_context(monkeypatch): + """Back demisto.context()/setContext() with an in-memory dict, mirroring platform behavior. + + Returns the backing store so tests can inspect exactly what was serialized to context. + """ + store: dict = {} + monkeypatch.setattr(BlockDomain.demisto, "context", lambda: dict(store)) + monkeypatch.setattr(BlockDomain.demisto, "setContext", lambda key, value: store.__setitem__(key, value)) + return store + + +def test_save_and_restore_responses_json_round_trip(monkeypatch): + """ + Given: + - A PanOs instance whose accumulated responses contain the realistic PAN-OS entry shape + (nested Contents dict, Metadata, None HumanReadable) written to context as JSON. + When: + - Calling save_responses on one polling cycle and restore_responses on the next. + Then: + - The context value is valid JSON (not a Python repr), and the responses survive the + json.dumps -> json.loads round-trip byte-for-byte equal to the reduced form. + """ + store = _install_fake_context(monkeypatch) + + pan_os = _pan_os(["evil.example.com"]) + pan_os.responses = [ + [ + { + "Type": 1, + "Contents": {"response": {"result": {"job": {"status": "FIN", "id": "42"}}}}, + "HumanReadable": None, + "Metadata": {"instance": "Panorama_QA", "brand": "Panorama"}, + "EntryContext": {"dropped": "not serialized"}, + } + ] + ] + expected_reduced = pan_os.reduce_responses() + + pan_os.save_responses() + + # Stored value must be real JSON that json.loads can parse (would fail on a Python repr). + stored = store["panorama_responses"] + assert json.loads(stored) == expected_reduced + + # A fresh instance restoring from the same context recovers the reduced responses exactly. + fresh = _pan_os(["evil.example.com"]) + fresh.restore_responses() + assert fresh.responses == expected_reduced + + +def test_finish_reads_rows_and_responses_as_json_then_clears_context(monkeypatch): + """ + Given: + - Context holds block_domain_rows and panorama_responses that were written as JSON by a + previous polling cycle. + When: + - Calling finish(). + Then: + - The rows are parsed back from JSON and returned; the accumulated responses are restored + onto the instance for verbose output; and all polling context keys are cleared. + """ + store = _install_fake_context(monkeypatch) + + rows = [ + { + "Domain": "evil.example.com", + "Brand": "Panorama", + "Status": STATUS_DONE, + "Result": RESULT_SUCCESS, + "RuleName": "Cortex - Block Domain", + "Message": "ok", + } + ] + responses = [[{"HumanReadable": "HR", "Contents": "ok", "Type": 1, "Metadata": None}]] + store["block_domain_rows"] = json.dumps(rows) + store["panorama_responses"] = json.dumps(responses) + store["commit_job_id"] = "999" + + pan_os = _pan_os(["evil.example.com"]) + result = pan_os.finish() + + assert result == rows + assert pan_os.responses == responses + # All polling context keys are cleared on finish. + assert store["commit_job_id"] == "" + assert store["push_job_id"] == "" + assert store["panorama_responses"] == "" + assert store["block_domain_rows"] == "" + + +def test_finish_tolerates_corrupt_context_data(monkeypatch): + """ + Given: + - Context holds a non-JSON (corrupt) block_domain_rows value. + When: + - Calling finish(). + Then: + - finish() does not raise; it degrades to an empty list and still clears context. + """ + store = _install_fake_context(monkeypatch) + store["block_domain_rows"] = "{not valid json" + store["panorama_responses"] = "also not json" + + pan_os = _pan_os(["evil.example.com"]) + result = pan_os.finish() + + assert result == [] + assert store["block_domain_rows"] == "" diff --git a/Packs/AggregatedScripts/Scripts/BlockDomain/README.md b/Packs/AggregatedScripts/Scripts/BlockDomain/README.md new file mode 100644 index 000000000000..584dc2fb802a --- /dev/null +++ b/Packs/AggregatedScripts/Scripts/BlockDomain/README.md @@ -0,0 +1,38 @@ +The script blocks a list of domain FQDNs in supported integrations. Currently only supports Palo Alto Networks Panorama. + +## Script Data + +--- + +| **Name** | **Description** | +| --- | --- | +| Script Type | python3 | +| Cortex XSOAR Version | 6.10.0 | + +## Inputs + +--- + +| **Argument Name** | **Description** | +| --- | --- | +| domain_list | A comma-separated list of domain FQDNs to block. Wildcard entries \(e.g. \*.evil.com\) are not supported and are skipped. | +| rule_name | The name of the rule which will be created in the relevant products. Default: `Cortex - Block Domain`. | +| log_forwarding_name | The Panorama log forwarding object name that specifies the Log Forwarding setting to apply to the PAN-OS custom rules. | +| address_group | The name of the PAN-OS Panorama or Firewall address group used to hold the blocked domain FQDN objects. Default: `Blocked Domains - Cortex`. | +| auto_commit | Whether to commit the new rule and push to the device group at the end of the run. Default: `true`. | +| tag | The designated tag name for the domain FQDN object. Applied to every object the script creates. Default: `cortex-blocked-domains`. | +| brands | A comma-separated list of integration brands to run the command for. If not provided, the command runs for all available integrations. | +| verbose | Whether to retrieve a human-readable entry for every command or only the final result. True retrieves a human-readable entry for every command. False retrieves a human-readable entry only for the final result. Default: `false`. | + +## Outputs + +--- + +| **Path** | **Description** | **Type** | +| --- | --- | --- | +| BlockDomain.Domain | The domain FQDN that was processed. | String | +| BlockDomain.Brand | The brand \(integration\) used to block the domain. | String | +| BlockDomain.Status | The lifecycle status of the action. One of Done, Pending, Skipped, Failed. | String | +| BlockDomain.Result | The result of the action. Success or Failed. | String | +| BlockDomain.RuleName | The name of the rule used for this integration. Empty if no rule was used. | String | +| BlockDomain.Message | A message concerning the result of the action. | String | diff --git a/Packs/AggregatedScripts/Scripts/BlockExternalIp/BlockExternalIp.yml b/Packs/AggregatedScripts/Scripts/BlockExternalIp/BlockExternalIp.yml index 07a6db1de48a..49d3d6146d65 100644 --- a/Packs/AggregatedScripts/Scripts/BlockExternalIp/BlockExternalIp.yml +++ b/Packs/AggregatedScripts/Scripts/BlockExternalIp/BlockExternalIp.yml @@ -84,7 +84,7 @@ type: python subtype: python3 compliantpolicies: - IP Blockage -dockerimage: demisto/python3:3.12.13.10116658 +dockerimage: demisto/python3:3.12.14.13495286 fromversion: 6.1.0 marketplaces: - xsoar diff --git a/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl.py b/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl.py new file mode 100644 index 000000000000..193f089c9543 --- /dev/null +++ b/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl.py @@ -0,0 +1,1300 @@ +import ast +import json +import re + +import demistomock as demisto +from CommonServerPython import * + + +POLLING = False +# Set by the commit and the push status functions when PAN-OS reported a terminal failure. Read by +# manage_pan_os_flow, which must then skip the push and report the failure instead of a success. +JOB_FAILURE_MESSAGE = "" + +SUPPORTED_BRANDS = ["Panorama"] +PAN_OS_BRAND = "Panorama" + +# PAN-OS returns this substring when an object does not exist. The generic "Request Failed." prefix +# is shared with real failures, so it must not be used for the not-found decision. +OBJECT_NOT_FOUND = "Object not present" +# Raised by pan-os-edit-custom-url-category / pan-os-edit-url-filter when the target object still +# carries uncommitted changes (@dirtyId). +DIRTY_OBJECT_ERROR = "Please commit the instance prior to editing" + +URL_LIST_TYPE = "URL List" +MAX_URL_LENGTH = 255 + +XML_SPECIAL_CHARS = {"<", ">", "&", '"', "'"} + +# The description stamped on every object this script creates, so an operator can tell them apart +# from user-managed objects. +OBJECT_DESCRIPTION = "Created by the Cortex block-url script." + +POLLING_INTERVAL = 30 +POLLING_TIMEOUT = 1200 + +# A commit job that finished with any result other than "OK" failed. Warnings are not a failure. +COMMIT_JOB_FINISHED_STATUS = "FIN" +COMMIT_JOB_SUCCESS_RESULT = "OK" +# The terminal statuses of a push job. Anything else, including an empty status, means "keep polling". +PUSH_COMPLETED_STATUS = "Completed" +PUSH_FAILURE_STATUSES = ("FAIL", "Failed") + + +class BlockUrlError(Exception): + """Raised when the PAN-OS flow cannot continue and the failure must be reported to the user.""" + + +""" HELPER FUNCTIONS """ + + +def _text(value: Any) -> str: + """Normalize a PAN-OS context value that may be wrapped in a '#text' dictionary. + + PAN-OS returns plain strings for committed objects, but dictionaries carrying '#text' and + '@dirtyId' keys while the object has uncommitted changes. + Args: + value (Any): The raw context value. + Returns: + The plain string value. + """ + if isinstance(value, dict): + return str(value.get("#text", "")) + if value is None: + return "" + return str(value) + + +def _text_list(value: Any) -> list[str]: + """Normalize a PAN-OS context value that holds zero, one or many possibly wrapped members. + + Args: + value (Any): The raw context value, a single member or a list of members. + Returns: + A list of plain string values, without empty entries. + """ + if value is None: + return [] + members = value if isinstance(value, list) else [value] + return [text for text in (_text(member) for member in members) if text] + + +def normalize_url(raw_url: str) -> tuple[str, str]: + """Normalize a URL into the form PAN-OS custom URL categories expect. + + PAN-OS validates almost nothing: it stores a scheme, a wildcard or a mixed-case host verbatim + and then silently fails to match them. Every rule below therefore prevents a silent non-block. + Args: + raw_url (str): The URL as supplied by the user. + Returns: + A tuple of (submitted_url, rejection_message). On rejection the submitted URL is empty. + """ + url = (raw_url or "").strip() + if not url: + return "", "URL is empty." + if re.search(r"\s", url): + return "", "URL contains whitespace, which is not supported by PAN-OS custom URL categories." + if "," in url: + return "", "URL contains a comma, which would split it into two entries when submitted to PAN-OS." + url = re.sub(r"^https?://", "", url, flags=re.IGNORECASE) + if "://" in url: + return "", "URL scheme is not supported. Only the http and https schemes are supported." + if "*" in url: + return "", "Wildcards are not supported and will not be passed to PAN-OS." + if xml_special := set(url) & XML_SPECIAL_CHARS: + return "", ( + f"URL contains the character(s) {', '.join(sorted(xml_special))}, which are not supported by " + "PAN-OS custom URL categories." + ) + url = url.rstrip("/") + if not url: + return "", "URL is empty." + if len(url) > MAX_URL_LENGTH: + return "", f"URL exceeds {MAX_URL_LENGTH} chars (PAN-OS limit)." + return lowercase_host(url), "" + + +def lowercase_host(url: str) -> str: + """Lowercase only the host part of a URL. PAN-OS does not fold case, and paths are case sensitive. + + Args: + url (str): A scheme-less URL. + Returns: + The URL with a lowercased host. + """ + host, separator, path = url.partition("/") + return f"{host.lower()}{separator}{path}" + + +def partition_urls(url_list: list[str]) -> tuple[list[dict], list[dict]]: + """Split the requested URLs into the ones that can be submitted and the ones that were rejected. + + Rejected URLs never reach any brand, and are reported with an empty SubmittedURL. + Args: + url_list (list[str]): The raw URLs supplied by the user. + Returns: + A tuple of (accepted entries, rejected entries). + """ + accepted: list[dict] = [] + rejected: list[dict] = [] + for raw_url in url_list: + submitted_url, rejection_message = normalize_url(raw_url) + if rejection_message: + rejected.append({"URL": raw_url, "SubmittedURL": "", "Result": "Failed", "Message": rejection_message}) + else: + accepted.append({"URL": raw_url, "SubmittedURL": submitted_url}) + demisto.debug(f"BU: partitioned the URLs into {len(accepted)} accepted and {len(rejected)} rejected.") + return accepted, rejected + + +def get_error_message(res: list[dict]) -> str: + """Collect the contents of every error entry in a command response. + + Args: + res (list[dict]): The response of demisto.executeCommand. + Returns: + The joined error messages, or an empty string when there was no error. + """ + messages = [str(entry.get("Contents", "")) for entry in res or [] if is_error(entry)] + return ", ".join(messages) + + +def raise_on_command_error(res: list[dict], object_description: str) -> None: + """Raise a BlockUrlError with an actionable message when a command response holds an error. + + Args: + res (list[dict]): The response of demisto.executeCommand. + object_description (str): A human readable description of the PAN-OS object being handled. + Raises: + BlockUrlError: When the response contains an error entry. + """ + message = get_error_message(res) + if not message: + return + if DIRTY_OBJECT_ERROR in message: + raise BlockUrlError( + f"The {object_description} has uncommitted changes on PAN-OS, so it cannot be edited. " + f"Commit the pending changes on PAN-OS and re-run the script. PAN-OS error: {message}" + ) + raise BlockUrlError(message) + + +def get_relevant_context(original_context: dict[str, Any], key: str) -> dict | list: + """Get the relevant context object from the execute_command response. + + Args: + original_context (dict[str, Any]): The original context ('EntryContext') of the response. + key (str): The key to extract from the original_context. + Returns: + A dict or a list that is the relevant command context. + """ + if not original_context: + return {} + if relevant_context := original_context.get(key, {}): + demisto.debug(f"BU: the {key=} was found in the first search.") + return relevant_context + for context_key in original_context: + if context_key.startswith(key): + demisto.debug(f"BU: the {key=} was found in the context as {context_key}") + return original_context.get(context_key, {}) + return {} + + +def get_context_entry(res: list[dict], key: str) -> dict: + """Get a single context entry of a given key from a command response. + + Args: + res (list[dict]): The response of demisto.executeCommand. + key (str): The context key to extract. + Returns: + The context entry as a dict, or an empty dict. + """ + if not res: + return {} + context = get_relevant_context(res[0].get("EntryContext", {}), key) + if isinstance(context, list): + return context[0] if context else {} + return context + + +def check_value_exist_in_context(value: str, context: list | dict, key: str) -> bool: + """Verify if a specific value of a specific key is present in the context. + + Args: + value (str): The value whose existence is checked. + context (list | dict): The command context. + key (str): The key to extract from the context. + Returns: + Whether the value is present. + """ + items = [context] if isinstance(context, dict) else context + for item in items or []: + if _text(item.get(key)) == value: + demisto.debug(f"BU: the {value=} was found.") + return True + demisto.debug(f"BU: the {value=} isn't in the context with the {key=}") + return False + + +def is_polling_resume() -> bool: + """Whether the current run is a polling round that resumes an earlier run. + + Returns: + True when a commit or a push job is already in flight. + """ + incident_context = demisto.context() + return bool(demisto.get(incident_context, "commit_job_id") or demisto.get(incident_context, "push_job_id")) + + +def update_brands_to_run(brands_to_run: list) -> tuple[list, set]: + """Delete the brands that were already executed from the list of brands to run. + + Used when at least one brand finished its execution before the polling of another brand is over. + Args: + brands_to_run (list): The list of brands that should be executed. + Returns: + The list of brands executed in previous runs, and the set of brands to execute now. + """ + if PAN_OS_BRAND not in brands_to_run: + return [], set(brands_to_run) + incident_context = demisto.context() + executed_brands = (incident_context.get("executed_brands", "[]")).replace("'", '"') + try: + executed_brands = json.loads(executed_brands) + except json.JSONDecodeError: + demisto.debug("BU: there was a failure in the json.loads for the executed_brands.") + executed_brands = [] + updated_brands_to_run = {brand for brand in brands_to_run if brand not in executed_brands} + demisto.debug(f"BU: removed {executed_brands=} from {brands_to_run=}") + return executed_brands, updated_brands_to_run + + +def run_execute_command(command_name: str, args: dict[str, Any]) -> list[dict]: + """Execute a command and return its raw results. + + Args: + command_name (str): The name of the command to execute. + args (dict[str, Any]): The arguments to pass to the command. + Returns: + A list of the command result entries. + """ + res = demisto.executeCommand(command_name, args) + return cast(list[dict], res) + + +""" CLIENT CLASS """ + + +class PanOs: + def __init__(self, args: dict) -> None: + self.args = args + self.responses: list = [] + self.is_panorama: bool = False + self.device_group: str = "" + self.already_present_urls: list[str] = [] + self.failure_message: str = "" + + """ TOPOLOGY """ + + def detect_topology(self) -> None: + """Detect whether the configured instance is a Panorama or a plain firewall. + + Panorama requires pre_post on every rule command and scopes objects by device group. + """ + res_pan_os = run_execute_command("pan-os", {"cmd": "", "type": "op"}) + self.responses.append(res_pan_os) + raise_on_command_error(res_pan_os, "PAN-OS system information") + context = get_relevant_context(res_pan_os[0].get("EntryContext", {}), "Panorama.Command") + model = context.get("response", {}).get("result", {}).get("system", {}).get("model", "") # type: ignore[union-attr] + self.is_panorama = model == PAN_OS_BRAND + demisto.debug(f"BU: detected {model=}, {self.is_panorama=}") + + def rule_scope_args(self) -> dict[str, Any]: + """The scoping arguments every rule command needs on Panorama and must not get on a firewall. + + Returns: + The extra arguments for the rule commands. + """ + return {"pre_post": "pre-rulebase"} if self.is_panorama else {} + + def capture_device_group(self, context: dict) -> None: + """Remember the device group PAN-OS echoed back, for the commit, the push and the xpath. + + Args: + context (dict): A PAN-OS command context entry. + """ + device_group = _text(context.get("DeviceGroup")) if isinstance(context, dict) else "" + if device_group: + self.device_group = device_group + demisto.debug(f"BU: captured {self.device_group=}") + + """ CUSTOM URL CATEGORY """ + + def get_custom_url_category(self) -> dict: + """Read the custom URL category, treating "Object not present" as missing rather than failed. + + Returns: + The category context entry, or an empty dict when the category does not exist. + Raises: + BlockUrlError: When the command failed for any other reason. + """ + category = self.args["url_category"] + res = run_execute_command("pan-os-get-custom-url-category", {"name": category}) + if error_message := get_error_message(res): + if OBJECT_NOT_FOUND in error_message: + demisto.debug(f"BU: the custom URL category {category=} does not exist yet.") + return {} + raise BlockUrlError(error_message) + self.responses.append(res) + context = get_context_entry(res, "Panorama.CustomURLCategory") + self.capture_device_group(context) + return context + + def ensure_url_category(self, existing: dict, urls: list[str]) -> list[str]: + """Create the custom URL category, or append only the URLs it does not hold yet. + + Args: + existing (dict): The existing category context, or an empty dict. + urls (list[str]): The normalized URLs to block. + Returns: + The URLs that were already present in the category. + Raises: + BlockUrlError: When the category exists with a type other than "URL List", or on failure. + """ + category = self.args["url_category"] + if not existing: + return self.create_url_category(urls) + category_type = _text(existing.get("Type")) + if category_type != URL_LIST_TYPE: + # PAN-OS silently converts a foreign-type category to a URL List and merges its members + # into the site list, destroying the object. There is no server-side protection. + raise BlockUrlError( + f"Won't modify a user-managed category '{category}' of type '{category_type}'. " + f"Only categories of type '{URL_LIST_TYPE}' are managed by this script. " + f"Use the url_category argument to point to a different category." + ) + existing_sites = _text_list(existing.get("Sites")) + missing_urls = [url for url in urls if url not in existing_sites] + already_present = [url for url in urls if url in existing_sites] + if not missing_urls: + demisto.debug(f"BU: every URL is already present in the category {category=}, skipping the edit.") + return already_present + res = run_execute_command("pan-os-edit-custom-url-category", {"name": category, "action": "add", "sites": missing_urls}) + self.responses.append(res) + raise_on_command_error(res, f"custom URL category '{category}'") + return already_present + + def create_url_category(self, urls: list[str]) -> list[str]: + """Create the custom URL category with the requested URLs. + + Args: + urls (list[str]): The normalized URLs to block. + Returns: + An empty list, since a new category holds none of the URLs yet. + Raises: + BlockUrlError: When the creation failed. + """ + category = self.args["url_category"] + res = run_execute_command( + "pan-os-create-custom-url-category", + {"name": category, "type": URL_LIST_TYPE, "sites": urls, "description": OBJECT_DESCRIPTION}, + ) + self.responses.append(res) + raise_on_command_error(res, f"custom URL category '{category}'") + self.capture_device_group(get_context_entry(res, "Panorama.CustomURLCategory")) + return [] + + """ URL FILTERING PROFILE """ + + def ensure_url_filtering_profile(self) -> None: + """Create the URL filtering profile, or attach the category to the existing one. + + Raises: + BlockUrlError: When the profile could not be read, created or edited. + """ + profile = self.args["url_filtering_profile"] + res = run_execute_command("pan-os-get-url-filter", {"name": profile}) + error_message = get_error_message(res) + if error_message and OBJECT_NOT_FOUND not in error_message: + raise BlockUrlError(error_message) + if error_message: + demisto.debug(f"BU: the URL filtering profile {profile=} does not exist yet.") + self.create_url_filtering_profile() + return + self.responses.append(res) + context = get_context_entry(res, "Panorama.URLFilter") + self.capture_device_group(context) + self.attach_category_to_profile(context) + + def create_url_filtering_profile(self) -> None: + """Create the URL filtering profile with the category blocked for Site Access. + + Raises: + BlockUrlError: When the creation failed. + """ + profile = self.args["url_filtering_profile"] + res = run_execute_command( + "pan-os-create-url-filter", + { + "name": profile, + "url_category": self.args["url_category"], + "action": "block", + "description": OBJECT_DESCRIPTION, + }, + ) + self.responses.append(res) + raise_on_command_error(res, f"URL filtering profile '{profile}'") + self.capture_device_group(get_context_entry(res, "Panorama.URLFilter")) + + def attach_category_to_profile(self, context: dict) -> None: + """Add the category to the profile block list when it is not attached yet. + + Args: + context (dict): The existing profile context. + Raises: + BlockUrlError: When the edit failed. + """ + profile = self.args["url_filtering_profile"] + category = self.args["url_category"] + if self.is_category_attached(context, category): + demisto.debug(f"BU: the category {category=} is already blocked by the profile {profile=}.") + return + res = run_execute_command( + "pan-os-edit-url-filter", + { + "name": profile, + "element_to_change": "block_categories", + "element_value": category, + "add_remove_element": "add", + }, + ) + self.responses.append(res) + raise_on_command_error(res, f"URL filtering profile '{profile}'") + + @staticmethod + def is_category_attached(context: dict, category: str) -> bool: + """Whether the category is already in the block list of the profile. + + Args: + context (dict): The profile context. + category (str): The custom URL category name. + Returns: + Whether the category is attached. + """ + if category in _text_list(context.get("block_categories")): + return True + categories = context.get("Category") or [] + if isinstance(categories, dict): + categories = [categories] + return any(_text(item.get("Name")) == category for item in categories) + + """ TAG AND SECURITY RULE """ + + def ensure_tag(self) -> None: + """Create the tag object before any rule references it. + + pan-os-create-rule fails with "tag '' is not a valid reference" when the tag object + does not exist. Re-creating an existing tag succeeds, so this call is idempotent. A failure + here is tolerated rather than aborting the whole flow. + """ + tag = self.args.get("tag", "") + if not tag: + return + res = run_execute_command("pan-os-create-tag", {"name": tag}) + if error_message := get_error_message(res): + demisto.debug(f"BU: could not create the tag {tag=}, continuing without it. {error_message=}") + return + self.responses.append(res) + + def ensure_security_rule(self) -> None: + """Create the security rule when it is missing, then attach the profile and move it to the top. + + Raises: + BlockUrlError: When the rulebase could not be read or the rule could not be created. + """ + rule_name = self.args["rule_name"] + res_list_rules = run_execute_command("pan-os-list-rules", self.rule_scope_args()) + self.responses.append(res_list_rules) + raise_on_command_error(res_list_rules, f"security rule '{rule_name}'") + context = get_relevant_context(res_list_rules[0].get("EntryContext", {}), "Panorama.SecurityRule") + if check_value_exist_in_context(rule_name, context, "Name"): + demisto.debug(f"BU: reusing the existing security rule {rule_name=}.") + self.add_tag_to_rule() + else: + self.create_security_rule() + self.apply_profile_to_rule() + self.move_rule_to_top() + + def add_tag_to_rule(self) -> None: + """Add the configured tag to an existing rule unconditionally. + + Uses behaviour=add so the rule's existing tags are preserved rather than replaced. The edit + is issued every run because the rule may have been changed outside this script (for example + through the PAN-OS UI), so the fetched tag list can be stale. Adding an already-present tag + is a no-op on the device, so this stays idempotent. + Raises: + BlockUrlError: When the edit failed. + """ + tag = self.args.get("tag", "") + if not tag: + return + rule_name = self.args["rule_name"] + edit_args: dict[str, Any] = { + "rulename": rule_name, + "element_to_change": "tag", + "element_value": tag, + "behaviour": "add", + } + edit_args |= self.rule_scope_args() + res = run_execute_command("pan-os-edit-rule", edit_args) + self.responses.append(res) + raise_on_command_error(res, f"security rule '{rule_name}'") + + def create_security_rule(self) -> None: + """Create the security rule that carries the URL filtering profile. + + Raises: + BlockUrlError: When the creation failed. + """ + rule_name = self.args["rule_name"] + create_rule_args: dict[str, Any] = { + "rulename": rule_name, + # The action is ALLOW by design. The URL filtering profile attached to this rule performs + # the blocking, and PAN-OS only inspects traffic with a profile when the rule allows it. + # A "deny" rule would drop the traffic before the profile runs, so do not change this. + "action": "allow", + "source": "any", + "destination": "any", + "application": "any", + "description": OBJECT_DESCRIPTION, + } + if tag := self.args.get("tag", ""): + create_rule_args["tags"] = tag + if log_forwarding_name := self.args.get("log_forwarding_name", ""): + create_rule_args["log_forwarding"] = log_forwarding_name + create_rule_args |= self.rule_scope_args() + res = run_execute_command("pan-os-create-rule", create_rule_args) + self.responses.append(res) + raise_on_command_error(res, f"security rule '{rule_name}'") + + def apply_profile_to_rule(self) -> None: + """Attach the URL filtering profile to the security rule. + + Raises: + BlockUrlError: When the command failed. + """ + rule_name = self.args["rule_name"] + apply_args: dict[str, Any] = { + "profile_type": "url-filtering", + "rule_name": rule_name, + "profile_name": self.args["url_filtering_profile"], + } + apply_args |= self.rule_scope_args() + res = run_execute_command("pan-os-apply-security-profile", apply_args) + self.responses.append(res) + raise_on_command_error(res, f"security rule '{rule_name}'") + + def move_rule_to_top(self) -> None: + """Move the security rule to the top of the rulebase. + + pan-os-create-rule appends the rule at the bottom, where an earlier allow rule would defeat + the block, so the move is mandatory. It is idempotent ("Rule already at the top"), therefore + it runs on every execution. + Raises: + BlockUrlError: When the command failed. + """ + rule_name = self.args["rule_name"] + move_args: dict[str, Any] = {"rulename": rule_name, "where": "top"} + move_args |= self.rule_scope_args() + res = run_execute_command("pan-os-move-rule", move_args) + self.responses.append(res) + raise_on_command_error(res, f"security rule '{rule_name}'") + + """ FLOW """ + + def reduce_pan_os_responses(self) -> list[list[dict]]: + """Reduce the stored responses to the parts the later polling rounds need. + + Returns: + A list containing the relevant parts of the command responses. + """ + demisto.debug("BU: updating the responses in reduce_pan_os_responses.") + reduced_responses = [] + for res in self.responses: + current_new_res = [] + for entry in res: + current_new_res.append( + { + "HumanReadable": entry.get("HumanReadable"), + "Contents": entry.get("Contents"), + "Type": entry.get("Type"), + "Metadata": entry.get("Metadata"), + } + ) + reduced_responses.append(current_new_res) + demisto.debug(f"BU: {len(reduced_responses)=}, {len(self.responses)=}") + return reduced_responses + + def save_state_to_context(self) -> None: + """Persist the state the next polling round needs.""" + demisto.setContext("panorama_responses", str(self.reduce_pan_os_responses())) + demisto.setContext("pan_os_device_group", self.device_group) + + def restore_state_from_context(self, incident_context: dict) -> None: + """Restore the state saved by the previous polling round. + + Args: + incident_context (dict): The incident context. + """ + self.responses = ast.literal_eval(incident_context.get("panorama_responses", "") or "[]") + self.device_group = demisto.get(incident_context, "pan_os_device_group") or "" + + def start_pan_os_flow(self) -> tuple[list, bool]: + """Run the configuration part of the PAN-OS flow. + + Returns: + A tuple of the command results, and whether the changes should be committed. The results + are populated only when the flow failed and there is nothing left to commit. + """ + try: + self.detect_topology() + existing_category = self.get_custom_url_category() + self.already_present_urls = self.ensure_url_category(existing_category, self.submitted_urls()) + self.ensure_url_filtering_profile() + self.ensure_tag() + self.ensure_security_rule() + except BlockUrlError as error: + demisto.debug(f"BU: the PAN-OS flow failed. {error=}") + self.failure_message = str(error) + return self.pan_os_finish(), False + return [], bool(self.args.get("auto_commit", True)) + + def submitted_urls(self) -> list[str]: + """The normalized URLs that are submitted to PAN-OS. + + Returns: + The list of submitted URLs. + """ + return [entry["SubmittedURL"] for entry in self.args["url_entries"]] + + def adopt_job_failure(self) -> bool: + """Adopt a commit or push job failure reported by the polling functions. + + The polling functions run outside the class and communicate a terminal PAN-OS job failure + through the JOB_FAILURE_MESSAGE global, since a failed job comes back as a type-1 success + entry that the is_error sweep in prepare_context_and_hr_multiple_executions cannot detect. + Returns: + Whether a job failure was adopted. + """ + global JOB_FAILURE_MESSAGE + if not JOB_FAILURE_MESSAGE: + return False + demisto.debug(f"BU: adopting the job failure {JOB_FAILURE_MESSAGE=}") + self.failure_message = JOB_FAILURE_MESSAGE + JOB_FAILURE_MESSAGE = "" + return True + + def pan_os_finish(self) -> list[CommandResults]: + """Clear the polling state from the context and build the final results. + + Returns: + The list of Command Results. + """ + demisto.setContext("push_job_id", "") + demisto.setContext("commit_job_id", "") + demisto.setContext("panorama_responses", "") + demisto.setContext("pan_os_device_group", "") + details = { + "brand": PAN_OS_BRAND, + "rule_name": self.args.get("rule_name", ""), + "url_category": self.args.get("url_category", ""), + "job_id": self.args.get("commit_job_id") or "", + "already_present": self.already_present_urls, + "failure_message": self.failure_message, + } + return prepare_context_and_hr_multiple_executions( + self.responses, bool(self.args.get("verbose", False)), self.args["url_entries"], details + ) + + def manage_pan_os_flow(self) -> CommandResults | list[CommandResults] | PollResult: + """Manage the different states of the PAN-OS flow. + + 1. The flow start: create or reuse the category, the profile, the tag and the rule. + 2. If auto_commit is true and there were changes, execute pan-os-commit. + 3. There is a commit job ID, check the status of the commit. + 4. The commit finished, on Panorama push the changes to the device group. + 5. There is a push job ID, check the status of the push. + 6. The push finished, build the final results. + Returns: + A PollResult while polling, otherwise a Command Result or a list of Command Results. + """ + incident_context = demisto.context() + auto_commit = self.args["auto_commit"] + commit_job_id = self.args.get("commit_job_id") + context_push_job_id = demisto.get(incident_context, "push_job_id") + context_commit_job_id = demisto.get(incident_context, "commit_job_id") + is_polling_reentry = bool(commit_job_id) + if not is_polling_reentry and (context_commit_job_id or context_push_job_id): + demisto.debug( + f"BU: stale polling context on a fresh invocation " + f"(commit={context_commit_job_id!r}, push={context_push_job_id!r}); clearing." + ) + demisto.setContext("commit_job_id", "") + demisto.setContext("push_job_id", "") + demisto.setContext("panorama_responses", "") + demisto.setContext("pan_os_device_group", "") + context_push_job_id = None + push_job_id = context_push_job_id if is_polling_reentry else None + # state 5 + if push_job_id: + demisto.debug(f"BU: has a {push_job_id=}") + self.restore_state_from_context(incident_context) + self.args["push_job_id"] = push_job_id + res_push_status = pan_os_push_status(self.args, self.responses) + # state 6 + if not POLLING: + demisto.debug("BU: finished polling, finishing the flow.") + self.adopt_job_failure() + return self.pan_os_finish() + self.save_state_to_context() + return res_push_status + # state 3 + if commit_job_id: + demisto.debug(f"BU: has a {commit_job_id=}") + self.args["commit_job_id"] = commit_job_id + self.restore_state_from_context(incident_context) + poll_commit_status = pan_os_commit_status(self.args, self.responses) + if POLLING: + self.save_state_to_context() + return poll_commit_status + # A commit that PAN-OS finished with a failure must not be pushed: the pushed candidate + # config never passed validation, and reporting success here would tell the user the URLs + # are blocked when they are not. + if self.adopt_job_failure(): + demisto.debug("BU: the commit job failed, skipping the push.") + return self.pan_os_finish() + # state 4 + self.detect_topology() + if not self.is_panorama: + demisto.debug("BU: not a Panorama instance, not pushing to the device group.") + return self.pan_os_finish() + self.args["device_group"] = self.device_group + poll_push_to_device = pan_os_push_to_device(self.args, self.responses) + if not POLLING: + demisto.debug("BU: nothing to push, finishing the flow.") + self.adopt_job_failure() + return self.pan_os_finish() + self.save_state_to_context() + return poll_push_to_device + + # state 1 + results, should_commit = self.start_pan_os_flow() + if not should_commit: + if results: + return results + not_committed = CommandResults( + readable_output=f"Not committing the changes in PAN-OS, since {auto_commit=}. " + f"Please do so manually for the changes to take effect." + ) + final_results = self.pan_os_finish() + final_results.append(not_committed) + return final_results + # state 2 + self.args["device_group"] = self.device_group + self.args["is_panorama"] = self.is_panorama + poll_result = pan_os_commit(self.args, self.responses) + if not POLLING: + return self.pan_os_finish() + self.save_state_to_context() + return poll_result + + +""" STANDALONE FUNCTION """ + + +def job_details(job: dict) -> str: + """Extract the human readable reason a PAN-OS job reports for its outcome. + + PAN-OS returns the reason either as a plain string or as a {"line": [...]} structure, and the + lines may themselves be nested lists. + Args: + job (dict): The job element of a pan-os-commit-status response. + Returns: + The joined details, or a placeholder when PAN-OS reported none. + """ + details = job.get("details") + if isinstance(details, dict): + details = details.get("line") + if isinstance(details, list): + flattened = [str(item) for line in details for item in (line if isinstance(line, list) else [line])] + return "; ".join(flattened) or "none reported" + return str(details) if details else "none reported" + + +def build_commit_args(args: dict) -> dict[str, Any]: + """Build the arguments of pan-os-commit, scoped as narrowly as PAN-OS allows. + + PAN-OS has no per-object commit, so the device group is the finest available granularity. + Args: + args (dict): The flow arguments. + Returns: + The arguments for pan-os-commit. + """ + url_count = len(args.get("url_entries", [])) + commit_args: dict[str, Any] = { + "polling": True, + "description": f"Block URL - {url_count} URL(s) - {args.get('incident_id', '')}", + "exclude_device_network_configuration": True, + } + device_group = args.get("device_group", "") + if args.get("is_panorama") and device_group: + commit_args["device-group"] = device_group + # Only safe when the objects live in the device group and not in /config/shared. + commit_args["exclude_shared_objects"] = True + return commit_args + + +def build_push_args(args: dict) -> dict[str, Any]: + """Build the arguments of pan-os-push-to-device-group. + + Args: + args (dict): The flow arguments. + Returns: + The arguments for pan-os-push-to-device-group. + """ + push_args: dict[str, Any] = { + "polling": True, + "description": f"Block URL - {args.get('incident_id', '')}", + } + if device_group := args.get("device_group", ""): + push_args["device-group"] = device_group + return push_args + + +def create_final_human_readable(failure_message: str, context: list[dict]) -> str: + """Create the human readable summary of the script. + + Args: + failure_message (str): A failure message if relevant. + context (list[dict]): The final context records. + Returns: + The human readable summary. + """ + headers = ["URL", "SubmittedURL", "Result", "Brand", "RuleName", "URLCategory", "JobID", "Message"] + name = "Failed to block the URL/s" if failure_message else "URL/s blocking summary" + demisto.debug(f"BU: creating the final human readable for {failure_message=}") + return tableToMarkdown(name=name, t=context, headers=headers, removeNull=True) + + +def create_final_context(used_integration: str, url_entries: list[dict], details: dict) -> list[dict]: + """Create the context records of the script, one per URL. + + Args: + used_integration (str): The integration that was used. + url_entries (list[dict]): The accepted URL entries, holding URL and SubmittedURL. + details (dict): The flow details, holding the rule name, the category, the job ID, the URLs + that were already present and the failure message. + Returns: + The list of context records. + """ + failure_message = details.get("failure_message", "") + already_present = details.get("already_present", []) + context = [] + for entry in url_entries: + submitted_url = entry.get("SubmittedURL", "") + if failure_message: + result, message = "Failed", failure_message + elif submitted_url in already_present: + result, message = "Skipped", "URL already present in the category." + else: + result, message = "Success", "URL was blocked successfully." + context.append( + { + "URL": entry.get("URL") or None, + "SubmittedURL": submitted_url or None, + "Brand": used_integration or None, + "Result": result or None, + "Message": message or None, + "RuleName": details.get("rule_name") or None, + "URLCategory": details.get("url_category") or None, + "JobID": details.get("job_id") or None, + } + ) + return context + + +def create_rejected_results(rejected: list[dict]) -> CommandResults: + """Create the results of the URLs that were rejected before any brand was contacted. + + Args: + rejected (list[dict]): The rejected URL entries. + Returns: + A Command Results holding the rejected records. + """ + context = [ + { + "URL": entry.get("URL") or None, + "SubmittedURL": None, + "Brand": None, + "Result": "Failed", + "Message": entry.get("Message") or None, + "RuleName": None, + "URLCategory": None, + "JobID": None, + } + for entry in rejected + ] + headers = ["URL", "SubmittedURL", "Result", "Message"] + return CommandResults( + readable_output=tableToMarkdown(name="URL/s that were not submitted", t=context, headers=headers, removeNull=True), + outputs_prefix="BlockURLResults", + outputs=context, + raw_response=context, + ) + + +def prepare_context_and_hr_multiple_executions( + responses: list[list[dict]], verbose: bool, url_entries: list[dict], details: dict +) -> list[CommandResults]: + """Create the context and the human readable of a flow made of multiple command executions. + + Args: + responses (list[list[dict]]): The responses returned from the command executions. + verbose (bool): Whether to return a human readable entry per command or only the summary. + url_entries (list[dict]): The accepted URL entries. + details (dict): The flow details. + Returns: + A list containing the relevant Command Results. + """ + demisto.debug(f"BU: in prepare_context_and_hr_multiple_executions, {len(responses)=}") + results = [] + failed_messages = [] + used_integration = details.get("brand", "") + if responses and responses[0]: + used_integration = responses[0][0].get("Metadata", {}).get("brand") or used_integration + + for res in responses: + for entry in res: + command_hr = entry.get("HumanReadable") + message = f"{used_integration}: {entry.get('Contents', '')}" + if is_error(entry): + demisto.debug(f"BU: a failure was found {message=}") + failed_messages.append(message) + elif command_hr and command_hr != str(None): + results.append(CommandResults(readable_output=f"{used_integration}:\n{command_hr}")) + elif message and isinstance(entry.get("Contents"), str): + results.append(CommandResults(readable_output=message)) + if explicit_failure := details.get("failure_message", ""): + failed_messages.append(explicit_failure) + details = details | {"failure_message": ", ".join(failed_messages)} + + final_context = create_final_context(used_integration, url_entries, details) + final_cr = CommandResults( + readable_output=create_final_human_readable(details["failure_message"], final_context), + outputs_prefix="BlockURLResults", + outputs=final_context, + raw_response=final_context, + ) + if verbose: + results.append(final_cr) + else: + results = [final_cr] + return results + + +""" COMMAND FUNCTION """ + + +@polling_function( + name="block-url", + interval=POLLING_INTERVAL, + timeout=POLLING_TIMEOUT, +) +def pan_os_commit(args: dict, responses: list) -> PollResult: + """Execute pan-os-commit. + + Args: + args (dict): The arguments of the flow. + responses (list): The responses of the command executions so far. + Returns: + A PollResult object. + """ + res_commit = run_execute_command("pan-os-commit", build_commit_args(args)) + responses.append(res_commit) + polling_args = res_commit[0].get("Metadata", {}).get("pollingArgs", {}) + job_id = polling_args.get("commit_job_id") + if job_id: + context_output = {"JobID": job_id, "Status": "Pending"} + continue_to_poll = True + commit_output = CommandResults( + outputs=context_output, readable_output=tableToMarkdown("Commit Status:", context_output, removeNull=True) + ) + demisto.debug(f"BU: initiated a commit execution {job_id=}") + demisto.setContext("commit_job_id", job_id) + else: # nothing to commit in PAN-OS, no reason to poll. + commit_output = res_commit[0].get("Contents") or "There are no changes to commit." # type: ignore[assignment] + demisto.debug(f"BU: no job_id, {commit_output}") + continue_to_poll = False + global POLLING + POLLING = continue_to_poll + + args_for_next_run = args | { + "commit_job_id": job_id, + "interval_in_seconds": arg_to_number(args.get("interval_in_seconds", POLLING_INTERVAL)), + "timeout": arg_to_number(args.get("timeout", POLLING_TIMEOUT)), + "polling": True, + } + return PollResult( + response=commit_output, + continue_to_poll=continue_to_poll, + args_for_next_run=args_for_next_run, + partial_result=CommandResults(readable_output=f"Waiting for commit job ID {job_id} to finish..."), + ) + + +@polling_function( + name="block-url", + interval=POLLING_INTERVAL, + timeout=POLLING_TIMEOUT, +) +def pan_os_commit_status(args: dict, responses: list) -> PollResult: + """Check the status of the commit in PAN-OS. + + A commit that finishes with warnings is a success: the lab returns warnings on every commit. A + commit that finishes with any result other than "OK" is a failure, and PAN-OS reports it as a + successful command entry carrying a failed job, so the failure is published through the + JOB_FAILURE_MESSAGE global for manage_pan_os_flow to skip the push and report it. + Args: + args (dict): The arguments of the flow. + responses (list): The responses of the command executions so far. + Returns: + A PollResult object. + """ + commit_job_id = args["commit_job_id"] + res_commit_status = run_execute_command("pan-os-commit-status", {"job_id": commit_job_id}) + responses.append(res_commit_status) + global POLLING, JOB_FAILURE_MESSAGE + raw_contents = res_commit_status[0].get("Contents") if res_commit_status else None + if is_error(res_commit_status) or not isinstance(raw_contents, dict): + POLLING = False + error_reason = get_error_message(res_commit_status) or str(raw_contents) + JOB_FAILURE_MESSAGE = ( + f"The PAN-OS commit job {commit_job_id} status could not be read, so the changes were not " + f"pushed and the URLs are not blocked. PAN-OS reason: {error_reason}" + ) + demisto.debug(f"BU: the commit status returned an error entry. {JOB_FAILURE_MESSAGE=}") + commit_output = {"JobID": commit_job_id, "Status": "Failure"} + return PollResult( + response=CommandResults( + outputs=commit_output, + outputs_key_field="JobID", + readable_output=tableToMarkdown("Commit Status:", commit_output, removeNull=True), + ), + args_for_next_run=args, + continue_to_poll=False, + ) + job = raw_contents.get("response", {}).get("result", {}).get("job", {}) + job_result = job.get("result") + continue_to_poll = job.get("status") != COMMIT_JOB_FINISHED_STATUS + job_failed = not continue_to_poll and job_result != COMMIT_JOB_SUCCESS_RESULT + commit_output = {"JobID": commit_job_id, "Status": "Failure" if job_failed else "Success"} + POLLING = continue_to_poll + if job_failed: + JOB_FAILURE_MESSAGE = ( + f"The PAN-OS commit job {commit_job_id} failed with the result '{job_result}', so the changes were " + f"not pushed and the URLs are not blocked. PAN-OS details: {job_details(job)}" + ) + demisto.debug(f"BU: the commit job failed. {JOB_FAILURE_MESSAGE=}") + demisto.debug(f"BU: after pan-os-commit-status {continue_to_poll=} {commit_job_id=} {job_result=}") + return PollResult( + response=CommandResults( + outputs=commit_output, + outputs_key_field="JobID", + readable_output=tableToMarkdown("Commit Status:", commit_output, removeNull=True), + ), + args_for_next_run=args, + continue_to_poll=continue_to_poll, + ) + + +@polling_function( + name="block-url", + interval=POLLING_INTERVAL, + timeout=POLLING_TIMEOUT, +) +def pan_os_push_to_device(args: dict, responses: list) -> PollResult: + """Execute pan-os-push-to-device-group. + + Args: + args (dict): The arguments of the flow. + responses (list): The responses of the command executions so far. + Returns: + A PollResult object. + """ + res_push_to_device = run_execute_command("pan-os-push-to-device-group", build_push_args(args)) + responses.append(res_push_to_device) + polling_args = res_push_to_device[0].get("Metadata", {}).get("pollingArgs", {}) + job_id = polling_args.get("push_job_id") + device_group = polling_args.get("device-group") + demisto.debug(f"BU: the polling args are {job_id=} {device_group=}") + if job_id: + context_output = {"DeviceGroup": device_group, "JobID": job_id, "Status": "Pending"} + continue_to_poll = True + push_cr = CommandResults( + outputs_key_field="JobID", + outputs=context_output, + readable_output=tableToMarkdown("Push to Device Group:", context_output, removeNull=True), + ) + demisto.setContext("push_job_id", job_id) + else: + push_cr = CommandResults(readable_output=res_push_to_device[0].get("Contents") or "There are no changes to push.") + continue_to_poll = False + global POLLING + POLLING = continue_to_poll + return PollResult( + response=push_cr, + continue_to_poll=continue_to_poll, + partial_result=CommandResults(readable_output=f"Waiting for Job-ID {job_id} to finish pushing the changes..."), + ) + + +@polling_function( + name="block-url", + interval=POLLING_INTERVAL, + timeout=POLLING_TIMEOUT, +) +def pan_os_push_status(args: dict, responses: list) -> PollResult: + """Check the status of the push in PAN-OS. + + The status is read from the context, since the human readable output of pan-os-push-status is + nearly empty while the job is still settling. + Args: + args (dict): The arguments of the flow. + responses (list): The responses of the command executions so far. + Returns: + A PollResult object. + """ + push_job_id = args["push_job_id"] + res_push_status = run_execute_command("pan-os-push-status", {"job_id": push_job_id}) + responses.append(res_push_status) + global POLLING, JOB_FAILURE_MESSAGE + if is_error(res_push_status): + POLLING = False + error_reason = get_error_message(res_push_status) + JOB_FAILURE_MESSAGE = ( + f"The PAN-OS push job {push_job_id} status could not be read, so the committed changes were " + f"not confirmed as applied and the URLs may not be blocked. PAN-OS reason: {error_reason}" + ) + demisto.debug(f"BU: the push status returned an error entry. {JOB_FAILURE_MESSAGE=}") + push_output = {"JobID": push_job_id, "Status": "Failure"} + return PollResult( + response=CommandResults( + outputs=push_output, + outputs_key_field="JobID", + readable_output=tableToMarkdown("Push to Device Group:", push_output, ["JobID", "Status"], removeNull=True), + ), + continue_to_poll=False, + ) + push_context = get_context_entry(res_push_status, "Panorama.Push") + push_status = _text(push_context.get("Status")) + push_failed = push_status in PUSH_FAILURE_STATUSES + # An empty status means the job is still settling, not that it succeeded. Treating it as done + # would report a full success for a push that never finished. The polling decorator timeout is + # the safety net for a status that never resolves. + continue_to_poll = not push_failed and push_status != PUSH_COMPLETED_STATUS + demisto.debug(f"BU: after pan-os-push-status {push_status=} {continue_to_poll=} {push_failed=}") + context_output = {"Status": push_status, "JobID": push_job_id} + push_cr = CommandResults( + outputs_key_field="JobID", + outputs=context_output, + readable_output=tableToMarkdown("Push to Device Group:", context_output, ["JobID", "Status"], removeNull=True), + ) + POLLING = continue_to_poll + if push_failed: + details = _text(push_context.get("Details")) or _text(push_context.get("Errors")) + JOB_FAILURE_MESSAGE = ( + f"The PAN-OS push job {push_job_id} failed with the status '{push_status}', so the committed changes " + f"were not applied to the managed firewalls and the URLs are not blocked. PAN-OS details: " + f"{details or 'none reported'}" + ) + demisto.debug(f"BU: the push job failed. {JOB_FAILURE_MESSAGE=}") + return PollResult( + response=push_cr, + continue_to_poll=continue_to_poll, + partial_result=CommandResults(readable_output=f"Waiting for Job-ID {push_job_id} to finish pushing the changes..."), + ) + + +""" MAIN FUNCTION """ + + +def main(): # pragma: no cover + try: + args = demisto.args() + demisto.debug(f"BU: the script block-url was called with the arguments {args=}") + url_list = argToList(args.get("url_list", [])) + brands_to_run = argToList(args.get("brands", ",".join(SUPPORTED_BRANDS))) + verbose = argToBoolean(args.get("verbose", False)) + modules = demisto.getModules() + enabled_brands = {module.get("brand") for module in modules.values() if module.get("state") == "active"} + demisto.debug(f"BU: the enabled modules are: {enabled_brands=}, {brands_to_run=}") + + executed_brands, updated_brands_to_run = update_brands_to_run(brands_to_run) + accepted_urls, rejected_urls = partition_urls(url_list) + + results: list = [] + if rejected_urls and not is_polling_resume(): + results.append(create_rejected_results(rejected_urls)) + + for brand in updated_brands_to_run: + demisto.debug(f"BU: the current brand is {brand}") + if brand not in enabled_brands: + results.append(CommandResults(readable_output=f"The brand {brand} isn't enabled.")) + executed_brands.append(brand) + continue + if brand != PAN_OS_BRAND: + return_error( + f"The brand {brand} isn't a part of the supported integrations for 'block-url'. " + f"The supported integrations are: {', '.join(SUPPORTED_BRANDS)}." + ) + if not accepted_urls: + results.append(CommandResults(readable_output="There are no valid URLs to block.")) + executed_brands.append(brand) + continue + brand_args = { + "url_list": [entry["SubmittedURL"] for entry in accepted_urls], + "url_entries": accepted_urls, + "rule_name": args.get("rule_name", "Cortex - Block URLs"), + "url_category": args.get("url_category", "Blocked URLs - Cortex"), + "url_filtering_profile": args.get("url_filtering_profile", "Cortex - Block URL profile"), + "log_forwarding_name": args.get("log_forwarding_name", ""), + "tag": args.get("tag", "cortex-blocked-urls"), + "auto_commit": argToBoolean(args.get("auto_commit", True)), + "verbose": verbose, + "brands": brands_to_run, + "commit_job_id": args.get("commit_job_id"), + "incident_id": demisto.incident().get("id", ""), + "polling": True, + } + results.append(PanOs(brand_args).manage_pan_os_flow()) + if not POLLING: + demisto.debug("BU: not in a polling mode, adding Panorama to the executed_brands.") + executed_brands.append(brand) + + if POLLING: + demisto.debug(f"BU: updating the executed_brands {executed_brands=}") + demisto.setContext("executed_brands", str(executed_brands)) + elif PAN_OS_BRAND in brands_to_run: + demisto.debug("BU: not in a polling mode, initializing the executed_brands.") + demisto.setContext("executed_brands", "") + return_results(results) + + except Exception as ex: + return_error(f"Failed to execute block-url. Error: {str(ex)}") + + +""" ENTRY POINT """ + + +if __name__ in ("__main__", "__builtin__", "builtins"): # pragma: no cover + main() diff --git a/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl.yml b/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl.yml new file mode 100644 index 000000000000..c306ab622775 --- /dev/null +++ b/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl.yml @@ -0,0 +1,91 @@ +args: +- description: A comma-separated list of URLs to block. The scheme is stripped before the URL is submitted to the firewall. + isArray: true + name: url_list + required: true +- description: |- + Which integration brands to run the command for. If not provided, the command will run for all available integrations. + For multi-select provide a comma-separated list. + isArray: true + name: brands + auto: PREDEFINED + predefined: + - 'Panorama' +- defaultValue: 'Cortex - Block URLs' + description: The name of the security rule which will be created in the relevant products. + name: rule_name +- defaultValue: 'Blocked URLs - Cortex' + description: The name of the PAN-OS custom URL category which holds the blocked URLs. + name: url_category +- defaultValue: 'Cortex - Block URL profile' + description: The name of the PAN-OS URL Filtering security profile which blocks the custom URL category. + name: url_filtering_profile +- description: The Panorama log forwarding object name. Indicates what type of Log Forwarding setting will be specified in the PAN-OS custom rules. + name: log_forwarding_name +- defaultValue: 'cortex-blocked-urls' + description: The designated tag name for the objects the script creates. + name: tag +- description: Whether to commit the new rule. + name: auto_commit + defaultValue: 'true' + auto: PREDEFINED + predefined: + - 'true' + - 'false' +- description: Whether to retrieve a human-readable entry for every command or only the final result. True retrieves a human-readable entry for every command. False retrieves a human-readable entry only for the final result. + name: verbose + defaultValue: 'false' + auto: PREDEFINED + predefined: + - 'true' + - 'false' +- description: The commit job ID to use in polling commands (automatically filled by polling). + name: commit_job_id + hidden: true +- description: The push job ID to use in polling commands (automatically filled by polling). + name: push_job_id + hidden: true +comment: Blocks a list of URLs in supported integrations. +commonfields: + id: block-url + version: -1 +name: block-url +outputs: +- contextPath: BlockURLResults.URL + description: The URL that was requested to be blocked, as it was supplied. + type: String +- contextPath: BlockURLResults.SubmittedURL + description: The normalized URL that was submitted to the brand. Empty when the URL was rejected before submission. + type: String +- contextPath: BlockURLResults.Brand + description: The name of the brand that was executed. + type: String +- contextPath: BlockURLResults.Result + description: The result of the action, Success, Failed or Skipped. + type: String +- contextPath: BlockURLResults.Message + description: A message concerning the result of the action. + type: String +- contextPath: BlockURLResults.RuleName + description: The name of the security rule that blocks the URL. + type: String +- contextPath: BlockURLResults.URLCategory + description: The name of the custom URL category that holds the URL. + type: String +- contextPath: BlockURLResults.JobID + description: The ID of the PAN-OS commit job. + type: String +script: '-' +system: false +timeout: 20m0s +type: python +subtype: python3 +dockerimage: demisto/python3:3.12.14.13495286 +fromversion: 6.10.0 +marketplaces: +- xsoar +- marketplacev2 +- platform +polling: true +tests: +- No tests (auto formatted) diff --git a/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl_test.py b/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl_test.py new file mode 100644 index 000000000000..9654d10568ba --- /dev/null +++ b/Packs/AggregatedScripts/Scripts/BlockUrl/BlockUrl_test.py @@ -0,0 +1,1375 @@ +import json +import os + +import pytest + +import BlockUrl +import demistomock as demisto +from BlockUrl import BlockUrlError, PanOs + +TEST_DATA_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "test_data") + + +def util_load_json(path): + with open(os.path.join(TEST_DATA_DIR, path), encoding="utf-8") as f: + return json.loads(f.read()) + + +RESPONSES = util_load_json("pan_os_responses.json") + + +def build_pan_os(**overrides) -> PanOs: + """Build a PanOs instance with the script defaults, overridden as the test needs.""" + args = { + "url_entries": [{"URL": "http://new.example.com", "SubmittedURL": "new.example.com"}], + "rule_name": "Cortex - Block URLs", + "url_category": "Blocked URLs - Cortex", + "url_filtering_profile": "Cortex - Block URL profile", + "log_forwarding_name": "", + "tag": "cortex-blocked-urls", + "auto_commit": True, + "verbose": False, + "incident_id": "1", + } + args.update(overrides) + pan_os = PanOs(args) + pan_os.is_panorama = overrides.pop("is_panorama", True) + pan_os.device_group = overrides.pop("device_group", "Test-Device-Group") + return pan_os + + +def executed_commands(mock) -> list[tuple[str, dict]]: + """The (command name, args) pairs that were executed, in order.""" + return [(call.args[0], call.args[1]) for call in mock.call_args_list] + + +def successful_flow_responses() -> list: + """The responses of a full start_pan_os_flow run on a Panorama holding none of the objects yet.""" + return [ + RESPONSES["system_info_panorama"], + RESPONSES["object_not_found"], + RESPONSES["category_created"], + RESPONSES["object_not_found"], + RESPONSES["url_filter_created"], + RESPONSES["tag_created"], + RESPONSES["list_rules_empty"], + RESPONSES["rule_created"], + RESPONSES["profile_applied"], + RESPONSES["rule_moved"], + ] + + +""" URL NORMALIZATION """ + + +@pytest.mark.parametrize( + "raw_url, expected", + [ + ("http://example.com", "example.com"), + ("https://example.com", "example.com"), + ("HTTPS://Example.COM", "example.com"), + ("example.com/", "example.com"), + ("https://example.com/", "example.com"), + ("UPPER.example.com/Some/Path", "upper.example.com/Some/Path"), + (" example.com ", "example.com"), + ("example.com/some/path", "example.com/some/path"), + ("a" * 240 + ".example.com", "a" * 240 + ".example.com"), + ], +) +def test_normalize_url_accepts(raw_url, expected): + """ + Given: + - A URL with a scheme, a trailing slash, mixed case or surrounding whitespace. + When: + - Normalizing it before submitting it to PAN-OS. + Then: + - The scheme and the trailing slash are stripped, the host is lowercased, the path keeps its + case, and no rejection message is returned. + """ + from BlockUrl import normalize_url + + submitted_url, rejection_message = normalize_url(raw_url) + assert submitted_url == expected + assert rejection_message == "" + + +@pytest.mark.parametrize( + "raw_url, expected_message_part", + [ + ("", "empty"), + (" ", "empty"), + ("*.example.com", "Wildcards are not supported"), + ("https://*.example.com", "Wildcards are not supported"), + ("a" * 256 + ".example.com", "exceeds 255 chars"), + ("example.com,other.example.com", "comma"), + ("example .com", "whitespace"), + ("ftp://example.com", "scheme is not supported"), + ("example.com", "not supported"), + ("exa&mple.com", "not supported"), + ('exa"mple.com', "not supported"), + ("exa'mple.com", "not supported"), + ("example.com/"} + assert incident_name(alert, "{description}") == "script{type_id}/script" + + +def test_incident_name_max_length(): + assert MAX_INCIDENT_NAME_LENGTH == 76 + name = incident_name({**CUSTOM_NAME_ALERT, "description": "x" * 1000}, "{description}") + assert len(name) == MAX_INCIDENT_NAME_LENGTH + assert name.endswith("...") + exact = incident_name({**CUSTOM_NAME_ALERT, "description": "x" * 76}, "{description}") + assert exact == "x" * 76 + + +def test_incident_name_long_example_is_truncated(): + name = incident_name(CUSTOM_NAME_ALERT, "{type_id}_{id} - {description}") + assert name == "NET:RST-FROM-SLAVE_1af93f46-65c1-4f52-a46a-2a181597fb0c - The slave 10.19..." + + +def test_parse_incident_uses_name_pattern(): + assert parse_incident(CUSTOM_NAME_ALERT, name_pattern="{type_id}_{id}")["name"] == ( + "NET:RST-FROM-SLAVE_1af93f46-65c1-4f52-a46a-2a181597fb0c" + ) + assert parse_incident(CUSTOM_NAME_ALERT, name_pattern=" ")["name"] == DEFAULT_NAME + assert parse_incident(CUSTOM_NAME_ALERT)["name"] == DEFAULT_NAME + + +def test_custom_incident_name_pattern_from_params(): + assert custom_incident_name_pattern({"customIncidentNamePattern": " {type_id}_{id} "}) == "{type_id}_{id}" + assert custom_incident_name_pattern({"customIncidentNamePattern": " "}) == "" + assert custom_incident_name_pattern({"customIncidentNamePattern": None}) == "" + assert custom_incident_name_pattern({}) == "" + + +def test_incidents_apply_custom_name_pattern(): + alerts = [{**CUSTOM_NAME_ALERT, "id": f"id-{n}"} for n in range(3)] + with patch("NozomiNetworks.incidents_better_than_time", return_value=alerts): + parsed, _ = incidents(0, {}, None, False, MagicMock(), "{type_id}_{id}") + default_named, _ = incidents(0, {}, None, False, MagicMock()) + assert [i["name"] for i in parsed] == [f"NET:RST-FROM-SLAVE_id-{n}" for n in range(3)] + assert [i["name"] for i in default_named] == [f"Link RST sent by Slave_id-{n}" for n in range(3)] + + +def test_fetch_incidents_reads_name_pattern_from_params(): + alerts = [CUSTOM_NAME_ALERT] + params = {"customIncidentNamePattern": "{type_id}_{id}", "fetchTime": "7 days"} + with ( + patch.object(demisto, "params", return_value=params), + patch.object(demisto, "getLastRun", return_value={}), + patch("NozomiNetworks.incidents_better_than_time", return_value=alerts), + ): + parsed, _ = fetch_incidents(MagicMock(), st="0", last_run={}, test_mode=True) + assert parsed[0]["name"] == "NET:RST-FROM-SLAVE_1af93f46-65c1-4f52-a46a-2a181597fb0c" diff --git a/Packs/NozomiNetworks/Integrations/NozomiNetworks/README.md b/Packs/NozomiNetworks/Integrations/NozomiNetworks/README.md index c3150b18be3d..65ae04b4af2b 100644 --- a/Packs/NozomiNetworks/Integrations/NozomiNetworks/README.md +++ b/Packs/NozomiNetworks/Integrations/NozomiNetworks/README.md @@ -2,6 +2,13 @@ The Nozomi Networks integration enables the collection of alert and asset inform This integration is used to gather alert and asset information from Nozomi. +## Features + +The integration provides the following capabilities: + +1. **Fetch Incidents**: Retrieve incidents directly from Nozomi Networks. +2. **Execute Commands**: Perform actions to gather and update data from Nozomi Networks. + ## Configure Nozomi Networks in Cortex | **Parameter** | **Description** | **Required** | @@ -16,6 +23,26 @@ This integration is used to gather alert and asset information from Nozomi. | riskFrom | Get incidents from risk level | False | | fecthAlsoIncidents | Fetch also nozomi incidents | False | | incidentPerRun | Specifies the maximum number of incidents to retrieve | False | +| customAlertFilter | Custom alert filter. An additional Nozomi query language filter appended to the alerts query, combined with the other fetch filters (for example `where risk >= 8`). The leading `\|` is optional. | False | +| defaultIncidentRole | Default incident role. The Cortex XSOAR role (or comma-separated roles) assigned to incidents created from fetched alerts. Leave empty to not assign a role. | False | +| Custom incident name pattern | The pattern used to build the incident name from Nozomi alert fields, for example `{type_id}_{id} - {description}`. Leave empty to use the default name `{name}_{id}`. See [Custom incident name](#custom-incident-name). | False | + +Fetched alerts are mapped to Cortex XSOAR incident severity based on the Nozomi risk: 1-5 Low, 6-7 Medium, 8-9 High, 10 Critical. + +### Custom incident name + +By default the incident name is `{name}_{id}`, built from the Nozomi alert name and id. Set **Custom incident name pattern** to build the name from other Nozomi alert fields. If the parameter is empty (or contains only whitespace), or the rendered name is empty, the default name is used, so existing instances are unaffected. + +Pattern syntax: + +- `{field}` is replaced with the value of that Nozomi alert field. Use the exact alert field name, for example `{id}`, `{name}`, `{description}`, `{type_id}`, `{type_name}`, `{ip_src}`, `{ip_dst}`, `{status}`. +- Missing, empty or unknown fields are rendered as empty text. +- Use `{{` and `}}` for literal braces. +- Values are inserted as plain text only (they are never evaluated or re-interpolated). HTML angle brackets (`<`, `>`) are removed, control characters and newlines are replaced by spaces, and repeated whitespace is collapsed. +- The name is limited to 76 characters (36 for the alert id plus 40; longer names are truncated with `...`). +- The name depends only on the alert data, so it is the same on every fetch. + +Example: `{type_id}_{id} - {description}` produces `NET:RST-FROM-SLAVE_1af93f46-65c1-4f52-a46a-2a181597fb0c - The slave 10.19...`. ## Commands diff --git a/Packs/NozomiNetworks/README.md b/Packs/NozomiNetworks/README.md index c02dcd44cf3e..6e1b9d66e1de 100644 --- a/Packs/NozomiNetworks/README.md +++ b/Packs/NozomiNetworks/README.md @@ -14,6 +14,7 @@ With the **NozomiNetworks** pack you can: * Manage incidents * Import and sync the Nozomi incidents * You have to look for _Nozomi Networks_ in the **Settings** > **Integrations** > **Servers & Services** section and add an instance. + * Optionally customize the incident name with the _Custom incident name pattern_ parameter (default: `{name}_{id}`); see the integration README for the pattern syntax, e.g. `{type_id}_{id} - {description}` * Through the _**nozomi-close-incidents-as-change**_ and _**nozomi-close-incidents-as-security**_ commands close the incidents inside the Nozomi platform * Find assets * the command _**nozomi-find-assets**_ return the asset filtered with the attributes passed. diff --git a/Packs/NozomiNetworks/ReleaseNotes/1_1_0.md b/Packs/NozomiNetworks/ReleaseNotes/1_1_0.md new file mode 100644 index 000000000000..4db029bacb74 --- /dev/null +++ b/Packs/NozomiNetworks/ReleaseNotes/1_1_0.md @@ -0,0 +1,8 @@ + +#### Integrations + +##### Nozomi Networks + +- Added support for **Custom alert filter** parameter that an additional Nozomi query language filter appended to the alerts query, combined with the other fetch filters. +- Added support for **Default incident role** parameter that the Cortex XSOAR role assigned to incidents created from fetched Nozomi alerts, for example analyst_l2. +- Updated the Docker image to: *demisto/python3:3.12.14.13495286*. diff --git a/Packs/NozomiNetworks/ReleaseNotes/1_2_0.md b/Packs/NozomiNetworks/ReleaseNotes/1_2_0.md new file mode 100644 index 000000000000..f89d4561a0b7 --- /dev/null +++ b/Packs/NozomiNetworks/ReleaseNotes/1_2_0.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### Nozomi Networks + +- Added the *Custom incident name pattern* integration parameter, which allows building the incident name by concatenating Nozomi alert fields, for example *{type_id}_{id} - {description}*. The default name (*{name}_{id}*) is used when the parameter is empty. diff --git a/Packs/NozomiNetworks/pack_metadata.json b/Packs/NozomiNetworks/pack_metadata.json index 89d72f818724..3d8e0b9f9dbb 100644 --- a/Packs/NozomiNetworks/pack_metadata.json +++ b/Packs/NozomiNetworks/pack_metadata.json @@ -4,7 +4,7 @@ "support": "partner", "author": "Nozomi Networks", "url": "https://www.nozominetworks.com", - "currentVersion": "1.0.24", + "currentVersion": "1.2.0", "email": "support@nozominetworks.com", "categories": [ "Network Security" @@ -15,7 +15,9 @@ "beta": false, "deprecated": false, "useCases": [], - "keywords": [], + "keywords": [ + "Nozomi Networks" + ], "dependencies": {}, "certification": "certified", "marketplaces": [ diff --git a/Packs/OPNSense/Integrations/OPNSense/OPNSense.yml b/Packs/OPNSense/Integrations/OPNSense/OPNSense.yml index 5d718cbe2040..9843f357ba48 100644 --- a/Packs/OPNSense/Integrations/OPNSense/OPNSense.yml +++ b/Packs/OPNSense/Integrations/OPNSense/OPNSense.yml @@ -959,7 +959,7 @@ script: description: Rollback revision. description: Revert config to given savepoint. outputs: [] - dockerimage: demisto/opnsense:1.0.0.10133006 + dockerimage: demisto/opnsense:1.0.0.12571702 subtype: python3 fromversion: 6.0.0 tests: diff --git a/Packs/OPNSense/ReleaseNotes/1_0_32.md b/Packs/OPNSense/ReleaseNotes/1_0_32.md new file mode 100644 index 000000000000..2abdb300ebda --- /dev/null +++ b/Packs/OPNSense/ReleaseNotes/1_0_32.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### OPNSense + +- Updated the Docker image to: *demisto/opnsense:1.0.0.12571702*. + diff --git a/Packs/OPNSense/pack_metadata.json b/Packs/OPNSense/pack_metadata.json index b5028c1964c6..905fc0514be1 100644 --- a/Packs/OPNSense/pack_metadata.json +++ b/Packs/OPNSense/pack_metadata.json @@ -2,7 +2,7 @@ "name": "OPNSense", "description": "Manage OPNsense Firewall", "support": "partner", - "currentVersion": "1.0.31", + "currentVersion": "1.0.32", "author": "SecInfra", "url": "http://secinfra.fr", "email": "support@secinfra.fr", diff --git a/Packs/OPSWAT-MetaDefender-Aether/Integrations/MetaDefenderAether/MetaDefenderAether.yml b/Packs/OPSWAT-MetaDefender-Aether/Integrations/MetaDefenderAether/MetaDefenderAether.yml index 77aa8583aeec..97e50da45406 100644 --- a/Packs/OPSWAT-MetaDefender-Aether/Integrations/MetaDefenderAether/MetaDefenderAether.yml +++ b/Packs/OPSWAT-MetaDefender-Aether/Integrations/MetaDefenderAether/MetaDefenderAether.yml @@ -35,7 +35,7 @@ script: script: "-" type: python subtype: python3 - dockerimage: demisto/fastapi:0.125.0.10158186 + dockerimage: demisto/fastapi:0.125.0.12689358 runonce: false commands: - name: metadefender-aether-scan-url diff --git a/Packs/OPSWAT-MetaDefender-Aether/ReleaseNotes/1_0_2.md b/Packs/OPSWAT-MetaDefender-Aether/ReleaseNotes/1_0_2.md new file mode 100644 index 000000000000..fa98de4bb576 --- /dev/null +++ b/Packs/OPSWAT-MetaDefender-Aether/ReleaseNotes/1_0_2.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### MetaDefender Aether + +- Updated the Docker image to: *demisto/fastapi:0.125.0.12689358*. + diff --git a/Packs/OPSWAT-MetaDefender-Aether/pack_metadata.json b/Packs/OPSWAT-MetaDefender-Aether/pack_metadata.json index c91621c0eb75..6f65fb047416 100644 --- a/Packs/OPSWAT-MetaDefender-Aether/pack_metadata.json +++ b/Packs/OPSWAT-MetaDefender-Aether/pack_metadata.json @@ -2,7 +2,7 @@ "name": "MetaDefender Aether", "description": "Next-generation unified Zero-Day detection solution", "support": "partner", - "currentVersion": "1.0.1", + "currentVersion": "1.0.2", "author": "OPSWAT", "url": "https://www.opswat.com/products/metadefender/aether", "email": "threatintel@opswat.com", diff --git a/Packs/OSQuery/ReleaseNotes/1_0_17.md b/Packs/OSQuery/ReleaseNotes/1_0_17.md new file mode 100644 index 000000000000..fda038442b86 --- /dev/null +++ b/Packs/OSQuery/ReleaseNotes/1_0_17.md @@ -0,0 +1,6 @@ + +#### Scripts + +##### OSQueryBasicQuery + +- Improved input sanitization for the *query* argument. diff --git a/Packs/OSQuery/Scripts/OSQueryBasicQuery/OSQueryBasicQuery.py b/Packs/OSQuery/Scripts/OSQueryBasicQuery/OSQueryBasicQuery.py index 78c1eb67fd22..b401f6aaae8b 100644 --- a/Packs/OSQuery/Scripts/OSQueryBasicQuery/OSQueryBasicQuery.py +++ b/Packs/OSQuery/Scripts/OSQueryBasicQuery/OSQueryBasicQuery.py @@ -1,10 +1,12 @@ import demistomock as demisto # noqa: F401 from CommonServerPython import * # noqa: F401 import json +import shlex -# ssh command to run, json format, param = query to execute -COMMAND = 'osqueryi --json "{0}"' +# ssh command to run, json format, param = query to execute. +# {0} is filled with a shlex.quote()-escaped value at call-site. +COMMAND = "osqueryi --json {0}" def main(): @@ -16,7 +18,8 @@ def main(): if query and systems: for system in systems: - temp_res = demisto.executeCommand("RemoteExec", {"cmd": COMMAND.format(str(query)), "system": system}) + safe_query = shlex.quote(str(query)) + temp_res = demisto.executeCommand("RemoteExec", {"cmd": COMMAND.format(safe_query), "system": system}) if isError(temp_res[0]): temp_res_contents = temp_res[0]["Contents"] error_res += [ diff --git a/Packs/OSQuery/Scripts/OSQueryBasicQuery/OSQueryBasicQuery_test.py b/Packs/OSQuery/Scripts/OSQueryBasicQuery/OSQueryBasicQuery_test.py index cdea43597c0b..0b754806789e 100644 --- a/Packs/OSQuery/Scripts/OSQueryBasicQuery/OSQueryBasicQuery_test.py +++ b/Packs/OSQuery/Scripts/OSQueryBasicQuery/OSQueryBasicQuery_test.py @@ -1,18 +1,19 @@ import demistomock as demisto +import pytest -def test_OSQueryBasicQuery(mocker): +def test_OSQueryBasicQuery_error(mocker): """ Given: - - The script args. + - The script args with a valid query that triggers a RemoteExec error. When: - Running the OSQueryBasicQuery script. Then: - - Validating the outputs as expected. + - The error entry is returned in results. """ from OSQueryBasicQuery import main - args_mock = mocker.patch.object(demisto, "args", return_value={"system": "system", "query": "query"}) # noqa: F841 + mocker.patch.object(demisto, "args", return_value={"system": "system", "query": "query"}) execute_command_res = [{"Type": 4, "Contents": "Error", "Brand": "brand"}] execute_mock = mocker.patch.object(demisto, "executeCommand", return_value=execute_command_res) results_mock = mocker.patch.object(demisto, "results") @@ -20,3 +21,68 @@ def test_OSQueryBasicQuery(mocker): assert execute_mock.call_count == 1 assert "An Error occurred on remote system" in results_mock.call_args[0][0][0]["Contents"] + + +@pytest.mark.parametrize( + "query_with_special_chars", + [ + 'SELECT * FROM users WHERE name = "admin"', + "SELECT path FROM file WHERE path LIKE '/etc/%'", + "SELECT $(echo test) FROM processes", + "SELECT `id` FROM users", + "SELECT pid, name FROM processes | head", + "SELECT * FROM users WHERE uid > 0 && uid < 100", + ], +) +def test_OSQueryBasicQuery_injection_payloads_are_quoted(mocker, query_with_special_chars): + """ + Given: + - A query argument containing shell special characters. + When: + - Running the OSQueryBasicQuery script. + Then: + - The command passed to RemoteExec, when parsed by a POSIX shell, yields the + query as a single literal argument to osqueryi — no shell interpretation. + """ + import shlex + from OSQueryBasicQuery import main + + mocker.patch.object(demisto, "args", return_value={"system": "target-host", "query": query_with_special_chars}) + execute_mock = mocker.patch.object(demisto, "executeCommand", return_value=[{"Type": 1, "Contents": "[]"}]) + mocker.patch.object(demisto, "results") + main() + + assert execute_mock.call_count == 1 + called_cmd = execute_mock.call_args[0][1]["cmd"] + + # Parse the command exactly as a POSIX shell would: the third token (index 2) + # must equal the raw query — proving shlex.quote neutralised all special characters + # and the shell would treat the entire query as one literal argument. + parsed_args = shlex.split(called_cmd) + assert len(parsed_args) == 3, f"Expected 3 shell tokens (osqueryi --json ), got {parsed_args!r}" + assert parsed_args[2] == query_with_special_chars, ( + f"Shell would not receive the query as a single literal argument. " + f"cmd={called_cmd!r}, parsed arg={parsed_args[2]!r}, expected={query_with_special_chars!r}" + ) + + +def test_OSQueryBasicQuery_normal_query_format(mocker): + """ + Given: + - A benign SQL query. + When: + - Running the OSQueryBasicQuery script. + Then: + - The command sent to RemoteExec matches the expected safe format. + """ + import shlex + from OSQueryBasicQuery import main, COMMAND + + query = "SELECT * FROM users" + mocker.patch.object(demisto, "args", return_value={"system": "host1", "query": query}) + execute_mock = mocker.patch.object(demisto, "executeCommand", return_value=[{"Type": 1, "Contents": "[]"}]) + mocker.patch.object(demisto, "results") + main() + + expected_cmd = COMMAND.format(shlex.quote(query)) + assert execute_mock.call_args[0][1]["cmd"] == expected_cmd diff --git a/Packs/OSQuery/pack_metadata.json b/Packs/OSQuery/pack_metadata.json index bb49f315bf88..62aea5cdb1a1 100644 --- a/Packs/OSQuery/pack_metadata.json +++ b/Packs/OSQuery/pack_metadata.json @@ -2,7 +2,7 @@ "name": "OS Query", "description": "Run OS query on a linux system.", "support": "xsoar", - "currentVersion": "1.0.16", + "currentVersion": "1.0.17", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/OTRS/Integrations/OTRS/OTRS.yml b/Packs/OTRS/Integrations/OTRS/OTRS.yml index 1a13d2c74ca1..e0f920b87609 100644 --- a/Packs/OTRS/Integrations/OTRS/OTRS.yml +++ b/Packs/OTRS/Integrations/OTRS/OTRS.yml @@ -510,7 +510,7 @@ script: - contextPath: OTRS.Ticket.Article.ContentType description: Ticket article content type. type: string - dockerimage: demisto/pyotrs:1.0.0.10133006 + dockerimage: demisto/pyotrs:1.0.0.13531018 isfetch: true runonce: false script: '-' diff --git a/Packs/OTRS/ReleaseNotes/1_1_12.md b/Packs/OTRS/ReleaseNotes/1_1_12.md new file mode 100644 index 000000000000..62ebc252cd02 --- /dev/null +++ b/Packs/OTRS/ReleaseNotes/1_1_12.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### OTRS + +- Updated the Docker image to: *demisto/pyotrs:1.0.0.12701738*. + diff --git a/Packs/OTRS/ReleaseNotes/1_1_13.md b/Packs/OTRS/ReleaseNotes/1_1_13.md new file mode 100644 index 000000000000..10e96d276ba9 --- /dev/null +++ b/Packs/OTRS/ReleaseNotes/1_1_13.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### OTRS + +- Updated the Docker image to: *demisto/pyotrs:1.0.0.13531018*. + diff --git a/Packs/OTRS/pack_metadata.json b/Packs/OTRS/pack_metadata.json index 66b022ad4300..5fc7aef7bfb7 100644 --- a/Packs/OTRS/pack_metadata.json +++ b/Packs/OTRS/pack_metadata.json @@ -2,7 +2,7 @@ "name": "OTRS Service Management XSOAR Pack", "description": "An Integration Pack to Create, Update, and Delete OTRS Tickets", "support": "partner", - "currentVersion": "1.1.11", + "currentVersion": "1.1.13", "author": "OTRS AG", "url": "https://otrs.com/", "email": "sales@otrs.com", diff --git a/Packs/OctoxLabs/Integrations/OctoxLabs/OctoxLabs.yml b/Packs/OctoxLabs/Integrations/OctoxLabs/OctoxLabs.yml index 50634c659db2..1d9ef3390fe8 100644 --- a/Packs/OctoxLabs/Integrations/OctoxLabs/OctoxLabs.yml +++ b/Packs/OctoxLabs/Integrations/OctoxLabs/OctoxLabs.yml @@ -814,7 +814,7 @@ script: script: '-' type: python subtype: python3 - dockerimage: demisto/octoxlabs:1.0.0.10133006 + dockerimage: demisto/octoxlabs:1.0.0.12833174 fromversion: 6.0.0 tests: - No tests (auto formatted) diff --git a/Packs/OctoxLabs/ReleaseNotes/2_1_5.md b/Packs/OctoxLabs/ReleaseNotes/2_1_5.md new file mode 100644 index 000000000000..02b713c2daa9 --- /dev/null +++ b/Packs/OctoxLabs/ReleaseNotes/2_1_5.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### OctoxLabs + +- Updated the Docker image to: *demisto/octoxlabs:1.0.0.12833174*. + diff --git a/Packs/OctoxLabs/pack_metadata.json b/Packs/OctoxLabs/pack_metadata.json index eff3b2fbb88b..11cc518c89fb 100644 --- a/Packs/OctoxLabs/pack_metadata.json +++ b/Packs/OctoxLabs/pack_metadata.json @@ -2,7 +2,7 @@ "name": "OctoxLabs", "description": "Octox Labs Cyber Security Asset Management platform", "support": "partner", - "currentVersion": "2.1.4", + "currentVersion": "2.1.5", "author": "OctoxLabs", "url": "https://octoxlabs.com", "email": "info@octoxlabs.com", diff --git a/Packs/Office365AndAzureAuditLog/Integrations/MicrosoftPolicyAndComplianceAuditLog/MicrosoftPolicyAndComplianceAuditLog.yml b/Packs/Office365AndAzureAuditLog/Integrations/MicrosoftPolicyAndComplianceAuditLog/MicrosoftPolicyAndComplianceAuditLog.yml index df5e6f6fd4b2..0e7cadcdf62b 100644 --- a/Packs/Office365AndAzureAuditLog/Integrations/MicrosoftPolicyAndComplianceAuditLog/MicrosoftPolicyAndComplianceAuditLog.yml +++ b/Packs/Office365AndAzureAuditLog/Integrations/MicrosoftPolicyAndComplianceAuditLog/MicrosoftPolicyAndComplianceAuditLog.yml @@ -154,7 +154,7 @@ script: runonce: false script: '-' type: powershell - dockerimage: demisto/pwsh-exchangev3:1.0.0.10133006 + dockerimage: demisto/pwsh-exchangev3:1.0.0.11550820 fromversion: 5.5.0 tests: - Audit Log - Test diff --git a/Packs/Office365AndAzureAuditLog/ReleaseNotes/2_0_11.md b/Packs/Office365AndAzureAuditLog/ReleaseNotes/2_0_11.md new file mode 100644 index 000000000000..adc93ea97cce --- /dev/null +++ b/Packs/Office365AndAzureAuditLog/ReleaseNotes/2_0_11.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### Microsoft Policy And Compliance (Audit Log) + +- Updated the Docker image to: *demisto/pwsh-exchangev3:1.0.0.11550820*. + diff --git a/Packs/Office365AndAzureAuditLog/pack_metadata.json b/Packs/Office365AndAzureAuditLog/pack_metadata.json index bd9773208d23..ab697e4f9ed1 100644 --- a/Packs/Office365AndAzureAuditLog/pack_metadata.json +++ b/Packs/Office365AndAzureAuditLog/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Office 365 and Azure (Audit Log)", "description": "Search the unified audit log to view user and administrator activity in your organization.", "support": "xsoar", - "currentVersion": "2.0.10", + "currentVersion": "2.0.11", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector.py b/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector.py index e50829cd667d..9ffc85fab8d4 100644 --- a/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector.py +++ b/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector.py @@ -1,5 +1,6 @@ import re from typing import cast +from urllib.parse import urlsplit, urlunsplit, parse_qsl import demistomock as demisto # noqa: F401 import urllib3 @@ -286,7 +287,16 @@ def get_events(self, since: Any, page_size: int = Config.PAGE_SIZE, next_link_ur """ if next_link_url: demisto.debug("[API Fetch] Requesting events using the pagination next_link") - return self._http_request(method="GET", full_url=next_link_url, resp_type="response") + # Build the request from the cursor carried in the link plus an explicit page + # size and ordering, so it does not depend on which parameters the link happens + # to include. A link that carries only the cursor would otherwise let the API + # apply its smaller default page size and default ordering, stalling the cursor. + split_url = urlsplit(next_link_url) + base_url = urlunsplit((split_url.scheme, split_url.netloc, split_url.path, "", "")) + link_params = dict(parse_qsl(split_url.query)) + link_params["limit"] = str(page_size) + link_params["sortOrder"] = Config.SORT_ORDER + return self._http_request(method="GET", full_url=base_url, params=link_params, resp_type="response") params = { "sortOrder": Config.SORT_ORDER, diff --git a/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector.yml b/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector.yml index 35d545236da6..f7f576cd1bb0 100644 --- a/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector.yml +++ b/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector.yml @@ -114,7 +114,7 @@ script: - contextPath: Okta.Event.severity description: The severity of the event. type: String - dockerimage: demisto/fastapi:0.125.0.10158186 + dockerimage: demisto/fastapi:0.125.0.12689358 isfetchevents: true subtype: python3 marketplaces: diff --git a/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector_test.py b/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector_test.py index db2c9343ab90..787a8ab3a540 100644 --- a/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector_test.py +++ b/Packs/Okta/Integrations/OktaEventCollector/OktaEventCollector_test.py @@ -1076,17 +1076,25 @@ def test_client_retry_policy_includes_server_errors(status_code): assert status_code in Client(BASE_URL, "my-token")._retry_policy.retryable_status_codes -def test_client_get_events_uses_full_url_for_next_link(dummy_client, mocker): +def test_client_get_events_forwards_next_link_cursor_as_params(dummy_client, mocker): """ - Given: A pagination cursor. + Given: A pagination cursor whose query string carries the resume parameters. When: Requesting the next page. - Then: The request targets the full cursor URL rather than rebuilding the query. + Then: The cursor query parameters are forwarded explicitly and the target URL keeps + only the path, so the resume parameters cannot be dropped by the HTTP client. + The page size is also set explicitly so the API does not fall back to a smaller + default. """ request_mock = mocker.patch.object(dummy_client, "_http_request", return_value=MockResponse()) dummy_client.get_events(since="ignored", next_link_url=NEXT_URL) - assert request_mock.call_args.kwargs["full_url"] == NEXT_URL + assert request_mock.call_args.kwargs["full_url"] == "https://okta.example.com/api/v1/logs" + assert request_mock.call_args.kwargs["params"] == { + "after": "cursor2", + "limit": str(Config.PAGE_SIZE), + "sortOrder": Config.SORT_ORDER, + } assert request_mock.call_args.kwargs["resp_type"] == "response" @@ -1352,3 +1360,55 @@ def test_get_last_run_cursor_round_trips(published): # endregion + +# region next_link pagination request + + +def test_next_link_request_preserves_query_parameters(scripted_client, capfd): + """ + Given: A pagination link that carries the full set of query parameters the API + needs to resume paging (ordering, page size and the cursor token). + When: The client follows that link to fetch the next page. + Then: Every query parameter is present on the outgoing request, so the API resumes + from the cursor with the expected page size and ordering. + """ + next_link = ( + "https://okta.example.com/api/v1/logs" + "?since=2022-04-17T00:00:00.000000&limit=1000&sortOrder=ASCENDING&after=cursor-token-123" + ) + client, transport, _ = scripted_client([okta_response(200, events=id1_pub)]) + + with capfd.disabled(): + client.get_events(since="unused", next_link_url=next_link) + + sent_query = dict(transport.requests[0].url.params) + assert sent_query == { + "since": "2022-04-17T00:00:00.000000", + "limit": "1000", + "sortOrder": "ASCENDING", + "after": "cursor-token-123", + } + + +def test_next_link_request_enforces_page_size_and_order_when_absent(scripted_client, capfd): + """ + Given: A pagination link that carries only the cursor token and omits the page size + and ordering. + When: The client follows that link to fetch the next page. + Then: The client sets the page size and ordering explicitly on the outgoing request, + so the API returns a full page in the expected order instead of falling back to + its smaller default page or its default order. The cursor token is preserved. + """ + next_link = "https://okta.example.com/api/v1/logs?after=cursor-token-123" + client, transport, _ = scripted_client([okta_response(200, events=id1_pub)]) + + with capfd.disabled(): + client.get_events(since="unused", next_link_url=next_link) + + sent_query = dict(transport.requests[0].url.params) + assert sent_query.get("after") == "cursor-token-123" + assert sent_query.get("limit") == str(Config.PAGE_SIZE) + assert sent_query.get("sortOrder") == Config.SORT_ORDER + + +# endregion diff --git a/Packs/Okta/Integrations/OktaEventCollector/README.md b/Packs/Okta/Integrations/OktaEventCollector/README.md index 84286d060542..1b8e414a2b5d 100644 --- a/Packs/Okta/Integrations/OktaEventCollector/README.md +++ b/Packs/Okta/Integrations/OktaEventCollector/README.md @@ -1,7 +1,5 @@ Collects the events log for authentication and Audit provided by Okta admin API -This is the default integration for this content pack when configured by the Data Onboarder in Cortex XSIAM. - ## Configure Okta Log in Cortex | **Parameter** | **Description** | **Required** | diff --git a/Packs/Okta/Integrations/Okta_IAM/Okta_IAM.py b/Packs/Okta/Integrations/Okta_IAM/Okta_IAM.py index 40aa04174bde..0e407d50a67a 100644 --- a/Packs/Okta/Integrations/Okta_IAM/Okta_IAM.py +++ b/Packs/Okta/Integrations/Okta_IAM/Okta_IAM.py @@ -1,949 +1,14 @@ import demistomock as demisto # noqa: F401 -import urllib3 from CommonServerPython import * # noqa: F401 -# Disable insecure warnings -urllib3.disable_warnings() +from OktaIAMApiModule import * # noqa: E402 -"""CONSTANTS""" -BATCH_SIZE = 2000 -DATE_FORMAT = "%Y-%m-%dT%H:%M:%SZ" -DEPROVISIONED_STATUS = "DEPROVISIONED" -USER_IS_DISABLED_MSG = "User is already disabled." -USER_IS_DISABLED_ERROR = "E0000007" -ERROR_CODES_TO_SKIP = [ - "E0000016", # user is already enabled - USER_IS_DISABLED_ERROR, -] -ERROR_CODES_TO_RETURN_ERROR = [ - "E0000047", # rate limit - resets after 1 minute -] +def main() -> None: + run_okta_iam_integration() -FETCH_QUERY_EXCEPTION_MSG = ( - 'If you marked the "Query only application events configured in IAM Configuration" ' - "checkbox in the instance configuration, you must add at least one application in " - "the IAM Configuration incident before fetching logs from Okta. " - "Alternatively, you can unmark this checkbox and provide a " - '"Fetch Query Filter" parameter instead.' -) -GET_USER_ATTRIBUTES = ["id", "login", "email"] -MAX_LOGS_LIMIT = 1000 - -"""CLIENT CLASS""" - - -class Client(BaseClient): - """ - Okta IAM Client class that implements logic to authenticate with Okta. - """ - - def test_connection(self): - uri = "users/me" - self._http_request(method="GET", url_suffix=uri) - - def get_user(self, filter_name: str, filter_value: str): - filter_name = filter_name if filter_name == "id" else f"profile.{filter_name}" - uri = "users" - query_params = {"filter": f'{filter_name} eq "{filter_value}"'} - - res = self._http_request(method="GET", url_suffix=uri, params=query_params) - - if res and len(res) == 1: - return res[0] - return None - - def deactivate_user(self, user_id): - uri = f"users/{user_id}/lifecycle/deactivate" - self._http_request(method="POST", url_suffix=uri) - - def activate_user(self, user_id): - query_params = {"sendEmail": "false"} - uri = f"users/{user_id}/lifecycle/activate" - self._http_request(method="POST", url_suffix=uri, params=query_params) - - def create_user(self, user_data): - # create a user in staged mode (not active) - body = {"profile": user_data} - uri = "users" - query_params = {"activate": "false", "provider": "true"} - res = self._http_request(method="POST", url_suffix=uri, json_data=body, params=query_params) - return res - - def update_user(self, user_id, user_data): - body = {"profile": user_data} - uri = f"users/{user_id}" - res = self._http_request(method="POST", url_suffix=uri, json_data=body) - return res - - def get_okta_fields(self): - okta_fields = {} - uri = "meta/schemas/user/default" - res = self._http_request(method="GET", url_suffix=uri) - - base_properties = res.get("definitions", {}).get("base", {}).get("properties", {}) - okta_fields.update({k: base_properties[k].get("title") for k in base_properties}) - - custom_properties = res.get("definitions", {}).get("custom", {}).get("properties", {}) - okta_fields.update({k: custom_properties[k].get("title") for k in custom_properties}) - - return okta_fields - - def http_request(self, method, url_suffix, full_url=None, params=None, data=None, headers=None): - if headers is None: - headers = self._headers - full_url = full_url if full_url else urljoin(self._base_url, url_suffix) - - res = requests.request(method, full_url, verify=self._verify, headers=headers, params=params, json=data) - return res - - def search_group(self, group_name): - uri = "groups" - query_params = {"q": encode_string_results(group_name)} - return self.http_request(method="GET", url_suffix=uri, params=query_params) - - def get_group_by_id(self, group_id): - uri = f"groups/{group_id}" - return self.http_request(method="GET", url_suffix=uri) - - def get_group_members(self, group_id): - uri = f"groups/{group_id}/users" - return self.get_paged_results(uri) - - def get_paged_results(self, uri, query_param=None): - response = self.http_request(method="GET", url_suffix=uri, params=query_param) - paged_results = response.json() - if response.status_code != 200: - raise Exception(f"Error occurred while calling Okta API: {response.request.url}. Response: {response.json()}") - while "next" in response.links and len(response.json()) > 0: - next_page = response.links.get("next").get("url") - response = self._http_request(method="GET", full_url=next_page, url_suffix="") - if response.status_code != 200: - raise Exception(f"Error occurred while calling Okta API: {response.request.url}. Response: {response.json()}") - paged_results += response.json() - return paged_results - - def get_app_user_assignment(self, application_id, user_id): - uri = f"/apps/{application_id}/users/{user_id}" - res = self._http_request(method="GET", url_suffix=uri, resp_type="response", ok_codes=(200, 404)) - return res - - def list_user_apps(self, user_id): - uri = "apps" - query_params = {"filter": f'user.id eq "{user_id}"'} - res = self._http_request(method="GET", url_suffix=uri, params=query_params) - return res - - def list_apps(self, query, page, limit): - query_params = {"q": query, "limit": limit} - - curr_page = 0 - apps_batch, next_page = self.list_apps_batch(url_suffix="/apps", params=query_params) - - while apps_batch and curr_page != page: - curr_page += 1 - apps_batch, next_page = self.list_apps_batch(full_url=next_page) - - if not apps_batch: - apps_batch = [] - return apps_batch - - def list_apps_batch(self, url_suffix="", params=None, full_url=""): - """Gets a batch of apps from Okta. - Args: - url_suffix (str): The apps API endpoint. - params (dict): The API query params. - full_url (str): The full url retrieved from the last API call. - - Return: - apps_batch (dict): The logs batch. - next_page (str): URL for next API call (equals '' on last batch). - """ - if not url_suffix and not full_url: - return None, None - - res = self._http_request(method="GET", url_suffix=url_suffix, params=params, full_url=full_url, resp_type="response") - - logs_batch = res.json() - next_page = res.links.get("next", {}).get("url") - - return logs_batch, next_page - - def get_logs( - self, - next_page=None, - last_run_time=None, - time_now=None, - query_filter=None, - auto_generate_filter=False, - context=None, - limit=None, - ): - logs = [] - - uri = "logs" - - if auto_generate_filter: - query_filter = get_query_filter(context) - - params = {"filter": query_filter, "since": last_run_time, "until": time_now, "limit": limit} - limit = int(limit) if limit else None - if limit and limit <= MAX_LOGS_LIMIT: - return self._http_request( - method="GET", - url_suffix=uri, - params=params, - ), None - - if limit and limit > MAX_LOGS_LIMIT: - params["limit"] = MAX_LOGS_LIMIT - - logs_batch, next_page = self.get_logs_batch(url_suffix=uri, params=params, full_url=next_page) - - try: - while logs_batch: - logs.extend(logs_batch) - if limit and len(logs) > limit: - return logs[:limit], next_page - logs_batch, next_page = self.get_logs_batch(full_url=next_page) - except DemistoException as e: - # in case of too many API calls, we return what we got and save the next_page for next fetch - if not is_rate_limit_error(e): - raise e - - return logs, next_page - - def get_logs_batch(self, url_suffix="", params=None, full_url=""): - """Gets a batch of logs from Okta. - Args: - url_suffix (str): The logs API endpoint. - params (dict): The API query params. - full_url (str): The full url retrieved from the last API call. Preferred over url_suffix if not empty. - - Return: - logs_batch (dict): The logs batch. - next_page (str): URL for next API call (equals '' on last batch). - """ - if not url_suffix and not full_url: - return None, None - - res = self._http_request(method="GET", url_suffix=url_suffix, params=params, full_url=full_url, resp_type="response") - - logs_batch = res.json() - next_page = res.links.get("next", {}).get("url") - - return logs_batch, next_page - - -"""HELPER FUNCTIONS""" - - -def get_all_user_profiles(): - query = 'type:"User Profile"' - email_to_user_profile = {} - - user_profiles: List[dict] = [] - search_indicators = IndicatorsSearcher(query=query, size=BATCH_SIZE) - for user_profile_res in search_indicators: - user_profiles.extend(user_profile_res.get("iocs") or []) - - for user_profile in user_profiles: - user_profile = user_profile.get("CustomFields", {}) - email_to_user_profile[user_profile.get("email")] = user_profile - - return email_to_user_profile - - -def get_event_username(log_entry): - for target in log_entry.get("target", []): - if target.get("type") == "User": - return target.get("alternateId") - return None - - -def should_drop_event(log_entry, email_to_user_profile): - """Returns a boolean value indicates whether the incident should be dropped. - - Args: - log_entry (dict): The log entry. - - Returns: - (bool) True iff the event should be dropped. - """ - username = get_event_username(log_entry) - if username is not None and email_to_user_profile.get(username) is None: - demisto.info(f"Dropping incident for user with username {username} - User Profile does not exist in XSOAR.") - return True - return False - - -def add_user_profile_data_to_entry(log_entry, email_to_user_profile): - username = get_event_username(log_entry) - user_profile = email_to_user_profile.get(username, {}) - log_entry.update(user_profile) - log_entry["UserProfile"] = user_profile - - -def get_query_filter(context): - iam_configuration = context.get("IAMConfiguration", []) - if not iam_configuration: - raise DemistoException(FETCH_QUERY_EXCEPTION_MSG) - - application_ids = [row["ApplicationID"] for row in iam_configuration] - - query_filter = '(eventType eq "application.user_membership.add" or eventType eq "application.user_membership.remove") and' - - query_filter += "(" + " or ".join([f'target.id co "{app_id}"' for app_id in application_ids]) + ")" - - return query_filter - - -def is_rate_limit_error(e): - if hasattr(e, "res") and e.res is not None: - return e.res.status_code == 429 - return False - - -def handle_exception(user_profile, e, action, okta_user=None): - """Handles failed responses from Okta API by setting the User Profile object with the results. - - Args: - user_profile (IAMUserProfile): The User Profile object. - e (Exception): The exception error. If DemistoException, holds the response json. - action (IAMActions): An enum represents the current action (get, update, create, etc). - """ - if e.__class__ is DemistoException and hasattr(e, "res") and e.res is not None: - try: - resp = e.res.json() - error_code = resp.get("errorCode") - error_message = get_error_details(resp) - except ValueError: - error_code = e.res.status_code - error_message = str(e) - else: - error_code = "" - error_message = str(e) - - if error_code == USER_IS_DISABLED_ERROR: - user_profile.set_user_is_already_disabled(okta_user) - - elif error_code in ERROR_CODES_TO_SKIP: - user_profile.set_result(action=action, skip=True, skip_reason=error_message) - else: - should_return_error = error_code in ERROR_CODES_TO_RETURN_ERROR - user_profile.set_result( - action=action, success=False, return_error=should_return_error, error_code=error_code, error_message=error_message - ) - - demisto.error(traceback.format_exc()) - - -def get_error_details(res): - """Parses the error details retrieved from Okta and outputs the resulted string. - - Args: - res (dict): The data retrieved from Okta. - - Returns: - (str) The parsed error details. - """ - error_msg = f'{res.get("errorSummary")}. ' - causes = "" - for idx, cause in enumerate(res.get("errorCauses", []), 1): - causes += f'{idx}. {cause.get("errorSummary")}\n' - if causes: - error_msg += f"Reason:\n{causes}" - return error_msg - - -"""COMMAND FUNCTIONS""" - - -def test_module(client, is_fetch, fetch_query_filter, auto_generate_query_filter, context, first_fetch_str): - if is_fetch: - if auto_generate_query_filter: - get_query_filter(context) # will raise an exception if configuration doesn't exist - elif not fetch_query_filter: - raise DemistoException(FETCH_QUERY_EXCEPTION_MSG) - try: - dateparser.parse(first_fetch_str).strftime(DATE_FORMAT) # type: ignore - except AttributeError: - raise DemistoException("First fetch timestamp parameter is not in the correct format.") - - client.test_connection() - - return_results("ok") - - -def get_mapping_fields_command(client): - okta_fields = client.get_okta_fields() - incident_type_scheme = SchemeTypeMapping(type_name=IAMUserProfile.DEFAULT_INCIDENT_TYPE) - - for field, description in okta_fields.items(): - incident_type_scheme.add_field(field, description) - - return GetMappingFieldsResponse([incident_type_scheme]) - - -def get_user_command(client, args, mapper_in, mapper_out): - user_profile = IAMUserProfile( - user_profile=args.get("user-profile"), mapper=mapper_out, incident_type=IAMUserProfile.UPDATE_INCIDENT_TYPE - ) - try: - iam_attr, iam_attr_value = user_profile.get_first_available_iam_user_attr(GET_USER_ATTRIBUTES) - okta_user = client.get_user(iam_attr, iam_attr_value) - if not okta_user: - error_code, error_message = IAMErrors.USER_DOES_NOT_EXIST - user_profile.set_result(action=IAMActions.GET_USER, success=False, error_code=error_code, error_message=error_message) - else: - user_profile.update_with_app_data(okta_user, mapper_in) - user_profile.set_result( - action=IAMActions.GET_USER, - success=True, - active=okta_user.get("status") != DEPROVISIONED_STATUS, - iden=okta_user.get("id"), - email=okta_user.get("profile", {}).get("email"), - username=okta_user.get("profile", {}).get("login"), - details=okta_user, - ) - - except Exception as e: - handle_exception(user_profile, e, IAMActions.GET_USER) - - return user_profile - - -def disable_user_command(client, args, is_command_enabled, mapper_out): - user_profile = IAMUserProfile( - user_profile=args.get("user-profile"), mapper=mapper_out, incident_type=IAMUserProfile.UPDATE_INCIDENT_TYPE - ) - okta_user = None - if not is_command_enabled: - user_profile.set_result(action=IAMActions.DISABLE_USER, skip=True, skip_reason="Command is disabled.") - else: - try: - iam_attr, iam_attr_value = user_profile.get_first_available_iam_user_attr(GET_USER_ATTRIBUTES) - okta_user = client.get_user(iam_attr, iam_attr_value) - if not okta_user: - _, error_message = IAMErrors.USER_DOES_NOT_EXIST - user_profile.set_result(action=IAMActions.DISABLE_USER, skip=True, skip_reason=error_message) - else: - client.deactivate_user(okta_user.get("id")) - user_profile.set_result( - action=IAMActions.DISABLE_USER, - success=True, - active=False, - iden=okta_user.get("id"), - email=okta_user.get("profile", {}).get("email"), - username=okta_user.get("profile", {}).get("login"), - details=okta_user, - ) - - except Exception as e: - handle_exception(user_profile, e, IAMActions.DISABLE_USER, okta_user) - - return user_profile - - -def create_user_command(client, args, mapper_out, is_command_enabled, is_update_user_enabled, is_enable_enabled): - user_profile = IAMUserProfile( - user_profile=args.get("user-profile"), mapper=mapper_out, incident_type=IAMUserProfile.CREATE_INCIDENT_TYPE - ) - if not is_command_enabled: - user_profile.set_result(action=IAMActions.CREATE_USER, skip=True, skip_reason="Command is disabled.") - else: - try: - iam_attr, iam_attr_value = user_profile.get_first_available_iam_user_attr(GET_USER_ATTRIBUTES) - okta_user = client.get_user(iam_attr, iam_attr_value) - if okta_user: - # if user exists, update its data - return update_user_command( - client, - args, - mapper_out, - is_update_user_enabled, - is_enable_enabled, - is_create_user_enabled=False, - create_if_not_exists=False, - ) - else: - okta_profile = user_profile.map_object(mapper_out, incident_type=IAMUserProfile.CREATE_INCIDENT_TYPE) - created_user = client.create_user(okta_profile) - client.activate_user(created_user.get("id")) - user_profile.set_result( - action=IAMActions.CREATE_USER, - success=True, - active=created_user.get("status") != DEPROVISIONED_STATUS, - iden=created_user.get("id"), - email=created_user.get("profile", {}).get("email"), - username=created_user.get("profile", {}).get("login"), - details=created_user, - ) - - except Exception as e: - handle_exception(user_profile, e, IAMActions.CREATE_USER) - - return user_profile - - -def update_user_command( - client, args, mapper_out, is_command_enabled, is_enable_enabled, is_create_user_enabled, create_if_not_exists -): - user_profile = IAMUserProfile( - user_profile=args.get("user-profile"), mapper=mapper_out, incident_type=IAMUserProfile.UPDATE_INCIDENT_TYPE - ) - allow_enable = args.get("allow-enable") == "true" - if not is_command_enabled: - user_profile.set_result(action=IAMActions.UPDATE_USER, skip=True, skip_reason="Command is disabled.") - else: - try: - iam_attr, iam_attr_value = user_profile.get_first_available_iam_user_attr(GET_USER_ATTRIBUTES, use_old_user_data=True) - okta_user = client.get_user(iam_attr, iam_attr_value) - if okta_user: - user_id = okta_user.get("id") - - if allow_enable and is_enable_enabled and okta_user.get("status") == DEPROVISIONED_STATUS: - client.activate_user(user_id) - user_profile.set_result( - action=IAMActions.ENABLE_USER, - success=True, - active=True, - iden=okta_user.get("id"), - email=okta_user.get("profile", {}).get("email"), - username=okta_user.get("profile", {}).get("login"), - details=okta_user, - ) - else: - okta_profile = user_profile.map_object(mapper_out, incident_type=IAMUserProfile.UPDATE_INCIDENT_TYPE) - updated_user = client.update_user(user_id, okta_profile) - user_profile.set_result( - action=IAMActions.UPDATE_USER, - success=True, - active=okta_user.get("status") != DEPROVISIONED_STATUS, - iden=updated_user.get("id"), - email=updated_user.get("profile", {}).get("email"), - username=updated_user.get("profile", {}).get("login"), - details=updated_user, - ) - else: - if create_if_not_exists: - return create_user_command(client, args, mapper_out, is_create_user_enabled, False, False) - else: - _, error_message = IAMErrors.USER_DOES_NOT_EXIST - user_profile.set_result(action=IAMActions.UPDATE_USER, skip=True, skip_reason=error_message) - - except Exception as e: - handle_exception(user_profile, e, IAMActions.UPDATE_USER) - - return user_profile - - -def get_app_user_assignment_command(client, args): - user_id = args.get("user_id") - application_id = args.get("application_id") - - res = client.get_app_user_assignment(application_id, user_id) - raw_response = res.json() - - is_user_assigned_to_app = res.status_code == 200 - - outputs = {"UserID": user_id, "AppID": application_id, "IsAssigned": is_user_assigned_to_app} - - readable_output = tableToMarkdown( - "App User Assignment", outputs, headers=["UserID", "AppID", "IsAssigned"], headerTransform=pascalToSpace - ) - - if is_user_assigned_to_app: - outputs["ProfileInApp"] = raw_response.get("profile") - profile_readable = tableToMarkdown("Profile in App", raw_response.get("profile"), removeNull=True) - readable_output += f"\n{profile_readable}" - - return CommandResults( - outputs=outputs, - outputs_prefix="Okta.AppUserAssignment", - outputs_key_field=["UserID", "AppID"], - readable_output=readable_output, - raw_response=raw_response, - ) - - -def list_apps_command(client, args): - query = args.get("query") - page = int(args.get("page")) - limit = min(int(args.get("limit")), 200) - - applications = client.list_apps(query, page, limit) - - outputs = [] - - for app in applications: - outputs.append( - { - "ID": app.get("id"), - "Name": app.get("name"), - "Label": app.get("label"), - "Logo": f"![]({app.get('_links', {}).get('logo', [{}])[0].get('href')})", - } - ) - - title = "Okta Applications" - - if applications: - from_idx = page * limit + 1 - to_idx = from_idx + len(applications) - 1 - title += f" ({from_idx} - {to_idx})" - - return CommandResults( - outputs=outputs, - outputs_prefix="Okta.Application", - outputs_key_field="ID", - readable_output=tableToMarkdown(title, outputs, headers=["ID", "Name", "Label", "Logo"]), - ) - - -def list_user_apps_command(client, args): - user_id = args.get("user_id") - applications = client.list_user_apps(user_id) - outputs = [] - - for app in applications: - outputs.append({"ID": app.get("id"), "Name": app.get("name"), "Label": app.get("label"), "Status": app.get("status")}) - - title = "Okta User Applications" - return CommandResults( - outputs=outputs, - outputs_prefix="Okta.Application", - outputs_key_field="ID", - readable_output=tableToMarkdown(title, outputs, headers=["ID", "Name", "Label", "Status"]), - ) - - -def get_configuration(context): - iam_configuration = context.get("IAMConfiguration", []) - - return CommandResults( - outputs=iam_configuration, - outputs_prefix="Okta.IAMConfiguration", - outputs_key_field="ApplicationID", - readable_output=tableToMarkdown("Okta IAM Configuration", iam_configuration), - ) - - -def set_configuration(args): - iam_configuration = json.loads(args.get("configuration")) - context = {"IAMConfiguration": iam_configuration} - return context - - -def fetch_incidents(client, last_run, first_fetch_str, fetch_limit, query_filter=None, auto_generate_filter=False, context=None): - """If no events were saved from last run, returns new events from Okta's /log API. Otherwise, - returns the events from last run. In both cases, no more than `fetch_limit` incidents will be returned, - and the rest of them will be saved for next run. - - Args: - client: (BaseClient) Okta client. - last_run: (dict) The "last run" object that was set on the previous run. - first_fetch_str: (str) First fetch time parameter (e.g. "1 day", "2 months", etc). - fetch_limit: (int) Maximum number of incidents to return. - query_filter: (str) Logs API query filter. - auto_generate_filter: (bool) Whether or not to automatically generate the query filter. - context: (dict) Integration Context object. - Returns: - incidents: (dict) Incidents/events that will be created in Cortex XSOAR - next_run: (dict) The "last run" object for the next run. - """ - - incidents = last_run.get("incidents", []) - last_run_full_url = last_run.get("last_run_full_url") - - first_fetch_date = dateparser.parse(first_fetch_str) - assert first_fetch_date is not None, f"could not parse {first_fetch_str}" - first_fetch = first_fetch_date.strftime(DATE_FORMAT) - last_run_time = last_run.get("last_run_time", first_fetch) # if last_run_time is undefined, use first_fetch - time_now = datetime.now().strftime(DATE_FORMAT) - - demisto.debug(f"Okta: Fetching logs from {last_run_time} to {time_now}.") - if not incidents: - email_to_user_profile = get_all_user_profiles() - - log_events, last_run_full_url = client.get_logs( - last_run_full_url, last_run_time, time_now, query_filter, auto_generate_filter, context - ) - for entry in log_events: - if not should_drop_event(entry, email_to_user_profile): - add_user_profile_data_to_entry(entry, email_to_user_profile) - incident = {"rawJSON": json.dumps(entry)} - incidents.append(incident) - - next_run = {"incidents": incidents[fetch_limit:], "last_run_time": time_now, "last_run_full_url": last_run_full_url} - - return incidents[:fetch_limit], next_run - - -class OutputContext: - """ - Class to build a generic output and context. - """ - - def __init__( - self, - success=None, - active=None, - id=None, - username=None, - email=None, - errorCode=None, - errorMessage=None, - details=None, - displayName=None, - members=None, - ): - self.instanceName = demisto.callingContext["context"]["IntegrationInstance"] - self.brand = demisto.callingContext["context"]["IntegrationBrand"] - self.command = demisto.command().replace("-", "_").title().replace("_", "") - self.success = success - self.active = active - self.id = id - self.username = username - self.email = email - self.errorCode = errorCode - self.errorMessage = errorMessage - self.details = details - self.displayName = displayName # Used in group - self.members = members # Used in group - self.data = { - "brand": self.brand, - "instanceName": self.instanceName, - "success": success, - "active": active, - "id": id, - "username": username, - "email": email, - "errorCode": errorCode, - "errorMessage": errorMessage, - "details": details, - "displayName": displayName, - "members": members, - } - # Remoove empty values - self.data = {k: v for k, v in self.data.items() if v is not None} - - -def get_group_command(client, args): - scim = safe_load_json(args.get("scim")) - group_id = scim.get("id") - group_name = scim.get("displayName") - - if not (group_id or group_name): - return_error("You must supply either 'id' or 'displayName' in the scim data") - - group_search_result = None - if not group_id: - res = client.search_group(group_name) - res_json = res.json() - - if res.status_code == 200: - if len(res_json) < 1: - generic_iam_context = OutputContext( - success=False, displayName=group_name, errorCode=404, errorMessage="Group Not Found", details=res_json - ) - else: - generic_iam_context = OutputContext() - group_search_result = res_json - else: - generic_iam_context = OutputContext( - success=False, - displayName=group_name, - id=group_id, - errorCode=res_json.get("errorCode"), - errorMessage=res_json.get("errorSummary"), - details=res_json, - ) - - if not group_search_result: - return CommandResults( - raw_response=generic_iam_context.data, - outputs_prefix=generic_iam_context.command, - outputs_key_field="id", - outputs=generic_iam_context.data, - readable_output=tableToMarkdown("Okta Get Group:", generic_iam_context.data, removeNull=True), - ) - - if group_search_result and len(group_search_result) > 1: - generic_iam_context_data_list = [] - - for group in group_search_result: - group_name = group.get("profile", {}).get("name") - generic_iam_context = OutputContext(success=True, id=group.get("id"), displayName=group_name) - generic_iam_context_data_list.append(generic_iam_context.data) - - return CommandResults( - raw_response=generic_iam_context_data_list, - outputs_prefix=generic_iam_context.command, - outputs_key_field="id", - outputs=generic_iam_context_data_list, - readable_output=tableToMarkdown("Okta Get Group:", generic_iam_context_data_list, removeNull=True), - ) - elif not group_id and isinstance(group_search_result, list): - group_id = group_search_result[0].get("id") - - res = client.get_group_by_id(group_id) - res_json = res.json() - if res.status_code == 200: - group_member_profiles = [] - include_members = args.get("includeMembers") - if include_members.lower() == "true": - group_members = client.get_group_members(group_id) - for member in group_members: - if member.get("status") != DEPROVISIONED_STATUS: - profile = member.get("profile", {}) - group_member_profile = {"value": member.get("id"), "display": profile.get("login")} - group_member_profiles.append(group_member_profile) - generic_iam_context = OutputContext( - success=True, - id=res_json.get("id"), - displayName=res_json.get("profile", {}).get("name"), - members=group_member_profiles, - ) - elif res.status_code == 404: - generic_iam_context = OutputContext( - success=False, displayName=group_name, id=group_id, errorCode=404, errorMessage="Group Not Found", details=res_json - ) - else: - generic_iam_context = OutputContext( - success=False, - displayName=group_name, - id=group_id, - errorCode=res_json.get("errorCode"), - errorMessage=res_json.get("errorSummary"), - details=res_json, - ) - - return CommandResults( - raw_response=generic_iam_context.data, - outputs_prefix=generic_iam_context.command, - outputs_key_field="id", - outputs=generic_iam_context.data, - readable_output=tableToMarkdown("Okta Get Group:", generic_iam_context.data, removeNull=True), - ) - - -def get_logs_command(client, args): - filter = args.get("filter") - since = args.get("since") - until = args.get("until") - limit = args.get("limit") - log_events, _ = client.get_logs(query_filter=filter, last_run_time=since, time_now=until, limit=limit) - - if not log_events: - return CommandResults(readable_output="No logs found.", outputs={}, raw_response=log_events) - - return CommandResults( - raw_response=log_events, - outputs_prefix="Okta.Logs.Events", - outputs_key_field="uuid", - outputs=log_events, - readable_output=tableToMarkdown("Okta Log Events:", log_events), - ) - - -def main(): - user_profile = None - params = demisto.params() - base_url = urljoin(params["url"].strip("/"), "/api/v1/") - token = params.get("credentials", {}).get("password", "") or params.get("apitoken", "") - - if not token: - raise ValueError("Missing API token.") - - mapper_in = params.get("mapper-in") - mapper_out = params.get("mapper-out") - verify_certificate = not params.get("insecure", False) - proxy = params.get("proxy", False) - command = demisto.command() - args = demisto.args() - - is_create_enabled = params.get("create-user-enabled") - is_enable_enabled = params.get("enable-user-enabled") - is_disable_enabled = params.get("disable-user-enabled") - is_update_enabled = demisto.params().get("update-user-enabled") - create_if_not_exists = demisto.params().get("create-if-not-exists") - - is_fetch = params.get("isFetch") - first_fetch_str = params.get("first_fetch") - fetch_limit = int(params.get("max_fetch", 1)) - auto_generate_query_filter = params.get("auto_generate_query_filter") - fetch_query_filter = params.get("fetch_query_filter") - context = demisto.getIntegrationContext() - - headers = {"Content-Type": "application/json", "Accept": "application/json", "Authorization": f"SSWS {token}"} - - client = Client(base_url=base_url, verify=verify_certificate, proxy=proxy, headers=headers, ok_codes=(200,)) - - demisto.debug(f"Command being called is {command}") - - if command == "iam-get-user": - user_profile = get_user_command(client, args, mapper_in, mapper_out) - - elif command == "iam-create-user": - user_profile = create_user_command(client, args, mapper_out, is_create_enabled, is_update_enabled, is_enable_enabled) - - elif command == "iam-update-user": - user_profile = update_user_command( - client, args, mapper_out, is_update_enabled, is_enable_enabled, is_create_enabled, create_if_not_exists - ) - - elif command == "iam-disable-user": - user_profile = disable_user_command(client, args, is_disable_enabled, mapper_out) - - if user_profile: - return_results(user_profile) - - try: - if command == "test-module": - test_module(client, is_fetch, fetch_query_filter, auto_generate_query_filter, context, first_fetch_str) - - elif command == "get-mapping-fields": - return_results(get_mapping_fields_command(client)) - - elif command == "okta-get-app-user-assignment": - return_results(get_app_user_assignment_command(client, args)) - - elif command == "okta-iam-list-applications": - return_results(list_apps_command(client, args)) - - elif command == "okta-iam-list-user-applications": - return_results(list_user_apps_command(client, args)) - - elif command == "okta-iam-get-configuration": - return_results(get_configuration(context)) - - elif command == "okta-iam-set-configuration": - context = set_configuration(args) - demisto.setIntegrationContext(context) - - elif command == "iam-get-group": - return_results(get_group_command(client, args)) - - elif command == "okta-get-logs": - return_results(get_logs_command(client, args)) - - elif command == "fetch-incidents": - last_run = demisto.getLastRun() - context = demisto.getIntegrationContext() - incidents, next_run = fetch_incidents( - client, last_run, first_fetch_str, fetch_limit, fetch_query_filter, auto_generate_query_filter, context - ) - demisto.incidents(incidents) - demisto.setLastRun(next_run) - - except Exception as e: - # For any other integration command exception, return an error - return_error(f"Failed to execute {command} command. Error: {e!s}") - - -from IAMApiModule import * # noqa: E402 +from CommonServerUserPython import * # noqa: E402 # pylint: disable=wrong-import-position if __name__ in ("__main__", "__builtin__", "builtins"): main() diff --git a/Packs/Okta/Integrations/Okta_IAM/Okta_IAM.yml b/Packs/Okta/Integrations/Okta_IAM/Okta_IAM.yml index e490e93af758..5323f200bc22 100644 --- a/Packs/Okta/Integrations/Okta_IAM/Okta_IAM.yml +++ b/Packs/Okta/Integrations/Okta_IAM/Okta_IAM.yml @@ -89,22 +89,34 @@ configuration: type: 0 section: Collect required: false + supportedModules: + - agentix + - xsiam - display: Fetch incidents name: isFetch type: 8 section: Collect required: false + supportedModules: + - agentix + - xsiam - defaultvalue: '1' display: Incidents Fetch Interval name: incidentFetchInterval type: 19 section: Collect required: false + supportedModules: + - agentix + - xsiam - display: Incident type name: incidentType type: 13 section: Connect required: false + supportedModules: + - agentix + - xsiam - additionalinfo: Configure an "IAM - Configuration" incident to specify the Okta apps you wish to sync to XSOAR. Further information about IAM configuration is detailed in the Help (the "?" icon). defaultvalue: 'true' display: Query only application events configured in IAM Configuration @@ -540,7 +552,7 @@ script: - contextPath: Okta.Logs.Events.target.displayName description: Display name of a target. type: String - dockerimage: demisto/python3:3.12.13.10116658 + dockerimage: demisto/python3:3.12.14.13053055 isfetch: true runonce: false script: '-' diff --git a/Packs/Okta/Integrations/Okta_IAM/Okta_IAM_test.py b/Packs/Okta/Integrations/Okta_IAM/Okta_IAM_test.py index 5c843c04780c..72c649949fb6 100644 --- a/Packs/Okta/Integrations/Okta_IAM/Okta_IAM_test.py +++ b/Packs/Okta/Integrations/Okta_IAM/Okta_IAM_test.py @@ -1,576 +1,49 @@ -import pytest -from CommonServerPython import EntryType -from IAMApiModule import * -from Okta_IAM import ( - Client, - create_user_command, - disable_user_command, - fetch_incidents, - get_app_user_assignment_command, - get_mapping_fields_command, - get_user_command, - update_user_command, -) -from requests import Response, Session - -OKTA_USER_OUTPUT = { - "id": "mock_id", - "status": "PROVISIONED", - "profile": { - "firstName": "mock_first_name", - "lastName": "mock_last_name", - "login": "testdemisto2@paloaltonetworks.com", - "email": "testdemisto2@paloaltonetworks.com", - }, -} - -OKTA_DISABLED_USER_OUTPUT = { - "id": "mock_id", - "status": "DEPROVISIONED", - "profile": { - "firstName": "mock_first_name", - "lastName": "mock_last_name", - "login": "testdemisto2@paloaltonetworks.com", - "email": "testdemisto2@paloaltonetworks.com", - }, -} - -BASE_URL = "https://test.com" - - -def mock_client(): - client = Client(base_url=BASE_URL) - return client - - -def get_outputs_from_user_profile(user_profile): - entry_context = user_profile.to_entry() - outputs = entry_context.get("Contents") - return outputs - - -@pytest.mark.parametrize( - "args, mock_url", - [ - ( - {"user-profile": {"email": "testdemisto2@paloaltonetworks.com"}}, - f'{BASE_URL}/users?filter=profile.email eq "testdemisto2@paloaltonetworks.com"', - ), - ( - {"user-profile": {"email": "testdemisto2@paloaltonetworks.com", "login": "testdemisto2@paloaltonetworks.com"}}, - f'{BASE_URL}/users?filter=profile.login eq "testdemisto2@paloaltonetworks.com"', - ), - ( - { - "user-profile": { - "email": "testdemisto2@paloaltonetworks.com", - "login": "testdemisto2@paloaltonetworks.com", - "id": "mock_id", - } - }, - f'{BASE_URL}/users?filter=id eq "mock_id"', - ), - ], -) -def test_get_user_command__existing_user(mocker, requests_mock, args, mock_url): - """ - Given: - - An Okta IAM client object - - A user-profile argument that contains an email of a user - When: - - The user exists in Okta - - Calling function get_user_command - Then: - - Ensure the resulted User Profile object holds the correct user details - """ - client = mock_client() - - requests_mock.get(mock_url, json=[OKTA_USER_OUTPUT]) - mocker.patch.object(IAMUserProfile, "update_with_app_data", return_value={}) - - user_profile = get_user_command(client, args, "mapper_in", "mapper_out") - outputs = get_outputs_from_user_profile(user_profile) - - assert outputs.get("action") == IAMActions.GET_USER - assert outputs.get("success") is True - assert outputs.get("active") is True - assert outputs.get("id") == "mock_id" - assert outputs.get("username") == "testdemisto2@paloaltonetworks.com" - assert outputs.get("details", {}).get("profile", {}).get("firstName") == "mock_first_name" - assert outputs.get("details", {}).get("profile", {}).get("lastName") == "mock_last_name" - - -def test_get_user_command__non_existing_user(mocker): - """ - Given: - - An Okta IAM client object - - A user-profile argument that contains an email a user - When: - - The user does not exist in Okta - - Calling function get_user_command - Then: - - Ensure the resulted User Profile object holds information about an unsuccessful result. - """ - client = mock_client() - args = {"user-profile": {"email": "testdemisto2@paloaltonetworks.com"}} - - mocker.patch.object(client, "get_user", return_value=None) - - user_profile = get_user_command(client, args, "mapper_in", "mapper_out") - outputs = get_outputs_from_user_profile(user_profile) - - assert outputs.get("action") == IAMActions.GET_USER - assert outputs.get("success") is False - assert outputs.get("errorCode") == IAMErrors.USER_DOES_NOT_EXIST[0] - assert outputs.get("errorMessage") == IAMErrors.USER_DOES_NOT_EXIST[1] - - -def test_get_user_command__bad_response(mocker): - """ - Given: - - An Okta IAM client object - - A user-profile argument that contains an email of a non-existing user in Okta - When: - - Calling function get_user_command - - A bad response (500) is returned from Okta's API - Then: - - Ensure the resulted User Profile object holds information about the bad response. - """ - import demistomock as demisto - - client = mock_client() - args = {"user-profile": {"email": "testdemisto2@paloaltonetworks.com"}} - - bad_response = Response() - bad_response.status_code = 500 - bad_response._content = ( - b'{"errorCode": "mock_error_code", ' - b'"errorSummary": "mock_error_summary", ' - b'"errorCauses": [{"errorSummary": "reason_1"}, ' - b'{"errorSummary": "reason_2"}]}' - ) - - mocker.patch.object(demisto, "error") - mocker.patch.object(Session, "request", return_value=bad_response) - - user_profile = get_user_command(client, args, "mapper_in", "mapper_out") - outputs = get_outputs_from_user_profile(user_profile) - - assert outputs.get("action") == IAMActions.GET_USER - assert outputs.get("success") is False - assert outputs.get("errorCode") == "mock_error_code" - assert outputs.get("errorMessage") == "mock_error_summary. Reason:\n1. reason_1\n2. reason_2\n" - - -def test_create_user_command__success(mocker): - """ - Given: - - An Okta IAM client object - - A user-profile argument that contains an email of a non-existing user in Okta - When: - - Calling function create_user_command - Then: - - Ensure a User Profile object with the user data is returned - """ - client = mock_client() - args = {"user-profile": {"email": "testdemisto2@paloaltonetworks.com"}} - - mocker.patch.object(client, "get_user", return_value=None) - mocker.patch.object(client, "create_user", return_value=OKTA_USER_OUTPUT) - mocker.patch.object(client, "activate_user", return_value=None) - - user_profile = create_user_command( - client, args, "mapper_out", is_command_enabled=True, is_update_user_enabled=True, is_enable_enabled=True - ) - outputs = get_outputs_from_user_profile(user_profile) - - assert outputs.get("action") == IAMActions.CREATE_USER - assert outputs.get("success") is True - assert outputs.get("active") is True - assert outputs.get("id") == "mock_id" - assert outputs.get("username") == "testdemisto2@paloaltonetworks.com" - assert outputs.get("details", {}).get("profile", {}).get("firstName") == "mock_first_name" - assert outputs.get("details", {}).get("profile", {}).get("lastName") == "mock_last_name" - - -def test_update_user_command__allow_enable(mocker): - """ - Given: - - An Okta IAM client object - - A user-profile argument that contains user data - When: - - The user is disabled in Okta - - allow-enable argument is true - - Calling function update_user_command - Then: - - Ensure the user is enabled at the end of the command execution. - """ - client = mock_client() - args = { - "user-profile": {"email": "testdemisto2@paloaltonetworks.com", "givenname": "mock_first_name"}, - "allow-enable": "true", - } - - mocker.patch.object(client, "get_user", return_value=OKTA_DISABLED_USER_OUTPUT) - mocker.patch.object(client, "activate_user", return_value=None) - - user_profile = update_user_command( - client, - args, - "mapper_out", - is_command_enabled=True, - is_enable_enabled=True, - is_create_user_enabled=False, - create_if_not_exists=False, - ) - outputs = get_outputs_from_user_profile(user_profile) - - assert outputs.get("action") == IAMActions.ENABLE_USER - assert outputs.get("success") is True - assert outputs.get("active") is True - assert outputs.get("id") == "mock_id" - assert outputs.get("username") == "testdemisto2@paloaltonetworks.com" - assert outputs.get("details", {}).get("profile", {}).get("firstName") == "mock_first_name" - assert outputs.get("details", {}).get("profile", {}).get("lastName") == "mock_last_name" - - -def test_update_user_command__non_existing_user(mocker): - """ - Given: - - An Okta IAM client object - - A user-profile argument that contains user data - When: - - The user does not exist in Okta - - create-if-not-exists parameter is checked - - Create User command is enabled - - Calling function update_user_command - Then: - - Ensure the create action is executed - - Ensure a User Profile object with the user data is returned - """ - client = mock_client() - args = {"user-profile": {"email": "testdemisto2@paloaltonetworks.com", "givenname": "mock_first_name"}} - - mocker.patch.object(client, "get_user", return_value=None) - mocker.patch.object(client, "create_user", return_value=OKTA_USER_OUTPUT) - mocker.patch.object(client, "activate_user", return_value=None) - - user_profile = update_user_command( - client, - args, - "mapper_out", - is_command_enabled=True, - is_enable_enabled=True, - is_create_user_enabled=True, - create_if_not_exists=True, - ) - outputs = get_outputs_from_user_profile(user_profile) - - assert outputs.get("action") == IAMActions.CREATE_USER - assert outputs.get("success") is True - assert outputs.get("active") is True - assert outputs.get("id") == "mock_id" - assert outputs.get("username") == "testdemisto2@paloaltonetworks.com" - assert outputs.get("details", {}).get("profile", {}).get("firstName") == "mock_first_name" - assert outputs.get("details", {}).get("profile", {}).get("lastName") == "mock_last_name" - - -def test_update_user_command__command_is_disabled(mocker): - """ - Given: - - An Okta IAM client object - - A user-profile argument that contains user data - When: - - Update User command is disabled - - Calling function update_user_command - Then: - - Ensure the command is considered successful and skipped - """ - client = mock_client() - args = {"user-profile": {"email": "testdemisto2@paloaltonetworks.com", "givenname": "mock_first_name"}} - - mocker.patch.object(client, "get_user", return_value=None) - mocker.patch.object(IAMUserProfile, "map_object", return_value={}) - mocker.patch.object(client, "update_user", return_value=OKTA_USER_OUTPUT) +"""Sanity tests for the Okta IAM shim integration. - user_profile = update_user_command( - client, - args, - "mapper_out", - is_command_enabled=False, - is_enable_enabled=True, - is_create_user_enabled=False, - create_if_not_exists=False, - ) - outputs = get_outputs_from_user_profile(user_profile) +Full behavioural coverage lives in the OktaIAMApiModule tests; this file +exists only to confirm the shim wires through to the ApiModule correctly. +""" - assert outputs.get("action") == IAMActions.UPDATE_USER - assert outputs.get("success") is True - assert outputs.get("skipped") is True - assert outputs.get("reason") == "Command is disabled." - - -def test_update_user_command__rate_limit_error(mocker): - """ - Given: - - An Okta IAM client object - - A user-profile argument - When: - - Calling function update_user_command - - API call exceeded rate limit - Then: - - Ensure an error entry is returned, as rate limit error code is in ERROR_CODES_TO_RETURN_ERROR list. - """ - import demistomock as demisto - - client = mock_client() - args = {"user-profile": {"email": "testdemisto2@paloaltonetworks.com", "givenname": "mock_first_name"}} - - bad_response = Response() - bad_response.status_code = 429 - bad_response._content = b'{"errorCode": "E0000047", "errorSummary": "API call exceeded rate limit due to too many requests."}' - - mocker.patch.object(demisto, "error") - mocker.patch.object(Session, "request", return_value=bad_response) - - user_profile = update_user_command( - client, - args, - "mapper_out", - is_command_enabled=True, - is_enable_enabled=True, - is_create_user_enabled=False, - create_if_not_exists=False, - ) - - entry_context = user_profile.to_entry() - outputs = entry_context.get("Contents") - - assert entry_context.get("Type") == EntryType.ERROR - assert outputs.get("action") == IAMActions.UPDATE_USER - assert outputs.get("success") is False - assert outputs.get("errorCode") == "E0000047" - - -def test_disable_user_command__user_is_already_disabled(mocker): - """ - Given: - - An Okta IAM client object - - A user-profile argument that contains an email of a user - When: - - The user is already disabled in Okta - - Calling function disable_user_command - Then: - - Ensure the command is considered successful and skipped - """ - import demistomock as demisto - - client = mock_client() - args = {"user-profile": {"email": "testdemisto2@paloaltonetworks.com"}} - - bad_response = Response() - bad_response.status_code = 400 - bad_response._content = ( - b'{"errorCode": "E0000007", ' - b'"errorSummary": "mock_error_summary", ' - b'"errorCauses": [{"errorSummary": "reason_1"}, ' - b'{"errorSummary": "reason_2"}]}' - ) - - mocker.patch.object(demisto, "error") - mocker.patch.object(Session, "request", return_value=bad_response) - - user_profile = disable_user_command(client, args, is_command_enabled=True, mapper_out="mapper_out") - outputs = get_outputs_from_user_profile(user_profile) - - assert outputs.get("action") == IAMActions.DISABLE_USER - assert outputs.get("success") is True - assert outputs.get("skipped") is True - - -def test_get_mapping_fields_command(mocker): - """ - Given: - - An Okta IAM client object - When: - - Okta user schema contains the fields 'field1' and 'field2' - - Calling function get_mapping_fields_command - Then: - - Ensure a GetMappingFieldsResponse object that contains the Okta fields is returned - """ - client = mock_client() - - mocker.patch.object(client, "get_okta_fields", return_value={"field1": "description1", "field2": "description2"}) - - mapping_response = get_mapping_fields_command(client) - mapping = mapping_response.extract_mapping() - - assert mapping.get(IAMUserProfile.DEFAULT_INCIDENT_TYPE, {}).get("field1") == "description1" - assert mapping.get(IAMUserProfile.DEFAULT_INCIDENT_TYPE, {}).get("field2") == "description2" - - -def test_get_app_user_assignment_command(mocker): - """ - Given: - - An Okta IAM client object - - Okta User ID - - Okta Application ID - When: - - Calling function get_assigned_user_for_app_command - Then: - - Ensure a User Assignment object to the application is retrieved in the correct format. - """ - client = mock_client() - - args = {"user_id": "mock_user_id", "application_id": "mock_app_id"} - - get_assignment_response = Response() - get_assignment_response.status_code = 200 - get_assignment_response._content = ( - b'{"id": "mock_user_id", ' - b'"profile": {}, ' - b'"created": "2020-11-03T09:59:30.000Z", ' - b'"credentials": {"userName": "mock_username"}, ' - b'"externalId": null, ' - b'"status": "ACTIVE"}' - ) - - mocker.patch.object(Session, "request", return_value=get_assignment_response) - - command_result = get_app_user_assignment_command(client, args) - - assert command_result.outputs.get("UserID") == "mock_user_id" - assert command_result.outputs.get("AppID") == "mock_app_id" - assert command_result.outputs.get("IsAssigned") is True - - -def test_fetch_incidents__two_logs_batches(mocker): - """ - Given: - - An Okta IAM client object and fetch-relevant instance parameters - When: - - Calling function fetch_incidents - - Events should come in two batches of two events in the first batch, and one event in the second batch. - Then: - - Ensure three events are returned in incident the correct format. - """ - import json - - mocker.patch.object(Client, "get_logs_batch", side_effect=mock_get_logs_batch) - mocker.patch("Okta_IAM.get_all_user_profiles", return_value={}) - events, _ = fetch_incidents( - client=mock_client(), last_run={}, query_filter="mock_query_filter", first_fetch_str="7 days", fetch_limit=5 - ) - - assert len(events) == 3 - assert json.loads(events[0]["rawJSON"]).get("mock_log1") == "mock_value1" - assert json.loads(events[1]["rawJSON"]).get("mock_log2") == "mock_value2" - assert json.loads(events[2]["rawJSON"]).get("mock_log3") == "mock_value3" - - -def test_fetch_incidents__fetch_limit(mocker): - """ - Given: - - An Okta IAM client object and fetch-relevant instance parameters - When: - - Calling function fetch_incidents - - Three events exist Okta logs. - - Fetch limit is 2. - Then: - - Ensure only two events are returned in incident the correct format. - """ - mocker.patch.object(Client, "get_logs_batch", side_effect=mock_get_logs_batch) - mocker.patch("Okta_IAM.get_all_user_profiles", return_value={}) - events, _ = fetch_incidents( - client=mock_client(), last_run={}, query_filter="mock_query_filter", first_fetch_str="7 days", fetch_limit=2 - ) - - assert len(events) == 2 - - -def test_fetch_incidents__last_run(mocker): - """ - Given: - - An Okta IAM client object and fetch-relevant instance parameters - - Last run object contains three incidents. - When: - - Calling function fetch_incidents - - Fetch Limit is 2. - Then: - - Ensure only the first two incidents from the last run are retrieved. - - Ensure that the next_run object returned contains the third incident. - - Ensure 'last_run_time' key exists and holds a datetime string in the correct format. - """ - from datetime import datetime - - last_run = {"incidents": [{"mock_log1": "mock_value1"}, {"mock_log2": "mock_value2"}, {"mock_log3": "mock_value3"}]} - mocker.patch("Okta_IAM.get_all_user_profiles", return_value={}) - - events, next_run = fetch_incidents( - client=mock_client(), last_run=last_run, query_filter="mock_query_filter", first_fetch_str="7 days", fetch_limit=2 - ) - - last_run_time = datetime.strptime(next_run.get("last_run_time"), "%Y-%m-%dT%H:%M:%SZ") - - assert len(events) == 2 - assert len(next_run.get("incidents")) == 1 - assert next_run["incidents"][0].get("mock_log3") == "mock_value3" - assert isinstance(last_run_time, datetime) - - -LOGS = [{"mock_log1": "mock_value1"}, {"mock_log2": "mock_value2"}, {"mock_log3": "mock_value3"}] +from unittest.mock import patch # noqa: F401 +import pytest +import Okta_IAM as integration_module -def mock_get_logs_batch(url_suffix="", params=None, full_url=""): - if url_suffix: - # first iteration - return LOGS[:2], "mock_next_page" - elif full_url: - # second iteration - return LOGS[2:], None +def test_shim_imports_run_entry_point(): + """The shim must re-export the ApiModule entry point under its expected name.""" + assert hasattr( + integration_module, "run_okta_iam_integration" + ), "OktaIAMApiModule.run_okta_iam_integration must be importable via the shim" - # third iteration - nothing is returned - return None, None +def test_shim_imports_client(): + """The shim must re-export the Okta IAM Client class.""" + assert hasattr(integration_module, "Client") -SHOULD_DROP_EVENT_ARGS = [ - # no user email in log entry - do not drop event - ({"target": [{"type": "Group", "alternateId": "testGroupId"}]}, {}, False), - # no user email in log entry - do not drop event - ({"target": [{"type": "Group", "alternateId": "testGroupId"}]}, {"test@example.com": {"username": "test"}}, False), - # user email in both log entry and in xsoar - do not drop event - ({"target": [{"type": "User", "alternateId": "test@example.com"}]}, {"test@example.com": {"username": "test"}}, False), - # user email in log entry but not in xsoar - drop event - ({"target": [{"type": "User", "alternateId": "test@example.com"}]}, {}, True), -] +def test_shim_imports_iam_api_module_symbols(): + """OktaIAMApiModule layers on IAMApiModule, so its primitives must reach the shim.""" + for name in ("IAMUserProfile", "IAMActions", "IAMErrors"): + assert hasattr(integration_module, name), f"Symbol {name!r} missing from shim" -@pytest.mark.parametrize("log_entry, email_to_user_profile, expected", SHOULD_DROP_EVENT_ARGS) -def test_should_drop_event(log_entry, email_to_user_profile, expected): - from Okta_IAM import should_drop_event - assert should_drop_event(log_entry, email_to_user_profile) == expected +def test_shim_imports_commands_and_constants(): + """Commands and constants used by playbooks/tests must remain accessible from the shim.""" + for name in ("get_user_command", "create_user_command", "fetch_incidents", "DEPROVISIONED_STATUS"): + assert hasattr(integration_module, name), f"Symbol {name!r} missing from shim" -LOGS_WITH_LIMIT = [(None, 3), (1, 1), (3, 3), (1001, 3)] +def test_main_delegates_to_api_module(mocker): + """`main()` must do nothing other than call the renamed ApiModule entry point.""" + mock_run = mocker.patch("Okta_IAM.run_okta_iam_integration") + integration_module.main() + mock_run.assert_called_once_with() -@pytest.mark.parametrize("limit, logs_amount", LOGS_WITH_LIMIT) -def test_get_logs_command(mocker, requests_mock, limit, logs_amount): - """ - Given: - - An Okta IAM client object. - When: - - Calling function okta-get-logs - - Events should come in two batches of two events in the first batch, and one event in the second batch. - Then: - - Ensure three events are returned in incident the correct format. - """ - from Okta_IAM import get_logs_command +def test_main_propagates_exceptions(mocker): + """`main()` is a thin shim and must not swallow exceptions from the ApiModule.""" + mocker.patch("Okta_IAM.run_okta_iam_integration", side_effect=RuntimeError("boom")) - mocker.patch.object(Client, "get_logs_batch", side_effect=mock_get_logs_batch) - requests_mock.get(f"{BASE_URL}/logs?limit={limit}", json=LOGS[:limit]) - args = {"limit": limit} - results = get_logs_command(client=mock_client(), args=args) - assert len(results.outputs) == logs_amount + with pytest.raises(RuntimeError, match="boom"): + integration_module.main() diff --git a/Packs/Okta/Integrations/Okta_v2/Okta_v2.py b/Packs/Okta/Integrations/Okta_v2/Okta_v2.py index c8dc3ead694d..7885394c8e17 100644 --- a/Packs/Okta/Integrations/Okta_v2/Okta_v2.py +++ b/Packs/Okta/Integrations/Okta_v2/Okta_v2.py @@ -1,1336 +1,14 @@ -from urllib.parse import parse_qs, urlencode, urlparse, urlunparse - import demistomock as demisto # noqa: F401 from CommonServerPython import * # noqa: F401 -from OktaApiModule import * # noqa: E402 - -# CONSTANTS -DATE_FORMAT = "%Y-%m-%dT%H:%M:%SZ" -SEARCH_LIMIT = 200 -OAUTH_TOKEN_SCOPES = [ # Scopes to request when generating an OAuth token - "okta.apps.manage", - "okta.apps.read", - "okta.groups.manage", - "okta.groups.read", - "okta.logs.read", - "okta.networkZones.manage", - "okta.networkZones.read", - "okta.sessions.manage", - "okta.sessions.read", - "okta.users.manage", - "okta.users.read", -] -PROFILE_ARGS = [ - "firstName", - "lastName", - "email", - "login", - "secondEmail", - "middleName", - "honorificPrefix", - "honorificSuffix", - "title", - "displayName", - "nickName", - "profileUrl", - "primaryPhone", - "mobilePhone", - "streetAddress", - "city", - "state", - "zipCode", - "countryCode", - "postalAddress", - "preferredLanguage", - "locale", - "timezone", - "userType", - "employeeNumber", - "costCenter", - "organization", - "division", - "department", - "managerId", - "manager", -] -GROUP_PROFILE_ARGS = ["name", "description"] - -MAX_LOGS_LIMIT = 1000 -DEFAULT_POLLING_TIME = 5 -DEFAULT_MAX_POLLING_CALLS = 10 - - -class Client(OktaClient): - # Getting Group Id with a given group name - def get_group_id(self, group_name): - uri = "/api/v1/groups" - query_params = {"q": encode_string_results(group_name)} - res = self.http_request(method="GET", url_suffix=uri, params=query_params) - return next((r.get("id") for r in (res or []) if r.get("profile", {}).get("name") == group_name), None) - - def get_app_id(self, app_name): - uri = "/api/v1/apps" - query_params = {"q": encode_string_results(app_name)} - res = self.http_request(method="GET", url_suffix=uri, params=query_params) - if res and len(res) == 1: - return res[0].get("id") - return None - - # Getting User Id with a given username - def get_user_id(self, username): - uri = "/api/v1/users" - query_params = {"filter": encode_string_results(f'profile.login eq "{username}"')} - res = self.http_request(method="GET", url_suffix=uri, params=query_params) - if res and len(res) == 1: - return res[0].get("id") - raise Exception(f"Failed to find userID for: {username} username.") - - def unlock_user(self, user_id): - """ - sending a POST request to unlock a specific user - """ - uri = f"/api/v1/users/{user_id}/lifecycle/unlock" - return self.http_request(method="POST", url_suffix=uri) - - def deactivate_user(self, user_id): - uri = f"/api/v1/users/{user_id}/lifecycle/deactivate" - return self.http_request(method="POST", url_suffix=uri) - - def activate_user(self, user_id): - uri = f"/api/v1/users/{user_id}/lifecycle/activate" - return self.http_request(method="POST", url_suffix=uri) - - def suspend_user(self, user_id): - uri = f"/api/v1/users/{user_id}/lifecycle/suspend" - return self.http_request(method="POST", url_suffix=uri) - - def unsuspend_user(self, user_id): - uri = f"/api/v1/users/{user_id}/lifecycle/unsuspend" - return self.http_request(method="POST", url_suffix=uri) - - def get_user_factors(self, user_id): - uri = f"/api/v1/users/{user_id}/factors" - return self.http_request(method="GET", url_suffix=uri) - - def reset_factor(self, user_id, factor_id): - uri = f"/api/v1/users/{user_id}/factors/{factor_id}" - return self.http_request(method="DELETE", url_suffix=uri, resp_type="text") - - def set_password(self, user_id, password): - uri = f"/api/v1/users/{user_id}" - body = {"credentials": {"password": {"value": password}}} - - return self.http_request(method="POST", url_suffix=uri, json_data=body) - - def revoke_session(self, user_id): - uri = f"/api/v1/users/{user_id}/lifecycle/expire_password_with_temp_password" - params = {"revokeSessions": "true"} - return self.http_request(method="POST", url_suffix=uri, params=params) - - def expire_password(self, user_id, args): - uri = f"/api/v1/users/{user_id}/lifecycle/expire_password" - params = {"tempPassword": args.get("temporary_password", "false")} - return self.http_request(method="POST", url_suffix=uri, params=params) - - def add_user_to_group(self, user_id, group_id): - uri = f"/api/v1/groups/{group_id}/users/{user_id}" - return self.http_request(method="PUT", url_suffix=uri, resp_type="text") - - def remove_user_from_group(self, user_id, group_id): - uri = f"/api/v1/groups/{group_id}/users/{user_id}" - return self.http_request(method="DELETE", url_suffix=uri, resp_type="text") - - def get_groups_for_user(self, user_id): - uri = f"/api/v1/users/{user_id}/groups" - return self.http_request(method="GET", url_suffix=uri) - - @staticmethod - def get_readable_groups(raw_groups): - groups = [] - raw_groups = raw_groups if isinstance(raw_groups, list) else [raw_groups] - for group in raw_groups: - group = { - "ID": group.get("id"), - "Created": group.get("created"), - "ObjectClass": group.get("objectClass"), - "LastUpdated": group.get("lastUpdated"), - "LastMembershipUpdated": group.get("lastMembershipUpdated"), - "Type": group.get("type"), - "Name": group.get("profile", {}).get("name"), - "Description": group.get("profile", {}).get("description"), - } - groups.append(group) - return groups - - @staticmethod - def get_readable_logs(raw_logs): - logs = [] - browser = "" - device = "" - os = "" - raw_logs = raw_logs if isinstance(raw_logs, list) else [raw_logs] - for log in raw_logs: - if log.get("client", {}).get("userAgent"): - browser = log.get("client", {}).get("userAgent").get("browser") - if (not browser) or browser.lower() == "unknown": - browser = "Unknown browser" - os = log.get("client", {}).get("userAgent").get("os") - if (not os) or os.lower() == "unknown": - os = "Unknown OS" - device = log.get("client", {}).get("device") - if (not device) or device.lower() == "unknown": - device = "Unknown device" - targets = "" - if log.get("target"): - for target in log.get("target"): - targets += f"{target.get('displayName')} ({target.get('type')})\n" - time_published = datetime.strptime(log.get("published"), "%Y-%m-%dT%H:%M:%S.%f%z").strftime("%m/%d/%Y, %H:%M:%S") - log = { - "Actor": f"{log.get('actor', {}).get('displayName')} ({log.get('actor', {}).get('type')})", - "ActorAlternaneId": log.get("actor", {}).get("alternateId"), - "EventInfo": log.get("displayMessage"), - "EventOutcome": log.get("outcome", {}).get("result") - + (f": {log.get('outcome', {}).get('reason')}" if log.get("outcome", {}).get("reason") else ""), - "EventSeverity": log.get("severity"), - "Client": f"{browser} on {os} {device}" if browser else "Unknown client", - "RequestIP": log.get("client", {}).get("ipAddress"), - "ChainIP": [ip_chain.get("ip") for ip_chain in log.get("request", {}).get("ipChain", [])], - "Targets": targets or "-", - "Time": time_published, - } - logs.append(log) - return logs - - @staticmethod - def get_readable_factors(raw_factors): - factors = [] - for factor in raw_factors: - factor = { - "ID": factor.get("id"), - "FactorType": factor.get("factorType"), - "Provider": factor.get("provider"), - "Status": factor.get("status"), - "Profile": factor.get("profile"), - } - factors.append(factor) - return factors - - def verify_push_factor(self, user_id, factor_id): - """ - Creates a new transaction and sends an asynchronous push notification to the device for the user to approve or reject. - You must poll the transaction to determine when it completes or expires. - """ - uri = f"/api/v1/users/{user_id}/factors/{factor_id}/verify" - return self.http_request(method="POST", url_suffix=uri) - - def poll_verify_push(self, url, polling_time: int = DEFAULT_POLLING_TIME, max_polling_calls: int = DEFAULT_MAX_POLLING_CALLS): - """ - Keep polling authentication transactions with WAITING result until the challenge completes or expires. - time limit defined by us = one minute - """ - counter = 0 - while counter < max_polling_calls: - response = self.http_request(method="GET", full_url=url, url_suffix="") - if response.get("factorResult") != "WAITING": - return response - counter += 1 - time.sleep(polling_time) # pylint: disable=E9003 - response["factorResult"] = "TIMEOUT" - return response - - def get_push_factor_status(self, polling_url: str): - """Get the status of the push factor""" - response = self.http_request(method="GET", full_url=polling_url, url_suffix="") - return response - - def search(self, term, limit, advanced_search): - uri = "/api/v1/users" - query_params = assign_params(limit=limit, q=encode_string_results(term), search=encode_string_results(advanced_search)) - return self.http_request(method="GET", url_suffix=uri, params=query_params) - - @staticmethod - def get_users_context(raw_users): - users = [] - raw_users = raw_users if isinstance(raw_users, list) else [raw_users] - for user in raw_users: - user = { - "ID": user.get("id"), - "Username": user.get("profile", {}).get("login"), - "DisplayName": f"{user.get('profile', {}).get('firstName', '')} {user.get('profile', {}).get('lastName', '')}", - "Email": user.get("profile", {}).get("email"), - "Status": user.get("status"), - "Type": "Okta", - "Created": user.get("created"), - "Activated": user.get("activated"), - "StatusChanged": user.get("statusChanged"), - "PasswordChanged": user.get("passwordChanged"), - "Manager": user.get("profile", {}).get("manager"), - "ManagerEmail": user.get("profile", {}).get("managerEmail"), - } - if user.get("group"): - user["Group"] = user.get("group") - users.append(user) - return users - - @staticmethod - def get_apps_context(raw_apps): - apps = [] - raw_apps = raw_apps if isinstance(raw_apps, list) else [raw_apps] - for app in raw_apps: - app = { - "ID": app.get("id"), - } - apps.append(app) - return apps - - @staticmethod - def get_groups_context(raw_groups): - groups = [] - raw_groups = raw_groups if isinstance(raw_groups, list) else [raw_groups] - for group in raw_groups: - group = { - "ID": group.get("id"), - "Name": group.get("profile", {}).get("name"), - "Description": group.get("profile", {}).get("description"), - "Type": group.get("type"), - } - groups.append(group) - return groups - - @staticmethod - def get_readable_users(raw_users, verbose="false"): - raw_users = raw_users if isinstance(raw_users, list) else [raw_users] - if verbose == "true": - users_verbose = "" - for user in raw_users: - profile = { - "First Name": user.get("profile", {}).get("firstName"), - "Last Name": user.get("profile", {}).get("lastName"), - "Mobile Phone": user.get("profile", {}).get("mobilePhone"), - "Login": user.get("profile", {}).get("login"), - "Email": user.get("profile", {}).get("email"), - "Second Email": user.get("profile", {}).get("secondEmail"), - "Manager": user.get("profile", {}).get("manager"), - "Manager Email": user.get("profile", {}).get("managerEmail"), - } - additionalData = { - "ID": user.get("id"), - "Status": user.get("status"), - "Created": user.get("created"), - "Activated": user.get("activated"), - "Status Changed": user.get("userChanged"), - "Last Login": user.get("lastLogin"), - "Last Updated": user.get("lastUpdated"), - "Password Changed": user.get("passwordChanged"), - "Type": user.get("type"), - "Credentials": user.get("credentials"), - "_links": user.get("_links"), - } - if user.get("group"): - additionalData["Group"] = user.get("group") - users_verbose += ( - f"### User:{profile.get('Login')}\n" - f"{tableToMarkdown('Profile', profile)}\n {tableToMarkdown('Additional Data', additionalData)}" - ) - return users_verbose - - else: - users = [] - for user in raw_users: - user = { - "ID": user.get("id"), - "Login": user.get("profile").get("login"), - "First Name": user.get("profile").get("firstName"), - "Last Name": user.get("profile").get("lastName"), - "Mobile Phone": user.get("profile").get("mobilePhone"), - "Last Login": user.get("lastLogin"), - "Status": user.get("status"), - "Manager": user.get("profile", {}).get("manager"), - "Manager Email": user.get("profile", {}).get("managerEmail"), - } - users.append(user) - return users - - def get_user(self, user_term): - uri = f"/api/v1/users/{encode_string_results(user_term)}" - return self.http_request(method="GET", url_suffix=uri) - - def create_user(self, cred, profile, group_ids, activate): - body = {"profile": profile, "groupIds": group_ids or [], "credentials": cred} - uri = "/api/v1/users" - query_params = {"activate": activate, "provider": "true" if cred.get("provider") else None} - return self.http_request(method="POST", url_suffix=uri, json_data=body, params=query_params) - - def create_group(self, profile): - body = { - "profile": profile, - } - uri = "/api/v1/groups" - return self.http_request(method="POST", url_suffix=uri, json_data=body) - - # Build profile dict with pre-defined keys (for user) - @staticmethod - def build_profile(args): - profile = {} - keys = args.keys() - for key in PROFILE_ARGS: - if key in keys: - profile[key] = args[key] - return profile - - # Build profile dict with pre-defined keys (for group) - @staticmethod - def build_group_profile(args): - profile = {} - keys = args.keys() - for key in GROUP_PROFILE_ARGS: - if key in keys: - profile[key] = args[key] - return profile - - # Build credentials dict with predefined keys (for user) - @staticmethod - def build_credentials(args): - cred = {} - if args.get("password"): - cred["password"] = {"value": args.get("password")} - if args.get("passwordQuestion") and args.get("passwordAnswer"): - cred["recovery_question"] = {"question": args.get("passwordQuestion"), "answer": args.get("passwordAnswer")} - if args.get("providerName") and args.get("providerType"): - cred["provider"] = {"name": args.get("providerName"), "type": args.get("providerType")} - return cred - - def update_user(self, user_id, profile, cred): - body = {"profile": profile, "credentials": cred} - uri = f"/api/v1/users/{user_id}" - return self.http_request(method="POST", url_suffix=uri, json_data=body) - - def get_paged_results(self, uri, query_param=None, max_limit=None): - response = self.http_request(method="GET", url_suffix=uri, resp_type="response", params=query_param) - paged_results = response.json() - while "next" in response.links and len(response.json()) > 0: - next_page = response.links.get("next").get("url") - response = self.http_request( - method="GET", full_url=next_page, url_suffix="", resp_type="response", params=query_param - ) - paged_results += response.json() - if max_limit and len(paged_results) >= max_limit: - return paged_results[:max_limit] - return paged_results - - def get_group_members(self, group_id, limit): - uri = f"/api/v1/groups/{group_id}/users" - if limit: - query_params = {"limit": limit} - return self.http_request(method="GET", url_suffix=uri, params=query_params) - return self.get_paged_results(uri) - - def list_users(self, args): - # Base url - if none of the above specified - returns all the users (default 200 items) - uri = "/api/v1/users" - query_params = {} - for key, value in args.items(): - if key == "query": - key = "q" - query_params[key] = encode_string_results(value) - limit = int(args.get("limit")) - response = self.http_request(method="GET", url_suffix=uri, resp_type="response", params=query_params) - paged_results = response.json() - if limit > 200: - query_params = {} - limit -= 200 - while limit > 0 and "next" in response.links and len(response.json()) > 0: - query_params["limit"] = encode_string_results(str(limit)) - next_page = delete_limit_param(response.links.get("next").get("url")) - response = self.http_request( - method="GET", full_url=next_page, url_suffix="", resp_type="response", params=query_params - ) - paged_results += response.json() - limit -= 200 - after = None - if "next" in response.links and len(response.json()) > 0: - after = get_after_tag(response.links.get("next").get("url")) - return (paged_results, after) - - def list_groups(self, args): - # Base url - if none of the the above specified - returns all the groups (default 200 items) - uri = "/api/v1/groups" - query_params = {} - for key, value in args.items(): - if key == "query": - key = "q" - query_params[key] = encode_string_results(value) - if args.get("limit"): - return self.http_request(method="GET", url_suffix=uri, params=query_params) - return self.get_paged_results(uri, query_params) - - def get_logs(self, args): - uri = "/api/v1/logs" - query_params = {} - for key, value in args.items(): - if key == "query": - key = "q" - query_params[key] = encode_string_results(value) - limit = args.get("limit") - limit = int(limit) if limit else None - if limit and limit <= MAX_LOGS_LIMIT: - return self.http_request(method="GET", url_suffix=uri, params=query_params) - if limit and limit > MAX_LOGS_LIMIT: - query_params["limit"] = MAX_LOGS_LIMIT - return self.get_paged_results(uri, query_params, max_limit=limit) - - def delete_user(self, user_term): - uri = f"/api/v1/users/{encode_string_results(user_term)}" - return self.http_request(method="DELETE", url_suffix=uri, resp_type="text") - - def clear_user_sessions(self, user_id, revoke_oauth_tokens=True): - uri = f"/api/v1/users/{user_id}/sessions" - params = {"oauthTokens": revoke_oauth_tokens} - return self.http_request(method="DELETE", url_suffix=uri, params=params, resp_type="text") - - def get_zone(self, zoneID): - uri = f"/api/v1/zones/{zoneID}" - return self.http_request(method="GET", url_suffix=uri) - - def list_zones(self, limit): - uri = "/api/v1/zones" - if limit: - query_params = {"limit": encode_string_results(limit)} - return self.http_request(method="GET", url_suffix=uri, params=query_params) - return self.get_paged_results(uri) - - def create_zone(self, zoneObject): - uri = "/api/v1/zones" - return self.http_request(method="POST", url_suffix=uri, json_data=zoneObject) - - def update_zone(self, zoneObject): - zoneID = zoneObject["id"] - uri = f"/api/v1/zones/{zoneID}" - - return self.http_request(method="PUT", url_suffix=uri, data=json.dumps(zoneObject)) - - def assign_group_to_app(self, group_id, app_id): - uri = f"/api/v1/apps/{app_id}/groups/{group_id}" - return self.http_request(method="PUT", url_suffix=uri, resp_type="text") - - -def module_test(client, args): - if client.auth_type == AuthType.OAUTH: - # For OAuth 2.0, there's no user the token belongs to, but an app. So the '/users/me' endpoint won't work. - uri = "/api/v1/users" - - else: - uri = "/api/v1/users/me" - - client.http_request(method="GET", url_suffix=uri) - return "ok", None, None - - -def unlock_user_command(client, args): - user_id = client.get_user_id(args.get("username")) - raw_response = client.unlock_user(user_id) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - - readable_output = f"### {args.get('username')} unlocked" - - return ( - readable_output, - outputs, - raw_response, # raw response - the original response - ) - - -def activate_user_command(client, args): - user_id = client.get_user_id(args.get("username")) - raw_response = client.activate_user(user_id) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - - readable_output = f"### {args.get('username')} is active now" - return (readable_output, outputs, raw_response) - - -def deactivate_user_command(client, args): - user_id = client.get_user_id(args.get("username")) - raw_response = client.deactivate_user(user_id) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - - readable_output = f"### User {args.get('username')} deactivated" - - return ( - readable_output, - outputs, - raw_response, # raw response - the original response - ) - - -def suspend_user_command(client, args): - user_id = client.get_user_id(args.get("username")) - raw_response = client.suspend_user(user_id) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - - readable_output = f"### {args.get('username')} status is Suspended" - return (readable_output, outputs, raw_response) - - -def unsuspend_user_command(client, args): - user_id = client.get_user_id(args.get("username")) - raw_response = client.unsuspend_user(user_id) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - - readable_output = f"### {args.get('username')} is no longer SUSPENDED" - return (readable_output, outputs, raw_response) - - -def get_user_factors_command(client, args): - user_id = args.get("userId") - - if not (args.get("username") or user_id): - raise Exception("You must supply either 'Username' or 'userId") - - if not user_id: - user_id = client.get_user_id(args.get("username")) - - raw_response = client.get_user_factors(user_id) - if not raw_response or len(raw_response) == 0: - return "No Factors found" - - factors = client.get_readable_factors(raw_response) - context = createContext(factors, removeNull=True) - outputs = { - "Account(val.ID && val.ID === obj.ID)": {"Factor": context, "ID": user_id}, - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"Factors for user: {user_id}\n {tableToMarkdown('Factors', factors)}" - return (readable_output, outputs, raw_response) - - -def reset_factor_command(client, args): - factor_id = args.get("factorId") - user_id = args.get("userId") - - if not (args.get("username") or user_id): - raise Exception("You must supply either 'Username' or 'userId") - - if not user_id: - user_id = client.get_user_id(args.get("username")) - - raw_response = client.reset_factor(user_id, factor_id) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"Factor: {factor_id} deleted" - return (readable_output, outputs, raw_response) - - -def set_password_command(client, args): - user_id = client.get_user_id(args.get("username")) - password = args.get("password") - - raw_response = client.set_password(user_id, password) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"{args.get('username')} password was last changed on {raw_response.get('passwordChanged')}" - - if argToBoolean(args.get("temporary_password", False)): - expire_password_response = client.expire_password(user_id, args) - expire_password_readable_output = tableToMarkdown("Okta Temporary Password", expire_password_response, removeNull=True) - readable_output = f"{readable_output}\n{expire_password_readable_output}" - - return readable_output, outputs, raw_response - - -def expire_password_command(client, args): - user_id = client.get_user_id(args.get("username")) - hide_password = argToBoolean(args.get("hide_password", False)) - revoke_session = argToBoolean(args.get("revoke_session", False)) - - if not (args.get("username") or user_id): - raise Exception("You must supply either 'Username' or 'userId") - if revoke_session is True: - raw_response = client.revoke_session(user_id) - else: - raw_response = client.expire_password(user_id, args) - if "tempPassword" in raw_response and hide_password: - raw_response["tempPassword"] = "Output removed by user. hide_password argument set to True" - user_context = client.get_users_context(raw_response) - - readable_output = tableToMarkdown("Okta Expired Password", raw_response, removeNull=True) - outputs = { - "Account(val.ID && val.ID === obj.ID)": createContext(user_context, removeNull=True), - "Okta.Metadata(true)": client.request_metadata, - } - - return (readable_output, outputs, raw_response) - - -def add_user_to_group_command(client, args): - group_id = args.get("groupId") - user_id = args.get("userId") - if (not (args.get("username") or user_id)) or (not (args.get("groupName") or group_id)): - raise Exception("You must supply either 'Username' or 'userId") - if not user_id: - user_id = client.get_user_id(args.get("username")) - if not group_id: - group_id = client.get_group_id(args.get("groupName")) - if group_id is None: - raise ValueError("The group was not found.") - raw_response = client.add_user_to_group(user_id, group_id) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"User: {user_id} added to group: {args.get('groupName')} successfully" - return (readable_output, outputs, raw_response) - - -def remove_from_group_command(client, args): - group_id = args.get("groupId") - user_id = args.get("userId") - - if (not (args.get("username") or user_id)) or (not (args.get("groupName") or group_id)): - raise Exception("You must supply either 'Username' or 'userId' and either 'groupName' or 'groupId'") - if not user_id: - user_id = client.get_user_id(args.get("username")) - if not group_id: - group_id = client.get_group_id(args.get("groupName")) - if group_id is None: - raise ValueError("The group was not found.") - raw_response = client.remove_user_from_group(user_id, group_id) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"User: {user_id} was removed from group: {args.get('groupName')} successfully" - return (readable_output, outputs, raw_response) - - -def get_groups_for_user_command(client, args): - user_id = client.get_user_id(args.get("username")) - raw_response = client.get_groups_for_user(user_id) - groups = client.get_readable_groups(raw_response) - - context = createContext(groups, removeNull=True) - outputs = { - "Account(val.ID && val.ID === obj.ID)": {"Group": context, "ID": user_id, "Type": "Okta"}, - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"Okta groups for user: {args.get('username')}\n {tableToMarkdown('Groups', groups)}" - - return (readable_output, outputs, raw_response) - - -def verify_push_factor_command(client, args): - user_id = args.get("userId") - factor_id = args.get("factorId") - polling_time = arg_to_number(args.get("polling_time", DEFAULT_POLLING_TIME)) - max_polling_calls = arg_to_number(args.get("max_polling_calls", DEFAULT_MAX_POLLING_CALLS)) - polling = argToBoolean(args.get("polling", True)) - - raw_response = client.verify_push_factor(user_id, factor_id) - poll_link = raw_response.get("_links").get("poll") - if not poll_link: - raise Exception("No poll link for the push factor challenge") - if not polling: - outputs = {"Okta.PollingStatusURL(true)": poll_link.get("href")} - readable_output = ( - "Push factor challenge has been initiated. To check the push factor challenge status, " - f"use the 'okta-verify-mfa-status' command with url: {poll_link.get('href')}" - ) - return (readable_output, outputs, raw_response) - - poll_response = client.poll_verify_push(poll_link.get("href"), polling_time, max_polling_calls) - - outputs = { - "Account(val.ID && val.ID === obj.ID)": {"ID": user_id, "VerifyPushResult": poll_response.get("factorResult")}, - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"Verify push factor result for user {user_id}: {poll_response.get('factorResult')}" - return (readable_output, outputs, raw_response) - - -def verify_mfa_status_command(client: Client, args: dict): - """Checks the status of a push factor procedure - - Args: - client (Client): The Okta client - args (dict): The command arguments - """ - polling_url = args.get("polling_url", "") - user_id, factor_id = extract_user_and_factor_id_from_url(polling_url) - - raw_response = client.get_push_factor_status(polling_url) - status = raw_response.get("factorResult") - outputs = { - "Okta.FactorResult(val.ID && val.ID === obj.ID)": { - "ID": factor_id, - "factorResult": status, - "UserID": user_id, - }, - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"The status of the push factor challenge is {status}" - return (readable_output, outputs, raw_response) - - -def search_command(client, args): - term = args.get("term") - limit = args.get("limit") or SEARCH_LIMIT - verbose = args.get("verbose") - advanced_search = args.get("advanced_search", "") - if not term and not advanced_search: - raise DemistoException("Please provide either the term or advanced_search argument") - raw_response = client.search(term, limit, advanced_search) - - if raw_response and len(raw_response) > 0: - users_context = client.get_users_context(raw_response) - users_readable = client.get_readable_users(raw_response, verbose) - context = createContext(users_context, removeNull=True) - outputs = { - "Account(val.ID && val.ID === obj.ID)": context, - "Okta.Metadata(true)": client.request_metadata, - } - if verbose == "true": - readable_output = f"### Okta users found:\n {users_readable}" - else: - readable_output = f"### Okta users found:\n {tableToMarkdown('Users:', users_readable)} " - return (readable_output, outputs, raw_response) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "No users found in Okta", outputs, raw_response - - -def get_user_command(client: Client, args: dict): - if not (args.get("username") or args.get("userId") or args.get("userEmail")): - raise Exception("You must supply either 'Username' or 'userId' or 'userEmail'") - - raw_response = None - if user_term := args.get("userEmail"): - raw_response, _ = client.list_users({"filter": f'profile.email eq "{user_term}"', "limit": 1}) - else: - user_term = args.get("userId") or args.get("username") - try: - raw_response = client.get_user(user_term) - except Exception as e: - if "404" not in str(e): - raise e - - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return (f"User {user_term!r} was not found.", outputs, {}) - - verbose = args.get("verbose") - - user_context = client.get_users_context(raw_response) - user_readable = client.get_readable_users(raw_response, verbose) - outputs = { - "Account(val.ID && val.ID === obj.ID)": createContext(user_context), - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = user_readable if verbose == "true" else f"{tableToMarkdown(f'User:{user_term}', user_readable)} " - return (readable_output, outputs, raw_response) - - -def create_user_command(client, args): - group_ids = args.get("groupIds") - cred = client.build_credentials(args) - profile = client.build_profile(args) - if group_ids: - group_ids = args.get("groupIds").split(",") - activate = "true" if args.get("activate") == "true" else "false" - raw_response = client.create_user(cred, profile, group_ids, activate) - user_context = client.get_users_context(raw_response) - outputs = { - "Account(val.ID && val.ID === obj.ID)": createContext(user_context), - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = tableToMarkdown(f"Okta User Created: {args.get('login')}:", client.get_readable_users(raw_response)) - return (readable_output, outputs, raw_response) - - -def update_user_command(client, args): - user_id = client.get_user_id(args.get("username")) - cred = client.build_credentials(args) - profile = client.build_profile(args) - profile["login"] = args.get("username") - raw_response = client.update_user(user_id, profile, cred) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = tableToMarkdown(f"Okta user: {args.get('username')} Updated:", raw_response.get("profile")) - return (readable_output, outputs, raw_response) - - -def get_group_members_command(client, args): - if not (args.get("groupId") or args.get("groupName")): - raise Exception("You must supply either 'groupName' or 'groupId") - limit = args.get("limit") - group_id = args.get("groupId") or client.get_group_id(args.get("groupName")) - if group_id is None: - raise ValueError("The group was not found.") - raw_members = client.get_group_members(group_id, limit) - users_context = client.get_users_context(raw_members) - users_readable = client.get_readable_users(raw_members, args.get("verbose")) - context = createContext(users_context, removeNull=True) - outputs = { - "Account(val.ID && val.ID === obj.ID)": context, - "Okta.Metadata(true)": client.request_metadata, - } - if args.get("verbose") == "true": - return (f"### Users for group: {args.get('groupName') or group_id}:\n {users_readable}", outputs, raw_members) - return (tableToMarkdown(f"Users for group: {args.get('groupName') or group_id}", users_readable), outputs, raw_members) - - -def list_users_command(client, args): - raw_response, after_tag = client.list_users(args) - verbose = args.get("verbose") - users = client.get_readable_users(raw_response, verbose) - user_context = client.get_users_context(raw_response) - context = createContext(user_context, removeNull=True) - if verbose == "true": - readable_output = f"### Okta users found:\n {users}" - else: - readable_output = f"### Okta users found:\n {tableToMarkdown('Users', users)} " - if after_tag: - readable_output += f"\n### tag: {after_tag}" - outputs = { - "Account(val.ID && val.ID == obj.ID)": context, - "Okta.User(val.tag)": {"tag": after_tag}, - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def list_groups_command(client, args): - raw_response = client.list_groups(args) - groups = client.get_readable_groups(raw_response) - context = createContext(groups, removeNull=True) - outputs = { - "Okta.Group(val.ID && val.ID === obj.ID)": context, - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = tableToMarkdown("Groups", groups) - - return (readable_output, outputs, raw_response) - - -def get_logs_command(client, args): - raw_response = client.get_logs(args) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "No logs found", outputs, raw_response - - logs = client.get_readable_logs(raw_response) - readable_output = tableToMarkdown("Okta Events", logs) - outputs = { - "Okta.Logs.Events(val.uuid && val.uuid === obj.uuid)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def get_failed_login_command(client, args): - args["filter"] = 'eventType eq "user.session.start" and outcome.result eq "FAILURE"' - raw_response = client.get_logs(args) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "No logs found", outputs, raw_response - logs = client.get_readable_logs(raw_response) - readable_output = tableToMarkdown("Failed Login Events", logs) - outputs = { - "Okta.Logs.Events(val.uuid && val.uuid === obj.uuid)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def get_group_assignments_command(client, args): - args["filter"] = 'eventType eq "group.user_membership.add"' - raw_response = client.get_logs(args) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "No logs found", outputs, raw_response - logs = client.get_readable_logs(raw_response) - readable_output = tableToMarkdown("Group Assignment Events", logs) - outputs = { - "Okta.Logs.Events(val.uuid && val.uuid === obj.uuid)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def get_application_assignments_command(client, args): - args["filter"] = 'eventType eq "application.user_membership.add"' - raw_response = client.get_logs(args) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "No logs found", outputs, raw_response - logs = client.get_readable_logs(raw_response) - readable_output = tableToMarkdown("Application Assignment Events", logs) - outputs = { - "Okta.Logs.Events(val.uuid && val.uuid === obj.uuid)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def get_application_authentication_command(client, args): - args["filter"] = 'eventType eq "user.authentication.sso"' - raw_response = client.get_logs(args) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "No logs found", outputs, raw_response - logs = client.get_readable_logs(raw_response) - readable_output = tableToMarkdown("Application Authentication Events", logs) - outputs = { - "Okta.Logs.Events(val.uuid && val.uuid === obj.uuid)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def delete_user_command(client, args): - if not (args.get("username") or args.get("userId")): - raise Exception("You must supply either 'Username' or 'userId") - user_term = args.get("userId") or args.get("username") - # Deletes a user permanently. This operation can only be performed on users that have a DEPROVISIONED status. - # This action cannot be recovered!This operation on a user that hasn't been deactivated - # causes that user to be deactivated. A second delete operation is required to delete the user. - user = client.get_user(user_term) - if user.get("status") != "DEPROVISIONED": - client.deactivate_user(args.get("userId") or client.get_user_id(args.get("username"))) - raw_response = client.delete_user(user_term) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"User: {user_term} was Deleted successfully" - return (readable_output, outputs, raw_response) - - -def clear_user_sessions_command(client, args): - user_id = args.get("userId") - revoke_oauth_tokens = argToBoolean(args.get("revokeOauthTokens", True)) - raw_response = client.clear_user_sessions(user_id, revoke_oauth_tokens) - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"### User session was cleared for: {user_id}" - - return readable_output, outputs, raw_response - - -def get_zone_command(client, args): - raw_response = client.get_zone(args.get("zoneID", "")) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "No zones found.", outputs, raw_response - readable_output = tableToMarkdown( - "Okta Zones", raw_response, headers=["name", "id", "gateways", "status", "system", "lastUpdated", "created"] - ) - outputs = { - "Okta.Zone(val.id && val.id === obj.id)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def list_zones_command(client, args): - raw_response = client.list_zones(args.get("limit")) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "No zones found.", outputs, raw_response - readable_output = tableToMarkdown( - "Okta Zones", raw_response, headers=["name", "id", "gateways", "status", "system", "lastUpdated", "created"] - ) - outputs = { - "Okta.Zone(val.id && val.id === obj.id)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def apply_zone_updates(zoneObject, zoneName, gatewayIPs, proxyIPs, updateType="OVERRIDE"): - # If user provided a new zone name - set it - if zoneName: - zoneObject["name"] = zoneName - - gateways = [] - proxies = [] - existing_gateways: list = zoneObject.get("gateways") - existing_proxies: list = zoneObject.get("proxies") - - if gatewayIPs: - for ip in gatewayIPs: - if "-" in ip: # Check for IP range notation - gateways.append({"type": "RANGE", "value": ip}) - else: # If not a range, treat it as a single IP - cidr_value = f"{ip}/32" if "/" not in ip else f"{ip}" - gateways.append({"type": "CIDR", "value": cidr_value}) - - if existing_gateways is not None and updateType == "APPEND": - zoneObject["gateways"] = existing_gateways + gateways - else: - zoneObject["gateways"] = gateways - - if proxyIPs: - for ip in proxyIPs: - if "-" in ip: # Check for IP range notation - proxies.append({"type": "RANGE", "value": ip}) - else: # If not a range, treat it as a single IP - cidr_value = f"{ip}/32" if "/" not in ip else f"{ip}" - proxies.append({"type": "CIDR", "value": cidr_value}) - - if existing_proxies is not None and updateType == "APPEND": - zoneObject["proxies"] = existing_proxies + proxies - else: - zoneObject["proxies"] = proxies - - return zoneObject - - -def create_zone_command(client, args): - zone_name = args.get("name") - gateway_ips = argToList(args.get("gateway_ips")) - proxies = argToList(args.get("proxies")) - if not (gateway_ips or proxies): - raise Exception("You must supply either 'gateway_ips' or 'proxies'.") - - zoneObject = {"name": "", "type": "IP", "status": "ACTIVE", "gateways": [], "proxies": []} - zoneObject = apply_zone_updates(zoneObject, zone_name, gateway_ips, proxies) - - raw_response = client.create_zone(zoneObject) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "Zone not created.", outputs, raw_response - readable_output = tableToMarkdown( - "Okta Zones", raw_response, headers=["name", "id", "gateways", "status", "system", "lastUpdated", "created"] - ) - outputs = { - "Okta.Zone(val.id && val.id === obj.id)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - - -def update_zone_command(client, args): - if not args.get("zoneName", "") and not args.get("gatewayIPs", "") and not args.get("proxyIPs", ""): - return ("Nothing to update", {}, "Nothing to update") - zoneID = args.get("zoneID", "") - updateType = args.get("updateType") - zoneObject = client.get_zone(zoneID) - if zoneID == zoneObject.get("id"): - zoneName = args.get("zoneName", "") - gatewayIPs = argToList(args.get("gatewayIPs", "")) - proxyIPs = argToList(args.get("proxyIPs", "")) - zoneObject = apply_zone_updates(zoneObject, zoneName, gatewayIPs, proxyIPs, updateType) - - raw_response = client.update_zone(zoneObject) - if not raw_response: - outputs = { - "Okta.Metadata(true)": client.request_metadata, - } - return "Got empty response.", outputs, raw_response - - readable_output = tableToMarkdown( - "Okta Zones", raw_response, headers=["name", "id", "gateways", "status", "system", "lastUpdated", "created"] - ) - outputs = { - "Okta.Zone(val.id && val.id === obj.id)": createContext(raw_response), - "Okta.Metadata(true)": client.request_metadata, - } - return (readable_output, outputs, raw_response) - else: - return "No zone found in Okta with this ID.", {}, {} - - -def assign_group_to_app_command(client, args): - group_id = args.get("groupId") - if not group_id: - group_id = client.get_group_id(args.get("groupName")) - if group_id is None: - raise ValueError("The group was not found.") - app_id = client.get_app_id(args.get("appName")) - raw_response = client.assign_group_to_app(group_id, app_id) - outputs = {"Okta.Metadata(true)": client.request_metadata} - readable_output = f"Group: {args.get('groupName')} added to PA App successfully" - return (readable_output, outputs, raw_response) - - -def create_group_command(client, args): - profile = client.build_group_profile(args) - raw_response = client.create_group(profile) - group_context = client.get_groups_context(raw_response) - outputs = { - "OktaGroup(val.ID && val.ID === obj.ID)": createContext(group_context), - "Okta.Metadata(true)": client.request_metadata, - } - readable_output = f"Group Created: [GroupID:{raw_response['id']}, GroupName: {raw_response['profile']['name']}]" - return (readable_output, outputs, raw_response) - - -def reset_auth_command(client, args): - reset_integration_context() - return CommandResults(readable_output="Authentication data cleared successfully.") - - -def get_after_tag(url): - """retrieve the after param from the url - - Args: - url: some url - - Returns: - String: the value of the 'after' query param. - """ - parsed_url = urlparse(url) - captured_value = parse_qs(parsed_url.query)["after"][0] - return captured_value - - -def delete_limit_param(url): - """Delete the limit param from the url - - Args: - url: some url - - Returns: - String: the url with the limit query param. - """ - parsed_url = urlparse(url) - query_dict = parse_qs(parsed_url.query) - query_dict.pop("limit") - return urlunparse(parsed_url._replace(query=urlencode(query_dict, True))) - - -def extract_user_and_factor_id_from_url(url: str) -> tuple[str, str]: - """Extracts the user ID and Factor ID from the polling url. - - Args: - url (str): The polling url. - - Returns: - tuple[str, str]: The user ID and Factor ID. - """ - parsed_url = urlparse(url) - path_parts = parsed_url.path.split("/") - # Example for url: https://example.okta.com/api/v1/users/user-id/factors/factor-id - # path_parts: ['', 'api', 'v1', 'users', 'user-id', 'factors', 'factor-id'] - try: - user_id = path_parts[path_parts.index("users") + 1] - factor_id = path_parts[path_parts.index("factors") + 1] - return user_id, factor_id - except (ValueError, IndexError): - raise DemistoException(f"Could not extract user ID and Factor ID from the polling URL: {url}") - - -def main(): - try: - params = demisto.params() - - demisto.debug(f"Command being called is {demisto.command()}") - commands = { - "test-module": module_test, - "okta-unlock-user": unlock_user_command, - "okta-deactivate-user": deactivate_user_command, - "okta-activate-user": activate_user_command, - "okta-suspend-user": suspend_user_command, - "okta-unsuspend-user": unsuspend_user_command, - "okta-reset-factor": reset_factor_command, - "okta-set-password": set_password_command, - "okta-expire-password": expire_password_command, - "okta-add-to-group": add_user_to_group_command, - "okta-remove-from-group": remove_from_group_command, - "okta-get-groups": get_groups_for_user_command, - "okta-get-user-factors": get_user_factors_command, - "okta-verify-push-factor": verify_push_factor_command, - "okta-verify-mfa-status": verify_mfa_status_command, - "okta-search": search_command, - "okta-get-user": get_user_command, - "okta-create-user": create_user_command, - "okta-update-user": update_user_command, - "okta-get-group-members": get_group_members_command, - "okta-list-users": list_users_command, - "okta-list-groups": list_groups_command, - "okta-get-logs": get_logs_command, - "okta-get-failed-logins": get_failed_login_command, - "okta-get-application-assignments": get_application_assignments_command, - "okta-get-group-assignments": get_group_assignments_command, - "okta-get-application-authentication": get_application_authentication_command, - "okta-delete-user": delete_user_command, - "okta-clear-user-sessions": clear_user_sessions_command, - "okta-list-zones": list_zones_command, - "okta-get-zone": get_zone_command, - "okta-update-zone": update_zone_command, - "okta-create-zone": create_zone_command, - "okta-create-group": create_group_command, - "okta-assign-group-to-app": assign_group_to_app_command, - "okta-auth-reset": reset_auth_command, - } - - command = demisto.command() - - client = Client( - base_url=params["url"].rstrip("/"), - verify=(not params.get("insecure", False)), - headers={ - "Accept": "application/json", - "Content-Type": "application/json", - }, - proxy=params.get("proxy", False), - ok_codes=(200, 201, 204), - api_token=params.get("credentials", {}).get("password") or params.get("apitoken"), - auth_type=AuthType.OAUTH if argToBoolean(params.get("use_oauth", False)) else AuthType.API_TOKEN, - client_id=params.get("client_id"), - scopes=OAUTH_TOKEN_SCOPES, - private_key=params.get("private_key"), - jwt_algorithm=JWTAlgorithm(params["jwt_algorithm"]) if params.get("jwt_algorithm") else None, - key_id=params.get("key_id", None), - ) - - if command in commands: - result = commands[command](client, demisto.args()) - - if isinstance(result, CommandResults | str | dict): - return_results(result) +from OktaApiModule import * # noqa: E402 - else: - human_readable, outputs, raw_response = result - return_outputs(readable_output=human_readable, outputs=outputs, raw_response=raw_response) - else: - raise NotImplementedError(f"Command {command} is not implemented.") +def main() -> None: + run_okta_v2_integration() - except Exception as e: - return_error(f"Failed to execute {demisto.command()} command. Error: {e!s}") +from CommonServerUserPython import * # noqa: E402 # pylint: disable=wrong-import-position if __name__ in ("__main__", "__builtin__", "builtins"): main() diff --git a/Packs/Okta/Integrations/Okta_v2/Okta_v2.yml b/Packs/Okta/Integrations/Okta_v2/Okta_v2.yml index b3f841191238..af0113ffe424 100644 --- a/Packs/Okta/Integrations/Okta_v2/Okta_v2.yml +++ b/Packs/Okta/Integrations/Okta_v2/Okta_v2.yml @@ -1887,7 +1887,7 @@ script: - description: Reset OAuth authentication data (authentication process will start from the beginning, and a new token will be generated). name: okta-auth-reset arguments: [] - dockerimage: demisto/crypto:1.0.0.5490413 + dockerimage: demisto/crypto:1.0.0.12833174 runonce: false script: "" subtype: python3 diff --git a/Packs/Okta/Integrations/Okta_v2/Okta_v2_test.py b/Packs/Okta/Integrations/Okta_v2/Okta_v2_test.py index 27e6bf7e3055..03de321e68f6 100644 --- a/Packs/Okta/Integrations/Okta_v2/Okta_v2_test.py +++ b/Packs/Okta/Integrations/Okta_v2/Okta_v2_test.py @@ -1,1440 +1,51 @@ -import json -from unittest.mock import MagicMock, patch +"""Sanity tests for the Okta v2 shim integration. -import pytest -import requests_mock -from Okta_v2 import ( - Client, - apply_zone_updates, - assign_group_to_app_command, - clear_user_sessions_command, - create_group_command, - create_user_command, - create_zone_command, - delete_limit_param, - get_after_tag, - get_group_members_command, - get_groups_for_user_command, - get_logs_command, - get_user_command, - get_user_factors_command, - get_zone_command, - list_users_command, - list_zones_command, - set_password_command, - update_zone_command, - verify_push_factor_command, - verify_mfa_status_command, - extract_user_and_factor_id_from_url, -) - -client = Client(base_url="demisto.com", api_token="XXX") - -user_data = { - "id": "TestID", - "status": "PROVISIONED", - "created": "2020-02-19T08:18:20.000Z", - "activated": "2020-02-20T11:44:43.000Z", - "statusChanged": "2020-02-20T11:45:24.000Z", - "lastLogin": "2020-02-23T11:45:24.000Z", - "lastUpdated": "2020-02-20T11:45:24.000Z", - "passwordChanged": "2020-02-19T08:18:21.000Z", - "type": {"id": "oty66lckcvDyVcGzS0h7"}, - "profile": { - "firstName": "test", - "lastName": "this", - "mobilePhone": "null", - "city": "Tel-Aviv", - "displayName": "test1", - "secondEmail": "test@this.com", - "login": "test@this.com", - "email": "test@this.com", - "manager": "manager", - "managerEmail": "manager@test.com", - }, - "credentials": {"provider": {"type": "OKTA", "name": "OKTA"}}, - "_links": {}, -} -factors_data = [ - { - "id": "mblpt21nffaaN5F060h7", - "factorType": "sms", - "provider": "OKTA", - "vendorName": "OKTA", - "status": "PENDING_ACTIVATION", - "created": "2020-02-18T11:48:16.000Z", - "lastUpdated": "2020-02-18T11:48:16.000Z", - "profile": {"phoneNumber": "+12025550191"}, - "_links": {}, - }, - { - "id": "uftpt24kdrDJ7fDOq0h7", - "factorType": "token:software:totp", - "provider": "GOOGLE", - "vendorName": "GOOGLE", - "status": "PENDING_ACTIVATION", - "created": "2020-02-18T11:45:14.000Z", - "lastUpdated": "2020-02-18T11:45:14.000Z", - "profile": {"credentialId": "woo@demisto.com"}, - "_links": {}, - }, - { - "id": "opfpt1joeaArlg27g0h7", - "factorType": "push", - "provider": "OKTA", - "vendorName": "OKTA", - "status": "PENDING_ACTIVATION", - "created": "2020-02-18T11:45:03.000Z", - "lastUpdated": "2020-02-18T11:45:03.000Z", - "_links": {"self": {}, "poll": {}, "user": {}}, - "_embedded": {"activation": {"factorResult": "TIMEOUT", "_links": {}}}, - }, -] -group_data = [ - { - "id": "00g66lckcsAJpLcNc0h7", - "created": "2016-04-12T15:01:50.000Z", - "lastUpdated": "2016-04-12T15:01:50.000Z", - "lastMembershipUpdated": "2020-02-19T09:01:32.000Z", - "objectClass": ["okta:user_group"], - "type": "BUILT_IN", - "profile": {"name": "Everyone", "description": "All users in your organization"}, - "_links": {}, - } -] -verify_push_factor_response = { - "factorResult": "WAITING", - "profile": { - "credentialId": "test@this.com", - "deviceType": "SmartPhone_IPhone", - "keys": [ - { - "kty": "EC", - "use": "sig", - "kid": "default", - "x": "3Y53lDoQYwzzVbjsbsPnqOnVaotIrVByQh5Sa-RwOHQ", - "y": "0zHY_y9rVh-bq_-lR-MrmzNtUZrrIMbTrsjtxUyUT2Q", - "crv": "P-256", - } - ], - "name": "iPhone (5)", - "platform": "IOS", - "version": "13.1.3", - }, - "expiresAt": "2020-02-24T11:37:08.000Z", - "_links": { - "cancel": { - "href": "https://test.com/api/v1/users/TestID/factors/FactorID/transactions/TransactionID", - "hints": {"allow": ["DELETE"]}, - }, - "poll": { - "href": "https://test.com/api/v1/users/TestID/factors/FactorID/transactions/TransactionID", - "hints": {"allow": ["GET"]}, - }, - }, -} -polling_response_success = {"factorResult": "SUCCESS"} -polling_response_rejected = { - "factorResult": "REJECTED", - "_links": { - "verify": {"href": "https://test.com/api/v1/users/TestID/factors/FactorID/verify", "hints": {"allow": ["POST"]}}, - "factor": {}, - }, -} -create_user_response = { - "id": "00ub0oNGTSWTBKOLGLNR", - "status": "STAGED", - "created": "2013-07-02T21:36:25.344Z", - "activated": "", - "statusChanged": "", - "lastLogin": "", - "lastUpdated": "2013-07-02T21:36:25.344Z", - "passwordChanged": "2013-07-02T21:36:25.344Z", - "profile": { - "firstName": "Testush", - "lastName": "Test", - "email": "test@this.com", - "login": "test@this.com", - "mobilePhone": "555-415-1337", - }, - "credentials": {"password": {}, "provider": {"type": "OKTA", "name": "OKTA"}}, - "_links": { - "activate": {"href": "https://test.com/api/v1/users/TestID/lifecycle/activate"}, - "self": {"href": "https://test.com/api/v1/users/TestID"}, - }, -} -create_group_response = { - "id": "00g3q8tjdyoOw6fJE1d7", - "created": "2022-05-20T14:59:29.000Z", - "lastUpdated": "2022-05-20T14:59:29.000Z", - "lastMembershipUpdated": "2022-05-20T14:59:29.000Z", - "objectClass": ["okta:user_group"], - "type": "OKTA_GROUP", - "profile": {"name": "TestGroup", "description": "Test Group Description"}, - "_links": { - "logo": [ - { - "name": "medium", - "href": "https://op3static.oktacdn.com/assets/img/logos/groups/odyssey/okta-medium.png", - "type": "image/png", - }, - { - "name": "large", - "href": "https://op3static.oktacdn.com/assets/img/logos/groups/odyssey/okta-large.png", - "type": "image/png", - }, - ], - "users": {"href": "https://test.com/api/v1/groups/00g3q8tjdyoOw6fJE1d7/users"}, - "apps": {"href": "https://test.com/api/v1/groups/00g3q8tjdyoOw6fJE1d7/apps"}, - }, -} -assign_group_to_app_response = { - "id": "00g3q8tjdyoOw6fJE1d7", - "lastUpdated": "2022-05-20T16:01:16.000Z", - "priority": 5, - "profile": {}, - "_links": { - "app": {"href": "https://test.com/api/v1/apps/0oa3ik9908vngPiMB1d7"}, - "self": {"href": "https://test.com/api/v1/apps/0oa3ik9908vngPiMB1d7/groups/00g3q8tjdyoOw6fJE1d7"}, - "group": {"href": "https://test.com/api/v1/groups/00g3q8tjdyoOw6fJE1d7"}, - }, -} -group_members = [ - { - "id": "TestID1", - "status": "ACTIVE", - "created": "2016-04-12T15:01:52.000Z", - "activated": "", - "statusChanged": "2020-02-12T15:05:06.000Z", - "lastLogin": "2020-02-24T11:40:36.000Z", - "lastUpdated": "2020-02-24T11:42:22.000Z", - "passwordChanged": "2020-02-24T11:40:08.000Z", - "type": {"id": "oty66lckcvDyVcGzS0h7"}, - "profile": { - "firstName": "Test1", - "lastName": "Test1", - "primaryPhone": "8888888888", - "mobilePhone": "", - "secondEmail": "", - "department": "everyone,admin,bla", - "login": "test@this.com", - "email": "test@this.com", - }, - "credentials": { - "password": {}, - "recovery_question": {"question": "born city"}, - "provider": {"type": "OKTA", "name": "OKTA"}, - }, - "_links": {}, - }, - { - "id": "TestID2", - "status": "STAGED", - "created": "2018-07-24T20:20:04.000Z", - "activated": "", - "statusChanged": "", - "lastLogin": "", - "lastUpdated": "2018-07-24T20:20:04.000Z", - "passwordChanged": "", - "type": {"id": "oty66lckcvDyVcGzS0h7"}, - "profile": { - "firstName": "Test2", - "lastName": "Test2", - "mobilePhone": "", - "secondEmail": "", - "login": "john@doe.com", - "email": "john@doe.com", - }, - "credentials": {"provider": {"type": "OKTA", "name": "OKTA"}}, - "_links": {}, - }, - { - "id": "TestID3", - "status": "PROVISIONED", - "created": "2018-07-31T12:48:33.000Z", - "activated": "2020-02-19T12:33:20.000Z", - "statusChanged": "2020-02-19T12:33:20.000Z", - "lastLogin": "", - "lastUpdated": "2020-02-19T12:33:20.000Z", - "passwordChanged": "2020-02-06T13:32:56.000Z", - "type": {"id": "oty66lckcvDyVcGzS0h7"}, - "profile": { - "firstName": "test", - "lastName": "that", - "manager": "MegaTester", - "mobilePhone": "", - "city": "TLV", - "displayName": "alsotest", - "secondEmail": "woo@demisto.com", - "login": "woo@demisto.com", - "email": "woo@demisto.com", - "employeeNumber": "123427", - }, - "credentials": {"provider": {"type": "OKTA", "name": "OKTA"}}, - "_links": {}, - }, -] -logs = [ - { - "actor": { - "id": "UserTestID1", - "type": "User", - "alternateId": "soso@demisto.com", - "displayName": "Test1 Testush", - "detailEntry": "", - }, - "client": { - "userAgent": {"rawUserAgent": "python-requests/2.22.0", "os": "Windows", "browser": "Chrome"}, - "zone": "null", - "device": "Computer", - "id": "", - "ipAddress": "8.8.8.8", - "geographicalContext": { - "city": "Tel Aviv", - "state": "Tel Aviv", - "country": "Israel", - "postalCode": "", - "geolocation": {"lat": 32.0678, "lon": 34.7647}, - }, - }, - "authenticationContext": { - "authenticationProvider": "", - "credentialProvider": "", - "credentialType": "", - "issuer": "", - "interface": "", - "authenticatio'nStep": 0, - "externalSessionId": "trsGDHiJe2ISM2GneNwg_tIWw", - }, - "displayMessage": "Add user to application membership", - "eventType": "application.user_membership.add", - "outcome": {"result": "SUCCESS", "reason": ""}, - "published": "2020-02-18T11:23:05.066Z", - "securityContext": {"asNumber": "", "asOrg": "", "isp": "", "domain": "", "isProxy": ""}, - "severity": "INFO", - "debugContext": { - "debugData": { - "requestId": "XkvJGFsS6hsPnC7KoFliVAAABzI", - "requestUri": "/api/v1/users", - "threatSuspected": "false", - "url": "/api/v1/users?activate=true", - } - }, - "legacyEventType": "app.generic.provision.assign_user_to_app", - "transaction": {"type": "WEB", "id": "XkvJGFsS6hsPnC7KoFliVAAABzI", "detail": {}}, - "uuid": "081c84f9-5241-11ea-ad7c-6125e916db06", - "version": "0", - "request": { - "ipChain": [ - { - "ip": "8.8.8.8", - "geographicalContext": { - "city": "Tel Aviv", - "state": "Tel Aviv", - "country": "Israel", - "postalCode": "", - "geolocation": {"lat": 32.0678, "lon": 34.7647}, - }, - "version": "V4", - "source": "", - } - ] - }, - "target": [ - { - "id": "UserTestID2", - "type": "AppUser", - "alternateId": "momo@demisto.com", - "displayName": "Test 1 that", - "detailEntry": "", - }, - { - "id": "0oabfkvxe1npBRdow0h7", - "type": "AppInstance", - "alternateId": "Demisto-SAML-OKTA", - "displayName": "Demisto-SAML-OKTA", - "detailEntry": "", - }, - { - "id": "00upt1h0w93PALT9v0h7", - "type": "User", - "alternateId": "momo@demisto.com", - "displayName": "Test 1 that", - "detailEntry": "", - }, - ], - }, - { - "actor": { - "id": "UserTestID2", - "type": "User", - "alternateId": "TestID2@demisto.com", - "displayName": "Testush2 test", - "detailEntry": "", - }, - "client": { - "userAgent": {"rawUserAgent": "python-requests/2.22.0", "os": "Unknown", "browser": "UNKNOWN"}, - "zone": "null", - "device": "Unknown", - "id": "", - "ipAddress": "8.8.8.8", - "geographicalContext": { - "city": "Tel Aviv", - "state": "Tel Aviv", - "country": "Israel", - "postalCode": "", - "geolocation": {"lat": 32.0678, "lon": 34.7647}, - }, - }, - "authenticationContext": { - "authenticationProvider": "", - "credentialProvider": "", - "credentialType": "", - "issuer": "", - "interface": "", - "authenticationStep": 0, - "externalSessionId": "trsGDHiJe2ISM2GneNwg_tIWw", - }, - "displayMessage": "Add user to application membership", - "eventType": "application.user_membership.add", - "outcome": {"result": "SUCCESS", "reason": ""}, - "published": "2020-02-18T11:23:04.791Z", - "securityContext": {"asNumber": "", "asOrg": "", "isp": "", "domain": "", "isProxy": ""}, - "severity": "INFO", - "debugContext": { - "debugData": { - "requestId": "XkvJGFsS6hsPnC7KoFliVAAABzI", - "requestUri": "/api/v1/users", - "threatSuspected": "false", - "url": "/api/v1/users?activate=true", - } - }, - "legacyEventType": "app.generic.provision.assign_user_to_app", - "transaction": {"type": "WEB", "id": "XkvJGFsS6hsPnC7KoFliVAAABzI", "detail": {}}, - "uuid": "07f28ec5-5241-11ea-ad7c-6125e916db06", - "version": "0", - "request": { - "ipChain": [ - { - "ip": "127.0.0.1", - "geographicalContext": { - "city": "Tel Aviv", - "state": "Tel Aviv", - "country": "Israel", - "postalCode": "", - "geolocation": {"lat": 32.0678, "lon": 34.7647}, - }, - "version": "V4", - "source": "", - } - ] - }, - "target": [ - { - "id": "0uapt1h0wbuz8uWvb0h7", - "type": "AppUser", - "alternateId": "momo@demisto.com", - "displayName": "Test 1 that", - "detailEntry": "", - }, - { - "id": "0oabe0e2jruaQccDf0h7", - "type": "AppInstance", - "alternateId": "ShrikSAML", - "displayName": "ShrikSAML", - "detailEntry": "", - }, - { - "id": "00upt1h0w93PALT9v0h7", - "type": "User", - "alternateId": "momo@demisto.com", - "displayName": "Test 1 that", - "detailEntry": "", - }, - ], - }, -] - -okta_zone = { - "_links": { - "deactivate": { - "hints": {"allow": ["POST"]}, - "href": "https://dev-530328.oktapreview.com/api/v1/zones/nzoqsmcx1qWYJ6wYF7q0/lifecycle/deactivate", - }, - "self": { - "hints": {"allow": ["GET", "PUT", "DELETE"]}, - "href": "https://dev-530328.oktapreview.com/api/v1/zones/nzoqsmcx1qWYJ6wYF7q0", - }, - }, - "created": "2020-04-06T22:23:12.000Z", - "gateways": [{"type": "CIDR", "value": "4.5.3.2/16"}, {"type": "CIDR", "value": "1.2.1.2/32"}], - "id": "nzoqsmcx1qWYJ6wYF7q0", - "lastUpdated": "2020-05-15T05:13:06.000Z", - "name": "Test Zone", - "proxies": None, - "status": "ACTIVE", - "system": False, - "type": "IP", -} - -LOGS = [{"mock_log1": "mock_value1"}, {"mock_log2": "mock_value2"}, {"mock_log3": "mock_value3"}] - - -def util_load_json(path: str): - """ - Utility to load json data from a local folder. - """ - with open(path, encoding="utf-8") as file: - return json.loads(file.read()) - - -@pytest.mark.parametrize( - # Write and define the expected - "args ,expected_context, expected_readable", - [ - ( - {"userId": "TestID", "username": "", "verbose": "false"}, - { - "ID": "TestID", - "Username": "test@this.com", - "DisplayName": "test this", - "Email": "test@this.com", - "Status": "PROVISIONED", - "Type": "Okta", - "Created": "2020-02-19T08:18:20.000Z", - "Activated": "2020-02-20T11:44:43.000Z", - "StatusChanged": "2020-02-20T11:45:24.000Z", - "PasswordChanged": "2020-02-19T08:18:21.000Z", - "Manager": "manager", - "ManagerEmail": "manager@test.com", - }, - "test@this.com", - ), - ( - {"userId": "", "username": "test@this.com", "verbose": "true"}, - { - "ID": "TestID", - "Username": "test@this.com", - "DisplayName": "test this", - "Email": "test@this.com", - "Status": "PROVISIONED", - "Type": "Okta", - "Created": "2020-02-19T08:18:20.000Z", - "Activated": "2020-02-20T11:44:43.000Z", - "StatusChanged": "2020-02-20T11:45:24.000Z", - "PasswordChanged": "2020-02-19T08:18:21.000Z", - "Manager": "manager", - "ManagerEmail": "manager@test.com", - }, - "Additional Data", - ), - ], -) -def test_get_user_command(mocker, args, expected_context, expected_readable): - mocker.patch.object(client, "get_user", return_value=user_data) - readable, outputs, _ = get_user_command(client, args) - assert outputs.get("Account(val.ID && val.ID === obj.ID)")[0] == expected_context - assert expected_readable in readable - - -def test_get_user_command_not_found_user(mocker): - """ - Given: - - Username. - - When: - - running get_user_command. - - Then: - - Ensure that no exception was raised, and assert the readable output. - """ - args = {"username": "test@this.com"} - mocker.patch.object(client, "get_user", side_effect=Exception("Error in API call [404] - Not found")) - readable, _, _ = get_user_command(client, args) - assert readable == "User 'test@this.com' was not found." - - -def test_get_user_command_email(mocker): - """ - Given: - - User email parameter. - When: - - Running get_user_command with userEmail. - Then: - - Ensure that the user is found via email search and returned correctly. - """ - args = {"userEmail": "isaac.brock@example.com", "verbose": "false"} - user_data = { - "id": "TestID", - "status": "ACTIVE", - "created": "2013-06-24T16:39:18.000Z", - "activated": "2013-06-24T16:39:19.000Z", - "statusChanged": "2013-06-24T16:39:19.000Z", - "lastLogin": "2013-06-24T17:39:19.000Z", - "lastUpdated": "2013-07-02T21:36:25.344Z", - "passwordChanged": "2013-07-02T21:36:25.344Z", - "profile": { - "login": "isaac.brock@example.com", - "firstName": "Isaac", - "lastName": "Brock", - "nickName": "issac", - "displayName": "Isaac Brock", - "email": "isaac.brock@example.com", - "secondEmail": "isaac@example.org", - "manager": "manager", - "managerEmail": "manager@test.com", - }, - } - expected_context = { - "ID": "TestID", - "Username": "isaac.brock@example.com", - "DisplayName": "Isaac Brock", - "Email": "isaac.brock@example.com", - "Status": "ACTIVE", - "Type": "Okta", - "Created": "2013-06-24T16:39:18.000Z", - "Activated": "2013-06-24T16:39:19.000Z", - "StatusChanged": "2013-06-24T16:39:19.000Z", - "PasswordChanged": "2013-07-02T21:36:25.344Z", - "Manager": "manager", - "ManagerEmail": "manager@test.com", - } - expected_readable = "isaac.brock@example.com" - - mocker.patch.object(client, "list_users", return_value=([user_data], None)) - mock_get_user = mocker.patch.object(client, "get_user", return_value=user_data) - readable, outputs, _ = get_user_command(client, args) - - assert outputs.get("Account(val.ID && val.ID === obj.ID)")[0] == expected_context - assert expected_readable in readable - mock_get_user.assert_not_called() - - -@pytest.mark.parametrize( - # Write and define the expected - "args ,expected_context, expected_readable", - [ - ( - {"userId": "TestID", "username": "", "verbose": "false"}, - { - "ID": "TestID", - "Username": "test@this.com", - "DisplayName": "test this", - "Email": "test@this.com", - "Status": "PROVISIONED", - "Type": "Okta", - "Created": "2020-02-19T08:18:20.000Z", - "Activated": "2020-02-20T11:44:43.000Z", - "StatusChanged": "2020-02-20T11:45:24.000Z", - "PasswordChanged": "2020-02-19T08:18:21.000Z", - "Manager": "manager", - "ManagerEmail": "manager@test.com", - }, - "test@this.com", - ), - ( - {"userId": "", "username": "test@this.com", "verbose": "true"}, - { - "ID": "TestID", - "Username": "test@this.com", - "DisplayName": "test this", - "Email": "test@this.com", - "Status": "PROVISIONED", - "Type": "Okta", - "Created": "2020-02-19T08:18:20.000Z", - "Activated": "2020-02-20T11:44:43.000Z", - "StatusChanged": "2020-02-20T11:45:24.000Z", - "PasswordChanged": "2020-02-19T08:18:21.000Z", - "Manager": "manager", - "ManagerEmail": "manager@test.com", - }, - "Additional Data", - ), - ], -) -def test_list_user_command(mocker, args, expected_context, expected_readable): - mocker.patch.object(client, "list_users", return_value=(user_data, "123dasu23c")) - readable, outputs, _ = list_users_command(client, args) - assert outputs.get("Account(val.ID && val.ID == obj.ID)")[0] == expected_context - assert expected_readable in readable - assert "tag: 123dasu23c" in readable - - -@pytest.mark.parametrize("args", [({"userId": "TestID", "username": "", "verbose": "false"})]) -def test_after_key_list_user_command(mocker, args): - """ - Given - - args. - - When - - Running list_users command. - - Then - - Validate that since there's no more results to show, there's no tag key in the readable output. - """ - mocker.patch.object(client, "list_users", return_value=(user_data, None)) - readable, _, _ = list_users_command(client, args) - assert "tag:" not in readable - - -@pytest.mark.parametrize( - "url, expected_after_tag", [("https://dev-725178.oktapreview.com/api/v1/users?limit=10&after=qazwsx123", "qazwsx123")] -) -def test_get_after_tag_function(url, expected_after_tag): - """ - Given - - url. - When - - Running get_after_tag function. - - Then - - Validate that tag was extracted correctly. - """ - after_tag = get_after_tag(url) - assert expected_after_tag == after_tag - - -@pytest.mark.parametrize( - "url, expected_url", - [ - ( - "https://dev-725178.oktapreview.com/api/v1/users?limit=10&after=qazwsx123", - "https://dev-725178.oktapreview.com/api/v1/users?after=qazwsx123", - ) - ], -) -def test_delete_limit_param_function(url, expected_url): - """ - Given - - url. - When - - Running delete_limit_param function. - - Then - - Ensure that the limit param was deleted. - """ - modified_url = delete_limit_param(url) - assert expected_url == modified_url - - -@pytest.mark.parametrize( - "args ,expected_context", - [ - ( - {"userId": "TestID"}, - { - "ID": "uftpt24kdrDJ7fDOq0h7", - "FactorType": "token:software:totp", - "Provider": "GOOGLE", - "Status": "PENDING_ACTIVATION", - "Profile": {"credentialId": "woo@demisto.com"}, - }, - ), - ( - {"username": "test@this.com"}, - { - "ID": "uftpt24kdrDJ7fDOq0h7", - "FactorType": "token:software:totp", - "Provider": "GOOGLE", - "Status": "PENDING_ACTIVATION", - "Profile": {"credentialId": "woo@demisto.com"}, - }, - ), - ], -) -def test_get_user_factors_command(mocker, args, expected_context): - mock_raw_response = MagicMock() - mock_raw_response.json.return_value = expected_context - mock_raw_response.headers = {"x-rate-limit-limit": "1", "x-rate-limit-remaining": "1", "x-rate-limit-reset": "1"} - mocker.patch.object(client, "get_user_id", return_value="TestID") - mocker.patch.object(client, "_http_request", return_value=mock_raw_response) - mocker.patch.object(client, "get_readable_factors", return_value=[expected_context]) - - result = get_user_factors_command(client, args) - readable_output, outputs, raw_response = result - assert expected_context == outputs.get("Account(val.ID && val.ID === obj.ID)").get("Factor")[0] - assert outputs.get("Account(val.ID && val.ID === obj.ID)").get("ID") == args.get("userId") or "TestID" - - -@pytest.mark.parametrize("args", [{"username": "test@this.com"}]) -def test_get_groups_for_user_command(mocker, args): - expected_context = { - "ID": "00g66lckcsAJpLcNc0h7", - "Created": "2016-04-12T15:01:50.000Z", - "ObjectClass": ["okta:user_group"], - "LastUpdated": "2016-04-12T15:01:50.000Z", - "LastMembershipUpdated": "2020-02-19T09:01:32.000Z", - "Type": "BUILT_IN", - "Name": "Everyone", - "Description": "All users in your organization", - } - mocker.patch.object(client, "get_user_id", return_value="TestID") - mocker.patch.object(client, "get_groups_for_user", return_value=group_data) - _, outputs, _ = get_groups_for_user_command(client, args) - assert outputs.get("Account(val.ID && val.ID === obj.ID)").get("Group")[0] == expected_context - assert outputs.get("Account(val.ID && val.ID === obj.ID)").get("ID") == "TestID" - - -@pytest.mark.parametrize( - "args, polling_response, result", - [ - ({"userId": "TestID", "factorId": "FactorID"}, polling_response_rejected, "REJECTED"), - ({"userId": "TestID", "factorId": "FactorID"}, polling_response_success, "SUCCESS"), - ], -) -def test_verify_push_factor_command(mocker, args, polling_response, result): - mocker.patch.object(client, "verify_push_factor", return_value=verify_push_factor_response) - mocker.patch.object(client, "poll_verify_push", return_value=polling_response) - _, outputs, _ = verify_push_factor_command(client, args) - assert outputs.get("Account(val.ID && val.ID === obj.ID)").get("ID") == "TestID" - assert outputs.get("Account(val.ID && val.ID === obj.ID)").get("VerifyPushResult") == result - - -@pytest.mark.parametrize( - "args, polling_time, max_polling_calls, expected_calls, expected_result", - [ - ( - {"userId": "TestID", "factorId": "FactorID", "polling_time": "2", "max_polling_calls": "3"}, - 2, - 3, - 3, - "TIMEOUT", - ), - ( - {"userId": "TestID", "factorId": "FactorID", "polling_time": "1", "max_polling_calls": "5"}, - 1, - 5, - 2, - "SUCCESS", - ), - ], -) -def test_verify_push_factor_command_polling_args(mocker, args, polling_time, max_polling_calls, expected_calls, expected_result): - """ - Given: - - Arguments for verify_push_factor_command with custom polling_time and max_polling_calls - - When: - - Running verify_push_factor_command with these arguments - - Then: - - Ensure poll_verify_push is called with the correct polling_time and max_polling_calls - - Ensure time.sleep is called the expected number of times with the correct polling_time - - Verify the expected result is returned - """ - # Mock the verify_push_factor method to return a response with a poll link - mocker.patch.object(client, "verify_push_factor", return_value=verify_push_factor_response) - - # Mock time.sleep to avoid actual waiting during tests - mocker.patch("time.sleep") - - # For the first test case (TIMEOUT), all responses should be "WAITING" until timeout - # For the second test case (SUCCESS), the second response should be "SUCCESS" - if expected_result == "TIMEOUT": - waiting_response = {"factorResult": "WAITING"} - timeout_response = {"factorResult": "TIMEOUT"} - - # Mock http_request to return "WAITING" for all calls - mocker.patch.object(client, "http_request", side_effect=[waiting_response] * max_polling_calls) - - # Mock poll_verify_push to simulate timeout - mocker.patch.object(client, "poll_verify_push", return_value=timeout_response) - else: - # For SUCCESS case, return SUCCESS on the second call - success_response = {"factorResult": "SUCCESS"} - - # Mock poll_verify_push to return SUCCESS after some calls - mocker.patch.object(client, "poll_verify_push", return_value=success_response) - - # Call the function - _, outputs, _ = verify_push_factor_command(client, args) - - # Verify poll_verify_push was called with correct arguments - client.poll_verify_push.assert_called_once_with( - "https://test.com/api/v1/users/TestID/factors/FactorID/transactions/TransactionID", polling_time, max_polling_calls - ) - - # Verify the result - assert outputs.get("Account(val.ID && val.ID === obj.ID)").get("ID") == "TestID" - assert outputs.get("Account(val.ID && val.ID === obj.ID)").get("VerifyPushResult") == expected_result - - -def test_verify_push_factor_command_no_polling(mocker): - """ - Given: - - Arguments for verify_push_factor_command with polling set to False - When: - - Running verify_push_factor_command - Then: - - Ensure that the polling URL is returned in the outputs and poll_verify_push is not called. - """ - args = {"userId": "TestID", "factorId": "FactorID", "polling": "false"} - mocker.patch.object(client, "verify_push_factor", return_value=verify_push_factor_response) - mock_poll = mocker.patch.object(client, "poll_verify_push") - - readable, outputs, raw_response = verify_push_factor_command(client, args) - - assert ( - outputs.get("Okta.PollingStatusURL(true)") - == "https://test.com/api/v1/users/TestID/factors/FactorID/transactions/TransactionID" - ) - assert "Push factor challenge has been initiated. To check the push factor challenge status" in readable - mock_poll.assert_not_called() - - -@pytest.mark.parametrize( - "args", - [({"firstName": "Testush", "lastName": "Test", "email": "test@this.com", "login": "test@this.com", "password": "Aa123456"})], -) -def test_create_user_command(mocker, args): - mocker.patch.object(client, "create_user", return_value=create_user_response) - readable, outputs, _ = create_user_command(client, args) - assert "STAGED" in readable - assert outputs.get("Account(val.ID && val.ID === obj.ID)")[0].get("Status") == "STAGED" - - -@pytest.mark.parametrize("args", [({"name": "TestGroup", "description": "Test Group Description"})]) -def test_create_group_command(mocker, args): - mocker.patch.object(client, "create_group", return_value=create_group_response) - readable, outputs, _ = create_group_command(client, args) - assert outputs.get("OktaGroup(val.ID && val.ID === obj.ID)")[0].get("Type") == "OKTA_GROUP" - - -@pytest.mark.parametrize("args", [({"groupName": "TestGroup", "appName": "TestApp"})]) -def test_assign_group_to_app_command(mocker, args): - mocker.patch.object(client, "get_group_id", return_value="00g3q8tjdyoOw6fJE1d7") - mocker.patch.object(client, "get_app_id", return_value="a456appid654") - mocker.patch.object(client, "assign_group_to_app", return_value=assign_group_to_app_response) - readable, outputs, _ = assign_group_to_app_command(client, args) - assert _.get("id") == "00g3q8tjdyoOw6fJE1d7" - - -@pytest.mark.parametrize( - "args, expected", - [ - ( - {"groupId": "Test Group", "limit": 5}, - { - "ID": "TestID2", - "Username": "john@doe.com", - "DisplayName": "Test2 Test2", - "Email": "john@doe.com", - "Status": "STAGED", - "Type": "Okta", - "Created": "2018-07-24T20:20:04.000Z", - }, - ) - ], -) -def test_get_group_members_command(mocker, args, expected): - mocker.patch.object(client, "get_group_members", return_value=group_members) - readable, outputs, _ = get_group_members_command(client, args) - assert "Test Group" in readable - assert expected == outputs.get("Account(val.ID && val.ID === obj.ID)")[1] - - -def test_get_logs_command(mocker): - mocker.patch.object(client, "get_logs", return_value=logs) - readable, outputs, _ = get_logs_command(client, {}) - assert logs == outputs.get("Okta.Logs.Events(val.uuid && val.uuid === obj.uuid)") - assert "Unknown browser on Unknown OS Unknown device" in readable - assert "Chrome on Windows Computer" in readable - - -@pytest.mark.parametrize("args", [({"zoneID": "nzoqsmcx1qWYJ6wYF7q0"})]) -def test_get_zone_command(mocker, args): - mocker.patch.object(client, "get_zone", return_value=okta_zone) - readable, outputs, _ = get_zone_command(client, args) - assert "Test Zone" in readable - assert outputs.get("Okta.Zone(val.id && val.id === obj.id)").get("id", "") == "nzoqsmcx1qWYJ6wYF7q0" - - -def test_list_zones_command(mocker): - mocker.patch.object(client, "list_zones", return_value=okta_zone) - readable, outputs, _ = list_zones_command(client, {}) - assert "Test Zone" in readable - assert outputs.get("Okta.Zone(val.id && val.id === obj.id)").get("id", "") == "nzoqsmcx1qWYJ6wYF7q0" +Full behavioural coverage lives in the OktaApiModule tests; this file +exists only to confirm the shim wires through to the ApiModule correctly. +""" +from unittest.mock import patch # noqa: F401 -@pytest.mark.parametrize("args", [({"zoneID": "nzoqsmcx1qWYJ6wYF7q0", "zoneName": "NewZoneName"})]) -def test_update_zone_command(mocker, args): - my_okta_zone = okta_zone - my_okta_zone["name"] = "NewZoneName" - mocker.patch.object(client, "get_zone", return_value=okta_zone) - mocker.patch.object(client, "update_zone", return_value=my_okta_zone) - readable, outputs, _ = update_zone_command(client, args) - assert outputs.get("Okta.Zone(val.id && val.id === obj.id)").get("name", "") == "NewZoneName" - - -@pytest.mark.parametrize("args", [({"gateway_ips": "8.8.8.8", "name": "NewZoneName"})]) -def test_create_zone_command(mocker, args): - my_okta_zone = okta_zone - my_okta_zone["name"] = "NewZoneName" - mocker.patch.object(client, "create_zone", return_value=okta_zone) - readable, outputs, _ = create_zone_command(client, args) - assert outputs.get("Okta.Zone(val.id && val.id === obj.id)").get("name", "") == "NewZoneName" - - -EXPEXTED_LOGS_RESULT = [ - { - "Actor": "dummy name (User)", - "ActorAlternaneId": "example", - "EventInfo": "Remove user from group membership", - "EventOutcome": "SUCCESS", - "EventSeverity": "INFO", - "Client": "Unknown browser on Unknown OS Unknown device", - "RequestIP": "8.8.8.8", - "ChainIP": ["8.8.8.8"], - "Targets": "test this (User)\ntest1 (UserGroup)\n", - "Time": "12/13/2021, 01:47:08", - }, - { - "Actor": "dummy name (User)", - "ActorAlternaneId": "example", - "EventInfo": "Remove user from group membership", - "EventOutcome": "SUCCESS", - "EventSeverity": "INFO", - "Client": "Unknown browser on Unknown OS Unknown device", - "RequestIP": "8.8.8.8", - "ChainIP": [], - "Targets": "test this (User)\ntest1 (UserGroup)\n", - "Time": "12/13/2021, 01:47:08", - }, -] - - -def test_get_readable_logs(): - logs_raw_response = util_load_json("test_data/get_logs_response.json") - result = client.get_readable_logs(logs_raw_response) - assert len(result) == 2 - assert result == EXPEXTED_LOGS_RESULT - - -def test_set_password_command(): - client = Client(base_url="https://demisto.com", api_token="XXX") - headers = { - "x-rate-limit-limit": "1", - "x-rate-limit-remaining": "1", - "x-rate-limit-reset": "1", - } - with requests_mock.Mocker() as m: - m.get('https://demisto.com/api/v1/users?filter=profile.login eq "test"', json=[{"id": "1234"}], headers=headers) - mock_request = m.post( - "https://demisto.com/api/v1/users/1234", json={"passwordChanged": "2020-03-26T13:57:13.000Z"}, headers=headers - ) - - result = set_password_command(client, {"username": "test", "password": "a1b2c3"}) - - assert result[0] == "test password was last changed on 2020-03-26T13:57:13.000Z" - assert mock_request.last_request.text == '{"credentials": {"password": {"value": "a1b2c3"}}}' - - -def test_set_temp_password_command(): - client = Client(base_url="https://demisto.com", api_token="XXX") - headers = { - "x-rate-limit-limit": "1", - "x-rate-limit-remaining": "1", - "x-rate-limit-reset": "1", - } - with requests_mock.Mocker() as m: - m.get('https://demisto.com/api/v1/users?filter=profile.login eq "test"', json=[{"id": "1234"}], headers=headers) - m.post("https://demisto.com/api/v1/users/1234", json={"passwordChanged": "2023-03-22T10:15:26.000Z"}, headers=headers) - m.post( - "https://demisto.com/api/v1/users/1234/lifecycle/expire_password?tempPassword=true", - json={"tempPassword": "cAn5N3gx"}, - headers=headers, - ) - - result = set_password_command(client, {"username": "test", "password": "a1b2c3", "temporary_password": "true"}) - expected_results = ( - "test password was last changed on 2023-03-22T10:15:26.000Z\n" - "### Okta Temporary Password\n|tempPassword|\n|---|\n| cAn5N3gx |\n" - ) - assert result[0] == expected_results - - -def mock_get_paged_results(url_suffix="", query_params=None, max_limit=None): - if max_limit: - return LOGS[:max_limit] - else: - return LOGS - - -LOGS_WITH_LIMIT = [(None, 3), (1, 1), (3, 3), (1001, 3)] - - -@pytest.mark.parametrize("limit, logs_amount", LOGS_WITH_LIMIT) -def test_get_logs_command_with_limit(mocker, requests_mock, limit, logs_amount): - """ - Given: - - An Okta IAM client object. - When: - - Calling function okta-get-logs - - Events should come in two batches of two events in the first batch, and one event in the second batch. - Then: - - Ensure three events are returned in incident the correct format. - """ - from Okta_v2 import get_logs_command - - client = Client(base_url="https://demisto.com", api_token="XXX") - mocker.patch.object(Client, "get_paged_results", side_effect=mock_get_paged_results) - mocker.patch.object(Client, "get_readable_logs", side_effect=mock_get_paged_results) - requests_mock.get( - f"https://demisto.com/api/v1/logs?limit={limit}", - json=LOGS[:limit], - headers={ - "x-rate-limit-limit": "1", - "x-rate-limit-remaining": "1", - "x-rate-limit-reset": "1", - }, - ) - args = {"limit": limit} - readable, outputs, raw_response = get_logs_command(client=client, args=args) - assert len(outputs.get("Okta.Logs.Events(val.uuid && val.uuid === obj.uuid)")) == logs_amount - - -def test_expire_password_with_revoke_session(): - """ - Given: - - A client object with mocked methods for getting user ID, revoking a session, and formatting the user context. - - Arguments for expire_password_command with username, hide_password set to False, and revoke_session set to True. - When: - - Calling expire_password_command with revoke_session set to True. - Then: - - Ensure the revoke_session method is called. - - Ensure the response includes the correct tempPassword and user context. - - Ensure the readable output is formatted as expected. - """ - from Okta_v2 import expire_password_command - - client = MagicMock() - args = {"username": "test_user", "hide_password": "False", "revoke_session": "True"} - - client.get_user_id.return_value = "user123" - client.revoke_session.return_value = {"tempPassword": "test_password"} - client.get_users_context.return_value = {"ID": "user123"} - - readable_output, outputs, raw_response = expire_password_command(client, args) - - client.revoke_session.assert_called_once_with("user123") - assert "Account(val.ID && val.ID === obj.ID)" in outputs - assert outputs["Account(val.ID && val.ID === obj.ID)"]["ID"] == "user123" - assert "test_password" in raw_response["tempPassword"] - assert readable_output == "### Okta Expired Password\n|tempPassword|\n|---|\n| test_password |\n" - - -def test_expire_password_without_revoke_session(): - """ - Given: - - Arguments for expire_password_command with username, hide_password set to False, and revoke_session set to False. - When: - - Calling expire_password_command. - Then: - - Ensure the expire_password method is called. - - Ensure the response includes the correct tempPassword and user context. - - Ensure the readable output is formatted as expected. - """ - from Okta_v2 import expire_password_command - - client = MagicMock() - args = {"username": "test_user", "hide_password": "False", "revoke_session": "False"} - - client.get_user_id.return_value = "user123" - client.expire_password.return_value = {"tempPassword": "test_password"} - client.get_users_context.return_value = {"ID": "user123"} - - readable_output, outputs, raw_response = expire_password_command(client, args) - - client.expire_password.assert_called_once_with("user123", args) - assert "Account(val.ID && val.ID === obj.ID)" in outputs - assert outputs["Account(val.ID && val.ID === obj.ID)"]["ID"] == "user123" - assert "test_password" in raw_response["tempPassword"] - assert readable_output == "### Okta Expired Password\n|tempPassword|\n|---|\n| test_password |\n" - - -def test_hide_password(): - """ - Given: - - Arguments for expire_password_command with username, hide_password set to True, and revoke_session set to False. - When: - - Calling expire_password_command. - Then: - - Ensure the tempPassword in the response is hidden. - - Ensure the readable output is formatted to indicate the password is hidden. - """ - from Okta_v2 import expire_password_command - - client = MagicMock() - args = {"username": "test_user", "hide_password": "True", "revoke_session": "False"} - - client.get_user_id.return_value = "user123" - client.expire_password.return_value = {"tempPassword": "test_password"} - client.get_users_context.return_value = {"ID": "user123"} - - readable_output, outputs, raw_response = expire_password_command(client, args) - - assert raw_response["tempPassword"] == "Output removed by user. hide_password argument set to True" - assert readable_output == ( - "### Okta Expired Password\n|tempPassword|\n|---|\n| Output removed by user. hide_password argument set to True |\n" - ) - - -def test_show_password(): - """ - Given: - - Arguments for expire_password_command with username, hide_password set to False, and revoke_session set to False. - When: - - Calling expire_password_command. - Then: - - Ensure the tempPassword in the response is shown. - - Ensure the readable output displays the tempPassword. - """ - from Okta_v2 import expire_password_command - - client = MagicMock() - args = {"username": "test_user", "hide_password": "False", "revoke_session": "False"} - - client.get_user_id.return_value = "user123" - client.expire_password.return_value = {"tempPassword": "test_password"} - client.get_users_context.return_value = {"ID": "user123"} - - readable_output, outputs, raw_response = expire_password_command(client, args) - - assert raw_response["tempPassword"] == "test_password" - assert readable_output == "### Okta Expired Password\n|tempPassword|\n|---|\n| test_password |\n" - - -def test_missing_username_and_user_id(): - """ - Given: - - Arguments for expire_password_command with hide_password and revoke_session but no username or user ID. - When: - - Calling expire_password_command without providing a username or user ID. - Then: - - Ensure an exception is raised indicating that either a username or user ID must be provided. - """ - from Okta_v2 import expire_password_command - - client = MagicMock() - args = {"hide_password": "False", "revoke_session": "False"} - - client.get_user_id.return_value = None - - try: - expire_password_command(client, args) - except Exception as e: - assert "You must supply either 'Username' or 'userId" in str(e) - - -@patch.object(Client, "http_request") -def test_revoke_session(mock_http_request): - """ - Given: - - A valid user ID. - When: - - Calling revoke_session with a user ID. - Then: - - Ensure http_request is called with the correct method, URL, and parameters. - """ - user_id = "12345" - expected_uri = f"/api/v1/users/{user_id}/lifecycle/expire_password_with_temp_password" - expected_params = {"revokeSessions": "true"} - client.revoke_session(user_id) - mock_http_request.assert_called_once_with(method="POST", url_suffix=expected_uri, params=expected_params) - - -def test_apply_zone_update_override(): - """ - Given: empty ZoneObeject - When: apply_zone_updates is called with "OVERRIDE" update type and specific gatewayIPs and proxyIPs, - Then: the zoneObject should be updated with the provided data. - """ - zoneObject = {} - updated_zone = apply_zone_updates( - zoneObject, "TestZone", ["192.168.1.1", "10.0.0.0-10.0.0.10"], ["10.0.0.1", "192.168.1.0-192.168.1.5"], "OVERRIDE" - ) - assert updated_zone == { - "name": "TestZone", - "gateways": [{"type": "CIDR", "value": "192.168.1.1/32"}, {"type": "RANGE", "value": "10.0.0.0-10.0.0.10"}], - "proxies": [{"type": "CIDR", "value": "10.0.0.1/32"}, {"type": "RANGE", "value": "192.168.1.0-192.168.1.5"}], - } - - -def test_apply_zone_update_append_with_range(): - """ - Given: zoneObject already with some gateways, - When: apply_zone_updates is called with "APPEND" update type and IP ranges, - Then: the new gateways with IP ranges should be appended to the existing ones in the zoneObject. - """ - zoneObject = {"gateways": [{"type": "CIDR", "value": "192.168.1.1/32"}]} - updated_zone = apply_zone_updates( - zoneObject, "NewZone", ["192.168.1.2-192.168.1.10"], ["10.0.0.1", "10.0.0.2-10.0.0.5"], "APPEND" - ) - assert updated_zone == { - "name": "NewZone", - "gateways": [{"type": "CIDR", "value": "192.168.1.1/32"}, {"type": "RANGE", "value": "192.168.1.2-192.168.1.10"}], - "proxies": [{"type": "CIDR", "value": "10.0.0.1/32"}, {"type": "RANGE", "value": "10.0.0.2-10.0.0.5"}], - } - - -@pytest.mark.parametrize( - "group_name, mock_response, expected_id", - [ - ( - "Existing Group", - [{"id": "123", "profile": {"name": "Existing Group"}}], - "123", - ), - ( - "Non-existent Group", - [{"id": "123", "profile": {"name": "Existing Group"}}], - None, - ), - ( - "Empty Response", - [], - None, - ), - ( - "Multiple Groups", - [ - {"id": "123", "profile": {"name": "Group A"}}, - {"id": "456", "profile": {"name": "Multiple Groups"}}, - ], - "456", - ), - ], -) -def test_get_group_id(mocker, group_name, mock_response, expected_id): - """ - Given: - - A group name to search for. - When: - - Calling the get_group_id method. - Then: - - Ensure the correct group ID is returned if found, otherwise None. - """ - mocker.patch.object(client, "http_request", return_value=mock_response) - mocker.patch("Okta_v2.encode_string_results", side_effect=lambda x: x) # Mock encode_string_results - - group_id = client.get_group_id(group_name) - assert group_id == expected_id - - -def test_verify_mfa_status_command(mocker): - """ - Given: - - polling_url. - - When: - - running verify_mfa_status_command. - - Then: - - Ensure that the status is returned correctly. - """ - args = {"polling_url": "https://test.com/api/v1/users/TestID/factors/FactorID/transactions/TransactionID"} - mock_response = {"id": "TransactionID", "factorResult": "SUCCESS"} - mocker.patch.object(client, "get_push_factor_status", return_value=mock_response) - - readable, outputs, raw_response = verify_mfa_status_command(client, args) - - assert "The status of the push factor challenge is SUCCESS" in readable - assert outputs.get("Okta.FactorResult(val.ID && val.ID === obj.ID)").get("factorResult") == "SUCCESS" - assert outputs.get("Okta.FactorResult(val.ID && val.ID === obj.ID)").get("ID") == "FactorID" - assert raw_response == mock_response +import pytest +import Okta_v2 as integration_module -@pytest.mark.parametrize( - "url, expected_user_id, expected_factor_id", - [ - ("https://example.okta.com/api/v1/users/user-id/factors/factor-id", "user-id", "factor-id"), - ("https://example.okta.com/api/v1/users/U123/factors/F456/transactions/T789", "U123", "F456"), - ("http://localhost/users/me/factors/my-factor", "me", "my-factor"), - ], -) -def test_extract_user_and_factor_id_from_url_success(url, expected_user_id, expected_factor_id): - """ - Given: - - A valid polling URL containing user and factor IDs. - When: - - Calling extract_user_and_factor_id_from_url. - Then: - - Ensure the user ID and factor ID are correctly extracted. - """ - user_id, factor_id = extract_user_and_factor_id_from_url(url) - assert user_id == expected_user_id - assert factor_id == expected_factor_id +def test_shim_imports_run_entry_point(): + """The shim must re-export the ApiModule entry point under its expected name.""" + assert hasattr( + integration_module, "run_okta_v2_integration" + ), "OktaApiModule.run_okta_v2_integration must be importable via the shim" -@pytest.mark.parametrize( - "url", - [ - "https://example.okta.com/api/v1/users/user-id", - "https://example.okta.com/api/v1/factors/factor-id", - "https://example.okta.com/api/v1/something/else", - "", - ], -) -def test_extract_user_and_factor_id_from_url_failure(url): - """ - Given: - - An invalid polling URL. - When: - - Calling extract_user_and_factor_id_from_url. - Then: - - Ensure a DemistoException is raised. - """ - from CommonServerPython import DemistoException +def test_shim_imports_client(): + """The shim must re-export the Okta v2 Client class and its OktaClient base.""" + assert hasattr(integration_module, "Client") + assert hasattr(integration_module, "OktaClient") + assert issubclass(integration_module.Client, integration_module.OktaClient) - with pytest.raises(DemistoException, match="Could not extract user ID and Factor ID from the polling URL"): - extract_user_and_factor_id_from_url(url) +def test_shim_imports_auth_symbols(): + """Auth helpers used by the UCP flow must remain accessible from the shim.""" + for name in ("AuthType", "JWTAlgorithm", "resolve_ucp_auth_type", "reset_integration_context"): + assert hasattr(integration_module, name), f"Symbol {name!r} missing from shim" -def test_clear_user_sessions_with_oauth_tokens(mocker): - """ - Given: - - Arguments for clear_user_sessions_command with revokeOauthTokens set to true. - When: - - Running clear_user_sessions_command. - Then: - - Ensure the clear_user_sessions method is called with revoke_oauth_tokens=True. - - Ensure the API is called with the oauthTokens query parameter. - """ - mock_http_request = mocker.patch.object(client, "http_request", return_value="") - client.request_metadata = {} - args = {"userId": "TestUserID456", "revokeOauthTokens": "true"} - readable_output, outputs, raw_response = clear_user_sessions_command(client, args) +def test_shim_imports_constants(): + """Module-level constants used by playbooks/tests must remain accessible from the shim.""" + for name in ("DATE_FORMAT", "OAUTH_TOKEN_SCOPES", "PROFILE_ARGS", "GROUP_PROFILE_ARGS"): + assert hasattr(integration_module, name), f"Constant {name!r} missing from shim" - mock_http_request.assert_called_once_with( - method="DELETE", - url_suffix="/api/v1/users/TestUserID456/sessions", - params={"oauthTokens": True}, - resp_type="text", - ) - assert "TestUserID456" in readable_output +def test_main_delegates_to_api_module(mocker): + """`main()` must do nothing other than call the renamed ApiModule entry point.""" + mock_run = mocker.patch("Okta_v2.run_okta_v2_integration") + integration_module.main() + mock_run.assert_called_once_with() -def test_clear_user_sessions_without_oauth_tokens(mocker): - """ - Given: - - Arguments for clear_user_sessions_command with revokeOauthTokens set to false. - When: - - Running clear_user_sessions_command. - Then: - - Ensure the clear_user_sessions method is called with revoke_oauth_tokens=False. - - Ensure the API is called without the oauthTokens query parameter. - """ - mock_http_request = mocker.patch.object(client, "http_request", return_value="") - client.request_metadata = {} - args = {"userId": "TestUserID789", "revokeOauthTokens": "false"} - readable_output, outputs, raw_response = clear_user_sessions_command(client, args) +def test_main_propagates_exceptions(mocker): + """`main()` is a thin shim and must not swallow exceptions from the ApiModule.""" + mocker.patch("Okta_v2.run_okta_v2_integration", side_effect=RuntimeError("boom")) - mock_http_request.assert_called_once_with( - method="DELETE", - url_suffix="/api/v1/users/TestUserID789/sessions", - params={"oauthTokens": False}, - resp_type="text", - ) - assert "TestUserID789" in readable_output + with pytest.raises(RuntimeError, match="boom"): + integration_module.main() diff --git a/Packs/Okta/ModelingRules/OktaModelingRules_2_0/OktaModelingRules_2_0.yml b/Packs/Okta/ModelingRules/OktaModelingRules_2_0/OktaModelingRules_2_0.yml index 30715a3637ae..96e6aaeb6509 100644 --- a/Packs/Okta/ModelingRules/OktaModelingRules_2_0/OktaModelingRules_2_0.yml +++ b/Packs/Okta/ModelingRules/OktaModelingRules_2_0/OktaModelingRules_2_0.yml @@ -1,4 +1,5 @@ fromversion: 8.4.0 +toversion: 8.14.0 id: Okta_2_0_ModelingRule name: Okta 2.0 Modeling Rule rules: '' diff --git a/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11.xif b/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11.xif new file mode 100644 index 000000000000..b65db0d27955 --- /dev/null +++ b/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11.xif @@ -0,0 +1,312 @@ + +[MODEL: dataset=okta_okta_raw] +filter + eventType in ("app.ad.api.user_import.account_locked","app.app_instance.csr.generate","app.app_instance.csr.publish","app.app_instance.csr.revoke","app.app_instance.provision_sync_job.completed","app.app_instance.provision_sync_job.failed","app.app_instance.provision_sync_job.started","app.audit_report.download.local.active","app.audit_report.download.local.deprov","app.audit_report.download.rogue.report","app.generic.unauth_app_access_attempt","app.inbound_del_auth.login_success","app.keys.clone","app.keys.generate","app.keys.rotate","app.ldap.password.change.failed","app.office365.api.change.domain.federation.success","app.office365.api.error.ad.user","app.office365.api.error.check.user.exists","app.office365.api.error.create.user","app.office365.api.error.deactivate.user","app.office365.api.error.download.custom.objects","app.office365.api.error.download.groups","app.office365.api.error.download.users","app.office365.api.error.endpoint.unavailable","app.office365.api.error.get.company.dirsync.failure","app.office365.api.error.get.company.dirsync.status.failure","app.office365.api.error.get.company.dirsync.status.pending","app.office365.api.error.get.object.ids.by.group.id","app.office365.api.error.group.create.failure","app.office365.api.error.group.create.failure.name.in.use","app.office365.api.error.group.delete.failure","app.office365.api.error.group.membership.update.assignment.failure","app.office365.api.error.group.membership.update.failure","app.office365.api.error.group.membership.update.group.not.found.failure","app.office365.api.error.group.membership.update.removal.failure","app.office365.api.error.group.update.failure","app.office365.api.error.group.update.failure.not.found","app.office365.api.error.import.profile","app.office365.api.error.no.endpoints.found","app.office365.api.error.push.password","app.office365.api.error.push.profile","app.office365.api.error.reactivate.user","app.office365.api.error.remove.domain.federation.failure","app.office365.api.error.remove.domain.federation.failure.access.denied","app.office365.api.error.remove.domain.federation.failure.domain.not.found","app.office365.api.error.revoke.refresh.token","app.office365.api.error.set.company.dirsync.failure","app.office365.api.error.set.company.dirsync.status.failure","app.office365.api.error.set.domain.federation.failure","app.office365.api.error.set.domain.federation.failure.access.denied","app.office365.api.error.set.domain.federation.failure.domain.default","app.office365.api.error.set.domain.federation.failure.domain.not.found","app.office365.api.error.sync.contact","app.office365.api.error.sync.finalize","app.office365.api.error.sync.group","app.office365.api.error.sync.not.activated","app.office365.api.error.sync.set.attribute","app.office365.api.error.sync.user","app.office365.api.error.unable.to.create.graph.client","app.office365.api.error.validate.admin.creds","app.office365.api.error.validate.creds","app.office365.api.error.validate.creds.unknown.exception","app.office365.api.error.x-ms-forwarded-client-ip-header.absent","app.office365.api.remove.domain.federation.success","app.office365.api.set.domain.federation.success","app.office365.api.sync.complete","app.office365.api.sync.heartbeat.sent","app.office365.api.sync.job.complete","app.office365.api.sync.job.complete.contact","app.office365.api.sync.job.complete.group","app.office365.api.sync.job.complete.user","app.office365.clientplatform.conversion.job.processing.app.instance","app.office365.clientplatform.conversion.job.skipping.migration","app.office365.dirsync.skipping.conflict-object","app.office365.dirsync.skipping.critical-system-object","app.office365.dirsync.skipping.non-security-group-invalid-mail","app.office365.dirsync.skipping.reserved-attribute-value","app.office365.dirsync.skipping.systemmailbox","app.office365.dirsync.skipping.without-name-and-displayname","app.office365.error.importing.user","app.office365.graph.api.error.no.mailbox.found","app.office365.graph.api.error.rate-limit.exceeded","app.office365.graph.api.error.service.principal.creation.failed","app.office365.service.principal.cleanup.job.complete","app.office365.service.principal.cleanup.job.invalid.credentials","app.office365.service.principal.cleanup.job.processing","app.office365.service.principal.cleanup.job.skipping.missing.creds","app.office365.service.principal.cleanup.job.skipping.no.service.principal","app.office365.service.principal.cleanup.job.unable.to.delete.service.principal","app.office365.user.delete.success","app.office365.user.lifecycle.action.failed","app.office365.user.remove.licenses.success","app.radius.agent.listener.failed","app.radius.agent.listener.succeeded","app.radius.agent.port_inaccessible","app.radius.agent.port_reaccessible","app.radius.info_access.no_permission","app.radius.info_access.partial_permission","app.realtimesync.import.details.add_user","app.realtimesync.import.details.delete_user","app.realtimesync.import.details.update_user","app.saml.sensitive.attribute.update","app.user_management.grouppush.mapping.created.from.rule","app.user_management.grouppush.mapping.created.from.rule.error.duplicate","app.user_management.grouppush.mapping.created.from.rule.error.validation","app.user_management.grouppush.mapping.created.from.rule.errors","app.user_management.grouppush.mapping.okta.users.ignored","app.user_management.push_new_user_success","app.user_management.update_from_master_failed","app.user_management.user_group_import.create_failure","app.user_management.user_group_import.delete_success","app.user_management.user_group_import.update_failure","app.user_management.user_group_import.upsert_fail","app.user_management.user_group_import.upsert_success","application.configuration.disable_delauth_outbound","application.configuration.disable_fed_broker_mode","application.configuration.enable_delauth_outbound","application.configuration.enable_fed_broker_mode","application.configuration.reset_logo","application.configuration.update","application.configuration.update_api_credentials_for_pass_change","application.configuration.update_logo","application.lifecycle.activate","application.lifecycle.create","application.lifecycle.deactivate","application.lifecycle.delete","application.lifecycle.update","application.policy.sign_on.deny_access","application.policy.sign_on.rule.create","application.policy.sign_on.rule.delete","application.policy.sign_on.update","application.provision.field_mapping_rule.change","application.provision.group.add","application.provision.group.import","application.provision.group.remove","application.provision.group.update","application.provision.group.verify_exists","application.provision.group_membership.add","application.provision.group_membership.import","application.provision.group_membership.remove","application.provision.group_membership.update","application.provision.group_push.activate_mapping","application.provision.group_push.delete_appgroup","application.provision.group_push.mapping.and.groups.deleted.rule.deleted","application.provision.group_push.mapping.app.group.renamed","application.provision.group_push.mapping.app.group.renamed.failed","application.provision.group_push.mapping.created","application.provision.group_push.mapping.created.from.rule.warning.duplicate.name","application.provision.group_push.mapping.created.from.rule.warning.duplicate.name.tobecreated","application.provision.group_push.mapping.created.from.rule.warning.upsertGroup.duplicate.name","application.provision.group_push.mapping.deactivated.source.group.renamed","application.provision.group_push.mapping.deactivated.source.group.renamed.failed","application.provision.group_push.mapping.update.or.delete.failed","application.provision.group_push.mapping.update.or.delete.failed.with.error","application.provision.group_push.push_memberships","application.provision.group_push.pushed","application.provision.group_push.removed","application.provision.group_push.updated","application.provision.integration.call_api","application.provision.user.activate","application.provision.user.deactivate","application.provision.user.deprovision","application.provision.user.import","application.provision.user.import_profile","application.provision.user.password","application.provision.user.push","application.provision.user.push_okta_password","application.provision.user.push_password","application.provision.user.push_profile","application.provision.user.reactivate","application.provision.user.sync","application.provision.user.verify_exists","application.registration_policy.lifecycle.create","application.registration_policy.lifecycle.update","application.user_membership.add","application.user_membership.approve","application.user_membership.change_password","application.user_membership.change_username","application.user_membership.deprovision","application.user_membership.provision","application.user_membership.remove","application.user_membership.restore","application.user_membership.restore_password","application.user_membership.revoke","application.user_membership.show_password","application.user_membership.update","credential.register","credential.revoke","device.enrollment.create","device.lifecycle.activate","device.lifecycle.deactivate","device.lifecycle.delete","device.lifecycle.suspend","device.lifecycle.unsuspend","device.platform.add","device.platform.delete","device.platform.update","device.user.add","device.user.remove","directory.app_user_profile.update","directory.mapping.update","directory.non_default_user_profile.create","directory.user_profile.update","event_hook.activated","event_hook.created","event_hook.deactivated","event_hook.deleted","event_hook.delivery","event_hook.updated","event_hook.verified","group.application_assignment.add","group.application_assignment.remove","group.application_assignment.update","group.lifecycle.create","group.lifecycle.delete","group.privilege.grant","group.privilege.revoke","group.profile.update","group.user_membership.add","group.user_membership.remove","group.user_membership.rule.add_exclusion","group.user_membership.rule.deactivated","iam.resourceset.bindings.add","iam.resourceset.bindings.delete","iam.resourceset.create","iam.resourceset.delete","iam.resourceset.resources.add","iam.resourceset.resources.delete","iam.role.create","iam.role.delete","iam.role.permissions.add","iam.role.permissions.delete","inline_hook.activated","inline_hook.created","inline_hook.deactivated","inline_hook.deleted","inline_hook.executed","master_application.user_membership.add","mim.createEnrollment.ANDROID","mim.createEnrollment.IOS","mim.createEnrollment.OSX","mim.createEnrollment.UNKNOWN","mim.createEnrollment.WINDOWS","network_zone.rule.disabled","omm.app.VPN.settings.changed","omm.app.WIFI.settings.changed","omm.app.eas.cert_based.settings.changed","omm.app.eas.disabled","omm.app.eas.settings.changed","omm.cma.created","omm.cma.deleted","omm.cma.updated","omm.enrollment.changed","pki.ca.add","pki.ca.delete","pki.cert.bind","pki.cert.issue","pki.cert.lifecycle.activate","pki.cert.lifecycle.delete","pki.cert.lifecycle.hold","pki.cert.lifecycle.revoke","pki.cert.lifecycle.suspend","pki.cert.renew","pki.cert.revoke","policy.evaluate_sign_on","policy.execute.user.start","policy.lifecycle.activate","policy.lifecycle.create","policy.lifecycle.deactivate","policy.lifecycle.delete","policy.lifecycle.overwrite","policy.lifecycle.update","policy.mapping.create","policy.rule.action.execute","policy.rule.activate","policy.rule.add","policy.rule.deactivate","policy.rule.delete","policy.rule.invalidate","policy.rule.update","policy.scheduled.execute","scheduled_action.user_suspension.canceled","scheduled_action.user_suspension.completed","scheduled_action.user_suspension.scheduled","scheduled_action.user_suspension.updated","security.authenticator.lifecycle.activate","security.authenticator.lifecycle.deactivate","security.device.add_request_blacklist_policy","security.device.remove_request_blacklist_policy","security.device.temporarily_disable_blacklisting","security.request.blocked","security.threat.configuration.update","security.threat.detected","security.voice.add_country_blacklist","security.voice.remove_country_blacklist","security.zone.make_blacklist","security.zone.remove_blacklist","self_service.disabled","self_service.enabled","system.agent.ad.connect","system.agent.ad.create","system.agent.ad.deactivate","system.agent.ad.delete","system.agent.ad.import_ou","system.agent.ad.import_user","system.agent.ad.invoke_dir","system.agent.ad.reactivate","system.agent.ad.read_dirsync","system.agent.ad.read_ldap","system.agent.ad.read_schema","system.agent.ad.read_topology","system.agent.ad.realtimesync","system.agent.ad.reset_user_password","system.agent.ad.start","system.agent.ad.unlock_user_account","system.agent.ad.update","system.agent.ad.update_user","system.agent.ad.upgrade","system.agent.ad.upload_iwa_log","system.agent.ad.upload_log","system.agent.ad.write_ldap","system.agent.auto_update","system.agent.connector.connect","system.agent.connector.deactivate","system.agent.connector.delete","system.agent.connector.reactivate","system.agent.ldap.change_user_password","system.agent.ldap.create_user_JIT","system.agent.ldap.disconnect","system.agent.ldap.reconnect","system.agent.ldap.reset_user_password","system.agent.ldap.unlock_user_account","system.agent_pools.auto_update","system.api_token.create","system.api_token.enable","system.api_token.revoke","system.brand.update","system.captcha.create","system.captcha.delete","system.captcha.update","system.custom_error.update","system.custom_signin.update","system.custom_url_domain.cert_renew","system.custom_url_domain.cert_upload","system.custom_url_domain.delete","system.custom_url_domain.initiate","system.directory.debugger.extend","system.directory.debugger.grant","system.directory.debugger.query_executed","system.directory.debugger.revoke","system.email.template.create","system.email.template.delete","system.email.template.update","system.idp.lifecycle.activate","system.idp.lifecycle.create","system.idp.lifecycle.deactivate","system.idp.lifecycle.delete","system.idp.lifecycle.update","system.import.clear.unconfirmed.users.summary","system.import.complete","system.import.complete_batch","system.import.custom_object.complete","system.import.custom_object.create","system.import.custom_object.delete","system.import.custom_object.update","system.import.download.complete","system.import.download.start","system.import.group.complete","system.import.group.create","system.import.group.delete","system.import.group.start","system.import.group.update","system.import.group_membership.complete","system.import.implicit_deletion.complete","system.import.implicit_deletion.start","system.import.import_profile","system.import.import_provisioning_info","system.import.membership_processing.complete","system.import.membership_processing.start","system.import.object_creation.complete","system.import.object_creation.start","system.import.roadblock","system.import.roadblock.reschedule_and_resume","system.import.roadblock.resume","system.import.roadblock.updated","system.import.start","system.import.user.complete","system.import.user.create","system.import.user.delete","system.import.user.match","system.import.user.start","system.import.user.suspend","system.import.user.unsuspend","system.import.user.unsuspend_after_confirm","system.import.user.update","system.import.user.update_user_lifecycle_from_master","system.import.user_csv.complete","system.import.user_csv.start","system.import.user_matching.complete","system.import.user_matching.start","system.iwa.create","system.iwa.go_offline","system.iwa.go_online","system.iwa.promote_primary","system.iwa.remove","system.iwa.update","system.iwa.use_default","system.ldapi.bind","system.ldapi.search","system.ldapi.unbind","system.log_stream.lifecycle.activate","system.log_stream.lifecycle.create","system.log_stream.lifecycle.deactivate","system.log_stream.lifecycle.delete","system.log_stream.lifecycle.update","system.mfa.factor.activate","system.mfa.factor.deactivate","system.org.captcha.activate","system.org.captcha.deactivate","system.org.lifecycle.create","system.org.rate_limit.warning","system.org.task.remove","system.theme.update","task.lifecycle.activate","task.lifecycle.create","task.lifecycle.deactivate","task.lifecycle.delete","task.lifecycle.update","user.account.access_super_user_app","user.account.lock","user.account.lock.limit","user.account.privilege.grant","user.account.privilege.revoke","user.account.report_suspicious_activity_by_enduser","user.account.reset_password","user.account.unlock","user.account.unlock_by_admin","user.account.unlock_failure","user.account.unlock_token","user.account.update_password","user.account.update_primary_email","user.account.update_profile","user.account.update_secondary_email","user.account.update_user_type","user.account.use_token","user.credential.enroll","user.identity_snapshot.attestation.create","user.lifecycle.activate","user.lifecycle.create","user.lifecycle.deactivate","user.lifecycle.delete.completed","user.lifecycle.delete.initiated","user.lifecycle.jit.error.read_only","user.lifecycle.password_mass_expiry","user.lifecycle.reactivate","user.lifecycle.suspend","user.lifecycle.unsuspend","user.session.access_admin_app","user.session.clear","user.session.end","user.session.expire","user.session.impersonation.end","user.session.impersonation.extend","user.session.impersonation.grant","user.session.impersonation.initiate","user.session.impersonation.revoke","workflows.user.connection.create","workflows.user.connection.delete","workflows.user.connection.reauthorize","workflows.user.connection.revoke","workflows.user.flow.activate","workflows.user.flow.create","workflows.user.flow.deactivate","workflows.user.flow.delete","workflows.user.flow.export","workflows.user.flow.import","workflows.user.flow.save","workflows.user.table.create","workflows.user.table.delete","workflows.user.table.export","workflows.user.table.import","workflows.user.table.update","workflows.user.table.view","zone.activate","zone.create","zone.deactivate","zone.delete","zone.make_blacklist","zone.remove_blacklist","zone.update","system.push.send_factor_verify_push") +| alter + outcome_result = json_extract_scalar(outcome, "$.result"), + actor_type = json_extract_scalar(actor, "$.type") +| alter + xdm.source.user.identifier = json_extract_scalar(actor, "$.id"), + xdm.source.identity.identifier = json_extract_scalar(actor, "$.id"), + xdm.source.user.user_type = if(actor_type in("User"), XDM_CONST.USER_TYPE_REGULAR, actor_type in("SystemPrincipal"), XDM_CONST.USER_TYPE_SERVICE_ACCOUNT,actor_type in("IP address"), XDM_CONST.USER_TYPE_MACHINE_ACCOUNT, to_string(actor_type)), + xdm.source.identity.user_type = if(actor_type in("User"), XDM_CONST.USER_TYPE_REGULAR, actor_type in("SystemPrincipal"), XDM_CONST.USER_TYPE_SERVICE_ACCOUNT,actor_type in("IP address"), XDM_CONST.USER_TYPE_MACHINE_ACCOUNT, to_string(actor_type)), + xdm.source.user.username = json_extract_scalar(actor, "$.alternateId"), + xdm.source.identity.username = json_extract_scalar(actor, "$.alternateId"), + xdm.target.cloud.zone = json_extract_scalar(client, "$.zone"), + xdm.source.ipv4 = json_extract_scalar(client,"$.ipAddress"), + xdm.target.ipv4="", + xdm.source.port=to_integer(0), + xdm.target.port=to_integer(0), + xdm.source.location.city = json_extract_scalar(client, "$.geographicalContext.city"), + xdm.source.location.country = json_extract_scalar(client, "$.geographicalContext.country"), + xdm.source.location.latitude = to_float(json_extract_scalar(client, "$.geographicalContext.geolocation.lat")), + xdm.source.location.longitude = to_float(json_extract_scalar(client, "$.geographicalContext.geolocation.lon")), + xdm.event.type = eventType, + xdm.event.outcome = if(outcome_result="SUCCESS", XDM_CONST.OUTCOME_SUCCESS, outcome_result="FAILURE", XDM_CONST.OUTCOME_FAILED, outcome_result="ALLOW", XDM_CONST.OUTCOME_SUCCESS, outcome_result="DENY", XDM_CONST.OUTCOME_FAILED, outcome_result="CHALLENGE", XDM_CONST.OUTCOME_PARTIAL, XDM_CONST.OUTCOME_UNKNOWN), + xdm.event.outcome_reason = json_extract_scalar(outcome, "$.reason"), + xdm.target.resource.id = arraystring (arraymap (json_extract_array (`target`,"$."), json_extract_scalar ("@element", "$.alternateId")),","), + xdm.target.resource.type = arraystring (arraymap (json_extract_array (`target`,"$."), json_extract_scalar ("@element", "$.type")),","), + xdm.source.user_agent = json_extract_scalar(client, "$.userAgent.rawUserAgent"), + xdm.source.host.os = json_extract_scalar(client, "$.userAgent.os"), + xdm.source.application.name = json_extract_scalar(client, "$.userAgent.browser"), + xdm.event.id = uuid, + xdm.source.asn.as_name = json_extract_scalar(securityContext,"$.asOrg") + //First and last name extraction + | alter displayName = json_extract_scalar(actor, "$.displayName") + | alter + lastName = arrayindex(regextract(displayName, "\s([\S]*)"),0), + firstName= arrayindex(regextract(displayName, "([\S]*)\s"),0) + | alter + xdm.source.user.first_name = firstName, + xdm.source.identity.first_name = firstName, + xdm.source.user.last_name = lastName, + xdm.source.identity.last_name = lastName; + +filter + eventType in ("user.session.start","user.mfa.okta_verify.deny_push_upgrade_needed","user.mfa.okta_verify.deny_push","user.mfa.okta_verify","user.mfa.factor.update","user.mfa.factor.unsuspend","user.mfa.factor.suspend","user.mfa.factor.reset_all","user.mfa.factor.deactivate","user.mfa.factor.activate","user.mfa.attempt_bypass","app.access_request.approver.approve","app.access_request.approver.deny","app.access_request.delete","app.access_request.deny","app.access_request.expire","app.access_request.grant","app.access_request.request","app.kerberos_rich_client.account_not_found","app.kerberos_rich_client.instance_not_found","app.kerberos_rich_client.multiple_accounts_found","app.kerberos_rich_client.user_authentication_successful","app.oauth2.admin.consent.grant","app.oauth2.admin.consent.revoke","app.oauth2.as.authorize","app.oauth2.as.authorize.code","app.oauth2.as.authorize.implicit.access_token","app.oauth2.as.authorize.implicit.id_token","app.oauth2.as.authorize.scope_denied","app.oauth2.as.consent.grant","app.oauth2.as.consent.revoke","app.oauth2.as.consent.revoke.implicit.as","app.oauth2.as.consent.revoke.implicit.client","app.oauth2.as.consent.revoke.implicit.scope","app.oauth2.as.consent.revoke.implicit.user","app.oauth2.as.consent.revoke.user","app.oauth2.as.consent.revoke.user.client","app.oauth2.as.evaluate.claim","app.oauth2.as.interact.interaction_code","app.oauth2.as.interact.interaction_handle","app.oauth2.as.key.rollover","app.oauth2.as.token.detect_reuse","app.oauth2.as.token.grant","app.oauth2.as.token.grant.access_token","app.oauth2.as.token.grant.device_secret","app.oauth2.as.token.grant.id_token","app.oauth2.as.token.grant.refresh_token","app.oauth2.as.token.revoke","app.oauth2.authorize","app.oauth2.authorize.code","app.oauth2.authorize.implicit.access_token","app.oauth2.authorize.implicit.id_token","app.oauth2.client.lifecycle.activate","app.oauth2.client.lifecycle.create","app.oauth2.client.lifecycle.deactivate","app.oauth2.client.lifecycle.delete","app.oauth2.client.lifecycle.update","app.oauth2.client_id_rate_limit_warning","app.oauth2.consent.grant","app.oauth2.interact.interaction_code","app.oauth2.interact.interaction_handle","app.oauth2.invalid_client_credentials","app.oauth2.invalid_client_ids","app.oauth2.key.rollover","app.oauth2.signon","app.oauth2.token.detect_reuse","app.oauth2.token.grant","app.oauth2.token.grant.access_token","app.oauth2.token.grant.id_token","app.oauth2.token.grant.refresh_token","app.oauth2.token.revoke","app.oauth2.token.revoke.implicit.as","app.oauth2.token.revoke.implicit.client","app.oauth2.token.revoke.implicit.user","app.office365.graph.api.error.service.principal.msgraph.authentication.failure","application.integration.authentication_failure","oauth2.as.activated","oauth2.as.created","oauth2.as.deactivated","oauth2.as.deleted","oauth2.as.updated","oauth2.claim.created","oauth2.claim.deleted","oauth2.claim.updated","oauth2.scope.created","oauth2.scope.deleted","oauth2.scope.updated","system.agent.ldap.realtimesync","system.agent.ldap.update_user","system.agent.ldap.update_user_password","system.iwa_agentless.auth","system.iwa_agentless.redirect","system.iwa_agentless.update","system.iwa_agentless.user.not_found","system.iwa_agentless_kerberos.update","user.authentication.auth","user.authentication.auth_via_AD_agent","user.authentication.auth_via_IDP","user.authentication.auth_via_LDAP_agent","user.authentication.auth_via_inbound_SAML","user.authentication.auth_via_inbound_delauth","user.authentication.auth_via_iwa","user.authentication.auth_via_mfa","user.authentication.auth_via_radius","user.authentication.auth_via_richclient","user.authentication.auth_via_social","user.authentication.authenticate","user.authentication.slo","user.authentication.sso","user.authentication.verify","user.import.password") +| alter + outcome_result = json_extract_scalar(outcome, "$.result"), + actor_type = lowercase(json_extract_scalar(actor, "$.type")), + actor_alternateId = json_extract_scalar(actor, "$.alternateId"), + actor_displayName = json_extract_scalar(actor, "$.displayName"), + actor_id = json_extract_scalar(actor, "$.id"), + target_id = coalesce(json_extract_scalar(arrayindex(arrayfilter(json_extract_array(`target`, "$"), json_extract_scalar("@element", "$.type") = "User"), 0), "$.id"), json_extract_scalar(arrayindex(json_extract_array(`target`, "$"), 0), "$.id")), + target_type = coalesce(json_extract_scalar(arrayindex(arrayfilter(json_extract_array(`target`, "$"), json_extract_scalar("@element", "$.type") = "User"), 0), "$.type"), json_extract_scalar(arrayindex(json_extract_array(`target`, "$"), 0), "$.type")), + target_alternateId = coalesce(json_extract_scalar(arrayindex(arrayfilter(json_extract_array(`target`, "$"), json_extract_scalar("@element", "$.type") = "User"), 0), "$.alternateId"), json_extract_scalar(arrayindex(json_extract_array(`target`, "$"), 0), "$.alternateId")), + target_displaynames = coalesce(json_extract_scalar(arrayindex(arrayfilter(json_extract_array(`target`, "$"), json_extract_scalar("@element", "$.type") = "User"), 0), "$.displayName"), json_extract_scalar(arrayindex(json_extract_array(`target`, "$"), 0), "$.displayName")), + debugdata_factor = lowercase(json_extract_scalar(debugContext, "$.debugData.factor")), + get_reason = json_extract_scalar(outcome, "$.reason"), + debugdata_errorcode = to_string(json_extract(debugContext, "$.debugData.errorCode")), + source_ip = json_extract_scalar(client, "$.ipAddress"), + client_device = json_extract_scalar(client, "$.device"), + client_ua_os = lowercase(json_extract_scalar(client, "$.userAgent.os")), + userAgent_rawUserAgent = json_extract_scalar(client, "$.userAgent.rawUserAgent"), + securityContext_isProxy = json_extract_scalar(securityContext, "$.isProxy"), + debugContext_debugData_risk_reasons = json_extract(debugContext, "$.debugData.risk.reasons"), + debugContext_debugData_behaviors = json_extract(debugContext, "$.debugData.behaviors"), + debugContext_debugData_threatSuspected = lowercase(to_string(json_extract(debugContext, "$.debugData.threatSuspected"))), + debugContext_debugData_originalPrincipal_type = json_extract(debugContext, "$.debugData.originalPrincipal.type"), + securityContext_domain = json_extract_scalar(securityContext, "$.domain"), + request_ipChain = request -> ipChain[] +| alter + source_ipv4 = if(source_ip !~= ":", source_ip, null), + source_ipv6 = if(source_ip ~= ":", source_ip, null), + convert_behaviors_array = arraymap(split(debugContext_debugData_behaviors, ","), rtrim(ltrim("@element", "{"), "}")), + lowercase_user_type = lowercase(debugContext_debugData_originalPrincipal_type), + check_debugData_threatSuspected = if(debugContext_debugData_threatSuspected = "false", "risky_signin: FALSE", "risky_signin: TRUE"), + check_actor_type = if(actor_type = "user", XDM_CONST.USER_TYPE_REGULAR, actor_type = "appuser", XDM_CONST.USER_TYPE_REGULAR, null), + check_target_type = if(target_type !~= "User", null, target_type = null, null, XDM_CONST.USER_TYPE_REGULAR), + check_actor_alternateId = if(actor_type = "user", actor_alternateId, actor_type = "appuser", actor_alternateId, null), + check_target_alternateId = if(target_type !~= "User", null, target_alternateId = null, null, target_alternateId), + check_actor_displayName = if(actor_type = "user", actor_displayName, actor_type = "appuser", actor_displayName, null), + check_target_displayName = if(target_type !~= "User", null, target_displaynames = null, null, target_displaynames), + check_actor_first_name = if(actor_type = "user", arrayindex(split(actor_displayName), 0) , actor_type = "appuser", arrayindex(split(actor_displayName), 0), null), + check_target_first_name = if(target_type ~= "User", arrayindex(split(target_displaynames), 0) , null), + check_actor_last_name = if(actor_type = "user", arrayindex(split(actor_displayName), 1) , actor_type = "appuser", arrayindex(split(actor_displayName), 1), null), + check_target_last_name = if(target_type ~= "User", arrayindex(split(target_displaynames), 1) , null), + check_actor_id = if(actor_type = "user", actor_id, actor_type = "appuser", actor_id, null), + check_target_id = if(target_type !~= "User", null, target_id = null, null, target_id), + check_securityContext_isProxy = if(securityContext_isProxy = "true", to_boolean("TRUE"), to_boolean("FALSE")), + check_debugData_risk_reasons1 = if(debugContext_debugData_risk_reasons ~= "Anonymizing Proxy", to_boolean("TRUE"), to_boolean("FALSE")), + check_debugData_risk_reasons2 = if(debugContext_debugData_risk_reasons ~= "Anonymizing Proxy", "anonymized_ip: TRUE", "anonymized_ip: FALSE"), + check_ipChain_v4 = arraymap(request_ipChain, if("@element" -> ip !~= ":", "@element" -> ip, null)), + check_securityContext_domain = if(securityContext_domain = ".", null, securityContext_domain), + check_ipChain_v6 = arraymap(request_ipChain, if("@element" -> ip ~= ":", "@element" -> ip, null)) +| alter + xdm.event.log_level = if(severity = "DEBUG", XDM_CONST.LOG_LEVEL_DEBUG, severity = "INFO", XDM_CONST.LOG_LEVEL_INFORMATIONAL, severity = "WARN", XDM_CONST.LOG_LEVEL_WARNING, severity = "ERROR", XDM_CONST.LOG_LEVEL_ERROR, severity = null, null, to_string(severity)), + xdm.event.operation_sub_type = if(eventType != "user.authentication.auth_via_mfa", null, debugdata_factor ~= "yubikey", "hardware_token", target_displaynames ~= "touch id", "hardware_token", target_displaynames ~= "hardware", "hardware_token", debugdata_factor = "fido_webauthn", "hardware_token", debugdata_factor = "password_as_factor", "password", target_displaynames = "authenticator", "application", target_displaynames = "android authenticator with safetyNet attestation", "application", target_displaynames = "okta verify", "application", target_displaynames = "windows hello software authenticator", "application", debugdata_factor = "okta_soft_token", "application", debugdata_factor = "okta_verify_push", "application", debugdata_factor = "email_factor", "email", debugdata_factor = "sms_factor", "sms", debugdata_factor = "call_factor", "voice", null), + xdm.event.type = if(eventType in ("user.authentication.sso", "user.session.start", "user.mfa.okta_verify.deny_push", "user.mfa.factor.update", "user.authentication.auth_via_mfa", "user.authentication.auth_via_AD_agent", "user.authentication.verify", "user.authentication.auth_via_radius", "user.authentication.auth_via_richclient", "system.push.send_factor_verify_push"), "authentication", ""), + xdm.event.description = displayMessage, + xdm.event.operation = if(eventType ~= "authentication|oauth2|mfa", XDM_CONST.OPERATION_TYPE_AUTH_MFA, eventType ~= "session|access_request|import|kerberos_rich_client|iwa|ldap", XDM_CONST.OPERATION_TYPE_AUTH_LOGIN, eventType = null, null, to_string(eventType)), + xdm.logon.type = if(actor_type = "user", XDM_CONST.LOGON_TYPE_INTERACTIVE, actor_type ~= "network", XDM_CONST.LOGON_TYPE_NETWORK, actor_type ~= "batch", XDM_CONST.LOGON_TYPE_BATCH, actor_type ~= "service", XDM_CONST.LOGON_TYPE_SERVICE, actor_type ~= "proxy", XDM_CONST.LOGON_TYPE_PROXY, actor_type ~= "unlock", XDM_CONST.LOGON_TYPE_UNLOCK, actor_type = "systemprincipal", null, actor_type = null, null, to_string(actor_type)), + xdm.auth.service = if(eventType = "user.authentication.auth_via_AD_agent", "IDP", eventType = "user.authentication.auth_via_radius", "IDP", eventType = "user.authentication.auth_via_richclient", "IDP", eventType = "user.session.start", "IDP", eventType = "policy.evaluate_sign_on", "IDP", eventType = "system.push.send_factor_verify_push", "IDP", eventType = "user.authentication.auth_via_mfa", "IDP", eventType = "user.authentication.verify", "IDP", eventType = "user.mfa.okta_verify.deny_push", "IDP", eventType = "user.authentication.sso", "SP", null), + xdm.source.host.os_family = if(client_ua_os ~= "chrome", XDM_CONST.OS_FAMILY_CHROMEOS, client_ua_os ~= "linux", XDM_CONST.OS_FAMILY_LINUX, client_ua_os ~= "ubuntu", XDM_CONST.OS_FAMILY_UBUNTU, client_ua_os ~= "windows|ms", XDM_CONST.OS_FAMILY_WINDOWS, client_ua_os ~= "mac", XDM_CONST.OS_FAMILY_MACOS, client_ua_os ~= "android", XDM_CONST.OS_FAMILY_ANDROID, client_ua_os ~= "ios", XDM_CONST.OS_FAMILY_IOS, client_ua_os ~= "debian", XDM_CONST.OS_FAMILY_DEBIAN, client_ua_os ~= "fedora", XDM_CONST.OS_FAMILY_FEDORA, client_ua_os ~= "centos", XDM_CONST.OS_FAMILY_CENTOS, client_ua_os ~= "centos", XDM_CONST.OS_FAMILY_CENTOS, client_ua_os ~= "solaris", XDM_CONST.OS_FAMILY_SOLARIS, client_ua_os ~= "scada", XDM_CONST.OS_FAMILY_SCADA, client_ua_os = null, null, to_string(client_ua_os)), + xdm.source.host.device_category = client_device, + xdm.source.location.region = json_extract_scalar(client, "$.geographicalContext.state"), + xdm.source.user.upn = check_actor_alternateId, + xdm.source.identity.upn = check_actor_alternateId, + xdm.target.user.upn = check_target_alternateId, + xdm.target.identity.upn = check_target_alternateId, + xdm.source.user.username = check_actor_displayName, + xdm.source.identity.username = check_actor_displayName, + xdm.target.user.username = check_target_displayName, + xdm.target.identity.username = check_target_displayName, + xdm.source.user.first_name = check_actor_first_name, + xdm.source.identity.first_name = check_actor_first_name, + xdm.target.user.first_name = check_target_first_name, + xdm.target.identity.first_name = check_target_first_name, + xdm.source.user.last_name = check_actor_last_name, + xdm.source.identity.last_name = check_actor_last_name, + xdm.target.user.last_name = check_target_last_name, + xdm.target.identity.last_name = check_target_last_name, + xdm.source.user.identifier = check_actor_id, + xdm.source.identity.identifier = check_actor_id, + xdm.target.user.identifier = check_target_id, + xdm.target.identity.identifier = check_target_id, + xdm.source.user.user_type = check_actor_type, + xdm.source.identity.user_type = check_actor_type, + xdm.target.user.user_type = check_target_type, + xdm.target.identity.user_type = check_target_type, + xdm.target.resource.id = if(target_type = "AppInstance", target_id, null), + xdm.target.resource.name = if(target_type = "AppInstance", target_alternateId, null), + xdm.source.asn.as_number = to_integer(json_extract_scalar(securityContext, "$.asNumber")), + xdm.source.asn.isp = json_extract_scalar(securityContext, "$.isp"), + xdm.source.asn.domain = check_securityContext_domain, + xdm.source.asn.is_proxy = check_securityContext_isProxy, + xdm.intermediate.is_proxy = check_debugData_risk_reasons1, + xdm.event.tags = arraycreate(XDM_CONST.EVENT_TAG_AUTHENTICATION), + xdm.alert.severity = coalesce(json_extract_scalar(securityContext, "$.risk.level"), json_extract_scalar(debugContext, "$.debugData.risk.level")), + xdm.alert.risks = arrayconcat(convert_behaviors_array, arraycreate(check_debugData_threatSuspected), arraycreate(check_debugData_risk_reasons2)), + xdm.session_context_id = json_extract_scalar(`transaction`, "$.id"), + xdm.observer.action = legacyEventType, + xdm.source.zone = json_extract_scalar(client, "$.zone"), + xdm.observer.type = json_extract_scalar(`transaction`, "$.type"), + xdm.observer.unique_identifier = json_extract_scalar(debugContext, "$.debugData.deviceFingerprint"), + xdm.logon.fingerprint = json_extract_scalar(debugContext, "$.debugData.dtHash"), + xdm.auth.mfa.client_details = json_extract_scalar(`transaction`, "$.detail.requestApiTokenId"), + xdm.intermediate.application.name = json_extract_scalar(debugContext, "$.debugData.proxyType"), + xdm.network.http.referrer = json_extract_scalar(debugContext, "$.debugData.origin"), + xdm.network.rule = json_extract_scalar(debugContext, "$.debugData.threatDetections"), + xdm.network.http.url = json_extract_scalar(debugContext, "$.debugData.url"), + xdm.auth.privilege_level = if(lowercase_user_type = "user", XDM_CONST.PRIVILEGE_LEVEL_USER, lowercase_user_type = "guest", XDM_CONST.PRIVILEGE_LEVEL_GUEST, lowercase_user_type = "admin", XDM_CONST.PRIVILEGE_LEVEL_ADMIN, lowercase_user_type = "system", XDM_CONST.PRIVILEGE_LEVEL_SYSTEM, lowercase_user_type = null, null, to_string(lowercase_user_type)), + xdm.auth.mfa.provider = json_extract_scalar(authenticationContext, "$.authenticationProvider.credentialProvider"), + xdm.auth.mfa.method = json_extract_scalar(authenticationContext, "$.authenticationProvider.credentialType"), + xdm.source.host.ipv4_addresses = check_ipChain_v4, + xdm.source.host.ipv6_addresses = check_ipChain_v6, + xdm.source.host.device_id = json_extract_scalar(client, "$.id"), + xdm.source.ipv4 = source_ipv4, + xdm.source.ipv6 = source_ipv6, + xdm.target.ipv4 = "", + xdm.target.ipv6 = "", + xdm.source.port = 0, + xdm.target.port = 0, + xdm.source.location.city = json_extract_scalar(client, "$.geographicalContext.city"), + xdm.source.location.country = json_extract_scalar(client, "$.geographicalContext.country"), + xdm.source.location.latitude = to_float(json_extract_scalar(client, "$.geographicalContext.geolocation.lat")), + xdm.source.location.longitude = to_float(json_extract_scalar(client, "$.geographicalContext.geolocation.lon")), + xdm.network.session_id = json_extract_scalar(authenticationContext, "$.authenticationProvider.externalSessionId"), + xdm.event.original_event_type = eventType, + xdm.event.outcome = if(outcome_result = "SUCCESS",XDM_CONST.OUTCOME_SUCCESS, outcome_result = "FAILURE", XDM_CONST.OUTCOME_FAILED, outcome_result = "ALLOW", XDM_CONST.OUTCOME_SUCCESS, outcome_result = "DENY", XDM_CONST.OUTCOME_FAILED, outcome_result = "CHALLENGE", XDM_CONST.OUTCOME_PARTIAL, outcome_result = "UNKNOWN", XDM_CONST.OUTCOME_UNKNOWN, outcome_result = "SKIPPED", XDM_CONST.OUTCOME_PARTIAL, outcome_result = null, null, to_string(outcome_result)), + xdm.event.outcome_reason = if(get_reason ~= "UNKNOWN_USER", "user_does_not_exist", get_reason ~= "Login denied. No matching user is assigned to Radius App RADIUS Application.", "user_does_not_exist", get_reason ~= "Login denied. No matching user", "user_does_not_exist", get_reason ~= "INVALID_CREDENTIALS", "bad_credentials", get_reason ~= "Authentication failed: bad username or password", "bad_credentials", debugdata_errorcode ~= "1326", "bad_credentials", get_reason ~= "Authentication failed: account is expired", "account_expired_or_disabled", get_reason ~= "USER_ACCOUNT_EXPIRE", "account_expired_or_disabled", get_reason ~= "GENERAL_NONSUCCESS", "account_expired_or_disabled", get_reason ~= "Authentication failed: password has expired", "account_expired_or_disabled", debugdata_errorcode ~= "1331", "account_expired_or_disabled", debugdata_errorcode ~= "1793", "account_expired_or_disabled", get_reason ~= "Authentication failed: account is locked", "account_locked", get_reason ~= "LOCKED_OUT", "account_locked", get_reason ~= "Login failed.", "failed_login", get_reason ~= "Login Failed", "failed_login", get_reason ~= "PASSWORD_BASED_LOGIN_DISALLOWED", "auth_policy_access_violation", get_reason ~= "login denied", "auth_policy_access_violation", get_reason ~= "VERIFICATION_ERROR", "mfa_failure", get_reason ~= "DEL_AUTH_TIMEOUT", "mfa_expired", get_reason ~= "User rejected Okta push verify", "user_reject", get_reason ~= "Login aborted", "user_cancelled", get_reason ~= "Login aborted.", "user_cancelled", get_reason ~= "NOT_SPECIFIED", "OTHER", get_reason = null, null, to_string(get_reason)), + xdm.target.host.device_category = target_type, + xdm.source.user_agent = if(lowercase(userAgent_rawUserAgent) = "unknown", null, userAgent_rawUserAgent), + xdm.source.host.os = json_extract_scalar(client, "$.userAgent.os"), + xdm.source.application.name = json_extract_scalar(client, "$.userAgent.browser"), + xdm.event.id = uuid, + xdm.source.asn.as_name = json_extract_scalar(securityContext,"$.asOrg"); + + + + +[MODEL: dataset=okta_sso_raw] +alter + outcome_result = json_extract_scalar(outcome, "$.result"), + actor_type = lowercase(json_extract_scalar(actor, "$.type")), + actor_alternateId = json_extract_scalar(actor, "$.alternateId"), + actor_displayName = json_extract_scalar(actor, "$.displayName"), + actor_id = json_extract_scalar(actor, "$.id"), + target_id = coalesce(json_extract_scalar(arrayindex(arrayfilter(json_extract_array(`target`, "$"), json_extract_scalar("@element", "$.type") = "User"), 0), "$.id"), json_extract_scalar(arrayindex(json_extract_array(`target`, "$"), 0), "$.id")), + target_type = coalesce(json_extract_scalar(arrayindex(arrayfilter(json_extract_array(`target`, "$"), json_extract_scalar("@element", "$.type") = "User"), 0), "$.type"), json_extract_scalar(arrayindex(json_extract_array(`target`, "$"), 0), "$.type")), + target_alternateId = coalesce(json_extract_scalar(arrayindex(arrayfilter(json_extract_array(`target`, "$"), json_extract_scalar("@element", "$.type") = "User"), 0), "$.alternateId"), json_extract_scalar(arrayindex(json_extract_array(`target`, "$"), 0), "$.alternateId")), + target_displaynames = coalesce(json_extract_scalar(arrayindex(arrayfilter(json_extract_array(`target`, "$"), json_extract_scalar("@element", "$.type") = "User"), 0), "$.displayName"), json_extract_scalar(arrayindex(json_extract_array(`target`, "$"), 0), "$.displayName")), + debugdata_factor = lowercase(json_extract_scalar(debugContext, "$.debugData.factor")), + get_reason = json_extract_scalar(outcome, "$.reason"), + debugdata_errorcode = to_string(json_extract(debugContext, "$.debugData.errorCode")), + source_ip = json_extract_scalar(client, "$.ipAddress"), + client_device = json_extract_scalar(client, "$.device"), + client_ua_os = lowercase(json_extract_scalar(client, "$.userAgent.os")), + userAgent_rawUserAgent = json_extract_scalar(client, "$.userAgent.rawUserAgent"), + securityContext_isProxy = json_extract_scalar(securityContext, "$.isProxy"), + debugContext_debugData_risk_reasons = json_extract(debugContext, "$.debugData.risk.reasons"), + debugContext_debugData_behaviors = json_extract(debugContext, "$.debugData.behaviors"), + debugContext_debugData_threatSuspected = lowercase(to_string(json_extract(debugContext, "$.debugData.threatSuspected"))), + debugContext_debugData_originalPrincipal_type = json_extract(debugContext, "$.debugData.originalPrincipal.type"), + securityContext_domain = json_extract_scalar(securityContext, "$.domain"), + request_ipChain = request -> ipChain[] +| alter + source_ipv4 = if(source_ip !~= ":", source_ip, null), + source_ipv6 = if(source_ip ~= ":", source_ip, null), + convert_behaviors_array = arraymap(split(debugContext_debugData_behaviors, ","), rtrim(ltrim("@element", "{"), "}")), + lowercase_user_type = lowercase(debugContext_debugData_originalPrincipal_type), + check_debugData_threatSuspected = if(debugContext_debugData_threatSuspected = "false", "risky_signin: FALSE", "risky_signin: TRUE"), + check_actor_type = if(actor_type = "user", XDM_CONST.USER_TYPE_REGULAR, actor_type = "appuser", XDM_CONST.USER_TYPE_REGULAR, null), + check_target_type = if(target_type !~= "User", null, target_type = null, null, XDM_CONST.USER_TYPE_REGULAR), + check_actor_alternateId = if(actor_type = "user", actor_alternateId, actor_type = "appuser", actor_alternateId, null), + check_target_alternateId = if(target_type !~= "User", null, target_alternateId = null, null, target_alternateId), + check_actor_displayName = if(actor_type = "user", actor_displayName, actor_type = "appuser", actor_displayName, null), + check_target_displayName = if(target_type !~= "User", null, target_displaynames = null, null, target_displaynames), + check_actor_first_name = if(actor_type in("user", "appuser"), arrayindex(regextract(actor_displayName, "(^[^,\s]+)"), 0), null), + check_target_first_name = if(target_type ~= "User", arrayindex(split(target_displaynames), 0) , null), + check_actor_last_name = if(actor_type in("user", "appuser"), arrayindex(regextract(actor_displayName, "^[^,\s]+[,\s]+(.*)$"), 0), null), + check_target_last_name = if(target_type ~= "User", arrayindex(split(target_displaynames), 1) , null), + check_actor_id = if(actor_type = "user", actor_id, actor_type = "appuser", actor_id, null), + check_target_id = if(target_type !~= "User", null, target_id = null, null, target_id), + check_securityContext_isProxy = if(securityContext_isProxy = "true", to_boolean("TRUE"), to_boolean("FALSE")), + check_debugData_risk_reasons1 = if(debugContext_debugData_risk_reasons ~= "Anonymizing Proxy", to_boolean("TRUE"), to_boolean("FALSE")), + check_debugData_risk_reasons2 = if(debugContext_debugData_risk_reasons ~= "Anonymizing Proxy", "anonymized_ip: TRUE", "anonymized_ip: FALSE"), + check_ipChain_v4 = arraymap(request_ipChain, if("@element" -> ip !~= ":", "@element" -> ip, null)), + check_securityContext_domain = if(securityContext_domain = ".", null, securityContext_domain), + check_ipChain_v6 = arraymap(request_ipChain, if("@element" -> ip ~= ":", "@element" -> ip, null)) +| alter + xdm.event.log_level = if(severity = "DEBUG", XDM_CONST.LOG_LEVEL_DEBUG, severity = "INFO", XDM_CONST.LOG_LEVEL_INFORMATIONAL, severity = "WARN", XDM_CONST.LOG_LEVEL_WARNING, severity = "ERROR", XDM_CONST.LOG_LEVEL_ERROR, severity = null, null, to_string(severity)), + xdm.event.operation_sub_type = if(eventType != "user.authentication.auth_via_mfa", null, debugdata_factor ~= "yubikey", "hardware_token", target_displaynames ~= "touch id", "hardware_token", target_displaynames ~= "hardware", "hardware_token", debugdata_factor = "fido_webauthn", "hardware_token", debugdata_factor = "password_as_factor", "password", target_displaynames = "authenticator", "application", target_displaynames = "android authenticator with safetyNet attestation", "application", target_displaynames = "okta verify", "application", target_displaynames = "windows hello software authenticator", "application", debugdata_factor = "okta_soft_token", "application", debugdata_factor = "okta_verify_push", "application", debugdata_factor = "email_factor", "email", debugdata_factor = "sms_factor", "sms", debugdata_factor = "call_factor", "voice", null), + xdm.event.type = if(eventType in ("user.authentication.sso", "user.session.start", "user.mfa.okta_verify.deny_push", "user.mfa.factor.update", "user.authentication.auth_via_mfa", "user.authentication.auth_via_AD_agent", "user.authentication.verify", "user.authentication.auth_via_radius", "user.authentication.auth_via_richclient", "system.push.send_factor_verify_push"), "authentication", ""), + xdm.event.description = displayMessage, +xdm.event.operation = if(eventType ~= "authentication|oauth2|mfa", XDM_CONST.OPERATION_TYPE_AUTH_MFA, eventType ~= "session|access_request|import|kerberos_rich_client|iwa|ldap", XDM_CONST.OPERATION_TYPE_AUTH_LOGIN, eventType = null, null, to_string(eventType)), + xdm.logon.type = if(actor_type = "user", XDM_CONST.LOGON_TYPE_INTERACTIVE, actor_type ~= "network", XDM_CONST.LOGON_TYPE_NETWORK, actor_type ~= "batch", XDM_CONST.LOGON_TYPE_BATCH, actor_type ~= "service", XDM_CONST.LOGON_TYPE_SERVICE, actor_type ~= "proxy", XDM_CONST.LOGON_TYPE_PROXY, actor_type ~= "unlock", XDM_CONST.LOGON_TYPE_UNLOCK, actor_type = "systemprincipal", null, actor_type = null, null, to_string(actor_type)), + xdm.auth.service = if(eventType = "user.authentication.auth_via_AD_agent", "IDP", eventType = "user.authentication.auth_via_radius", "IDP", eventType = "user.authentication.auth_via_richclient", "IDP", eventType = "user.session.start", "IDP", eventType = "policy.evaluate_sign_on", "IDP", eventType = "system.push.send_factor_verify_push", "IDP", eventType = "user.authentication.auth_via_mfa", "IDP", eventType = "user.authentication.verify", "IDP", eventType = "user.mfa.okta_verify.deny_push", "IDP", eventType = "user.authentication.sso", "SP", null), + xdm.source.host.os_family = if(client_ua_os ~= "chrome", XDM_CONST.OS_FAMILY_CHROMEOS, client_ua_os ~= "linux", XDM_CONST.OS_FAMILY_LINUX, client_ua_os ~= "ubuntu", XDM_CONST.OS_FAMILY_UBUNTU, client_ua_os ~= "windows|ms", XDM_CONST.OS_FAMILY_WINDOWS, client_ua_os ~= "mac", XDM_CONST.OS_FAMILY_MACOS, client_ua_os ~= "android", XDM_CONST.OS_FAMILY_ANDROID, client_ua_os ~= "ios", XDM_CONST.OS_FAMILY_IOS, client_ua_os ~= "debian", XDM_CONST.OS_FAMILY_DEBIAN, client_ua_os ~= "fedora", XDM_CONST.OS_FAMILY_FEDORA, client_ua_os ~= "centos", XDM_CONST.OS_FAMILY_CENTOS, client_ua_os ~= "centos", XDM_CONST.OS_FAMILY_CENTOS, client_ua_os ~= "solaris", XDM_CONST.OS_FAMILY_SOLARIS, client_ua_os ~= "scada", XDM_CONST.OS_FAMILY_SCADA, client_ua_os = null, null, to_string(client_ua_os)), + xdm.source.host.device_category = client_device, + xdm.source.location.region = json_extract_scalar(client, "$.geographicalContext.state"), + xdm.source.user.upn = check_actor_alternateId, + xdm.source.identity.upn = check_actor_alternateId, + xdm.target.user.upn = check_target_alternateId, + xdm.target.identity.upn = check_target_alternateId, + xdm.source.user.username = check_actor_displayName, + xdm.source.identity.username = check_actor_displayName, + xdm.target.user.username = check_target_displayName, + xdm.target.identity.username = check_target_displayName, + xdm.source.user.first_name = check_actor_first_name, + xdm.source.identity.first_name = check_actor_first_name, + xdm.target.user.first_name = check_target_first_name, + xdm.target.identity.first_name = check_target_first_name, + xdm.source.user.last_name = check_actor_last_name, + xdm.source.identity.last_name = check_actor_last_name, + xdm.target.user.last_name = check_target_last_name, + xdm.target.identity.last_name = check_target_last_name, + xdm.source.user.identifier = check_actor_id, + xdm.source.identity.identifier = check_actor_id, + xdm.target.user.identifier = check_target_id, + xdm.target.identity.identifier = check_target_id, + xdm.source.user.user_type = check_actor_type, + xdm.source.identity.user_type = check_actor_type, + xdm.target.user.user_type = check_target_type, + xdm.target.identity.user_type = check_target_type, + xdm.target.resource.id = if(target_type = "AppInstance", target_id, null), + xdm.target.resource.name = if(target_type = "AppInstance", target_alternateId, null), + xdm.source.asn.as_number = to_integer(json_extract_scalar(securityContext, "$.asNumber")), + xdm.source.asn.isp = json_extract_scalar(securityContext, "$.isp"), + xdm.source.asn.domain = check_securityContext_domain, + xdm.source.asn.is_proxy = check_securityContext_isProxy, + xdm.intermediate.is_proxy = check_debugData_risk_reasons1, + xdm.event.tags = arraycreate(XDM_CONST.EVENT_TAG_AUTHENTICATION), + xdm.alert.severity = coalesce(json_extract_scalar(securityContext, "$.risk.level"), json_extract_scalar(debugContext, "$.debugData.risk.level")), + xdm.alert.risks = arrayconcat(convert_behaviors_array, arraycreate(check_debugData_threatSuspected), arraycreate(check_debugData_risk_reasons2)), + xdm.session_context_id = json_extract_scalar(`transaction`, "$.id"), + xdm.observer.action = legacyEventType, + xdm.source.zone = json_extract_scalar(client, "$.zone"), + xdm.observer.type = json_extract_scalar(`transaction`, "$.type"), + xdm.observer.unique_identifier = json_extract_scalar(debugContext, "$.debugData.deviceFingerprint"), + xdm.logon.fingerprint = json_extract_scalar(debugContext, "$.debugData.dtHash"), + xdm.auth.mfa.client_details = json_extract_scalar(`transaction`, "$.detail.requestApiTokenId"), + xdm.intermediate.application.name = json_extract_scalar(debugContext, "$.debugData.proxyType"), + xdm.network.http.referrer = json_extract_scalar(debugContext, "$.debugData.origin"), + xdm.network.rule = json_extract_scalar(debugContext, "$.debugData.threatDetections"), + xdm.network.http.url = json_extract_scalar(debugContext, "$.debugData.url"), + xdm.auth.privilege_level = if(lowercase_user_type = "user", XDM_CONST.PRIVILEGE_LEVEL_USER, lowercase_user_type = "guest", XDM_CONST.PRIVILEGE_LEVEL_GUEST, lowercase_user_type = "admin", XDM_CONST.PRIVILEGE_LEVEL_ADMIN, lowercase_user_type = "system", XDM_CONST.PRIVILEGE_LEVEL_SYSTEM, lowercase_user_type = null, null, to_string(lowercase_user_type)), + xdm.auth.mfa.provider = json_extract_scalar(authenticationContext, "$.authenticationProvider.credentialProvider"), + xdm.auth.mfa.method = json_extract_scalar(authenticationContext, "$.authenticationProvider.credentialType"), + xdm.source.host.ipv4_addresses = check_ipChain_v4, + xdm.source.host.ipv6_addresses = check_ipChain_v6, + xdm.source.host.device_id = json_extract_scalar(client, "$.id"), + xdm.source.ipv4 = source_ipv4, + xdm.source.ipv6 = source_ipv6, + xdm.target.ipv4 = "", + xdm.target.ipv6 = "", + xdm.source.port = 0, + xdm.target.port = 0, + xdm.source.location.city = json_extract_scalar(client, "$.geographicalContext.city"), + xdm.source.location.country = json_extract_scalar(client, "$.geographicalContext.country"), + xdm.source.location.latitude = to_float(json_extract_scalar(client, "$.geographicalContext.geolocation.lat")), + xdm.source.location.longitude = to_float(json_extract_scalar(client, "$.geographicalContext.geolocation.lon")), + xdm.network.session_id = json_extract_scalar(authenticationContext, "$.authenticationProvider.externalSessionId"), + xdm.event.original_event_type = eventType, + xdm.event.outcome = if(outcome_result = "SUCCESS",XDM_CONST.OUTCOME_SUCCESS, outcome_result = "FAILURE", XDM_CONST.OUTCOME_FAILED, outcome_result = "ALLOW", XDM_CONST.OUTCOME_SUCCESS, outcome_result = "DENY", XDM_CONST.OUTCOME_FAILED, outcome_result = "CHALLENGE", XDM_CONST.OUTCOME_PARTIAL, outcome_result = "UNKNOWN", XDM_CONST.OUTCOME_UNKNOWN, outcome_result = "SKIPPED", XDM_CONST.OUTCOME_PARTIAL, outcome_result = null, null, to_string(outcome_result)), + xdm.event.outcome_reason = if(get_reason ~= "UNKNOWN_USER", "user_does_not_exist", get_reason ~= "Login denied. No matching user is assigned to Radius App RADIUS Application.", "user_does_not_exist", get_reason ~= "Login denied. No matching user", "user_does_not_exist", get_reason ~= "INVALID_CREDENTIALS", "bad_credentials", get_reason ~= "Authentication failed: bad username or password", "bad_credentials", debugdata_errorcode ~= "1326", "bad_credentials", get_reason ~= "Authentication failed: account is expired", "account_expired_or_disabled", get_reason ~= "USER_ACCOUNT_EXPIRE", "account_expired_or_disabled", get_reason ~= "GENERAL_NONSUCCESS", "account_expired_or_disabled", get_reason ~= "Authentication failed: password has expired", "account_expired_or_disabled", debugdata_errorcode ~= "1331", "account_expired_or_disabled", debugdata_errorcode ~= "1793", "account_expired_or_disabled", get_reason ~= "Authentication failed: account is locked", "account_locked", get_reason ~= "LOCKED_OUT", "account_locked", get_reason ~= "Login failed.", "failed_login", get_reason ~= "Login Failed", "failed_login", get_reason ~= "PASSWORD_BASED_LOGIN_DISALLOWED", "auth_policy_access_violation", get_reason ~= "login denied", "auth_policy_access_violation", get_reason ~= "VERIFICATION_ERROR", "mfa_failure", get_reason ~= "DEL_AUTH_TIMEOUT", "mfa_expired", get_reason ~= "User rejected Okta push verify", "user_reject", get_reason ~= "Login aborted", "user_cancelled", get_reason ~= "Login aborted.", "user_cancelled", get_reason ~= "NOT_SPECIFIED", "OTHER", get_reason = null, null, to_string(get_reason)), + xdm.target.host.device_category = target_type, + xdm.source.user_agent = if(lowercase(userAgent_rawUserAgent) = "unknown", null, userAgent_rawUserAgent), + xdm.source.host.os = json_extract_scalar(client, "$.userAgent.os"), + xdm.source.application.name = json_extract_scalar(client, "$.userAgent.browser"), + xdm.event.id = uuid, + xdm.source.asn.as_name = json_extract_scalar(securityContext,"$.asOrg"), + xdm.network.ip_protocol=XDM_CONST.IP_PROTOCOL_TCP; \ No newline at end of file diff --git a/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11.yml b/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11.yml new file mode 100644 index 000000000000..fb6c868440f2 --- /dev/null +++ b/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: Okta_2_0_ModelingRule +name: Okta 2.0 Modeling Rule +rules: '' +schema: '' +supportedModules: +- xsiam +tags: okta diff --git a/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11_schema.json b/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11_schema.json new file mode 100644 index 000000000000..157ac5920fba --- /dev/null +++ b/Packs/Okta/ModelingRules/OktaModelingRules_2_11/OktaModelingRules_2_11_schema.json @@ -0,0 +1,126 @@ +{ + "okta_okta_raw": { + "client": { + "type": "string", + "is_array": false + }, + "actor": { + "type": "string", + "is_array": false + }, + "uuid": { + "type": "string", + "is_array": false + }, + "securityContext": { + "type": "string", + "is_array": false + }, + "eventType": { + "type": "string", + "is_array": false + }, + "outcome": { + "type": "string", + "is_array": false + }, + "target": { + "type": "string", + "is_array": false + }, + "debugContext": { + "type": "string", + "is_array": false + }, + "request": { + "type": "string", + "is_array": false + }, + "severity": { + "type": "string", + "is_array": false + }, + "displayMessage": { + "type": "string", + "is_array": false + }, + "transaction": { + "type": "string", + "is_array": false + }, + "legacyEventType": { + "type": "string", + "is_array": false + }, + "authenticationContext": { + "type": "string", + "is_array": false + }, + "published": { + "type": "datetime", + "is_array": false + } + }, + "okta_sso_raw": { + "client": { + "type": "string", + "is_array": false + }, + "actor": { + "type": "string", + "is_array": false + }, + "uuid": { + "type": "string", + "is_array": false + }, + "securityContext": { + "type": "string", + "is_array": false + }, + "eventType": { + "type": "string", + "is_array": false + }, + "outcome": { + "type": "string", + "is_array": false + }, + "target": { + "type": "string", + "is_array": false + }, + "debugContext": { + "type": "string", + "is_array": false + }, + "request": { + "type": "string", + "is_array": false + }, + "severity": { + "type": "string", + "is_array": false + }, + "displayMessage": { + "type": "string", + "is_array": false + }, + "transaction": { + "type": "string", + "is_array": false + }, + "legacyEventType": { + "type": "string", + "is_array": false + }, + "authenticationContext": { + "type": "string", + "is_array": false + }, + "published": { + "type": "datetime", + "is_array": false + } + } +} \ No newline at end of file diff --git a/Packs/Okta/ReleaseNotes/3_4_10.md b/Packs/Okta/ReleaseNotes/3_4_10.md new file mode 100644 index 000000000000..88caa4031d2d --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_10.md @@ -0,0 +1,3 @@ +## Okta + +- Documentation and metadata improvements. diff --git a/Packs/Okta/ReleaseNotes/3_4_2.md b/Packs/Okta/ReleaseNotes/3_4_2.md new file mode 100644 index 000000000000..34f04981c9cd --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_2.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### Okta Event Collector + +- Improved the reliability of the event fetching mechanism. diff --git a/Packs/Okta/ReleaseNotes/3_4_3.md b/Packs/Okta/ReleaseNotes/3_4_3.md new file mode 100644 index 000000000000..c6ea15d5dc81 --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_3.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: Okta 2.0 Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/Okta/ReleaseNotes/3_4_4.md b/Packs/Okta/ReleaseNotes/3_4_4.md new file mode 100644 index 000000000000..57ba877521fe --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_4.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### Okta IAM + +- Documentation and metadata improvements. diff --git a/Packs/Okta/ReleaseNotes/3_4_5.md b/Packs/Okta/ReleaseNotes/3_4_5.md new file mode 100644 index 000000000000..bc38dca9f007 --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_5.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### Okta Event Collector + +- Updated the Docker image to: *demisto/fastapi:0.125.0.12689358*. + diff --git a/Packs/Okta/ReleaseNotes/3_4_6.md b/Packs/Okta/ReleaseNotes/3_4_6.md new file mode 100644 index 000000000000..05ecc8ca7f09 --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_6.md @@ -0,0 +1,6 @@ + +#### Modeling Rules + +##### Okta 2.0 Modeling Rule + +- Added a mapping of *debugData.dtHash* to the *xdm.logon.fingerprint* field for the *okta_okta_raw* and *okta_sso_raw* datasets. diff --git a/Packs/Okta/ReleaseNotes/3_4_7.md b/Packs/Okta/ReleaseNotes/3_4_7.md new file mode 100644 index 000000000000..e2d879e02b52 --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_7.md @@ -0,0 +1,7 @@ + +#### Scripts + +##### IAMInitOktaUser + +- Updated the Docker image to: *demisto/py3-tools:1.0.0.13176850*. + diff --git a/Packs/Okta/ReleaseNotes/3_4_8.md b/Packs/Okta/ReleaseNotes/3_4_8.md new file mode 100644 index 000000000000..b35987373899 --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_8.md @@ -0,0 +1,15 @@ + +#### Integrations + +##### Okta v2 + +- Added support for Unified Connector Platform (UCP) brokered authentication. +- Updated the ***okta-auth-reset*** command to invalidate the UCP credential cache when running under the Unified Connector Platform. +- Refactored the integration to consume shared logic from the `OktaApiModule`. No user-visible behavior change. +- Updated the Docker image to: *demisto/crypto:1.0.0.12833174*. + +##### Okta IAM + +- Added support for Unified Connector Platform (UCP) brokered authentication. +- Refactored the integration to consume shared logic from the new `OktaIAMApiModule`. No user-visible behavior change. +- Updated the Docker image to: *demisto/python3:3.12.14.13053055*. diff --git a/Packs/Okta/ReleaseNotes/3_4_9.md b/Packs/Okta/ReleaseNotes/3_4_9.md new file mode 100644 index 000000000000..f3cbbb76f1f5 --- /dev/null +++ b/Packs/Okta/ReleaseNotes/3_4_9.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### Okta 2.0 Modeling Rule + +- Updated the Okta 2.0 Modeling Rule to resolve the target user fields from the target element of type *User*. +- Updated the Okta 2.0 Modeling Rule to map the risk level to the *xdm.alert.severity* field. diff --git a/Packs/Okta/Scripts/IAMInitOktaUser/IAMInitOktaUser.yml b/Packs/Okta/Scripts/IAMInitOktaUser/IAMInitOktaUser.yml index 7c79f94e28bb..99656fa7585f 100644 --- a/Packs/Okta/Scripts/IAMInitOktaUser/IAMInitOktaUser.yml +++ b/Packs/Okta/Scripts/IAMInitOktaUser/IAMInitOktaUser.yml @@ -81,7 +81,7 @@ outputs: type: String scripttarget: 0 subtype: python3 -dockerimage: demisto/py3-tools:1.0.0.11597437 +dockerimage: demisto/py3-tools:1.0.0.13176850 runas: DBotWeakRole fromversion: 6.5.0 tests: diff --git a/Packs/Okta/TestPlaybooks/OktaV2-Test.yml b/Packs/Okta/TestPlaybooks/OktaV2-Test.yml index cfcc854c4547..54d0fb4d66b8 100644 --- a/Packs/Okta/TestPlaybooks/OktaV2-Test.yml +++ b/Packs/Okta/TestPlaybooks/OktaV2-Test.yml @@ -5,10 +5,10 @@ starttaskid: '0' tasks: '0': id: '0' - taskid: c8a98379-7a6f-4cb6-8a98-ff9971eafd79 + taskid: 16d44437-894f-4718-8d1d-d33cc973de38 type: start task: - id: c8a98379-7a6f-4cb6-8a98-ff9971eafd79 + id: 16d44437-894f-4718-8d1d-d33cc973de38 version: -1 name: '' iscommand: false @@ -21,8 +21,8 @@ tasks: view: |- { "position": { - "x": 377.5, - "y": -130 + "x": 695, + "y": 50 } } note: false @@ -32,12 +32,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '1': id: '1' - taskid: 9383a473-e924-4468-8059-9e3910912bf5 + taskid: 06a64bf1-ac61-43a4-83bb-e3e2c7b1d811 type: regular task: - id: 9383a473-e924-4468-8059-9e3910912bf5 + id: 06a64bf1-ac61-43a4-83bb-e3e2c7b1d811 version: -1 name: DeleteContext description: Delete field from context @@ -55,8 +56,8 @@ tasks: view: |- { "position": { - "x": 377.5, - "y": 35 + "x": 695, + "y": 210 } } note: false @@ -66,23 +67,23 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '5': id: '5' - taskid: 650405f6-867e-4ca9-8009-4d3d4c2ec794 + taskid: 8a906748-548e-4940-a0eb-1a35af7c0ea0 type: regular task: - id: 650405f6-867e-4ca9-8009-4d3d4c2ec794 + id: 8a906748-548e-4940-a0eb-1a35af7c0ea0 version: -1 name: create user with activation - description: Creates a new user with an option of setting password, recovery question and answer.The new user will immediately be able to login after activation with the assigned password. This flow is common when developing a custom user registration experience. - script: Okta v2|||okta-create-user + description: Creates a new user with an option of setting a password, recovery question, and answer. The new user will immediately be able to log in after activation with the assigned password. This flow is common when developing a custom user registration experience. + script: '|||okta-create-user' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '7' - - '36' scriptarguments: activate: simple: 'true' @@ -116,8 +117,8 @@ tasks: view: |- { "position": { - "x": 377.5, - "y": 545 + "x": 695, + "y": 735 } } note: false @@ -127,19 +128,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '7': id: '7' - taskid: 3e52684c-fc15-43d2-883a-8187c59cd272 + taskid: bd7e302e-94de-4986-8f19-14d960321a71 type: regular task: - id: 3e52684c-fc15-43d2-883a-8187c59cd272 + id: bd7e302e-94de-4986-8f19-14d960321a71 version: -1 name: deactivate user description: Deactivate User. - script: Okta v2|||okta-deactivate-user + script: '|||okta-deactivate-user' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '9' @@ -150,8 +152,8 @@ tasks: view: |- { "position": { - "x": 162.5, - "y": 720 + "x": 695, + "y": 910 } } note: false @@ -161,16 +163,17 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '8': id: '8' - taskid: 3f27c908-2025-45fd-8a74-a22e4e1e3749 + taskid: ced12544-5b43-406c-8d38-5965bcaf9fa1 type: regular task: - id: 3f27c908-2025-45fd-8a74-a22e4e1e3749 + id: ced12544-5b43-406c-8d38-5965bcaf9fa1 version: -1 name: add user to Demisto group description: Adds a user to a group with OKTA_GROUP type - script: Okta v2|||okta-add-to-group + script: '|||okta-add-to-group' type: regular iscommand: true brand: '' @@ -187,8 +190,8 @@ tasks: view: |- { "position": { - "x": 1022.5, - "y": 1070 + "x": 265, + "y": 1610 } } note: false @@ -198,19 +201,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '9': id: '9' - taskid: baabbffc-bdd5-451a-8759-8deb807d2073 + taskid: a9813344-8cb6-423d-8e0c-2bbdf3bfb5f4 type: regular task: - id: baabbffc-bdd5-451a-8759-8deb807d2073 + id: a9813344-8cb6-423d-8e0c-2bbdf3bfb5f4 version: -1 name: activate user description: Activates a specific user. - script: Okta v2|||okta-activate-user + script: '|||okta-activate-user' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '15' @@ -222,8 +226,8 @@ tasks: view: |- { "position": { - "x": 162.5, - "y": 895 + "x": 695, + "y": 1085 } } note: false @@ -233,19 +237,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '10': id: '10' - taskid: a00fa662-ecfb-4ffc-88ce-93733b47e2ba + taskid: 64db5ca8-68ec-42b1-85d8-671147a279d4 type: regular task: - id: a00fa662-ecfb-4ffc-88ce-93733b47e2ba + id: 64db5ca8-68ec-42b1-85d8-671147a279d4 version: -1 name: get Demisto group members description: Enumerates all users that are members of a group - script: Okta v2|||okta-get-group-members + script: '|||okta-get-group-members' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '23' @@ -256,8 +261,8 @@ tasks: view: |- { "position": { - "x": 1452.5, - "y": 1245 + "x": 480, + "y": 1785 } } note: false @@ -267,19 +272,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '11': id: '11' - taskid: a07224f6-9330-4480-85c9-ed967a9db650 + taskid: be8bd78e-1b90-48a0-81f6-a6a2f4daebce type: regular task: - id: a07224f6-9330-4480-85c9-ed967a9db650 + id: be8bd78e-1b90-48a0-81f6-a6a2f4daebce version: -1 name: get groups for user description: Returns all user groups associated with a specified user. - script: Okta v2|||okta-get-groups + script: '|||okta-get-groups' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '22' @@ -290,8 +296,8 @@ tasks: view: |- { "position": { - "x": 910, - "y": 1245 + "x": 50, + "y": 1785 } } note: false @@ -301,23 +307,23 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '13': id: '13' - taskid: 9df091f8-ac8d-4169-89fa-86fdc55edfaa + taskid: eb06cece-2468-4a33-809e-a8ebee139c80 type: regular task: - id: 9df091f8-ac8d-4169-89fa-86fdc55edfaa + id: eb06cece-2468-4a33-809e-a8ebee139c80 version: -1 name: remove from group description: Removes a user from a group with OKTA_GROUP type - script: Okta v2|||okta-remove-from-group + script: '|||okta-remove-from-group' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '40' - - '29' scriptarguments: groupName: simple: Demisto @@ -327,8 +333,8 @@ tasks: view: |- { "position": { - "x": 1022.5, - "y": 1595 + "x": 265, + "y": 2135 } } note: false @@ -338,19 +344,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '14': id: '14' - taskid: b5d33a85-4fff-4d58-8691-d00a046cbcb8 + taskid: ce5ce5fc-39a8-4a17-8e30-661cd4fd1ce0 type: regular task: - id: b5d33a85-4fff-4d58-8691-d00a046cbcb8 + id: ce5ce5fc-39a8-4a17-8e30-661cd4fd1ce0 version: -1 name: update user description: Update user with a given login, all fields are optional, fields which are not set will not be overriden. - script: Okta v2|||okta-update-user + script: '|||okta-update-user' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '27' @@ -363,8 +370,8 @@ tasks: view: |- { "position": { - "x": 50, - "y": 1420 + "x": 910, + "y": 1610 } } note: false @@ -374,19 +381,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '15': id: '15' - taskid: ae90bf75-72e9-4097-8b89-de3735af422d + taskid: a3a2f780-2eaf-4aea-80cb-b64cc76ca713 type: regular task: - id: ae90bf75-72e9-4097-8b89-de3735af422d + id: a3a2f780-2eaf-4aea-80cb-b64cc76ca713 version: -1 name: suspend user description: Suspends a user. This operation can only be performed on users with an ACTIVE status. The user has a status of SUSPENDED when the process is completed - script: Okta v2|||okta-suspend-user + script: '|||okta-suspend-user' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '16' @@ -397,8 +405,8 @@ tasks: view: |- { "position": { - "x": 162.5, - "y": 1070 + "x": 1125, + "y": 1260 } } note: false @@ -408,19 +416,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '16': id: '16' - taskid: 97498508-5ff4-40df-8aa6-472059d3e1ae + taskid: 478605fa-8a85-483a-83b4-bffae9384633 type: regular task: - id: 97498508-5ff4-40df-8aa6-472059d3e1ae + id: 478605fa-8a85-483a-83b4-bffae9384633 version: -1 name: unsuspend user description: Returns a user to ACTIVE status. This operation can only be performed on users that have a SUSPENDED status. - script: Okta v2|||okta-unsuspend-user + script: '|||okta-unsuspend-user' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '14' @@ -432,8 +441,8 @@ tasks: view: |- { "position": { - "x": 162.5, - "y": 1245 + "x": 1125, + "y": 1435 } } note: false @@ -443,12 +452,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '22': id: '22' - taskid: 58e6ad72-d969-4940-8c29-be9489ede7e8 + taskid: 548646c8-c2e5-4b0c-803d-372a397faa8b type: condition task: - id: 58e6ad72-d969-4940-8c29-be9489ede7e8 + id: 548646c8-c2e5-4b0c-803d-372a397faa8b version: -1 name: verify results type: condition @@ -472,8 +482,8 @@ tasks: view: |- { "position": { - "x": 910, - "y": 1420 + "x": 50, + "y": 1960 } } note: false @@ -483,12 +493,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '23': id: '23' - taskid: 9aa642ad-d0f1-43d8-868f-216606879af6 + taskid: f2a39d80-c084-441c-898b-0e1f72553fb3 type: condition task: - id: 9aa642ad-d0f1-43d8-868f-216606879af6 + id: f2a39d80-c084-441c-898b-0e1f72553fb3 version: -1 name: verify results type: condition @@ -513,8 +524,8 @@ tasks: view: |- { "position": { - "x": 1452.5, - "y": 1420 + "x": 480, + "y": 1960 } } note: false @@ -524,19 +535,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '27': id: '27' - taskid: 6277e71f-abc4-41d3-8e66-d129af8dfc6e + taskid: 6d21ccf9-76a0-45d6-8f8e-f44b0c6f2e97 type: regular task: - id: 6277e71f-abc4-41d3-8e66-d129af8dfc6e + id: 6d21ccf9-76a0-45d6-8f8e-f44b0c6f2e97 version: -1 name: get-user description: Fetches information for a specific user. You must enter one or more parameters for the command to run - script: Okta v2|||okta-get-user + script: '|||okta-get-user' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '28' @@ -547,8 +559,8 @@ tasks: view: |- { "position": { - "x": 162.5, - "y": 1595 + "x": 1125, + "y": 1785 } } note: false @@ -558,12 +570,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '28': id: '28' - taskid: 90c1aaf9-e43f-4a81-8015-78d648a8e54a + taskid: 920dda09-4514-41d3-8814-c085d25c9b53 type: condition task: - id: 90c1aaf9-e43f-4a81-8015-78d648a8e54a + id: 920dda09-4514-41d3-8814-c085d25c9b53 version: -1 name: verify result type: condition @@ -587,8 +600,8 @@ tasks: view: |- { "position": { - "x": 162.5, - "y": 1770 + "x": 1125, + "y": 1960 } } note: false @@ -598,19 +611,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '29': id: '29' - taskid: 34fa30fd-bf9b-45c5-8a17-bd756486b630 + taskid: 475f0210-6b13-4324-8f3e-71b36a1499f2 type: regular task: - id: 34fa30fd-bf9b-45c5-8a17-bd756486b630 + id: 475f0210-6b13-4324-8f3e-71b36a1499f2 version: -1 name: okta-delete-user description: Delete specified user - script: Okta v2|||okta-delete-user + script: '|||okta-delete-user' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '39' @@ -621,8 +635,8 @@ tasks: view: |- { "position": { - "x": 265, - "y": 2295 + "x": 1125, + "y": 2485 } } note: false @@ -632,19 +646,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '30': id: '30' - taskid: 7f06b3a6-2911-4087-8c7a-cd6087ac7648 + taskid: c73fbca2-3e8f-4f25-81f1-81e45c7b4853 type: regular task: - id: 7f06b3a6-2911-4087-8c7a-cd6087ac7648 + id: c73fbca2-3e8f-4f25-81f1-81e45c7b4853 version: -1 name: set-password description: Sets passwords without validating existing user credentials - script: Okta v2|||okta-set-password + script: '|||okta-set-password' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '27' @@ -657,8 +672,8 @@ tasks: view: |- { "position": { - "x": 490, - "y": 1420 + "x": 1340, + "y": 1610 } } note: false @@ -668,19 +683,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '34': id: '34' - taskid: 14c46606-c34e-45c3-873c-567885bfed16 + taskid: 72aac069-b11d-48cf-8bfe-7d846b8028f6 type: regular task: - id: 14c46606-c34e-45c3-873c-567885bfed16 + id: 72aac069-b11d-48cf-8bfe-7d846b8028f6 version: -1 name: okta-search description: Search for Okta users - script: Okta v2|||okta-search + script: '|||okta-search' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '35' @@ -691,8 +707,8 @@ tasks: view: |- { "position": { - "x": 162.5, - "y": 1945 + "x": 1125, + "y": 2135 } } note: false @@ -702,12 +718,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '35': id: '35' - taskid: 0c3eefc7-334a-4b83-8b90-80a94b65dc37 + taskid: c38d49f8-c94a-4dde-82b9-0828b1309d57 type: condition task: - id: 0c3eefc7-334a-4b83-8b90-80a94b65dc37 + id: c38d49f8-c94a-4dde-82b9-0828b1309d57 version: -1 name: verify result type: condition @@ -732,86 +749,8 @@ tasks: view: |- { "position": { - "x": 162.5, - "y": 2120 - } - } - note: false - timertriggers: [] - ignoreworker: false - skipunavailable: false - quietmode: 0 - isoversize: false - isautoswitchedtoquietmode: false - '36': - id: '36' - taskid: 15135df8-bce4-4586-8a76-3cfde30d2b30 - type: regular - task: - id: 15135df8-bce4-4586-8a76-3cfde30d2b30 - version: -1 - name: get-user-factors - description: Returns all the enrolled facors for the specified user. - script: '|||okta-get-user-factors' - type: regular - iscommand: true - brand: '' - nexttasks: - '#none#': - - '37' - scriptarguments: - userId: - simple: 00upt1w8t40wFQM2v0h7 - separatecontext: false - view: |- - { - "position": { - "x": 1042.5, - "y": 720 - } - } - note: false - timertriggers: [] - ignoreworker: false - skipunavailable: false - quietmode: 0 - isoversize: false - isautoswitchedtoquietmode: false - '37': - id: '37' - taskid: f928cd2a-c5ef-4dbc-84ce-766999ce418c - type: condition - task: - id: f928cd2a-c5ef-4dbc-84ce-766999ce418c - version: -1 - name: verify result - type: condition - iscommand: false - brand: '' - nexttasks: - yes: - - '8' - separatecontext: false - conditions: - - label: yes - condition: - - - operator: containsString - left: - value: - complex: - root: Account - accessor: Factor.FactorType - transformers: - - operator: FirstArrayElement - iscontext: true - right: - value: - simple: sms - view: |- - { - "position": { - "x": 1042.5, - "y": 895 + "x": 1125, + "y": 2310 } } note: false @@ -821,12 +760,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '38': id: '38' - taskid: 28a14f17-0339-4998-8229-44fa39e7d38d + taskid: c320a226-41aa-4c49-81d7-0f20b7030cb4 type: title task: - id: 28a14f17-0339-4998-8229-44fa39e7d38d + id: c320a226-41aa-4c49-81d7-0f20b7030cb4 version: -1 name: Done type: title @@ -837,8 +777,8 @@ tasks: view: |- { "position": { - "x": 830, - "y": 3600 + "x": 695, + "y": 3710 } } note: false @@ -848,12 +788,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '39': id: '39' - taskid: 437df453-54ee-4d2d-889a-3a2398f06c45 + taskid: 8d90b8db-8ed2-4ff9-8c1a-3c0d7cfa06a0 type: regular task: - id: 437df453-54ee-4d2d-889a-3a2398f06c45 + id: 8d90b8db-8ed2-4ff9-8c1a-3c0d7cfa06a0 version: -1 name: okta-search description: Search for Okta users @@ -871,8 +812,8 @@ tasks: view: |- { "position": { - "x": 265, - "y": 2470 + "x": 1125, + "y": 2660 } } note: false @@ -882,12 +823,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '40': id: '40' - taskid: eb9cc005-e2e9-480e-8b3c-0d3b08151d01 + taskid: fe42decf-1592-456a-8bf8-2fa9dde7b4da type: regular task: - id: eb9cc005-e2e9-480e-8b3c-0d3b08151d01 + id: fe42decf-1592-456a-8bf8-2fa9dde7b4da version: -1 name: get Demisto group members description: Enumerates all users that are members of a group @@ -905,8 +847,8 @@ tasks: view: |- { "position": { - "x": 1022.5, - "y": 1770 + "x": 265, + "y": 2310 } } note: false @@ -916,12 +858,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '41': id: '41' - taskid: bf297581-1b21-432e-8888-c62210de67b3 + taskid: 68cbc010-2612-426a-8f05-14fa2498d444 type: condition task: - id: bf297581-1b21-432e-8888-c62210de67b3 + id: 68cbc010-2612-426a-8f05-14fa2498d444 version: -1 name: verify results type: condition @@ -945,8 +888,8 @@ tasks: view: |- { "position": { - "x": 1022.5, - "y": 1945 + "x": 265, + "y": 2485 } } note: false @@ -956,12 +899,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '42': id: '42' - taskid: 3b1e8549-ac5b-437f-8390-c2ad0cd1eeeb + taskid: 4e7fcedf-5890-4205-8a36-473edcce8153 type: condition task: - id: 3b1e8549-ac5b-437f-8390-c2ad0cd1eeeb + id: 4e7fcedf-5890-4205-8a36-473edcce8153 version: -1 name: verify result type: condition @@ -985,8 +929,8 @@ tasks: view: |- { "position": { - "x": 265, - "y": 2645 + "x": 1125, + "y": 2835 } } note: false @@ -996,19 +940,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '43': id: '43' - taskid: 0ee45839-9826-429d-8847-d744de957402 + taskid: 2834b370-fceb-4a54-89a7-b536540569aa type: regular task: - id: 0ee45839-9826-429d-8847-d744de957402 + id: 2834b370-fceb-4a54-89a7-b536540569aa version: -1 name: list groups with query =de description: Lists groups in your organization. A subset of groups can be returned that match a supported filter expression or query. - script: Okta v2|||okta-list-groups + script: '|||okta-list-groups' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '44' @@ -1019,8 +964,8 @@ tasks: view: |- { "position": { - "x": 1022.5, - "y": 2120 + "x": 265, + "y": 2660 } } note: false @@ -1030,12 +975,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '44': id: '44' - taskid: 5733cf66-3abb-4d61-855e-f7db5ca10d94 + taskid: a87f37f8-2a94-476d-8c3d-60d1057e6841 type: condition task: - id: 5733cf66-3abb-4d61-855e-f7db5ca10d94 + id: a87f37f8-2a94-476d-8c3d-60d1057e6841 version: -1 name: verify results type: condition @@ -1059,8 +1005,8 @@ tasks: view: |- { "position": { - "x": 1125, - "y": 2295 + "x": 265, + "y": 2835 } } note: false @@ -1070,12 +1016,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '47': id: '47' - taskid: 10b42e04-6003-4bb0-80df-b028316a630b + taskid: f7374d33-b4ab-42c5-8516-598f60507a9e type: regular task: - id: 10b42e04-6003-4bb0-80df-b028316a630b + id: f7374d33-b4ab-42c5-8516-598f60507a9e version: -1 name: get-group-assignments description: Get events for when a user added to a group @@ -1093,8 +1040,8 @@ tasks: view: |- { "position": { - "x": 1022.5, - "y": 2645 + "x": 50, + "y": 3185 } } note: false @@ -1104,12 +1051,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '48': id: '48' - taskid: 4bf88e0e-6e65-4f39-868e-f48906dae5c0 + taskid: 0843dccd-2e69-474a-8680-ff5559213724 type: condition task: - id: 4bf88e0e-6e65-4f39-868e-f48906dae5c0 + id: 0843dccd-2e69-474a-8680-ff5559213724 version: -1 name: verify results type: condition @@ -1135,8 +1083,8 @@ tasks: view: |- { "position": { - "x": 1022.5, - "y": 2820 + "x": 50, + "y": 3360 } } note: false @@ -1146,12 +1094,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '50': id: '50' - taskid: cc533b99-6c42-40cc-8a88-d10ea0108988 + taskid: b7bda056-0f47-41fe-81d6-99adc83d5e97 type: condition task: - id: cc533b99-6c42-40cc-8a88-d10ea0108988 + id: b7bda056-0f47-41fe-81d6-99adc83d5e97 version: -1 name: verify results type: condition @@ -1177,8 +1126,8 @@ tasks: view: |- { "position": { - "x": 1452.5, - "y": 2820 + "x": 480, + "y": 3360 } } note: false @@ -1188,12 +1137,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '51': id: '51' - taskid: fa1278c5-9be3-4d0c-81e0-370e6bf44ec8 + taskid: c1dc64e7-06be-4522-8c30-bedc5e7fb391 type: title task: - id: fa1278c5-9be3-4d0c-81e0-370e6bf44ec8 + id: c1dc64e7-06be-4522-8c30-bedc5e7fb391 version: -1 name: Logs type: title @@ -1208,8 +1158,8 @@ tasks: view: |- { "position": { - "x": 1125, - "y": 2485 + "x": 265, + "y": 3017.5 } } note: false @@ -1219,12 +1169,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '52': id: '52' - taskid: 994b1ff4-ecf5-4e30-82eb-50ef3d27793e + taskid: 42cb8eba-dede-4626-8d82-5cf5270d57c9 type: title task: - id: 994b1ff4-ecf5-4e30-82eb-50ef3d27793e + id: 42cb8eba-dede-4626-8d82-5cf5270d57c9 version: -1 name: End of Logs type: title @@ -1238,8 +1189,8 @@ tasks: view: |- { "position": { - "x": 1125, - "y": 2995 + "x": 265, + "y": 3542.5 } } note: false @@ -1249,12 +1200,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '53': id: '53' - taskid: 3f6b35ee-1bc8-46e3-897a-73b0aa9ba66b + taskid: 6ab0dc69-3123-41f5-845b-b90605f2f65f type: regular task: - id: 3f6b35ee-1bc8-46e3-897a-73b0aa9ba66b + id: 6ab0dc69-3123-41f5-845b-b90605f2f65f version: -1 name: application-assignments description: Returnes events for when a user was assigned to an application @@ -1272,8 +1224,8 @@ tasks: view: |- { "position": { - "x": 1452.5, - "y": 2645 + "x": 480, + "y": 3185 } } note: false @@ -1283,12 +1235,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '60': id: '60' - taskid: 3e0ea1c8-3741-498c-861b-f171ea6817ad + taskid: 328322e8-557e-4663-803d-02626b016bba type: regular task: - id: 3e0ea1c8-3741-498c-861b-f171ea6817ad + id: 328322e8-557e-4663-803d-02626b016bba version: -1 name: 'Generate a random login prefix ' description: Generates random string @@ -1314,8 +1267,8 @@ tasks: view: |- { "position": { - "x": 377.5, - "y": 210 + "x": 695, + "y": 385 } } note: false @@ -1325,12 +1278,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '61': id: '61' - taskid: 751b5eb8-60b2-4dbc-896a-4239ca40895e + taskid: 547dfeda-bf97-4245-8b69-1da0221a6c41 type: regular task: - id: 751b5eb8-60b2-4dbc-896a-4239ca40895e + id: 547dfeda-bf97-4245-8b69-1da0221a6c41 version: -1 name: GeneratePassword description: "This function generates a password and allows various parameters to customize the properties of the password depending on the use case (e.g. password complexity requirements). The default behavior is to generate a password of *random length* including all four character classes (upper, lower, digits, symbols) with at least five and at most ten characters per class. \n\nThe min_* values all default to 0. This means that if the command is executed in this way:\n!GeneratePassword max_lcase=10\nIt is possible that a password of length zero could be generated. It is therefore recommended to always include a min_* parameter that matches. \n\nThe debug parameter will print certain properties of the command into the WarRoom for easy diagnostics." @@ -1362,8 +1316,8 @@ tasks: view: |- { "position": { - "x": 377.5, - "y": 380 + "x": 695, + "y": 560 } } note: false @@ -1373,19 +1327,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '63': id: '63' - taskid: 8f2938e1-9305-45f7-83dd-ab48e362768b + taskid: fec12244-a381-4dde-8c1b-62b42dd2e037 type: regular task: - id: 8f2938e1-9305-45f7-83dd-ab48e362768b + id: fec12244-a381-4dde-8c1b-62b42dd2e037 version: -1 name: list-users description: Lists users in your organization. - script: Okta v2|||okta-list-users + script: '|||okta-list-users' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '64' @@ -1396,8 +1351,8 @@ tasks: view: |- { "position": { - "x": 265, - "y": 2820 + "x": 1125, + "y": 3010 } } note: false @@ -1407,12 +1362,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '64': id: '64' - taskid: 7280e61c-ad9f-4575-8177-fec8cab75fe9 + taskid: dc2a990c-7707-4710-88b6-7849bd5c1e29 type: condition task: - id: 7280e61c-ad9f-4575-8177-fec8cab75fe9 + id: dc2a990c-7707-4710-88b6-7849bd5c1e29 version: -1 name: verify results type: condition @@ -1433,8 +1389,8 @@ tasks: view: |- { "position": { - "x": 265, - "y": 2995 + "x": 1125, + "y": 3185 } } note: false @@ -1444,19 +1400,20 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '65': id: '65' - taskid: f19162ac-081c-4a37-8f7b-e8b0dc85f5a6 + taskid: ee32a071-0494-4b12-89b4-0f80e8b2b7e8 type: regular task: - id: f19162ac-081c-4a37-8f7b-e8b0dc85f5a6 + id: ee32a071-0494-4b12-89b4-0f80e8b2b7e8 version: -1 name: list-zones description: Get an Okta Zone object - script: Okta v2|||okta-list-zones + script: '|||okta-list-zones' type: regular iscommand: true - brand: Okta v2 + brand: "" nexttasks: '#none#': - '66' @@ -1467,8 +1424,8 @@ tasks: view: |- { "position": { - "x": 265, - "y": 3170 + "x": 1125, + "y": 3360 } } note: false @@ -1478,12 +1435,13 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" '66': id: '66' - taskid: 00f1981d-e1d3-41ea-8ba7-69e1b5628271 + taskid: 3d355c2d-2424-4be7-8c2b-ee97c40d6ce1 type: condition task: - id: 00f1981d-e1d3-41ea-8ba7-69e1b5628271 + id: 3d355c2d-2424-4be7-8c2b-ee97c40d6ce1 version: -1 name: verify results type: condition @@ -1504,8 +1462,8 @@ tasks: view: |- { "position": { - "x": 265, - "y": 3340 + "x": 1125, + "y": 3535 } } note: false @@ -1515,6 +1473,7 @@ tasks: quietmode: 0 isoversize: false isautoswitchedtoquietmode: false + continueonerrortype: "" view: |- { "linkLabelsPosition": {}, diff --git a/Packs/Okta/pack_metadata.json b/Packs/Okta/pack_metadata.json index 6207e23b93e9..c8f36ef87f77 100644 --- a/Packs/Okta/pack_metadata.json +++ b/Packs/Okta/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Okta", "description": "Integration with Okta's cloud-based identity management service.", "support": "xsoar", - "currentVersion": "3.4.1", + "currentVersion": "3.4.10", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -12,13 +12,14 @@ ], "tags": [], "useCases": [], - "keywords": [], + "keywords": [ + "Okta" + ], "marketplaces": [ "xsoar", "marketplacev2", "platform" ], - "defaultDataSource": "Okta Event Collector", "supportedModules": [ "agentix", "cloud_runtime_security", diff --git a/Packs/OktaASA/ModelingRules/OktaASA/OktaASA.yml b/Packs/OktaASA/ModelingRules/OktaASA/OktaASA.yml index 593e5e61b206..1cfe21e1c138 100644 --- a/Packs/OktaASA/ModelingRules/OktaASA/OktaASA.yml +++ b/Packs/OktaASA/ModelingRules/OktaASA/OktaASA.yml @@ -1,4 +1,5 @@ fromversion: 8.11.0 +toversion: 8.14.0 id: Okta_ASA_ModelingRule name: Okta ASA Modeling Rule rules: '' diff --git a/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11.xif b/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11.xif new file mode 100644 index 000000000000..9b560cad866a --- /dev/null +++ b/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11.xif @@ -0,0 +1,135 @@ +/* This modeling rule is designed to extract and map fields from all Okta ASA event types. + For the list of event types, see here: https://developer.okta.com/docs/reference/api/event-types/ +*/ +[MODEL: dataset = "okta_asa_raw"] +alter //Extraction of fields + type = details -> type, + details_user_status = details -> user_status, + details_from_address = details -> from_address, + details_client_ip = details -> client_ip, + user_status = user -> status, + user_type = user -> user_type, + client_os = lowercase(client -> os), + server_project_name = server -> project_name, + server_os = lowercase(server -> os), + server_cloud_provider = lowercase(server -> cloud_provider), + server_source_details_cloud_provider = lowercase(server -> source_details.cloud_provider), + server_instance_details_external_ip = server -> instance_details.external_ip, + server_instance_details_internal_ip = server -> instance_details.internal_ip +| alter //Mapping to XDM fields + xdm.event.id = id, + xdm.event.tags = arraycreate(XDM_CONST.EVENT_TAG_SAAS), + xdm.event.operation = if( + type IN("*enroll", "*create", "*add"), XDM_CONST.OPERATION_TYPE_CREATE, + details_user_status ~= "DELETE" OR type IN ("*remove","*delete"), XDM_CONST.OPERATION_TYPE_DELETE, + details_user_status ~= "UPDATE" OR type IN("active_directory.account_discovery.complete", "server_account.discovered", "*reveal"), XDM_CONST.OPERATION_TYPE_READ, + type IN("server_account.password_change.initiated", "server_account.password_change.out_of_band", "active_directory.account_rule.applied", "*activate", "incoming_federation.request"), XDM_CONST.OPERATION_TYPE_EXECUTION, + type IN ("*creds*", "*auth_token*", "*federation.approve", "security_policy.evaluate", "resource.*") ,XDM_CONST.OPERATION_TYPE_AUTHENTICATION, + type IN ("*ssh_login"),XDM_CONST.OPERATION_TYPE_AUTH_LOGIN, + type IN("*assign", "*update", "*rotate", "*bulk_membership_change", "offline_disabled_event", "offline_enabled_event","password.change", "password.reset", "permission.change", "service_account.password_rotation"), XDM_CONST.OPERATION_TYPE_UPDATE, + type), //Maps event operation based on the possible event types. For 'user' events, the 'details.user_status' field provides more specific information about the action. For the full list of the event types, see the docs: https://developer.okta.com/docs/reference/api/event-types/. + xdm.event.operation_sub_type = coalesce(details -> session_type,details_user_status),// Maps the operation sub-type: uses 'session_type' for 'auth_token_issue' event type and 'user_status' for user events. + xdm.observer.unique_identifier = details -> team_id, + xdm.observer.name = details -> team_name, + xdm.session_context_id = details -> trace_id, + xdm.event.type = type, + xdm.target.user.identifier = details -> target_user, //Related to 'user' events + xdm.target.identity.identifier = details -> target_user, //Related to 'user' events + xdm.target.user.username = coalesce(details -> unix_user_name, details -> user), //Related to 'server' events + xdm.target.identity.username = coalesce(details -> unix_user_name, details -> user), //Related to 'server' events + xdm.source.ipv4 = if( + is_ipv4(details_from_address) = TRUE, details_from_address, + is_ipv4(details_client_ip) = TRUE, details_client_ip), + xdm.source.ipv6 = if( + is_ipv6(details_from_address) = TRUE,details_from_address, + is_ipv6(details_client_ip) = TRUE, details_client_ip), + xdm.auth.auth_method = details -> user_access_method, //Related to 'user_creds.issue' event + xdm.source.user.upn = user -> details.email, + xdm.source.identity.upn = user -> details.email, + xdm.source.user.first_name = user -> details.first_name, + xdm.source.identity.first_name = user -> details.first_name, + xdm.source.user.last_name = user -> details.last_name, + xdm.source.identity.last_name = user -> details.last_name, + xdm.source.user.identifier = user -> id, + xdm.source.identity.identifier = user -> id, + xdm.source.user.username = user -> name, + xdm.source.identity.username = user -> name, + xdm.source.user.is_disabled = if( + user_status ~= "ACTIVE|UNKNOWN|PENDING", FALSE, + user_status ~= "DISABLE", TRUE, + null), + xdm.source.identity.is_disabled = if( + user_status ~= "ACTIVE|UNKNOWN|PENDING", FALSE, + user_status ~= "DISABLE", TRUE, + null), + xdm.source.user.user_type = if( + user_type = "human", XDM_CONST.USER_TYPE_REGULAR, + user_type = "service" OR details -> unix_user_name != null, XDM_CONST.USER_TYPE_SERVICE_ACCOUNT, + user_type), + xdm.source.identity.user_type = if( + user_type = "human", XDM_CONST.USER_TYPE_REGULAR, + user_type = "service" OR details -> unix_user_name != null, XDM_CONST.USER_TYPE_SERVICE_ACCOUNT, + user_type), + xdm.source.user.roles = json_extract_scalar_array(user, "$.role_grants"), + xdm.source.identity.roles = json_extract_scalar_array(user, "$.role_grants"), + xdm.source.host.hostname = client -> hostname, + xdm.source.host.device_id = client -> id, + xdm.source.host.os = client -> os, + xdm.source.host.os_family = if( + client_os CONTAINS "windows", XDM_CONST.OS_FAMILY_WINDOWS, + client_os CONTAINS "macos", XDM_CONST.OS_FAMILY_MACOS, + client_os CONTAINS "linux", XDM_CONST.OS_FAMILY_LINUX, + client_os CONTAINS "ubuntu", XDM_CONST.OS_FAMILY_UBUNTU, + client -> os), + xdm.intermediate.cloud.project = project -> name, + xdm.intermediate.cloud.project_id = project -> id, + xdm.target.host.device_id = server -> id, + xdm.target.cloud.project = server_project_name, + xdm.target.host.hostname = server -> hostname, + xdm.intermediate.host.hostname = server -> bastion, + xdm.target.host.os = server -> os, + xdm.target.host.os_family = if( + server_os CONTAINS "windows", XDM_CONST.OS_FAMILY_WINDOWS, + server_os CONTAINS "macos", XDM_CONST.OS_FAMILY_MACOS, + server_os CONTAINS "linux", XDM_CONST.OS_FAMILY_LINUX, + server_os CONTAINS "ubuntu", XDM_CONST.OS_FAMILY_UBUNTU, + server_os CONTAINS "debian", XDM_CONST.OS_FAMILY_DEBIAN, + server_os CONTAINS "fedora", XDM_CONST.OS_FAMILY_FEDORA, + server_os CONTAINS "centos", XDM_CONST.OS_FAMILY_CENTOS, + server_os CONTAINS "solaris", XDM_CONST.OS_FAMILY_SOLARIS, + server -> os_type), + xdm.target.cloud.provider = if( + server_cloud_provider IN("aws","amazon*"), XDM_CONST.CLOUD_PROVIDER_AWS, + server_cloud_provider IN("gce","gcp"), XDM_CONST.CLOUD_PROVIDER_GCP, + server_cloud_provider CONTAINS "azure", XDM_CONST.CLOUD_PROVIDER_AZURE, + server_cloud_provider CONTAINS "alibaba", XDM_CONST.CLOUD_PROVIDER_ALIBABA, + server_cloud_provider), + xdm.target.cloud.project_id = server -> instance_details.project_id, + xdm.target.cloud.zone = server -> instance_details.zone_id, + xdm.target.cloud.project_hierarchy = split(server -> instance_details.network, "/"), // Splits the cloud network path to populate the project hierarchy + xdm.target.host.ipv4_public_addresses = if( + is_ipv4(server_instance_details_external_ip) = TRUE, arraycreate(server_instance_details_external_ip)), + xdm.target.host.ipv6_public_addresses = if( + is_ipv6(server_instance_details_internal_ip) = TRUE, arraycreate(server_instance_details_internal_ip)), + xdm.target.host.ipv4_addresses = if( + is_ipv4(server_instance_details_internal_ip) = TRUE, arraycreate(server_instance_details_internal_ip)), + xdm.target.host.ipv6_addresses = if( + is_ipv6(server_instance_details_internal_ip) = TRUE, arraycreate(server_instance_details_internal_ip)), + xdm.target.ipv4 = if( + is_ipv4(server_instance_details_internal_ip) = TRUE, server_instance_details_internal_ip, + is_ipv4(server_instance_details_external_ip) = TRUE, server_instance_details_external_ip), + xdm.target.ipv6 = if( + is_ipv6(server_instance_details_internal_ip) = TRUE, server_instance_details_internal_ip, + is_ipv6(server_instance_details_external_ip) = TRUE, server_instance_details_external_ip), + xdm.target.is_internal_ip = if( + server_instance_details_internal_ip NOT IN ("""""", NULL), TRUE, + server_instance_details_external_ip NOT IN ("""""", NULL), TRUE, + NULL), + xdm.source.cloud.provider = if( + server_source_details_cloud_provider IN("aws","amazon*"), XDM_CONST.CLOUD_PROVIDER_AWS, + server_source_details_cloud_provider IN("gce","gcp"), XDM_CONST.CLOUD_PROVIDER_GCP, + server_source_details_cloud_provider CONTAINS "azure", XDM_CONST.CLOUD_PROVIDER_AZURE, + server_source_details_cloud_provider CONTAINS "alibaba", XDM_CONST.CLOUD_PROVIDER_ALIBABA, + server_source_details_cloud_provider), + xdm.source.cloud.project = server -> source_details.cloud_account, + xdm.logon.logon_guid = server -> source_details.ad_connection_id; \ No newline at end of file diff --git a/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11.yml b/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11.yml new file mode 100644 index 000000000000..99d8120b3a12 --- /dev/null +++ b/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: Okta_ASA_ModelingRule +name: Okta ASA Modeling Rule +rules: '' +schema: '' +tags: '' +supportedModules: +- xsiam diff --git a/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11_schema.json b/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11_schema.json new file mode 100644 index 000000000000..95a28a9e6d9c --- /dev/null +++ b/Packs/OktaASA/ModelingRules/OktaASA_2_11/OktaASA_2_11_schema.json @@ -0,0 +1,32 @@ +{ + "okta_asa_raw": { + "id": { + "type": "string", + "is_array": false + }, + "details": { + "type": "string", + "is_array": false + }, + "user": { + "type": "string", + "is_array": false + }, + "client": { + "type": "string", + "is_array": false + }, + "server": { + "type": "string", + "is_array": false + }, + "project": { + "type": "string", + "is_array": false + }, + "group": { + "type": "string", + "is_array": false + } + } + } \ No newline at end of file diff --git a/Packs/OktaASA/ReleaseNotes/1_0_13.md b/Packs/OktaASA/ReleaseNotes/1_0_13.md new file mode 100644 index 000000000000..138ed01c948d --- /dev/null +++ b/Packs/OktaASA/ReleaseNotes/1_0_13.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: OktaASA Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OktaASA/ReleaseNotes/1_0_14.md b/Packs/OktaASA/ReleaseNotes/1_0_14.md new file mode 100644 index 000000000000..e4d67c67d730 --- /dev/null +++ b/Packs/OktaASA/ReleaseNotes/1_0_14.md @@ -0,0 +1,4 @@ + +## OktaASA + +- Documentation and metadata improvements. diff --git a/Packs/OktaASA/ReleaseNotes/1_0_15.md b/Packs/OktaASA/ReleaseNotes/1_0_15.md new file mode 100644 index 000000000000..b48534f93fd9 --- /dev/null +++ b/Packs/OktaASA/ReleaseNotes/1_0_15.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: Okta ASA Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OktaASA/pack_metadata.json b/Packs/OktaASA/pack_metadata.json index 15951ac82e26..2b1e4de6c05c 100644 --- a/Packs/OktaASA/pack_metadata.json +++ b/Packs/OktaASA/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Okta ASA", "description": "Okta Server access management.", "support": "xsoar", - "currentVersion": "1.0.12", + "currentVersion": "1.0.15", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -27,4 +27,4 @@ "supportedModules": [ "xsiam" ] -} \ No newline at end of file +} diff --git a/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules/OktaAuth0ModelingRules.yml b/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules/OktaAuth0ModelingRules.yml index 45905a0a8d5d..498046accba1 100644 --- a/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules/OktaAuth0ModelingRules.yml +++ b/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules/OktaAuth0ModelingRules.yml @@ -1,4 +1,5 @@ fromversion: 8.2.0 +toversion: 8.14.0 id: auth0_okta_ModelingRule name: Auth0 Okta Modeling Rule rules: '' diff --git a/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11.xif b/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11.xif new file mode 100644 index 000000000000..e7a4a66e99ac --- /dev/null +++ b/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11.xif @@ -0,0 +1,42 @@ +[MODEL: dataset=okta_auth0_raw] +alter + extracted_ip = json_extract_scalar(details, "$.request.ip"), + method = json_extract_scalar(details, "$.request.method"), + status_code = json_extract_scalar(details, "$.response.statusCode") +| alter + ip_v4 = if(ip !~= ":", ip, null), + ip_v6 = if(ip ~= ":", ip, null), + extracted_ipv4 = if(extracted_ip !~= ":", extracted_ip, null), + extracted_ipv6 = if(extracted_ip ~= ":", extracted_ip, null) +| alter + xdm.event.type = type, + xdm.event.description = description, + xdm.event.operation_sub_type = connection, + xdm.session_context_id = connection_id, + xdm.source.agent.identifier = client_id, + xdm.source.agent.content_version = client_name, + xdm.source.ipv4 = ip_v4, + xdm.source.ipv6 = ip_v6, + xdm.source.host.hostname = hostname, + xdm.source.user.identifier = user_id, + xdm.source.identity.identifier = user_id, + xdm.source.user.username = coalesce(user_name, json_extract_scalar(details, "$.request.auth.user.name"), json_extract_scalar(details, "$.request.auth.user.email")), + xdm.source.identity.username = coalesce(user_name, json_extract_scalar(details, "$.request.auth.user.name"), json_extract_scalar(details, "$.request.auth.user.email")), + xdm.observer.content_version = coalesce(audience, json_extract_scalar(details, "$.response.body.audience")), + xdm.source.user.scope = to_string(scope), + xdm.source.identity.scope = to_string(scope), + xdm.observer.name = coalesce(strategy, json_extract_scalar(details, "$.request.auth.strategy")), + xdm.observer.type = strategy_type, + xdm.event.id = log_id, + xdm.source.host.device_category = if(isMobile = true, "mobile", isMobile = false, "desktop/laptop/server", null), + xdm.network.http.method = if(method = "ACL", XDM_CONST.HTTP_METHOD_ACL, method = "BASELINE_CONTROL", XDM_CONST.HTTP_METHOD_BASELINE_CONTROL , method = "BIND", XDM_CONST.HTTP_METHOD_BIND, method = "CHECKIN", XDM_CONST.HTTP_METHOD_CHECKIN, method = "CHECKOUT", XDM_CONST.HTTP_METHOD_CHECKOUT, method = "CONNECT", XDM_CONST.HTTP_METHOD_CONNECT, method = "COPY", XDM_CONST.HTTP_METHOD_COPY, method = "DELETE", XDM_CONST.HTTP_METHOD_DELETE, method = "GET", XDM_CONST.HTTP_METHOD_GET, method = "HEAD", XDM_CONST.HTTP_METHOD_HEAD, method = "LABEL", XDM_CONST.HTTP_METHOD_LABEL, method = "LINK", XDM_CONST.HTTP_METHOD_LINK, method = "LOCK", XDM_CONST.HTTP_METHOD_LOCK, method = "MERGE", XDM_CONST.HTTP_METHOD_MERGE, method = "MKACTIVITY", XDM_CONST.HTTP_METHOD_MKACTIVITY, method = "MKCALENDAR", XDM_CONST.HTTP_METHOD_MKCALENDAR, method = "MKCOL", XDM_CONST.HTTP_METHOD_MKCOL, method = "MKREDIRECTREF", XDM_CONST.HTTP_METHOD_MKREDIRECTREF, method = "MKWORKSPACE", XDM_CONST.HTTP_METHOD_MKWORKSPACE, method = "MOVE", XDM_CONST.HTTP_METHOD_MOVE, method = "OPTIONS", XDM_CONST.HTTP_METHOD_OPTIONS, method = "ORDERPATCH", XDM_CONST.HTTP_METHOD_ORDERPATCH, method = "PATCH", XDM_CONST.HTTP_METHOD_PATCH, method = "POST", XDM_CONST.HTTP_METHOD_POST, method = "PRI", XDM_CONST.HTTP_METHOD_PRI, method = "PROPFIND", XDM_CONST.HTTP_METHOD_PROPFIND, method = "PROPPATCH", XDM_CONST.HTTP_METHOD_PROPPATCH, method = "PUT", XDM_CONST.HTTP_METHOD_PUT, method = "REBIND", XDM_CONST.HTTP_METHOD_REBIND, method = "REPORT", XDM_CONST.HTTP_METHOD_REPORT, method = "SEARCH", XDM_CONST.HTTP_METHOD_SEARCH, method = "TRACE", XDM_CONST.HTTP_METHOD_TRACE, method = "UNBIND", XDM_CONST.HTTP_METHOD_UNBIND, method = "UNCHECKOUT", XDM_CONST.HTTP_METHOD_UNCHECKOUT, method = "UNLINK", XDM_CONST.HTTP_METHOD_UNLINK, method = "UNLOCK", XDM_CONST.HTTP_METHOD_UNLOCK, method = "UPDATE", XDM_CONST.HTTP_METHOD_UPDATE, method = "UPDATEREDIRECTREF", XDM_CONST.HTTP_METHOD_UPDATEREDIRECTREF, method = "VERSION_CONTROL", XDM_CONST.HTTP_METHOD_VERSION_CONTROL, method = null, null, to_string(method)), + xdm.source.user_agent = coalesce(json_extract_scalar(details, "$.request.userAgent"), user_agent), + xdm.network.http.url = json_extract_scalar(details, "$.request.channel"), + xdm.source.host.ipv4_addresses = arraycreate(extracted_ipv4), + xdm.source.host.ipv6_addresses = arraycreate(extracted_ipv6), + xdm.target.host.hostname = coalesce(json_extract_scalar(details, "$.response.body.name"), json_extract_scalar(details, "$.response.body.friendly_name")), + xdm.auth.auth_method = json_extract_scalar(details, "$.response.body.default_directory"), + xdm.network.http.content_type = arraystring(json_extract_scalar_array(details, "$.response.body.enabled_locales"), ", "), + xdm.network.rule = details -> response.body.flags{}, + xdm.network.http.response_code = if(status_code = "100", XDM_CONST.HTTP_RSP_CODE_CONTINUE, status_code = "101", XDM_CONST.HTTP_RSP_CODE_SWITCHING_PROTOCOLS, status_code = "102", XDM_CONST.HTTP_RSP_CODE_PROCESSING, status_code = "103", XDM_CONST.HTTP_RSP_CODE_EARLY_HINTS, status_code = "200", XDM_CONST.HTTP_RSP_CODE_OK, status_code = "201", XDM_CONST.HTTP_RSP_CODE_CREATED, status_code = "202", XDM_CONST.HTTP_RSP_CODE_ACCEPTED, status_code = "203", XDM_CONST.HTTP_RSP_CODE_NON__AUTHORITATIVE_INFORMATION, status_code = "204", XDM_CONST.HTTP_RSP_CODE_NO_CONTENT, status_code = "205", XDM_CONST.HTTP_RSP_CODE_RESET_CONTENT, status_code = "206", XDM_CONST.HTTP_RSP_CODE_PARTIAL_CONTENT, status_code = "207", XDM_CONST.HTTP_RSP_CODE_MULTI__STATUS, status_code = "208", XDM_CONST.HTTP_RSP_CODE_ALREADY_REPORTED, status_code = "226", XDM_CONST.HTTP_RSP_CODE_IM_USED, status_code = "300", XDM_CONST.HTTP_RSP_CODE_MULTIPLE_CHOICES, status_code = "301", XDM_CONST.HTTP_RSP_CODE_MOVED_PERMANENTLY, status_code = "302", XDM_CONST.HTTP_RSP_CODE_FOUND, status_code = "303", XDM_CONST.HTTP_RSP_CODE_SEE_OTHER, status_code = "304", XDM_CONST.HTTP_RSP_CODE_NOT_MODIFIED, status_code = "305", XDM_CONST.HTTP_RSP_CODE_USE_PROXY, status_code = "307", XDM_CONST.HTTP_RSP_CODE_TEMPORARY_REDIRECT, status_code = "308", XDM_CONST.HTTP_RSP_CODE_PERMANENT_REDIRECT, status_code = "400", XDM_CONST.HTTP_RSP_CODE_BAD_REQUEST, status_code = "401", XDM_CONST.HTTP_RSP_CODE_UNAUTHORIZED, status_code = "402", XDM_CONST.HTTP_RSP_CODE_PAYMENT_REQUIRED, status_code = "403", XDM_CONST.HTTP_RSP_CODE_FORBIDDEN, status_code = "404", XDM_CONST.HTTP_RSP_CODE_NOT_FOUND, status_code = "405", XDM_CONST.HTTP_RSP_CODE_METHOD_NOT_ALLOWED, status_code = "406", XDM_CONST.HTTP_RSP_CODE_NOT_ACCEPTABLE, status_code = "407", XDM_CONST.HTTP_RSP_CODE_PROXY_AUTHENTICATION_REQUIRED, status_code = "408", XDM_CONST.HTTP_RSP_CODE_REQUEST_TIMEOUT, status_code = "409", XDM_CONST.HTTP_RSP_CODE_CONFLICT, status_code = "410", XDM_CONST.HTTP_RSP_CODE_GONE, status_code = "411", XDM_CONST.HTTP_RSP_CODE_LENGTH_REQUIRED, status_code = "412", XDM_CONST.HTTP_RSP_CODE_PRECONDITION_FAILED, status_code = "413", XDM_CONST.HTTP_RSP_CODE_CONTENT_TOO_LARGE, status_code = "414", XDM_CONST.HTTP_RSP_CODE_URI_TOO_LONG, status_code = "415", XDM_CONST.HTTP_RSP_CODE_UNSUPPORTED_MEDIA_TYPE, status_code = "416", XDM_CONST.HTTP_RSP_CODE_RANGE_NOT_SATISFIABLE, status_code = "417", XDM_CONST.HTTP_RSP_CODE_EXPECTATION_FAILED, status_code = "421", XDM_CONST.HTTP_RSP_CODE_MISDIRECTED_REQUEST, status_code = "422", XDM_CONST.HTTP_RSP_CODE_UNPROCESSABLE_CONTENT, status_code = "423", XDM_CONST.HTTP_RSP_CODE_LOCKED, status_code = "424", XDM_CONST.HTTP_RSP_CODE_FAILED_DEPENDENCY, status_code = "425", XDM_CONST.HTTP_RSP_CODE_TOO_EARLY, status_code = "426", XDM_CONST.HTTP_RSP_CODE_UPGRADE_REQUIRED, status_code = "428", XDM_CONST.HTTP_RSP_CODE_PRECONDITION_REQUIRED, status_code = "429", XDM_CONST.HTTP_RSP_CODE_TOO_MANY_REQUESTS, status_code = "431", XDM_CONST.HTTP_RSP_CODE_REQUEST_HEADER_FIELDS_TOO_LARGE, status_code = "451", XDM_CONST.HTTP_RSP_CODE_UNAVAILABLE_FOR_LEGAL_REASONS, status_code = "500", XDM_CONST.HTTP_RSP_CODE_INTERNAL_SERVER_ERROR, status_code = "501", XDM_CONST.HTTP_RSP_CODE_NOT_IMPLEMENTED, status_code = "502", XDM_CONST.HTTP_RSP_CODE_BAD_GATEWAY, status_code = "503", XDM_CONST.HTTP_RSP_CODE_SERVICE_UNAVAILABLE, status_code = "504", XDM_CONST.HTTP_RSP_CODE_GATEWAY_TIMEOUT, status_code = "505", XDM_CONST.HTTP_RSP_CODE_HTTP_VERSION_NOT_SUPPORTED, status_code = "506", XDM_CONST.HTTP_RSP_CODE_VARIANT_ALSO_NEGOTIATES, status_code = "507", XDM_CONST.HTTP_RSP_CODE_INSUFFICIENT_STORAGE, status_code = "508", XDM_CONST.HTTP_RSP_CODE_LOOP_DETECTED, status_code = "511", XDM_CONST.HTTP_RSP_CODE_NETWORK_AUTHENTICATION_REQUIRED, status_code = null, null, to_string(status_code)), + xdm.target.module.directory = json_extract_scalar(details, "$.request.path"); diff --git a/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11.yml b/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11.yml new file mode 100644 index 000000000000..77b6b71ae51d --- /dev/null +++ b/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: auth0_okta_ModelingRule +name: Auth0 Okta Modeling Rule +rules: '' +schema: '' +tags: auth0 +supportedModules: +- xsiam diff --git a/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11_schema.json b/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11_schema.json new file mode 100644 index 000000000000..7f57909ddf3c --- /dev/null +++ b/Packs/OktaAuth0/ModelingRules/OktaAuth0ModelingRules_2_11/OktaAuth0ModelingRules_2_11_schema.json @@ -0,0 +1,76 @@ +{ + "okta_auth0_raw": { + "type": { + "type": "string", + "is_array": false + }, + "description": { + "type": "string", + "is_array": false + }, + "connection": { + "type": "string", + "is_array": false + }, + "connection_id": { + "type": "string", + "is_array": false + }, + "client_id": { + "type": "string", + "is_array": false + }, + "client_name": { + "type": "string", + "is_array": false + }, + "ip": { + "type": "string", + "is_array": false + }, + "hostname": { + "type": "string", + "is_array": false + }, + "user_id": { + "type": "string", + "is_array": false + }, + "user_name": { + "type": "string", + "is_array": false + }, + "audience": { + "type": "string", + "is_array": false + }, + "details": { + "type": "string", + "is_array": false + }, + "scope": { + "type": "string", + "is_array": false + }, + "strategy": { + "type": "string", + "is_array": false + }, + "strategy_type": { + "type": "string", + "is_array": false + }, + "log_id": { + "type": "string", + "is_array": false + }, + "isMobile": { + "type": "boolean", + "is_array": false + }, + "user_agent": { + "type": "string", + "is_array": false + } + } +} \ No newline at end of file diff --git a/Packs/OktaAuth0/ReleaseNotes/1_0_12.md b/Packs/OktaAuth0/ReleaseNotes/1_0_12.md new file mode 100644 index 000000000000..a6fbbcf92f59 --- /dev/null +++ b/Packs/OktaAuth0/ReleaseNotes/1_0_12.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: Auth0 Okta Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OktaAuth0/pack_metadata.json b/Packs/OktaAuth0/pack_metadata.json index ee8427c6d6a3..0377321c15b8 100644 --- a/Packs/OktaAuth0/pack_metadata.json +++ b/Packs/OktaAuth0/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Okta Auth0", "description": "Identity platform to manage access to your applications.", "support": "xsoar", - "currentVersion": "1.0.11", + "currentVersion": "1.0.12", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/OktaOAG/ModelingRules/OktaOAG_1_3/OktaOAG_1_3.yml b/Packs/OktaOAG/ModelingRules/OktaOAG_1_3/OktaOAG_1_3.yml index f998af804866..e7a97f3f4b7f 100644 --- a/Packs/OktaOAG/ModelingRules/OktaOAG_1_3/OktaOAG_1_3.yml +++ b/Packs/OktaOAG/ModelingRules/OktaOAG_1_3/OktaOAG_1_3.yml @@ -1,4 +1,5 @@ fromversion: 6.10.0 +toversion: 8.14.0 id: okta_Access_gateway_modeling_rule name: Okta Access Gateway rules: '' diff --git a/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11.xif b/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11.xif new file mode 100644 index 000000000000..3f81cf5a621c --- /dev/null +++ b/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11.xif @@ -0,0 +1,92 @@ +[MODEL: dataset="okta_oag_raw"] +filter _raw_log contains "monitor" +|alter + event = arrayindex(regextract(_raw_log,"\s(\w+)\s"),2), + description = arrayindex(regextract(_raw_log,"\]\s(.*)$"),0), + logLevel = arrayindex(regextract(_raw_log,"\s([A-Z]+)\s\S+\s\["),0), + hostname=arrayindex(regextract(_raw_log,"HOSTNAME\=\"([^\"]+)\""),0), + outcome=arrayindex(regextract(_raw_log,"STATUS\=\"([^\"]+)\""),0), + component=arrayindex(regextract(_raw_log,"\s([A-Z]+)\s\S+\s"),0) +| alter xdm.target.user.username = hostname, + xdm.target.identity.username = hostname, + xdm.event.operation = event, + xdm.event.description = description, + xdm.target.resource.sub_type = component, + xdm.alert.severity = logLevel, + xdm.event.outcome=outcome; + +filter _raw_log contains "ACCESS" +|alter event = arrayindex(regextract(_raw_log,"\s(\w+)\s\["),0), +description = arrayindex(regextract(_raw_log,"\]\s(.*)$"),0), +logLevel = arrayindex(regextract(_raw_log,"\s([A-Z]+)\s\S+\s\["),0), +component=arrayindex(regextract(_raw_log,"\s([A-Z]+)\s\S+\s"),0), +app=arrayindex(regextract(_raw_log,"APP\=\"([^\"]+)\""),0), +targetIP=arrayindex(regextract(_raw_log,"REMOTE_IP\=\"([^\"]+)\""),0), +userAgent=arrayindex(regextract(_raw_log,"USER_AGENT\=\"([^\"]+)\""),0), +outcome=arrayindex(regextract(_raw_log,"RESULT\=\"([^\"]+)\""),0), +outcomeReason=arrayindex(regextract(_raw_log,"REASON\=\"([^\"]+)\""),0), +username=arrayindex(regextract(_raw_log,"SUBJECT\=\"([^\"]+)\""),0), +username2=arrayindex(regextract(_raw_log,"username\=(\w+)\s"),0), +sessionID=arrayindex(regextract(_raw_log,"SESSION_ID\=\"([^\"]+)\""),0), +HTTPmethod=arrayindex(regextract(_raw_log,"METHOD\=\"([^\"]+)\""),0), +url=arrayindex(regextract(_raw_log,"url\=(.*)\"\sM"),0) + +|alter xdm.event.operation = event, +xdm.event.description = description, +xdm.target.resource.sub_type = component, +xdm.alert.severity = logLevel, +xdm.event.outcome=outcome, +xdm.source.application.name=app, +xdm.target.ipv4=targetIP, +xdm.event.outcome_reason=outcomeReason, +xdm.source.user.username=coalesce(username,username2), +xdm.source.identity.username=coalesce(username,username2), +xdm.source.user_agent=userAgent, +xdm.session_context_id=sessionID, +xdm.network.http.method=HTTPmethod, +xdm.network.http.url=url; + +filter _raw_log contains "CHECK" and _raw_log NOT contains "SCRIPT" and _raw_log NOT contains "HOST_IP_CHECK" +|alter event = arrayindex(regextract(_raw_log,"\s([\w\_]+)\s"),0), +logLevel = arrayindex(regextract(_raw_log,"\s(\w+)\s"),1), +component=arrayindex(regextract(_raw_log,"\s([\w\.]+)\s"),1), +targetIP=arrayindex(regextract(_raw_log,"([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})"),0), +username=arrayindex(regextract(_raw_log,"USER\=\"([^\"]+)\""),0) +|alter xdm.event.operation = event, +xdm.target.resource.sub_type = component, +xdm.alert.severity = logLevel, +xdm.target.ipv4=targetIP, +xdm.source.user.username=username, +xdm.source.identity.username=username; + +filter _raw_log contains "CHECK" and _raw_log contains "SCRIPT" and _raw_log NOT contains "HOST_IP_CHECK" +|alter event = arrayindex(regextract(_raw_log,"\s([\w\_]+)\s"),0), +logLevel = arrayindex(regextract(_raw_log,"\s([A-Z]+)\s\w+\s\["),0), +username=arrayindex(regextract(_raw_log,"USER\=\"([^\"]+)\""),0) +|alter xdm.event.operation = event, +xdm.alert.severity = logLevel, +xdm.source.user.username=username, +xdm.source.identity.username=username; + +filter _raw_log contains "HOST_IP_CHECK" +|alter event = arrayindex(regextract(_raw_log,"\s([\w\_]+)\s"),0), +logLevel = arrayindex(regextract(_raw_log,"\s([A-Z]+)\s\w+\s\["),0), +username=arrayindex(regextract(_raw_log,"USER\=\"([^\"]+)\""),0), +description = arrayindex(regextract(_raw_log,"\]\s(.*)$"),0) +|alter xdm.event.operation = event, +xdm.alert.severity = logLevel, +xdm.source.user.username=username, +xdm.source.identity.username=username, +xdm.event.description = description; + +filter (_raw_log contains "GET" or _raw_log contains "POST") and _raw_log NOT contains "ACCESS" +|alter url=arrayindex(regextract(_raw_log,"\s([\w\/.]+)\sHTTP"),0), +httpMethod=arrayindex(regextract(_raw_log,"\s\"([\"\w]+)\s\/"),0), +statusCode=arrayindex(regextract(_raw_log,"\"\s(\w+)\s"),0), +userAgent=arrayindex(regextract(_raw_log,"\"\s\"(.*)\"\s\""),0), +domain=arrayindex(regextract(_raw_log,"\s([\w.-]+)\s"),1) +|alter xdm.source.user_agent = userAgent, +xdm.network.http.method=httpMethod, +xdm.network.http.url=url, +xdm.network.http.response_code=statusCode, +xdm.target.host.fqdn=domain; \ No newline at end of file diff --git a/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11.yml b/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11.yml new file mode 100644 index 000000000000..ba98b482f67c --- /dev/null +++ b/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: OktaOAG_ModelingRule +name: OktaOAG Modeling Rule +rules: '' +schema: '' +tags: Okta Access Gateway +supportedModules: +- xsiam diff --git a/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11_schema.json b/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11_schema.json new file mode 100644 index 000000000000..fb8688ed261a --- /dev/null +++ b/Packs/OktaOAG/ModelingRules/OktaOAG_2_11/OktaOAG_2_11_schema.json @@ -0,0 +1,8 @@ +{ + "okta_oag_raw": { + "_raw_log": { + "type": "string", + "is_array": false + } + } +} \ No newline at end of file diff --git a/Packs/OktaOAG/ReleaseNotes/1_0_12.md b/Packs/OktaOAG/ReleaseNotes/1_0_12.md new file mode 100644 index 000000000000..cb2bb296ce5b --- /dev/null +++ b/Packs/OktaOAG/ReleaseNotes/1_0_12.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: OktaOAG Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OktaOAG/pack_metadata.json b/Packs/OktaOAG/pack_metadata.json index 3354a56c7995..7b91b8c5c8ea 100644 --- a/Packs/OktaOAG/pack_metadata.json +++ b/Packs/OktaOAG/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Okta Access Gateway", "description": "Okta Access Gateway is a reverse proxy based virtual application, designed to secure web applications that don't natively support SAML or OIDC.", "support": "xsoar", - "currentVersion": "1.0.11", + "currentVersion": "1.0.12", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/OktaStandardConnector/.pack-ignore b/Packs/OktaStandardConnector/.pack-ignore new file mode 100644 index 000000000000..7444c0b71896 --- /dev/null +++ b/Packs/OktaStandardConnector/.pack-ignore @@ -0,0 +1,14 @@ +[file:README.md] +ignore=RM104 + +[file:OktaStandardConnector.yml] +ignore=PR101,BA129,BA127 + +[file:OktaStandardConnector_image.png] +ignore=IM111 + +[file:OktaIAMStandardConnector.yml] +ignore=PR101,BA129,BA127 + +[file:OktaIAMStandardConnector_image.png] +ignore=IM111 diff --git a/Packs/OktaStandardConnector/.secrets-ignore b/Packs/OktaStandardConnector/.secrets-ignore new file mode 100644 index 000000000000..d6bd0341728a --- /dev/null +++ b/Packs/OktaStandardConnector/.secrets-ignore @@ -0,0 +1,42 @@ +momo@demisto.com +TestID2@demisto.com +test2@demisto.com +soso@demisto.com +8.8.8.8 +woo@demisto.com +john@doe.com +test@this.com +https://test.com +127.0.0.1 +Test1@test.com +https://developer.okta.com +test@that.com +user2@demisto.com +factor@test.com +https://dev-725178.oktapreview.com +bartest@test.com +testForDocs@test.co +Test@demisto.com +XSOAR@demisto.com +https://yourdomain.okta.com +1.1.0.0 +https://dev-530328.oktapreview.com +1.2.1.2 +4.5.3.2 +34.103.1.108 +2.2.2.3 +5.3.246.229 +5.3.246.228 +3.5.146.103 +5.3.143.103 +1.3.1.5 +https://dev-950355.oktapreview.com +3.5.1.229 +1.1.3.5 +3.5.1.228 +3.3.3.1 +5.5.5.3 +1.3.1.3 +https://op1static.oktacdn.com +dbot@xsoar.com +XDM_CONST.OUTCOME_PARTIAL, diff --git a/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector.py b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector.py new file mode 100644 index 000000000000..0e407d50a67a --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector.py @@ -0,0 +1,14 @@ +import demistomock as demisto # noqa: F401 +from CommonServerPython import * # noqa: F401 + +from OktaIAMApiModule import * # noqa: E402 + + +def main() -> None: + run_okta_iam_integration() + + +from CommonServerUserPython import * # noqa: E402 # pylint: disable=wrong-import-position + +if __name__ in ("__main__", "__builtin__", "builtins"): + main() diff --git a/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector.yml b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector.yml new file mode 100644 index 000000000000..f4a6e2feaae7 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector.yml @@ -0,0 +1,507 @@ +category: Authentication & Identity Management +provider: Okta +sectionorder: +- Connect +- Collect +commonfields: + id: OktaIAMStandardConnector + version: -1 +configuration: +- display: Okta URL (https://.okta.com) + name: url + required: true + type: 0 + section: Connect +- name: credentials + type: 9 + displaypassword: API Token (see detailed instructions) + hiddenusername: true + section: Connect + required: false +- display: API Token (See detailed instructions) + name: apitoken + type: 4 + hidden: true + section: Connect + required: false +- display: Trust any certificate (not secure) + name: insecure + type: 8 + section: Connect + advanced: true + required: false +- display: Use system proxy settings + name: proxy + type: 8 + section: Connect + advanced: true + required: false +- defaultvalue: 'true' + display: Allow creating users + name: create-user-enabled + type: 8 + section: Collect + advanced: true + required: false +- defaultvalue: 'true' + display: Allow updating users + name: update-user-enabled + type: 8 + section: Collect + advanced: true + required: false +- defaultvalue: 'true' + display: Allow enabling users + name: enable-user-enabled + type: 8 + section: Collect + advanced: true + required: false +- defaultvalue: 'true' + display: Allow disabling users + name: disable-user-enabled + type: 8 + section: Collect + advanced: true + required: false +- defaultvalue: 'true' + display: Automatically create user if not found in update command + name: create-if-not-exists + type: 8 + section: Collect + advanced: true + required: false +- defaultvalue: User Profile - Okta (Incoming) + display: Incoming Mapper + name: mapper-in + required: false + type: 0 + section: Connect +- defaultvalue: User Profile - Okta (Outgoing) + display: Outgoing Mapper + name: mapper-out + required: false + type: 0 + section: Connect +description: 'This integration is configured automatically as part of the Okta Standard Connector. Do not configure this integration directly — set it up from the connector page instead.' +display: Okta IAM (Standard Connector) +name: OktaIAMStandardConnector +script: + commands: + - arguments: + - description: User Profile indicator details. + name: user-profile + required: true + - auto: PREDEFINED + defaultValue: 'true' + description: When set to true, after the command execution the status of the user in the 3rd-party integration will be active. + name: allow-enable + predefined: + - 'true' + - 'false' + description: Creates a user. + execution: true + name: iam-create-user + outputs: + - contextPath: IAM.Vendor.active + description: When true, indicates that the employee's status is active in the 3rd-party integration. + type: Boolean + - contextPath: IAM.Vendor.brand + description: Name of the integration. + type: String + - contextPath: IAM.Vendor.details + description: Provides the raw data from the 3rd-party integration. + type: string + - contextPath: IAM.Vendor.email + description: The employee's email address. + type: String + - contextPath: IAM.Vendor.errorCode + description: HTTP error response code. + type: Number + - contextPath: IAM.Vendor.errorMessage + description: Reason why the API failed. + type: String + - contextPath: IAM.Vendor.id + description: The employee's user ID in the app. + type: String + - contextPath: IAM.Vendor.instanceName + description: Name of the integration instance. + type: string + - contextPath: IAM.Vendor.success + description: When true, indicates that the command was executed successfully. + type: Boolean + - contextPath: IAM.Vendor.username + description: The employee's username in the app. + type: String + - arguments: + - description: A User Profile indicator. + name: user-profile + required: true + - auto: PREDEFINED + defaultValue: 'true' + description: When set to true, after the command execution the status of the user in the 3rd-party integration will be active. + name: allow-enable + predefined: + - 'true' + - 'false' + description: Updates an existing user with the data passed in the user-profile argument. + execution: true + name: iam-update-user + outputs: + - contextPath: IAM.Vendor.active + description: When true, indicates that the employee's status is active in the 3rd-party integration. + type: Boolean + - contextPath: IAM.Vendor.brand + description: Name of the integration. + type: String + - contextPath: IAM.Vendor.details + description: Provides the raw data from the 3rd-party integration. + type: string + - contextPath: IAM.Vendor.email + description: The employee's email address. + type: String + - contextPath: IAM.Vendor.errorCode + description: HTTP error response code. + type: Number + - contextPath: IAM.Vendor.errorMessage + description: Reason why the API failed. + type: String + - contextPath: IAM.Vendor.id + description: The employee's user ID in the app. + type: String + - contextPath: IAM.Vendor.instanceName + description: Name of the integration instance. + type: string + - contextPath: IAM.Vendor.success + description: When true, indicates that the command was executed successfully. + type: Boolean + - contextPath: IAM.Vendor.username + description: The employee's username in the app. + type: String + compliantpolicies: + - User Hard Remediation + - arguments: + - description: A User Profile indicator. + name: user-profile + required: true + description: Retrieves a single user resource. + name: iam-get-user + outputs: + - contextPath: IAM.Vendor.active + description: When true, indicates that the employee's status is active in the 3rd-party integration. + type: Boolean + - contextPath: IAM.Vendor.brand + description: Name of the integration. + type: String + - contextPath: IAM.Vendor.details + description: Provides the raw data from the 3rd-party integration. + type: string + - contextPath: IAM.Vendor.email + description: The employee's email address. + type: String + - contextPath: IAM.Vendor.errorCode + description: HTTP error response code. + type: Number + - contextPath: IAM.Vendor.errorMessage + description: Reason why the API failed. + type: String + - contextPath: IAM.Vendor.id + description: The employee's user ID in the app. + type: String + - contextPath: IAM.Vendor.instanceName + description: Name of the integration instance. + type: string + - contextPath: IAM.Vendor.success + description: When true, indicates that the command was executed successfully. + type: Boolean + - contextPath: IAM.Vendor.username + description: The employee's username in the app. + type: String + - arguments: + - description: A User Profile indicator. + name: user-profile + required: true + description: Disable an active user. + execution: true + name: iam-disable-user + outputs: + - contextPath: IAM.Vendor.active + description: When true, indicates that the employee's status is active in the 3rd-party integration. + type: Boolean + - contextPath: IAM.Vendor.brand + description: Name of the integration. + type: String + - contextPath: IAM.Vendor.details + description: Provides the raw data from the 3rd-party integration. + type: string + - contextPath: IAM.Vendor.email + description: The employee's email address. + type: String + - contextPath: IAM.Vendor.errorCode + description: HTTP error response code. + type: Number + - contextPath: IAM.Vendor.errorMessage + description: Reason why the API failed. + type: String + - contextPath: IAM.Vendor.id + description: The employee's user ID in the app. + type: String + - contextPath: IAM.Vendor.instanceName + description: Name of the integration instance. + type: string + - contextPath: IAM.Vendor.success + description: When true, indicates that the command was executed successfully. + type: Boolean + - contextPath: IAM.Vendor.username + description: The employee's username in the app. + type: String + compliantpolicies: + - User Hard Remediation + - description: Retrieves a User Profile schema which holds all of the user fields within the application. Used for outgoing-mapping through the Get Schema option. + name: get-mapping-fields + - arguments: + - description: ID of the user for which to get information. + name: user_id + required: true + - description: ID of the application for which to get information. + name: application_id + required: true + description: Returns information about application's user assignment. + name: okta-get-app-user-assignment + outputs: + - contextPath: Okta.AppUserAssignment.UserID + description: ID of the user. + type: string + - contextPath: Okta.AppUserAssignment.AppID + description: ID of the application. + type: string + - contextPath: Okta.AppUserAssignment.IsAssigned + description: When True, indicates that the user is assigned to the application. + type: boolean + - contextPath: Okta.AppUserAssignment.ProfileInApp + description: The user profile data in the application. + type: unknown + - arguments: + - description: Search for applications by their names. + name: query + - defaultValue: '0' + description: Page number (0-based). + name: page + - defaultValue: '50' + description: Maximum number of apps to retrieve (maximal value is 200). + name: limit + description: Returns a list of Okta applications data. + name: okta-iam-list-applications + outputs: + - contextPath: Okta.Application.ID + description: ID of the application. + type: string + - contextPath: Okta.Application.Name + description: Name of the application. + type: string + - contextPath: Okta.Application.Label + description: Label of the application. + type: string + - contextPath: Okta.Application.Logo + description: Logo of the application. + type: string + - arguments: + - description: IAM configuration data. + name: configuration + required: true + description: Updates IAM configuration data in the integration context. + name: okta-iam-set-configuration + - description: Gets the IAM configuration data from the integration context. + name: okta-iam-get-configuration + outputs: + - contextPath: Okta.IAMConfiguration.ApplicationID + description: An Okta application ID. + type: String + - contextPath: Okta.IAMConfiguration.Name + description: Name of the Okta application. + type: String + - contextPath: Okta.IAMConfiguration.Label + description: Label of the Okta application. + type: String + - contextPath: Okta.IAMConfiguration.Logo + description: Logo of the application. + type: String + - contextPath: Okta.IAMConfiguration.Instance + description: An XSOAR IAM integration instance name. + type: String + - arguments: + - description: ID of the user for which to get the information. + name: user_id + required: true + description: List all applications assigned to a user. + name: okta-iam-list-user-applications + outputs: + - contextPath: Okta.Application.ID + description: ID of the application. + type: String + - contextPath: Okta.Application.Label + description: Label of the application. + type: String + - contextPath: Okta.Application.Name + description: Name of the application. + type: String + - contextPath: Okta.Application.Status + description: Status of the application. + type: String + - arguments: + - description: Group SCIM Data. + name: scim + required: true + - auto: PREDEFINED + defaultValue: 'true' + description: 'Field to indicate if members need to be included in the response. ' + name: includeMembers + predefined: + - 'true' + - 'false' + required: true + description: Retrieves the group information, including its members. + name: iam-get-group + outputs: + - contextPath: GetGroup.id + description: ID of the group. + type: String + - contextPath: GetGroup.displayName + description: The display name of the group. + type: String + - contextPath: GetGroup.members.display + description: The display name of the group member. + type: String + - contextPath: GetGroup.members.value + description: ID of the group member. + type: String + - contextPath: GetGroup.success + description: Indicates whether the command succeeded. + type: Boolean + - contextPath: GetGroup.errorCode + description: HTTP error response code. + type: Number + - contextPath: GetGroup.errorMessage + description: Reason why the API failed. + type: String + - arguments: + - description: 'Useful for performing structured queries where constraints on LogEvent attribute values can be explicitly targeted. The following expressions are supported for events with the filter query parameter: eventType eq " :eventType" -Events that have a specific action; eventType target.id eq ":id" - Events published with a specific target id; actor.id eq ":id"- Events published with a specific actor ID. For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering' + name: filter + - description: 'Filters the lower time bound of the log events in the Internet Date/Time Format profile of ISO 8601. For example: 2017-05-03T16:22:18Z.' + name: since + - description: 'Filters the upper time bound of the log events in the Internet Date/Time Format profile of ISO 8601. For example: 2017-05-03T16:22:18Z.' + name: until + - auto: PREDEFINED + defaultValue: ASCENDING + description: The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". + name: sortOrder + predefined: + - ASCENDING + - DESCENDING + - defaultValue: '100' + description: The maximum number of results to return, between 0 and 1000 maximum. The default is 100. + name: limit + description: Gets logs by providing optional filters. + name: okta-get-logs + outputs: + - contextPath: Okta.Logs.Events.actor.alternateId + description: Alternative ID of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.displayName + description: Display name of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.id + description: ID of the actor. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.rawUserAgent + description: A raw string representation of user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.os + description: The operating system on which the client runs. For example, Microsoft Windows 10. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.browser + description: Identifies the type of web browser, if relevant. For example, Chrome. + type: String + - contextPath: Okta.Logs.Events.client.device + description: Type of device from which the client operated. For example, Computer. + type: String + - contextPath: Okta.Logs.Events.client.id + description: For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. + type: String + - contextPath: Okta.Logs.Events.client.ipAddress + description: IP address from which the client made its request. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.displayMessage + description: The display message for an event. + type: String + - contextPath: Okta.Logs.Events.eventType + description: Type of event that was published. + type: String + - contextPath: Okta.Logs.Events.outcome.result + description: Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". + type: String + - contextPath: Okta.Logs.Events.outcome.reason + description: Reason for the result. For example, INVALID_CREDENTIALS. + type: String + - contextPath: Okta.Logs.Events.published + description: Timestamp when the event was published. + type: String + - contextPath: Okta.Logs.Events.severity + description: The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". + type: String + - contextPath: Okta.Logs.Events.securityContext.asNumber + description: Autonomous system number associated with the autonomous system that the event request was sourced to. + type: Number + - contextPath: Okta.Logs.Events.securityContext.asOrg + description: Organization associated with the autonomous system that the event request was sourced to. + type: String + - contextPath: Okta.Logs.Events.securityContext.isp + description: Internet service provider used to send the event's request. + type: String + - contextPath: Okta.Logs.Events.securityContext.domain + description: Specifies whether an event's request is from a known proxy. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.IP + description: IP address. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.source + description: Details regarding the source. + type: String + - contextPath: Okta.Logs.Events.target.id + description: ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.type + description: Type of a target. + type: String + - contextPath: Okta.Logs.Events.target.alternateId + description: Alternative ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.displayName + description: Display name of a target. + type: String + dockerimage: demisto/python3:3.12.14.13053055 + runonce: false + script: '-' + subtype: python3 + type: python +tests: +- No tests +fromversion: 8.15.0 diff --git a/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_dark.svg b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_dark.svg new file mode 100644 index 000000000000..80498f5edf45 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_dark.svg @@ -0,0 +1,10 @@ + + + + + + + + + + diff --git a/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_description.md b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_description.md new file mode 100644 index 000000000000..d50e1224efc4 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_description.md @@ -0,0 +1,27 @@ +## Get Your Okta API Token + +In order to use this integration, you need to enter your Okta API token in the **API Token** integration instance parameter. For information on getting your Okta API token, see the [Okta documentation](https://developer.okta.com/docs/api/getting_started/getting_a_token). + +## Enable/Disable CRUD Commands +You can select which CRUD commands are enabled in the integration instance configuration settings. By default, all commands are enabled. + +## Add Custom Indicator Fields +Follow these steps to add custom fields to the User Profile indicator. + +1. In XSOAR, create the custom indicator and incident field, for example, **Middle Name**. +2. Duplicate the **User Profile - Okta (Incoming)** mapper and/or the **User Profile - Okta (Outging)** mapper. +3. Add and map the custom field to the necessary mapper(s). +4. Go to the Okta IAM integration instance and in the mapper textbox, replace the name of the default mapper with the custom mapper you created. + +## Automatically create user if not found in update command +The *create-if-not-exists* parameter specifies if a new user should be created when the User Profile passed was not found in the 3rd-party integration. + +## Fetch incidents using an "IAM - Configuration" incident +When the "Query only application events configured in IAM Configuration" checkbox is selected, add or remove event types for the applications you configured in the **IAM Configuration** incident are retrieved. You must have at least one application configured in XSOAR to fetch incidents from Okta. + +## Fetch incidents using a manual query filter expression +**Note: Cortex XSOAR recommends you use the Query only application events configured in IAM Configuration option to generate the fetch-incidents query filter. The following following method should be used primarily for debugging purposes.** +Clear the "Query only application events configured in IAM Configuration" checkbox to use a custom fetch query filter expression. The expression must be in SCIM syntax, and include the add and remove event types, as well as the application ID. +For example: `(eventType eq "application.user_membership.add" or eventType eq "application.user_membership.remove") and target.id eq "0oar418fvkm67MWGd0h7"` +You may also use the advanced search in Okta's System Logs to generate the filter expression. +For more details, visit [Okta API reference](https://developer.okta.com/docs/reference/api/system-log/#expression-filter). diff --git a/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_image.png b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_image.png new file mode 100644 index 000000000000..e1ac854dd8b7 Binary files /dev/null and b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_image.png differ diff --git a/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_light.svg b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_light.svg new file mode 100644 index 000000000000..aaa67e475168 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_light.svg @@ -0,0 +1,10 @@ + + + + + + + + + + diff --git a/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_test.py b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_test.py new file mode 100644 index 000000000000..75e0f58b90f1 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/OktaIAMStandardConnector_test.py @@ -0,0 +1,43 @@ +"""Sanity tests for the OktaIAMStandardConnector shim integration. + +Full behavioural coverage lives in the OktaIAMApiModule tests; this file +exists only to confirm the shim wires through to the ApiModule correctly. +""" + +from unittest.mock import patch # noqa: F401 + +import OktaIAMStandardConnector as integration_module +import pytest + + +def test_shim_imports_run_entry_point(): + """The shim must re-export the ApiModule entry point under its expected name.""" + assert hasattr( + integration_module, "run_okta_iam_integration" + ), "OktaIAMApiModule.run_okta_iam_integration must be importable via the shim" + + +def test_shim_imports_client(): + """The shim must re-export the Okta IAM Client class.""" + assert hasattr(integration_module, "Client") + + +def test_shim_imports_iam_api_module_symbols(): + """OktaIAMApiModule layers on IAMApiModule, so its primitives must reach the shim.""" + for name in ("IAMUserProfile", "IAMActions", "IAMErrors"): + assert hasattr(integration_module, name), f"Symbol {name!r} missing from shim" + + +def test_main_delegates_to_api_module(mocker): + """`main()` must do nothing other than call the renamed ApiModule entry point.""" + mock_run = mocker.patch("OktaIAMStandardConnector.run_okta_iam_integration") + integration_module.main() + mock_run.assert_called_once_with() + + +def test_main_propagates_exceptions(mocker): + """`main()` is a thin shim and must not swallow exceptions from the ApiModule.""" + mocker.patch("OktaIAMStandardConnector.run_okta_iam_integration", side_effect=RuntimeError("boom")) + + with pytest.raises(RuntimeError, match="boom"): + integration_module.main() diff --git a/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/README.md b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/README.md new file mode 100644 index 000000000000..0c3fff292ba5 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaIAMStandardConnector/README.md @@ -0,0 +1,468 @@ +> Note: This integration should be used along with our ILM premium pack. For further details, visit our ILM pack documentation. + +Integrate with Okta's Identity Access Management service to execute CRUD operations to employee lifecycle processes. +This integration was integrated and tested with version v1 of the Okta integration. +For more information, refer to the [Identity Lifecycle Management article](https://xsoar.pan.dev/docs/reference/articles/identity-lifecycle-management). + +## Configure Okta IAM in Cortex + +| **Parameter** | **Description** | **Required** | +| --- | --- | --- | +| url | Okta URL \(https://<domain>.okta.com\) | True | +| apitoken | API Token \(see Detailed Instructions\) | True | +| insecure | Trust any certificate \(not secure\) | False | +| proxy | Use system proxy settings | False | +| create-user-enabled | Allow creating users | False | +| update-user-enabled | Allow updating users | False | +| enable-user-enabled | Allow enabling users | False | +| disable-user-enabled | Allow disabling users | False | +| create-if-not-exists | Automatically create user if not found in update command | False | +| mapper-in | Incoming Mapper | True | +| mapper-out | Outgoing Mapper | True | +| max_fetch | Fetch Limit \(recommended less than 200\) | False | +| isFetch | Fetch incidents | False | +| incidentFetchInterval | Incidents Fetch Interval | False | +| incidentType | Incident type | False | +| auto_generate_query_filter | Query only application events configured in the IAM Configuration | False | +| fetch_query_filter | Fetch Query Filter (Okta system log events) | True | +| first_fetch | First fetch timestamp (<number> <time unit>, e.g., 12 hours, 7 days) | False | + +* To allow the integration to access the mapper from within the code, as required by the ILM pack, both mappers have to be configured in their proper respective fields and not in the "Mapper (outgoing)" dropdown list selector. + +## Fetch incidents using an "IAM - Configuration" incident + +When the "Query only application events configured in IAM Configuration" checkbox is selected, add or remove event types for the applications you configured in the **IAM Configuration** incident are retrieved. You must have at least one application configured in XSOAR to fetch incidents from Okta. + +## Fetch incidents using a manual query filter expression + +**Note: Cortex XSOAR recommends you use the Query only application events configured in IAM Configuration option to generate the fetch-incidents query filter. The following following method should be used primarily for debugging purposes.** +Clear the "Query only application events configured in IAM Configuration" checkbox to use a custom fetch query filter expression. The expression must be in SCIM syntax, and include the add and remove event types, as well as the application ID. +For example: `(eventType eq "application.user_membership.add" or eventType eq "application.user_membership.remove") and target.id eq "0oar418fvkm67MWGd0h7"` +You may also use the advanced search in Okta's System Logs to generate the filter expression. +For more details, visit [Okta API reference](https://developer.okta.com/docs/reference/api/system-log/#expression-filter). + +## Commands + +You can execute these commands from the CLI, as part of an automation, or in a playbook. +After you successfully execute a command, a DBot message appears in the War Room with the command details. + +### iam-create-user + +*** +Creates a user. + +#### Base Command + +`iam-create-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| user-profile | User Profile indicator details. | Required | +| allow-enable | When set to true, after the command execution the status of the user in the 3rd-party integration will be active. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| IAM.Vendor.active | Boolean | When true, indicates that the employee's status is active in the 3rd-party integration. | +| IAM.Vendor.brand | String | Name of the integration. | +| IAM.Vendor.details | string | Provides the raw data from the 3rd-party integration. | +| IAM.Vendor.email | String | The employee's email address. | +| IAM.Vendor.errorCode | Number | HTTP error response code. | +| IAM.Vendor.errorMessage | String | Reason why the API failed. | +| IAM.Vendor.id | String | The employee's user ID in the app. | +| IAM.Vendor.instanceName | string | Name of the integration instance. | +| IAM.Vendor.success | Boolean | When true, indicates that the command was executed successfully. | +| IAM.Vendor.username | String | The employee's username in the app. | + +#### Command Example + +```!iam-create-user user-profile={\"email\":\"testdemisto2@paloaltonetworks.com\", \"surname\":\"Test\",\"givenname\":\"Demisto\"}``` + +#### Human Readable Output + +##### Create User Results (Okta IAM) + +|brand|instanceName|success|active|id|username|email|details| +|---|---|---|---|---|---|---|---| +| Okta IAM | Okta IAM_instance_1 | true | true | 00uujxnbh3uJw4tWA0h7 | testdemisto2@paloaltonetworks.com | testdemisto2@paloaltonetworks.com | id: 00uujxnbh3uJw4tWA0h7
status: PROVISIONED
created: 2020-10-18T17:54:30.000Z
activated: 2020-10-18T17:54:30.000Z
statusChanged: 2020-10-18T17:54:30.000Z
lastLogin: null
lastUpdated: 2020-10-18T17:54:30.000Z
passwordChanged: null
type: {"id": "oty8zfz6plq7b0r830h7"}
profile: {"firstName": "Demisto", "lastName": "Test", "mobilePhone": null, "secondEmail": null, "login": "testdemisto2@paloaltonetworks.com", "email": "testdemisto44@paloaltonetworks.com"}
credentials: {"provider": {"type": "OKTA", "name": "OKTA"}}
_links: {"suspend": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/suspend", "method": "POST"}, "schema": {"href": "https://panw-test.oktapreview.com/api/v1/meta/schemas/user/osc8zfz6plq7b0r830h7"}, "resetPassword": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/reset_password", "method": "POST"}, "reactivate": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/reactivate", "method": "POST"}, "self": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7"}, "type": {"href": "https://panw-test.oktapreview.com/api/v1/meta/types/user/oty8zfz6plq7b0r830h7"}, "deactivate": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/deactivate", "method": "POST"}} | + +### iam-update-user + +*** +Updates an existing user with the data passed in the user-profile argument. + +#### Base Command + +`iam-update-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| user-profile | A User Profile indicator. | Required | +| allow-enable | When set to true, after the command execution the status of the user in the 3rd-party integration will be active. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| IAM.Vendor.active | Boolean | When true, indicates that the employee's status is active in the 3rd-party integration. | +| IAM.Vendor.brand | String | Name of the integration. | +| IAM.Vendor.details | string | Provides the raw data from the 3rd-party integration. | +| IAM.Vendor.email | String | The employee's email address. | +| IAM.Vendor.errorCode | Number | HTTP error response code. | +| IAM.Vendor.errorMessage | String | Reason why the API failed. | +| IAM.Vendor.id | String | The employee's user ID in the app. | +| IAM.Vendor.instanceName | string | Name of the integration instance. | +| IAM.Vendor.success | Boolean | When true, indicates that the command was executed successfully. | +| IAM.Vendor.username | String | The employee's username in the app. | + +#### Command Example + +```!iam-update-user user-profile={\"email\":\"testdemisto2@paloaltonetworks.com\", \"givenname\":\"Demisto-Test\"}``` + +#### Human Readable Output + +##### Update User Results (Okta IAM) + +|brand|instanceName|success|active|id|username|email|details| +|---|---|---|---|---|---|---|---| +| Okta IAM | Okta IAM_instance_1 | true | true | 00uujxnbh3uJw4tWA0h7 | testdemisto2@paloaltonetworks.com | testdemisto2@paloaltonetworks.com | id: 00uujxnbh3uJw4tWA0h7
status: PROVISIONED
created: 2020-10-18T17:54:30.000Z
activated: 2020-10-18T17:54:30.000Z
statusChanged: 2020-10-18T17:54:30.000Z
lastLogin: null
lastUpdated: 2020-10-18T17:56:53.000Z
passwordChanged: null
type: {"id": "oty8zfz6plq7b0r830h7"}
profile: {"firstName": "Demisto-Test", "lastName": "Test", "mobilePhone": null, "secondEmail": null, "login": "testdemisto2@paloaltonetworks.com", "email": "testdemisto2@paloaltonetworks.com"}
credentials: {"provider": {"type": "OKTA", "name": "OKTA"}}
_links: {"suspend": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/suspend", "method": "POST"}, "schema": {"href": "https://panw-test.oktapreview.com/api/v1/meta/schemas/user/osc8zfz6plq7b0r830h7"}, "resetPassword": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/reset_password", "method": "POST"}, "reactivate": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/reactivate", "method": "POST"}, "self": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7"}, "type": {"href": "https://panw-test.oktapreview.com/api/v1/meta/types/user/oty8zfz6plq7b0r830h7"}, "deactivate": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/deactivate", "method": "POST"}} | + +### iam-get-user + +*** +Retrieves a single user resource. + +#### Base Command + +`iam-get-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| user-profile | A User Profile indicator. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| IAM.Vendor.active | Boolean | When true, indicates that the employee's status is active in the 3rd-party integration. | +| IAM.Vendor.brand | String | Name of the integration. | +| IAM.Vendor.details | string | Provides the raw data from the 3rd-party integration. | +| IAM.Vendor.email | String | The employee's email address. | +| IAM.Vendor.errorCode | Number | HTTP error response code. | +| IAM.Vendor.errorMessage | String | Reason why the API failed. | +| IAM.Vendor.id | String | The employee's user ID in the app. | +| IAM.Vendor.instanceName | string | Name of the integration instance. | +| IAM.Vendor.success | Boolean | When true, indicates that the command was executed successfully. | +| IAM.Vendor.username | String | The employee's username in the app. | + +#### Command Example + +```!iam-get-user user-profile={\"email\":\"testdemisto2@paloaltonetworks.com\"}``` + +#### Human Readable Output + +##### Get User Results (Okta IAM) + +|brand|instanceName|success|active|id|username|email|details| +|---|---|---|---|---|---|---|---| +| Okta IAM | Okta IAM_instance_1 | true | true | 00uujxnbh3uJw4tWA0h7 | testdemisto2@paloaltonetworks.com | testdemisto2@paloaltonetworks.com | id: 00uujxnbh3uJw4tWA0h7
status: PROVISIONED
created: 2020-10-18T17:54:30.000Z
activated: 2020-10-18T17:54:30.000Z
statusChanged: 2020-10-18T17:54:30.000Z
lastLogin: null
lastUpdated: 2020-10-18T17:56:53.000Z
passwordChanged: null
type: {"id": "oty8zfz6plq7b0r830h7"}
profile: {"firstName": "Demisto-Test", "lastName": "Test", "mobilePhone": null, "secondEmail": null, "login": "testdemisto2@paloaltonetworks.com", "email": "testdemisto2@paloaltonetworks.com"}
credentials: {"provider": {"type": "OKTA", "name": "OKTA"}}
_links: {"suspend": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/suspend", "method": "POST"}, "schema": {"href": "https://panw-test.oktapreview.com/api/v1/meta/schemas/user/osc8zfz6plq7b0r830h7"}, "resetPassword": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/reset_password", "method": "POST"}, "reactivate": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/reactivate", "method": "POST"}, "self": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7"}, "type": {"href": "https://panw-test.oktapreview.com/api/v1/meta/types/user/oty8zfz6plq7b0r830h7"}, "deactivate": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7/lifecycle/deactivate", "method": "POST"}} | + +### iam-disable-user + +*** +Disable an active user. + +#### Base Command + +`iam-disable-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| user-profile | A User Profile indicator. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| IAM.Vendor.active | Boolean | When true, indicates that the employee's status is active in the 3rd-party integration. | +| IAM.Vendor.brand | String | Name of the integration. | +| IAM.Vendor.details | string | Provides the raw data from the 3rd-party integration. | +| IAM.Vendor.email | String | The employee's email address. | +| IAM.Vendor.errorCode | Number | HTTP error response code. | +| IAM.Vendor.errorMessage | String | Reason why the API failed. | +| IAM.Vendor.id | String | The employee's user ID in the app. | +| IAM.Vendor.instanceName | string | Name of the integration instance. | +| IAM.Vendor.success | Boolean | When true, indicates that the command was executed successfully. | +| IAM.Vendor.username | String | The employee's username in the app. | + +#### Command Example + +```!iam-disable-user user-profile={\"email\":\"testdemisto2@paloaltonetworks.com\"}``` + +#### Human Readable Output + +##### Disable User Results (Okta IAM) + +|brand|instanceName|success|active|id|username|email|details| +|---|---|---|---|---|---|---|---| +| Okta IAM | Okta IAM_instance_1 | true | false | 00uujxnbh3uJw4tWA0h7 | testdemisto2@paloaltonetworks.com | testdemisto2@paloaltonetworks.com | id: 00uujxnbh3uJw4tWA0h7
status: PROVISIONED
created: 2020-10-18T17:54:30.000Z
activated: 2020-10-18T17:54:30.000Z
statusChanged: 2020-10-18T17:54:30.000Z
lastLogin: null
lastUpdated: 2020-10-18T17:56:53.000Z
passwordChanged: null
type: {"id": "oty8zfz6plq7b0r830h7"}
profile: {"firstName": "Demisto-Test", "lastName": "Test", "mobilePhone": null, "secondEmail": null, "login": "testdemisto2@paloaltonetworks.com", "email": "testdemisto2@paloaltonetworks.com"}
credentials: {"provider": {"type": "OKTA", "name": "OKTA"}}
_links: {"self": {"href": "https://panw-test.oktapreview.com/api/v1/users/00uujxnbh3uJw4tWA0h7"}} | + +### okta-get-assigned-user-for-app + +*** +Gets a specific user assignment for an application by id. + +#### Base Command + +`okta-get-app-user-assignment` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| user_id | ID of the user for which to get information. | Required | +| application_id | ID of the application for which to get information. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.AppUserAssignment.UserID | string | ID of the user. | +| Okta.AppUserAssignment.AppID | string | ID of the application. | +| Okta.AppUserAssignment.IsAssigned | boolean | When True, indicates that the user is assigned to the application. | +| Okta.AppUserAssignment.ProfileInApp | unknown | The user profile data in the application. | + +#### Command Example + +```!okta-get-app-user-assignment user_id=00uuv6y8t1iy8YXm94h7 application_id=0oae3ioe51sQ64Aui2h7``` + +#### Human Readable Output + +##### App User Assignment + +|App ID|Is Assigned|User ID| +|---|---|---| +| 0oae3ioe51sQ64Aui2h7 | true | 00uuv6y8t1iy8YXm94h7 | + +### okta-list-applications + +*** +Returns a list of Okta applications data. + +#### Base Command + +`okta-iam-list-applications` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| query | Search for applications by their names. | Optional | +| page | Page number (0-based). Default is 0. | Optional | +| limit | Maximum number of apps to retrieve (maximal value is 200). Default is 50. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Application.ID | string | ID of the application. | +| Okta.Application.Name | string | Name of the application. | +| Okta.Application.Label | string | Label of the application. | +| Okta.Application.Logo | string | Logo of the application. | + +#### Command Example + +``` !okta-iam-list-applications limit=5 query="Workday" ``` + +#### Human Readable Output + +##### Okta Applications (1 - 3) + +|ID|Name|Label|Logo| +|---|---|---|---| +| 0ob8zlypk6GVPRr2T0h7 | workday | Workday - Preview | ![](../../doc_files/gfsnda403rf16Qe790h7) | +| 0oabz0ozy5dDpEKyA0h7 | workday | Workday - Prod - DryRun | ![](../../doc_files/gfsnda403rf16Qe790h7) | +| 0oae3ioe51sQ64Aui2h7 | workday | Workday - Impl1 | ![](../../doc_files/gfsnda403rf16Qe790h7) | + +### okta-list-user-applications + +*** +Returns a list of Okta applications data. + +#### Base Command + +`okta-iam-list-user-applications` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| user_id | ID of the user for which to get the information. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Application.ID | string | ID of the application. | +| Okta.Application.Name | string | Name of the application. | +| Okta.Application.Label | string | Label of the application. | +| Okta.Application.Status | string | Status of the application. | + +#### Command Example + +``` !okta-iam-list-user-applications user_id=00ux9v19bvTfQIjur0h7" ``` + +#### Human Readable Output + +##### Okta User Applications + +|ID|Name|Label|Status| +|---|---|---|---| +| 0ob8zlypk6GVPRr2T0h7 | active_directory | pantest.local | ACTIVE| +| 0oabz0ozy5dDpEKyA0h7 | test_app | martsheet Test App | ACTIVE | + +### okta-iam-get-configuration + +*** +Gets the IAM configuration data from the integration context. + +#### Base Command + +`okta-iam-get-configuration` + +#### Input + +There are no input arguments for this command. + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.IAMConfiguration.ApplicationID | String | An Okta application ID. | +| Okta.IAMConfiguration.Name | String | Name of the Okta application. | +| Okta.IAMConfiguration.Label | String | Label of the Okta application. | +| Okta.IAMConfiguration.Logo | String | Logo of the application. | +| Okta.IAMConfiguration.Instance | String | An XSOAR IAM integration instance name. | + +#### Command Example + +```!okta-iam-get-configuration using="Okta IAM_instance_1_copy"``` + +#### Human Readable Output + +##### Okta IAM Configuration + +|ApplicationID|Instance|Label|Logo|Name| +|---|---|---|---|---| +| 0oc8zlypk6GVPRr2G0h7 | ServiceNow IAM_instance_1 | ServiceNow | ![](../../doc_files/gfskliw1i51ScX6pf0h7) | servicenow | + +### okta-iam-set-configuration + +*** +Updates IAM configuration data in the integration context. + +#### Base Command + +`okta-iam-set-configuration` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| configuration | IAM configuration data. | Required | + +#### Context Output + +There is no context output for this command. + +### iam-get-group + +*** +Retrieves the group information, including its members. + +#### Base Command + +`iam-get-group` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| scim | Group SCIM Data. | Required | +| includeMembers | Field to indicate if members need to be included in the response. . Possible values are: true, false. Default is true. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| GetGroup.id | String | ID of the group. | +| GetGroup.displayName | String | The display name of the group. | +| GetGroup.members.display | String | The display name of the group member. | +| GetGroup.members.value | String | ID of the group member. | +| GetGroup.success | Boolean | Indicates whether the command succeeded. | +| GetGroup.errorCode | Number | HTTP error response code. | +| GetGroup.errorMessage | String | Reason why the API failed. | + +### okta-get-logs + +*** +Gets logs by providing optional filters. + +#### Base Command + +`okta-get-logs` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| filter | Useful for performing structured queries where constraints on LogEvent attribute values can be explicitly targeted. The following expressions are supported for events with the filter query parameter: eventType eq " :eventType" -Events that have a specific action; eventType target.id eq ":id" - Events published with a specific target id; actor.id eq ":id"- Events published with a specific actor ID. For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering. | Optional | +| since | Filters the lower time bound of the log events in the Internet Date/Time Format profile of ISO 8601. For example: 2017-05-03T16:22:18Z. | Optional | +| until | Filters the upper time bound of the log events in the Internet Date/Time Format profile of ISO 8601. For example: 2017-05-03T16:22:18Z. | Optional | +| sortOrder | The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". Possible values are: ASCENDING, DESCENDING. Default is ASCENDING. | Optional | +| limit | The maximum number of results to return. The default and maximum is 100. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Logs.Events.actor.alternateId | String | Alternative ID of the actor. | +| Okta.Logs.Events.actor.displayName | String | Display name of the actor. | +| Okta.Logs.Events.actor.id | String | ID of the actor. | +| Okta.Logs.Events.client.userAgent.rawUserAgent | String | A raw string representation of user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. | +| Okta.Logs.Events.client.userAgent.os | String | The operating system on which the client runs. For example, Microsoft Windows 10. | +| Okta.Logs.Events.client.userAgent.browser | String | Identifies the type of web browser, if relevant. For example, Chrome. | +| Okta.Logs.Events.client.device | String | Type of device from which the client operated. For example, Computer. | +| Okta.Logs.Events.client.id | String | For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. | +| Okta.Logs.Events.client.ipAddress | String | IP address from which the client made its request. | +| Okta.Logs.Events.client.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.client.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.client.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.displayMessage | String | The display message for an event. | +| Okta.Logs.Events.eventType | String | Type of event that was published. | +| Okta.Logs.Events.outcome.result | String | Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". | +| Okta.Logs.Events.outcome.reason | String | Reason for the result. For example, INVALID_CREDENTIALS. | +| Okta.Logs.Events.published | String | Timestamp when the event was published. | +| Okta.Logs.Events.severity | String | The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". | +| Okta.Logs.Events.securityContext.asNumber | Number | Autonomous system number associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.asOrg | String | Organization associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.isp | String | Internet service provider used to send the event's request. | +| Okta.Logs.Events.securityContext.domain | String | Specifies whether an event's request is from a known proxy. | +| Okta.Logs.Events.request.ipChain.IP | String | IP address. | +| Okta.Logs.Events.request.ipChain.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.request.ipChain.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.request.ipChain.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.request.ipChain.source | String | Details regarding the source. | +| Okta.Logs.Events.target.id | String | ID of a target. | +| Okta.Logs.Events.target.type | String | Type of a target. | +| Okta.Logs.Events.target.alternateId | String | Alternative ID of a target. | +| Okta.Logs.Events.target.displayName | String | Display name of a target. | diff --git a/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector.py b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector.py new file mode 100644 index 000000000000..7885394c8e17 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector.py @@ -0,0 +1,14 @@ +import demistomock as demisto # noqa: F401 +from CommonServerPython import * # noqa: F401 + +from OktaApiModule import * # noqa: E402 + + +def main() -> None: + run_okta_v2_integration() + + +from CommonServerUserPython import * # noqa: E402 # pylint: disable=wrong-import-position + +if __name__ in ("__main__", "__builtin__", "builtins"): + main() diff --git a/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector.yml b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector.yml new file mode 100644 index 000000000000..2f2427e33e6e --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector.yml @@ -0,0 +1,1897 @@ +category: Authentication & Identity Management +provider: Okta +sectionorder: +- Connect +- Collect +commonfields: + id: OktaStandardConnector + version: -1 +configuration: +- display: Okta URL (https://.okta.com) + name: url + required: true + type: 0 + section: Connect +- name: credentials + type: 9 + displaypassword: API Token + hiddenusername: true + section: Connect + required: false + display: '' +- display: API Token + name: apitoken + type: 4 + hidden: true + section: Connect + required: false +- display: Use OAuth 2.0 Authentication + name: use_oauth + type: 8 + section: Connect + required: false + additionalinfo: See detailed instructions on the 'Help' tab. +- display: Client ID + name: client_id + type: 0 + section: Connect + required: false + additionalinfo: Required and used if OAuth 2.0 is used for authentication. See detailed instructions on the 'Help' tab. +- display: Private Key + additionalinfo: In PEM format. Required and used if OAuth 2.0 is used for authentication. See detailed instructions on the 'Help' tab. + name: private_key + type: 14 + section: Connect + required: false +- display: JWT Signing Algorithm + additionalinfo: Algorithm to sign generated JWT tokens with. Doesn't affect integration's functionality. Required and used if OAuth 2.0 is used for authentication. See detailed instructions on the 'Help' tab. + name: jwt_algorithm + type: 15 + section: Connect + required: false + options: + - RS256 + - RS384 + - RS512 + - ES256 + - ES384 + - ES512 + defaultvalue: RS256 +- display: Key ID + name: key_id + type: 0 + section: Connect + required: false + additionalinfo: Required and used if more than one key is used for signing JWT tokens. +- display: Trust any certificate (not secure) + name: insecure + type: 8 + section: Connect + advanced: true + required: false +- display: Use system proxy settings + name: proxy + type: 8 + section: Connect + advanced: true + required: false +description: 'This integration is configured automatically as part of the Okta Standard Connector. Do not configure this integration directly — set it up from the connector page instead.' +display: Okta (Standard Connector) +name: OktaStandardConnector +script: + commands: + - arguments: + - default: true + description: Username to unlock. + name: username + required: true + description: Unlocks a single user. + execution: true + name: okta-unlock-user + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Username to deactivate. + name: username + required: true + description: Deactivates a single user. + name: okta-deactivate-user + compliantpolicies: + - User Hard Remediation + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - default: true + description: Username to activate. + name: username + required: true + description: Activates a single user. + name: okta-activate-user + compliantpolicies: + - User Soft Remediation + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - default: true + description: Username to suspend. + name: username + required: true + description: Suspends a single user. This operation can only be performed on users with an ACTIVE status. After the porcess is completed, the user's status is SUSPENDED. + name: okta-suspend-user + compliantpolicies: + - User Hard Remediation + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - default: true + description: Username to change the status to ACTIVE. + name: username + required: true + description: Returns a single user to ACTIVE status. This operation can only be performed on users that have a SUSPENDED status. + name: okta-unsuspend-user + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Username for which to return all enrolled factors. + name: username + - description: User ID of the user for which to get all enrolled factors. + name: userId + description: Returns all the enrolled factors for the specified user. + name: okta-get-user-factors + outputs: + - contextPath: Account.ID + description: Okta account ID. + type: String + - contextPath: Account.Factor.ID + description: Okta account factor ID. + type: String + - contextPath: Account.Factor.Provider + description: Okta account factor provider. + type: String + - contextPath: Account.Factor.Profile + description: Okta account factor profile. + type: String + - contextPath: Account.Factor.FactorType + description: Okta account factor type. + type: String + - contextPath: Account.Factor.Status + description: Okta account factor status. + type: Unknown + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: The user ID. + name: userId + - description: Username for which to un-enroll an existing factor. + name: username + - description: The ID of the factor to reset. + name: factorId + required: true + description: Un-enrolls an existing factor for the specified user. This enables the user to enroll a new factor. + name: okta-reset-factor + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - default: true + description: Okta username for which to set the password. + name: username + required: true + - description: The new password to set for the user. + name: password + required: true + - auto: PREDEFINED + defaultValue: 'false' + description: When true, you'll need to change the password in the next login. + name: temporary_password + predefined: + - 'true' + - 'false' + description: Sets passwords without validating existing user credentials. + execution: true + name: okta-set-password + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: ID of the user to add to the group. + name: userId + - description: Name of the user to add to the group. + name: username + - description: ID of the group to add the user to. + name: groupId + - description: Name of the group to add the user to. + name: groupName + description: Adds a user to a group with OKTA_GROUP type. + name: okta-add-to-group + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: ID of the user to remove from the group. + name: userId + - description: Name of the user to remove from the group. + name: username + - description: ID of the group to remove the user from. + name: groupId + - description: Name of the group to remove the user from. + name: groupName + description: Removes a user from a group with OKTA_GROUP type. + name: okta-remove-from-group + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - default: true + description: Username in Okta for which to get the associated groups. + name: username + required: true + description: Returns all user groups associated with a specified user. + name: okta-get-groups + outputs: + - contextPath: Account.Group + description: Okta groups with which the account is associated. + type: Unknown + - contextPath: Account.ID + description: Okta account ID. + type: String + - contextPath: Account.Type + description: Okta account type. + type: String + - contextPath: Account.Group.ID + description: Unique key for the group. + type: String + - contextPath: Account.Group.Created + description: Timestamp when the group was created. + type: Date + - contextPath: Account.Group.ObjectClass + description: The object class, which determines the group's profile. + type: String + - contextPath: Account.Group.LastUpdated + description: Timestamp when the group's profile was last updated. + type: Date + - contextPath: Account.Group.LastMembershipUpdated + description: Timestamp when the group's memberships were last updated. + type: Date + - contextPath: Account.Group.Type + description: Group type, which determines how a group's profile and memberships are managed. + type: String + - contextPath: Account.Group.Description + description: Description of the group. + type: String + - contextPath: Account.Group.Name + description: Name of the group. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: The ID of the user to enroll and verify. + name: userId + required: true + - description: The push factor ID. + name: factorId + required: true + - description: Time to wait between subsequent polling calls. Value is in seconds. + name: polling_time + defaultValue: 5 + - description: Maximum number of polling calls. + name: max_polling_calls + defaultValue: 10 + - auto: PREDEFINED + description: Specifies whether the command should poll the status of the push factor. + name: polling + defaultValue: 'true' + predefined: + - 'true' + - 'false' + description: Enrolls and verifies a push factor for the specified user. + name: okta-verify-push-factor + outputs: + - contextPath: Account.ID + description: Okta user ID. + type: String + - contextPath: Account.VerifyPushResult + description: Okta user push factor result. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - contextPath: Okta.PollingStatusURL + description: The polling URL for the push factor challenge. + type: String + compliantpolicies: + - User Soft Remediation + - arguments: + - description: The url to poll. + name: polling_url + required: true + description: Checks the status of the push factor mfa. + name: okta-verify-mfa-status + outputs: + - contextPath: Okta.FactorResult.ID + description: The push factor challenge ID. + type: String + - contextPath: Okta.FactorResult.factorResult + description: The status of the push factor. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - default: true + description: Term by which to search. Can be a first name, last name, or email address. The argument `term` or `advanced_search` is required. + name: term + - description: Searches for users with a supported filtering expression for most properties, including custom-defined properties. The argument `term` or `advanced_search` is required. + name: advanced_search + - description: The maximum number of results to return. The default and maximum is 200. + name: limit + - auto: PREDEFINED + defaultValue: 'false' + description: Whether to return details of users that match the found term. Can be "true" or "false". The default is "false". + name: verbose + predefined: + - 'true' + - 'false' + description: Searches for Okta users. + name: okta-search + outputs: + - contextPath: Account.ID + description: Okta account IDs returned by the search. + type: String + - contextPath: Account.Username + description: Okta account usernames returned by the search. + type: String + - contextPath: Account.Email + description: Okta account emails returned by the search. + type: String + - contextPath: Account.DisplayName + description: Okta account display names returned by the search. + type: String + - contextPath: Account.Type + description: Okta account type returned by the search. + type: String + - contextPath: Account.Status + description: Okta account current status. + type: String + - contextPath: Account.Created + description: Timestamp for when the user was created. + type: Date + - contextPath: Account.Activated + description: Timestamp for when the user was activated. + type: Date + - contextPath: Account.StatusChanged + description: Timestamp for when the user's status was last changed. + type: Date + - contextPath: Account.PasswordChanged + description: Timestamp for when the user's password was last changed. + type: Date + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Okta username for which to get information. + name: username + - description: User ID of the user for which to get information. + name: userId + - description: Email of the user for which to get information. + name: userEmail + - auto: PREDEFINED + defaultValue: 'false' + description: Whether to return extended user information. Can be "true" or "false". The default is "false". + name: verbose + predefined: + - 'true' + - 'false' + description: Fetches information for a single user. You must enter one or more parameters for the command to run. + name: okta-get-user + outputs: + - contextPath: Account.ID + description: Okta account ID. + type: String + - contextPath: Account.Email + description: Okta account email. + type: String + - contextPath: Account.Username + description: Okta account username. + type: String + - contextPath: Account.DisplayName + description: Okta account display name. + type: String + - contextPath: Account.Status + description: Okta account status. + type: String + - contextPath: Account.Created + description: Timestamp for when the user was created. + type: Date + - contextPath: Account.Activated + description: Timestamp for when the user was activated. + type: Date + - contextPath: Account.StatusChanged + description: Timestamp for when the user's status was last changed. + type: Date + - contextPath: Account.PasswordChanged + description: Timestamp for when the user's password was last changed. + type: Date + - contextPath: Account.Manager + description: The manager. + type: String + - contextPath: Account.ManagerEmail + description: The manager email. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: "The cursor in which to retrive the results from and on. if the query didn't reach the end of results, the tag can be obtained from the bottom of the grid in the readable output, or in the context path Okta.User.tag." + name: after + - description: The maximum number of results to return. + name: limit + default: true + defaultValue: '200' + - auto: PREDEFINED + defaultValue: 'false' + description: Whether to return extended user information. + name: verbose + predefined: + - 'true' + - 'false' + - description: Searches the name property of groups for matching values. + name: query + - description: "Useful for performing structured queries where constraints on group attribute values can be explicitly targeted. \nThe following expressions are supported(among others) for groups with the filter query parameter: \n type eq \"OKTA_GROUP\" - Groups that have a type of OKTA_GROUP; lastUpdated lt \"yyyy-MM-dd''T''HH:mm:ss.SSSZ\" - Groups with profile last updated before a specific timestamp; lastMembershipUpdated eq \"yyyy-MM-dd''T''HH:mm:ss.SSSZ\" - Groups with memberships last updated at a specific timestamp; id eq \"00g1emaKYZTWRYYRRTSK\" - Group with a specified ID. For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering" + name: filter + description: Lists users in your organization. + name: okta-list-users + outputs: + - contextPath: Account.ID + description: Okta account ID. + type: String + - contextPath: Account.Email + description: Okta account email. + type: String + - contextPath: Account.Username + description: Okta account username. + type: String + - contextPath: Account.DisplayName + description: Okta account display name. + type: String + - contextPath: Account.Status + description: Okta account status. + type: String + - contextPath: Account.Created + description: Timestamp for when the user was created. + type: Date + - contextPath: Account.Activated + description: Timestamp for when the user was activated. + type: Date + - contextPath: Account.StatusChanged + description: Timestamp for when the user's status was last changed. + type: Date + - contextPath: Account.PasswordChanged + description: Timestamp for when the user's password was last changed. + type: Date + - contextPath: Okta.User.tag + description: The location of the next item, used with after param. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: First name of the user (givenName). + name: firstName + required: true + - description: Family name of the user (familyName). + name: lastName + required: true + - description: Primary email address of the user. + name: email + required: true + - description: Unique identifier for the user (username). + name: login + required: true + - description: Secondary email address of user. Usually used for account recovery. + name: secondEmail + - description: Middle name(s) of the user. + name: middleName + - description: A comma-separated list of honorific prefix(es) of the user, or title in most Western languages. + name: honorificPrefix + - description: A comma-separated list of honorific suffix(es) of the user. + name: honorificSuffix + - description: User's title. for example, Vice President. + name: title + - description: Display name of the user. + name: displayName + - description: Casual way to address the user (nick name). + name: nickName + - description: URL of the user online profile. For example, a web page. + name: profileUrl + - description: Primary phone number of the user. + name: primaryPhone + - description: Mobile phone number of the user. + name: mobilePhone + - description: Full street address component of the user's address. + name: streetAddress + - description: City or locality component of the user's address (locality). + name: city + - description: State or region component of the user's address (region). + name: state + - description: Zip code or postal code component of the user's address (postalCode). + name: zipCode + - description: Country name component of the user's address (country). + name: countryCode + - description: Mailing address component of the user's address. + name: postalAddress + - description: User's preferred written or spoken languages. + name: preferredLanguage + - description: User's default location, for purposes of localizing items such as currency, date-time format, numerical representations, etc. + name: locale + - description: User's time zone. + name: timezone + - description: The user type, which is used to identify the organization-to-user relationship such as "Employee" or "Contractor". + name: userType + - description: Organization or company assigned unique identifier for the user. + name: employeeNumber + - description: Name of a cost center the user is assigned to. + name: costCenter + - description: Name of the user's organization. + name: organization + - description: Name of the user's division. + name: division + - description: Name of the user's department. + name: department + - description: ID of the user's manager. + name: managerId + - description: Display name of the user's manager. + name: manager + - description: Password for the new user. + name: password + - description: Password question for the new user. + name: passwordQuestion + - description: Password answer for question. + name: passwordAnswer + - auto: PREDEFINED + description: The provider type. Can be "OKTA", "ACTIVE_DIRECTORY", "LDAP", "FEDERATION", or "SOCIAL". + name: providerType + predefined: + - OKTA + - ACTIVE_DIRECTORY + - LDAP + - FEDERATION + - SOCIAL + - description: Name of the provider. + name: providerName + - description: IDs of groups that the user will be immediately added to at time of creation (does Not include default group). + name: groupIds + - auto: PREDEFINED + description: Whether to activate the lifecycle operation when creating the user. Can be "true" or "false". + name: activate + predefined: + - 'true' + - 'false' + description: Creates a new user with an option of setting a password, recovery question, and answer. The new user will immediately be able to log in after activation with the assigned password. This flow is common when developing a custom user registration experience. + execution: true + name: okta-create-user + outputs: + - contextPath: Account.ID + description: Created Okta account ID. + type: String + - contextPath: Account.Email + description: Created Okta account email address. + type: String + - contextPath: Account.Username + description: Created okta account username. + type: String + - contextPath: Account.DisplayName + description: Created Okta account display name. + type: String + - contextPath: Account.Type + description: Type of created account - Okta. + type: String + - contextPath: Account.Status + description: Okta account current status. + type: String + - contextPath: Account.Created + description: Timestamp for when the user was created. + type: Date + - contextPath: Account.Activated + description: Timestamp for when the user was activated. + type: Date + - contextPath: Account.StatusChanged + description: Timestamp for when the user's status was last changed. + type: Date + - contextPath: Account.PasswordChanged + description: Timestamp for when the user's password was last changed. + type: Date + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: First name of the user (given name). + name: firstName + - description: Family name of the user. + name: lastName + - description: Primary email address of the user. + name: email + - description: Unique identifier for the user (login). + name: username + required: true + - description: Secondary email address of the user (typically used for account recovery. + name: secondEmail + - description: Middle name(s) of the user. + name: middleName + - description: Honorific prefix(es) of the user, or title in most Western languages. + name: honorificPrefix + - description: Honorific suffix(es) of the user. + name: honorificSuffix + - description: User's title. For example, Vice President. + name: title + - description: Display name of the user. + name: displayName + - description: Casual way to address the user in real life (nick name). + name: nickName + - description: URL of the user's online profile. For example, a web page. + name: profileUrl + - description: Primary phone number of the user. + name: primaryPhone + - description: Mobile phone number of the user. + name: mobilePhone + - description: Full street address component of the user's address. + name: streetAddress + - description: City or locality component of the user's address (locality). + name: city + - description: State or region component of the user's address (region). + name: state + - description: Zip code or postal code component of the user's address (postalCode). + name: zipCode + - description: Country name component of the user's address (country). + name: countryCode + - description: Mailing address component of the user's address. + name: postalAddress + - description: User's preferred written or spoken languages. + name: preferredLanguage + - description: User's default location for purposes of localizing items such as currency, date-time format, numerical representations, etc. + name: locale + - description: User time zone. + name: timezone + - description: The user type, which is used to identify the organization-to-user relationship such as "Employee" or "Contractor". + name: userType + - description: Organization or company assigned unique identifier for the user. + name: employeeNumber + - description: Name of a cost center the user is assigned to. + name: costCenter + - description: Name of the user's organization. + name: organization + - description: Name of the user's division. + name: division + - description: Name of the user's department. + name: department + - description: ID of the user's manager. + name: managerId + - description: Display name of the user's manager. + name: manager + - description: New password for the specified user. + name: password + - description: Password question for the specified user. + name: passwordQuestion + - description: Password answer for the question. + name: passwordAnswer + - auto: PREDEFINED + description: The provider type. Can be "OKTA", "ACTIVE_DIRECTORY", "LDAP", "FEDERATION", or "SOCIAL". + name: providerType + predefined: + - OKTA + - ACTIVE_DIRECTORY + - FEDERATION + - SOCIAL + - description: Name of the provider. + name: providerName + description: Updates a user with a given login. All fields are optional. Fields which are not set, will not be overwritten. + execution: true + name: okta-update-user + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: ID of the group. + name: groupId + - description: The maximum number of results to return. + name: limit + - auto: PREDEFINED + defaultValue: 'false' + description: Whether to print extended user details. Can be "true" or "false". The default is "false". + name: verbose + predefined: + - 'true' + - 'false' + - description: Name of the group. + name: groupName + description: Enumerates all users that are members of a group. + name: okta-get-group-members + outputs: + - contextPath: Account.ID + description: Okta account ID. + type: String + - contextPath: Account.Email + description: Okta account email address. + type: String + - contextPath: Account.Username + description: Okta account username. + type: String + - contextPath: Account.DisplayName + description: Okta account display name. + type: String + - contextPath: Account.Type + description: Account type - Okta. + type: String + - contextPath: Account.Status + description: Okta account current status. + type: String + - contextPath: Account.Created + description: Timestamp for when the user was created. + type: Date + - contextPath: Account.Activated + description: Timestamp for when the user was activated. + type: Date + - contextPath: Account.StatusChanged + description: Timestamp for when the user's status was last changed. + type: Date + - contextPath: Account.PasswordChanged + description: Timestamp for when the user's password was last changed. + type: Date + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Searches the name property of groups for matching values. + name: query + - description: "Useful for performing structured queries where constraints on group attribute values can be explicitly targeted. \nThe following expressions are supported(among others) for groups with the filter query parameter: \n type eq \"OKTA_GROUP\" - Groups that have a type of OKTA_GROUP; lastUpdated lt \"yyyy-MM-dd''T''HH:mm:ss.SSSZ\" - Groups with profile last updated before a specific timestamp; lastMembershipUpdated eq \"yyyy-MM-dd''T''HH:mm:ss.SSSZ\" - Groups with memberships last updated at a specific timestamp; id eq \"00g1emaKYZTWRYYRRTSK\" - Group with a specified ID. For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering" + name: filter + - defaultValue: '200' + description: The maximum number of results to return. The default is 200. + name: limit + description: Lists groups in your organization. A subset of groups can be returned that match a supported filter expression or query. + name: okta-list-groups + outputs: + - contextPath: Okta.Group.ID + description: Unique key for the group. + type: String + - contextPath: Okta.Group.Created + description: Timestamp for when the group was created. + type: Date + - contextPath: Okta.Group.ObjectClass + description: The group's profile. + type: Unknown + - contextPath: Okta.Group.LastUpdated + description: Timestamp for when the group's profile was last updated. + type: Date + - contextPath: Okta.Group.LastMembershipUpdated + description: Timestamp for when the group's membership was last updated. + type: Date + - contextPath: Okta.Group.Type + description: The group type, which determines how a group's profile and membership are managed. Can be "OKTA_GROUP", "APP_GROUP", or "BUILT_IN". + type: String + - contextPath: Okta.Group.Name + description: Name of the group. + type: String + - contextPath: Okta.Group.Description + description: Description of the group. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: 'Filters the lower time bound of the log events in the Internet Date/Time Format profile of ISO 8601. An example: 2017-05-03T16:22:18Z.' + name: since + - description: 'Filters the upper time bound of the log events in the Internet Date/Time Format profile of ISO 8601. An example: 2017-05-03T16:22:18Z.' + name: until + - auto: PREDEFINED + defaultValue: ASCENDING + description: The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". + name: sortOrder + predefined: + - ASCENDING + - DESCENDING + - description: The maximum number of results to return. The default is 100. + name: limit + description: Returns failed login events. + name: okta-get-failed-logins + outputs: + - contextPath: Okta.Logs.Events.actor.alternateId + description: Alternative ID of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.displayName + description: Display name of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.id + description: ID of the actor. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.rawUserAgent + description: A raw string representation of the user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.os + description: The operating system on which the client runs. For example, Microsoft Windows 10. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.browser + description: Identifies the browser type, if relevant. For example, Chrome. + type: String + - contextPath: Okta.Logs.Events.client.device + description: Type of device that client operated from. For example, Computer. + type: String + - contextPath: Okta.Logs.Events.client.id + description: For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. + type: String + - contextPath: Okta.Logs.Events.client.ipAddress + description: IP address from which the client made its request. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.displayMessage + description: The display message for an event. + type: String + - contextPath: Okta.Logs.Events.eventType + description: Type of event that was published. + type: String + - contextPath: Okta.Logs.Events.outcome.result + description: Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", "UNKNOWN". + type: String + - contextPath: Okta.Logs.Events.outcome.reason + description: Reason for the result. For example, INVALID_CREDENTIALS. + type: String + - contextPath: Okta.Logs.Events.published + description: Timestamp when the event was published. + type: String + - contextPath: Okta.Logs.Events.severity + description: The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". + type: String + - contextPath: Okta.Logs.Events.securityContext.asNumber + description: Autonomous system number associated with the autonomous system that the event request was sourced to. + type: Number + - contextPath: Okta.Logs.Events.securityContext.asOrg + description: Organization associated with the autonomous system that the event request was sourced to. + type: String + - contextPath: Okta.Logs.Events.securityContext.isp + description: Internet service provider used to send the event's request. + type: String + - contextPath: Okta.Logs.Events.securityContext.domain + description: The domain name associated with the IP address of the inbound event request. + type: String + - contextPath: Okta.Logs.Events.securityContext.isProxy + description: Specifies whether an event's request is from a known proxy. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.IP + description: IP address. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.source + description: Details regarding the source. + type: String + - contextPath: Okta.Logs.Events.target.id + description: ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.type + description: Type of a target. + type: String + - contextPath: Okta.Logs.Events.target.alternateId + description: Alternative ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.displayName + description: Display name of a target. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: "Useful for performing structured queries where constraints on LogEvent attribute values can be explicitly targeted. \nThe following expressions are supported for events with the filter query parameter: eventType eq \" :eventType\" \n-Events that have a specific action; eventType target.id eq \":id\" \n- Events published with a specific target id; actor.id eq \":id\"\n- Events published with a specific actor ID. For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering" + name: filter + - description: "The query parameter can be used to perform keyword matching against a LogEvents object’s attribute values. To satisfy the constraint, all supplied keywords must be matched exactly. Note that matching is case-insensitive. The following are some examples of common keyword filtering: \nEvents that mention a specific city: query=San Francisco; \nEvents that mention a specific url: query=interestingURI.com; \nEvents that mention a specific person: query=firstName lastName." + name: query + - description: 'Filters the lower time bound of the log events in the Internet Date/Time Format profile of ISO 8601. For example: 2017-05-03T16:22:18Z.' + name: since + - description: 'Filters the upper time bound of the log events in the Internet Date/Time Format profile of ISO 8601. For example: 2017-05-03T16:22:18Z.' + name: until + - auto: PREDEFINED + defaultValue: ASCENDING + description: The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". + name: sortOrder + predefined: + - ASCENDING + - DESCENDING + - defaultValue: '100' + description: The maximum number of results to return. The default is 100. + name: limit + description: Gets logs by providing optional filters. + name: okta-get-logs + outputs: + - contextPath: Okta.Logs.Events.actor.alternateId + description: Alternative ID of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.displayName + description: Display name of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.id + description: ID of the actor. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.rawUserAgent + description: A raw string representation of user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.os + description: The operating system on which the client runs. For example, Microsoft Windows 10. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.browser + description: Identifies the type of web browser, if relevant. For example, Chrome. + type: String + - contextPath: Okta.Logs.Events.client.device + description: Type of device from which the client operated. For example, Computer. + type: String + - contextPath: Okta.Logs.Events.client.id + description: For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. + type: String + - contextPath: Okta.Logs.Events.client.ipAddress + description: IP address from which the client made its request. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.displayMessage + description: The display message for an event. + type: String + - contextPath: Okta.Logs.Events.eventType + description: Type of event that was published. + type: String + - contextPath: Okta.Logs.Events.outcome.result + description: Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". + type: String + - contextPath: Okta.Logs.Events.outcome.reason + description: Reason for the result. For example, INVALID_CREDENTIALS. + type: String + - contextPath: Okta.Logs.Events.published + description: Timestamp when the event was published. + type: String + - contextPath: Okta.Logs.Events.severity + description: The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". + type: String + - contextPath: Okta.Logs.Events.securityContext.asNumber + description: Autonomous system number associated with the autonomous system that the event request was sourced to. + type: Number + - contextPath: Okta.Logs.Events.securityContext.asOrg + description: Organization associated with the autonomous system that the event request was sourced to. + type: String + - contextPath: Okta.Logs.Events.securityContext.isp + description: Internet service provider used to send the event's request. + type: String + - contextPath: Okta.Logs.Events.securityContext.domain + description: The domain name associated with the IP address of the inbound event request. + type: String + - contextPath: Okta.Logs.Events.securityContext.isProxy + description: Specifies whether an event's request is from a known proxy. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.IP + description: IP address. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.source + description: Details regarding the source. + type: String + - contextPath: Okta.Logs.Events.target.id + description: ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.type + description: Type of a target. + type: String + - contextPath: Okta.Logs.Events.target.alternateId + description: Alternative ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.displayName + description: Display name of a target. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Filters the lower time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. + name: since + - description: Filters the upper time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. + name: until + - auto: PREDEFINED + defaultValue: ASCENDING + description: The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". + name: sortOrder + predefined: + - ASCENDING + - DESCENDING + - defaultValue: '100' + description: The maximum number of results to return. The default is 100. + name: limit + description: Gets events for when a user was added to a group. + name: okta-get-group-assignments + outputs: + - contextPath: Okta.Logs.Events.actor.alternateId + description: Alternative ID of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.displayName + description: Display name of the actor. + type: String + - contextPath: Okta.Logs.Event.actor.id + description: ID of the actor. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.rawUserAgent + description: A raw string representation of user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.os + description: The operating system on which the client runs. For example, Microsoft Windows 10. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.browser + description: Identifies the type of web browser, if relevant. For example, Chrome. + type: String + - contextPath: Okta.Logs.Events.client.device + description: Type of device from which the client operated. For example, Computer. + type: String + - contextPath: Okta.Logs.Events.client.id + description: For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. + type: String + - contextPath: Okta.Logs.Events.client.ipAddress + description: IP address from which the client made its request. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.state + description: Full name of the state or province encompassing in the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.displayMessage + description: The display message for an event. + type: String + - contextPath: Okta.Logs.Events.eventType + description: Type of event that was published. + type: String + - contextPath: Okta.Logs.Events.outcome.result + description: Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". + type: String + - contextPath: Okta.Logs.Events.outcome.reason + description: Reason for the result. For example INVALID_CREDENTIALS. + type: Unknown + - contextPath: Okta.Logs.Events.published + description: Timestamp when the event was published. + type: String + - contextPath: Okta.Logs.Events.severity + description: The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". + type: String + - contextPath: Okta.Logs.Events.securityContext.asNumber + description: Autonomous system number associated with the autonomous system that the event request was sourced to. + type: Number + - contextPath: Okta.Logs.Events.securityContext.asOrg + description: Organization associated with the autonomous system that the event request was sourced to. + type: String + - contextPath: Okta.Logs.Events.securityContext.isp + description: Internet service provider used to send the event's request. + type: String + - contextPath: Okta.Logs.Events.securityContext.domain + description: The domain name associated with the IP address of the inbound event request. + type: String + - contextPath: Okta.Logs.Events.securityContext.isProxy + description: Specifies whether an event's request is from a known proxy. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.IP + description: IP address. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.source + description: Details regarding the source. + type: String + - contextPath: Okta.Logs.Events.target.id + description: ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.type + description: Target type. + type: String + - contextPath: Okta.Logs.Events.target.alternateId + description: Alternative ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.displayName + description: Display name of a target. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Filters the lower time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. + name: since + - description: Filters the upper time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. + name: until + - auto: PREDEFINED + defaultValue: ASCENDING + description: The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". + name: sortOrder + predefined: + - ASCENDING + - DESCENDING + - defaultValue: '100' + description: The maximum number of results to return. The default is 100. + name: limit + description: Returns events for when a user was assigned to an application. + name: okta-get-application-assignments + outputs: + - contextPath: Okta.Logs.Events.actor.alternateId + description: Alternative ID of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.displayName + description: Display name of the actor. + type: String + - contextPath: Okta.Logs.Event.actor.id + description: ID of the actor. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.rawUserAgent + description: A raw string representation of the user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.os + description: The OS on which the client runs. For example, Microsoft Windows 10. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.browser + description: Identifies the type of web browser, if relevant. For example, Chrome. + type: String + - contextPath: Okta.Logs.Events.client.device + description: Type of device from which the client operated. For example, Computer. + type: String + - contextPath: Okta.Logs.Events.client.id + description: For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. + type: String + - contextPath: Okta.Logs.Events.client.ipAddress + description: IP address from which the client made its request. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.displayMessage + description: The display message for an event. + type: String + - contextPath: Okta.Logs.Events.eventType + description: Type of event that was published. + type: String + - contextPath: Okta.Logs.Events.outcome.result + description: Result of the action. For example, "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". + type: String + - contextPath: Okta.Logs.Events.outcome.reason + description: Reason for the result. For example INVALID_CREDENTIALS. + type: String + - contextPath: Okta.Logs.Events.published + description: Timestamp when the event was published. + type: String + - contextPath: Okta.Logs.Events.severity + description: The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". + type: String + - contextPath: Okta.Logs.Events.securityContext.asNumber + description: Autonomous system number associated with the autonomous system that the event request was sourced to. + type: Number + - contextPath: Okta.Logs.Events.securityContext.asOrg + description: Organization associated with the autonomous system that the event request was sourced to. + type: String + - contextPath: Okta.Logs.Events.securityContext.isp + description: Internet service provider used to send the event's request. + type: String + - contextPath: Okta.Logs.Events.securityContext.domain + description: The domain name associated with the IP address of the inbound event request. + type: String + - contextPath: Okta.Logs.Events.securityContext.isProxy + description: Specifies whether an event's request is from a known proxy. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.IP + description: IP address. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.source + description: Details regarding the source. + type: String + - contextPath: Okta.Logs.Events.target.id + description: ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.type + description: Type of a target. + type: String + - contextPath: Okta.Logs.Events.target.alternateId + description: Alternative ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.displayName + description: Display name of a target. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Filters the lower time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. + name: since + - description: Filters the upper time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. + name: until + - auto: PREDEFINED + defaultValue: ASCENDING + description: The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". + name: sortOrder + predefined: + - ASCENDING + - DESCENDING + - defaultValue: '100' + description: The maximum number of results to return. The default is 100. + name: limit + description: Returns logs using specified filters. + name: okta-get-application-authentication + outputs: + - contextPath: Okta.Logs.Events.actor.alternateId + description: Alternative ID of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.displayName + description: Display name of the actor. + type: String + - contextPath: Okta.Logs.Events.actor.id + description: ID of the actor. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.rawUserAgent + description: A raw string representation of user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.os + description: The operating system on which the client runs. For example, Microsoft Windows 10. + type: String + - contextPath: Okta.Logs.Events.client.userAgent.browser + description: Identifies the type of web browser, if relevant. For example, Chrome. + type: String + - contextPath: Okta.Logs.Events.client.device + description: Type of device from which the client operated. For example, Computer. + type: String + - contextPath: Okta.Logs.Events.client.id + description: For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. + type: String + - contextPath: Okta.Logs.Events.client.ipAddress + description: IP address from which the client made its request. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.client.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.displayMessage + description: The display message for an event. + type: String + - contextPath: Okta.Logs.Events.eventType + description: Type of event that was published. + type: String + - contextPath: Okta.Logs.Events.outcome.result + description: Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". + type: String + - contextPath: Okta.Logs.Events.outcome.reason + description: Reason for the result. For example INVALID_CREDENTIALS. + type: String + - contextPath: Okta.Logs.Events.published + description: Timestamp when the event was published. + type: String + - contextPath: Okta.Logs.Events.severity + description: The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". + type: String + - contextPath: Okta.Logs.Events.securityContext.asNumber + description: Autonomous system number associated with the autonomous system that the event request was sourced to. + type: Number + - contextPath: Okta.Logs.Events.securityContext.asOrg + description: Organization associated with the autonomous system that the event request was sourced to. + type: String + - contextPath: Okta.Logs.Events.securityContext.isp + description: Internet service provider used to send the event's request. + type: String + - contextPath: Okta.Logs.Events.securityContext.domain + description: The domain name associated with the IP address of the inbound event request. + type: String + - contextPath: Okta.Logs.Events.securityContext.isProxy + description: Specifies whether an event's request is from a known proxy. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.IP + description: IP address. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.city + description: The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.state + description: Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.geographicalContext.country + description: Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. + type: String + - contextPath: Okta.Logs.Events.request.ipChain.source + description: Details regarding the source. + type: String + - contextPath: Okta.Logs.Events.target.id + description: ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.type + description: Type of a target. + type: String + - contextPath: Okta.Logs.Events.target.alternateId + description: Alternative ID of a target. + type: String + - contextPath: Okta.Logs.Events.target.displayName + description: Display name of a target. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Okta User ID. + name: userId + - description: Username of the user. + name: username + description: Deletes the specified user. + execution: true + name: okta-delete-user + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + compliantpolicies: + - User Hard Remediation + - arguments: + - description: Okta User ID. + name: userId + required: true + - description: When true, revokes OpenID Connect and OAuth refresh and access tokens issued to the user. + name: revokeOauthTokens + auto: PREDEFINED + defaultValue: 'true' + predefined: + - 'true' + - 'false' + description: |- + Removes all active identity provider sessions. This forces the user to authenticate upon the next operation. By default, OpenID Connect and OAuth refresh and access tokens issued to the user are revoked. Token revocation can be disabled if needed. + For more information and examples: + https://developer.okta.com/docs/reference/api/users/#user-sessions + name: okta-clear-user-sessions + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + compliantpolicies: + - User Soft Remediation + - arguments: + - description: The maximum number of results to return. + name: limit + description: Get an Okta Zone object. + name: okta-list-zones + outputs: + - contextPath: Okta.Zone.created + description: Zone creation timestamp, in the format 2020-04-06T22:23:12.000Z. + type: Date + - contextPath: Okta.Zone.gateways.type + description: Gateways IP entry type, e.g., CIDR. + type: String + - contextPath: Okta.Zone.gateways.value + description: Gateways IP entry value, e.g., 34.103.1.108/32. + type: String + - contextPath: Okta.Zone.id + description: Zone ID, e.g., nzoqsmcx1qWYJ6wYF0h7. + type: String + - contextPath: Okta.Zone.lastUpdated + description: Zone last update timestamp, e.g., 2020-04-06T22:23:12.000Z. + type: Date + - contextPath: Okta.Zone.name + description: Zone name. + type: String + - contextPath: Okta.Zone.proxies.type + description: Proxies IP entry type e.g. CIDR. + type: String + - contextPath: Okta.Zone.proxies.value + description: Proxies IP entry value, e.g., 34.103.1.108/32. + type: Unknown + - contextPath: Okta.Zone.status + description: Zone status, e.g., ACTIVE. + type: String + - contextPath: Okta.Zone.system + description: True if this is a system zone, false if user-created. + type: Number + - contextPath: Okta.Zone.type + description: Zone type, e.g., IP. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Zone ID to update, e.g., nzoqsmcx1qWYJ6wYF0h7. + name: zoneID + required: true + - description: Updates the zone name. + name: zoneName + - description: 'Updates Gateway IP addresses: CIDR range (1.1.0.0/16), single IP address (2.2.2.2) or an IP range (1.2.3.4-1.2.3.5).' + isArray: true + name: gatewayIPs + - description: 'Update Proxy IP addresses: CIDR range (1.1.0.0/16), single IP address (2.2.2.2) or an IP range (1.2.3.4-1.2.3.5).' + isArray: true + name: proxyIPs + - description: 'Indicate the update type: APPEND - add an IP to an existing list of zone IPs. OVERRIDE: overriding the existing IP list with the given IP.' + isArray: false + name: updateType + auto: PREDEFINED + predefined: + - "APPEND" + - "OVERRIDE" + defaultValue: 'OVERRIDE' + description: Update an Okta Zone. + name: okta-update-zone + outputs: + - contextPath: Okta.Zone.created + description: Zone creation timestamp, e.g., 2020-04-06T22:23:12.000Z. + type: Date + - contextPath: Okta.Zone.gateways.type + description: Gateways IP entry type, e.g., CIDR. + type: String + - contextPath: Okta.Zone.gateways.value + description: Gateways IP entry value, e.g., 34.103.1.108/32. + type: String + - contextPath: Okta.Zone.id + description: Okta Zone ID, e.g., nzoqsmcx1qWYJ6wYF0h7. + type: String + - contextPath: Okta.Zone.lastUpdated + description: Zone last update timestamp, in the format 2020-04-06T22:23:12.000Z. + type: Date + - contextPath: Okta.Zone.name + description: Zone name. + type: String + - contextPath: Okta.Zone.proxies.type + description: Proxies IP entry type, e.g., CIDR. + type: String + - contextPath: Okta.Zone.proxies.value + description: Proxies IP entry value, e.g., 34.103.1.108/32. + type: Unknown + - contextPath: Okta.Zone.status + description: Zone status, e.g., ACTIVE. + type: String + - contextPath: Okta.Zone.system + description: True if this is a system zone, false if user-created. + type: Number + - contextPath: Okta.Zone.type + description: Zone type, e.g., IP. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Zone ID to get, e.g., nzoqsmcx1qWYJ6wYF0h.7. + name: zoneID + required: true + description: Get a Zone by its ID. + name: okta-get-zone + outputs: + - contextPath: Okta.Zone.created + description: Zone creation timestamp, in the format 2020-04-06T22:23:12.000Z. + type: Date + - contextPath: Okta.Zone.gateways.type + description: Gateways IP entry type, e.g., CIDR. + type: String + - contextPath: Okta.Zone.gateways.value + description: Gateways IP entry value, e.g., 34.103.1.108/32. + type: String + - contextPath: Okta.Zone.id + description: Okta Zone ID, e.g., nzoqsmcx1qWYJ6wYF0h7. + type: String + - contextPath: Okta.Zone.lastUpdated + description: Zone last update timestamp, in the format 2020-04-06T22:23:12.000Z. + type: Date + - contextPath: Okta.Zone.name + description: Zone name. + type: String + - contextPath: Okta.Zone.proxies.type + description: Proxies IP entry type, e.g., CIDR. + type: String + - contextPath: Okta.Zone.proxies.value + description: Proxies IP entry value, e.g., 34.103.1.108/32. + type: Unknown + - contextPath: Okta.Zone.status + description: Zone status, e.g,. ACTIVE. + type: String + - contextPath: Okta.Zone.system + description: True if this is a system zone, false if user-created. + type: Number + - contextPath: Okta.Zone.type + description: Zone type, e.g., IP. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Zone name. + name: name + required: true + - description: 'Update Gateway IP addresses: CIDR range (1.1.0.0/16), single IP address (2.2.2.2) or an IP range (1.2.3.4-1.2.3.5).' + name: gateway_ips + - description: 'Update Proxy IP addresses: CIDR range (1.1.0.0/16), single IP address (2.2.2.2) or an IP range (1.2.3.4-1.2.3.5).' + name: proxies + description: Creates a Zone with the specified name. + name: okta-create-zone + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Name of the group to add. + name: name + required: true + - description: Description of the group to add. + name: description + description: Create a new group in Okta tenant. + execution: true + name: okta-create-group + outputs: + - contextPath: OktaGroup.ID + description: Group ID in Okta. + type: Unknown + - contextPath: OktaGroup.Name + description: Group name in Okta. + type: Unknown + - contextPath: OktaGroup.Description + description: Group description in Okta. + type: Unknown + - contextPath: OktaGroup.Type + description: Group type in Okta. + type: Unknown + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - description: Name of the group to assign to the app. + name: groupName + - description: ID of the group to assign to the app. + name: groupId + - description: Friendly name of the app that the group will be assigned to. + name: appName + description: Assign a group to an application. + name: okta-assign-group-to-app + outputs: + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + - arguments: + - default: true + description: Okta username for which to expire the password. + name: username + required: true + - auto: PREDEFINED + defaultValue: 'false' + description: When true, you'll need to change the password in the next login. + name: temporary_password + predefined: + - 'true' + - 'false' + - auto: PREDEFINED + defaultValue: 'false' + description: Revokes the user's existing sessions and returns a temporary password, regardless of the temporary_password value. When true, you'll need to change the password in the next login. + name: revoke_session + predefined: + - 'true' + - 'false' + - auto: PREDEFINED + defaultValue: 'false' + description: Prevents the password from being saved in the War Room. + name: hide_password + predefined: + - 'true' + - 'false' + description: Expires a password for an existing Okta user. + execution: true + name: okta-expire-password + outputs: + - contextPath: Account.Activated + description: Timestamp for when the user was activated. + type: Date + - contextPath: Account.Created + description: Timestamp for when the user was created. + type: Date + - contextPath: Account.DisplayName + description: Okta account display name. + type: String + - contextPath: Account.Email + description: Okta account email. + type: String + - contextPath: Account.ID + description: Created Okta account ID. + type: String + - contextPath: Account.PasswordChanged + description: Timestamp for when the user's password was last changed. + type: Date + - contextPath: Account.Status + description: Okta account current status. + type: String + - contextPath: Account.StatusChanged + description: Timestamp for when the user's status was last changed. + type: Date + - contextPath: Account.Type + description: Okta account type. + type: String + - contextPath: Account.Username + description: Okta account usernames returned by the search. + type: String + - contextPath: Okta.Metadata.x-rate-limit-limit + description: The rate limit ceiling that’s applicable for the current request. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-remaining + description: The amount of requests left for the current rate-limit window. + type: Number + - contextPath: Okta.Metadata.x-rate-limit-reset + description: The time at which the rate limit resets, specified in UTC epoch time (in seconds). + type: Number + compliantpolicies: + - User Soft Remediation + - description: Reset OAuth authentication data (authentication process will start from the beginning, and a new token will be generated). + name: okta-auth-reset + arguments: [] + dockerimage: demisto/crypto:1.0.0.12833174 + runonce: false + script: "" + subtype: python3 + type: python +fromversion: 8.15.0 +tests: +- No tests diff --git a/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_dark.svg b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_dark.svg new file mode 100644 index 000000000000..c6e7f41bec2e --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_dark.svg @@ -0,0 +1,10 @@ + + + + + + + + + + diff --git a/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_description.md b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_description.md new file mode 100644 index 000000000000..d5ee83c257e6 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_description.md @@ -0,0 +1,53 @@ +## Authentication using API Token +Okta API tokens are used to authenticate requests to Okta APIs. + +### Prerequisites +1. Sign in to your Okta organization as a user **with administrator privileges**. +2. In the Admin Console, select **Security** > **API** from the menu, and then select the **Tokens** tab. +3. Click **Create Token**. +4. Name your token and click **Create Token**. + +#### Notes +- API tokens have the same permissions as the user who creates them, and if the permissions of a user change, so do the permissions of the API token. + +For more information, see the '[Create an API token](https://developer.okta.com/docs/guides/create-an-api-token/main/)' official documentation article. + +## Authentication using OAuth 2.0 Authentication +As an alternative to Okta API tokens, you can interact with Okta APIs using scoped OAuth 2.0 access tokens for a number of Okta endpoints. +Each access token enables the bearer to perform specific actions on specific Okta endpoints, with that ability controlled by which scopes the access token contains. + +### Required Scopes +The following scopes are required for the Okta v2 integration to work properly: +- okta.apps.manage +- okta.apps.read +- okta.groups.manage +- okta.groups.read +- okta.logs.read +- okta.networkZones.manage +- okta.networkZones.read +- okta.sessions.manage +- okta.sessions.read +- okta.users.manage +- okta.users.read + + +### Prerequisites +1. Sign in to Okta Admin Console. +2. In the Admin Console, go to **Applications** > **Applications**. +3. Click **Create App Integration**. +4. Select **API Services** as the sign-in method, and click **Next**. +5. Enter the desired name for the created app (e.g., "Cortex XSOAR"), and click **Save**. +6. In the app configuration page, under the **General** tab and the **Client Credentials** section, select **Public key / Private key** for the **Client authentication** option. +7. Under the newly added **PUBLIC KEYS** section, click **Add Key**. +8. In the **Add Public Key** dialog box, click **Generate new key**. Make sure to copy the generated private key (in PEM format) to somewhere safe, and click **Save**. +9. Under the **General Settings** section: + 1. Next to the **Proof of possession** label, uncheck the **Require Demonstrating Proof of Possession (DPoP) header in token requests** option if it's selected. + 2. Next to the **Grant type** label, make sure the **Client Credentials** option is selected, and that the **Token Exchange** option is not selected. + 3. Click **Save**. +10. Under the **Okta API Scopes** tab, grant the required scopes mentioned above for the app. +11. Under the **Admin roles** tab: + 1. Click **Edit assignments**. + 2. In the dropdown list under "Role", select **Super Administrator**. + 3. Click **Save changes** at the top. + +For more information, see the '[Implement OAuth for Okta](https://developer.okta.com/docs/guides/implement-oauth-for-okta/main/)' official documentation article. \ No newline at end of file diff --git a/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_image.png b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_image.png new file mode 100644 index 000000000000..e1ac854dd8b7 Binary files /dev/null and b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_image.png differ diff --git a/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_light.svg b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_light.svg new file mode 100644 index 000000000000..9ce2ac6758ca --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_light.svg @@ -0,0 +1,10 @@ + + + + + + + + + + diff --git a/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_test.py b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_test.py new file mode 100644 index 000000000000..4d50aea4009e --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/OktaStandardConnector_test.py @@ -0,0 +1,45 @@ +"""Sanity tests for the OktaStandardConnector shim integration. + +Full behavioural coverage lives in the OktaApiModule tests; this file +exists only to confirm the shim wires through to the ApiModule correctly. +""" + +from unittest.mock import patch # noqa: F401 + +import OktaStandardConnector as integration_module +import pytest + + +def test_shim_imports_run_entry_point(): + """The shim must re-export the ApiModule entry point under its expected name.""" + assert hasattr( + integration_module, "run_okta_v2_integration" + ), "OktaApiModule.run_okta_v2_integration must be importable via the shim" + + +def test_shim_imports_client(): + """The shim must re-export the Okta v2 Client class and its OktaClient base.""" + assert hasattr(integration_module, "Client") + assert hasattr(integration_module, "OktaClient") + assert issubclass(integration_module.Client, integration_module.OktaClient) + + +def test_shim_imports_auth_symbols(): + """Auth helpers used by the UCP flow must remain accessible from the shim.""" + for name in ("AuthType", "JWTAlgorithm", "resolve_ucp_auth_type", "reset_integration_context"): + assert hasattr(integration_module, name), f"Symbol {name!r} missing from shim" + + +def test_main_delegates_to_api_module(mocker): + """`main()` must do nothing other than call the renamed ApiModule entry point.""" + mock_run = mocker.patch("OktaStandardConnector.run_okta_v2_integration") + integration_module.main() + mock_run.assert_called_once_with() + + +def test_main_propagates_exceptions(mocker): + """`main()` is a thin shim and must not swallow exceptions from the ApiModule.""" + mocker.patch("OktaStandardConnector.run_okta_v2_integration", side_effect=RuntimeError("boom")) + + with pytest.raises(RuntimeError, match="boom"): + integration_module.main() diff --git a/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/README.md b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/README.md new file mode 100644 index 000000000000..4e1fafe6f064 --- /dev/null +++ b/Packs/OktaStandardConnector/Integrations/OktaStandardConnector/README.md @@ -0,0 +1,2815 @@ +Integration with Okta's cloud-based identity management service. + +## Configure Okta v2 in Cortex + +### API Token Authentication Prerequisites + +1. Sign in to your Okta organization as a user with administrator privileges. +2. On the **Admin Console**, select **Security** > **API** from the menu, and then select the **Tokens** tab. +3. Click **Create Token**. +4. Name your token and click **Create Token**. + +#### Notes + +- API tokens have the same permissions as the user who creates them, and if the permissions of a user change, so do the permissions of the API token. +- If more than one certificate is assigned to the application, the Key ID parameter is required to specify which + certificate to use for signing the JWT token. + +For more information, see the '[Create an API token](https://developer.okta.com/docs/guides/create-an-api-token/main/)' official documentation article. + +### OAuth 2.0 Authentication Prerequisites + +#### Required Scopes + +The following scopes are required for the Okta v2 integration to work properly: + +- okta.apps.manage +- okta.apps.read +- okta.groups.manage +- okta.groups.read +- okta.logs.read +- okta.networkZones.manage +- okta.networkZones.read +- okta.sessions.manage +- okta.sessions.read +- okta.users.manage +- okta.users.read + +1. Sign in to Okta Admin Console. +2. In the Admin Console, go to **Applications** > **Applications**. +3. Click **Create App Integration**. +4. Select **API Services** as the sign-in method, and click **Next**. +5. Enter the desired name for the created app (e.g., "Cortex XSOAR"), and click **Save**. +6. In the app configuration page, under the **General** tab and the **Client Credentials** section, select **Public key / Private key** for the **Client authentication** option. +7. Under the newly added **PUBLIC KEYS** section, click **Add Key**. +8. In the **Add Public Key** dialog box, click **Generate new key**. Make sure to copy the generated private key (in PEM format) to somewhere safe, and click **Save**. +9. Under the **General Settings** section: + 1. Next to the **Proof of possession** label, uncheck the **Require Demonstrating Proof of Possession (DPoP) header in token requests** option if it's selected. + 2. Next to the **Grant type** label, make sure the **Client Credentials** option is selected, and that the **Token Exchange** option is not selected. + 3. Click **Save**. +10. Under the **Okta API Scopes** tab, grant the required scopes mentioned above for the app. +11. Under the **Admin roles** tab: + 1. Click **Edit assignments**. + 2. In the dropdown list under "Role", select **Super Administrator**. + 3. Click **Save changes** at the top. + +For more information, see the '[Implement OAuth for Okta](https://developer.okta.com/docs/guides/implement-oauth-for-okta/main/)' official documentation article. + +**Note:** OAuth 2.0 authentication is confirmed to support the 'Revoke all user sessions' functionality. When using the `okta-clear-user-sessions` command with `revoke_oauth_tokens=true`, it revokes OpenID Connect and OAuth refresh and access tokens issued to the user. + +### Instance Configuration + +| **Parameter** | **Description** | **Required** | +|--------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| +| Okta URL (https://<domain>.okta.com) | | True | +| API Token | | False | +| Use OAuth 2.0 Authentication | See detailed instructions on the 'Help' tab. | False | +| Client ID | Required and used if OAuth 2.0 is used for authentication. See detailed instructions on the 'Help' tab. | False | +| Private Key | In PEM format. Required and used if OAuth 2.0 is used for authentication. See detailed instructions on the 'Help' tab. | False | +| JWT Signing Algorithm | Algorithm to sign generated JWT tokens with. Doesn't affect integration's functionality. Required and used if OAuth 2.0 is used for authentication. See detailed instructions on the 'Help' tab. | False | +| Key ID | Required and used if more than one key is used for signing JWT tokens. | False | +| Trust any certificate (not secure) | | False | +| Use system proxy settings | | False | + +## Commands + +You can execute these commands from the CLI, as part of an automation, or in a playbook. +After you successfully execute a command, a DBot message appears in the War Room with the command details. + +### okta-unlock-user + +*** +Unlocks a single user. + +#### Base Command + +`okta-unlock-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Username to unlock. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-unlock-user username=testForDocs@test.com``` + +##### Human Readable Output +> +>User testForDocs@test.com unlocked + +### okta-deactivate-user + +*** +Deactivates a single user. + +#### Base Command + +`okta-deactivate-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Username to deactivate. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-deactivate-user username=testForDocs@test.com``` + +##### Human Readable Output +> +>User testForDocs@test.com deactivated + +### okta-activate-user + +*** +Activates a single user. + +#### Base Command + +`okta-activate-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Username to activate. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-activate-user username=testForDocs@test.com``` + +##### Human Readable Output +> +>### testForDocs@test.com is active now + +### okta-suspend-user + +*** +Suspends a single user. This operation can only be performed on users with an ACTIVE status. After the porcess is completed, the user's status is SUSPENDED. + +#### Base Command + +`okta-suspend-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Username to suspend. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-suspend-user username=testForDocs@test.com``` + +##### Human Readable Output +> +>### testForDocs@test.com status is Suspended + +### okta-unsuspend-user + +*** +Returns a single user to ACTIVE status. This operation can only be performed on users that have a SUSPENDED status. + +#### Base Command + +`okta-unsuspend-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Username to change the status to ACTIVE. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-unsuspend-user username=testForDocs@test.com``` + +##### Human Readable Output +> +>### testForDocs@test.com is no longer SUSPENDED + +### okta-get-user-factors + +*** +Returns all the enrolled factors for the specified user. + +#### Base Command + +`okta-get-user-factors` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Username for which to return all enrolled factors. | Optional | +| userId | User ID of the user for which to get all enrolled factors. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.ID | String | Okta account ID. | +| Account.Factor.ID | String | Okta account factor ID. | +| Account.Factor.Provider | String | Okta account factor provider. | +| Account.Factor.Profile | String | Okta account factor profile. | +| Account.Factor.FactorType | String | Okta account factor type. | +| Account.Factor.Status | Unknown | Okta account factor status. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-user-factors username=factor@test.com``` + +##### Context Example + +``` +{ + "Account": { + "Factor": [ + { + "FactorType": "sms", + "ID": "mblpt21nffaaN5F060h7", + "Profile": { + "phoneNumber": "+12025550191" + }, + "Provider": "OKTA", + "Status": "PENDING_ACTIVATION" + }, + { + "FactorType": "token:software:totp", + "ID": "uftpt24kdrDJ7fDOq0h7", + "Profile": { + "credentialId": "factor@test.com" + }, + "Provider": "GOOGLE", + "Status": "PENDING_ACTIVATION" + }, + { + "FactorType": "push", + "ID": "opfpt1joeaArlg27g0h7", + "Provider": "OKTA", + "Status": "PENDING_ACTIVATION" + } + ], + "ID": "00upt1w8tgFQM2v6t4" + } +} +``` + +##### Human Readable Output +> +>Factors for user: 00upt1w8tgFQM2v0h7 +> +>### Factors +> +>|FactorType|ID|Profile|Provider|Status| +>|---|---|---|---|---| +>| sms | mbgt21nffaaN5F060h7 | phoneNumber: +12025550191 | OKTA | PENDING_ACTIVATION | +>| token:software:totp | uftptgdrDJ7fDOq0h7 | credentialId: factor@test.com | GOOGLE | PENDING_ACTIVATION | +>| push | opfg1joeaArlg27g0h7 | | OKTA | PENDING_ACTIVATION | + +### okta-reset-factor + +*** +Un-enrolls an existing factor for the specified user. This enables the user to enroll a new factor. + +#### Base Command + +`okta-reset-factor` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| userId | The user ID. | Optional | +| username | Username for which to un-enroll an existing factor. | Optional | +| factorId | The ID of the factor to reset. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-reset-factor factorId=ufsq7cvptfbjQa72c0h7 userId=00upt1w8t40wFQM2v6t4``` + +##### Human Readable Output +> +>Factor: ufsq7cvptfbjQa72c0h7 deleted + +### okta-set-password + +*** +Sets passwords without validating existing user credentials. + +#### Base Command + +`okta-set-password` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Okta username for which to set the password. | Required | +| password | The new password to set for the user. | Required | +| temporary_password | When true, you'll need to change the password in the next login. Possible values are: true, false. Default is false. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-set-password username=testForDocs@test.com password=N3wPa55word!``` + +##### Human Readable Output +> +>testForDocs@test.com password was last changed on 2020-03-26T13:57:13.000Z + +### okta-add-to-group + +*** +Adds a user to a group with OKTA_GROUP type. + +#### Base Command + +`okta-add-to-group` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| userId | ID of the user to add to the group. | Optional | +| username | Name of the user to add to the group. | Optional | +| groupId | ID of the group to add the user to. | Optional | +| groupName | Name of the group to add the user to. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-add-to-group groupName=Demisto username=testForDocs@test.com``` + +##### Human Readable Output +> +>User: 00uqk1qesl3k0SRbH0h7 added to group: Demisto successfully + +### okta-remove-from-group + +*** +Removes a user from a group with OKTA_GROUP type. + +#### Base Command + +`okta-remove-from-group` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| userId | ID of the user to remove from the group. | Optional | +| username | Name of the user to remove from the group. | Optional | +| groupId | ID of the group to remove the user from. | Optional | +| groupName | Name of the group to remove the user from. | Optional | + +#### Context Output + +There is no context output for this command. + +##### Command Example + +```!okta-remove-from-group groupName=demisto username=testForDocs@test.com``` + +##### Human Readable Output +> +>User: 00uqk1qesl3k0SRbH0h7 was removed from group: demisto successfully + +### okta-get-groups + +*** +Returns all user groups associated with a specified user. + +#### Base Command + +`okta-get-groups` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Username in Okta for which to get the associated groups. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.Group | Unknown | Okta groups with which the account is associated. | +| Account.ID | String | Okta account ID. | +| Account.Type | String | Okta account type. | +| Account.Group.ID | String | Unique key for the group. | +| Account.Group.Created | Date | Timestamp when the group was created. | +| Account.Group.ObjectClass | String | The object class, which determines the group's profile. | +| Account.Group.LastUpdated | Date | Timestamp when the group's profile was last updated. | +| Account.Group.LastMembershipUpdated | Date | Timestamp when the group's memberships were last updated. | +| Account.Group.Type | String | Group type, which determines how a group's profile and memberships are managed. | +| Account.Group.Description | String | Description of the group. | +| Account.Group.Name | String | Name of the group. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-groups username=testForDocs@test.com``` + +##### Context Example + +``` +{ + "Account": { + "Group": [ + { + "Created": "2016-04-12T15:01:50.000Z", + "Description": "All users in your organization", + "ID": "00g66lckcsAJpLcNc0h7", + "LastMembershipUpdated": "2020-03-26T13:56:49.000Z", + "LastUpdated": "2016-04-12T15:01:50.000Z", + "Name": "Everyone", + "ObjectClass": [ + "okta:user_group" + ], + "Type": "BUILT_IN" + }, + { + "Created": "2018-01-19T02:02:06.000Z", + "ID": "00gdougcq3zEaf7c50h7", + "LastMembershipUpdated": "2020-03-26T13:49:47.000Z", + "LastUpdated": "2018-01-19T02:02:06.000Z", + "Name": "Demisto", + "ObjectClass": [ + "okta:user_group" + ], + "Type": "OKTA_GROUP" + } + ], + "ID": "00uqk1qesl3k0SRbH0h7", + "Type": "Okta" + } +} +``` + +##### Human Readable Output +> +>Okta groups for user: testForDocs@test.com +> +>### Groups +> +>|Created|Description|ID|LastMembershipUpdated|LastUpdated|Name|ObjectClass|Type| +>|---|---|---|---|---|---|---|---| +>| 2016-04-12T15:01:50.000Z | All users in your organization | 00g66lckgAJpLcNc0h7 | 2020-03-26T13:56:49.000Z | 2016-04-12T15:01:50.000Z | Everyone | okta:user_group | BUILT_IN | +>| 2018-01-19T02:02:06.000Z | | 00gdougcgzEaf7c50h7 | 2020-03-26T13:49:47.000Z | 2018-01-19T02:02:06.000Z | Demisto | okta:user_group | OKTA_GROUP | + +### okta-verify-push-factor + +*** +Enrolls and verifies a push factor for the specified user. + +#### Base Command + +`okta-verify-push-factor` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| userId | The ID of the user to enroll and verify. | Required | +| factorId | The push factor ID. | Required | +| polling_time | Time to wait between subsequent polling calls. Value is in seconds. Default time is 5 seconds. | Optional | +| max_polling_calls | Maximum number of polling calls. Default value is 10. | Optional | +| polling | Whether to poll for the push factor challenge result. Possible values are: true, false. Default is true. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.ID | String | Okta user ID. | +| Account.VerifyPushResult | String | Okta user push factor result. | +| Okta.PollingStatusURL | String | The polling URL for the push factor challenge. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-verify-push-factor factorId=opfpt1joeaArlg27g0h7 userId=00upt1w8t40wFQM2v0h7``` + +##### Human Readable Output +> +>Verify push factor result for user 00upt1w8t40wgQM2v0h7: WAITING + +##### Context Example + +``` +{ + "factorResult": "WAITING", + "profile": { + "credentialId": "test@this.com", + "deviceType": "SmartPhone_IPhone", + "keys": [ + { + "kty": "EC", + "use": "sig", + "kid": "default", + "x": "3Y53lDoQYwzzVbjsbsPnqOnVaotIrVByQh5Sa-RwOHQ", + "y": "0zHY_y9rVh-bq_-lR-MrmzNtUZrrIMbTrsjtxUyUT2Q", + "crv": "P-256" + } + ], + "name": "iPhone (5)", + "platform": "IOS", + "version": "13.1.3" + }, + "expiresAt": "2020-02-24T11:37:08.000Z", + "_links": { + "cancel": { + "href": "https://test.com/api/v1/users/00upt1w8t40wgQM2v0h7/factors/FactorID/transactions/TransactionID", + "hints": { + "allow": [ + "DELETE" + ] + } + }, + "poll": { + "href": "https://test.com/api/v1/users/00upt1w8t40wgQM2v0h7/factors/FactorID/transactions/TransactionID", + "hints": { + "allow": [ + "GET" + ] + } + } + } +} +``` + +### okta-verify-mfa-status + +*** +Verifies the status of a push factor challenge. + +#### Base Command + +`okta-verify-mfa-status` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| polling_url | The polling URL for the push factor challenge. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.FactorResult.ID | String | The push factor challenge ID. | +| Okta.FactorResult.factorResult | String | The result of the factor challenge. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-verify-mfa-status polling_url="https://test.com/api/v1/users/TestID/factors/FactorID/transactions/TransactionID"``` + +##### Human Readable Output +> +>The status of the push factor is SUCCESS + +### okta-search + +*** +Searches for Okta users. + +#### Base Command + +`okta-search` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| term | Term by which to search. Can be a first name, last name, or email address. The argument `term` or `advanced_search` is required. | Optional | +| advanced_search | Searches for users with a supported filtering expression for most properties, including custom-defined properties. The argument `term` or `advanced_search` is required. | Optional | +| limit | The maximum number of results to return. The default and maximum is 200. | Optional | +| verbose | Whether to return details of users that match the found term. Can be "true" or "false". The default is "false". Possible values are: true, false. Default is false. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.ID | String | Okta account IDs returned by the search. | +| Account.Username | String | Okta account usernames returned by the search. | +| Account.Email | String | Okta account emails returned by the search. | +| Account.DisplayName | String | Okta account display names returned by the search. | +| Account.Type | String | Okta account type returned by the search. | +| Account.Status | String | Okta account current status. | +| Account.Created | Date | Timestamp for when the user was created. | +| Account.Activated | Date | Timestamp for when the user was activated. | +| Account.StatusChanged | Date | Timestamp for when the user's status was last changed. | +| Account.PasswordChanged | Date | Timestamp for when the user's password was last changed. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-search term=test verbose=true``` + +##### Context Example + +``` +{ + "Account": [ + { + "Activated": "2020-02-12T14:03:51.000Z", + "Created": "2020-02-12T14:03:50.000Z", + "DisplayName": "bar test", + "Email": "bartest@test.com", + "ID": "00uppjeleqJQ2kkN80h7", + "Status": "PROVISIONED", + "StatusChanged": "2020-02-12T14:03:51.000Z", + "Type": "Okta", + "Username": "bartest@test.com" + }, + { + "Activated": "2020-02-19T12:33:20.000Z", + "Created": "2018-07-31T12:48:33.000Z", + "DisplayName": "test that", + "Email": "test@that.com", + "ID": "00ufufhqits3y78Ju0h7", + "PasswordChanged": "2020-02-06T13:32:56.000Z", + "Status": "PROVISIONED", + "StatusChanged": "2020-02-19T12:33:20.000Z", + "Type": "Okta", + "Username": "test@that.com" + }, + { + "Activated": "2020-03-26T13:56:52.000Z", + "Created": "2020-03-26T13:56:49.000Z", + "DisplayName": "test that", + "Email": "testForDocs@test.com", + "ID": "00uqk1qesl3k0SRbH0h7", + "PasswordChanged": "2020-03-26T13:56:50.000Z", + "Status": "ACTIVE", + "StatusChanged": "2020-03-26T13:56:52.000Z", + "Type": "Okta", + "Username": "testForDocs@test.com" + } + ] +} +``` + +##### Human Readable Output +> +>Okta users found: +>User:bartest@test.com +> +>### Profile +> +>|Email|First Name|Last Name|Login|Mobile Phone|Second Email| +>|---|---|---|---|---|---| +>| bartest@test.com | bar | test | bartest@test.com | | | + +##### Additional Data + +|Activated|Created|Credentials|ID|Last Login|Last Updated|Password Changed|Status|Status Changed|Type|_links| +|---|---|---|---|---|---|---|---|---|---|---| +| 2020-02-12T14:03:51.000Z | 2020-02-12T14:03:50.000Z | provider: {"type": "OKTA", "name": "OKTA"} | 00uppjeleqJQ2kkN80h7 | | 2020-02-12T14:03:51.000Z | | PROVISIONED | | id: oty66lckcvDyVcGzS0h7 | self: {"href": "https://yourdomain.okta.com/api/v1/users/00uppjeleqJQ2kkN80h7"} | + +##### User:test@that.com + +##### Profile + +|Email|First Name|Last Name|Login|Mobile Phone|Second Email| +|---|---|---|---|---|---| +| test@that.com | test | that | test@that.com | | test@that.com | + +##### Additional Data + +|Activated|Created|Credentials|ID|Last Login|Last Updated|Password Changed|Status|Status Changed|Type|_links| +|---|---|---|---|---|---|---|---|---|---|---| +| 2020-02-19T12:33:20.000Z | 2018-07-31T12:48:33.000Z | provider: {"type": "OKTA", "name": "OKTA"} | 00ufufhqits3y78Ju0h7 | | 2020-02-19T12:33:20.000Z | 2020-02-06T13:32:56.000Z | PROVISIONED | | id: oty66lckcvDyVcGzS0h7 | self: {"href": "https://yourdomain.okta.com/api/v1/users/00ufufhqits3y78Ju0h7"} | + +##### User:testForDocs@test.com + +##### Profile + +|Email|First Name|Last Name|Login|Mobile Phone|Second Email| +|---|---|---|---|---|---| +| testForDocs@test.com | test | that | testForDocs@test.com | | | + +##### Additional Data + +|Activated|Created|Credentials|ID|Last Login|Last Updated|Password Changed|Status|Status Changed|Type|_links| +|---|---|---|---|---|---|---|---|---|---|---| +| 2020-03-26T13:56:52.000Z | 2020-03-26T13:56:49.000Z | password: {}recovery_question: {"question": "whats is your favourite integration"}provider: {"type": "OKTA", "name": "OKTA"} | 00uqk1qesl3k0SRbH0h7 | | 2020-03-26T13:56:52.000Z | 2020-03-26T13:56:50.000Z | ACTIVE | | id: oty66lckcvDyVcGzS0h7 | self: {"href": "https://yourdomain.okta.com/api/v1/users/00uqk1qesl3k0SRbH0h7"} | + +### okta-get-user + +*** +Fetches information for a single user. You must enter one or more parameters for the command to run. + +#### Base Command + +`okta-get-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Okta username for which to get information. | Optional | +| userId | User ID of the user for which to get information. | Optional | +| userEmail | Email of the user for which to get information. | Optional | +| verbose | Whether to return extended user information. Can be "true" or "false". The default is "false". Possible values are: true, false. Default is false. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.ID | String | Okta account ID. | +| Account.Email | String | Okta account email. | +| Account.Username | String | Okta account username. | +| Account.DisplayName | String | Okta account display name. | +| Account.Status | String | Okta account status. | +| Account.Created | Date | Timestamp for when the user was created. | +| Account.Activated | Date | Timestamp for when the user was activated. | +| Account.StatusChanged | Date | Timestamp for when the user's status was last changed. | +| Account.PasswordChanged | Date | Timestamp for when the user's password was last changed. | +| Account.Manager | String | The manager. | +| Account.ManagerEmail | String | The manager email. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-user username=testForDocs@test.com verbose=true``` + +##### Context Example + +``` +{ + "Account": { + "Activated": "2020-03-26T13:56:52.000Z", + "Created": "2020-03-26T13:56:49.000Z", + "DisplayName": "test that", + "Email": "testForDocs@test.com", + "ID": "00uqk1qesl3k0SRbH0h7", + "Manager": "manager@test.com", + "ManagerEmail": null, + "PasswordChanged": "2020-03-26T13:56:50.000Z", + "Status": "ACTIVE", + "StatusChanged": "2020-03-26T13:56:52.000Z", + "Type": "Okta", + "Username": "testForDocs@test.com" + } +} +``` + +##### Human Readable Output +> +>### User:testForDocs@test.com +> +>### Profile +> +>|Email|First Name|Last Name|Login|Manager|Manager Email|Mobile Phone|Second Email| +>|---|---|---|---|---|---|---|---| +>| testForDocs@test.com | test | that | testForDocs@test.com | manager@test.com | | | | + +##### Additional Data + +|Activated|Created|Credentials|ID|Last Login|Last Updated|Password Changed|Status|Status Changed|Type|_links| +|---|---|---|---|---|---|---|---|---|---|---| +| 2020-03-26T13:56:52.000Z | 2020-03-26T13:56:49.000Z | password: {}recovery_question: {"question": "whats is your favourite integration"} provider: {"type": "OKTA", "name": "OKTA"} | 00uqk1qesl3k0SRbH0h7 | | 2020-03-26T13:56:52.000Z | 2020-03-26T13:56:50.000Z | ACTIVE | | id: oty66lckcvDyVcGzS0h7 | links| + +### okta-list-users + +*** +Lists users in your organization. + +#### Base Command + +`okta-list-users` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| after | The cursor in which to retrive the results from and on. if the query didn't reach the end of results, the tag can be obtained from the bottom of the grid in the readable output, or in the context path Okta.User.tag. | Optional | +| limit | The maximum number of results to return. Default is 200. | Optional | +| verbose | Whether to return extended user information. Possible values are: true, false. Default is false. | Optional | +| query | Searches the name property of groups for matching values. | Optional | +| filter | Useful for performing structured queries where constraints on group attribute values can be explicitly targeted.
The following expressions are supported(among others) for groups with the filter query parameter:
type eq "OKTA_GROUP" - Groups that have a type of OKTA_GROUP; lastUpdated lt "yyyy-MM-dd''T''HH:mm:ss.SSSZ" - Groups with profile last updated before a specific timestamp; lastMembershipUpdated eq "yyyy-MM-dd''T''HH:mm:ss.SSSZ" - Groups with memberships last updated at a specific timestamp; id eq "00g1emaKYZTWRYYRRTSK" - Group with a specified ID. For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.ID | String | Okta account ID. | +| Account.Email | String | Okta account email. | +| Account.Username | String | Okta account username. | +| Account.DisplayName | String | Okta account display name. | +| Account.Status | String | Okta account status. | +| Account.Created | Date | Timestamp for when the user was created. | +| Account.Activated | Date | Timestamp for when the user was activated. | +| Account.StatusChanged | Date | Timestamp for when the user's status was last changed. | +| Account.PasswordChanged | Date | Timestamp for when the user's password was last changed. | +| Okta.User.tag | String | The location of the next item, used with after param. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +### okta-create-user + +*** +Creates a new user with an option of setting a password, recovery question, and answer. The new user will immediately be able to log in after activation with the assigned password. This flow is common when developing a custom user registration experience. + +#### Base Command + +`okta-create-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| firstName | First name of the user (givenName). | Required | +| lastName | Family name of the user (familyName). | Required | +| email | Primary email address of the user. | Required | +| login | Unique identifier for the user (username). | Required | +| secondEmail | Secondary email address of user. Usually used for account recovery. | Optional | +| middleName | Middle name(s) of the user. | Optional | +| honorificPrefix | A comma-separated list of honorific prefix(es) of the user, or title in most Western languages. | Optional | +| honorificSuffix | A comma-separated list of honorific suffix(es) of the user. | Optional | +| title | User's title. for example, Vice President. | Optional | +| displayName | Display name of the user. | Optional | +| nickName | Casual way to address the user (nick name). | Optional | +| profileUrl | URL of the user online profile. For example, a web page. | Optional | +| primaryPhone | Primary phone number of the user. | Optional | +| mobilePhone | Mobile phone number of the user. | Optional | +| streetAddress | Full street address component of the user's address. | Optional | +| city | City or locality component of the user's address (locality). | Optional | +| state | State or region component of the user's address (region). | Optional | +| zipCode | Zip code or postal code component of the user's address (postalCode). | Optional | +| countryCode | Country name component of the user's address (country). | Optional | +| postalAddress | Mailing address component of the user's address. | Optional | +| preferredLanguage | User's preferred written or spoken languages. | Optional | +| locale | User's default location, for purposes of localizing items such as currency, date-time format, numerical representations, etc. | Optional | +| timezone | User's time zone. | Optional | +| userType | The user type, which is used to identify the organization-to-user relationship such as "Employee" or "Contractor". | Optional | +| employeeNumber | Organization or company assigned unique identifier for the user. | Optional | +| costCenter | Name of a cost center the user is assigned to. | Optional | +| organization | Name of the user's organization. | Optional | +| division | Name of the user's division. | Optional | +| department | Name of the user's department. | Optional | +| managerId | ID of the user's manager. | Optional | +| manager | Display name of the user's manager. | Optional | +| password | Password for the new user. | Optional | +| passwordQuestion | Password question for the new user. | Optional | +| passwordAnswer | Password answer for question. | Optional | +| providerType | The provider type. Can be "OKTA", "ACTIVE_DIRECTORY", "LDAP", "FEDERATION", or "SOCIAL". Possible values are: OKTA, ACTIVE_DIRECTORY, LDAP, FEDERATION, SOCIAL. | Optional | +| providerName | Name of the provider. | Optional | +| groupIds | IDs of groups that the user will be immediately added to at time of creation (does Not include default group). | Optional | +| activate | Whether to activate the lifecycle operation when creating the user. Can be "true" or "false". Possible values are: true, false. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.ID | String | Created Okta account ID. | +| Account.Email | String | Created Okta account email address. | +| Account.Username | String | Created okta account username. | +| Account.DisplayName | String | Created Okta account display name. | +| Account.Type | String | Type of created account - Okta. | +| Account.Status | String | Okta account current status. | +| Account.Created | Date | Timestamp for when the user was created. | +| Account.Activated | Date | Timestamp for when the user was activated. | +| Account.StatusChanged | Date | Timestamp for when the user's status was last changed. | +| Account.PasswordChanged | Date | Timestamp for when the user's password was last changed. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-create-user email=testForDocs@test.com firstName=test lastName=that login=testForDocs@test.com password=Pa55word! passwordQuestion="whats is your favourite integration" passwordAnswer="Okta of course"``` + +##### Context Example + +``` +{ + "Account": { + "Activated": null, + "Created": "2020-03-26T13:56:49.000Z", + "DisplayName": "test that", + "Email": "testForDocs@test.com", + "ID": "00uqk1qesl3k0SRbH0h7", + "PasswordChanged": "2020-03-26T13:56:50.000Z", + "Status": "STAGED", + "StatusChanged": null, + "Type": "Okta", + "Username": "testForDocs@test.com" + } +} +``` + +##### Human Readable Output +> +>### Okta User Created: testForDocs@test.com +> +>|First Name|ID|Last Login|Last Name|Login|Mobile Phone|Status| +>|---|---|---|---|---|---|---| +>| test | 00uqk1qesl3k0SRbH0h7 | | that | testForDocs@test.com | | STAGED | + +### okta-update-user + +*** +Updates a user with a given login. All fields are optional. Fields which are not set, will not be overwritten. + +#### Base Command + +`okta-update-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| firstName | First name of the user (given name). | Optional | +| lastName | Family name of the user. | Optional | +| email | Primary email address of the user. | Optional | +| username | Unique identifier for the user (login). | Required | +| secondEmail | Secondary email address of the user (typically used for account recovery. | Optional | +| middleName | Middle name(s) of the user. | Optional | +| honorificPrefix | Honorific prefix(es) of the user, or title in most Western languages. | Optional | +| honorificSuffix | Honorific suffix(es) of the user. | Optional | +| title | User's title. For example, Vice President. | Optional | +| displayName | Display name of the user. | Optional | +| nickName | Casual way to address the user in real life (nick name). | Optional | +| profileUrl | URL of the user's online profile. For example, a web page. | Optional | +| primaryPhone | Primary phone number of the user. | Optional | +| mobilePhone | Mobile phone number of the user. | Optional | +| streetAddress | Full street address component of the user's address. | Optional | +| city | City or locality component of the user's address (locality). | Optional | +| state | State or region component of the user's address (region). | Optional | +| zipCode | Zip code or postal code component of the user's address (postalCode). | Optional | +| countryCode | Country name component of the user's address (country). | Optional | +| postalAddress | Mailing address component of the user's address. | Optional | +| preferredLanguage | User's preferred written or spoken languages. | Optional | +| locale | User's default location for purposes of localizing items such as currency, date-time format, numerical representations, etc. | Optional | +| timezone | User time zone. | Optional | +| userType | The user type, which is used to identify the organization-to-user relationship such as "Employee" or "Contractor". | Optional | +| employeeNumber | Organization or company assigned unique identifier for the user. | Optional | +| costCenter | Name of a cost center the user is assigned to. | Optional | +| organization | Name of the user's organization. | Optional | +| division | Name of the user's division. | Optional | +| department | Name of the user's department. | Optional | +| managerId | ID of the user's manager. | Optional | +| manager | Display name of the user's manager. | Optional | +| password | New password for the specified user. | Optional | +| passwordQuestion | Password question for the specified user. | Optional | +| passwordAnswer | Password answer for the question. | Optional | +| providerType | The provider type. Can be "OKTA", "ACTIVE_DIRECTORY", "LDAP", "FEDERATION", or "SOCIAL". Possible values are: OKTA, ACTIVE_DIRECTORY, FEDERATION, SOCIAL. | Optional | +| providerName | Name of the provider. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-update-user username=testForDocs@test.com firstName="First Name Updated"``` + +##### Human Readable Output +> +>### Okta user: testForDocs@test.com Updated +> +>|email|firstName|lastName|login|mobilePhone|secondEmail| +>|---|---|---|---|---|---| +>| testForDocs@test.com | First Name Updated | that | testForDocs@test.com | | | + +### okta-get-group-members + +*** +Enumerates all users that are members of a group. + +#### Base Command + +`okta-get-group-members` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| groupId | ID of the group. | Optional | +| limit | The maximum number of results to return. | Optional | +| verbose | Whether to print extended user details. Can be "true" or "false". The default is "false". Possible values are: true, false. Default is false. | Optional | +| groupName | Name of the group. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.ID | String | Okta account ID. | +| Account.Email | String | Okta account email address. | +| Account.Username | String | Okta account username. | +| Account.DisplayName | String | Okta account display name. | +| Account.Type | String | Account type - Okta. | +| Account.Status | String | Okta account current status. | +| Account.Created | Date | Timestamp for when the user was created. | +| Account.Activated | Date | Timestamp for when the user was activated. | +| Account.StatusChanged | Date | Timestamp for when the user's status was last changed. | +| Account.PasswordChanged | Date | Timestamp for when the user's password was last changed. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-group-members groupName=Demisto limit=1 verbose=true``` + +##### Context Example + +``` +{ + "Account": { + "Created": "2016-04-12T15:01:52.000Z", + "DisplayName": "Test Demisto", + "Email": "XSOAR@demisto.com", + "ID": "00u66ljhpjidYi0h7", + "PasswordChanged": "2020-02-24T11:40:08.000Z", + "Status": "ACTIVE", + "StatusChanged": "2016-04-12T15:05:06.000Z", + "Type": "Okta", + "Username": "XSOAR@demisto.com" + } +} +``` + +##### Human Readable Output +> +>### Users for group: Demisto +> +>### User:Test@demisto.com +> +>### Profile +> +>|Email|First Name|Last Name|Login|Mobile Phone|Second Email| +>|---|---|---|---|---|---| +>| XSOAR@demisto.com | Test | Demisto | XSOAR@demisto.com | | | + +##### Additional Data + +|Activated|Created|Credentials|ID|Last Login|Last Updated|Password Changed|Status|Status Changed|Type|_links| +|---|---|---|---|---|---|---|---|---|---|---| +| | 2016-04-12T15:01:52.000Z | password: {} recovery_question: {"question": "born city"} provider: {"type": "OKTA", "name": "OKTA"} | 00u66lckd7lpjidYi0h7 | 2020-03-12T09:54:36.000Z | 2020-02-24T11:42:22.000Z | 2020-02-24T11:40:08.000Z | ACTIVE | | id: oty66lckcyVcGzS0h7 | self: {"href": "https://yourdomain.okta.com/api/v1/users/00uclpjidYi0h7"} | + +### okta-list-groups + +*** +Lists groups in your organization. A subset of groups can be returned that match a supported filter expression or query. + +#### Base Command + +`okta-list-groups` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| query | Searches the name property of groups for matching values. | Optional | +| filter | Useful for performing structured queries where constraints on group attribute values can be explicitly targeted.
The following expressions are supported(among others) for groups with the filter query parameter:
type eq "OKTA_GROUP" - Groups that have a type of OKTA_GROUP; lastUpdated lt "yyyy-MM-dd''T''HH:mm:ss.SSSZ" - Groups with profile last updated before a specific timestamp; lastMembershipUpdated eq "yyyy-MM-dd''T''HH:mm:ss.SSSZ" - Groups with memberships last updated at a specific timestamp; id eq "00g1emaKYZTWRYYRRTSK" - Group with a specified ID. For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering. | Optional | +| limit | The maximum number of results to return. The default is 200. Default is 200. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Group.ID | String | Unique key for the group. | +| Okta.Group.Created | Date | Timestamp for when the group was created. | +| Okta.Group.ObjectClass | Unknown | The group's profile. | +| Okta.Group.LastUpdated | Date | Timestamp for when the group's profile was last updated. | +| Okta.Group.LastMembershipUpdated | Date | Timestamp for when the group's membership was last updated. | +| Okta.Group.Type | String | The group type, which determines how a group's profile and membership are managed. Can be "OKTA_GROUP", "APP_GROUP", or "BUILT_IN". | +| Okta.Group.Name | String | Name of the group. | +| Okta.Group.Description | String | Description of the group. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-list-groups filter=`type eq "OKTA_GROUP" and lastUpdated lt "2019-04-30T00:00:00.000Z" and lastMembershipUpdated gt "2019-04-30T00:00:00.000Z"` query=demisto``` + +##### Context Example + +``` +{ + "Okta": { + "Group": { + "Created": "2018-01-19T02:02:06.000Z", + "ID": "00gdout3zEaf7c50h7", + "LastMembershipUpdated": "2020-03-26T13:56:56.000Z", + "LastUpdated": "2018-01-19T02:02:06.000Z", + "Name": "Demisto", + "ObjectClass": [ + "okta:user_group" + ], + "Type": "OKTA_GROUP" + } + } +} +``` + +##### Human Readable Output +> +>### Groups +> +>|Created|Description|ID|LastMembershipUpdated|LastUpdated|Name|ObjectClass|Type| +>|---|---|---|---|---|---|---|---| +>| 2018-01-19T02:02:06.000Z | | 00gdougctEaf7c50h7 | 2020-03-26T13:56:56.000Z | 2018-01-19T02:02:06.000Z | Demisto | okta:user_group | OKTA_GROUP | + +### okta-get-failed-logins + +*** +Returns failed login events. + +#### Base Command + +`okta-get-failed-logins` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| since | Filters the lower time bound of the log events in the Internet Date/Time Format profile of ISO 8601. An example: 2017-05-03T16:22:18Z. | Optional | +| until | Filters the upper time bound of the log events in the Internet Date/Time Format profile of ISO 8601. An example: 2017-05-03T16:22:18Z. | Optional | +| sortOrder | The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". Possible values are: ASCENDING, DESCENDING. Default is ASCENDING. | Optional | +| limit | The maximum number of results to return. The default is 100. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Logs.Events.actor.alternateId | String | Alternative ID of the actor. | +| Okta.Logs.Events.actor.displayName | String | Display name of the actor. | +| Okta.Logs.Events.actor.id | String | ID of the actor. | +| Okta.Logs.Events.client.userAgent.rawUserAgent | String | A raw string representation of the user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. | +| Okta.Logs.Events.client.userAgent.os | String | The operating system on which the client runs. For example, Microsoft Windows 10. | +| Okta.Logs.Events.client.userAgent.browser | String | Identifies the browser type, if relevant. For example, Chrome. | +| Okta.Logs.Events.client.device | String | Type of device that client operated from. For example, Computer. | +| Okta.Logs.Events.client.id | String | For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. | +| Okta.Logs.Events.client.ipAddress | String | IP address from which the client made its request. | +| Okta.Logs.Events.client.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.client.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example Montana, Incheon. | +| Okta.Logs.Events.client.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.displayMessage | String | The display message for an event. | +| Okta.Logs.Events.eventType | String | Type of event that was published. | +| Okta.Logs.Events.outcome.result | String | Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", "UNKNOWN". | +| Okta.Logs.Events.outcome.reason | String | Reason for the result. For example, INVALID_CREDENTIALS. | +| Okta.Logs.Events.published | String | Timestamp when the event was published. | +| Okta.Logs.Events.severity | String | The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". | +| Okta.Logs.Events.securityContext.asNumber | Number | Autonomous system number associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.asOrg | String | Organization associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.isp | String | Internet service provider used to send the event's request. | +| Okta.Logs.Events.securityContext.domain | String | The domain name associated with the IP address of the inbound event request. | +| Okta.Logs.Events.securityContext.isProxy | String | Specifies whether an event's request is from a known proxy. | +| Okta.Logs.Events.request.ipChain.IP | String | IP address. | +| Okta.Logs.Events.request.ipChain.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.request.ipChain.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.request.ipChain.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.request.ipChain.source | String | Details regarding the source. | +| Okta.Logs.Events.target.id | String | ID of a target. | +| Okta.Logs.Events.target.type | String | Type of a target. | +| Okta.Logs.Events.target.alternateId | String | Alternative ID of a target. | +| Okta.Logs.Events.target.displayName | String | Display name of a target. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-failed-logins since="2019-04-30T00:00:00.000Z" limit=1``` + +##### Context Example + +``` +{ + "Okta": { + "Logs": { + "Events": { + "actor": { + "alternateId": "goo@test.com", + "detailEntry": null, + "displayName": "unknown", + "id": "unknown", + "type": "User" + }, + "authenticationContext": { + "authenticationProvider": null, + "authenticationStep": 0, + "credentialProvider": null, + "credentialType": null, + "externalSessionId": "unknown", + "interface": null, + "issuer": null + }, + "client": { + "device": "Computer", + "geographicalContext": { + "city": "Tel Aviv", + "country": "Israel", + "geolocation": { + "lat": 32.0678, + "lon": 34.7647 + }, + "postalCode": null, + "state": "Tel Aviv" + }, + "id": null, + "ipAddress": "127.0.0.1", + "userAgent": { + "browser": "CHROME", + "os": "Windows 10", + "rawUserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36" + }, + "zone": "null" + }, + "debugContext": { + "debugData": { + "loginResult": "VERIFICATION_ERROR", + "requestId": "XYM92q-0Fs0ewvRQZVAlzwAABL8", + "requestUri": "/api/v1/authn", + "url": "/api/v1/authn?" + } + }, + "displayMessage": "User login to Okta", + "eventType": "user.session.start", + "legacyEventType": "core.user_auth.login_failed", + "outcome": { + "reason": "VERIFICATION_ERROR", + "result": "FAILURE" + }, + "published": "2019-09-19T08:35:38.353Z", + "request": { + "ipChain": [ + { + "geographicalContext": { + "city": "Tel Aviv", + "country": "Israel", + "geolocation": { + "lat": 32.0678, + "lon": 34.7647 + }, + "postalCode": null, + "state": "Tel Aviv" + }, + "ip": "127.0.0.1", + "source": null, + "version": "V4" + } + ] + }, + "securityContext": { + "asNumber": null, + "asOrg": null, + "domain": null, + "isProxy": null, + "isp": null + }, + "severity": "INFO", + "target": null, + "transaction": { + "detail": {}, + "id": "XYM92q-0Fs0ewvRQZVAlzwAABL8", + "type": "WEB" + }, + "uuid": "7503c6b4-dab8-11e9-b336-d163e95fbe00", + "version": "0" + } + } + } +} +``` + +##### Human Readable Output +> +>### Failed Login Events +> +>|Actor|ActorAlternaneId|ChainIP|Client|EventInfo|EventOutcome|EventSeverity|RequestIP|Targets|Time| +>|---|---|---|---|---|---|---|---|---|---| +>| unknown (User) | admin | 127.0.0.1 | CHROME on Windows 10 Computer | User login to Okta | FAILURE: VERIFICATION_ERROR | INFO | 127.0.0.1 | - | 09/30/2019, 18:42:38 | + +### okta-get-logs + +*** +Gets logs by providing optional filters. + +#### Base Command + +`okta-get-logs` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| filter | Useful for performing structured queries where constraints on LogEvent attribute values can be explicitly targeted.
The following expressions are supported for events with the filter query parameter: eventType eq " :eventType"
-Events that have a specific action; eventType target.id eq ":id"
- Events published with a specific target id; actor.id eq ":id"
- Events published with a specific actor ID. For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering. | Optional | +| query | The query parameter can be used to perform keyword matching against a LogEvents object’s attribute values. To satisfy the constraint, all supplied keywords must be matched exactly. Note that matching is case-insensitive. The following are some examples of common keyword filtering:
Events that mention a specific city: query=San Francisco;
Events that mention a specific url: query=interestingURI.com;
Events that mention a specific person: query=firstName lastName. | Optional | +| since | Filters the lower time bound of the log events in the Internet Date/Time Format profile of ISO 8601. For example: 2017-05-03T16:22:18Z. | Optional | +| until | Filters the upper time bound of the log events in the Internet Date/Time Format profile of ISO 8601. For example: 2017-05-03T16:22:18Z. | Optional | +| sortOrder | The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". Possible values are: ASCENDING, DESCENDING. Default is ASCENDING. | Optional | +| limit | The maximum number of results to return. The default is 100. Default is 100. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Logs.Events.actor.alternateId | String | Alternative ID of the actor. | +| Okta.Logs.Events.actor.displayName | String | Display name of the actor. | +| Okta.Logs.Events.actor.id | String | ID of the actor. | +| Okta.Logs.Events.client.userAgent.rawUserAgent | String | A raw string representation of user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. | +| Okta.Logs.Events.client.userAgent.os | String | The operating system on which the client runs. For example, Microsoft Windows 10. | +| Okta.Logs.Events.client.userAgent.browser | String | Identifies the type of web browser, if relevant. For example, Chrome. | +| Okta.Logs.Events.client.device | String | Type of device from which the client operated. For example, Computer. | +| Okta.Logs.Events.client.id | String | For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. | +| Okta.Logs.Events.client.ipAddress | String | IP address from which the client made its request. | +| Okta.Logs.Events.client.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.client.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.client.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.displayMessage | String | The display message for an event. | +| Okta.Logs.Events.eventType | String | Type of event that was published. | +| Okta.Logs.Events.outcome.result | String | Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". | +| Okta.Logs.Events.outcome.reason | String | Reason for the result. For example, INVALID_CREDENTIALS. | +| Okta.Logs.Events.published | String | Timestamp when the event was published. | +| Okta.Logs.Events.severity | String | The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". | +| Okta.Logs.Events.securityContext.asNumber | Number | Autonomous system number associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.asOrg | String | Organization associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.isp | String | Internet service provider used to send the event's request. | +| Okta.Logs.Events.securityContext.domain | String | The domain name associated with the IP address of the inbound event request. | +| Okta.Logs.Events.securityContext.isProxy | String | Specifies whether an event's request is from a known proxy. | +| Okta.Logs.Events.request.ipChain.IP | String | IP address. | +| Okta.Logs.Events.request.ipChain.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.request.ipChain.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.request.ipChain.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.request.ipChain.source | String | Details regarding the source. | +| Okta.Logs.Events.target.id | String | ID of a target. | +| Okta.Logs.Events.target.type | String | Type of a target. | +| Okta.Logs.Events.target.alternateId | String | Alternative ID of a target. | +| Okta.Logs.Events.target.displayName | String | Display name of a target. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-logs filter=`actor.id eq "00u66lckvpjidYi0h7"` query=Boardman since="2020-03-03T20:23:17.573Z" limit=1``` + +##### Context Example + +``` +{ + "Okta": { + "Logs": { + "Events": { + "actor": { + "alternateId": "Test@demisto.com", + "detailEntry": null, + "displayName": "Test Demisto", + "id": "00u66lvd7lpjidYi0h7", + "type": "User" + }, + "authenticationContext": { + "authenticationProvider": null, + "authenticationStep": 0, + "credentialProvider": null, + "credentialType": null, + "externalSessionId": "trs3hs_F_UQT9K5FOPG7m4i1g", + "interface": null, + "issuer": null + }, + "client": { + "device": "Unknown", + "geographicalContext": { + "city": "Boardman", + "country": "United States", + "geolocation": { + "lat": 45.8491, + "lon": -119.7143 + }, + "postalCode": "97818", + "state": "Oregon" + }, + "id": null, + "ipAddress": "127.0.0.1", + "userAgent": { + "browser": "UNKNOWN", + "os": "Unknown", + "rawUserAgent": "Demisto/1.0" + }, + "zone": "null" + }, + "debugContext": { + "debugData": { + "requestId": "Xl68tch@7iZvNo0k8vPc5gAAAmE", + "requestUri": "/api/v1/groups/00g8mo0l5wuTxmoIC0h7/users/00uptu0jj9V91p5QM0h7", + "threatSuspected": "false", + "url": "/api/v1/groups/00g8mo0l5wuTxmoIC0h7/users/00uptu0jj9V91p5QM0h7?" + } + }, + "displayMessage": "Remove user from group membership", + "eventType": "group.user_membership.remove", + "legacyEventType": "core.user_group_member.user_remove", + "outcome": { + "reason": null, + "result": "SUCCESS" + }, + "published": "2020-03-03T20:23:17.573Z", + "request": { + "ipChain": [ + { + "geographicalContext": { + "city": "Boardman", + "country": "United States", + "geolocation": { + "lat": 45.8491, + "lon": -119.7143 + }, + "postalCode": "97818", + "state": "Oregon" + }, + "ip": "127.0.0.1", + "source": null, + "version": "V4" + } + ] + }, + "securityContext": { + "asNumber": null, + "asOrg": null, + "domain": null, + "isProxy": null, + "isp": null + }, + "severity": "INFO", + "target": [ + { + "alternateId": "test@this.com", + "detailEntry": null, + "displayName": "test this", + "id": "00uptu0jj9V91p5QM0h7", + "type": "User" + }, + { + "alternateId": "unknown", + "detailEntry": null, + "displayName": "test1", + "id": "00g8mo0l5wuTxmoIC0h7", + "type": "UserGroup" + } + ], + "transaction": { + "detail": {}, + "id": "Xl68tch@7iZvNo0k8vPc5gAAAmE", + "type": "WEB" + }, + "uuid": "d14117f9-5d8c-11ea-a9cb-1f2fbd3b03f7", + "version": "0" + } + } + } +} +``` + +##### Human Readable Output +> +>### Okta Events +> +>|Actor|ActorAlternaneId|ChainIP|Client|EventInfo|EventOutcome|EventSeverity|RequestIP|Targets|Time| +>|---|---|---|---|---|---|---|---|---|---| +>| Test Demisto (User) | Test@demisto.com | 127.0.0.1 | Unknown browser on Unknown OS Unknown device | Remove user from group membership | SUCCESS | INFO | 127.0.0.1 | test this (User) test1 (UserGroup) | 03/03/2020, 20:23:17 | + +### okta-get-group-assignments + +*** +Gets events for when a user was added to a group. + +#### Base Command + +`okta-get-group-assignments` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| since | Filters the lower time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. | Optional | +| until | Filters the upper time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. | Optional | +| sortOrder | The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". Possible values are: ASCENDING, DESCENDING. Default is ASCENDING. | Optional | +| limit | The maximum number of results to return. The default is 100. Default is 100. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Logs.Events.actor.alternateId | String | Alternative ID of the actor. | +| Okta.Logs.Events.actor.displayName | String | Display name of the actor. | +| Okta.Logs.Event.actor.id | String | ID of the actor. | +| Okta.Logs.Events.client.userAgent.rawUserAgent | String | A raw string representation of user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. | +| Okta.Logs.Events.client.userAgent.os | String | The operating system on which the client runs. For example, Microsoft Windows 10. | +| Okta.Logs.Events.client.userAgent.browser | String | Identifies the type of web browser, if relevant. For example, Chrome. | +| Okta.Logs.Events.client.device | String | Type of device from which the client operated. For example, Computer. | +| Okta.Logs.Events.client.id | String | For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. | +| Okta.Logs.Events.client.ipAddress | String | IP address from which the client made its request. | +| Okta.Logs.Events.client.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.client.geographicalContext.state | String | Full name of the state or province encompassing in the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.client.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.displayMessage | String | The display message for an event. | +| Okta.Logs.Events.eventType | String | Type of event that was published. | +| Okta.Logs.Events.outcome.result | String | Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". | +| Okta.Logs.Events.outcome.reason | Unknown | Reason for the result. For example INVALID_CREDENTIALS. | +| Okta.Logs.Events.published | String | Timestamp when the event was published. | +| Okta.Logs.Events.severity | String | The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". | +| Okta.Logs.Events.securityContext.asNumber | Number | Autonomous system number associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.asOrg | String | Organization associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.isp | String | Internet service provider used to send the event's request. | +| Okta.Logs.Events.securityContext.domain | String | The domain name associated with the IP address of the inbound event request. | +| Okta.Logs.Events.securityContext.isProxy | String | Specifies whether an event's request is from a known proxy. | +| Okta.Logs.Events.request.ipChain.IP | String | IP address. | +| Okta.Logs.Events.request.ipChain.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.request.ipChain.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.request.ipChain.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.request.ipChain.source | String | Details regarding the source. | +| Okta.Logs.Events.target.id | String | ID of a target. | +| Okta.Logs.Events.target.type | String | Target type. | +| Okta.Logs.Events.target.alternateId | String | Alternative ID of a target. | +| Okta.Logs.Events.target.displayName | String | Display name of a target. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-group-assignments since="2019-04-30T00:00:00.000Z" limit=1``` + +##### Context Example + +``` +{ + "Okta": { + "Logs": { + "Events": { + "actor": { + "alternateId": "Test@demisto.com", + "detailEntry": null, + "displayName": "Test Demisto", + "id": "00u66lckd7lpjidYi0h7", + "type": "User" + }, + "authenticationContext": { + "authenticationProvider": null, + "authenticationStep": 0, + "credentialProvider": null, + "credentialType": null, + "externalSessionId": "trs4IvlVrvVR9G8RPsPtFjwBA", + "interface": null, + "issuer": null + }, + "client": { + "device": "Unknown", + "geographicalContext": { + "city": "Boardman", + "country": "United States", + "geolocation": { + "lat": 45.8491, + "lon": -119.7143 + }, + "postalCode": "97818", + "state": "Oregon" + }, + "id": null, + "ipAddress": "127.0.0.1", + "userAgent": { + "browser": "UNKNOWN", + "os": "Unknown", + "rawUserAgent": "Demisto/1.0" + }, + "zone": "null" + }, + "debugContext": { + "debugData": { + "requestId": "XXxTqUauHPkBXo4-TEcw9QAAAq0", + "requestUri": "/api/v1/groups/00g8mo0l5wuTxmoIC0h7/users/00ued6gq9jItNhAsN0h7", + "url": "/api/v1/groups/00g8mo0l5wuTxmoIC0h7/users/00ued6gq9jItNhAsN0h7?" + } + }, + "displayMessage": "Add user to group membership", + "eventType": "group.user_membership.add", + "legacyEventType": "core.user_group_member.user_add", + "outcome": { + "reason": null, + "result": "SUCCESS" + }, + "published": "2019-09-14T02:42:49.379Z", + "request": { + "ipChain": [ + { + "geographicalContext": { + "city": "Boardman", + "country": "United States", + "geolocation": { + "lat": 45.8491, + "lon": -119.7143 + }, + "postalCode": "97818", + "state": "Oregon" + }, + "ip": "127.0.0.1", + "source": null, + "version": "V4" + } + ] + }, + "securityContext": { + "asNumber": null, + "asOrg": null, + "domain": null, + "isProxy": null, + "isp": null + }, + "severity": "INFO", + "target": [ + { + "alternateId": "test@this.com", + "detailEntry": null, + "displayName": "test this", + "id": "00ued6gq9jItNhAsN0h7", + "type": "User" + }, + { + "alternateId": "unknown", + "detailEntry": null, + "displayName": "test1", + "id": "00g8mo0l5wuTxmoIC0h7", + "type": "UserGroup" + } + ], + "transaction": { + "detail": {}, + "id": "XXxTqUauHPkBXo4-TEcw9QAAAq0", + "type": "WEB" + }, + "uuid": "5741ef53-d699-11e9-a08c-d549acc8afb2", + "version": "0" + } + } + } +} +``` + +##### Human Readable Output +> +>### Group Assignment Events +> +>|Actor|ActorAlternaneId|ChainIP|Client|EventInfo|EventOutcome|EventSeverity|RequestIP|Targets|Time| +>|---|---|---|---|---|---|---|---|---|---| +>| Test Demisto (User) | Test@demisto.com | 127.0.0.1 | Unknown browser on Unknown OS Unknown device | Add user to group membership | SUCCESS | INFO | 127.0.0.1 | test this (User) test1 (UserGroup) | 09/29/2019, 03:47:46 | + +### okta-get-application-assignments + +*** +Returns events for when a user was assigned to an application. + +#### Base Command + +`okta-get-application-assignments` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| since | Filters the lower time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. | Optional | +| until | Filters the upper time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. | Optional | +| sortOrder | The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". Possible values are: ASCENDING, DESCENDING. Default is ASCENDING. | Optional | +| limit | The maximum number of results to return. The default is 100. Default is 100. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Logs.Events.actor.alternateId | String | Alternative ID of the actor. | +| Okta.Logs.Events.actor.displayName | String | Display name of the actor. | +| Okta.Logs.Event.actor.id | String | ID of the actor. | +| Okta.Logs.Events.client.userAgent.rawUserAgent | String | A raw string representation of the user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. | +| Okta.Logs.Events.client.userAgent.os | String | The OS on which the client runs. For example, Microsoft Windows 10. | +| Okta.Logs.Events.client.userAgent.browser | String | Identifies the type of web browser, if relevant. For example, Chrome. | +| Okta.Logs.Events.client.device | String | Type of device from which the client operated. For example, Computer. | +| Okta.Logs.Events.client.id | String | For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. | +| Okta.Logs.Events.client.ipAddress | String | IP address from which the client made its request. | +| Okta.Logs.Events.client.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.client.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.client.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.displayMessage | String | The display message for an event. | +| Okta.Logs.Events.eventType | String | Type of event that was published. | +| Okta.Logs.Events.outcome.result | String | Result of the action. For example, "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". | +| Okta.Logs.Events.outcome.reason | String | Reason for the result. For example INVALID_CREDENTIALS. | +| Okta.Logs.Events.published | String | Timestamp when the event was published. | +| Okta.Logs.Events.severity | String | The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". | +| Okta.Logs.Events.securityContext.asNumber | Number | Autonomous system number associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.asOrg | String | Organization associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.isp | String | Internet service provider used to send the event's request. | +| Okta.Logs.Events.securityContext.domain | String | The domain name associated with the IP address of the inbound event request. | +| Okta.Logs.Events.securityContext.isProxy | String | Specifies whether an event's request is from a known proxy. | +| Okta.Logs.Events.request.ipChain.IP | String | IP address. | +| Okta.Logs.Events.request.ipChain.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.request.ipChain.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.request.ipChain.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.request.ipChain.source | String | Details regarding the source. | +| Okta.Logs.Events.target.id | String | ID of a target. | +| Okta.Logs.Events.target.type | String | Type of a target. | +| Okta.Logs.Events.target.alternateId | String | Alternative ID of a target. | +| Okta.Logs.Events.target.displayName | String | Display name of a target. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-application-assignments since="2019-04-30T00:00:00.000Z" until="2020-02-30T00:00:00.000Z" sortOrder=DESCENDING limit=1``` + +##### Context Example + +``` +{ + "Okta": { + "Logs": { + "Events": { + "actor": { + "alternateId": "Test@demisto.com", + "detailEntry": null, + "displayName": "Test Demisto", + "id": "00u66lckd7lpjidYi0h7", + "type": "User" + }, + "authenticationContext": { + "authenticationProvider": null, + "authenticationStep": 0, + "credentialProvider": null, + "credentialType": null, + "externalSessionId": "trsFSV6XXY4TMCSB_xzJrZ85A", + "interface": null, + "issuer": null + }, + "client": { + "device": "Unknown", + "geographicalContext": { + "city": "Boardman", + "country": "United States", + "geolocation": { + "lat": 45.8491, + "lon": -119.7143 + }, + "postalCode": "97818", + "state": "Oregon" + }, + "id": null, + "ipAddress": "127.0.0.1", + "userAgent": { + "browser": "UNKNOWN", + "os": "Unknown", + "rawUserAgent": "python-requests/2.22.0" + }, + "zone": "null" + }, + "debugContext": { + "debugData": { + "requestId": "XlgCgAEBMJsNo5Yh9rHtZAAACPw", + "requestUri": "/api/v1/users/00upywm7l0rL1V0zt0h7/lifecycle/activate", + "threatSuspected": "false", + "url": "/api/v1/users/00upywm7l0rL1V0zt0h7/lifecycle/activate?" + } + }, + "displayMessage": "Add user to application membership", + "eventType": "application.user_membership.add", + "legacyEventType": "app.generic.provision.assign_user_to_app", + "outcome": { + "reason": null, + "result": "SUCCESS" + }, + "published": "2020-02-27T17:55:12.949Z", + "request": { + "ipChain": [ + { + "geographicalContext": { + "city": "Boardman", + "country": "United States", + "geolocation": { + "lat": 45.8491, + "lon": -119.7143 + }, + "postalCode": "97818", + "state": "Oregon" + }, + "ip": "127.0.0.1", + "source": null, + "version": "V4" + } + ] + }, + "securityContext": { + "asNumber": null, + "asOrg": null, + "domain": null, + "isProxy": null, + "isp": null + }, + "severity": "INFO", + "target": [ + { + "alternateId": "Test1@test.com", + "detailEntry": null, + "displayName": "Test 1 that", + "id": "0uapywj3yxcjhpjSQ0h7", + "type": "AppUser" + }, + { + "alternateId": "ShrikSAML", + "detailEntry": null, + "displayName": "ShrikSAML", + "id": "0oabe0e2jruaQccDf0h7", + "type": "AppInstance" + }, + { + "alternateId": "Test1@test.com", + "detailEntry": null, + "displayName": "Test 1 that", + "id": "00upywm7l0rL1V0zt0h7", + "type": "User" + } + ], + "transaction": { + "detail": {}, + "id": "XlgCgAEBMJsNo5Yh9rHtZAAACPw", + "type": "WEB" + }, + "uuid": "4d8a4e9a-598a-11ea-a594-b9fb637659a5", + "version": "0" + } + } + } +} +``` + +##### Human Readable Output +> +>### Application Assignment Events +> +>|Actor|ActorAlternaneId|ChainIP|Client|EventInfo|EventOutcome|EventSeverity|RequestIP|Targets|Time| +>|---|---|---|---|---|---|---|---|---|---| +>| Test Demisto (User) | Test@demisto.com | 127.0.0.1 | Unknown browser on Unknown OS Unknown device | Add user to application membership | SUCCESS | INFO | 127.0.0.1 | Test 1 that (AppUser) ShrikSAML (AppInstance) Test 1 that (User) | 02/27/2020, 17:55:12 | + +### okta-get-application-authentication + +*** +Returns logs using specified filters. + +#### Base Command + +`okta-get-application-authentication` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| since | Filters the lower time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. | Optional | +| until | Filters the upper time bound of the log event in the Internet Date\Time format profile of ISO 8601. For example, 2020-02-14T16:00:18Z. | Optional | +| sortOrder | The order of the returned events. Can be "ASCENDING" or "DESCENDING". The default is "ASCENDING". Possible values are: ASCENDING, DESCENDING. Default is ASCENDING. | Optional | +| limit | The maximum number of results to return. The default is 100. Default is 100. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Logs.Events.actor.alternateId | String | Alternative ID of the actor. | +| Okta.Logs.Events.actor.displayName | String | Display name of the actor. | +| Okta.Logs.Events.actor.id | String | ID of the actor. | +| Okta.Logs.Events.client.userAgent.rawUserAgent | String | A raw string representation of user agent, formatted according to section 5.5.3 of HTTP/1.1 Semantics and Content. Both the browser and the OS fields can be derived from this field. | +| Okta.Logs.Events.client.userAgent.os | String | The operating system on which the client runs. For example, Microsoft Windows 10. | +| Okta.Logs.Events.client.userAgent.browser | String | Identifies the type of web browser, if relevant. For example, Chrome. | +| Okta.Logs.Events.client.device | String | Type of device from which the client operated. For example, Computer. | +| Okta.Logs.Events.client.id | String | For OAuth requests, the ID of the OAuth client making the request. For SSWS token requests, the ID of the agent making the request. | +| Okta.Logs.Events.client.ipAddress | String | IP address from which the client made its request. | +| Okta.Logs.Events.client.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.client.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.client.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.displayMessage | String | The display message for an event. | +| Okta.Logs.Events.eventType | String | Type of event that was published. | +| Okta.Logs.Events.outcome.result | String | Result of the action. Can be "SUCCESS", "FAILURE", "SKIPPED", or "UNKNOWN". | +| Okta.Logs.Events.outcome.reason | String | Reason for the result. For example INVALID_CREDENTIALS. | +| Okta.Logs.Events.published | String | Timestamp when the event was published. | +| Okta.Logs.Events.severity | String | The event severity. Can be "DEBUG", "INFO", "WARN", or "ERROR". | +| Okta.Logs.Events.securityContext.asNumber | Number | Autonomous system number associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.asOrg | String | Organization associated with the autonomous system that the event request was sourced to. | +| Okta.Logs.Events.securityContext.isp | String | Internet service provider used to send the event's request. | +| Okta.Logs.Events.securityContext.domain | String | The domain name associated with the IP address of the inbound event request. | +| Okta.Logs.Events.securityContext.isProxy | String | Specifies whether an event's request is from a known proxy. | +| Okta.Logs.Events.request.ipChain.IP | String | IP address. | +| Okta.Logs.Events.request.ipChain.geographicalContext.city | String | The city encompassing the area containing the geo-location coordinates, if available. For example, Seattle, San Francisco. | +| Okta.Logs.Events.request.ipChain.geographicalContext.state | String | Full name of the state or province encompassing the area containing the geo-location coordinates. For example, Montana, Incheon. | +| Okta.Logs.Events.request.ipChain.geographicalContext.country | String | Full name of the country encompassing the area containing the geo-location coordinates. For example, France, Uganda. | +| Okta.Logs.Events.request.ipChain.source | String | Details regarding the source. | +| Okta.Logs.Events.target.id | String | ID of a target. | +| Okta.Logs.Events.target.type | String | Type of a target. | +| Okta.Logs.Events.target.alternateId | String | Alternative ID of a target. | +| Okta.Logs.Events.target.displayName | String | Display name of a target. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-get-application-authentication since="2019-04-30T00:00:00.000Z" until="2020-02-30T00:00:00.000Z" limit=1``` + +##### Context Example + +``` +{ + "Okta": { + "Logs": { + "Events": { + "actor": { + "alternateId": "Test@demisto.com", + "detailEntry": null, + "displayName": "Test Demisto", + "id": "00u66lckd7lpjidYi0h7", + "type": "User" + }, + "authenticationContext": { + "authenticationProvider": null, + "authenticationStep": 0, + "credentialProvider": null, + "credentialType": null, + "externalSessionId": "102ejbJMP0RSE2zwIOhr_PpHA", + "interface": null, + "issuer": null + }, + "client": { + "device": "Computer", + "geographicalContext": { + "city": "Tel Aviv", + "country": "Israel", + "geolocation": { + "lat": 32.0678, + "lon": 34.7647 + }, + "postalCode": null, + "state": "Tel Aviv" + }, + "id": null, + "ipAddress": "127.0.0.1", + "userAgent": { + "browser": "CHROME", + "os": "Mac OS X", + "rawUserAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36" + }, + "zone": "null" + }, + "debugContext": { + "debugData": { + "authnRequestId": "XYI-PiWDTYEsVNLm7sc0vwAACi0", + "initiationType": "SP_INITIATED", + "requestId": "XYI-PzNoI1UMTtFvio-9LAAACAc", + "requestUri": "/app/demistodev725178_benzi_1/exkm30ffmuhcL0rFv0h7/sso/saml", + "signOnMode": "SAML 2.0", + "url": "/app/demistodev725178_benzi_1/exkm30ffmuhcL0rFv0h7/sso/saml?RelayState=&SAMLRequest=PHNhbWxwOkF1dGhuUmVxdWVzdCB4bWxuczpzYW1scD0idXJuOm9hc2lzOm5hbWVzOnRjOlNBTUw6Mi4wOnByb3RvY29sIiB4bWxuczpzYW1sPSJ1cm46b2FzaXM6bmFtZXM6dGM6U0FNTDoyLjA6YXNzZXJ0aW9uIiBJRD0iXzRiODZjMGUzLWI0NzktNGU2NS00ZjIwLWJiNTE0YTc3NTUyMiIgVmVyc2lvbj0iMi4wIiBQcm90b2NvbEJpbmRpbmc9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDpiaW5kaW5nczpIVFRQLVBPU1QiIEFzc2VydGlvbkNvbnN1bWVyU2VydmljZVVSTD0iaHR0cHM6Ly9lYzItNTItNDgtMTItMTIzLmV1LXdlc3QtMS5jb21wdXRlLmFtYXpvbmF3cy5jb20vc2FtbCIgSXNzdWVJbnN0YW50PSIyMDE5LTA5LTE4VDE0OjIyOjM3WiI%2BPHNhbWw6SXNzdWVyPmh0dHBzOi8vZGV2LTcyNTE3OC5va3RhcHJldmlldy5jb20vYXBwL2V4a20zMGZmbXVoY0wwckZ2MGg3L3Nzby9zYW1sL21ldGFkYXRhPC9zYW1sOklzc3Vlcj48c2FtbHA6TmFtZUlEUG9saWN5IEFsbG93Q3JlYXRlPSJ0cnVlIiBGb3JtYXQ9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDpuYW1laWQtZm9ybWF0OnBlcnNpc3RlbnQiPjwvc2FtbHA6TmFtZUlEUG9saWN5Pjwvc2FtbHA6QXV0aG5SZXF1ZXN0Pg%3D%3D&fromLoginToken=OODncZp5PTgZ-8QwGMHgbJ6psmQES6dUWiMohfLGGm7GWKko8LFXHz3faG7ZkoocPX2ixv-dyUSOF7qJ9DZDVoVkeETM7n6MskWQ01woQFqUcUVdM1xDBmplZlK1DMhX6ozpZQV2XNK073FfDt4bASBEFFgGkFi5ygH-LmBFSfcoiLWHM5MGJ-JEUB97peJxOL41inNX2r333FMJoDzem0dLAOf4cfApoVz7VDdY06r8i6Lt0vuxmxKyZRWJkCHroKyc3ysag9gbUMR5tSoe3hRPJvCBozjYtzkpVlLBP_6V01eGL2YVP8JR2rkpI8MvBYNDLIoJry1e_eBOF3kzJA" + } + }, + "displayMessage": "User single sign on to app", + "eventType": "user.authentication.sso", + "legacyEventType": "app.auth.sso", + "outcome": { + "reason": null, + "result": "SUCCESS" + }, + "published": "2019-09-18T14:29:19.329Z", + "request": { + "ipChain": [ + { + "geographicalContext": { + "city": "Tel Aviv", + "country": "Israel", + "geolocation": { + "lat": 32.0678, + "lon": 34.7647 + }, + "postalCode": null, + "state": "Tel Aviv" + }, + "ip": "127.0.0.1", + "source": null, + "version": "V4" + } + ] + }, + "securityContext": { + "asNumber": null, + "asOrg": null, + "domain": null, + "isProxy": null, + "isp": null + }, + "severity": "INFO", + "target": [ + { + "alternateId": "benzi_master", + "detailEntry": { + "signOnModeType": "SAML_2_0" + }, + "displayName": "benzi_master", + "id": "0oam30ffmvM5cLzxo0h7", + "type": "AppInstance" + }, + { + "alternateId": "Test@demisto.com", + "detailEntry": null, + "displayName": "Test Demisto", + "id": "0uam30nfffJqV3I4M0h7", + "type": "AppUser" + } + ], + "transaction": { + "detail": {}, + "id": "XYI-PzNoI1UMTtFvio-9LAAACAc", + "type": "WEB" + }, + "uuid": "b349fd35-da20-11e9-81c0-95908eb13131", + "version": "0" + } + } + } +} +``` + +##### Human Readable Output +> +>### Application Authentication Events +> +>|Actor|ActorAlternaneId|ChainIP|Client|EventInfo|EventOutcome|EventSeverity|RequestIP|Targets|Time| +>|---|---|---|---|---|---|---|---|---|---| +>| Test Demisto (User) | Test@demisto.com | 127.0.0.1 | CHROME on Mac OS X Computer | User single sign on to app | SUCCESS | INFO | 127.0.0.1 | BenziPermanent (AppInstance) Test Demisto (AppUser) | 10/14/2019, 12:16:53 | + +### okta-delete-user + +*** +Deletes the specified user. + +#### Base Command + +`okta-delete-user` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| userId | Okta User ID. | Optional | +| username | Username of the user. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-delete-user username=testForDocs@test.com``` + +##### Human Readable Output +> +>User: testForDocs@test.com was Deleted successfully + +### okta-clear-user-sessions + +*** +Removes all active identity provider sessions. This forces the user to authenticate upon the next operation. By default, OpenID Connect and OAuth refresh and access tokens issued to the user are revoked. Token revocation can be disabled if needed. +For more information and examples: +https://developer.okta.com/docs/reference/api/users/#user-sessions + +#### Base Command + +`okta-clear-user-sessions` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| userId | Okta User ID. | Required | +| revokeOauthTokens | When true, revokes OpenID Connect and OAuth refresh and access tokens issued to the user. Possible values are: true, false. Default is true. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +##### Command Example + +```!okta-clear-user-sessions userId=00ui5brmwtJpMdoZZ0h7``` + +##### Human Readable Output +> +>### User session was cleared for: 00ui5brmwtJpMdoZZ0h7 + +### okta-list-zones + +*** +Get an Okta Zone object. + +#### Base Command + +`okta-list-zones` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| limit | The maximum number of results to return. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Zone.created | Date | Zone creation timestamp, in the format 2020-04-06T22:23:12.000Z. | +| Okta.Zone.gateways.type | String | Gateways IP entry type, e.g., CIDR. | +| Okta.Zone.gateways.value | String | Gateways IP entry value, e.g., 34.103.1.108/32. | +| Okta.Zone.id | String | Zone ID, e.g., nzoqsmcx1qWYJ6wYF0h7. | +| Okta.Zone.lastUpdated | Date | Zone last update timestamp, e.g., 2020-04-06T22:23:12.000Z. | +| Okta.Zone.name | String | Zone name. | +| Okta.Zone.proxies.type | String | Proxies IP entry type e.g. CIDR. | +| Okta.Zone.proxies.value | Unknown | Proxies IP entry value, e.g., 34.103.1.108/32. | +| Okta.Zone.status | String | Zone status, e.g., ACTIVE. | +| Okta.Zone.system | Number | True if this is a system zone, false if user-created. | +| Okta.Zone.type | String | Zone type, e.g., IP. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +#### Command Example + +```!okta-list-zones``` + +#### Context Example + +``` +{ + "Okta": { + "Zone": [ + { + "_links": { + "deactivate": { + "hints": { + "allow": [ + "POST" + ] + }, + "href": "https://dev-950355.oktapreview.com/api/v1/zones/nzo9rbw8evGOFV1VE0h7/lifecycle/deactivate" + }, + "self": { + "hints": { + "allow": [ + "GET", + "PUT", + "DELETE" + ] + }, + "href": "https://dev-950355.oktapreview.com/api/v1/zones/nzo9rbw8evGOFV1VE0h7" + } + }, + "created": "2017-03-03T22:05:24.000Z", + "gateways": [ + { + "type": "CIDR", + "value": "2.2.2.2/32" + } + ], + "id": "nzo9rbw8evGOFV1VE0h7", + "lastUpdated": "2020-04-23T08:58:55.000Z", + "name": "LegacyIpZone", + "proxies": null, + "status": "ACTIVE", + "system": true, + "type": "IP" + }, + { + "_links": { + "deactivate": { + "hints": { + "allow": [ + "POST" + ] + }, + "href": "https://dev-950355.oktapreview.com/api/v1/zones/nzoqsmcx1qWYJ6wY33h7/lifecycle/deactivate" + }, + "self": { + "hints": { + "allow": [ + "GET", + "PUT", + "DELETE" + ] + }, + "href": "https://dev-950355.oktapreview.com/api/v1/zones/nzoqsmcx1qWYJ6wY33h7" + } + }, + "created": "2020-04-06T22:23:12.000Z", + "gateways": [ + { + "type": "CIDR", + "value": "1.1.1.2/32" + }, + { + "type": "CIDR", + "value": "1.1.1.3/32" + }, + { + "type": "CIDR", + "value": "2.2.2.2/32" + }, + { + "type": "CIDR", + "value": "2.2.2.3/32" + } + ], + "id": "nzoqsmcx1qWYJ6wY33h7", + "lastUpdated": "2020-06-05T08:57:57.000Z", + "name": "MyZone", + "proxies": null, + "status": "ACTIVE", + "system": false, + "type": "IP" + } + ] + } +} +``` + +#### Human Readable Output +> +>### Okta Zones +> +>|name|id|gateways|status|system|lastUpdated|created| +>|---|---|---|---|---|---|---| +>| LegacyIpZone | nzo9rbw8evGOFV1VE0h7 | {'type': 'CIDR', 'value': '2.2.2.2/32'} | ACTIVE | true | 2020-04-23T08:58:55.000Z | 2017-03-03T22:05:24.000Z | +>| MyZone | nzoqsmcx1qWYJ6wY33h7 | {'type': 'CIDR', 'value': '3.3.3.4/32'},
{'type': 'CIDR', 'value': '5.5.5.3/32'},
{'type': 'CIDR', 'value': '3.3.3.1/32'},
{'type': 'CIDR', 'value': '2.2.2.3/32'} | ACTIVE | false | 2020-06-05T08:57:57.000Z | 2020-04-06T22:23:12.000Z | + +### okta-update-zone + +*** +Update an Okta Zone. + +#### Base Command + +`okta-update-zone` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- |---------------------------------------------------------------------------------------------------------------------------------------------------------| --- | +| zoneID | Zone ID to update, e.g., nzoqsmcx1qWYJ6wYF0h7. | Required | +| zoneName | Updates the zone name. | Optional | +| gatewayIPs | Updates Gateway IP addresses: CIDR range (1.1.0.0/16) or single IP address (2.2.2.2). | Optional | +| proxyIPs | Update Proxy IP addresses: CIDR range (1.1.0.0/16) or single IP address (2.2.2.2). | Optional | +| updateType | Indicate the action of adding an IP to an existing zone or overriding the existing IPs. Possible values are: "APPEND", "OVERRIDE". Default is "OVERRIDE". | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Zone.created | Date | Zone creation timestamp, e.g., 2020-04-06T22:23:12.000Z. | +| Okta.Zone.gateways.type | String | Gateways IP entry type, e.g., CIDR. | +| Okta.Zone.gateways.value | String | Gateways IP entry value, e.g., 34.103.1.108/32. | +| Okta.Zone.id | String | Okta Zone ID, e.g., nzoqsmcx1qWYJ6wYF0h7. | +| Okta.Zone.lastUpdated | Date | Zone last update timestamp, in the format 2020-04-06T22:23:12.000Z. | +| Okta.Zone.name | String | Zone name. | +| Okta.Zone.proxies.type | String | Proxies IP entry type, e.g., CIDR. | +| Okta.Zone.proxies.value | Unknown | Proxies IP entry value, e.g., 34.103.1.108/32. | +| Okta.Zone.status | String | Zone status, e.g., ACTIVE. | +| Okta.Zone.system | Number | True if this is a system zone, false if user-created. | +| Okta.Zone.type | String | Zone type, e.g., IP. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +#### Command Example + +```!okta-update-zone zoneID=nzoqsmcx1qWYJ6wY33h7 zoneName=MyZone``` + +#### Context Example + +``` +{ + "Okta": { + "Zone": { + "_links": { + "deactivate": { + "hints": { + "allow": [ + "POST" + ] + }, + "href": "https://dev-950355.oktapreview.com/api/v1/zones/nzoqsmcx1qWYJ6wY33h7/lifecycle/deactivate" + }, + "self": { + "hints": { + "allow": [ + "GET", + "PUT", + "DELETE" + ] + }, + "href": "https://dev-950355.oktapreview.com/api/v1/zones/nzoqsmcx1qWYJ6wY33h7" + } + }, + "created": "2020-04-06T22:23:12.000Z", + "gateways": [ + { + "type": "CIDR", + "value": "1.1.3.5/32" + }, + { + "type": "CIDR", + "value": "5.3.143.103/32" + }, + { + "type": "CIDR", + "value": "5.3.246.228/32" + }, + { + "type": "CIDR", + "value": "5.3.246.229/32" + } + ], + "id": "nzoqsmcx1qWYJ6wY33h7", + "lastUpdated": "2020-06-05T08:57:57.000Z", + "name": "MyZone", + "proxies": null, + "status": "ACTIVE", + "system": false, + "type": "IP" + } + } +} +``` + +#### Human Readable Output + +>### Okta Zones +> +>|name|id|gateways|status|system|lastUpdated|created| +>|---|---|---|---|---|---|---| +>| MyZone | nzoqsmcx1qWYJ6wY33h7 | {'type': 'CIDR', 'value': '1.3.1.5/32'},
{'type': 'CIDR', 'value': '1.3.1.5/32'},
{'type': 'CIDR', 'value': '1.3.1.5/32'},
{'type': 'CIDR', 'value': '1.3.1.5/32'} | ACTIVE | false | 2020-06-05T08:57:57.000Z | 2020-04-06T22:23:12.000Z | + +### okta-get-zone + +*** +Get a Zone by its ID. + +#### Base Command + +`okta-get-zone` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| zoneID | Zone ID to get, e.g., nzoqsmcx1qWYJ6wYF0h.7. | Required | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Zone.created | Date | Zone creation timestamp, in the format 2020-04-06T22:23:12.000Z. | +| Okta.Zone.gateways.type | String | Gateways IP entry type, e.g., CIDR. | +| Okta.Zone.gateways.value | String | Gateways IP entry value, e.g., 34.103.1.108/32. | +| Okta.Zone.id | String | Okta Zone ID, e.g., nzoqsmcx1qWYJ6wYF0h7. | +| Okta.Zone.lastUpdated | Date | Zone last update timestamp, in the format 2020-04-06T22:23:12.000Z. | +| Okta.Zone.name | String | Zone name. | +| Okta.Zone.proxies.type | String | Proxies IP entry type, e.g., CIDR. | +| Okta.Zone.proxies.value | Unknown | Proxies IP entry value, e.g., 34.103.1.108/32. | +| Okta.Zone.status | String | Zone status, e.g,. ACTIVE. | +| Okta.Zone.system | Number | True if this is a system zone, false if user-created. | +| Okta.Zone.type | String | Zone type, e.g., IP. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +#### Command Example + +```!okta-get-zone zoneID=nzoqsmcx1qWYJ6wY33h7``` + +#### Context Example + +``` +{ + "Okta": { + "Zone": { + "_links": { + "deactivate": { + "hints": { + "allow": [ + "POST" + ] + }, + "href": "https://dev-950355.oktapreview.com/api/v1/zones/nzoqsmcx1qWYJ6wY33h7/lifecycle/deactivate" + }, + "self": { + "hints": { + "allow": [ + "GET", + "PUT", + "DELETE" + ] + }, + "href": "https://dev-950355.oktapreview.com/api/v1/zones/nzoqsmcx1qWYJ6wY33h7" + } + }, + "created": "2020-04-06T22:23:12.000Z", + "gateways": [ + { + "type": "CIDR", + "value": "1.3.1.3/32" + }, + { + "type": "CIDR", + "value": "3.5.146.103/32" + }, + { + "type": "CIDR", + "value": "3.5.1.228/32" + }, + { + "type": "CIDR", + "value": "3.5.1.229/32" + } + ], + "id": "nzoqsmcx1qWYJ6wY33h7", + "lastUpdated": "2020-06-05T08:57:57.000Z", + "name": "MyZone", + "proxies": null, + "status": "ACTIVE", + "system": false, + "type": "IP" + } + } +} +``` + +#### Human Readable Output + +>### Okta Zones +> +>|name|id|gateways|status|system|lastUpdated|created| +>|---|---|---|---|---|---|---| +>| MyZone | nzoqsmcx1qWYJ6wY33h7 | {'type': 'CIDR', 'value': '1.3.1.3/32'},
{'type': 'CIDR', 'value': '3.5.146.103/32'},
{'type': 'CIDR', 'value': '3.5.1.228/32'},
{'type': 'CIDR', 'value': '3.5.1.229/32'} | ACTIVE | false | 2020-06-05T08:57:57.000Z | 2020-04-06T22:23:12.000Z | + +### okta-list-users + +*** +Lists users in your organization. + +#### Base Command + +`okta-list-users` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| verbose | Whether to return extended user information. Can be "true" or "false". The default is "false". Possible values are: true, false. Default is false. | Optional | +| limit | The maximum number of results to return. | Optional | +| query | Searches the name property of groups for matching values. | Optional | +| filter | Useful for performing structured queries where constraints on group attribute values can be explicitly targeted.
The following expressions are supported(among others) for groups with the filter query parameter:
type eq "OKTA_GROUP" - Groups that have a type of OKTA_GROUP; lastUpdated lt "yyyy-MM-dd''T''HH:mm:ss.SSSZ" - Groups with profile last updated before a specific timestamp; lastMembershipUpdated eq "yyyy-MM-dd''T''HH:mm:ss.SSSZ" - Groups with memberships last updated at a specific timestamp; id eq "00g1emaKYZTWRYYRRTSK" - Group with a specified ID. +For more information about filtering, visit https://developer.okta.com/docs/api/getting_started/design_principles#filtering. | Optional | +| after | The cursor in which to retrive the results from and on. If the query didn't reach the end of results, the tag will be found in the readable output under the tag key. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.ID | String | Okta account ID. | +| Account.Email | String | Okta account email. | +| Account.Username | String | Okta account username. | +| Account.DisplayName | String | Okta account display name. | +| Account.Status | String | Okta account status. | +| Account.Created | Date | Timestamp for when the user was created. | +| Account.Activated | Date | Timestamp for when the user was activated. | +| Account.StatusChanged | Date | Timestamp for when the user's status was last changed. | +| Account.PasswordChanged | Date | Timestamp for when the user's password was last changed. | +| Okta.User.tag| String | The location of the next item, used with after param. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +#### Command Example + +```!okta-list-users``` + +#### Context Example + +```json +{ + "Okta": + { + "User": + { + "tag": "test12tag" + } + }, + "Account": [ + { + "Created": "2018-07-24T20:20:04.000Z", + "DisplayName": "Dbot XSOAR", + "Email": "dbot@xsoar.com", + "ID": "XXXXXXXXX", + "Status": "STAGED", + "Type": "Okta", + "Username": "dbot@xsoar.com" + } + ] +} +``` + +#### Human Readable Output + +>### Okta users found +> +> ### Users +> +>|First Name|ID|Last Login|Last Name|Login|Mobile Phone|Status| +>|---|---|---|---|---|---|---| +>| Dbot | XXXXX | | XSOAR | dbot@xsoar.com | | STAGED | +> +> ### tag: test12tag +> +### okta-create-zone + +*** +Creates a Zone with the specified name. + +#### Base Command + +`okta-create-zone` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| name | Zone name. | Required | +| gateway_ips | Update Gateway IP addresses: CIDR range (1.1.0.0/16) or single IP address (2.2.2.2). | Optional | +| proxies | Update Proxy IP addresses: CIDR range (1.1.0.0/16) or single IP address (2.2.2.2). | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +### okta-create-group + +*** +Create a new group in Okta tenant. + +#### Base Command + +`okta-create-group` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| name | Name of the group to add. | Required | +| description | Description of the group to add. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| OktaGroup.ID | Unknown | Group ID in Okta. | +| OktaGroup.Name | Unknown | Group name in Okta. | +| OktaGroup.Description | Unknown | Group description in Okta. | +| OktaGroup.Type | Unknown | Group type in Okta. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +#### Command example + +```!okta-create-group name="TestGroup" description="TestGroup description."``` + +#### Context Example + +```json +{ + "OktaGroup": { + "Description": "TestGroup description.", + "ID": "00g3qb398kItYXzKd1d7", + "Name": "TestGroup", + "Type": "OKTA_GROUP" + } +} +``` + +#### Human Readable Output + +>Group Created: [GroupID:00g3qb398kItYXzKd1d7, GroupName: TestGroup] + +### okta-assign-group-to-app + +*** +Assign a group to an application. + +#### Base Command + +`okta-assign-group-to-app` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| groupName | Name of the group to assign to the app. | Optional | +| groupId | ID of the group to assign to the app. | Optional | +| appName | Friendly name of the app that the group will be assigned to. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +#### Command example + +```!okta-assign-group-to-app appName="Default-App" groupName="TestGroup"``` + +#### Human Readable Output + +>Group: TestGroup added to PA App successfully +> +### okta-expire-password + +*** +Expires a password for an existing Okta user. + +#### Base Command + +`okta-expire-password` + +#### Input + +| **Argument Name** | **Description** | **Required** | +| --- | --- | --- | +| username | Okta username for which to expire the password. | Required | +| temporary_password | When true, you'll need to change the password in the next login. Possible values are: true, false. Default is false. | Optional | +| revoke_session | When true, revokes the user's existing sessions. | Optional | +| hide_password | When true, prevents the password from being saved in the war room. | Optional | + +#### Context Output + +| **Path** | **Type** | **Description** | +| --- | --- | --- | +| Account.Activated | Date | Timestamp for when the user was activated. | +| Account.Created | Date | Timestamp for when the user was created. | +| Account.DisplayName | String | Okta account display name. | +| Account.Email | String | Okta account email. | +| Account.ID | String | Created Okta account ID. | +| Account.PasswordChanged | Date | Timestamp for when the user's password was last changed. | +| Account.Status | String | Okta account current status. | +| Account.StatusChanged | Date | Timestamp for when the user's status was last changed. | +| Account.Type | String | Okta account type. | +| Account.Username | String | Okta account usernames returned by the search. | +| Okta.Metadata.x-rate-limit-limit | Number | The rate limit ceiling that’s applicable for the current request. | +| Okta.Metadata.x-rate-limit-remaining | Number | The amount of requests left for the current rate-limit window. | +| Okta.Metadata.x-rate-limit-reset | Number | The time at which the rate limit resets, specified in UTC epoch time (in seconds). | + +#### Command example + +```!okta-expire-password username="4x1xh5rl@test.com" temporary_password="false"``` + +#### Context Example + +```json +{ + "Account": { + "Activated": "2022-06-20T04:48:04.000Z", + "Created": "2022-06-20T04:47:59.000Z", + "DisplayName": "Test 1 Test1", + "Email": "4x1xh5rl@test.com", + "ID": "00u19cr5qv91HjELI0h8", + "PasswordChanged": "2022-06-20T04:48:07.000Z", + "Status": "PASSWORD_EXPIRED", + "StatusChanged": "2023-09-10T12:56:04.000Z", + "Type": "Okta", + "Username": "4x1xh5rl@test.com" + } +} +``` + +#### Human Readable Output + +>### Okta Expired Password +> +>|_links|activated|created|credentials|id|lastUpdated|passwordChanged|profile|status|statusChanged|type| +>|---|---|---|---|---|---|---|---|---|---|---| +>| suspend: {"href": "https://test.oktapreview.com/api/v1/users/00u19cr5qv91HjELI0h8/lifecycle/suspend", "method": "POST"}
schema: {"href": "https://test.oktapreview.com/api/v1/meta/schemas/user/osc66lckcvDyVcGzS0h7"}
resetPassword: {"href": "https://test.oktapreview.com/api/v1/users/00u19cr5qv91HjELI0h8/lifecycle/reset_password", "method": "POST"}
forgotPassword: {"href": "https://test.oktapreview.com/api/v1/users/00u19cr5qv91HjELI0h8/credentials/forgot_password", "method": "POST"}
expirePassword: {"href": "https://test.oktapreview.com/api/v1/users/00u19cr5qv91HjELI0h8/lifecycle/expire_password", "method": "POST"}
changeRecoveryQuestion: {"href": "https://test.oktapreview.com/api/v1/users/00u19cr5qv91HjELI0h8/credentials/change_recovery_question", "method": "POST"}
self: {"href": "https://test.oktapreview.com/api/v1/users/00u19cr5qv91HjELI0h8"}
type: {"href": "https://test.oktapreview.com/api/v1/meta/types/user/oty66lckcvDyVcGzS0h7"}
changePassword: {"href": "https://test.oktapreview.com/api/v1/users/00u19cr5qv91HjELI0h8/credentials/change_password", "method": "POST"}
deactivate: {"href": "https://test.oktapreview.com/api/v1/users/00u19cr5qv91HjELI0h8/lifecycle/deactivate", "method": "POST"} | 2022-06-20T04:48:04.000Z | 2022-06-20T04:47:59.000Z | password: {}
recovery_question: {"question": "whats the first school?"}
provider: {"type": "OKTA", "name": "OKTA"} | 00u19cr5qv91HjELI0h8 | 2023-09-10T12:56:04.000Z | 2022-06-20T04:48:07.000Z | firstName: Test 1
lastName: Test1
preferredLanguage: en
mobilePhone: null
city: Tel-Aviv
displayName: Test 1 that
nickName: Testush
secondEmail: null
login: 4x1xh5rl@test.com
email: 4x1xh5rl@test.com
employeeNumber: 12345 | PASSWORD_EXPIRED | 2023-09-10T12:56:04.000Z | id: oty66lckcvDyVcGzS0h7 | + +### okta-auth-reset + +*** +Reset OAuth authentication data (authentication process will start from the beginning, and a new token will be generated). + +#### Base Command + +`okta-auth-reset` + +#### Input + +There are no input arguments for this command. + +#### Context Output + +There is no context output for this command. diff --git a/Packs/OktaStandardConnector/README.md b/Packs/OktaStandardConnector/README.md new file mode 100644 index 000000000000..1135646d0216 --- /dev/null +++ b/Packs/OktaStandardConnector/README.md @@ -0,0 +1,3 @@ +# Okta (Standard Connector) + +Satellite pack — shares logic with the Okta pack via the OktaApiModule and OktaIAMApiModule. Do not install standalone. diff --git a/Packs/OktaStandardConnector/ReleaseNotes/1_0_1.md b/Packs/OktaStandardConnector/ReleaseNotes/1_0_1.md new file mode 100644 index 000000000000..ede8f932a0c6 --- /dev/null +++ b/Packs/OktaStandardConnector/ReleaseNotes/1_0_1.md @@ -0,0 +1,3 @@ +## Okta (Standard Connector) + +- Documentation and metadata improvements. \ No newline at end of file diff --git a/Packs/OktaStandardConnector/ReleaseNotes/1_0_2.md b/Packs/OktaStandardConnector/ReleaseNotes/1_0_2.md new file mode 100644 index 000000000000..66dcc4c3a1af --- /dev/null +++ b/Packs/OktaStandardConnector/ReleaseNotes/1_0_2.md @@ -0,0 +1,9 @@ + +#### Integrations + +##### Okta (Standard Connector) + +- Documentation and metadata improvements. +##### Okta IAM (Standard Connector) + +- Documentation and metadata improvements. diff --git a/Packs/OktaStandardConnector/doc_files/gfskliw1i51ScX6pf0h7 b/Packs/OktaStandardConnector/doc_files/gfskliw1i51ScX6pf0h7 new file mode 100644 index 000000000000..3a8884cdee43 Binary files /dev/null and b/Packs/OktaStandardConnector/doc_files/gfskliw1i51ScX6pf0h7 differ diff --git a/Packs/OktaStandardConnector/doc_files/gfsnda403rf16Qe790h7 b/Packs/OktaStandardConnector/doc_files/gfsnda403rf16Qe790h7 new file mode 100644 index 000000000000..eb13b1e8189c Binary files /dev/null and b/Packs/OktaStandardConnector/doc_files/gfsnda403rf16Qe790h7 differ diff --git a/Packs/OktaStandardConnector/pack_metadata.json b/Packs/OktaStandardConnector/pack_metadata.json new file mode 100644 index 000000000000..fbbc2dfac058 --- /dev/null +++ b/Packs/OktaStandardConnector/pack_metadata.json @@ -0,0 +1,29 @@ +{ + "name": "Okta (Standard Connector)", + "description": "Satellite pack of Okta used for the Standard Connector deployment. Shares core logic with the Okta pack via the OktaApiModule and OktaIAMApiModule.", + "support": "xsoar", + "currentVersion": "1.0.2", + "serverMinVersion": "8.15.0", + "author": "Cortex XSOAR", + "url": "https://www.paloaltonetworks.com/cortex", + "email": "", + "created": "2026-09-24T00:00:00Z", + "categories": [ + "Identity and Access Management" + ], + "tags": [], + "useCases": [], + "keywords": [], + "githubUser": [], + "marketplaces": [ + "platform" + ], + "supportedModules": [ + "agentix", + "xsiam", + "cloud_posture", + "cloud_runtime_security", + "data_security" + ], + "hidden": false +} \ No newline at end of file diff --git a/Packs/OnboardingIntegration/Integrations/OnboardingIntegration/OnboardingIntegration.yml b/Packs/OnboardingIntegration/Integrations/OnboardingIntegration/OnboardingIntegration.yml index d614d215d78e..bd9ad05b6e1f 100644 --- a/Packs/OnboardingIntegration/Integrations/OnboardingIntegration/OnboardingIntegration.yml +++ b/Packs/OnboardingIntegration/Integrations/OnboardingIntegration/OnboardingIntegration.yml @@ -1,4 +1,6 @@ category: Utilities +sectionorder: +- Collect provider: Open Source commonfields: id: OnboardingIntegration @@ -9,23 +11,35 @@ configuration: name: incidents_per_minute type: 0 required: false + section: Collect - defaultvalue: '10' display: Maximum number of incidents to create name: max_num_of_incidents type: 0 required: false + section: Collect - display: How often to create new incidents (in minutes) name: frequency type: 0 required: false + section: Collect - display: Fetch incidents name: isFetch type: 8 required: false + section: Collect - display: Incident type name: incidentType type: 13 required: false + section: Collect +- display: Incidents Fetch Interval + name: incidentFetchInterval + defaultvalue: '1' + required: false + type: 19 + advanced: true + section: Collect description: 'Creates mock email incidents using one of two randomly selected HTML templates. Textual content is randomly generated and defined to include some text (100 random words) and the following data (at least 5 of each data type): IP addresses, URLs, SHA-1 hashes, SHA-256 hashes, MD5 hashes, email addresses, domain names.' display: OnboardingIntegration name: OnboardingIntegration @@ -182,7 +196,7 @@ script: - contextPath: DBotScore.Indicator description: The indicator that was tested. type: String - dockerimage: demisto/faker3:1.0.0.10120494 + dockerimage: demisto/faker3:1.0.0.13766451 isfetch: true subtype: python3 tests: diff --git a/Packs/OnboardingIntegration/ReleaseNotes/1_1_15.md b/Packs/OnboardingIntegration/ReleaseNotes/1_1_15.md new file mode 100644 index 000000000000..c3af8f70c05b --- /dev/null +++ b/Packs/OnboardingIntegration/ReleaseNotes/1_1_15.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### OnboardingIntegration + +- Updated the Fetch Incidents configuration parameters. diff --git a/Packs/OnboardingIntegration/ReleaseNotes/1_1_16.md b/Packs/OnboardingIntegration/ReleaseNotes/1_1_16.md new file mode 100644 index 000000000000..bf323833e080 --- /dev/null +++ b/Packs/OnboardingIntegration/ReleaseNotes/1_1_16.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### OnboardingIntegration + +- Updated the Docker image to: *demisto/faker3:1.0.0.12844876*. + diff --git a/Packs/OnboardingIntegration/ReleaseNotes/1_1_17.md b/Packs/OnboardingIntegration/ReleaseNotes/1_1_17.md new file mode 100644 index 000000000000..cad9aed07ad2 --- /dev/null +++ b/Packs/OnboardingIntegration/ReleaseNotes/1_1_17.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### OnboardingIntegration + +- Updated the Docker image to: *demisto/faker3:1.0.0.13766451*. + diff --git a/Packs/OnboardingIntegration/pack_metadata.json b/Packs/OnboardingIntegration/pack_metadata.json index 2d3a1281b923..40ab328bc427 100644 --- a/Packs/OnboardingIntegration/pack_metadata.json +++ b/Packs/OnboardingIntegration/pack_metadata.json @@ -2,7 +2,7 @@ "name": "OnboardingIntegration", "description": "Creates mock email incidents using one of two randomly selected HTML templates. Textual content is randomly generated and defined to include some text (100 random words) and the following data (at least 5 of each data type): IP addresses, URLs, SHA-1 hashes, SHA-256 hashes, MD5 hashes, email addresses, domain names.", "support": "xsoar", - "currentVersion": "1.1.14", + "currentVersion": "1.1.17", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/OneLogin/ModelingRules/OneLoginModelingRules_1_3/OneLoginModelingRules_1_3.yml b/Packs/OneLogin/ModelingRules/OneLoginModelingRules_1_3/OneLoginModelingRules_1_3.yml index a94a5dc9750e..7b4d82b41b9f 100644 --- a/Packs/OneLogin/ModelingRules/OneLoginModelingRules_1_3/OneLoginModelingRules_1_3.yml +++ b/Packs/OneLogin/ModelingRules/OneLoginModelingRules_1_3/OneLoginModelingRules_1_3.yml @@ -1,6 +1,7 @@ name: OneLogin Modeling Rule id: onelogin_ModelingRule fromversion: 6.10.0 +toversion: 8.14.0 tags: onelogin rules: '' schema: '' diff --git a/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11.xif b/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11.xif new file mode 100644 index 000000000000..4e537710082a --- /dev/null +++ b/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11.xif @@ -0,0 +1,38 @@ +[MODEL: dataset=onelogin_iam_raw] +alter + event_type_id = to_string(event_type_id), + ipv4 = if(ipaddr ~= "^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", ipaddr, null), + ipv6 = if(ipaddr ~= "\:[a-fA-F\d]{1,4}|[a-fA-F\d]{1,4}\:", ipaddr, null), + target_user_id = to_string(user_id) +| alter + // map event type id according to https://api.onelogin.com/api/1/events/types (https://developers.onelogin.com/api-docs/1/events/event-types) + event_name = if(event_type_id="1", "APP_ADDED_TO_ROLE", event_type_id="2", "APP_REMOVED_FROM_ROLE", event_type_id="3", "USER_ASSUMED_USER", event_type_id="4", "USER_ASSIGNED_ROLE",event_type_id="5", "USER_LOGGED_INTO_ONELOGIN",event_type_id="6", "USER_FAILED_ONELOGIN_LOGIN",event_type_id="7", "USER_LOGGED_OUT_OF_ONELOGIN",event_type_id="8", "USER_LOGGED_INTO_APP",event_type_id="9", "USER_FAILED_APP_LOGIN",event_type_id="10", "USER_REQUESTED_NEW_PASSWORD",event_type_id="11", "USER_CHANGED_PASSWORD",event_type_id="12", "UNLOCKED_USER",event_type_id="13", "CREATED_USER",event_type_id="14", "UPDATED_USER",event_type_id="15", "DEACTIVATED_USER",event_type_id="16", "ACTIVATED_USER",event_type_id="17", "DELETED_USER",event_type_id="18", "ADMIN_APPROVED_PASSWORD_REQUEST",event_type_id="19", "USER_LOCKED",event_type_id="20", "REACHED_USERS_LIMIT",event_type_id="21", "SUSPENDED_USER",event_type_id="22", "USER_ADDED_OTP_DEVICE",event_type_id="23", "USER_BULK_OPERATION",event_type_id="24", "USER_REMOVED_OTP_DEVICE",event_type_id="25", "PROVISIONING_EXCEPTION",event_type_id="26", "PROVISIONING_EVENT",event_type_id="27", "USER_DOWNLOADED_CERT",event_type_id="28", "USER_RECENTLY_REMOVED",event_type_id="29", "USER_LOGGED_OUT_OF_APP",event_type_id="30", "UPDATED_PAYMENT_INFO",event_type_id="31", "FAILED_UPDATE_PAYMENT_INFO",event_type_id="32", "REACTIVATED_USER",event_type_id="33", "USERS_IMPORTED_FROM_DIRECTORY",event_type_id="34", "USER_REQUESTED_APP",event_type_id="35", "USER_LOCKED_OUT_OF_APP",event_type_id="36", "USER_LOST_OTP_DEVICE",event_type_id="37", "USER_JOIN_REQUEST",event_type_id="38", "APP_REACHED_USER_LIMIT",event_type_id="39", "CONNECTOR_BROKEN",event_type_id="40", "USER_UNLOCKED_OTP_DEVICE",event_type_id="41", "AD_CONNECTOR_STARTED",event_type_id="42", "AD_CONNECTOR_STOPPED",event_type_id="43", "AD_CONNECTOR_CONFIG_RELOAD",event_type_id="44", "AD_CONNECTOR_NOTIFICATION",event_type_id="45", "AD_CONNECTOR_EXCEPTION_OLD",event_type_id="46", "AD_CONNECTOR_FAIL_OVER",event_type_id="47", "AD_CONNECTOR_EXCEPTION",event_type_id="48", "IMPORTED_USER",event_type_id="49", "UPDATE_USER_FAILED",event_type_id="50", "REJECTED_USER",event_type_id="51", "USER_CREATED_IN_APP",event_type_id="52", "USER_UPDATED_IN_APP",event_type_id="53", "USER_SUSPENDED_IN_APP",event_type_id="54", "USER_REACTIVATED_IN_APP",event_type_id="55", "USER_DELETED_IN_APP",event_type_id="56", "UNMATCHED_USERS",event_type_id="57", "RABBIT_DOWN",event_type_id="58", "RABBIT_RESTARTED",event_type_id="59", "USER_LINKED_IN_APP",event_type_id="60", "PROVISIONING_DEPROVISIONING_MODE_DO_NOTHING_WARNING",event_type_id="61", "USER_FAILED_SUSPENDING_IN_APP",event_type_id="62", "USER_FAILED_REACTIVATING_IN_APP",event_type_id="63", "USER_FAILED_DELETING_IN_APP",event_type_id="64", "USER_FAILED_CREATING_IN_APP",event_type_id="65", "USER_FAILED_UPDATING_IN_APP",event_type_id="66", "NO_USERS_TO_IMPORT_FROM_DIRECTORY",event_type_id="67", "DIRECTORY_IMPORT_EXCEPTION",event_type_id="68", "USER_AUTHENTICATED_BY_RADIUS",event_type_id="69", "USER_REJECTED_BY_RADIUS",event_type_id="70", "PRIVILEGE_GRANTED_TO_ACCOUNT",event_type_id="71", "PRIVILEGE_REVOKED_FROM_ACCOUNT",event_type_id="72", "PRIVILEGE_GRANTED_TO_USER",event_type_id="73", "PRIVILEGE_REVOKED_FROM_USER",event_type_id="74", "TRUSTED_IDP_ADDED",event_type_id="75", "TRUSTED_IDP_REMOVED",event_type_id="76", "TRUSTED_IDP_MODIFIED",event_type_id="77", "USER_FAILED_PROXY_LOGIN",event_type_id="78", "USER_SUCCEEDED_PROXY_LOGIN",event_type_id="79", "AD_CONNECTOR_PROVISIONING_ERROR",event_type_id="80", "USER_CREATED_IN_DIRECTORY",event_type_id="81", "USER_UPDATED_IN_DIRECTORY",event_type_id="82", "USER_SUSPENDED_IN_DIRECTORY",event_type_id="83", "USER_REACTIVATED_IN_DIRECTORY",event_type_id="84", "USER_DELETED_IN_DIRECTORY",event_type_id="85", "APP_COULD_NOT_AUTHENTICATE",event_type_id="86", "USER_FAILED_REMOTE_AUTHENTICATION",event_type_id="87", "USER_VIEWED_NOTE",event_type_id="88", "USER_EDITED_NOTE",event_type_id="89", "USER_DELETED_NOTE",event_type_id="90", "USER_UNAUTHORIZED_APP_ACCESS",event_type_id="91", "USER_UNDETERMINED_BY_RADIUS",event_type_id="92", "USER_NTHASH_REQUESTED_BY_RADIUS",event_type_id="95", "DIRECTORY_GROUPS_WRITE_BACK_IMPORT_STARTED",event_type_id="96", "DIRECTORY_GROUPS_WRITE_BACK_IMPORT_FINISHED",event_type_id="100", "SELF_REGISTRATION_REQUEST",event_type_id="101", "SELF_REGISTRATION_APPROVED",event_type_id="102", "SELF_REGISTRATION_DENIED",event_type_id="103", "SELF_REGISTRATION_REQUEST_UNVERIFIED",event_type_id="104", "SELF_REGISTRATION_REQUEST_VERIFIED",event_type_id="105", "SMS_FAILURE",event_type_id="106", "USER_CHANGE_PASSWORD_FAILED",event_type_id="110", "APP_LOGINS_UPDATED",event_type_id="111", "APP_LOGINS_UPDATE_FAILED",event_type_id="112", "TRUSTED_IDP_MADE_DEFAULT",event_type_id="113", "DIRECTORY_IMPORT_STARTED",event_type_id="114", "DIRECTORY_IMPORT_FINISHED",event_type_id="115", "USER_INVITED",event_type_id="116", "CREATE_USER_FAILED",event_type_id="117", "DIRECTORY_SYNC_RUN_ID",event_type_id="118", "SAML_ACS_FAILED",event_type_id="119", "TRUSTED_IDP_REMOVED_AS_DEFAULT",event_type_id="120", "UNLOCKED_USER_IN_DIRECTORY",event_type_id="121", "SCRIPTLET_ERROR",event_type_id="122", "USER_AUTHENTICATED_BY_API",event_type_id="123", "USER_REJECTED_BY_API",event_type_id="124", "ENTITLEMENTS_CACHE_ACTION",event_type_id="125", "ENTITLEMENT_ACTION",event_type_id="126", "DIRECTORY_CONNECTOR_ENABLED",event_type_id="127", "DIRECTORY_CONNECTOR_DISABLED",event_type_id="128", "NO_ACTIVE_ACTIVE_DIRECTORY_CONNECTORS",event_type_id="129", "VLDAP_BIND_FAILURE",event_type_id="130", "VLDAP_BIND_SUCCESS",event_type_id="131", "DIRECTORY_EXPORT_STARTED",event_type_id="132", "DIRECTORY_EXPORT_FINISHED",event_type_id="133", "DIRECTORY_EXPORT_EXCEPTION",event_type_id="134", "DIRECTORY_REFRESH_SCHEMA_EXCEPTION",event_type_id="135", "CERTIFICATE_EXPIRES",event_type_id="136", "DIRECTORY_FIELDS_IMPORT_STARTED",event_type_id="137", "USER_APP_REQUEST_APPROVED",event_type_id="138", "USER_APP_REQUEST_DENIED",event_type_id="139", "DIRECTORY_FIELDS_IMPORT_FINISHED",event_type_id="140", "SOCIAL_SIGN_IN",event_type_id="141", "SOCIAL_SIGN_IN_FAILURE",event_type_id="145", "USER_SMART_PASSWORD_UPDATED",event_type_id="146", "USER_SMART_PASSWORD_UPDATE_FAILED",event_type_id="147", "USER_MANUALLY_ADDED_TO_ROLE",event_type_id="148", "USER_MANUALLY_REMOVED_FROM_ROLE",event_type_id="149", "USER_AUTO_ADDED_TO_ROLE",event_type_id="150", "USER_AUTO_REMOVED_FROM_ROLE",event_type_id="151", "USER_ROLE_MANAGEMENT_GRANTED",event_type_id="152", "USER_ROLE_MANAGEMENT_REVOKED",event_type_id="153", "MAC_LOGIN_SUCCESS",event_type_id="154", "MAC_LOGIN_FAILURE",event_type_id="155", "DIRECTORY_FIELDS_IMPORT_EXCEPTION",event_type_id="156", "POLICY_CREATED",event_type_id="157", "POLICY_UPDATED",event_type_id="158", "POLICY_DELETED",event_type_id="159", "PROXY_AGENT_CREATED",event_type_id="160", "PROXY_AGENT_DELETED",event_type_id="161", "RADIUS_CONFIG_CREATED",event_type_id="162", "RADIUS_CONFIG_UPDATED",event_type_id="163", "RADIUS_CONFIG_DELETED",event_type_id="164", "VPN_ENABLED",event_type_id="165", "VPN_SETTINGS_UPDATED",event_type_id="166", "VPN_DISABLED",event_type_id="167", "EMBEDDING_ENABLED",event_type_id="168", "EMBEDDING_SETTINGS_UPDATED",event_type_id="169", "EMBEDDING_DISABLED",event_type_id="170", "AUTHENTICATION_FACTOR_CREATED",event_type_id="171", "AUTHENTICATION_FACTOR_UPDATED",event_type_id="172", "AUTHENTICATION_FACTOR_DELETED",event_type_id="173", "SECURITY_QUESTIONS_UPDATED",event_type_id="174", "DESKTOP_SSO_SETTINGS_UPDATED",event_type_id="175", "DESKTOP_SSO_ENABLED",event_type_id="176", "DESKTOP_SSO_DISABLED",event_type_id="177", "CERTIFICATE_CREATED",event_type_id="178", "CERTIFICATE_DELETED",event_type_id="179", "API_CREDENTIAL_CREATED",event_type_id="180", "API_CREDENTIAL_DELETED",event_type_id="181", "API_CREDENTIAL_ENABLED",event_type_id="182", "API_CREDENTIAL_DISABLED",event_type_id="183", "VLDAP_ENABLED",event_type_id="184", "VLDAP_DISABLED",event_type_id="185", "VLDAP_SETTINGS_UPDATED",event_type_id="186", "BRANDING_ENABLED",event_type_id="187", "BRANDING_DISABLED",event_type_id="188", "BRANDING_UPDATED",event_type_id="189", "MAPPING_ADDED",event_type_id="190", "MAPPING_DELETED",event_type_id="191", "MAPPING_DISABLED",event_type_id="192", "MAPPING_ENABLED",event_type_id="193", "MAPPING_UPDATED",event_type_id="194", "USER_FIELD_ADDED",event_type_id="195", "USER_FIELD_DELETED",event_type_id="196", "COMPANY_INFO_UPDATED",event_type_id="197", "ACCOUNT_SETTINGS_UPDATED",event_type_id="198", "DIRECTORY_CREATED",event_type_id="199", "DIRECTORY_DESTROYED",event_type_id="200", "DIRECTORY_CONNECTOR_INSTANCE_ADDED",event_type_id="201", "DIRECTORY_CONNECTOR_INSTANCE_DELETED",event_type_id="202", "REAPPLIED_MAPPINGS",event_type_id="203", "SELF_REGISTRATION_PROFILE_CREATED",event_type_id="204", "SELF_REGISTRATION_PROFILE_UPDATED",event_type_id="205", "SELF_REGISTRATION_PROFILE_DESTROYED",event_type_id="206", "MANUALLY_ADDED_LOGIN",event_type_id="207", "MANUALLY_REMOVED_LOGIN",event_type_id="208", "RETRIED_PROVISIONING",event_type_id="209", "DIRECTORY_USER_IMPORT_WARNING",event_type_id="210", "LDAP_CONNECTOR_EXCEPTION",event_type_id="211", "ADMIN_CHANGED_USER_PASSWORD",event_type_id="212", "DIRECTORY_LOCKED",event_type_id="213", "PROFILE_PICTURE_UPLOADED",event_type_id="214", "PROFILE_PICTURE_DELETED",event_type_id="215", "ADMIN_CHANGED_ACCOUNT_SETTINGS",event_type_id="216", "JOB_IN_QUEUE",event_type_id="217", "DIRECTORY_IMPORT_LIMIT_REACHED",event_type_id="218", "REAPPLIED_MAPPINGS_FAILED",event_type_id="219", "WORKDAY_REAL_TIME_NOTIFICATION",event_type_id="220", "ADMIN_CREATED_PAYMENT_RECORD",event_type_id="221", "ADMIN_UPDATED_PAYMENT_RECORD",event_type_id="222", "ADMIN_DELETED_PAYMENT_RECORD",event_type_id="223", "USER_UNLICENSED",event_type_id="224", "USER_LICENSED_MANUALLY",event_type_id="225", "USER_UNLICENSED_MANUALLY",event_type_id="226", "USER_UNLICENSED_AUTOMATICALLY",event_type_id="227", "USER_LICENSE_FAILED",event_type_id="228", "USERS_LICENSED_BULK",event_type_id="229", "ACCOUNT_NEAR_LIMIT",event_type_id="230", "ACCOUNT_IN_LIMIT",event_type_id="231", "USERS_IN_UNLICENSED_STATE",event_type_id="232", "USER_AGREED_TERMS",event_type_id="233", "USER_DENIED_TERMS",event_type_id="234", "ADMIN_ENABLED_TERMS",event_type_id="235", "ADMIN_UPDATED_TERMS",event_type_id="236", "ADMIN_DISABLED_TERMS",event_type_id="237", "DELETE_USER_FAILED",event_type_id="238", "USER_REDIRECTED_FOR_PASSWORD_CHANGE",event_type_id="239", "IMPORT_USER_FAILED",event_type_id="240", "USER_REVEALED_PASSWORD",event_type_id="241", "CSV_IMPORT_FAILED",event_type_id="242", "JOB_START_FAILED",event_type_id="243", "JOB_TERMINATED",event_type_id="244", "REPORT_GENERATED",event_type_id="245", "REPORT_GENERATION_FAILED",event_type_id="246", "REPORT_GENERATION_TERMINATED",event_type_id="247", "USER_MAPPINGS_FAILED",event_type_id="248", "USER_MAPPINGS_SUCCEEDED",event_type_id="249", "USER_BULK_OPERATION_FAILED",event_type_id="250", "PROVISIONING_APP_CONFIG_ERROR",event_type_id="251", "PROVISIONING_APP_THROTTLED",event_type_id="252", "USER_REMOVELOGINS_FAILED",event_type_id="253", "ENTITLEMENT_MAPPINGS_FAILED",event_type_id="254", "ENTITLEMENT_MAPPINGS_REAPPLIED",event_type_id="255", "MANUALLY_UPDATED_LOGIN",event_type_id="291", "USER_CREATED_BY_TIDP",event_type_id="300", "LDAP_CONNECTOR_STARTED",event_type_id="301", "LDAP_CONNECTOR_NOTIFICATION",event_type_id="303", "LDAP_CONNECTOR_CONFIG_RELOAD",event_type_id="304", "LDAP_CONNECTOR_STOPPED",event_type_id="305", "LDAP_CONNECTOR_FAIL_OVER",event_type_id="306", "MANUALLY_ADDED_LOGIN_FAILURE",event_type_id="307", "LDAP_CONNECTOR_PROVISIONING_ERROR",event_type_id="330", "USER_DISASSOCIATED_FROM_DIRECTORY",event_type_id="331", "USER_ASSOCIATED_TO_DIRECTORY",event_type_id="332", "USER_DIRECTORY_EXTERNAL_ID_UPDATED",event_type_id="333", "USER_DIRECTORY_EXTERNAL_ID_DELETED",event_type_id="334", "USER_NOT_UPDATED_IN_APP",event_type_id="400", "API_BAD_REQUEST",event_type_id="401", "API_UNAUTHORIZED",event_type_id="402", "MAPPING_SKIPPED",event_type_id="410", "BROADCASTER_CREATED",event_type_id="411", "BROADCASTER_UPDATED",event_type_id="412", "BROADCASTER_DELETED",event_type_id="501", "API_INDEX_ACTION",event_type_id="502", "API_SHOW_ACTION",event_type_id="503", "API_RES_ACTION",event_type_id="510", "API_SET_PWD_SALT",event_type_id="511", "API_SET_PWD_CLEAR_TEXT",event_type_id="512", "API_SET_CUSTOM_ATTRS",event_type_id="513", "API_ADD_ROLES",event_type_id="514", "API_REMOVE_ROLES",event_type_id="515", "API_AUTH_ISSUE_TOKEN",event_type_id="516", "API_LOGOUT",event_type_id="517", "API_SET_PWD_SALT_FAILED",event_type_id="518", "API_SET_PWD_CLEAR_TEXT_FAILED",event_type_id="519", "API_SET_CUSTOM_ATTRS_FAILED",event_type_id="520", "API_ADD_ROLES_FAILED",event_type_id="521", "API_REMOVE_ROLES_FAILED",event_type_id="522", "API_AUTH_ISSUE_TOKEN_FAILED",event_type_id="523", "API_LOGOUT_FAILED",event_type_id="524", "API_DESTROY_USER_FAILED",event_type_id="525", "API_GET_INVITE_LINK_FAILED",event_type_id="526", "API_LOCK_USER_FAILED",event_type_id="527", "API_VERIFY_FACTOR_FAILED",event_type_id="528", "API_VERIFY_FACTOR",event_type_id="529", "API_UPDATE_USER",event_type_id="530", "API_DESTROY_USER",event_type_id="531", "API_LOCK_USER",event_type_id="532", "API_UPDATE_USER_FAILED",event_type_id="533", "API_CREATE_USER",event_type_id="534", "API_CREATE_USER_FAILED",event_type_id="535", "API_GET_INVITE_LINK",event_type_id="536", "API_USER_OTPS_RETRIEVED",event_type_id="537", "API_CONFIRM_FACTOR",event_type_id="538", "API_CONFIRM_FACTOR_FAILED",event_type_id="539", "API_TRIGGER_FACTOR",event_type_id="540", "API_ADDED_OTP_DEVICE",event_type_id="541", "DIRECTORY_UPDATED",event_type_id="542", "DIRECTORY_OUS_CHANGED",event_type_id="545", "API_SEND_INVITE_LINK_FAILED",event_type_id="546", "API_SEND_INVITE_LINK",event_type_id="550", "FORCE_LOGOUT_USER",event_type_id="551", "SUSPENDED_USER_VIA_API",event_type_id="552", "REACTIVATED_USER_VIA_API",event_type_id="553", "USER_LOCKED_VIA_API",event_type_id="554", "UNLOCKED_USER_VIA_API",event_type_id="555", "EXTERNAL_ASSUME_USER",event_type_id="600", "APP_CREATED_BY_USER",event_type_id="601", "APP_UPDATED_BY_USER",event_type_id="602", "APP_DELETED_BY_USER",event_type_id="700", "CONNECTOR_CREATED",event_type_id="701", "CONNECTOR_CREATE_FAILED",event_type_id="702", "CONNECTOR_UPDATED",event_type_id="703", "CONNECTOR_UPDATE_FAILED",event_type_id="704", "CONNECTOR_DELETED",event_type_id="705", "CONNECTOR_DELETE_FAILED",event_type_id="706", "CONNECTOR_STATS_UPDATE",event_type_id="800", "PARAMETER_CREATED",event_type_id="801", "PARAMETER_CREATE_FAILED",event_type_id="802", "PARAMETER_UPDATED",event_type_id="803", "PARAMETER_UPDATE_FAILED",event_type_id="804", "PARAMETER_DELETED",event_type_id="805", "PARAMETER_DELETE_FAILED",event_type_id="900", "ONELOGIN_DESKTOP_MAC_LOGIN_SUCCESS",event_type_id="901", "ONELOGIN_DESKTOP_MAC_LOGIN_FAILURE",event_type_id="902", "ONELOGIN_DESKTOP_DEVICE_DELETED",event_type_id="903", "ONELOGIN_DESKTOP_DEVICE_UNBIND",event_type_id="904", "ONELOGIN_DESKTOP_LOGIN_SUCCESS",event_type_id="905", "ONELOGIN_DESKTOP_LOGIN_FAILURE",event_type_id="906", "ONELOGIN_DESKTOP_USER_FAILED_ONELOGIN_LOGIN",event_type_id="907", "DIRECTORY_EXPORT_SUCCESS",event_type_id="911", "ONELOGIN_DESKTOP_REVOKE_CERT_FOR_USER",event_type_id="912", "ONELOGIN_DESKTOP_REVOKE_CERT_FOR_DEVICE",event_type_id="931", "ADAPTIVE_LOGIN_ENABLED",event_type_id="932", "ADAPTIVE_LOGIN_DISABLED",event_type_id="950", "OL_OTP_PUSH_REJECT",event_type_id="1001", "USER_LOGIN_CHALLENGE",event_type_id="1002", "USER_LOGIN_CHALLENGE_FAILED",event_type_id="1010", "USER_REAUTH_SUCCESS",event_type_id="1100", "TEMP_OTP_TOKEN_GENERATED",event_type_id="1101", "TEMP_OTP_TOKEN_REVOKED",event_type_id="1200", "DELEGATED_APP_PRIVILEGE_DENIED",event_type_id="1201", "DELEGATED_USER_PRIVILEGE_DENIED",event_type_id="1244", "USER_ADDED_PHONE_NUMBER",event_type_id="1245", "USER_UPDATED_PHONE_NUMBER",event_type_id="1300", "API_APP_CREATED",event_type_id="1301", "API_APP_CREATE_FAILED",event_type_id="1302", "API_APP_UPDATED",event_type_id="1303", "API_APP_UPDATE_FAILED",event_type_id="1304", "API_APP_DESTROYED",event_type_id="1305", "API_APP_DESTROY_FAILED",event_type_id="1400", "USER_VERIFIED_OTP_DEVICE",event_type_id="1401", "API_AUTH_APP_CREATE_FAILED",event_type_id="1402", "API_AUTH_APP_UPDATED",event_type_id="1403", "API_AUTH_APP_UPDATE_FAILED",event_type_id="1404", "API_AUTH_APP_DESTROYED",event_type_id="1405", "API_AUTH_APP_DESTROY_FAILED",event_type_id="1406", "API_AUTH_SCOPE_CREATED",event_type_id="1407", "API_AUTH_SCOPE_CREATE_FAILED",event_type_id="1408", "API_AUTH_SCOPE_UPDATED",event_type_id="1409", "API_AUTH_SCOPE_UPDATE_FAILED",event_type_id="1410", "API_AUTH_SCOPE_DESTROYED",event_type_id="1411", "API_AUTH_SCOPE_DESTROY_FAILED",event_type_id="1412", "API_AUTH_CLAIM_CREATED",event_type_id="1413", "API_AUTH_CLAIM_CREATE_FAILED",event_type_id="1414", "API_AUTH_CLAIM_UPDATED",event_type_id="1415", "API_AUTH_CLAIM_UPDATE_FAILED",event_type_id="1416", "API_AUTH_CLAIM_DESTROYED",event_type_id="1417", "API_AUTH_CLAIM_DESTROY_FAILED",event_type_id="1418", "API_AUTH_CLIENT_CREATED",event_type_id="1419", "API_AUTH_CLIENT_CREATE_FAILED",event_type_id="1420", "API_AUTH_CLIENT_UPDATED",event_type_id="1421", "API_AUTH_CLIENT_UPDATE_FAILED",event_type_id="1422", "API_AUTH_CLIENT_DESTROYED",event_type_id="1423", "API_AUTH_CLIENT_DESTROY_FAILED",event_type_id="1424", "API_AUTH_APP_CREATED",event_type_id="1500", "SANDBOX_SYNC_STARTED",event_type_id="1501", "SANDBOX_SYNC_FAILED",event_type_id="1502", "SANDBOX_SYNCED",event_type_id="1503", "SANDBOX_DELETED",event_type_id="1504", "SANDBOX_DELETE_FAILED",event_type_id="1505", "SANDBOX_CREATED",event_type_id="1506", "SANDBOX_CREATION_FAILED",event_type_id="1507", "SANDBOX_UPDATED",event_type_id="1508", "SANDBOX_UPDATE_FAILED",event_type_id="1509", "SANDBOX_DELETED_BY_API",event_type_id="1510", "SANDBOX_DELETE_FAILED_BY_API",event_type_id="1511", "SANDBOX_CREATED_BY_API",event_type_id="1512", "SANDBOX_CREATION_FAILED_BY_API",event_type_id="1513", "SANDBOX_UPDATED_BY_API",event_type_id="1514", "SANDBOX_UPDATE_FAILED_BY_API",event_type_id="1600", "PROFILE_DEVICES_DELETE_DEVICE",event_type_id="1601", "PROFILE_DEVICES_RENAME_DEVICE",event_type_id="1602", "PROFILE_DEVICES_UPDATE_DEFAULT",event_type_id="1603", "PROFILE_SETTINGS_UPDATE_LOCALE",event_type_id="1604", "PROFILE_SETTINGS_UPDATE_PHONE",event_type_id="1605", "PROFILE_SETTINGS_UPDATE_DEFAULT_TAB",event_type_id="1606", "PROFILE_SETTINGS_UPDATE_PROFILE_PHOTO",event_type_id="1607", "PROFILE_SETTINGS_UPDATE_APP_AUTO_DETECT",event_type_id="1608", "PROFILE_CHANGE_PASSWORD",event_type_id="1609", "PROFILE_SETTINGS_UPDATE_SHOW_TABS",event_type_id="1700", "RADIUS_ATTRIBUTE_CREATED",event_type_id="1701", "RADIUS_ATTRIBUTE_UPDATED",event_type_id="1702", "RADIUS_ATTRIBUTE_DELETED",event_type_id="1801", "ROLE_CREATED",event_type_id="1802", "ROLE_DELETED",event_type_id="1900", "API_BRAND_CREATED",event_type_id="1901", "API_BRAND_CREATE_FAILED",event_type_id="1902", "API_BRAND_UPDATED",event_type_id="1903", "API_BRAND_UPDATE_FAILED",event_type_id="1904", "API_BRAND_DESTROYED",event_type_id="1905", "API_BRAND_DESTROY_FAILED",event_type_id="2000", "HOOKS_LIST_FUNCTION",event_type_id="2001", "CUSTOM_SMTP_ERROR",event_type_id="2002", "SMTP_SETTINGS_UPDATED",event_type_id="2003", "HOOKS_CREATE_FUNCTION",event_type_id="2004", "HOOKS_CREATE_FUNCTION_FAILED",event_type_id="2005", "HOOKS_GET_FUNCTION",event_type_id="2006", "HOOKS_GET_FUNCTION_LOGS",event_type_id="2007", "HOOKS_UPDATE_FUNCTION",event_type_id="2008", "HOOKS_UPDATE_FUNCTION_FAILED",event_type_id="2009", "HOOKS_DELETE_FUNCTION",event_type_id="2010", "HOOKS_DELETE_FUNCTION_FAILED",event_type_id="2011", "HOOKS_LIST_ENVVAR",event_type_id="2012", "HOOKS_CREATE_ENVVAR",event_type_id="2013", "HOOKS_CREATE_ENVVAR_FAILED",event_type_id="2014", "HOOKS_GET_ENVVAR",event_type_id="2015", "HOOKS_UPDATE_ENVVAR",event_type_id="2016", "HOOKS_UPDATE_ENVVAR_FAILED",event_type_id="2017", "HOOKS_DELETE_ENVVAR",event_type_id="2018", "HOOKS_DELETE_ENVVAR_FAILED",event_type_id="2100", "DELEGATED_PRIVILEGE_CREATED_VIA_API",event_type_id="2101", "DELEGATED_PRIVILEGE_CREATED_BY_USER",event_type_id="2102", "DELEGATED_PRIVILEGE_UPDATED_VIA_API",event_type_id="2103", "DELEGATED_PRIVILEGE_UPDATED_BY_USER",event_type_id="2104", "DELEGATED_PRIVILEGE_DELETED_VIA_API",event_type_id="2105", "DELEGATED_PRIVILEGE_DELETED_BY_USER",event_type_id="2106", "DELEGATED_PRIVILEGE_ASSIGNED_TO_USER_VIA_API",event_type_id="2107", "DELEGATED_PRIVILEGE_ASSIGNED_TO_USER_BY_USER",event_type_id="2108", "DELEGATED_PRIVILEGE_REMOVED_FROM_USER_VIA_API",event_type_id="2109", "DELEGATED_PRIVILEGE_REMOVED_FROM_USER_BY_USER",event_type_id="2110", "DELEGATED_PRIVILEGE_ASSIGNED_TO_ROLE_VIA_API",event_type_id="2111", "DELEGATED_PRIVILEGE_ASSIGNED_TO_ROLE_BY_USER",event_type_id="2112", "DELEGATED_PRIVILEGE_REMOVED_FROM_ROLE_VIA_API",event_type_id="2113", "DELEGATED_PRIVILEGE_REMOVED_FROM_ROLE_BY_USER",event_type_id="2114", "DELEGATED_ROLE_PRIVILEGE_DENIED",event_type_id="2201", "REPORT_CREATED_BY_USER",event_type_id="2202", "REPORT_UPDATED_BY_USER",event_type_id="2203", "REPORT_CLONED_BY_USER",event_type_id="2204", "REPORT_DESTROYED_BY_USER",event_type_id="3000", "OIDC_GENERAL_FAIL",event_type_id="3001", "OIDC_IMPLICIT_FLOW_SUCCESS",event_type_id="3002", "OIDC_IMPLICIT_FLOW_FAILED",event_type_id="3003", "OIDC_GET_CODE_SUCCESS",event_type_id="3004", "OIDC_GET_CODE_FAILED",event_type_id="3005", "OIDC_AUTHORIZATION_CODE_SUCCESS",event_type_id="3006", "OIDC_AUTHORIZATION_CODE_FAILED",event_type_id="3007", "OIDC_CLIENT_CREDENTIALS_SUCCESS",event_type_id="3008", "OIDC_CLIENT_CREDENTIALS_FAILED",event_type_id="3009", "OIDC_PASSWORD_SUCCESS",event_type_id="3010", "OIDC_PASSWORD_FAILED",event_type_id="3011", "OIDC_REFRESH_TOKEN_SUCCESS",event_type_id="3012", "OIDC_REFRESH_TOKEN_FAILED",event_type_id="3013", "OIDC_VALIDATE_TOKEN_SUCCESS",event_type_id="3014", "OIDC_VALIDATE_TOKEN_FAILED",event_type_id="3015", "OIDC_REVOKE_TOKEN_SUCCESS",event_type_id="3016", "OIDC_REVOKE_TOKEN_FAILED",event_type_id="3017", "OIDC_USER_INFO_SUCCESS",event_type_id="3018", "OIDC_USER_INFO_FAILED",event_type_id="3019", "NOTIFICATION_WAS_SENT",event_type_id="3020", "GROUP_CREATED",event_type_id="3021", "GROUP_UPDATED",event_type_id="3022", "GROUP_DESTROYED",event_type_id="3023", "USER_CREATED_NOTE",event_type_id="3024", "DELEGATED_GROUP_PRIVILEGE_DENIED",event_type_id="3025", "DELEGATED_POLICY_PRIVILEGE_DENIED",event_type_id="3026", "PROFILE_DEVICES_UNSET_DEFAULT",event_type_id="3027", "DELEGATED_REPORT_PRIVILEGE_DENIED",event_type_id="3028", "DELEGATED_TRUSTED_IDP_PRIVILEGE_DENIED",event_type_id="9000", "USER_ENABLED_WORKFLOW",event_type_id="9001", "USER_DISABLED_WORKFLOW",event_type_id="9002", "USER_INITIATED_WORKFLOW",event_type_id="9003", "USER_COMPLETED_TASK",event_type_id="9004", "USER_MARKED_TASK_COMPLETE",event_type_id="9005", "USER_MARKED_WORKFLOW_COMPLETE",event_type_id="9006", "USER_MARKED_TASK_INCOMPLETE",event_type_id="9007", "USER_ENABLED_ONBOARDING",event_type_id="9008", "USER_DISABLED_ONBOARDING",event_type_id="9009", "USER_ENABLED_OFFBOARDING",event_type_id="9010", "USER_DISABLED_OFFBOARDING",event_type_id="9011", "USER_INITIATED_OFFBOARDING",event_type_id="9012", "USER_INITIATED_ONBOARDING",event_type_id="9013", "USER_COMPLETED_WORKFLOW",event_type_id="9014", "APP_RULES_LIST_SUCCESS",event_type_id="9015", "APP_RULES_LIST_FAILED",event_type_id="9016", "APP_RULES_CREATE_SUCCESS",event_type_id="9017", "APP_RULES_CREATE_FAILED",event_type_id="9018", "APP_RULES_UPDATE_SUCCESS",event_type_id="9019", "APP_RULES_UPDATE_FAILED",event_type_id="9020", "APP_RULES_GET_SUCCESS",event_type_id="9021", "APP_RULES_GET_FAILED",event_type_id="9022", "APP_RULES_DRYRUN_SUCCESS",event_type_id="9023", "APP_RULES_DRYRUN_FAILED",event_type_id="9024", "APP_RULES_DELETE_SUCCESS",event_type_id="9025", "APP_RULES_DELETE_FAILED",event_type_id="9026", "APP_RULES_SORT_SUCCESS",event_type_id="9027", "APP_RULES_SORT_FAILED",event_type_id="9028", "APP_RULES_APPLY_SUCCESS",event_type_id="9029", "APP_RULES_APPLY_FAILED",event_type_id="9030", "APP_RULES_REFRESH_ENTITLEMENTS_SUCCESS",event_type_id="9031", "APP_RULES_REFRESH_ENTITLEMENTS_FAILED",event_type_id="9032", "APP_RULES_LIST_CONDITIONS_SUCCESS",event_type_id="9033", "APP_RULES_LIST_CONDITIONS_FAILED",event_type_id="9034", "APP_RULES_LIST_CONDITION_OPERATORS_SUCCESS",event_type_id="9035", "APP_RULES_LIST_CONDITION_OPERATORS_FAILED",event_type_id="9036", "APP_RULES_LIST_ACTIONS_SUCCESS",event_type_id="9037", "APP_RULES_LIST_ACTIONS_FAILED",event_type_id="9038", "APP_RULES_LIST_ACTION_VALUES_SUCCESS",event_type_id="9039", "APP_RULES_LIST_ACTION_VALUES_FAILED",event_type_id="9040", "USER_ROLE_MANAGEMENT_GRANTED_FAILED",event_type_id="9041", "USER_ROLE_MANAGEMENT_REVOKED_FAILED",event_type_id="9042", "APP_ADDED_TO_ROLE_FAILED",event_type_id="9043", "APP_REMOVED_FROM_ROLE_FAILED",event_type_id="9044", "USER_MANUALLY_ADDED_TO_ROLE_FAILED",event_type_id="9045", "USER_MANUALLY_REMOVED_FROM_ROLE_FAILED",event_type_id="9046", "ROLE_CREATE_FAILED",event_type_id="9047", "ROLE_DELETE_FAILED",event_type_id="9048", "ROLE_LIST_SUCCESS",event_type_id="9049", "ROLE_LIST_FAILED",event_type_id="9050", "ROLE_GET_SUCCESS",event_type_id="9051", "ROLE_GET_FAILED",event_type_id="9052", "ROLE_UPDATE_SUCCESS",event_type_id="9053", "ROLE_UPDATE_FAILED",event_type_id="9054", "ROLE_LIST_APPS_SUCCESS",event_type_id="9055", "ROLE_LIST_APPS_FAILED",event_type_id="9056", "ROLE_LIST_USERS_SUCCESS",event_type_id="9057", "ROLE_LIST_USERS_FAILED",event_type_id="9058", "ROLE_LIST_ADMINISTRATORS_SUCCESS",event_type_id="9059", "ROLE_LIST_ADMINISTRATORS_FAILED", event_type_id) +| alter + xdm.event.id = to_string(id), + xdm.event.original_event_type = event_type_id, + xdm.event.type = event_name, + xdm.event.description = notes, + xdm.alert.severity = to_string(risk_score), + xdm.alert.description = risk_reasons, + xdm.network.rule = policy_name, + xdm.source.user.username = actor_user_name, + xdm.source.identity.username = actor_user_name, + xdm.source.user.identifier = to_string(actor_user_id), + xdm.source.identity.identifier = to_string(actor_user_id), + xdm.source.ipv4 = ipv4, + xdm.source.host.ipv4_addresses = if(ipv4 != null, arraycreate(ipv4), null), + xdm.source.ipv6 = ipv6, + xdm.source.host.ipv6_addresses = if(ipv6 != null, arraycreate(ipv6), null), + xdm.target.user.username = user_name, + xdm.target.identity.username = user_name, + xdm.target.user.identifier = target_user_id, + xdm.target.identity.identifier = target_user_id, + xdm.target.user.employee_id = target_user_id, + xdm.target.identity.employee_id = target_user_id, + xdm.target.application.name = app_name, + xdm.target.ipv4 = ipv4, + xdm.target.host.ipv4_addresses = if(ipv4 != null, arraycreate(ipv4), null), + xdm.target.ipv6 = ipv6, + xdm.target.host.ipv6_addresses = if(ipv6 != null, arraycreate(ipv6), null), + xdm.target.host.hostname = otp_device_name, + xdm.target.host.device_id = to_string(otp_device_id); \ No newline at end of file diff --git a/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11.yml b/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11.yml new file mode 100644 index 000000000000..5382a2145972 --- /dev/null +++ b/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11.yml @@ -0,0 +1,8 @@ +name: OneLogin Modeling Rule +id: onelogin_ModelingRule +fromversion: 8.15.0 +tags: onelogin +rules: '' +schema: '' +supportedModules: +- xsiam diff --git a/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11_schema.json b/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11_schema.json new file mode 100644 index 000000000000..9453f9078acb --- /dev/null +++ b/Packs/OneLogin/ModelingRules/OneLoginModelingRules_2_11/OneLoginModelingRules_2_11_schema.json @@ -0,0 +1,60 @@ +{ + "onelogin_iam_raw": { + "policy_name": { + "type": "string", + "is_array": false + }, + "actor_user_name": { + "type": "string", + "is_array": false + }, + "id": { + "type": "int", + "is_array": false + }, + "notes": { + "type": "string", + "is_array": false + }, + "event_type_id": { + "type": "int", + "is_array": false + }, + "user_id": { + "type": "int", + "is_array": false + }, + "ipaddr": { + "type": "string", + "is_array": false + }, + "user_name": { + "type": "string", + "is_array": false + }, + "risk_score": { + "type": "int", + "is_array": false + }, + "risk_reasons": { + "type": "string", + "is_array": false + }, + "app_name": { + "type": "string", + "is_array": false + }, + "actor_user_id": { + "type": "int", + "is_array": false + }, + "otp_device_name": { + "type": "string", + "is_array": false + }, + "otp_device_id": { + "type": "int", + "is_array": false + } + } +} diff --git a/Packs/OneLogin/ReleaseNotes/1_0_22.md b/Packs/OneLogin/ReleaseNotes/1_0_22.md new file mode 100644 index 000000000000..2b0037a96eb0 --- /dev/null +++ b/Packs/OneLogin/ReleaseNotes/1_0_22.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: OneLogin Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OneLogin/ReleaseNotes/1_0_23.md b/Packs/OneLogin/ReleaseNotes/1_0_23.md new file mode 100644 index 000000000000..b9584d9a3a14 --- /dev/null +++ b/Packs/OneLogin/ReleaseNotes/1_0_23.md @@ -0,0 +1,4 @@ + +## OneLogin + +- Documentation and metadata improvements. diff --git a/Packs/OneLogin/ReleaseNotes/1_0_24.md b/Packs/OneLogin/ReleaseNotes/1_0_24.md new file mode 100644 index 000000000000..2b0037a96eb0 --- /dev/null +++ b/Packs/OneLogin/ReleaseNotes/1_0_24.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: OneLogin Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OneLogin/pack_metadata.json b/Packs/OneLogin/pack_metadata.json index 7ebf9d169c0a..7b392f270710 100644 --- a/Packs/OneLogin/pack_metadata.json +++ b/Packs/OneLogin/pack_metadata.json @@ -2,7 +2,7 @@ "name": "OneLogin", "description": "Simple customer authentication and streamlined workforce identity operations with APIs.", "support": "xsoar", - "currentVersion": "1.0.21", + "currentVersion": "1.0.24", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -19,4 +19,4 @@ "supportedModules": [ "xsiam" ] -} \ No newline at end of file +} diff --git a/Packs/OnePassword/ModelingRules/OnePassword/OnePassword.yml b/Packs/OnePassword/ModelingRules/OnePassword/OnePassword.yml index 3307d0317aa4..1bb264df4be5 100644 --- a/Packs/OnePassword/ModelingRules/OnePassword/OnePassword.yml +++ b/Packs/OnePassword/ModelingRules/OnePassword/OnePassword.yml @@ -1,4 +1,5 @@ fromversion: 8.4.0 # Will be updated with XSIAM version updates +toversion: 8.14.0 id: OnePassword_OnePassword_ModelingRule name: OnePassword OnePassword Modeling Rule rules: '' diff --git a/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11.xif b/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11.xif new file mode 100644 index 000000000000..0ad441d77856 --- /dev/null +++ b/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11.xif @@ -0,0 +1,73 @@ +[RULE: 1Password_Common_Fields] +//Recurring fields for all log types +alter + xdm.event.type = SOURCE_LOG_TYPE, + xdm.event.operation = action, + xdm.source.location.country = location -> country, + xdm.source.location.region = location -> region, + xdm.source.location.city = location -> city, + xdm.source.location.latitude = to_float(location -> latitude), + xdm.source.location.longitude = to_float(location -> longitude); + +[RULE: 1Password_Common_Fields_For_Item_and_SignIns] +// Recurring fields only for Item usage actions and Sign in attempts +alter + host_os_name = client -> os_name, + host_os_version = client -> os_version, + extract_ip_address = client -> ip_address +| alter + full_os_name = concat(host_os_name , " ", host_os_version), + extract_ipv4 = if(extract_ip_address ~= "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}",extract_ip_address ,null), + extract_ipv6 = if(extract_ip_address ~= "^((?:[a-fA-F\d]{0,4}\:){2,7}[a-fA-F\d]{0,4})$",extract_ip_address ,null) +| alter + xdm.target.host.os = full_os_name, + xdm.target.application.name = client -> app_name, + xdm.target.application.version = client -> app_version, + xdm.source.ipv4 = extract_ipv4, + xdm.source.ipv6 = extract_ipv6, + xdm.target.host.os_family = if(host_os_name contains "Mac", XDM_CONST.OS_FAMILY_MACOS, host_os_name contains "Windows", XDM_CONST.OS_FAMILY_WINDOWS, host_os_name contains "Linux", XDM_CONST.OS_FAMILY_LINUX, host_os_name) ; + +[MODEL: dataset="1password_1password_raw"] +//1Password - Audit events modeling rules +filter SOURCE_LOG_TYPE = "Audit events" +| call 1Password_Common_Fields +| alter + extract_ip_address = session -> ip +| alter + extract_ipv4 = if(extract_ip_address ~= "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}",extract_ip_address ,null), + extract_ipv6 = if(extract_ip_address ~= "^((?:[a-fA-F\d]{0,4}\:){2,7}[a-fA-F\d]{0,4})$",extract_ip_address ,null) +| alter + xdm.source.user.identifier = actor_details -> uuid, + xdm.source.identity.identifier = actor_details -> uuid, + xdm.source.user.username = actor_details -> name, + xdm.source.identity.username = actor_details -> name, + xdm.source.user.upn = actor_details -> email, + xdm.source.identity.upn = actor_details -> email, + xdm.source.ipv4 = extract_ipv4, + xdm.source.ipv6 = extract_ipv6; +//1Password - Item usage action modeling rules +filter SOURCE_LOG_TYPE = "Item usage actions" +| call 1Password_Common_Fields +| call 1Password_Common_Fields_For_Item_and_SignIns +| alter + xdm.target.resource.parent_id = vault_uuid, + xdm.target.resource.id = item_uuid, + xdm.source.user.identifier = user -> uuid, + xdm.source.identity.identifier = user -> uuid, + xdm.source.user.username = user -> name, + xdm.source.identity.username = user -> name, + xdm.source.user.upn = user -> email, + xdm.source.identity.upn = user -> email; +//1Password - Sign in atempts modeling rules +filter SOURCE_LOG_TYPE = "Sign in attempts" +| call 1Password_Common_Fields +| call 1Password_Common_Fields_For_Item_and_SignIns +| alter + xdm.event.outcome = category, + xdm.event.outcome_reason = type, + xdm.target.user.identifier = target_user -> uuid, + xdm.target.identity.identifier = target_user -> uuid, + xdm.target.user.username = target_user -> name, + xdm.target.identity.username = target_user -> name, + xdm.target.user.upn = target_user -> email, + xdm.target.identity.upn = target_user -> email; diff --git a/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11.yml b/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11.yml new file mode 100644 index 000000000000..982f21aa08fc --- /dev/null +++ b/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: OnePassword_OnePassword_ModelingRule +name: OnePassword OnePassword Modeling Rule +rules: '' +schema: '' +tags: '' +supportedModules: +- xsiam diff --git a/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11_schema.json b/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11_schema.json new file mode 100644 index 000000000000..1d2a822407ef --- /dev/null +++ b/Packs/OnePassword/ModelingRules/OnePassword_2_11/OnePassword_2_11_schema.json @@ -0,0 +1,53 @@ + +{ + "1password_1password_raw": { + "SOURCE_LOG_TYPE": { + "type": "string", + "is_array": false + }, + "action": { + "type": "string", + "is_array": false + }, + "location": { + "type": "string", + "is_array": false + }, + "client": { + "type": "string", + "is_array": false + }, + "session": { + "type": "string", + "is_array": false + }, + "actor_details": { + "type": "string", + "is_array": false + }, + "vault_uuid": { + "type": "string", + "is_array": false + }, + "item_uuid": { + "type": "string", + "is_array": false + }, + "user": { + "type": "string", + "is_array": false + }, + "category": { + "type": "string", + "is_array": false + }, + "type": { + "type": "string", + "is_array": false + }, + "target_user": { + "type": "string", + "is_array": false + } + } + } \ No newline at end of file diff --git a/Packs/OnePassword/ReleaseNotes/1_0_14.md b/Packs/OnePassword/ReleaseNotes/1_0_14.md new file mode 100644 index 000000000000..7f200b41f83e --- /dev/null +++ b/Packs/OnePassword/ReleaseNotes/1_0_14.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: OnePassword OnePassword Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OnePassword/ReleaseNotes/1_0_15.md b/Packs/OnePassword/ReleaseNotes/1_0_15.md new file mode 100644 index 000000000000..047a7fa52b13 --- /dev/null +++ b/Packs/OnePassword/ReleaseNotes/1_0_15.md @@ -0,0 +1,4 @@ + +## OnePassword + +- Documentation and metadata improvements. diff --git a/Packs/OnePassword/ReleaseNotes/1_1_0.md b/Packs/OnePassword/ReleaseNotes/1_1_0.md new file mode 100644 index 000000000000..7f200b41f83e --- /dev/null +++ b/Packs/OnePassword/ReleaseNotes/1_1_0.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: OnePassword OnePassword Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OnePassword/pack_metadata.json b/Packs/OnePassword/pack_metadata.json index a05e6f3f6950..716814b08742 100644 --- a/Packs/OnePassword/pack_metadata.json +++ b/Packs/OnePassword/pack_metadata.json @@ -2,7 +2,7 @@ "name": "OnePassword", "description": "1Password is a password manager used for storing and managing your account credentials, financial information, documents, and other sensitive data. It provides secure password generation, quick form filling, and cross-device synchronization. It also offers features like secure password sharing and monitoring for compromised accounts.", "support": "xsoar", - "currentVersion": "1.0.13", + "currentVersion": "1.1.0", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", @@ -21,4 +21,4 @@ "supportedModules": [ "xsiam" ] -} \ No newline at end of file +} diff --git a/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3.py b/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3.py index 449ed2932f9c..9b3dea2ab4ed 100644 --- a/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3.py +++ b/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3.py @@ -51,7 +51,7 @@ class Config: COMPLIANCE_PAGE_SIZE = 100 MAX_PAGES_PER_FETCH = 50 # Safety cap on pagination loops. - DEFAULT_FIRST_FETCH = "1 minute ago" + DEFAULT_FIRST_FETCH = "1 hour ago" # Test-module probe: per-stream max-events ceiling when test_module exercises the collector # via the same fetch_stream pipeline (mirrors Koi's TEST_MODULE_MAX_EVENTS=1). @@ -490,6 +490,15 @@ def get_audit_logs( # endregion # region Event Collector - Compliance Logs (ChatGPT Platform - `Connect - Compliance` section) + def _compliance_headers(self) -> dict[str, str]: + """Build the auth headers for the ChatGPT Compliance API. + + The credential MUST carry the `Bearer` scheme - a bare key is not a well-formed + `Authorization` value and the API rejects it with 401 "Access token is missing" + without ever evaluating the key. + """ + return {"Authorization": f"Bearer {self.compliance_api_key}", "Accept": "application/json"} + def list_compliance_logs( self, workspace_id: str, @@ -519,10 +528,10 @@ def list_compliance_logs( params.append(("limit", effective_limit)) full_url = self.chatgpt_base_url + ApiPaths.compliance_logs(workspace_id) - headers = {"Authorization": self.compliance_api_key, "Accept": "application/json"} + headers = self._compliance_headers() demisto.debug( - f"[API Compliance List] Listing logs | event_types_count={len(event_types)} | " - f"after_set={bool(after)} | limit={limit}" + f"[API Compliance List] Listing logs | url={full_url} | auth_scheme=Bearer | " + f"event_types_count={len(event_types)} | after_set={bool(after)} | limit={limit}" ) # Fetch as text first so we can defensively handle single-JSON, concatenated-JSON, and empty @@ -565,8 +574,8 @@ def get_compliance_log_content(self, workspace_id: str, log_id: str) -> list[dic raise DemistoException("Compliance API Key is required to fetch OpenAI Compliance log content.") full_url = self.chatgpt_base_url + ApiPaths.compliance_log_content(workspace_id, log_id) - headers = {"Authorization": self.compliance_api_key, "Accept": "application/json"} - demisto.debug("[API Compliance Content] Fetching content for one log entry.") + headers = self._compliance_headers() + demisto.debug(f"[API Compliance Content] Fetching content for one log entry | url={full_url} | auth_scheme=Bearer") # The response body is a stream of concatenated JSON objects (or a JSONL file) - fetch raw text. raw_body = self._http_request(method="GET", full_url=full_url, headers=headers, resp_type="text", **Config.RETRY_POLICY) diff --git a/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3.yml b/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3.yml index 15c57f6ce6f2..b23bbbcf8ff5 100644 --- a/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3.yml +++ b/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3.yml @@ -647,7 +647,7 @@ script: script: '' type: python subtype: python3 - dockerimage: demisto/parse-emails:0.1.48.10120494 + dockerimage: demisto/parse-emails:0.2.0.12624354 isfetchevents: true fromversion: 6.0.0 tests: diff --git a/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3_test.py b/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3_test.py index f7508a2fea4b..8aa0a173917d 100644 --- a/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3_test.py +++ b/Packs/OpenAI/Integrations/OpenAiChatGPTV3/OpenAiChatGPTV3_test.py @@ -509,9 +509,9 @@ def test_parse_first_fetch_to_datetime_happy_path(mocker, first_fetch_input, exp def test_parse_first_fetch_to_datetime_bad_input_falls_back_to_default(mocker, bad_input): """Unparseable input MUST fall back to `Config.DEFAULT_FIRST_FETCH`, never to a hardcoded window. - This locks the regression where a typo silently widened the lookback 1440× from - "1 minute ago" to "1 day ago". Whitespace, empty strings, and made-up unit names - all must reach the fallback path. + This locks the regression where a typo silently widened the lookback window beyond + the documented default. Whitespace, empty strings, and made-up unit names all must + reach the fallback path. """ # Suppress the demisto.error stdout under pytest; the error-log contract is asserted by # test_parse_first_fetch_to_datetime_emits_error_log_on_bad_input. @@ -521,17 +521,13 @@ def test_parse_first_fetch_to_datetime_bad_input_falls_back_to_default(mocker, b assert isinstance(result, datetime) assert result.tzinfo is not None, "Fallback must also be timezone-aware." - # The fallback MUST equal Config.DEFAULT_FIRST_FETCH (currently "1 minute ago"), - # NOT a hardcoded 1-day window. + # The fallback MUST equal Config.DEFAULT_FIRST_FETCH, never a hardcoded window. expected_fallback = arg_to_datetime(Config.DEFAULT_FIRST_FETCH, is_utc=True) assert expected_fallback is not None if expected_fallback.tzinfo is None: expected_fallback = expected_fallback.replace(tzinfo=UTC) drift = abs((result - expected_fallback).total_seconds()) - assert drift < 30, ( - f"Fallback drifted {drift:.1f}s from Config.DEFAULT_FIRST_FETCH. " - f"If this exceeds the 1-minute window, the bug regressed." - ) + assert drift < 30, f"Fallback drifted {drift:.1f}s from Config.DEFAULT_FIRST_FETCH - the bug regressed." def test_parse_first_fetch_to_datetime_emits_error_log_on_bad_input(mocker): @@ -2872,3 +2868,163 @@ def test_create_moderation_command_multi_text_more_results_than_texts(mocker): # endregion + + +# region Compliance API authentication header +def test_compliance_headers_carry_bearer_scheme_and_accept(): + """ + Given: A client configured with a Compliance API key. + When: The shared compliance header helper builds the headers. + Then: Authorization carries the `Bearer` scheme and Accept requests JSON. + """ + headers = _make_client()._compliance_headers() + + assert headers == {"Authorization": "Bearer COMPLIANCE_KEY", "Accept": "application/json"} + + +def test_list_compliance_logs_sends_bearer_authorization_header(mocker): + """ + Given: A client configured with a valid Compliance API key. + When: Listing compliance logs. + Then: The Authorization header carries the `Bearer` scheme, which the ChatGPT Compliance + API requires. A bare key is rejected with 401 "Access token is missing". + """ + client = _make_client() + http_mock = mocker.patch.object(OpenAiClient, "_http_request", return_value=json.dumps({"data": [], "last_end_time": None})) + + client.list_compliance_logs( + workspace_id="FAKE_WORKSPACE_ID", + event_types=["AUDIT_LOG"], + after="2099-01-01T00:00:00Z", + limit=10, + ) + + assert http_mock.call_args.kwargs["headers"]["Authorization"] == "Bearer COMPLIANCE_KEY" + + +def test_get_compliance_log_content_sends_bearer_authorization_header(mocker): + """ + Given: A client configured with a valid Compliance API key. + When: Fetching the content of a single compliance log entry (step 2 of the two-step flow). + Then: The Authorization header carries the `Bearer` scheme. + """ + client = _make_client() + http_mock = mocker.patch.object(OpenAiClient, "_http_request", return_value="{}") + + client.get_compliance_log_content(workspace_id="FAKE_WORKSPACE_ID", log_id="FAKE_LOG_ID") + + assert http_mock.call_args.kwargs["headers"]["Authorization"] == "Bearer COMPLIANCE_KEY" + + +def test_audit_and_compliance_use_their_own_keys_with_bearer(mocker): + """ + Given: A client with distinct admin and compliance keys. + When: The audit endpoint and the compliance endpoint are each called. + Then: Each sends its own key, and both carry the `Bearer` scheme - the audit path must not + borrow the compliance credential, nor the reverse. + """ + client = _make_client() + + audit_mock = mocker.patch.object(OpenAiClient, "_http_request", return_value={"data": [], "has_more": False}) + client.get_audit_logs(after=None, effective_at_gt=1) + assert audit_mock.call_args.kwargs["headers"]["Authorization"] == "Bearer ADMIN_KEY" + + compliance_mock = mocker.patch.object( + OpenAiClient, "_http_request", return_value=json.dumps({"data": [], "last_end_time": None}) + ) + client.list_compliance_logs(workspace_id="FAKE_WORKSPACE_ID", event_types=["AUDIT_LOG"], after="2099-01-01T00:00:00Z") + assert compliance_mock.call_args.kwargs["headers"]["Authorization"] == "Bearer COMPLIANCE_KEY" + + +def test_compliance_request_never_sends_a_bare_key(mocker): + """ + Bad path: a bare, scheme-less credential must not reappear in any form. + + Given: A client with a Compliance API key. + When: The compliance listing is requested. + Then: The Authorization value is never the bare key, and never a doubled scheme. + """ + client = _make_client() + http_mock = mocker.patch.object(OpenAiClient, "_http_request", return_value=json.dumps({"data": []})) + + client.list_compliance_logs(workspace_id="FAKE_WORKSPACE_ID", event_types=["AUDIT_LOG"], after="2099-01-01T00:00:00Z") + + sent = http_mock.call_args.kwargs["headers"]["Authorization"] + assert sent != "COMPLIANCE_KEY", "regression: the raw key was sent with no auth scheme" + assert not sent.startswith("Bearer Bearer "), "the Bearer scheme was applied twice" + assert sent.startswith("Bearer ") + + +@pytest.mark.parametrize( + "status_code, message", + [ + pytest.param(401, "Unauthorized - Access token is missing", id="bad-401-unauthorized"), + pytest.param(403, "Forbidden", id="bad-403-forbidden"), + pytest.param(429, "Too Many Requests", id="bad-429-rate-limited"), + pytest.param(500, "Internal Server Error", id="bad-500-server-error"), + ], +) +def test_list_compliance_logs_propagates_api_errors(mocker, status_code, message): + """ + Bad path: the Compliance API rejects or fails the request. + + Given: The Compliance API returns an error status. + When: Listing compliance logs. + Then: The error surfaces to the caller rather than being swallowed into an empty result, + so the instance test reports a real failure instead of silently collecting nothing. + """ + client = _make_client() + mocker.patch.object( + OpenAiClient, "_http_request", side_effect=DemistoException(f"Error in API call [{status_code}] - {message}") + ) + + with pytest.raises(DemistoException) as exc_info: + client.list_compliance_logs(workspace_id="FAKE_WORKSPACE_ID", event_types=["AUDIT_LOG"], after="2099-01-01T00:00:00Z") + + assert str(status_code) in str(exc_info.value) + + +def test_get_compliance_log_content_propagates_unauthorized(mocker): + """ + Bad path: step 2 of the two-step flow is rejected. + + Given: The compliance content endpoint returns 401. + When: Fetching the content of a log entry. + Then: The error surfaces rather than yielding an empty record list, which would look + like a log entry that legitimately had no content. + """ + client = _make_client() + mocker.patch.object( + OpenAiClient, + "_http_request", + side_effect=DemistoException("Error in API call [401] - Unauthorized - Access token is missing"), + ) + + with pytest.raises(DemistoException) as exc_info: + client.get_compliance_log_content(workspace_id="FAKE_WORKSPACE_ID", log_id="FAKE_LOG_ID") + + assert "401" in str(exc_info.value) + + +@pytest.mark.parametrize( + "body", + [ + pytest.param("", id="bad-empty-body"), + pytest.param(" ", id="bad-whitespace-only-body"), + ], +) +def test_get_compliance_log_content_handles_empty_body(mocker, body): + """ + Bad path: the content endpoint returns nothing at all. + + Given: The compliance content response body is empty or whitespace. + When: Fetching the content of a log entry. + Then: An empty list is returned rather than raising, so one empty entry cannot abort a fetch. + """ + client = _make_client() + mocker.patch.object(OpenAiClient, "_http_request", return_value=body) + + assert client.get_compliance_log_content(workspace_id="FAKE_WORKSPACE_ID", log_id="FAKE_LOG_ID") == [] + + +# endregion diff --git a/Packs/OpenAI/Integrations/OpenAiChatGPTV3/README.md b/Packs/OpenAI/Integrations/OpenAiChatGPTV3/README.md index 2931ada6f483..c9732eb5aab3 100644 --- a/Packs/OpenAI/Integrations/OpenAiChatGPTV3/README.md +++ b/Packs/OpenAI/Integrations/OpenAiChatGPTV3/README.md @@ -362,3 +362,13 @@ Runs text or an image through the OpenAI Moderations API and returns per-categor | OpenAiChatGPTV3.Moderation.Flagged | Boolean | Whether the content was flagged by the moderation model. | | OpenAiChatGPTV3.Moderation.Categories | Unknown | The object of boolean values indicating which categories were flagged. | | OpenAiChatGPTV3.Moderation.CategoryScores | Unknown | The object of float values indicating the confidence score for each category. | + +<~PLATFORM> + +## License Requirements + +The following configuration parameters require the **Cortex XSIAM** license: + +- Fetch events + + diff --git a/Packs/OpenAI/ModelingRules/OpenAIModelingRules/OpenAIModelingRules.yml b/Packs/OpenAI/ModelingRules/OpenAIModelingRules/OpenAIModelingRules.yml index 013501b17530..4802924cd5b9 100644 --- a/Packs/OpenAI/ModelingRules/OpenAIModelingRules/OpenAIModelingRules.yml +++ b/Packs/OpenAI/ModelingRules/OpenAIModelingRules/OpenAIModelingRules.yml @@ -1,4 +1,5 @@ fromversion: 8.5.0 +toversion: 8.14.0 id: OpenAI_ModelingRule name: OpenAI Modeling Rule rules: '' diff --git a/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11.xif b/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11.xif new file mode 100644 index 000000000000..9558a74fa651 --- /dev/null +++ b/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11.xif @@ -0,0 +1,173 @@ +[MODEL: dataset="openai_admin_audit_raw"] +// OpenAI Organization Admin Audit Logs -> XDM +// Supports two stories: +// * Authentication (AUTH) -> login.* events +// * SaaS Audit -> all non-login admin/audit events (api_key.*, organization.*, etc.) +alter + xdm.event.type = if(type ~= "^login\.", "authentication", "saas audit"), + xdm.event.original_event_type = type, + xdm.event.outcome = if( + type ~= "\.succeeded$", XDM_CONST.OUTCOME_SUCCESS, + type ~= "\.failed$", XDM_CONST.OUTCOME_FAILED, + XDM_CONST.OUTCOME_SUCCESS), + xdm.event.outcome_reason = if( + type in ("login.succeeded", "logout.succeeded"), null, + coalesce( + json_extract_scalar(login, "$.failed.error_code"), + json_extract_scalar(logout, "$.failed.error_code"))), + xdm.source.user.upn = json_extract_scalar(actor, "$.session.user.email"), + xdm.source.identity.upn = json_extract_scalar(actor, "$.session.user.email"), + xdm.source.user.identifier = if(type ~= "^login\.", + json_extract_scalar(actor, "$.session.user.id"), + coalesce( + json_extract_scalar(actor, "$.session.user.id"), + json_extract_scalar(actor, "$.api_key.user.id"), + json_extract_scalar(actor, "$.api_key.service_account.id"))), + xdm.source.identity.identifier = if(type ~= "^login\.", + json_extract_scalar(actor, "$.session.user.id"), + coalesce( + json_extract_scalar(actor, "$.session.user.id"), + json_extract_scalar(actor, "$.api_key.user.id"), + json_extract_scalar(actor, "$.api_key.service_account.id"))), + xdm.source.user.username = if(type ~= "^login\.", + json_extract_scalar(actor, "$.session.user.email"), + coalesce( + json_extract_scalar(actor, "$.session.user.email"), + json_extract_scalar(actor, "$.api_key.user.email"))), + xdm.source.identity.username = if(type ~= "^login\.", + json_extract_scalar(actor, "$.session.user.email"), + coalesce( + json_extract_scalar(actor, "$.session.user.email"), + json_extract_scalar(actor, "$.api_key.user.email"))), + xdm.source.user.identity_type = if(type ~= "^login\.", null, + if(json_extract_scalar(actor, "$.type") = "session" + or (json_extract_scalar(actor, "$.type") = "api_key" and json_extract_scalar(actor, "$.api_key.type") = "user"), + XDM_CONST.IDENTITY_TYPE_USER, + XDM_CONST.IDENTITY_TYPE_APPLICATION)), + xdm.source.identity.identity_type = if(type ~= "^login\.", null, + if(json_extract_scalar(actor, "$.type") = "session" + or (json_extract_scalar(actor, "$.type") = "api_key" and json_extract_scalar(actor, "$.api_key.type") = "user"), + XDM_CONST.IDENTITY_TYPE_USER, + XDM_CONST.IDENTITY_TYPE_APPLICATION)), + xdm.source.ipv4 = json_extract_scalar(actor, "$.session.ip_address"), + xdm.source.user_agent = json_extract_scalar(actor, "$.session.user_agent"), + xdm.source.port = to_integer(0), + xdm.target.ipv4 = "", + xdm.target.port = to_integer(0), + xdm.network.ip_protocol = XDM_CONST.IP_PROTOCOL_TCP, + xdm.event.operation = if(type ~= "^login\.", XDM_CONST.OPERATION_TYPE_AUTH_LOGIN, null), + xdm.auth.service = "SP", + xdm.event.tags = if(type ~= "^login\.", arraycreate(XDM_CONST.EVENT_TAG_AUTHENTICATION), null), + xdm.source.location.country = json_extract_scalar(actor, "$.session.ip_address_details.country"), + xdm.source.location.city = json_extract_scalar(actor, "$.session.ip_address_details.city"), + xdm.source.location.region = json_extract_scalar(actor, "$.session.ip_address_details.region"), + xdm.source.location.latitude = to_float(json_extract_scalar(actor, "$.session.ip_address_details.latitude")), + xdm.source.location.longitude = to_float(json_extract_scalar(actor, "$.session.ip_address_details.longitude")), + xdm.observer.type = "OpenAI API Platform", + xdm.observer.sub_type = "Audit Logs", + xdm.target.resource.id = if( + type in ("login.succeeded", "login.failed", "logout.succeeded", "logout.failed", + "resource.deleted", "tunnel.created", "tunnel.updated", "tunnel.deleted", + "ip_allowlist.config.activated", "ip_allowlist.config.deactivated", + "certificates.activated", "certificates.deactivated"), null, + type = "api_key.created", json_extract_scalar(api_key_created, "$.id"), + type = "api_key.updated", json_extract_scalar(api_key_updated, "$.id"), + type = "api_key.deleted", json_extract_scalar(api_key_deleted, "$.id"), + type = "invite.sent", json_extract_scalar(invite_sent, "$.id"), + type = "invite.accepted", json_extract_scalar(invite_accepted, "$.id"), + type = "invite.deleted", json_extract_scalar(invite_deleted, "$.id"), + type = "user.added", json_extract_scalar(user_added, "$.id"), + type = "user.updated", json_extract_scalar(user_updated, "$.id"), + type = "user.deleted", json_extract_scalar(user_deleted, "$.id"), + type = "project.created", json_extract_scalar(project_created, "$.id"), + type = "project.updated", json_extract_scalar(project_updated, "$.id"), + type = "project.archived", json_extract_scalar(project_archived, "$.id"), + type = "service_account.created", json_extract_scalar(service_account_created, "$.id"), + type = "service_account.updated", json_extract_scalar(service_account_updated, "$.id"), + type = "service_account.deleted", json_extract_scalar(service_account_deleted, "$.id"), + type = "role.created", json_extract_scalar(role_created, "$.id"), + type = "role.updated", json_extract_scalar(role_updated, "$.id"), + type = "group.created", json_extract_scalar(group_created, "$.id"), + type = "group.updated", json_extract_scalar(group_updated, "$.id"), + type = "certificate.created", json_extract_scalar(certificate_created, "$.id"), + type = "certificate.updated", json_extract_scalar(certificate_updated, "$.id"), + type = "certificate.deleted", json_extract_scalar(certificate_deleted, "$.id"), + type = "organization.updated", json_extract_scalar(organization_updated, "$.id"), + null), + xdm.target.resource.name = if( + type in ("login.succeeded", "login.failed", "logout.succeeded", "logout.failed"), "OpenAI API Platform", + type not in ("certificate.created", "certificate.updated", "certificate.deleted", + "ip_allowlist.created", "ip_allowlist.deleted", + "role.created", "role.updated", "group.created", "group.updated", + "invite.sent", "project.created", "organization.updated"), null, + type = "certificate.created", json_extract_scalar(certificate_created, "$.name"), + type = "certificate.updated", json_extract_scalar(certificate_updated, "$.name"), + type = "certificate.deleted", json_extract_scalar(certificate_deleted, "$.name"), + type = "ip_allowlist.created", json_extract_scalar(ip_allowlist_created, "$.name"), + type = "ip_allowlist.deleted", json_extract_scalar(ip_allowlist_deleted, "$.name"), + type = "role.created", json_extract_scalar(role_created, "$.role_name"), + type = "role.updated", json_extract_scalar(role_updated, "$.changes_requested.role_name"), + type = "group.created", json_extract_scalar(group_created, "$.data.group_name"), + type = "group.updated", json_extract_scalar(group_updated, "$.changes_requested.group_name"), + type = "invite.sent", json_extract_scalar(invite_sent, "$.data.email"), + type = "project.created", json_extract_scalar(project_created, "$.data.name"), + coalesce( + json_extract_scalar(organization_updated, "$.changes_requested.name"), + json_extract_scalar(organization_updated, "$.changes_requested.title"))); + +[MODEL: dataset="openai_chatgpt_compliance_raw"] +// OpenAI ChatGPT Enterprise Compliance API Logs -> XDM +// Primary story: SaaS Audit - admin/workspace audit activity. +// Authentication (AUTH): only INVITE_ACCEPT carries a real user identity/auth-relevant. +alter + xdm.event.type = if(action = "INVITE_ACCEPT", "authentication", "saas audit"), + xdm.event.original_event_type = action, + xdm.event.outcome = if( + action_result = "SUCCESS", XDM_CONST.OUTCOME_SUCCESS, + action_result in("ERROR", "BLOCKED"), XDM_CONST.OUTCOME_FAILED, + action_result = null, null, + XDM_CONST.OUTCOME_FAILED), + xdm.event.outcome_reason = if( + action_result = "BLOCKED", "auth_policy_access_violation", + action_result = "ERROR", "OTHER", + "NOT_SPECIFIED"), + xdm.source.user.identifier = coalesce( + json_extract_scalar(actor, "$.user_id"), + json_extract_scalar(actor, "$.redacted_id")), + xdm.source.identity.identifier = coalesce( + json_extract_scalar(actor, "$.user_id"), + json_extract_scalar(actor, "$.redacted_id")), + xdm.source.user.upn = json_extract_scalar(actor, "$.user_email"), + xdm.source.identity.upn = json_extract_scalar(actor, "$.user_email"), + xdm.source.user.username = json_extract_scalar(actor, "$.user_email"), + xdm.source.identity.username = json_extract_scalar(actor, "$.user_email"), + xdm.source.user.user_type = if( + json_extract_scalar(actor, "$.type") = "ACCOUNT_USER", XDM_CONST.USER_TYPE_REGULAR, + json_extract_scalar(actor, "$.type") = "API_KEY", XDM_CONST.USER_TYPE_SERVICE_ACCOUNT, + null), + xdm.source.identity.user_type = if( + json_extract_scalar(actor, "$.type") = "ACCOUNT_USER", XDM_CONST.USER_TYPE_REGULAR, + json_extract_scalar(actor, "$.type") = "API_KEY", XDM_CONST.USER_TYPE_SERVICE_ACCOUNT, + null), + xdm.auth.privilege_level = if( + action_privilege = "ADMIN", XDM_CONST.PRIVILEGE_LEVEL_ADMIN, + action_privilege = "STANDARD_USER", XDM_CONST.PRIVILEGE_LEVEL_USER, + null), + xdm.source.ipv4 = json_extract_scalar(request_metadata, "$.client_ip"), + xdm.source.user_agent = json_extract_scalar(request_metadata, "$.client_user_agent"), + xdm.source.port = to_integer(0), + xdm.target.ipv4 = "", + xdm.target.port = to_integer(0), + xdm.network.ip_protocol = XDM_CONST.IP_PROTOCOL_TCP, + xdm.event.operation = if(action = "INVITE_ACCEPT", XDM_CONST.OPERATION_TYPE_AUTH_LOGIN, null), + xdm.auth.service = "SP", + xdm.event.tags = if(action = "INVITE_ACCEPT", arraycreate(XDM_CONST.EVENT_TAG_AUTHENTICATION), null), + xdm.source.location.country = json_extract_scalar(request_metadata, "$.client_ip_details.country"), + xdm.source.location.city = json_extract_scalar(request_metadata, "$.client_ip_details.city"), + xdm.source.location.region = json_extract_scalar(request_metadata, "$.client_ip_details.region"), + xdm.source.location.latitude = to_float(json_extract_scalar(request_metadata, "$.client_ip_details.latitude")), + xdm.source.location.longitude = to_float(json_extract_scalar(request_metadata, "$.client_ip_details.longitude")), + xdm.observer.type = "OpenAI API Platform", + xdm.target.resource.id = workspace_id, + xdm.target.resource.name = json_extract_scalar(request_metadata, "$.destination_hostname"), + xdm.network.session_id = conversation_id; diff --git a/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11.yml b/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11.yml new file mode 100644 index 000000000000..5a815e71c471 --- /dev/null +++ b/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11.yml @@ -0,0 +1,8 @@ +fromversion: 8.15.0 +id: OpenAI_ModelingRule +name: OpenAI Modeling Rule +rules: '' +schema: '' +tags: '' +supportedModules: +- xsiam diff --git a/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11_schema.json b/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11_schema.json new file mode 100644 index 000000000000..ff0f28a59838 --- /dev/null +++ b/Packs/OpenAI/ModelingRules/OpenAIModelingRules_2_11/OpenAIModelingRules_2_11_schema.json @@ -0,0 +1,182 @@ +{ + "openai_admin_audit_raw": { + "type": { + "type": "string", + "is_array": false + }, + "actor": { + "type": "string", + "is_array": false + }, + "object": { + "type": "string", + "is_array": false + }, + "effective_at": { + "type": "string", + "is_array": false + }, + "id": { + "type": "string", + "is_array": false + }, + "project": { + "type": "string", + "is_array": false + }, + "login": { + "type": "string", + "is_array": false + }, + "logout": { + "type": "string", + "is_array": false + }, + "api_key_created": { + "type": "string", + "is_array": false + }, + "api_key_updated": { + "type": "string", + "is_array": false + }, + "api_key_deleted": { + "type": "string", + "is_array": false + }, + "invite_sent": { + "type": "string", + "is_array": false + }, + "invite_accepted": { + "type": "string", + "is_array": false + }, + "invite_deleted": { + "type": "string", + "is_array": false + }, + "user_added": { + "type": "string", + "is_array": false + }, + "user_updated": { + "type": "string", + "is_array": false + }, + "user_deleted": { + "type": "string", + "is_array": false + }, + "project_created": { + "type": "string", + "is_array": false + }, + "project_updated": { + "type": "string", + "is_array": false + }, + "project_archived": { + "type": "string", + "is_array": false + }, + "service_account_created": { + "type": "string", + "is_array": false + }, + "service_account_updated": { + "type": "string", + "is_array": false + }, + "service_account_deleted": { + "type": "string", + "is_array": false + }, + "role_created": { + "type": "string", + "is_array": false + }, + "role_updated": { + "type": "string", + "is_array": false + }, + "group_created": { + "type": "string", + "is_array": false + }, + "group_updated": { + "type": "string", + "is_array": false + }, + "certificate_created": { + "type": "string", + "is_array": false + }, + "certificate_updated": { + "type": "string", + "is_array": false + }, + "certificate_deleted": { + "type": "string", + "is_array": false + }, + "ip_allowlist_created": { + "type": "string", + "is_array": false + }, + "ip_allowlist_deleted": { + "type": "string", + "is_array": false + }, + "organization_updated": { + "type": "string", + "is_array": false + } + }, + "openai_chatgpt_compliance_raw": { + "type": { + "type": "string", + "is_array": false + }, + "actor": { + "type": "string", + "is_array": false + }, + "action": { + "type": "string", + "is_array": false + }, + "action_result": { + "type": "string", + "is_array": false + }, + "action_privilege": { + "type": "string", + "is_array": false + }, + "principal": { + "type": "string", + "is_array": false + }, + "request_metadata": { + "type": "string", + "is_array": false + }, + "workspace_id": { + "type": "string", + "is_array": false + }, + "conversation_id": { + "type": "string", + "is_array": false + }, + "event_id": { + "type": "string", + "is_array": false + }, + "timestamp": { + "type": "string", + "is_array": false + } + } +} diff --git a/Packs/OpenAI/ReleaseNotes/2_1_5.md b/Packs/OpenAI/ReleaseNotes/2_1_5.md new file mode 100644 index 000000000000..c4cdf99c03c5 --- /dev/null +++ b/Packs/OpenAI/ReleaseNotes/2_1_5.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### OpenAI GPT + +- Fixed an authentication issue with fetching Compliance logs. diff --git a/Packs/OpenAI/ReleaseNotes/2_1_6.md b/Packs/OpenAI/ReleaseNotes/2_1_6.md new file mode 100644 index 000000000000..1d205af1f8ee --- /dev/null +++ b/Packs/OpenAI/ReleaseNotes/2_1_6.md @@ -0,0 +1,7 @@ + +#### Modeling Rules + +##### New: OpenAI Modeling Rule + +- New: Added a new modeling rule version that maps the `xdm.*.identity.*` fields alongside the existing `xdm.*.user.*` fields. +<~XSIAM> (Available from Cortex XSIAM 2.11). diff --git a/Packs/OpenAI/ReleaseNotes/2_1_7.md b/Packs/OpenAI/ReleaseNotes/2_1_7.md new file mode 100644 index 000000000000..e64d3e22ea4b --- /dev/null +++ b/Packs/OpenAI/ReleaseNotes/2_1_7.md @@ -0,0 +1,7 @@ + +#### Integrations + +##### OpenAI GPT + +- Updated the Docker image to: *demisto/parse-emails:0.2.0.12624354*. + diff --git a/Packs/OpenAI/pack_metadata.json b/Packs/OpenAI/pack_metadata.json index 0543c0f9d775..53067ad9334e 100644 --- a/Packs/OpenAI/pack_metadata.json +++ b/Packs/OpenAI/pack_metadata.json @@ -2,7 +2,7 @@ "name": "OpenAI", "description": "The OpenAI API can be applied to virtually any task that involves understanding or generating natural language or code.", "support": "xsoar", - "currentVersion": "2.1.4", + "currentVersion": "2.1.7", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "", diff --git a/Packs/OpsGenie/Integrations/OpsGenieV3/OpsGenieV3.yml b/Packs/OpsGenie/Integrations/OpsGenieV3/OpsGenieV3.yml index 83e90c665d0b..16eea259a5eb 100644 --- a/Packs/OpsGenie/Integrations/OpsGenieV3/OpsGenieV3.yml +++ b/Packs/OpsGenie/Integrations/OpsGenieV3/OpsGenieV3.yml @@ -1,4 +1,7 @@ category: Case Management +sectionorder: +- Connect +- Collect provider: Atlassian commonfields: id: OpsGenieV3 @@ -9,25 +12,30 @@ configuration: name: url required: true type: 0 + section: Connect - displaypassword: API Token name: credentials type: 9 required: true hiddenusername: true + section: Connect - display: Fetch incidents name: isFetch required: false type: 8 + section: Collect - defaultvalue: 3 days display: First fetch timestamp (