Skip to content

Export Resource Metadata Labels #115

Open

Description

@raskasa

Background

My company has an internal monitoring/alerting solution that serves both our on-premise
and cloud needs; it's backed by Prometheus and Grafana. We get Cloud Monitoring (née
Stackdriver) metrics into our internal solution using this exporter.

I'm currently building some Grafana graphs for Dataflow jobs.

  • Metric types under the dataflow.googleapis.com
    namespace can be filtered down to a specific Dataflow job using the job_name label
    since this label's value includes the Dataflow job name. (Technically, the job_name
    label is attached to the dataflow_job
    monitored resource type).
  • Metric types under the compute.googleapis.com
    namespace can be filtered down to a specific Dataflow job using the instance_name
    label since this label's value includes the Dataflow job name.
  • Metric types under the agent.googleapis.com
    namespace CANNOT be filtered down to a specific Dataflow job. None of the labels
    include the Dataflow job name. The closest label that gives me what I need is the
    instance_id label that comes attached to the gce_instance monitored resource type.
    instance_id is numeric and there's no way to know what Dataflow job it belongs to.

Because I can't filter down to a specific Dataflow job with the agent.googleapis.com
metric types, I can't build a complete Grafana dashboard that visualizes all the metrics
I care about. Because of this, I still have to go back to using a combination of
Dataflow's Job Metrics UI and Cloud Monitoring's Metrics Explorer UI just to visualize
those metrics alongside others. I'm taking on the overhead of context-switching between
3 different monitoring tools. This slows down any analysis work that I want to do.

Request

As I understand it, in addition to the labels that are attached to metric types and
monitored resource types, Cloud Monitoring also attaches
resource metadata labels
to the monitored resource types. Specifically, the metadata.system_labels.name label
includes the GCE instance name which also includes the Dataflow job name. I would be
able to use this label to filter the agent.googleapis.com metric types down to a
specific Dataflow job.

The Stackdriver exporter doesn't seem to export these resource metadata labels though.

Is there a way we can start exporting these labels by default? Or, add a configuration
that allows users to enable collection of these extra labels?

References:

Happy to be corrected on anything stated here.

Activity

  1. xbglowx commented on Sep 3, 2021

    @xbglowx
    Contributor

    It would be nice to have something similar to aws_resource_info from the cloudwatch-exporter. That has allowed me to create custom tags on the aws resources themselves, i.e. owner, so that I can come up with a generic alerting rule and pass it along to alertmanager for proper routing.

    From what I have seen currently, in order to do something like that with stackdriver-exporter, one would need to create multiple alerting rules for each resource, that hardcodes the needed labels?

  2. ibeauvais commented on Nov 17, 2026

    @ibeauvais

    Proposed an alternative based on the Cloud Monitoring PromQL API, which avoids the aggregation configuration: #569

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions