Skip to main content
Ent+
This feature is available as an add-on for the Honeycomb Enterprise plan. Please contact your Honeycomb account team for details.

​
Supported Types

MetricsLogsTraces
✓✓✓

​
How It Works

  1. This source rehydrates data previously stored by Google Cloud Storage.
  2. It will process both uncompressed JSON objects and objects compressed with gzip.
  3. You can authenticate to Google Cloud using the provided credentials, credentials_file, or by using Application Default Credentials.
  4. Your authentication credentials must have the Storage Admin permission to read and delete objects.

​
Notes

This is not a traditional source that continually produces data. Instead, it rehydrates all objects found within a specified time range. Once all objects in that time range have been rehydrated, the source will stop producing data.

​
Configuration

ParameterTypeDefaultDescription
telemetry_types*telemetrySelectorLogs, Metrics, TracesSpecifies which types of telemetry to rehydrate.
bucket_name*string""The name of the bucket to rehydrate from.
project_idstring""The ID of the Google Cloud project the bucket belongs to. Will be read from credentials if not configured.
auth_typeenumautoThe method used for authenticating to Google Cloud. Valid values are “auto”, “json”, or “file”.
credentialsstring""JSON value from a Google Service Account credential file. Required if auth_type is “json”.
credentials_filestring""Path to a Google Service Account credential file. Required if auth_type is “file”.
starting_time*dateTime""The UTC start time for rehydration. Must be in the format “YYYY-MM-DDTHH:MM”.
ending_time*dateTime""The UTC end time for rehydration. Must be in the format “YYYY-MM-DDTHH:MM”.
folder_namestring""Restricts rehydration to objects in a specific folder within the bucket.
batch_sizeint30The number of objects to download at once. This impacts performance by controlling the number of concurrent object downloads.
delete_on_readboolfalseIf true, objects will be deleted after being rehydrated.
storage_enablebooltrueEnable to specify a storage extension for rehydration progress.
storage_directorystring$OIQ_OTEL_COLLECTOR_HOME/storageDirectory for storing rehydration state. Useful for maintaining state and resuming operations after disruptions.
* required field

​
Example Configurations

​
Basic Configuration

This configuration authenticates using Application Default Credentials and rehydrates data in the specified bucket, folder, and time range. Standalone Source: 
apiVersion: bindplane.observiq.com/v1
kind: Source
metadata:
  id: google_cloud_storage_rehydration
  name: google_cloud_storage_rehydration
spec:
  type: google_cloud_storage_rehydration
  parameters:
    - name: telemetry_types
      value: ['Logs', 'Metrics', 'Traces']
    - name: bucket_name
      value: 'my-bucket'
    - name: auth_type
      value: 'auto'
    - name: starting_time
      value: '2025-03-03T16:00'
    - name: ending_time
      value: '2025-03-03T17:00'
    - name: folder_name
      value: 'my-folder-name'
    - name: batch_size
      value: 30
    - name: storage_enable
      value: false

​
Complete Configuration

This configuration demonstrates all available options for the Google Cloud Storage Rehydration source, including authentication, storage settings, and delete on read functionality. Standalone Source: 
apiVersion: bindplane.observiq.com/v1
kind: Source
metadata:
  id: google_cloud_storage_rehydration
  name: google_cloud_storage_rehydration
spec:
  type: google_cloud_storage_rehydration
  parameters:
    - name: telemetry_types
      value: ['Logs', 'Metrics', 'Traces']
    - name: bucket_name
      value: 'my-bucket'
    - name: project_id
      value: 'my-project'
    - name: auth_type
      value: 'file'
    - name: credentials_file
      value: '/path/to/googlecloud/credentials/file'
    - name: starting_time
      value: '2025-03-03T16:00'
    - name: ending_time
      value: '2025-03-03T17:00'
    - name: folder_name
      value: 'my-folder'
    - name: batch_size
      value: 30
    - name: delete_on_read
      value: true
    - name: storage_enable
      value: true
    - name: storage_directory
      value: '/custom/storage/path'