Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt

Use this file to discover all available pages before exploring further.

So you want to use Honeycomb. Maybe this is because you want high cardinality or faster querying times. Maybe you are at the beginning of a project to onboard to tracing or OpenTelemetry. But right now, you rely on unstructured logs for core analysis. Event logs are good for debugging, easy to generate, and unstructured logs are often the easiest to implement. The downside? The lack of standardization makes debugging using unstructured logs challenging, and querying unstructured logs is highly inefficient. You can sometimes do post-processing with regular expressions, but if you have to use regular expressions to understand what is happening in production, you are wasting your analytical skills on what amounts to a data formatting problem instead of actually analyzing the information the system is sending. The best solution is to transform your unstructured logs into a structured format before sending to Honeycomb. But regardless of whether you are ready to migrate to structured logs, if you have unstructured logs that you would like to get into Honeycomb, then you are in the right place.
If you have semi-structured logs, you may also find the resources shared in Start Building: Get Started with Structured Logs helpful.

​
Unstructured Data in Honeycomb

Honeycomb refers to unstructured data as information that does not follow a predefined data model. Often this information takes the form of logs emitted by applications to log files, but it can also include raw text or any kind of event data that has not been formatted into a structured format like JSON or key-value pairs. Some examples:
  • Raw log files: Traditional log files generated by systems or applications. Example: 2024-10-15 12:00:00 ERROR Connection timeout while accessing database
  • Text-based application logs: Logs that contain text messages, but no clear structure or predefined format. Example: User login failed: username=admin, reason=invalid password
  • Freeform event data: Events that capture information in a narrative or free-form text. Example: Server xyz failed to respond due to high CPU usage
  • Error or debug messages: Messages output during application runtime, typically for debugging or error reporting, but without a structured schema. Example: Stacktrace: at main.py: line 23
If you send unstructured logs to Honeycomb without transforming them into structured logs, then we will accept the data, but treat it as a single, opaque field, which will be harder to filter or query on. This also means you will miss out on some powerful observability features, such as tracing, high-cardinality querying, and exploring data interactively. To avoid this, we recommend transforming unstructured data into structured data. For example, you might transform this unstructured log: 
2024-10-15 12:00:00 ERROR connection timeout
Into this structured log: 
{"timestamp": "2024-10-15T12:00:00Z", "status": "error", "message": "connection timeout"}

​
Sending Unstructured Logs to Honeycomb

If you’re ready to transform your unstructured logs into structured logs, you can use any of the following options:
  • Honeycomb Telemetry Pipeline: Use the Honeycomb Telemetry Pipeline, which lets you standardize the entirety of telemetry operations on OpenTelemetry, whatever the format of existing logs, traces, and metrics.
  • OpenTelemetry Collector: Use the OpenTelemetry Collector as a logging agent along with the Filelog Receiver to parse unstructured logs and sending them to Honeycomb as structured logs.
  • HoneyTail + RegEx: Ingest unstructured logs using custom regular expressions and Honeytail, our lightweight tool that will tail your existing log files, parse the content, and send it to Honeycomb.

​
Enhancing Data in Honeycomb

Once your data is in Honeycomb, you will want to enhance it. If you’ve transformed your data to structured logs, you can use the resources shared in Start Building: Structured Logs. Otherwise, consider adding metadata like timestamps or categories to help you query later.

​
Exploring Log Data in Honeycomb

For structured logs, Honeycomb lets you easily query based on fields like status or timestamp. For unstructured logs, you can still use Honeycomb, but it will be harder to filter or analyze your data because the information is contained in one text blob. To learn how to analyze your data in Honeycomb, visit Investigate Log Data in Honeycomb.