We use cookies or similar technologies to personalize your online experience and tailor marketing to you. Many of our product features require cookies to function properly. Your use of this site and online product constitutes your consent to these personalization technologies. Read our Privacy Policy to find out more.

X

Python SDK

Installation

pip install libhoney

Initialization

Initialize the library by passing in your Team API key and the default dataset name to which it should send events.

import libhoney

libhoney.init(writekey="YOUR_API_KEY", dataset="honeycomb-python-example", debug=True)

# ... Do work and capture events

You are currently logged in to the team, so we have populated the write key here to the first write key for that team.

Further configuration options can be found in the API reference.

Note: the init params may contain an api_host key, which defaults to Honeycomb’s API server. Overriding this with an empty string is a good way to drop events in a test environment.

Building and sending events

Once initialized, libhoney is ready to send events. Events go through three phases:

Upon calling .send(), the event is dispatched to be sent to Honeycomb. All libraries set defaults that will allow your application to function as smoothly as possible during error conditions. When creating events faster than they can be sent, overflowed events will be dropped instead of backing up and slowing down your application.

In its simplest form, you can add a single attribute to an event with the .add_field(k, v) method. If you add the same key multiple times, only the last value added will be kept.

More complex structures (dicts and objects—things that can be serialized into a JSON object) can be added to an event with the .add(data) method.

Events can have metadata associated with them that is not sent to Honeycomb. This metadata is used to identify the event when processing the response. More detail about metadata is below in the Response section.

Handling responses

Sending an event is an asynchronous action and will avoid blocking by default. .send() will enqueue the event to be sent as soon as possible (thus, the return value doesn’t indicate that the event was successfully sent). Use the queue returned by .responses() to check whether events were successfully received by Honeycomb’s servers.

Before sending an event, you have the option to attach metadata to that event. This metadata is not sent to Honeycomb; instead, it’s used to help you match up individual responses with sent events. When sending an event, libhoney will take the metadata from the event and attach it to the response object for you to consume. Add metadata by calling .add_metadata(k, v) on an event.

Responses are represented as dicts with the following keys:

You don’t have to process responses if you’re not interested in them—simply ignoring them is perfectly safe. Unread responses will be dropped.

Examples

Honeycomb can calculate all sorts of statistics, so send the data you care about and let us crunch the averages, percentiles, lower/upper bounds, cardinality—whatever you want—for you.

Simple: send a blob of data

import libhoney

libhoney.init(writekey="YOUR_API_KEY", dataset="honeycomb-python-example", debug=True)

# create a new event
ev = libhoney.new_event()
# add data up front
ev.add({
  "method": "get",
  "hostname": "appserver15",
  "payload_length": 27
})

# do some work, maybe take some measurements

# add another field
ev.add_field("duration_ms", 153.12)
ev.send()

You are currently logged in to the team, so we have populated the write key here to the first write key for that team.

Intermediate: override some attributes

# ... Initialization code ...
params = {
  "hostname": "foo.local",
  "built": false,
  "user_id": -1
}

libhoney.add(params)

builder = libhoney.Builder()
builder.add_field("built", true)

# Spawn a new event and override the timestamp
event = builder.new_event()
event.add_field("user_id", 15)
event.add_field("latency_ms", datetime.datetime.now() - start)
event.created_at = datetime.datetime(2016, 2, 29, 1, 1, 1)
event.send()

Further examples can be found on GitHub.

Middleware examples: Django

For automatic instrumentation of Django and other application frameworks, see our Python Beeline. This example is intended to demonstrate using the SDK directly, but you’ll save time and get richer events (and tracing!) using the Beeline.

Django is a widely-used, high-level Python Web framework. Each inbound HTTP request as received by a framework like Django maps nicely to Honeycomb events, representing “a single thing of interest that happened” in a given system.

Django middlewares are simply classes that have access to the request object (and optionally the response object) in the application’s request-response chain.

As such, you can define a simple honey_middleware.py file as in the following:

import os
import time
import libhoney

class HoneyMiddleware(object):
  def __init__(self):
    libhoney.init(writekey=os.environ["YOUR_API_KEY"],
                  dataset="django-requests", debug=True)


  def process_request(self, request):
    request.start_time = time.time()

    return None


  def process_response(self, request, response):
    response_time = time.time() - request.start_time

    ev = libhoney.Event(data={
      "method": request.method,
      "scheme": request.scheme,
      "path": request.path,
      "query": request.GET,
      "isSecure": request.is_secure(),
      "isAjax": request.is_ajax(),
      "isUserAuthenticated": request.user.is_authenticated(),
      "username": request.user.username,
      "host": request.get_host(),
      "ip": request.META['REMOTE_ADDR'],
      "responseTime_ms": response_time * 1000,
    })
    ev.send()

    return response

See the examples/ directory on GitHub for more sample code demonstrating how to use events, builders, fields, and dynamic fields, specifically in the context of Django middleware.

Advanced usage: utilizing builders

Builders are, at their simplest, a convenient way to avoid repeating common attributes that may not apply globally. Creating a builder for a given component allows a variety of different events to be spawned and sent within the component, without having to repeat the component name as an attribute for each.

You can clone builders—the cloned builder will have a copy of all the fields and dynamic fields in the original. As your application forks down into more and more specific functionality, you can create more detailed builders. The final event creation in the leaves of your application’s tree will have all the data you’ve added along the way in addition to the specifics of this event.

The global scope is essentially a specialized builder, for capturing attributes that are likely useful to all events (e.g. hostname, environment, etc). Adding this kind of peripheral and normally unavailable information to every event gives you enormous power to identify patterns that would otherwise be invisible in the context of a single request.

Advanced usage: dynamic fields

The top-level libhoney and Builders support .add_dynamic_field(func). Adding a dynamic field to a Builder or top-level libhoney ensures that each time an event is created, the provided function is executed and the returned key/value pair is added to the event. This may be useful for including dynamic process information such as memory used, number of threads, concurrent requests, and so on to each event. Adding this kind of dynamic data to an event makes it easy to understand the application’s context when looking at an individual event or error condition.

Troubleshooting

Debug Mode

Our Python SDK supports an optional debug mode. Simply pass debug=True to the init function at startup to get verbose logging of events sent to and responses from the Honeycomb API.

Using the Python SDK with Python Pre-fork Models

Popular servers like uWSGI and Gunicorn utilize a pre-fork model where requests are delegated to separate Python processes.

Initializing the SDK before the fork happens can lead to a state where events cannot be sent. To initialize the SDK correctly, you will need to run your init code inside a post-fork hook.

uWSGI

Users of uWSGI can use a postfork decorator. Simply add the @postfork decorator to the function that initializes the Python Beeline, and it will be executed post-fork.

import logging
import os

import libhoney
from uwsgidecorators import postfork

@postfork
def init_libhoney():
    logging.info(f'libhoney initialization in process pid {os.getpid()}')
    libhoney.init(writekey="YOUR_API_KEY", dataset="honeycomb-uwsgi-example", debug=True)

Gunicorn

Gunicorn users can define a post_worker_init function in the Gunicorn config, and initialize the SDK there.

 # conf.py
import logging
import os
import libhoney

def post_worker_init(worker):
    logging.info(f'libhoney initialization in process pid {os.getpid()}')
    libhoney.init(writekey="YOUR_API_KEY", dataset="honeycomb-gunicorn-example", debug=True)

Then start gunicorn with the -c option:

gunicorn -c /path/to/conf.py

Celery

Celery uses a pre-fork approach to create worker processes. You can specify a worker_process_init decorated function to initialize the Python SDK after each worker has started.

import logging
import os
import libhoney

from celery.signals import worker_process_init
@worker_process_init.connect
def initialize_honeycomb(**kwargs):
    logging.info(f'libhoney initialization in process pid {os.getpid()}')
    libhoney.init(writekey="YOUR_API_KEY", dataset="honeycomb-celery-example", debug=True)

Contributions

Features, bug fixes and other changes to libhoney are gladly accepted. Please open issues or a pull request with your change via GitHub. Remember to add your name to the CONTRIBUTORS file!

All contributions will be released under the Apache License 2.0.