Integrating and Migrating Existing Logging

If you have an existing code base, you likely have existing log messages. This document will explain how to migrate and integrate existing logging into your new Eliot log setup. In particular, this will focus on the Python standard library logging package, but the same principles apply to other logging libraries.

Route existing logs to Eliot

Eliot includes a logging.Handler that can take standard library log messages and route them into Eliot. These log messages will automatically appear in the correct place in the action tree! Once you add actions to your code these log messages will automatically benefit from Eliot’s causal information.

To begin with, however, we’ll just add routing of log messages to Eliot:

# Add Eliot Handler to root Logger. You may wish to only route specific
# Loggers to Eliot.
import logging
from eliot.stdlib import EliotHandler

Add actions at entry points and other key points

Simply by adding a few key actions—the entry points to the code, as well as key sub-actions—you can start getting value from Eliot’s functionality while still getting information from your existing logs. You can leave existing log messages in place, replacing them with Eliot logging opportunistically; they will still be included in your output.

Example of routing standard library logging to Eliot.

The assumption is you have legacy logging using stdlib, and are switching over
to Eliot.

import logging
import sys

from eliot.stdlib import EliotHandler
from eliot import start_action, to_file

# A Logger left over from before switch to Eliot
LEGACY_LOGGER = logging.Logger("mypackage")

def do_a_thing(i):
    with start_action(action_type="mypackage:do_a_thing"):
        # run your business logic....
        if i == 3:
            LEGACY_LOGGER.error("The number 3 is a bad number, don't use it.")
            raise ValueError("I hate the number 3")

def main():
    with start_action(action_type="mypackage:main"):
        for i in [1, 3]:
            except ValueError:
      "Number {} was rejected.".format(i))

if __name__ == '__main__':
    # Hook up stdlib logging to Eliot:
    # Write Eliot logs to stdout:
    # Run the code:

The stdlib logging messages will be included in the correct part of the tree:

$ python examples/  | eliot-tree
└── mypackage:main/1 ⇒ started
    ├── timestamp: 2018-07-15 16:50:39.230467
    ├── mypackage:do_a_thing/2/1 ⇒ started
    │   ├── timestamp: 2018-07-15 16:50:39.230709
    │   └── mypackage:do_a_thing/2/2 ⇒ succeeded
    │       └── timestamp: 2018-07-15 16:50:39.230836
    ├── mypackage:do_a_thing/3/1 ⇒ started
    │   ├── timestamp: 2018-07-15 16:50:39.230980
    │   ├── eliot:stdlib/3/2
    │   │   ├── log_level: ERROR
    │   │   ├── logger: mypackage
    │   │   ├── message: The number 3 is a bad number, don't use it.
    │   │   └── timestamp: 2018-07-15 16:50:39.231157
    │   └── mypackage:do_a_thing/3/3 ⇒ failed
    │       ├── exception: builtins.ValueError
    │       ├── reason: I hate the number 3
    │       └── timestamp: 2018-07-15 16:50:39.231364
    ├── eliot:stdlib/4
    │   ├── log_level: INFO
    │   ├── logger: mypackage
    │   ├── message: Number 3 was rejected.
    │   └── timestamp: 2018-07-15 16:50:39.231515
    └── mypackage:main/5 ⇒ succeeded
        └── timestamp: 2018-07-15 16:50:39.231641