Eliot: Logging for Complex & Distributed Systems

Build Status

Eliot is a Python logging system that outputs causal chains of actions happening within and across process boundaries: a logical trace of the system’s operation. In particular, Eliot can be used to generate meaningful, useful logs; tools like Logstash and ElasticSearch are still necessary to aggregate and store logs.

Eliot was originally created by ClusterHQ and is maintained by Itamar Turner-Trauring and others, under the Apache 2.0 License. Download from PyPI, read the documentation, file bugs at Github. Need help? Join the #eliot IRC channel on irc.freenode.net.

To install:

$ pip install eliot

Features:

  • Structured, optionally-typed log messages and actions.
  • Logged actions can span processes and threads.
  • Excellent support for unit testing your code’s logging.
  • Optional Twisted support.
  • Native journald support, easily usable by Logstash/Elasticsearch.
  • Supports CPython 2.7, 3.4, 3.5, 3.6 and PyPy.

Eliot is supported by 3rd party libraries like `eliot-tree`_, eliot-profiler, and eliot-profiler-analysis. A preliminary JavaScript implementation is also available.

Here’s an example of using Eliot, and the output rendered by the third-party `eliot-tree`_ utility:

import sys
from eliot import start_action, to_file
import requests
to_file(sys.stdout)


def check_links(urls):
    with start_action(action_type="check_links", urls=urls):
        for url in urls:
            try:
                with start_action(action_type="download", url=url):
                    response = requests.get(url)
                    response.raise_for_status()
            except Exception as e:
                raise ValueError(str(e))

check_links(["http://google.com"], ["http://nosuchurl"])
$ python examples/linkcheck.py | eliot-tree
4c42a789-76f5-4f0b-b154-3dd0e3041445
+-- check_links@1/started
    `-- urls: [u'http://google.com', u'http://nosuchurl']
    +-- download@2,1/started
        `-- url: http://google.com
        +-- download@2,2/succeeded
    +-- download@3,1/started
        `-- url: http://nosuchurl
        +-- download@3,2/failed
            |-- exception: requests.exceptions.ConnectionError
            |-- reason: ('Connection aborted.', gaierror(-2, 'Name or service not known'))
    +-- check_links@4/failed
        |-- exception: exceptions.ValueError
        |-- reason: ('Connection aborted.', gaierror(-2, 'Name or service not known'))

Contents:

Indices and tables