Zero Effort Fully Automatic Distributed Tracing For Python

This post was orginally featured on the Instana Blog.

Today Instana is proud to announce an industry first - fully automatic distributed tracing and monitoring of Python applications. Seriously, it’s completely automatic and it’s never been done before.

At Instana, monitoring automation and intelligence is our lifeblood. In today’s complex microservice architectures, imposing disruptive manual steps onto an organization for performance visibility is no longer acceptable. Today, many solutions claim automatic instrumentation but don’t mention all of the manual steps (and on-going maintenance) required to achieve that claimed automatic-ness.

The purpose of this post is twofold: First, to illustrate the true cost of adding instrumentation to your application and second, to announce general availability of our new fully automatic Python instrumentation that bypasses all of the identified organizational costs.

The True Cost of “Non-Automatic” Instrumentation

The use of microservices has exploded in the last few years and as a corollary, so has overall application complexity. This isn’t limited only to application architectures. That complexity also includes the many platforms and cloud providers in use today. The last thing an organization needs is to add more complexity; organizations need solutions that simplify.

Figure 1: Screenshot of Instana Infrastructure Map showing massive monitored environment

At Instana, we have unquestionably large customers with very complex application environments built from many code bases. For organizations such as these, to add instrumentation from other APM vendors usually involves the following costly steps:

1. Install a packageThis sounds benign but for an organization that has 30+ code bases, this is not a trivial task. Also consider whether developer resources need to be scheduled. To install a package across an organization, it’s teams and code bases can take weeks to months or more.
2. Configure that package  Most APM vendors require you to manually name your application (why?), set a customer key, or configure where to report performance instrumentation data.
3. Write CodeImport a package? Add tracing code? Initialize the package in your application? How long will this take to schedule developer time, do this across teams and against 30+ code bases? And further, to QA the changes and move them through the deploy cycle to production?
4. Change Execution Context  Some vendors even require organizations to change how they run their applications such as adding -javaagent to the command line to point out a custom monitoring jar or even running their entire application inside of a wrapper script (!). Such changes then require investment and ongoing support from your Operations and/or Site Reliability Engineers. Then multiply this effort by the number of code bases and platforms that this applies to…
5. Upgrades  Bugs do exist and software evolves to offer new features. Need to fix a bug or get a new monitoring feature? Then be prepared to handle all of the above steps again to upgrade and all the (potentially breaking) changes that come with it.

What we find here is that in the APM world, the term automatic is used often but when viewed with a modicum of scrutiny as above, the existing market of APM offerings are anything but automatic.

This is what we, at Instana, have been working to improve. To bypass this organizational cost completely. To make monitoring transparent, truly automatic, and continuous for organizations.

To that end, we’ve always offered Python instrumentation that requires no application code changes but today we take that effort one giant leap forward…

Instana’s Truly Automatic Python Instrumentation

Today we’re announcing the general availability of our automatic remote Python instrumentation utilizing Instana’s AutoTrace technology.

We don’t require any of the steps identified in the previous section - Instana will now identify your remote Python web servers, apply instrumentation, and start tracing/monitoring automatically.

To be clear, to get this new functionality requires exactly zero steps from your organization beyond installing the base Instana agent:

• No installing packages
• No modifying code
• No setting environment variables
• No deploying code
• No naming applications
• No configuring packages
• No changing command lines
• No upgrading packages
• No restarting processes

What you get:

• Instant Python gratification
• Contextual visualization of your Python services
• Automatic performance monitoring of those services
• Precise root cause analysis

With the Instana agent installed on your host, your Python web servers will be continuously detected, instrumented, and simply show up in your Instana dashboard as they come and go - true continuous and automatic performance monitoring of Python.

This makes Instana the first APM vendor to provide fully automatic and continuous remote monitoring and tracing of Python applications.

Note that for this first release, we’re limiting this to just Gunicorn and UWSGI web server processes but will expand this list over time. If you want to jump ahead and add more processes to this list, see our documentation.

To better visualize the significance of this announcement, let’s compare the true cost of instrumentation of other vendors to Instana. (❎ == Automatic, no work required)

The True Cost of Instrumentation	Other Vendors	Instana
Install Software (a package, jar, gem etc.)	✔	❎
Configure Software (INI files, YAML files)	✔	❎
Set Environment Variables	✔	❎
Name the Application	✔	❎
Code Changes Required? (import package, init code)	✔	❎
Change execution context (run your app inside a wrapper? -javaagent?)	✔	❎
Periodically upgrade instrumentation and revalidate all of the above?	✔	❎
Restart applications?	✔	❎
Periodic package upgrades?	✔	❎

P.S. Did you know that we already offer this same Auto-Trace functionality for Java and PHP? Python is the latest addition.

As Always - More Python Instrumentation

Outside of this new functionality, we’re also continually adding new instrumentation to the Python sensor. Most recently we’ve added support for SQL Alchemy and Asynqp. Stay tuned because much more is on the way.

Figure 2: Screenshot of Instana showing a distributed trace of a single request

Have a request for something not covered? Let us know!

A Final Word on Instana Auto-Trace for Python

I hope this post properly illustrates the true cost of non-automatic instrumentation for organizations and how Instana is rewriting the rules for APM. My hope is that this will help more than a few organizations make well informed decisions before choosing a solution that will cost too much organizationally.

We work hard at Instana on extensive, continuous, and automatic monitoring so you don’t have to. You can try Instana Auto-Trace for Python in your own applications today by signing up for a free trial using the “Try Instana” button at the top of this page.