Python Profiling and Performance Tuning in Production

Presented by Joe Gordon
Thursday 2:15 p.m.–3 p.m.
Target audience: Developer

Abstract

Pinterest decreased latency and shrunk their front-end fleet by over 40% with less than 100 lines of python. This was done by understanding how a significant amount of latency and consumed resources were the result of gevent’s cooperative multitasking. 3 different open source python profilers, were then used together to identify and fix bottlenecks and inefficiencies in the code base. Most profilers are too slow to use in production, so profiling of web servers often occurs in synthetic environments. But to get an accurate picture of what is happening you need production profiling data, so a production ready Linux signal (SIGVTALRM) based python profiler was used. This talk will cover how Pinterest shrunk their front-end fleet, the techniques and tooling used to improve performance and how they can be applied elsewhere.

Presented by

Joe Gordon

Joe is an SRE at Pinterest, where he works on search and performance. Before that he spent several years working on OpenStack. He has previously spoken at numerous conferences such as EuroPython, LinuxCon, LCA.

©2016 Linux Australia and linux.conf.au 2017. Linux is a registered trademark of Linus Torvalds. Site design by Takeflight. Image credits can be found on our Colophon.