Profiling the Toolchain

The toolchain interacts with a moderate amount of external tools and sublibraries, the exact set generally depends on which compilation and linker flags were used. If you are seeing abnormal compilation times, or if you are developing the Emscripten toolchain itself, it may be useful to profile the toolchain performance itself as it compiles your project. Emscripten has a built-in toolchain wide emprofile.py profiler that can be used for this purpose.

Quick Example

To try out the toolchain profiler, run the following set of commands:

cd path/to/emscripten
tools/emprofile.py --reset
export EM_PROFILE_TOOLCHAIN=1
emcc tests/hello_world.c -O3 -o a.html
tools/emprofile.py --graph

On Windows, replace the export keyword with set instead. The last command should generate a HTML file of form toolchain_profiler.results_yyyymmdd_hhmm.html that can be opened in the web browser to view the results.

Details

The toolchain profiler is active whenever the toolchain is invoked with the environment variable EM_PROFILE_TOOLCHAIN=1 being set. In this mode, each called tool will accumulate profiling instrumentation data to a set of .json files under the Emscripten temp directory.

Profiling Tool Commands

The command tools/emprofile.py --reset deletes all previously stored profiling data. Call this command to erase the profiling session to a fresh empty state. To start profiling, call Emscripten tool commands with the environment variable EM_PROFILE_TOOLCHAIN=1 set either system-wide as shown in the example, or on a per command basis, like this:

cd path/to/emscripten
tools/emprofile.py --reset
EM_PROFILE_TOOLCHAIN=1 emcc tests/hello_world.c -o a.bc
EM_PROFILE_TOOLCHAIN=1 emcc a.bc -O3 -o a.html
tools/emprofile.py --graph --outfile=myresults.html

Any number of commands can be profiled within one session, and when tools/emprofile.py --graph is finally called, it will pick up records from all Emscripten tool invocations up to that point. Calling --graph also clears the recorded profiling data.

The output HTML filename can be chosen with the optional --outfile=myresults.html parameter.

Instrumenting Python Scripts

In order for the toolchain to work, each “top level” Python script (a script that is directly called from the command line, or by a subprocess spawn) should have the following preamble in the beginning of the script:

from tools.toolchain_profiler import ToolchainProfiler
if __name__ == '__main__':
  ToolchainProfiler.record_process_start()

Additionally, at the end of the script when the script is about to exit, it should do so by explicitly calling either the sys.exit(<returncode>) function, or by calling the ToolchainProfiler.record_process_exit(<returncode>) function, whichever is more convenient for the script. The function ToolchainProfiler.record_process_exit() does not exit by itself, but only records that the process is quitting.

These two blocks ensure that the toolchain profiler will be aware of all tool invocations that occur. In the graphed output, the process spawns will be shown in green color.

Python Profiling Blocks

Graphing the subprocess start and end times alone might sometimes be a bit too coarse view into what is happening. In Python code, it is possible to hierarchically annotate individual blocks of code to break down execution into custom tasks. These blocks will be shown in blue in the output graph. To add a custom profiling block, use the Python with keyword to add a profile_block section:

with ToolchainProfiler.profile_block('my_custom_task'):
  do_some_tasks()
  call_another_function()
  more_code()

this_is_outside_the_block()

This will show the three functions in the same scope under a block ‘my_custom_task’ drawn in blue in the profiling swimlane.

In some cases it may be cumbersome to wrap the code inside a with section. For these scenarios, it is also possible to use low level C-style enter_block and exit_block statements.

ToolchainProfiler.enter_block('my_code_block')
try:
  do_some_tasks()
  call_another_function()
  more_code()
finally:
  ToolchainProfiler.exit_block('my_code_block')

However when using this form one must be cautious to ensure that each call to ToolchainProfiler.enter_block() is matched by exactly one call to ToolchainProfiler.exit_block() in all code flows, so wrapping the code in a try-finally statement is a good idea.