Invoking Command Line Tools

As with many compilers, IREE’s compiler consists of many command line tools, some of which are designed for compiler devs and are only accessible via source builds. User level tools are distributed via the Python packages and are also accessible via dedicated Python APIs, documented here.

Core Compiler (iree-compile)

This module contains Python wrappers for various IREE command-line tools.

This top-level API provides access to the iree-compiler tool, which compiles MLIR ASM via IREE’s compiler to a supported output format (i.e. VM FlatBuffer, C source code, etc).

Example

import iree.compiler.tools

SIMPLE_MUL_ASM = """
func.func @simple_mul(%arg0: tensor<4xf32>, %arg1: tensor<4xf32>) -> tensor<4xf32> {
    %0 = "tosa.mul"(%arg0, %arg1) {shift = 0 : i32} : (tensor<4xf32>, tensor<4xf32>) -> tensor<4xf32>
    return %0 : tensor<4xf32>
}
"""

# Also see compile_file()
# There are many keyword options available.
# See iree.compiler.CompilerOptions
binary = iree.compiler.tools.compile_str(
    SIMPLE_MUL_ASM, input_type="tosa", target_backends=["llvm-cpu"])
iree.compiler.tools.compile_file(input_file: str, **kwargs)

Invokes the IREE compiler on an input file.

Parameters

  • input_file – File containing MLIR assembly to compile.

  • **kwargs – Keyword arguments corresponding to CompilerOptions.

Returns

Either a byte buffer of the compiled content or None if output_file was specified in the options.

iree.compiler.tools.compile_str(input_str: Union[str, bytes], **kwargs)

Invokes the IREE compiler with an input string.

Parameters

  • input_str – MLIR assembly to parse/compile (str or bytes).

  • **kwargs – Keyword arguments corresponding to CompilerOptions.

Returns

Either a byte buffer of the compiled content or None if output_file was specified in the options.

class iree.compiler.tools.CompilerOptions(output_file: Optional[str] = None, target_backends: Sequence[str] = (), input_type: Union[iree.compiler.tools.core.InputType, str] = InputType.NONE, output_format: Union[iree.compiler.tools.core.OutputFormat, str] = OutputFormat.FLATBUFFER_BINARY, extra_args: Sequence[str] = (), optimize: bool = True, output_mlir_debuginfo: bool = True, output_generic_mlir: bool = False, extended_diagnostics: bool = False, strip_debug_ops: bool = False, strip_source_map: bool = False, crash_reproducer_path: Optional[str] = None, enable_tflite_bindings: bool = False, enable_benchmark: bool = False)

Options to the compiler backend.

Parameters

  • output_file – Optionally save the compiled binary to a file instead of returning it.

  • target_backends – List of str names of target backends to compile into the binary. The resulting binary will run on targets that match one or more of the compiled backends.

  • input_type – The type of input legalization to perform prior to full compilation. Values can either be an InputType enum value or a case-insensitive name. Defaults to InputType.NONE.

  • output_format – Override the output format. See the OutputFormat enum. Values can either be an enum value or a case-insensitive name of the option. Typically used for debugging Defaults to OutputFormat.FLATBUFFER_BINARY.

  • extra_args – Optional list of additional arguments to pass to the compiler. Example: [“–mlir-print-ir-after-all”, “–some-other-arg”]. Individual arguments must be separate items in the list.

  • optimize – Whether to apply some default high level optimizations (default True).

  • output_mlir_debuginfo – Include debuginfo (including paths) in any saved or returned MLIR.

  • output_generic_mlir – Use the generic (and more portable) MLIR formatting for any saved or returned MLIR instead of the per-dialect custom assembly.

  • extended_diagnostics – Outputs extended information on diagnostics, potentially outputting very verbosely (defaults to False).

  • strip_debug_ops – Whether to strip high level operations used to aid debugging.

  • strip_source_map – Whether to strip source map information (used to generate better errors).

  • crash_reproducer_path – File name to output an MLIR crash dump to if there is a compiler failure.

  • enable_tflite_bindings – Support the IREE TFLite runtime bindings API shim.

  • enable_benchmark – Whether to generate instrumented binaries suitable for benchmarking.

enum iree.compiler.tools.InputType(value)

Enumeration of allowable input types to the compiler.

An instance of this enum or the string form can be passed to CompilerOptions.input_type.

Valid values are as follows:

NONE
MHLO
TOSA
TM_TENSOR
XLA
enum iree.compiler.tools.OutputFormat(value)

The output format of the compiler.

Valid values are as follows:

FLATBUFFER_BINARY
FLATBUFFER_TEXT
MLIR_TEXT

Debugging

A number of optional arguments to the compiler can be useful for debugging:

  • extended_diagnostics=True - Outputs verbose attached operations to diagnostics. Can output a large volume of information.

  • crash_reproducer_path=… some .mlir file path… - On a crash or error,a reproducer will be output at the listed path.

  • extra_args=[…] - Passes extra arguments to the compiler. Useful for various standard features of MLIR based compilers like -mlir-print-ir-after-all.

In addition, the core compiler and frontend compiler APIs have a unified mechanism for saving their temporary files, which are often useful for post mortem debugging. Since the need for this is often as part of a larger system, it is exposed both via an environment variable and an API.

In order to save all temporaries and reproducers, set the IREE_SAVE_TEMPS environment variable to a directory in which to dump artifacts. For complex programs that invoke the compiler many times, it will typically be necessary to further qualify the path, and there are a few placeholders that will be expanded:

  • {id} - A per-process monotonically increasing number for each compiler invocation. Can be overridden by the API if a better symbolic name is available (i.e. test case, etc).

  • {pid} - Process ID of the current process.

  • {main} - Basename of sys.argv[0], which is typically the name of the Python main file.

For interactive use, the following (on a Unix-like system) should provide value:

export IREE_SAVE_TEMPS="/tmp/ireedumps/{main}/{id}"

For the context manager based API, refer to the iree.compiler.tools.debugging.TempFileSaver class.

class iree.compiler.tools.debugging.TempFileSaver(temp_path_pattern: Optional[str] = None, *, invocation_id: Optional[str] = None)

Manages the saving of temp files resulting from tool invocations.

The TempFileSaver is a thread-local context bound object. An attempt to create a new one will return the most recent instance created and entered as a context manager. This allows up-stack callers to establish the policy for saving temporaries and deep implementations will inherit it.

Proper usage from users wishing to establish a saver context:

with TempFileSaver():
  # Do things with temp files.

Proper usage for implementors wishing to use an established saver context or set up a new one:

with TempFileSaver.implicit() as tfs:
  # Do things with temp files.

The outer-most creator can customize it with explicit arguments to __init__ but these will be ignored if an instance is already thread bound.

TEMP_PATH_ENV_KEY = 'IREE_SAVE_TEMPS'
alloc_optional(file_name: str, *, export_as: Optional[str] = None) → Optional[str]

Allocates an optional temporary file.

When in non-retained mode, the return value is ‘export_as’, meaning that the file is just some user specified output file.

When in retained mode, the output file will be an index-mangled variant of ‘file_name’ under the temp_path. In addition, a mapping will be added so that upon finalization, the file is also exported to ‘export_as’ if specified.

Returns None if neither a user-specified ‘export_as’ is specified nor in retained mode.

The distinction between retained temporaries and exports is to help in cases for when the caller has requested that an artifact be written to a specific place (i.e. an output file) but for debuggability, we also want to save it as a temporary. In this case, we save it to the temporary location and then conclude by moving artifacts to their final location once the saver goes out of scope.

static current()
static implicit()

TFLite Importer (iree-import-tflite)

Using the API below to access iree-import-tflite presumes that the tool itself is installed via the appropriate PIP package.

Imports TFLite binaries via the iree-import-tflite tool.

iree.compiler.tools.tflite.compile_file(fb_path: str, **kwargs)

Compiles a TFLite FlatBuffer file to an IREE binary.

Parameters

  • fb_path – Path to the FlatBuffer.

  • **kwargs – Keyword args corresponding to ImportOptions or CompilerOptions.

Returns

A bytes-like object with the compiled output or None if output_file= was specified.

iree.compiler.tools.tflite.compile_str(input_bytes: bytes, **kwargs)

Compiles in-memory TFLite FlatBuffer to an IREE binary.

Parameters

  • input_bytes – Flatbuffer content as bytes or IR string.

  • **kwargs – Keyword args corresponding to ImportOptions or CompilerOptions.

Returns

A bytes-like object with the compiled output or None if output_file= was specified.

class iree.compiler.tools.tflite.ImportOptions(output_file: Optional[str] = None, target_backends: Sequence[str] = (), input_type: Optional[str] = 'tosa', output_format: Union[OutputFormat, str] = OutputFormat.FLATBUFFER_BINARY, extra_args: Sequence[str] = (), optimize: bool = True, output_mlir_debuginfo: bool = True, output_generic_mlir: bool = False, extended_diagnostics: bool = False, strip_debug_ops: bool = False, strip_source_map: bool = False, crash_reproducer_path: Optional[str] = None, enable_tflite_bindings: bool = False, enable_benchmark: bool = False, input_arrays: Sequence[str] = (), output_arrays: Sequence[str] = (), import_only: bool = False, import_extra_args: Sequence[str] = (), save_temp_tfl_input: Optional[str] = None, save_temp_iree_input: Optional[str] = None)

Import options layer on top of the backend compiler options.

Parameters

  • input_arrays – Sequence of input array node names (if different from default).

  • output_arrays – Sequence of output array node names (if different from default).

  • import_only – Only import the module. If True, the result will be textual MLIR that can be further fed to the IREE compiler. If False (default), the result will be the fully compiled IREE binary. In both cases, bytes-like output is returned. Note that if the output_file= is specified and import_only=True, then the MLIR form will be written to the output file.

  • import_extra_args – Extra arguments to pass to the iree-import-tf tool.

  • save_temp_tfl_input – Optionally save the IR that results from importing the flatbuffer (prior to any further transformations).

  • save_temp_iree_input – Optionally save the IR that is the result of the import (ready to be passed to IREE).

TensorFlow Importer (iree-import-tf)

Using the API below to access iree-import-tf presumes that the tool itself is installed via the appropriate PIP package.

Imports TensorFlow artifacts via the `iree-import-tf tool.

iree.compiler.tools.tf.compile_module(module, saved_model_dir: Optional[str] = None, **kwargs)

Compiles a tf.Module to an IREE binary (by saving to disk).

Parameters

  • module – The tf.Module instance to convert to MLIR

  • saved_model_dir – Optional path to save the tf.Module to. The module will not be persisted on disk outside of this call if this is not provided.

  • **kwargs – Keyword args corresponding to ImportOptions or CompilerOptions.

Returns

Same as compile_saved_model().

iree.compiler.tools.tf.compile_saved_model(saved_model_dir: str, **kwargs)

Compiles an on-disk saved model to an IREE binary.

Parameters

  • saved_model_dir – Path to directory where the model was saved.

  • **kwargs – Keyword args corresponding to ImportOptions or CompilerOptions.

Returns

A bytes-like object with the compiled output or None if output_file= was specified.

class iree.compiler.tools.tf.ImportOptions(output_file: Optional[str] = None, target_backends: Sequence[str] = (), input_type: Union[InputType, str] = InputType.XLA, output_format: Union[OutputFormat, str] = OutputFormat.FLATBUFFER_BINARY, extra_args: Sequence[str] = (), optimize: bool = True, output_mlir_debuginfo: bool = True, output_generic_mlir: bool = False, extended_diagnostics: bool = False, strip_debug_ops: bool = False, strip_source_map: bool = False, crash_reproducer_path: Optional[str] = None, enable_tflite_bindings: bool = False, enable_benchmark: bool = False, exported_names: Sequence[str] = (), import_only: bool = False, import_type: Union[ImportType, str] = ImportType.OBJECT_GRAPH, saved_model_tags: Set[str] = <factory>, import_extra_args: Sequence[str] = (), save_temp_tf_input: Optional[str] = None, save_temp_mid_level_input: Optional[str] = None, save_temp_iree_input: Optional[str] = None, use_tosa: bool = False)

Import options layer on top of the backend compiler options.

Parameters

  • exported_names – Optional sequence representing the exported names to keep (object graph/v2 models only).

  • import_only – Only import the module. If True, the result will be textual MLIR that can be further fed to the IREE compiler. If False (default), the result will be the fully compiled IREE binary. In both cases, bytes-like output is returned. Note that if the output_file= is specified and import_only=True, then the MLIR form will be written to the output file.

  • import_type – Type of import to perform. See ImportType enum.

  • saved_model_tags – Set of tags to export (signature def/v1 saved models only).

  • import_extra_args – Extra arguments to pass to the iree-import-tf tool.

  • save_temp_tf_input – Optionally save the IR that is input to the TensorFlow pipeline.

  • save_temp_mid_level_input – Optionally save the IR that is input to the mid level IR.

  • save_temp_iree_input – Optionally save the IR that is the result of the import (ready to be passed to IREE).

enum iree.compiler.tools.tf.ImportType(value)

Import type of the model.

Valid values are as follows:

OBJECT_GRAPH
SIGNATURE_DEF