Compiling circuit models in Caphe

One of the strong points of Caphe is the ability to compile pure Python models to highly optimized C++ models. This is done using the popular module cython, which is basically a compiler to compile python code into optimized C code. For more information about Cython, please visit their web-site.

The result of compiling circuit models is typically a 10-100-fold increase in simulation speed compared to a pure Python model (depending on the specific case).

Note

IPKISS.flow is bundled with a C++ compiler, so nothing additionally has to be installed in order to take advantage of this.

Note

This is an advanced topic. This topic is relevant when you start performing large circuit simulations or large parameter sweeps, and you want to optimize the simulation speed.

Files used in this guide:

  • waveguidemodel.py : this file contains a simple waveguide model, used for explaining how to compile models.
  • waveguidemodel_extraphase.py : in this example, we added additional variables, and include type information to keep the model efficient.

Optimizing models

We will illustrate how to optimize models using a simple example. In the code below, we have a simple model for a photonic waveguide. It has a property length and effective index (n_eff). The transmission contains a phase rotation, determined by the wavelength, length of the waveguide, and it’s effective index:

from ipkiss3 import all as i3
import numpy as np
pi = np.pi
exp = np.exp

class Waveguide(i3.PCell):
    class Netlist(i3.NetlistView):
        def _generate_terms(self, terms):
            terms += i3.OpticalTerm(name="in")
            terms += i3.OpticalTerm(name="out")
            return terms
        
    class CapheModel(i3.CapheSModelView):
        length = i3.PositiveNumberProperty(default=100.0, doc="Length of the waveguide")
        n_eff = i3.PositiveNumberProperty(default=2.86, doc="Effective index of the waveguide")

        
        #@i3.compile_function()
        def _calculate_S(self, environment, term1, term2, mode1, mode2):
            phase = 2 * pi / environment.wavelength * self.n_eff * self.length
            if (term1.name == 'in' and term2.name == 'out') or (term1.name == 'out' and term2.name == 'in'):
                return exp(1j * phase)
            return 0
            
w1 = Waveguide()
w1_cm = w1.CapheModel(length=100, n_eff=2.86)
S = w1_cm.get_smatrix(wavelengths=[1.55])          
print S['in', 'out']

This will print [-0.99486932-0.10116832j].

To optimize this model (i.e., compile it to C++), simply add the compile decorator to the _calculate_S function:

class Waveguide(i3.PCell):
    class Netlist(i3.NetlistView):
        def _generate_terms(self, terms):
            terms += i3.OpticalTerm(name="in")
            terms += i3.OpticalTerm(name="out")
            return terms

    class CapheModel(i3.CapheSModelView):
        length = i3.PositiveNumberProperty(default=100.0, doc="Length of the waveguide")
        n_eff = i3.PositiveNumberProperty(default=2.86, doc="Effective index of the waveguide")

        @i3.compile_function()
        def _calculate_S(self, environment, term1, term2, mode1, mode2):
            phase = 2 * pi / environment.wavelength * self.n_eff * self.length
            if (term1.name == 'in' and term2.name == 'out') or (term1.name == 'out' and term2.name == 'in'):
                return exp(1j * phase)
            return 0

When running the code, execution will take a bit longer first, but the output will be the same: [-0.99486932-0.10116832j].

Each subsequent run will now use the optimized method.

Obviously, as long as your circuit is very small, you will not see a large speed gain (in this case, when making a wavelength sweep, at most a factor 2 speed is gained). Once your circuit becomes more complex, with more components, and different models, the speed gain can go up to a factor 100.

Limitations

Being able to compile code to C++ comes with certain limitations. Most importantly, all properties used in your circuit model must be valid integers, floats, complex numbers or numpy arrays. Only these properties can be interpreted by the compiler. This means you are safe when using IntProperty, NumberProperty, ComplexNumberProperty (or any other property derived from this, such as PositiveNumberProperty). If for some reason you need to use a DefinitionProperty (which is not recommended), make sure the type of the variable is always consistently the same. For example, if you first supply the value ‘1’ to a DefinitionProperty, then run a simulation, then supply a value ‘1.0’ and re-run the simulation, this will lead to undefined behavior.

Note

The standard typed properties (such as IntProperty, NumberProperty, ...) automatically cast the value to the correct type. For example, when a ComplexNumberProperty is given a integer value, it is casted to the complex equivalent.

Numpy arrays are also allowed. The dimensions should not change during the execution of the simulation.

Optimization strategies

If your simulations are running slow, you can try some of the following tricks shown below (in order of importance):

  1. Make sure there are no print statements inside the _calculate_S function. Obviously, print statements are very powerful when debugging your simulation, but once you want to do large simulations, they can significantly slow things down.
  2. Make sure that each model you use in your circuit is compiled. Even if 1 model in your circuit is not compiled, it will immediately become the bottleneck.
  3. Preferrably, all variables used during simulation are ‘simple’ types, such as integers, floats, complex numbers or numpy matrices. Other types cannot be optimized efficiently.
  4. Finally, you may want to check whether your model is written efficiently. For this, please check the following sections (how optimizing works and debugging slow models).

How optimizing works

Executing the code above with the compile decorator will create a .temp.compiled folder, located in the folder where the simulation is started. This folder contains all compiled models. Below is a table that summarized which files are generated:

Number File Purpose
1 ipkiss_caphe_{NAME}_calculate_S.py A model that can be interpreted by Caphe
2 _optimized_cy_{NAME}_calculate_S.pyx Generated from (1): This file contains the code that will be optimized
3 _optimized_cy_{NAME}_calculate_S.cpp/h Generated from (2) by Cython. It contains C++ code that will be compiled and linked into Caphe
4 _optimized_cy_{NAME}_calculate_S.html Generated from (2) by Cython. It contains a comparison between the Python and generated C++ code. Color codes are used to indicate how fast the code runs (white: fast, dark yellow: slow)
5 _optimized_cy_{NAME}_calculate_S.pyd/so Generated from (3) by the gcc compiler. This file is loaded by Caphe.

It’s not the purpose to modify these files, because they are all automatically generated. It is worth opening the html file, because it can give insight in why a model is fast/slow.

Debugging slow models

This section explains how you can check the generated models. Please go to the .temp.compiled folder in your working directory. Look for html files with the name of your model (for the waveguide example here, you should look for _optimized_cy_Waveguide_CapheModel_calculate_S.html). It shows the python lines and the corresponding C++ code. Yellow code indicates slow code (the darker, the slower).

We will use the example from this guide to show how to debug a model. If you look for the html file in the compilation folder, you’ll see something that looks like this:

Initial model compiled.

Fig. 88 In the initially compiled model, there’s no yellow code, so this code will run fast.

Suppose now, we want to add an additional phase extra_phase, and we want to multiply the return result by c_val:

@i3.compile_function()
def _calculate_S(self, environment, term1, term2, mode1, mode2):
   extra_phase = pi / 2
   c_val = 1j
   phase = extra_phase + 2 * pi / environment.wavelength * self.n_eff * self.length
   if (term1.name == 'in' and term2.name == 'out') or (term1.name == 'out' and term2.name == 'in'):
       return c_val * exp(1j * phase)
   return 0

This looks trivial, right? Nonetheless, if you run simulations, you’ll notice it runs much, much slower! The reason can be found by looking again to the generated html file:

Using a generic Python object will reduce the model speed considerably.

Fig. 89 Using a generic Python object will reduce the model speed considerably.

As you can see, the variable extra_phase was interpreted as a generic Python object, and the generic Python C API is used to perform all calculations following - instead of using simple C algorithmics on C native datatypes. How can we let Cython know that this should be a simple double? This can be done by adding the code cdef double extra_phase to the function call. In practice, this is supplied to the compiler decorator (and while we’re at it, we’ll also provide type information for c_val):

@i3.compile_function(append="cdef double extra_phase\n"+
                            "cdef complex c_val")
def _calculate_S(self, environment, term1, term2, mode1, mode2):
   extra_phase = 1
   c_val = 1j
   phase = extra_phase + 2 * pi / environment.wavelength * self.n_eff * self.length
   if (term1.name == 'in' and term2.name == 'out') or (term1.name == 'out' and term2.name == 'in'):
       return c_val * exp(1j * phase)
   return 0

You will notice that in the generated pyx file, some extra typing information has been added to the function. Below is the result of the html file:

Note

If you want to specify multiple variables using the append argument, you have to separate them with an ‘enter’ (i.e., '\n'). For example: cdef double extra_phase\ncdef complex c_val. In the previous example, we split the lines for readability.

Fixing the slow model.

Fig. 90 We can fix the slow model by specifying the data type of the variables we use.

Great, this will again run smoothly!