Project

General

Profile

OpenWalnut Module Concept

In this article, we give an overview of OpenWalnut's module concept. We begin with describing the module in the context of OpenWalnut and the possibilities it provides, its tasks, and limitations. We will describe how modules can be combined to even more complex structures and how modules communicate with the user and other modules.

This article is NOT about writing modules. It is about understanding the mechanisms behind it. It is strongly recommended to understand these basics before getting started with module development.

Data Flow Networks and Modules

To understand what modules in OpenWalnut are, it is important to understand the way OpenWalnut handles and processes data. Basically, OpenWalnut uses the principle of Data Flow Networks. In these kind of networks you usually have sources, sinks and processor-nodes. They are organized as a graph and therefore describing the flow and processing of data.

In OpenWalnut, we call the data flow network simply module graph. Modules can serve as a source, a sink and a processor. Typically, data loaders are sources, whereas visualization algorithms usually can be seen as sinks. But OpenWalnut does not divide modules this strict. It is even possible to have modules which are source and sink at the same time. This ensures flexibility and does not limit the possibilities of the module developer.

Modules transfer data to other modules using so called connectors. They allow the connection of one module's output to another module's input. This way, OpenWalnut reflects the flow of data between the modules. A quite simple example is shown in the following figure:

Example Module Graph

The data module loads some data from disk and outputs it on its output connector. The Gauss Filtering module takes this data and smooths it. The resulting data is then transferred to a module calculating the gradient vectors in the data. The smoothed data is also transferred to the Isosurface module directly. The Isosurface module renders the data and uses the calculated gradients for proper lighting.

Now, one question may arise: How does the module communicate with the user? How is it possible to adjust the iso-value? In OpenWalnut, this is done via properties. Each module in OpenWalnut can have such properties. They define how an algorithm should behave or what the parameters of an algorithm are. Now, lets assume you change the number of Gauss iterations in the Gauss Filtering module via the corresponding property. The Gauss module now recalculates the data and updates its output. This updated is then propagated through the graph. All modules connected to this output will wake up from their sleeping state and update their outputs accordingly to the new, modified input. At this point, it comes in handy that each module runs in its own thread. They go asleep if nothing is to be done and wake up if the user modifies a property or some data has updated on one of the inputs. This creates an inherent multi-tasking and, therefore, utilizes today's multi-core architectures.

By keeping this in mind, you can directly dive into module development! For this, you should also have a look at our extensively documented example module source:src/modules/template and the external module development tutorial. The following sections cover several more advanced topics and might not yet be that interesting for you.

Data Exchange: Connectors

In the last section, we already gave an overview on how modules are organized and how they communicate. With this in mind, this section will dive deeper into the specifics of our input and output connectors.

The Basics

The most important things to know are:

  1. Each connector is either input or output
  2. Each connector is typed
  3. The typing respects class inheritance hierarchy during runtime

In code, connector declarations looks like this:

// inputs
WModuleInputData< WDataSetVector >::PtrType myInput;
WModuleInputData< WDataSet >::PtrType myInputLessStrictTyped;

// outputs
WModuleOutputData< WDataSetScalar >::PtrType myOutput;
WModuleOutputData< WDataSetVector >::PtrType myOutputVector;
WModuleOutputData< WDataSet >::PtrType myOutputLessStrictTyped;

This declares two inputs and three outputs. Their template parameters define the type you can send/receive with this input/output. What are the practical implications? Lets discuss the possibilities.

NOTE: at this point, we assume the data instances put into the output connectors have the same runtime type as the connector type. This is important here, due to point 3 in the above list. Below the following examples, we will have a deeper look into these cases where the runtime type plays an important role.

  • myOutputVector -> myInput
    • This works since both types match.
  • myOutput -> myInput
    • This won't work, since this would send a scalar field to an input requesting a vector field.
  • myOutput -> myInputLessStrictTyped:
    • This will work, because connectors respect class hierarchies (3). A WDataSetScalar is derived from WDataSet. So, the input only wants to get some arbitrary dataset. A scalar field is an arbitrary datatset.
  • myOutputLessStrictTyped -> myInput:
    • This won't work. Although connectors respect class hierarchy, they only do it in direction of input to output. If this looks strange, just take a minute and think about this example. You expect a vector field on myInput but you only receive a very unspecific WDataSet.
  • myInput -> myInputLessStrictTyped or myOutput -> myOutputLessStrictTyped
    • Although the types match (in class hierarchy), you cannot connect inputs with inputs or outputs with outputs
  • myOutputVector -> myInputLessStrictTyped and myOutput -> myInput
    • Works. It is no problem to connect multiple inputs to only one output
    • The other way around is not possible. It is not possible to connect two outputs to one input

Ok, this might be a bit confusing but OpenWalnut decides whether to connectors are compatible in exactly the same way. Each time you want to connect the connectors of two modules, OpenWalnut checks them against the above criteria.

An interesting thing to know here is the fact that OpenWalnut checks the type criteria during runtime! In code, this means you can do:

boost::shared_ptr< WDataSetVector > someVectors = .....
myOutputLessStrictTyped->updateData( someVectors );


This sets a vector dataset to an output which only wants a WDataSet. Ok. Not very interesting. But now assume you do this in your module. Furthermore assume OpenWalnut now checks compatibility of a connection between myOutputLessStrictTyped and myInput. Are they compatible? A fast look-up in the above table would say NO. But that is wrong. OpenWalnut would say YES because the type check is done during runtime. And you can dynamic_cast the WDataSet pointer in myOutputLessStrictTyped to a a WDataSetVector, because it IS a WDataSetVector. This allows some interesting things. For example, you can create modules where you not know the exact type of output. A nice example is a data loader. For example, the NIFTI format allows storing scalar and vector data (besides others). Your data loader can now provide an output of type WDataSet because you do not know the exact type contained in the file the user wants to load.

Handle Events properly

The last section showed the different types of connectors and how they handle the transferred data types. This section is about the correct handling of events in input connectors. For this, assume you have a simple module with one input connector. You usually define these connectors in the module's connectors() method. Lets call our connector m_input.

At first, you will never get notified of anything in your connector automatically. You need to specify in what you are interested in. Most modules do it this way:

void WMMyModule::moduleMain()
{
    // ...
    m_moduleState.add( m_input->getDataChangedCondition() );
    // ...
    ready();
    while( !m_shutdownFlag() )
    {
        m_moduleState.wait();
        // ...
        // how do we handle the wake-up call?
    }
}


As you remember, each module runs in its own thread. To let threads wait for events very CPU-friendly, so called condition variables are used. Have a look at Wikipedia for more details about this topic. Each input connector provides such a condition variable, which gets notified if something has changed on the input. This means, the main loop of the module waits for events (notified conditions) using its m_moduleState. Each time the input got some change, they thread is woken up. How do we handle the wake-up call? Basically, there can be three things here:
  • the value in m_input is NULL
    • Because the input was disconnected
    • Or the sending module set its output to NULL
  • the value in m_input is the same
    • Because m_moduleState is also used for properties and your custom condition variables
  • the value in m_input is really different

These three cases need to be handled. Usually this is done this way:

void WMMyModule::moduleMain()
{
    // ...
    m_moduleState.add( m_input->getDataChangedCondition() );
    // ...
    ready();

    // stores the current dataset
    boost::shared_ptr< WDataSetScalar > currentDataSet = boost::shared_ptr< WDataSetScalar >();

    while( !m_shutdownFlag() )
    {
        m_moduleState.wait();
        // ...

        boost::shared_ptr< WDataSetScalar > newDataSet = m_input->getData();
        bool dataValid   = ( dataSet );
        bool dataUpdated = m_input->updated() && ( newDataSet != currentDataSet );
        currentDataSet = newDataSet;

        // valid data available?
        if( !dataValid )
        {
            // remove renderings if no data is available anymore and set own outputs to NULL
        }
        else if( dataUpdated )
        {
            // re-calc something and update rendering
        }
    }
}

This code already shows that m_input->updated() does not say whether the actual value in the connector has changed. It's purpose is to tell you whether m_input has triggered a wake-up of the thread. The remaining code is self-explanatory. If the new value is NULL, set your own outputs to NULL and remove your renderings, because you do not have any valid data anymore. If the data is not invalid and really has changed, do all the updates needed with the new data.

This is the recommended way of working with input connectors. The problem with condition variables often is the uncertainty who has caused a wake-up. If you want more fine-grained information on events on input connectors, you can subscribe to certain callbacks. The input connectors therefore provide a method named subscribeSignal. It allows subscribing callbacks fired during connection, disconnection and data update. For more details, have a look at source:src/core/kernel/WModuleConnectorSignals.h and source:src/core/kernel/WModuleInputConnector.h.

Transfer your own Data

Have you red the external module development tutorial? Maybe you want to write a toolbox with a lot of modules which handle some awesome new medical data type: NNSHRSEWAD scanning data (this is the abbreviation for No Noise Super High Resolution See Everything Without Any Drawbacks). For this, you wrote an all new dataset type: WDataSetSuperData. Now you wonder how to use this type with our connectors. A fast look into source:src/core/dataHandler/WDataSet.h reveals the needed thing: you need to derive from WTransferable, which actually is a WPrototyped. Technically, you do not need to derive from this class if you derive from WDataSet or any of its derived classes.

As described earlier, OpenWalnut does a dynamic type check during compatibility check of two connectors. This type check is based on a prototype instance of the type specified to the connector. Why is this prototype instance needed? Assume we do not have this prototype instance. If OpenWalnut now checks the compatibility of two connectors and the output connector has some data, everything is nice. The prototype is not needed since the connectors can check compatibility by casting this data (previously set data pointer) to the target type (the type of the input connector). But if the user connects an output, which not yet has any data? How to check compatibility? Here, the prototype comes into play. In these cases, the prototype instance of your WDataSetSuperData is used in the output connector. To provide this prototype, you need to provide a static method getPrototype. This method needs to return a shared_ptr to a WPrototype, which is of dynamic type WDataSetSuperData:

// NOTE: if you do not derive from WDataSet or one of its derived classes, you need to derive from WTransferable.
class WDataSetSuperData : public WDataSet
{
public:
    // ......

    /**
     * Gets the name of this prototype.
     *
     * \return the name.
     */
    virtual const std::string getName() const;

    /**
     * Gets the description for this prototype.
     *
     * \return the description
     */
    virtual const std::string getDescription() const;

    /**
     * Returns a prototype instantiated with the true type of the deriving class.
     *
     * \return the prototype.
     */
    static boost::shared_ptr< WPrototyped > getPrototype();
private:
    // ......

    /**
     * The prototype as singleton.
     */
    static boost::shared_ptr< WPrototyped > m_prototype;
}

And in the CPP file, you need to implement the function:

// prototype instance as singleton
boost::shared_ptr< WPrototyped > WDataSetSuperData::m_prototype = boost::shared_ptr< WPrototyped >();

boost::shared_ptr< WPrototyped > WDataSetSuperData::getPrototype()
{
    if( !m_prototype )
    {
        m_prototype = boost::shared_ptr< WPrototyped >( new WDataSetSuperData() );
    }

    return m_prototype;
}

const std::string WDataSetSuperData::getName() const
{
    return "Useful name";
}

const std::string WDataSetSuperData::getDescription() const
{
    return "Provide a useful description here, telling the user what your data is.";
}


During the first compatibility check, the prototype instance is created and returned. The compatibility check then does a simple dynamic_cast< InputConnectorType >( getPrototype().get() ). This way, you ensure that your data can be transferred and through the OpenWalnut connector mechanism and that the type hierarchy is respected during connector compatibility check. The name and description should be overwritten and should return some useful text describing your datatype. Although this has no technical purpose, these information might be interesting for the user.

User Communication: Properties

The last sections gave some insight into the connector mechanism. This section tries to give you an similar deep insight into our property mechanism.
TODO: write me

Combining Modules: Module Containers

TODO: write me

graph_example.png - Example Module Graph (10.8 KB) Sebastian Eichelbaum, 04/11/2012 11:17 AM