Flow: Visual Graph Creation
You can use command tofu flow to start a graphical user interface in which UFO
tasks are represented as nodes which you can connect together. Once you have
created your flow you can execute it.
Nodes
An operation on data is represented by a node in a flow. A node has inputs and outputs, which have data types. An input or output of a node is represented by a port, which is a circle on the left of the node in case of an input and on the right in case of output. Every port has a data type which is represented by color. There are two data types:
UFO: you can connect all UFO nodes together
Array: a numpy array which comes out from UFO’s
memory_outnode and may be used to visualize the processing result byimage_viewer
A node may have properties (almost all UFO nodes do, e.g. path in the
read node) which are listed and can be set inside the node. If you hover the
mouse over a property field, a tooltip will be shown describing that property.
When you right click on a node which holds properties, a context menu pops up
and let’s you choose which properties you want to be visible and which not. Some
nodes, like general_backproject have many properties, many of which may be
considered expert options which are not needed most of the times. By hiding
these properties, you can avoid clutter. There is a pre-defined set of
properties, which are shown by default. When you create a node in the scene,
this setting is applied and you can check which properties are hidden by default
by clicking on a node right after its creation. In case a node doesn’t have
properties, right click either doesn’t take effect or pops up a context menu
relevant for that node. E.g. image_viewer’s context menu allows you to
configure the viewer’s behavior.
A node might implement an action on a double click, e.g. read node opens a
dialog allowing you to choose the data path, image_viewer pops up an
external image window which can be enlarged and put on another display for
convenience. Current nodes which implement double clicks are:
Composite: opens a new window with a scene displaying internal composite nodesread: opens a dialog which allows you to choose the inputpathwrite: opens a dialog which allows you to choose the outputfilenameimage_viewer: opens the image in a new window
read node currently supports an auto fill option, which may be invoked via
the main menu bar. The node sets its number property to the number of
detected images found in the specified path.
UFO Nodes
An UFO node represents an UFO task and holds properties which are the Properties of this UFO task. Please check the UFO Filters Reference for the complete list of UFO tasks and their properties. When you create an UFO node, its properties are the default properties of the encapsulated UFO task.
Composite Nodes
In order to reduce clutter, you can combine several nodes in a composite node (main menu’s Nodes->Create Composite) and you can also nest composites, i.e. have a composite node and create another composite node with the first one inside. Internal nodes are listed as groups in the composite node in the scene and similarly to property nodes, you can show and hide different internal nodes from the listing. The input and output ports of a composite node are the ports of its internal nodes which are not connected at the time of composite node’s creation.
A double click on a composite node opens its internal nodes in a separate window, where you can edit their properties but you can’t add new nodes or change connections. You can open this window also by pressing Nodes->Edit Composite in the main menu.
In order to store a composite node for later usage, you can export it into a file via the main menu’s Nodes->Export Composite. You can import composite node definitions by Nodes->Import Composites, which are then available in the flow scene’s context menu in the Composite category.
There are several pre-defined composite nodes available via the scene’s context menu (category Composite), they are:
CFlatFieldCorrectencapsulates readers and averagers and theflat_field_correctnode itselfCPhaseRetrieveencapsulates padding, fourier tranformation and the phase retrieval itself
General Backproject
This is a versatile back projection node which can reconstruct tomographic,
laminographic, parallel and cone beam data. It has one parameter which is not
part of the UFO task, slice-memory-coeff. This parameter sets the fraction
(0 - 1) of a graphic card’s memory which will be used to store the reconstructed
volume. If you are working with graphic cards which have other processes running
on them and these processes use a lot of memory, then you might need to reduce
this parameter.
Phase Retrieval
retrieve_phase node may have varying number of inputs in order to support
multi-distance phase retrieval. You specify the number of inputs in a dialog
when you create the node. If you specify more than one input, the retrieval
method will be the multidistance contrast transfer function and the method
field will be fixed to ctf_multidistance. In this case, fields distance-x
and distance-y will be disabled. If you specify one input, you may choose
different methods via the method field. In this case, you can either specify
one value in the distance field, or specify separate distances for x and
y directions via distance-x and distance-y fields (they take
precedence over distance field in case they are both non-zero).
Image Viewer
image_viewer lets you display the results of your flow. It is composed of
the image itself and three text boxes with sliders, which allow you to specify
the image index shown, the black point and white point. In case only one image
is input the first slider is hidden. Right click on the node opens a context
menu which allows you to reset the black and white points (Reset), set them
automatically (Auto Levels) and specify whether they should be automatically
adjusted when new images are on input or left unchanged (Auto Levels on New
Image). Double click opens the image in a new window by using the PyQtGraph
library. In case a separate window is open, image index, black and white point
settings can be set eigher in the flow node or in the window and they are
reflected in both the node and the window.
Flows
On right click in the flow scene a context menu will pop up and you will be able
to add nodes. Then you can connect them by dragging a node’s output port into
another node’s input port if those ports have the same data type, which are
distinguished by port colors. By connecting node ports you create your flow
which you may later execute. Every node in the scene must have a unique
caption, so when you create a read node, the caption will be Read,
when you create another read node, the caption will be Read 2 and so on.
This is important for setting property links explained below.
The roots of the flow in the scene must be UFO nodes and leaves may have UFO or
Array type. It is not possible to go from UFO to Array and back to UFO,
i.e. the UFO portion of the flow in the scene must be one contiguous component
of the flow. There may be only one flow in the scene and it must be completely
connected (there can’t be disconnected ports, e.g. flat_field_correct’s
darks port).
You can delete the current flow by pressing Flow->New, you can save a flow into a flow file (.flow) by Flow->Save and open such files by Flow->Open.
Property Links
A property of a node might depend on another node’s property, e.g. the number of
dimensions of an ifft node depends on the number of dimensions of the
predecessing fft task. In order to reduce the number of properties you need
to set, you can link properties together, i.e. when you set one node’s
property, all the linked nodes’ properties will be updated (e.g. when you change
the number of dimensions of an ifft node, the number of dimensions of the
linked fft node will be updated as well.
You can create property links in the Property Links window (open via main menu
bar’s View field). At the top of the window, there is a tree view of the nodes
in the scene. Its items are the nodes in the flow scene, and in case there are
composites, they are listed recursively. The last level of the view are the
properties of the nodes in the flow. You can drag these properties into the list
in the second half of the window to start creating links. If you drag a property
to a new row or a row doesn’t exist yet, it is automatically added. If you drag
a property into an existing row (over an existing cell), it is appended to this
row and a link is created. Links are allowed only for properties with compatible
data types, e.g. you cannot link read’s path (a string) to fft’s
dimensions (a number). Also keep in mind that nodes which are able to
process batches have their fields which are responsible for receiving different
batches (e.g. number of the memory_out node) have string data type (so
that you can type {region} inside)
Execution
Execution of the flow starts with executing the UFO part of the flow, and if
there is a memory_out and subsequent nodes, they get the result of the
UFO processing as the batches are finished (or just one batch if no
batch-capable nodes are in the flow). You start it by invoking main menu bar’s
Flow->run action. You can abort the execution but invoking Flow->abort.
Batch Processing
Some nodes require a lot of GPU memory and they can’t process all the input data
at once (e.g. general_backproject). Based on your system, they can split the
work on their own and tell the execution mechanism to run multiple batches. If
your system has multiple GPUs, tofu flow may create several batches and each
of these batches may be executed on one or more cards in your system.
Currently, only one batch processing task is allowed in the flow and only
general_backproject supports batch processing.
In case your flow contains a node which is able to produce batches, then your
consumer nodes must be able to process batches and they must be notified of the
fact that they will get more batches on input. Currently, write and
memory_out support batches and this is how you set them up for it:
write:filenamemust contain {region} somewhere in it, e.g. slices-{region}.tifmemory_out:numberfield must be set to {region}
The {region} template is then replaced by the current batch identifier provided by the producer node which is capable of batch processing, e.g. slices-0.tif, slices-100.tif and so on.
If there is no node capable of producing batches, this is how you set them up for normal, non-batch processing:
write:filenamefield set to normal file name, e.g. slices.tifmemory_out:numberfield set to the number of input images
Python Console
Main menu’s View->Python Console opens up a Python interpreter console with
attribute scene set to the flow scene, which allows you to interact with the
nodes programatically, see qtpynodeeditor docs
more details on flow scene functionality.