Introduction to nodes

The first thing to do if you want to create a node is to import the Node base class:

Your first node

from nodify import Node

Let’s say that after years of developing you have come up with a function that performs a very complex calculation:

def my_sum(a: int, b: int):
    print(f"SUMMING {a} + {b}")
    return a + b

We have good news, you have already done the hardest part!

Converting it to a node class is as simple as:

my_sum = Node.from_func(my_sum)

Using a decorator

You can also use Node.from_func as a decorator:

@Node.from_func
def my_sum(a: int, b: int):
    print(f"SUMMING {a} + {b}")
    return a + b

Lazyness and updating

There are some important differences between your normal function and the node that you have created:

• Nodes compute lazily. That is, they only run when you explicitly ask for the result.

• Node instances represent a computation. The inputs of this computation can be updated to recompute the function.

• Nodes support batching.

• Nodes can be connected to other nodes to create workflows.

That is why when you call your new node class you will get a node instance, not directly the result:

my_sum(2, 5)

<__main__.my_sum at 0x7f38380e88d0>

If you want the result, you need to call .get() on it. This will trigger the computation.

sum_value = my_sum(2, 5)

sum_value.get()

SUMMING 2 + 5

7

The result is then stored in the node.

If you keep requesting it the node will not need to recompute, it will just return the result. You can see that the message is not printed:

sum_value.get()

7

You can update the inputs of a node by using the update_inputs method:

sum_value.update_inputs(a=8)

<__main__.my_sum at 0x7f38380eafd0>

As you can see, the node hasn’t recomputed anything yet. It will not do so until you request the result again!

sum_value.get()

SUMMING 8 + 5

13

Linking nodes

At this point, you might be asking yourself: Why would I complicate my functions so much?

Well, the power of nodifying does not lie in each individual node, but the collective effect they acheive when they are connected.

Let’s say that our sum_value can be used to compute some other thing. E.g. a multiplication:

@Node.from_func
def my_multiplication(a: int, b: int):
    print(f"MULTIPLYING {a} * {b}")
    return a * b


final_value = my_multiplication(sum_value, 4)

Again, no computation is performed until the result is explicitly requested:

final_value.get()

MULTIPLYING 13 * 4

52

We can now update the inputs of the first sum:

sum_value.update_inputs(a=2)

<__main__.my_sum at 0x7f38380eafd0>

And requesting the final value will now trigger the recomputation of all outdated nodes on which it depends.

final_value.get()

SUMMING 2 + 5
MULTIPLYING 7 * 4

28

Just like that, you have created your first node graph!!

Computation context

The context of a node defines how the node behaves, but not its result. You can, for example, update it so that it does not compute lazily.

For example, by setting the context of our final value to lazy=False:

final_value.context["lazy"] = False

We are telling it to update as soon as it notices that it has become outdated.

A node can become outdated because one of its inputs has changed, but also because the inputs of the nodes on which it depends have changed. Since our final value depends on the sum value, if we update the inputs of the sum, everything will be recomputed.

sum_value.update_inputs(a=3)

SUMMING 3 + 5
MULTIPLYING 8 * 4

<__main__.my_sum at 0x7f38380eafd0>

We also provide a context manager to modify the context temporarily:

from nodify import temporal_context

with temporal_context(lazy=True):
    sum_value.update_inputs(a=4)

The nodes were told to be lazy within the context manager, so nothing has been recomputed as expected.

Batching

Another interesting fact is that nodes support batching. When they receive a batch (nodify.Batch), they also return a batch:

from nodify import Batch

factors = Batch(1, 2)

final_value.update_inputs(b=factors)

SUMMING 4 + 5
MULTIPLYING 9 * 1
MULTIPLYING 9 * 2

<__main__.my_multiplication at 0x7f383804a2d0>

As you can see, the multiplication has been performed twice, but the sum just once. Batched computations are performed only when needed. This can come very handy in workflows to help save computation and memory!

Now, if you get the result, you will get a batch:

final_value.get()

<nodify.node.Batch at 0x7f383bfb2490>

Batches are also propagated through the node graph. Let’s add yet another layer on top of our computation, a subtraction:

@Node.from_func
def my_sub(a: int, b: int):
    print(f"SUBTRACTING {a} - {b}")
    return a - b


shifted = my_sub(final_value, 1)

And ask for the result:

shifted.get()

SUBTRACTING 9 - 1
SUBTRACTING 18 - 1

<nodify.node.Batch at 0x7f3838101b90>

You can retreive the values of a batch like:

list(shifted.get())

[8, 17]

Batches can also interact with each other. Let’s also pass a batch to the second argument of the subtraction:

shifts = Batch(1, 2)
shifted.update_inputs(b=shifts)

<__main__.my_sub at 0x7f38381002d0>

And get the result:

shifted.get()

SUBTRACTING 9 - 1
SUBTRACTING 18 - 2

<nodify.node.Batch at 0x7f38380f6150>

As you can see the batches have been zipped. But we can change that behavior with, you’ve guessed it, context!

with temporal_context(batch_iter="product"):
    shifted.get()

MULTIPLYING 9 * 1
MULTIPLYING 9 * 2
SUBTRACTING 9 - 1
SUBTRACTING 9 - 2
SUBTRACTING 18 - 1
SUBTRACTING 18 - 2

And now the batches have interacted with a product, creating 4 results!

Implicit node creation

Nodes will also be created if you perform operations on existing nodes:

implicit = shifted + 2
implicit.get()

<nodify.node.Batch at 0x7f3821caa910>

This might come handy if you don’t want to convert functions to nodes, you can use them directly to generate new nodes as long as they only have operations in which the node can determine the behavior. This includes:

• Arithmetic operations.

• Comparisons.

• Getting items and getting attributes.

• Numpy functions.

It is particularly interesting to use Constant for that. Let’s say that we have a function that we have imported from somewhere:

def some_function(a, b):
    return a[0] + b == 3

Then we create a constant and pass it through it:

from nodify import Constant

val = Constant([2, 3, 4])

result = some_function(val, 3)
result

<nodify.syntax_nodes.CompareNode at 0x7f38218fd690>

And now we have a node graph, we can just call get on the result node:

result.get()

False

Update the constant and get it again:

val.update_inputs(value=[0, 2])
result.get()

True

You can even do that with batches:

vals = Batch(0, 1, 2, 3)
batched_result = some_function([0, 1], vals)
list(batched_result.get())

[False, False, False, True]

Graph to python code

Another useful feature of nodes is that you can convert all their tree to python code:

from nodify.conversions import node_to_python_script

code = node_to_python_script(result, as_function=True, function_name="is_correct")

print(code)

def is_correct(obj=[0, 2]):
    left = obj[0]

    left_1 = left + 3

    return left_1 == 3

The variable names are not super meaningful (you can solve that), but it works :)

Notice how in this node graph, the b argument of some_function was fixed to 3, that’s why b is not an argument of the defined function, but instead its value is hardcoded.