Joe Wood Notes

Data Processing

How different languages handle the processing of data through projection and aggregation.

Python Language

Map

Data projection from a list or array-like data structure.

numbers = [1, 2, 3, 4, 5]
squares = list(map(lambda x: x * x, numbers))
print(squares)  # [1, 4, 9, 16, 25]
evens = list(filter(lambda x: x % 2 == 0, numbers))
print(evens)  # [2, 4]

Better implemented using a list comprehension:

numbers = [1, 2, 3, 4, 5]
squares = [x * x for x in numbers]

Error Loading

Missing

Reduce

Reduction of a list or array-like data structure into a new aggregate object.

from functools import reduce

sum_numbers = reduce(lambda acc, x: acc + x, numbers)

print(sum_numbers)  # 15

Better implemented using a list comprehension:

numbers = [1, 2, 3, 4, 5]
squares = [x * x for x in numbers]
sum_of_squares = sum(x * x for x in numbers)

Missing

Deferred Execution

Execution of operations is delayed until necessary.

To defer execution of the call graph in Python, you can use lambda functions, generators, or callables to delay evaluation until explicitly invoked.

Using a lambda function:

def compute():
    print("Running compute")
    return 42

lazy_result = lambda: compute()  # Deferred
print("Before calling")
result = lazy_result()  # Triggers execution
print("Result:", result)

Using a generator:

def lazy_range(n):
    for i in range(n):
        print(f"Yielding {i}")
        yield i

for num in lazy_range(3):
    print("Got:", num)

Missing

Deferred Execution with AST

Execution is deferred and represented as an Abstract Syntax Tree (AST).

To defer execution and be able to analyze the AST of the expression passed in, use the ast and inspect modules to inspect a function or lambda expression.

import ast
import inspect

f = lambda x: x * x + 1
source = inspect.getsource(f)
tree = ast.parse(source)

print(ast.dump(tree, indent=4))

This gives you full access to the internal structure of the function before execution, enabling you to:

  • Perform static analysis
  • Rewrite or transform code safely
  • Enforce constraints or validations

For advanced manipulation, consider using libraries like astor, libcst, or RedBaron for AST round-tripping and source regeneration.

Missing