Skip to main content

2 posts tagged with "map"

View All Tags

whereq.com is open source — the map, the crawler, and everything it reads

· 8 min read
whereq.comwhereq.com

A pin is a very small sentence​

Stand on a street in Toronto and look at the address in front of you. A map gives you one claim about it: there is a pin here. A coffee shop. A hospital. A school. Maybe a parking garage.

But the place itself is a pile of claims, and the pin is the least interesting one of them.

Which school catchment is that address in? Can I walk to a library, and which one? Is the fuel station on the corner charging 1.42or1.42 or 1.79? Is that building inside a ward, or inside a neighbourhood, or in a ward and a neighbourhood? Did anything happen on this street last week?

Understanding map() with Multiple Iterables and Lambda Functions

· 10 min read

📌 Core Concept Breakdown​

The Code Explained​

a = [1, 2, 3]
b = [10, 20, 30]
sums = list(map(lambda x, y: x + y, a, b))

Result: [11, 22, 33]

🔍 Step-by-Step Execution​

Visual Execution Flow​

Step 1: Initialize
a = [1, 2, 3]
↑ ↑ ↑
b = [10, 20, 30]
↑ ↑ ↑
| | |
Step 2: map() pairs elements
map(lambda x, y: x + y, a, b)
| | |
| | |
Step 3: Process each pair
Pair 1: x=1, y=10 → 1 + 10 = 11
Pair 2: x=2, y=20 → 2 + 20 = 22
Pair 3: x=3, y=30 → 3 + 30 = 33

Step 4: Convert to list
sums = [11, 22, 33]

What's Happening Internally​

# Equivalent explicit code
result = []
for i in range(min(len(a), len(b))): # map() stops at shortest iterable
x = a[i]
y = b[i]
sum_result = (lambda x, y: x + y)(x, y) # Apply lambda
result.append(sum_result)

sums = result # [11, 22, 33]

🎯 Anatomy of the map() Function​

map() Function Signature​

def map(function: Callable, *iterables: Iterable) -> map object:
"""
Parameters:
function: A callable that takes as many arguments as there are iterables
*iterables: One or more iterables to process in parallel

Returns:
An iterator that yields function(i1, i2, ..., in)
where i1 from iterable1, i2 from iterable2, etc.
"""

Multiple Iterables Behavior​

# With single iterable
result1 = list(map(lambda x: x*2, [1, 2, 3]))
# result1 = [2, 4, 6]

# With two iterables
result2 = list(map(lambda x, y: x+y, [1, 2, 3], [10, 20, 30]))
# result2 = [11, 22, 33]

# With three iterables
result3 = list(map(lambda x, y, z: x+y+z,
[1, 2, 3],
[10, 20, 30],
[100, 200, 300]))
# result3 = [111, 222, 333]

📊 Visual Representation​

Diagram: map() with Multiple Iterables​

┌──────────────────────────────────────────────────────┐
│ map() Parallel Processing │
├──────────────────────────────────────────────────────┤
│ Iterable 1: [1, 2, 3] │
│ Iterable 2: [10, 20, 30] │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ Function: (1,10) (2,20) (3,30) │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ Results: [11, 22, 33] │
└──────────────────────────────────────────────────────┘

Animation-like Explanation​

Time | Action
-----|---------------------------------------------------
t0 | map() starts with first elements: a[0]=1, b[0]=10
t1 | lambda(1, 10) → 1 + 10 = 11
t2 | yield 11
t3 | Move to next elements: a[1]=2, b[1]=20
t4 | lambda(2, 20) → 2 + 20 = 22
t5 | yield 22
t6 | Move to next elements: a[2]=3, b[2]=30
t7 | lambda(3, 30) → 3 + 30 = 33
t8 | yield 33
t9 | No more elements → StopIteration

🔧 Detailed Technical Analysis​

The Lambda Function​

# Breakdown of the lambda
lambda x, y: x + y

# Equivalent to:
def anonymous_function(x, y):
return x + y

# Type hints for clarity
from typing import Callable
adder: Callable[[int, int], int] = lambda x, y: x + y

map() Object Characteristics​

# map() returns an iterator, not a list
result = map(lambda x, y: x + y, a, b)
print(type(result)) # <class 'map'>
print(iter(result) is result) # True, it's its own iterator

# You need to consume it to get values
print(list(result)) # [11, 22, 33]

# Once consumed, it's empty
print(list(result)) # [] (iterator exhausted)

💡 Key Features and Behaviors​

1. Lazy Evaluation​

a = [1, 2, 3]
b = [10, 20, 30]

# Nothing is computed yet
mapper = map(lambda x, y: x + y, a, b)
print("No computation happened yet")

# Computation happens when we iterate
first_sum = next(mapper) # Computes 1 + 10 = 11
print(f"First sum: {first_sum}")

remaining = list(mapper) # Computes remaining: [22, 33]
print(f"Remaining: {remaining}")

2. Different Length Iterables​

# map() stops at the shortest iterable
a = [1, 2, 3, 4, 5]
b = [10, 20, 30] # Only 3 elements

result = list(map(lambda x, y: x + y, a, b))
print(result) # [11, 22, 33] - only 3 results!

# Using itertools.zip_longest for different lengths
from itertools import zip_longest

a = [1, 2, 3, 4]
b = [10, 20]

# Default fill with None
result1 = list(map(lambda x, y: (x or 0) + (y or 0),
a, b))
print(result1) # [11, 22, 3, 4]

# With fillvalue
result2 = list(map(
lambda x, y: x + y,
a,
b + [0] * (len(a) - len(b)) # Pad with zeros
))
print(result2) # [11, 22, 3, 4]

3. Memory Efficiency​

# map() is memory efficient for large datasets
import sys

# Generate large sequences without storing intermediate results
large_a = range(1_000_000) # 1 to 1,000,000
large_b = range(10_000_000, 11_000_000) # 10M to 11M

# Memory efficient - doesn't create intermediate lists
mapper = map(lambda x, y: x + y, large_a, large_b)

# Process in chunks
total = 0
for i, value in enumerate(mapper):
total += value
if i >= 1000: # Just sample first 1000
break

print(f"Sum of first 1000: {total}")
print(f"Memory used by map object: {sys.getsizeof(mapper)} bytes")

🚀 Modern Python Alternatives​

1. List Comprehension (Pythonic Alternative)​

# Equivalent using zip() and list comprehension
a = [1, 2, 3]
b = [10, 20, 30]

# Method 1: zip() + list comprehension
sums = [x + y for x, y in zip(a, b)]
# sums = [11, 22, 33]

# Method 2: With type hints
from typing import List
sums: List[int] = [x + y for x, y in zip(a, b)]

# Performance comparison
import timeit

setup = """
a = list(range(1000))
b = list(range(1000, 2000))
"""

map_code = "list(map(lambda x, y: x + y, a, b))"
comp_code = "[x + y for x, y in zip(a, b)]"

print("map + lambda:", timeit.timeit(map_code, setup, number=1000))
print("List comprehension:", timeit.timeit(comp_code, setup, number=1000))

2. Using operator.add​

from operator import add

a = [1, 2, 3]
b = [10, 20, 30]

# More efficient than lambda for simple addition
sums = list(map(add, a, b))
# sums = [11, 22, 33]

# operator module provides many such functions
from operator import mul, sub, truediv

operations = [
("Add", add, [11, 22, 33]),
("Multiply", mul, [10, 40, 90]),
("Subtract", sub, [-9, -18, -27]),
("Divide", truediv, [0.1, 0.1, 0.1])
]

for name, op, expected in operations:
result = list(map(op, a, b))
print(f"{name}: {result} (expected: {expected})")

3. NumPy Vectorized Operations​

# For numerical computations, NumPy is faster
import numpy as np

a = np.array([1, 2, 3])
b = np.array([10, 20, 30])

# Vectorized addition (much faster for large arrays)
sums = a + b # [11, 22, 33]

# Multiple operations
results = {
"sum": a + b,
"product": a * b,
"difference": a - b,
"power": a ** 2 + b ** 2
}

for operation, result in results.items():
print(f"{operation}: {result}")

🔄 Advanced Usage Patterns​

1. Chaining map() Operations​

# Processing pipeline
a = [1, 2, 3]
b = [10, 20, 30]
c = [100, 200, 300]

# Chain multiple operations
step1 = map(lambda x, y: x + y, a, b) # [11, 22, 33]
step2 = map(lambda x: x * 2, step1) # [22, 44, 66]
step3 = map(lambda x, y: x + y, step2, c) # [122, 244, 366]

result = list(step3)
print(f"Chained result: {result}")

# All in one (functional style)
from functools import reduce

final = list(map(
lambda args: reduce(lambda acc, x: acc + x, args),
zip(a, b, c)
))
print(f"Functional result: {final}") # [111, 222, 333]

2. Using with filter()​

# Combine map and filter
a = [1, 2, 3, 4, 5]
b = [10, 20, 30, 40, 50]

# Only keep sums that are even
result = list(filter(
lambda x: x % 2 == 0,
map(lambda x, y: x + y, a, b)
))
# [12, 24, 36, 44] (skipped 11, 15, etc.)

# More complex filtering
complex_result = [
sum_val
for sum_val in map(lambda x, y: x + y, a, b)
if sum_val > 25 and sum_val % 10 == 0
]
# [30, 40, 50]

3. Partial Application with functools.partial​

from functools import partial
from operator import add, mul

a = [1, 2, 3]
b = [10, 20, 30]

# Create specialized functions
add_10 = partial(add, 10)
multiply_by_2 = partial(mul, 2)

# Apply to single list
result1 = list(map(add_10, a)) # [11, 12, 13]

# Apply to pairs with different operations
result2 = list(map(
lambda x, y: add_10(x) + multiply_by_2(y),
a, b
))
# [(1+10) + (10*2), (2+10) + (20*2), (3+10) + (30*2)]
# = [31, 52, 73]

🎯 Real-World Examples​

Example 1: Data Processing Pipeline​

# Process sales data from multiple sources
prices = [19.99, 29.99, 14.99]
quantities = [10, 5, 20]
discounts = [0.1, 0.2, 0.05] # 10%, 20%, 5% discounts

# Calculate total revenue after discounts
revenues = list(map(
lambda price, qty, discount: price * qty * (1 - discount),
prices, quantities, discounts
))

print(f"Individual revenues: {revenues}")
print(f"Total revenue: ${sum(revenues):.2f}")

Example 2: Coordinate Transformation​

# Transform 2D coordinates
x_coords = [1, 2, 3, 4]
y_coords = [5, 6, 7, 8]

# Translate coordinates
translated = list(map(
lambda x, y: (x + 10, y - 5),
x_coords, y_coords
))
# [(11, 0), (12, 1), (13, 2), (14, 3)]

# Calculate distances from origin
distances = list(map(
lambda x, y: (x**2 + y**2) ** 0.5,
x_coords, y_coords
))
# [5.099..., 6.324..., 7.615..., 8.944...]

Example 3: String Processing​

# Process multiple string lists
names = ["alice", "bob", "charlie"]
titles = ["engineer", "manager", "director"]
salaries = [50000, 75000, 100000]

# Create formatted strings
employee_info = list(map(
lambda name, title, salary: (
f"{name.title()} ({title}): ${salary:,.2f}"
),
names, titles, salaries
))

for info in employee_info:
print(info)
# Alice (engineer): $50,000.00
# Bob (manager): $75,000.00
# Charlie (director): $100,000.00

⚠️ Common Pitfalls and Solutions​

Pitfall 1: Different Length Iterables​

# ❌ Problem: Silent truncation
a = [1, 2, 3, 4]
b = [10, 20]
result = list(map(lambda x, y: x + y, a, b))
print(result) # [11, 22] - Lost elements!

# ✅ Solution 1: Check lengths
if len(a) != len(b):
raise ValueError("Iterables must have same length")

# ✅ Solution 2: Use zip_longest with fillvalue
from itertools import zip_longest

result = list(map(
lambda x, y: (x or 0) + (y or 0),
a,
b
))
# With zip_longest: [11, 22, 3, 4]

Pitfall 2: Modifying Iterables During Iteration​

# ❌ Problem: Unexpected behavior
a = [1, 2, 3]
b = [10, 20, 30]

def problematic_map():
result = []
for x, y in zip(a, b):
a.append(x * 10) # Modifying during iteration
result.append(x + y)
return result

print(problematic_map()) # Unpredictable results

# ✅ Solution: Create copies
a_copy = a.copy()
b_copy = b.copy()
result = list(map(lambda x, y: x + y, a_copy, b_copy))

Pitfall 3: Lambda Side Effects​

# ❌ Problem: Lambda with side effects
counter = 0
a = [1, 2, 3]
b = [10, 20, 30]

bad_lambda = lambda x, y: (
globals().__setitem__('counter', counter + 1),
x + y
)[1]

result = list(map(bad_lambda, a, b))
print(f"Result: {result}")
print(f"Counter: {counter}") # Unpredictable in parallel contexts

# ✅ Solution: Pure functions only
pure_lambda = lambda x, y: x + y
result = list(map(pure_lambda, a, b))

📈 Performance Optimization​

Benchmark Comparison​

import timeit
import numpy as np
from operator import add

# Setup
size = 1000000
a = list(range(size))
b = list(range(size, size * 2))

# Different implementations
implementations = [
("map + lambda", "list(map(lambda x, y: x + y, a, b))"),
("map + operator", "list(map(add, a, b))"),
("list comprehension", "[x + y for x, y in zip(a, b)]"),
("numpy", "np.array(a) + np.array(b)"),
]

# Run benchmarks
print("Performance Comparison (1 million elements):")
print("-" * 50)

for name, code in implementations:
if "numpy" in name:
setup = f"import numpy as np; a=list(range({size})); b=list(range({size}, {size}*2))"
else:
setup = f"from operator import add; a=list(range({size})); b=list(range({size}, {size}*2))"

time = timeit.timeit(code, setup, number=1)
print(f"{name:20} {time:.4f} seconds")

🎓 Summary​

Key Takeaways​

  1. map(lambda x, y: x + y, a, b) processes pairs from a and b in parallel
  2. Stops at shortest iterable - beware of different lengths
  3. Returns an iterator - use list() to get all results
  4. Memory efficient - processes elements lazily
  5. Modern alternatives exist - list comprehensions often more Pythonic

When to Use This Pattern​

# ✅ Good for:
# - Parallel element-wise operations
# - Functional programming style
# - Large datasets (memory efficiency)
# - Chaining with other functional tools

# ❌ Consider alternatives for:
# - Simple operations (use list comprehensions)
# - Numerical arrays (use NumPy)
# - Complex logic (use explicit loops)

Memory Diagram​

BEFORE map():
a ──→ [1, 2, 3] (3 elements)
b ──→ [10, 20, 30] (3 elements)

DURING map() iteration:
map object ──→ generates values on demand:
1. Pull (1, 10) → compute 11 → yield
2. Pull (2, 20) → compute 22 → yield
3. Pull (3, 30) → compute 33 → yield

AFTER list():
sums ──→ [11, 22, 33] (new list)

This pattern is fundamental to functional programming in Python and understanding it deeply helps write more efficient and expressive code!