Python Callable: Comprehensive Guide
ยท ้
่ฏป้ 14 ๅ้
๐ Introduction to Callable Objectsโ
What is a Callable?โ
In Python, a Callable is any object that can be called like a function using parentheses () and optionally passing arguments.
# Everything here is a Callable:
def regular_func(x): return x * 2
lambda_func = lambda x: x * 2
class MyClass:
def __call__(self): return "called"
instance = MyClass()
builtin_func = print
# All can be "called":
regular_func(5) # 10
lambda_func(5) # 10
instance() # "called"
builtin_func("Hello") # Hello
The Callable Type Hintโ
from typing import Callable, TypeVar
# Basic Callable type hint
func: Callable[[int], int] = lambda x: x * 2
# With multiple parameters
adder: Callable[[int, int], int] = lambda x, y: x + y
# With keyword arguments
formatter: Callable[[str, str], str] = lambda text, prefix: f"{prefix}: {text}"
๐ฏ Anatomy of a Callableโ
Callable Protocol Diagramโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Callable Protocol โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 1. Has __call__() method โ
โ 2. Can be invoked with () โ
โ 3. May accept arguments and return values โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Types Implementing Protocol โ
โโโโโโโโโโโโฌโโโโโโโโโโฌโโโโโโโโโโโฌโโโโโโโโโโโฌโโโโโโโโโโโโโโค
โ Function โ Lambda โ Method โ Class โ Instance โ
โ โ โ โ (__new__)โ (__call__) โ
โโโโโโโโโโโโดโโโโโโโโโโดโโโโโโโโโโโดโโโโโโโโโโโดโโโโโโโโโโโโโโ
Type Hierarchy of Callablesโ
from typing import Any, Callable, Protocol, runtime_checkable
@runtime_checkable
class CallableProtocol(Protocol):
"""The protocol that defines callable behavior."""
def __call__(self, *args: Any, **kwargs: Any) -> Any: ...
# All these satisfy the protocol
def func(x: int) -> int: return x
lambda_func = lambda x: x
class CallableClass:
def __call__(self, x: int) -> int: return x
# Type checking
print(isinstance(func, CallableProtocol)) # True
print(isinstance(lambda_func, CallableProtocol)) # True
print(isinstance(CallableClass(), CallableProtocol)) # True
๐ง Creating Callablesโ
1. Regular Functionsโ
from typing import Callable
def power(exponent: int) -> Callable[[float], float]:
"""Factory function returning a callable."""
def inner(base: float) -> float:
return base ** exponent
return inner
square: Callable[[float], float] = power(2)
cube: Callable[[float], float] = power(3)
print(square(5.0)) # 25.0
print(cube(5.0)) # 125.0
2. Lambda Functions (Modern Syntax)โ
from typing import Callable, TypeAlias
# Type aliases for clarity
MathOperation: TypeAlias = Callable[[float, float], float]
StringTransformer: TypeAlias = Callable[[str], str]
# Basic lambda with type hints
add: MathOperation = lambda x, y: x + y
multiply: MathOperation = lambda x, y: x * y
# Walrus operator in lambda (Python 3.8+)
process: Callable[[str], tuple[str, int]] = (
lambda s: (s.upper(), (length := len(s)))
)
# TypeGuard pattern (Python 3.10+)
from typing import TypeGuard
is_even: Callable[[int], TypeGuard[int]] = lambda x: x % 2 == 0
numbers = [1, 2, 3, 4, 5]
evens = [n for n in numbers if is_even(n)] # Type narrowed to int
3. Classes with __call__โ
from typing import Callable, Any
from dataclasses import dataclass
@dataclass
class Counter:
"""Callable class maintaining state."""
count: int = 0
def __call__(self) -> int:
self.count += 1
return self.count
# Usage
counter: Counter = Counter()
print(counter()) # 1
print(counter()) # 2
print(counter()) # 3
# Type hint for callable instance
callable_counter: Callable[[], int] = counter
4. Partial Functionsโ
from functools import partial
from typing import Callable
# Create new callables from existing ones
def power(base: float, exponent: float) -> float:
return base ** exponent
# Partial application
square: Callable[[float], float] = partial(power, exponent=2)
cube: Callable[[float], float] = partial(power, exponent=3)
print(square(5.0)) # 25.0
print(cube(5.0)) # 125.0
# With type hints
from typing import ParamSpec, Concatenate
P = ParamSpec('P')
R = TypeVar('R')
def typed_partial(
func: Callable[Concatenate[int, P], R],
fixed_arg: int
) -> Callable[P, R]:
"""Type-safe partial function."""
return partial(func, fixed_arg)
๐ Callable Type Systemโ
Modern Type Hint Syntax (Python 3.10+)โ
from typing import Callable, TypeAlias, ParamSpec, TypeVar, Concatenate
from collections.abc import Iterator
# Type variables for generics
T = TypeVar('T')
R = TypeVar('R')
P = ParamSpec('P')
# 1. Basic callable
SimpleFunc: TypeAlias = Callable[[int, int], int]
# 2. Generic callable
Transformer: TypeAlias = Callable[[T], T]
# 3. Callable with ParamSpec (Python 3.10)
def add_logging(func: Callable[P, R]) -> Callable[P, R]:
"""Decorator preserving parameter specification."""
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
print(f"Calling {func.__name__}")
return func(*args, **kwargs)
return wrapper
# 4. Concatenate for adding parameters
def add_prefix(
func: Callable[P, R]
) -> Callable[Concatenate[str, P], R]:
"""Add a prefix parameter to a function."""
def wrapper(prefix: str, *args: P.args, **kwargs: P.kwargs) -> R:
print(f"[{prefix}] ", end="")
return func(*args, **kwargs)
return wrapper
Type Checking Diagramโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Type Checking Flow โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ 1. Define Callable Type: โ
โ Callable[[int, str], bool] โ
โ โ
โ 2. Assign Implementation: โ
โ def check(x: int, s: str) -> bool: return True โ
โ lambda x, s: len(s) > x โ
โ โ
โ 3. Type Checker Validates: โ
โ โ
Parameter count matches โ
โ โ
Parameter types compatible โ
โ โ
Return type matches โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ Callable Composition Patternsโ
1. Function Compositionโ
from typing import Callable, TypeVar
T1 = TypeVar('T1')
T2 = TypeVar('T2')
T3 = TypeVar('T3')
def compose(
f: Callable[[T1], T2],
g: Callable[[T2], T3]
) -> Callable[[T1], T3]:
"""Compose two functions: g โ f"""
return lambda x: g(f(x))
# Modern syntax with walrus operator
compose_modern: Callable[
[Callable[[T1], T2], Callable[[T2], T3]],
Callable[[T1], T3]
] = lambda f, g: lambda x: (temp := f(x), g(temp))[1]
# Usage
add_one: Callable[[int], int] = lambda x: x + 1
double: Callable[[int], int] = lambda x: x * 2
add_one_then_double = compose(add_one, double)
print(add_one_then_double(3)) # 8
2. Pipeline Patternโ
from typing import Callable, Any, Sequence
def create_pipeline(
*funcs: Callable[[Any], Any]
) -> Callable[[Any], Any]:
"""Create a processing pipeline from callables."""
from functools import reduce
return lambda data: reduce(
lambda result, func: func(result),
funcs,
data
)
# Type-safe pipeline (Python 3.12+)
def typed_pipeline(
initial: T1,
*funcs: Callable[[Any], Any]
) -> Any:
"""Type hinted pipeline."""
result = initial
for func in funcs:
result = func(result)
return result
# Usage
pipeline = create_pipeline(
lambda x: x * 2,
lambda x: x + 10,
str,
lambda s: f"Result: {s}"
)
print(pipeline(5)) # "Result: 20"
3. Strategy Pattern with Callablesโ
from typing import Callable, Protocol, runtime_checkable
from dataclasses import dataclass
from enum import Enum
class Operation(Enum):
ADD = "add"
MULTIPLY = "multiply"
POWER = "power"
@runtime_checkable
class MathStrategy(Protocol):
"""Protocol for math operations."""
def __call__(self, a: float, b: float) -> float: ...
@dataclass
class MathProcessor:
"""Uses callables as strategies."""
strategy: MathStrategy
def execute(self, a: float, b: float) -> float:
return self.strategy(a, b)
# Strategies as lambdas
strategies: dict[Operation, MathStrategy] = {
Operation.ADD: lambda a, b: a + b,
Operation.MULTIPLY: lambda a, b: a * b,
Operation.POWER: lambda a, b: a ** b,
}
# Usage
processor = MathProcessor(strategies[Operation.POWER])
print(processor.execute(2, 3)) # 8.0
๐ฏ Advanced Callable Patternsโ
1. Callable with Positional-Only Parametersโ
from typing import Callable
# Python 3.8+ positional-only syntax in callables
pos_only_lambda: Callable[[int, int, /], int] = (
lambda x, y, /: x + y # '/' makes parameters positional-only
)
# Valid usage
print(pos_only_lambda(10, 20)) # 30
# pos_only_lambda(x=10, y=20) # TypeError
# Regular function with positional-only
def pos_only_func(x: int, y: int, /) -> int:
return x * y
# Type hint for such functions
pos_only_callable: Callable[[int, int, /], int] = pos_only_func
2. Callable with Keyword-Only Parametersโ
from typing import Callable, Unpack, TypedDict
class Point(TypedDict):
x: float
y: float
# Keyword-only lambda (Python 3.8+)
kw_only_lambda: Callable[[], int] = (
lambda *, x=0, y=0: x + y # '*' makes parameters keyword-only
)
# With TypedDict unpacking (Python 3.11+)
def process_point(**kwargs: Unpack[Point]) -> float:
return (kwargs['x'] ** 2 + kwargs['y'] ** 2) ** 0.5
# Type hint
point_processor: Callable[[Unpack[Point]], float] = process_point
3. Async Callablesโ
from typing import Callable, Awaitable
import asyncio
# Async callable type hints
AsyncOperation = Callable[[str], Awaitable[str]]
# Async lambda (Python 3.7+)
async def process_data(data: str) -> str:
return data.upper()
# Type hint
async_processor: AsyncOperation = process_data
# Async callable class
class AsyncProcessor:
def __init__(self, delay: float):
self.delay = delay
async def __call__(self, data: str) -> str:
await asyncio.sleep(self.delay)
return f"Processed: {data}"
# Usage
async def main():
processor = AsyncProcessor(1.0)
result = await processor("test")
print(result)
# Run: asyncio.run(main())
๐ Callable in Data Processingโ
1. Map/Filter/Reduce with Typed Callablesโ
from typing import Callable, TypeVar, Iterable, List
from functools import reduce
import operator
T = TypeVar('T')
R = TypeVar('R')
def typed_map(
func: Callable[[T], R],
iterable: Iterable[T]
) -> List[R]:
"""Type-safe map operation."""
return [func(item) for item in iterable]
def typed_filter(
predicate: Callable[[T], bool],
iterable: Iterable[T]
) -> List[T]:
"""Type-safe filter operation."""
return [item for item in iterable if predicate(item)]
def typed_reduce(
func: Callable[[R, T], R],
iterable: Iterable[T],
initial: R
) -> R:
"""Type-safe reduce operation."""
return reduce(func, iterable, initial)
# Usage with modern syntax
numbers = [1, 2, 3, 4, 5]
# Map with lambda and type hints
squared: Callable[[int], int] = lambda x: x ** 2
result_map = typed_map(squared, numbers)
# Filter with walrus operator
has_large_square: Callable[[int], bool] = (
lambda x: (sq := x ** 2) > 10
)
result_filter = typed_filter(has_large_square, numbers)
# Reduce with operator module
sum_squares = typed_reduce(operator.add, result_map, 0)
2. Data Validation Pipelineโ
from typing import Callable, Any, Optional
from pydantic import BaseModel, validator
# Validation callables
ValidationRule = Callable[[Any], Optional[str]]
def create_validator(*rules: ValidationRule) -> Callable[[Any], bool]:
"""Create a validator from multiple rules."""
def validate(data: Any) -> bool:
for rule in rules:
if (error := rule(data)) is not None:
print(f"Validation error: {error}")
return False
return True
return validate
# Validation rules as lambdas
validators = [
lambda x: None if isinstance(x, (int, float)) else "Not a number",
lambda x: None if x >= 0 else "Must be non-negative",
lambda x: None if x <= 100 else "Must be โค 100",
]
# Create validator
validate_number = create_validator(*validators)
print(validate_number(50)) # True
print(validate_number(-5)) # False with error message
print(validate_number("abc")) # False with error message
๐๏ธ Callable in OOP Patternsโ
1. Factory Patternโ
from typing import Callable, Type, TypeVar, Any
from dataclasses import dataclass
T = TypeVar('T')
@dataclass
class Product:
name: str
price: float
ProductFactory = Callable[[str, float], Product]
# Factory lambdas
factories: dict[str, ProductFactory] = {
"simple": lambda name, price: Product(name, price),
"discounted": lambda name, price: Product(
f"Discounted {name}",
price * 0.8
),
"premium": lambda name, price: Product(
f"Premium {name}",
price * 1.2
),
}
def create_product(
factory_type: str,
name: str,
price: float
) -> Product:
factory = factories.get(factory_type, factories["simple"])
return factory(name, price)
# Usage
product = create_product("discounted", "Laptop", 1000.0)
print(product) # Product(name='Discounted Laptop', price=800.0)
2. Observer Patternโ
from typing import Callable, Protocol, List
from dataclasses import dataclass, field
class Observer(Protocol):
"""Observer protocol using callable."""
def __call__(self, data: Any) -> None: ...
@dataclass
class Observable:
"""Subject that notifies observers."""
observers: List[Observer] = field(default_factory=list)
def attach(self, observer: Observer) -> None:
self.observers.append(observer)
def notify(self, data: Any) -> None:
for observer in self.observers:
observer(data)
# Observers as lambdas
observable = Observable()
# Attach observers
observable.attach(lambda data: print(f"Logger: {data}"))
observable.attach(lambda data: print(f"Metrics: processed {type(data).__name__}"))
observable.attach(lambda data: (
globals().__setitem__('last_data', data),
print(f"Cached: {data}")
)[1]) # Using tuple for multiple expressions
# Notify all observers
observable.notify("Test Data")
๐ง Callable Inspection and Introspectionโ
1. Signature Inspectionโ
import inspect
from typing import Callable, get_type_hints
def inspect_callable(func: Callable) -> None:
"""Inspect callable properties."""
print(f"Name: {func.__name__ if hasattr(func, '__name__') else 'Anonymous'}")
print(f"Module: {func.__module__ if hasattr(func, '__module__') else 'Unknown'}")
# Get signature
try:
sig = inspect.signature(func)
print(f"Signature: {sig}")
# Get type hints
hints = get_type_hints(func)
print(f"Type hints: {hints}")
# Parameter details
print("\nParameters:")
for param in sig.parameters.values():
print(f" {param.name}: {param.annotation}")
except ValueError:
print("No signature available")
# Test with different callables
inspect_callable(lambda x: x * 2)
print("\n" + "="*50)
inspect_callable(print)
print("\n" + "="*50)
def typed_func(x: int, y: str = "default") -> bool:
return len(y) > x
inspect_callable(typed_func)
2. Callable Comparison and Identityโ
from typing import Callable
# Lambda identity vs equality
lambda1: Callable[[int], int] = lambda x: x * 2
lambda2: Callable[[int], int] = lambda x: x * 2
lambda3 = lambda1
print(f"lambda1 is lambda2: {lambda1 is lambda2}") # False
print(f"lambda1 == lambda2: {lambda1 == lambda2}") # False
print(f"lambda1 is lambda3: {lambda1 is lambda3}") # True
# Function identity
def square(x: int) -> int:
return x ** 2
square_alias = square
print(f"\nsquare is square_alias: {square is square_alias}") # True
# Checking callable nature
callables_to_test = [
lambda x: x,
print,
str.upper,
[].append,
]
for obj in callables_to_test:
print(f"{obj!r}: callable={callable(obj)}")
๐ Performance Considerationsโ
Benchmark: Lambda vs Function vs Methodโ
import timeit
from typing import Callable
def regular_func(x: int) -> int:
return x * 2
class Multiplier:
def __call__(self, x: int) -> int:
return x * 2
def method(self, x: int) -> int:
return x * 2
# Create callables
lambda_func: Callable[[int], int] = lambda x: x * 2
regular = regular_func
instance = Multiplier()
callable_instance: Callable[[int], int] = instance
method = instance.method
# Benchmark
test_data = 42
iterations = 1000000
setup = """
from __main__ import lambda_func, regular, callable_instance, method
"""
tests = [
("Lambda", "lambda_func(42)"),
("Regular function", "regular(42)"),
("Callable instance", "callable_instance(42)"),
("Method", "method(42)"),
]
print("Performance Comparison (lower is better):")
print("-" * 50)
for name, code in tests:
time = timeit.timeit(code, setup, number=iterations)
print(f"{name:20} {time:.6f}s ({time/iterations*1e9:.2f} ns/call)")
๐จ Modern Python Features with Callablesโ
1. Pattern Matching (Python 3.10+)โ
from typing import Callable, Union
def process_with_pattern(
processor: Callable[[Union[int, str]], str]
) -> Callable[[Union[int, str]], str]:
"""Wrap processor with pattern matching."""
def wrapper(data: Union[int, str]) -> str:
match data:
case int() if data >= 0:
return f"Positive number: {processor(data)}"
case int():
return f"Negative number: {processor(abs(data))}"
case str():
return f"String: {processor(data.upper())}"
case _:
return f"Unknown: {processor(str(data))}"
return wrapper
# Usage
basic_processor: Callable[[Union[int, str]], str] = str
enhanced = process_with_pattern(basic_processor)
print(enhanced(42)) # "Positive number: 42"
print(enhanced("hello")) # "String: HELLO"
2. Structural Pattern Matching with Callablesโ
from typing import Callable
def handle_command(
command: str,
*args: str,
handlers: dict[str, Callable[..., str]]
) -> str:
"""Route commands to appropriate handlers."""
match command.split():
case ["add", item]:
return handlers.get("add", lambda x: f"Add: {x}")(item)
case ["remove", item]:
return handlers.get("remove", lambda x: f"Remove: {x}")(item)
case ["search", *terms]:
return handlers.get("search", lambda *t: f"Search: {t}")(*terms)
case _:
return handlers.get("default", lambda: "Unknown command")()
# Define handlers as lambdas
handlers = {
"add": lambda item: f"Adding {item} to database",
"remove": lambda item: f"Removing {item} from database",
"search": lambda *terms: f"Searching for: {', '.join(terms)}",
"default": lambda: "No handler found"
}
print(handle_command("add apple", handlers=handlers))
print(handle_command("search python java", handlers=handlers))
๐ Best Practicesโ
1. Type Hinting Guidelinesโ
from typing import Callable, ParamSpec, TypeVar, Concatenate
P = ParamSpec('P')
R = TypeVar('R')
# โ
DO: Use precise type hints
def process_numbers(
processor: Callable[[int, int], float]
) -> list[float]:
return [processor(i, i*2) for i in range(10)]
# โ
DO: Use ParamSpec for decorators
def log_calls(func: Callable[P, R]) -> Callable[P, R]:
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
print(f"Calling {func.__name__}")
return func(*args, **kwargs)
return wrapper
# โ DON'T: Use vague type hints
def bad_example(func): # No type hint!
return func()
# โ
DO: Use TypeAlias for complex types
from typing import TypeAlias
MathOp: TypeAlias = Callable[[float, float], float]
StringProcessor: TypeAlias = Callable[[str], str]
2. Lambda vs Named Functions Decision Treeโ
Start
โ
Is it a one-liner? โ NO โ Use named function
โ YES
Will it be reused? โ YES โ Consider named function
โ NO
Need documentation? โ YES โ Use named function
โ NO
Used as simple callback? โ YES โ Use lambda
โ NO
Complex logic needed? โ YES โ Use named function
โ NO
Use lambda (keep it readable)
๐ฏ Summary Cheat Sheetโ
Callable Type Hint Patternsโ
# Basic patterns
Callable[[int], str] # (int) -> str
Callable[[int, str], bool] # (int, str) -> bool
Callable[..., int] # (*args, **kwargs) -> int
# With generics
Callable[[T], R] # Generic function
Callable[[T], List[T]] # Returns list of same type
# With ParamSpec (Python 3.10+)
Callable[P, R] # Preserves parameters
Callable[Concatenate[str, P], R] # Adds parameter
# Async callables
Callable[[str], Awaitable[str]] # Async function
Common Use Casesโ
# 1. Callbacks
def on_complete(callback: Callable[[str], None]) -> None:
callback("Done")
# 2. Strategies/Plugins
strategies: dict[str, Callable[[int], int]] = {
"double": lambda x: x * 2,
"square": lambda x: x ** 2,
}
# 3. Factory functions
def create_validator(
condition: Callable[[Any], bool]
) -> Callable[[Any], bool]:
return lambda x: condition(x)
# 4. Event handlers
class Button:
def __init__(self, on_click: Callable[[], None]):
self.on_click = on_click
Performance Characteristicsโ
Callable Type | Overhead | Use When
---------------------|----------|------------------
Lambda | Low | Simple one-liners
Named Function | Very Low | Reusable logic
Callable Class | Medium | Stateful operations
Partial Function | Medium | Pre-configured functions
Method | Low | Object-oriented design
๐ฎ Future Trends (Python 3.12+)โ
1. Generic TypeVar Syntaxโ
# Python 3.12+ generic syntax
from typing import Callable
def process[T](items: list[T], func: Callable[[T], T]) -> list[T]:
return [func(item) for item in items]
# Type variable with bounds
def numeric_operation[T: (int, float)](
a: T,
b: T,
op: Callable[[T, T], T]
) -> T:
return op(a, b)
2. Improved Error Messagesโ
# Python 3.11+ better error messages for callables
def expects_callable(func: Callable[[int], str]) -> str:
return func(42)
# Type checker gives better error messages
expects_callable(lambda x: x) # Error: expected str, got int
๐ References and Further Readingโ
Key Concepts Summaryโ
- Callable Protocol: Any object with
__call__()method - Type Safety: Use
Callable[[types...], return_type]for hints - Composition: Combine callables for complex behavior
- Performance: Lambdas are fast but have limitations
- Best Practice: Use named functions for complex logic
When to Use Whatโ
- Lambda: Simple transformations, callbacks, one-time use
- Named Function: Reusable logic, documentation, complex operations
- Callable Class: Stateful operations, configuration
- Partial: Pre-configured functions, parameter binding