Everything in Python is an object. And that includes functions. Let’s see what I learned while I was trying to work with Google cloud functions with python runtime.

Python Functions

Since functions too are objects, we can see what all attributes a function contains as following

>>> def hello(name):
...     print(f"Hello, {name}!")
...
>>> dir(hello)
['__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__',
'__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__get__', '__getattribute__', '__globals__', '__gt__',
'__hash__', '__init__', '__init_subclass__', '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__',
'__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', 
'__subclasshook__']

While there are a lot of them, let’s look at some interesting ones

globals

This attribute, as the name suggests, has references of global variables. If you ever need to know what all global variables are in the scope of this function, this will tell you. See how the function start seeing the new variable in globals

>>> hello.__globals__
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'hello': <function hello at 0x7fe4e82554c0>}

# adding new global variable
>>> GLOBAL="g_val"
>>> hello.__globals__
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'hello': <function hello at 0x7fe4e82554c0>, 'GLOBAL': 'g_val'}

code

This is an interesting one! As everything in python is an object, this includes the bytecode too. The compiled python bytecode is a python code object. Which is accessible via __code__ attribute here. A function has an associated code object which carries some interesting information.

# the file in which function is defined
# stdin here since this is run in an interpreter
>>> hello.__code__.co_filename
'<stdin>'

# number of arguments the function takes
>>> hello.__code__.co_argcount
1

# local variable names
>>> hello.__code__.co_varnames
('name',)

# the function code's compiled bytecode
>>> hello.__code__.co_code
b't\x00d\x01|\x00\x9b\x00d\x02\x9d\x03\x83\x01\x01\x00d\x00S\x00'

There are more code attributes which you can enlist by >>> dir(hello.__code__)

Function Attributes

This might sound weird at first but you can do function_name.foo='bar', like it is an object! (it is, though) These are function attributes and it will be associated with the function.

>>> hello.foo="bar"
>>> hello.foo
'bar'

How does this work? The famous __dict__ attribute which carries these fellas.

>>> hello.__dict__
{'foo': 'bar'}

I personally have not yet come across any use for this feature. There are some explained in this feature’s proposal at PEP 232

The Cloud Functions

Cloud functions, as the name suggest, run a unit of code, that is a function. Here is what a Hello World would look like

def hello_world(request):
    """Responds to any HTTP request.
    Args:
        request (flask.Request): HTTP request object.
    Returns:
        The response text or any set of values that can be turned into a
        Response object using
        `make_response <http://flask.pocoo.org/docs/1.0/api/#flask.Flask.make_response>`.
    """
    request_json = request.get_json()
    if request.args and 'message' in request.args:
        return request.args.get('message')
    elif request_json and 'message' in request_json:
        return request_json['message']
    else:
        return 'Hello World!'

A cloud function should be expecting a Flask’s request object as a parameter. This function will be accessible under https://<some-domain>.com/hello_world. The endpoint name is the same as the function name

Now, of course, you won’t have just one function and you would want to test/run these functions locally. Since the injected request parameter is of flask, there needs to be a way to run this function with flask locally

Flask

While working with flask, functions are generally decorated with flask’s @app.route decorator and flask would be injecting parameters to that functions if any. HTTP request’s content is accessed by a global variable like from flask import request while the cloud function expects it as a parameter to function.

To overcome this, we can create a function that wraps the google cloud’s above function. Something like

from flask import Flask, request
app=Flask("test")
@app.route("/hello_world")
def hello_test():
    return hello_world(request)

Sweet! This works as expected. But there is one issue though. If there are a lot of google functions, an equal number of test functions will be required.

Enters the Metaprogrammer

Python is a dynamic language, so let’s make use of it. Here’s the plan

Discover the google functions dynamically
Create wrapper function which generates flask compatible functions out of step 1 above
Register them to our test flask app dynamically

Let’s Discover

If all the functions are in one file, discovering them would look like this. We import the module which contains all the functions, we go through each variable in that module and take a note of all the callable ones (ie: the functions)

# suppose all the functions are defined in main.py
local_vars = importlib.import_module("main").__dict__
for name, value in local_vars.items():
    if callable(value):
        print(name)

Wrapper function

GCF’s function takes request as a parameter which would be incompatible with the function that flask’s route decorator takes. For this, what we can do is create a wrapper function which will call our original google function. And original function takes imported request arg. So make_function here is a closure.

from flask import request
app = flask.Flask(__name__)

def make_function(f):
    def _function():
        return f(request)
    return _function

wrapped_hello_world = make_function(hello_world)

Register Routes

Once we have the wrapped function, we can register it with flask.

app.route("/hello_world")(wrapped_hello_world)

This works well. But as soon as the second wrapper function is registered, it greets us with

>>> app.route("/hi")(make_function(hi))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sanket/venv/lib/python3.8/site-packages/flask/app.py", line 1315, in decorator
    self.add_url_rule(rule, endpoint, f, **options)
  File "/home/sanket/venv/lib/python3.8/site-packages/flask/app.py", line 98, in wrapper_func
    return f(self, *args, **kwargs)
  File "/home/sanket/venv/lib/python3.8/site-packages/flask/app.py", line 1282, in add_url_rule
    raise AssertionError(
AssertionError: View function mapping is overwriting an existing endpoint function: _function

The error is saying a function called _function already exits for another endpoint. If you look closer, the wrapper returns a function named _function and Flask seems to be using function’s name while registering routes. So we cannot have functions with the same name under different routes. Though functions returned by the make_function are different, they have the same name.

Now scrolling back up, a function has an attribute called __name__. Flask would be using this to get the function’s name. Since we know functions we return are different functions, we can change the function’s name before returning it.

def make_function(f):
    def _function():
        return f(request)
    
    _function.__name__ = f.__name__
    
    return _function

And then it goes smooth!

So here is the final version

import importlib
from flask import request
app = flask.Flask(__name__)

def make_function(f):
    def _function():
        return f(request)
    # setting the name correctly so that flask does not complain
    _function.__name__ = f.__name__
    return _function

local_vars = importlib.import_module("main").__dict__
for name, value in local_vars.items():
    if callable(value):
        app.route(f"/{name}")(make_function(value))

app.run()

Oh and later one day I realized functools.wraps also renames the function along with couple other things.

Python Functions#

globals#

code#

Function Attributes#

The Cloud Functions#

Flask#

Enters the Metaprogrammer#

Let’s Discover#

Wrapper function#

Register Routes#