The Pain Of Building a Centralized Error Handler in FastAPI
Disclaimer 1
100% human, 0% chatgpt
Disclaimer 2
The blog contains code snippets and error logs, it’s better you follow along by spinning up the respective code.
All the code used in this blog post can be used here https://github.com/bhavaniravi/fastapi-centralized-exception-handler-demo đź”—
I have been using FastAPI for almost five months extensively. Every day, I wake up to writing new APIs and test cases for our business function. The project is growing day by day, and we wanted to have a centralized error handler.
This blog post covers the hurdles I faced while implementing it and how I overcame them.
Setting the Stage
To walk through this experiment with me, you need a FastAPI app. Let’s call this project Playground
and our custom exception will be the Playground Exception
# exception.py
class PlaygroundError(Exception):
def __init__(self, message, http_code):
self.message = message
self.http_code = http_code
super().__init__(message)
The FastAPI app will look something like this.
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def index():
raise PlaygroundError("exception raised", 400)
@app.get("/divide")
def divide():
return 1 / 0 # raises zero division error
To handle these errors in a centralized fashion throughout the project, there is a hook in FastAPI to plugin a function
async def playground_exception_handler(request, exc):
print ("playground exception handled")
return JSONResponse(
status_code=exc.code,
content={"message": exc.message},
)
@app.add_exception_handler(PlaygroundError, playground_exception_handler)
You can also handle the ZeroDivisionError
the same way.
async def handle_exception(request: Request, exc: Exception):
print ("handler for generic exception")
return JSONResponse(
status_code=500,
content={"message": str(exc)},
)
app.add_exception_handler(ZeroDivisionError, handle_exception)
When you run the application with uvicorn app:app --reload
and hit the API localhost:8000/
you will get a nicely formed JSON response with the error message and http status code 400.
Looks nice, easy, and simple, right? What’s the problem?
Introducing Background Tasks
The project I was working on predominantly used background tasks to run the business logic. Given large data processing.
When an API throws an error in the background, the process exception_handler
hook no longer catches them. Because by the time the exception reaches, the response is already generated.
Don’t trust me? Try this.
async def background_task():
raise PlaygroundError("custom error", 3000)
@app.get("/background")
def background(bg: BackgroundTasks):
bg.add_task(background_task)
return {"message": "Hello World"}
Now, when you hit this API with a background task localhost:8000/background
You will receive a RuntimeError
traceback.
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
await self.app(scope, receive, sender)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__
await route.handle(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/routing.py", line 69, in app
await response(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/responses.py", line 174, in __call__
await self.background()
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/background.py", line 43, in __call__
await task()
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/background.py", line 26, in __call__
await self.func(*self.args, **self.kwargs)
File "/Users/bhavaniravi/invisible/playground/fastapi_exceptions/app.py", line 37, in background_task
raise PlaygroundError("custom error", 3000)
exception.PlaygroundError: ('custom error', 3000)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
return await self.app(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/fastapi/applications.py", line 276, in __call__
await super().__call__(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__
await self.middleware_stack(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
await self.app(scope, receive, _send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 83, in __call__
raise RuntimeError(msg) from exc
RuntimeError: Caught handled exception, but response already started
Let’s deconstruct this error a bit.
- The error is caused by
starlette/middleware/exceptions.py
and theExceptionMiddleware
class - The message
print ("playground exception handled")
was never printed in the stack trace, showing the handler wasn’t called The above exception was the direct cause of the following exception:
andCaught handled exception, but response already started
in stack trace shows that RuntimeError is a direct cause of mishandling the exception.
We need a better way that
- Calls the exception handler for background task
- But, does not print the crazy stack trace
Let’s try different alternatives of exception handler hook to work around this error.
Version 1 - Capturing RuntimeError
Instead of handling global exceptions, how about we handle the RuntimeError
?
async def handle_exception(request, exc):
if isinstance(exc, RuntimeError):
print("handling runtime exception", exc)
Having just the runtime error handler won’t handle PlaygroundError
since it is the cause of RuntimeError
app.add_exception_handler(RuntimeError, handle_exception)
Having both PlaygroundError
and RuntimeError
still, result in RuntimeError
since it’s the result of the ExceptionMiddleware
unable to gracefully handle the PlaygroundError
app.add_exception_handler(PlaygroundError, handle_exception)
app.add_exception_handler(RuntimeError, handle_exception)
Version 2 - Capturing Global Exception
RuntimeError
is a type of Exception
so why not add a global exception handler and handle specific cases inside the handler function?
app.add_exception_handler(ACEException, handle_exception)
async def handle_exception(request, exc):
if isinstance(exc, RuntimeError) and isinstance(exc.__cause__, ACEException):
print("handling runtime exception", exc)
if isinstance(exc, PlaygroundError):
print("handling custom exception", exc)
else:
print("handling other exception", exc)
return JSONResponse({"detail": str(exc)})
app.add_exception_handler(Exception, handle_exception)
Two things’s different about this version
- This version doesn’t throw a runtime error
- The handler is called and
handling custom exception
message is printed
What’s the catch?
There is still the exception log that looks like this, making it hard to understand whether the exception was handled cleanly
handling custom exception ('custom error', 3000)
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
return await self.app(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/fastapi/applications.py", line 276, in __call__
await super().__call__(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__
await self.middleware_stack(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
await self.app(scope, receive, _send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
raise exc
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
await self.app(scope, receive, sender)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__
await route.handle(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/routing.py", line 69, in app
await response(scope, receive, send)
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/responses.py", line 174, in __call__
await self.background()
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/background.py", line 43, in __call__
await task()
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/background.py", line 26, in __call__
await self.func(*self.args, **self.kwargs)
File "/Users/bhavaniravi/invisible/playground/fastapi_exceptions/app.py", line 37, in background_task
raise PlaygroundError("custom error", 3000)
exception.PlaygroundError: ('custom error', 3000)
Maybe it’s just an error log
Maybe it is. But…
- How can you differentiate?
- When debugging an error after 6 months, how can you know if this is a result of a handled or unhandled exception
- This will create logs that might trigger alerts from Datadog or Sentry.
Before considering alternative approaches, we have to ensure that we aren’t doing anything wrong and there is no other way possible. For that, we need answers to the following two questions.
Why is this happening?
Going through the error logs deeper will bring out a few things.
The following line from RuntimeError
Version 1
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
The following line from PlaygroundErorr
Version 2
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
It says one thing clearly… Starlette is the culprit
The error is being raised by Starlette
not FastAPI
. To verify that, I created a Starlette app and boom. It was the culprit.
Try running the following Starlette app. It also raises the same error.
from starlette.applications import Starlette
from starlette.background import BackgroundTask
from starlette.middleware.errors import ServerErrorMiddleware
from starlette.responses import JSONResponse, Response
from starlette.routing import Route
from middleware import CustomMiddleware
from starlette.middleware import Middleware
from exception import PlaygroundError
def error_handler(request, exc):
print("error handled gracefully")
def raise_exception():
raise PlaygroundError("Something went wrong")
async def endpoint(request):
return Response("Hello, world!", background=BackgroundTask(raise_exception))
app = Starlette(
routes=[Route("/", endpoint=endpoint)],
exception_handlers={Exception: error_handler},
middleware=[Middleware(CustomMiddleware, debug=True)],
)
Where is the error log coming from?
This line in uvicorn 🔗 is where it is coming from. Though it’s not raising the exception, it is still alarming to have the log.
What can we do about it?
We can find the first point of contact FastAPI
and figure out a way to hook exception-handling Behavior, and BOOM!
File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
There we have it. It’s a middleware that’s raising the exception. How about we write one to suppress the exception we need?
Version 3 - Let’s Write a Middleware
Let’s write a custom middleware in Starlette style and capture the playground error.
# custom_middleware.py
class CustomExceptionHandlingMiddleware:
def __init__(self, app: ASGIApp) -> None:
self.app = app
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] == "http":
request = Request(scope, receive)
try:
await self.app(scope, receive, send)
except PlaygroundError as err:
logging.exception("Error occured while making request to ACE")
return handle_exception(request, err)
else:
await self.app(scope, receive, send)
return None
Link the middleware to the app
# app.py
app.add_middleware(CustomExceptionHandlingMiddleware)
Hitting the background API after adding the middleware localhost:8000/background
captures the error gracefully without spitting those huge logs.
Does it work for all cases?
Case 1 - Synchronous API with Exceptions
With just the middleware and not the exception_handler
Synchronous APIs go haywire.
Create an endpoint that throws the error
@app.get("/")
def index():
raise PlaygroundError("custom error", 3000)
Removing all exception handlers and just keeping the middleware will result in the following error
ERROR: ASGI callable returned without starting response.
INFO: 127.0.0.1:59350 - "GET / HTTP/1.1" 500 Internal Server Error
Solution
Clup them both, use exception_handlers
and CustomMiddleware
Why does this happen?
Yet to find an answer
Case 2 - Error in both sync API and background task
If the API has an error in both sync and background, according to the FastAPI the background tasks won’t be executed, hence we are good.
@app.get("/")
def index(bg: BackgroundTasks):
bg.add_task(background_task)
raise PlaygroundError("custom error", 3000)
return {"message": "Hello World"}
Case 3 - TestCases
At this point, I was happy with my solution. Everything was working smoothly and then came the test cases.
Even with the CustomExceptionHandlingMiddleware
I couldn’t get rid of RuntimeError: Caught handled exception, but response already started.
the error. That is because the TestClient
we use with FastAPI has raise_server_exceptions
set to True
by default.
We can, of course, set it to False
but in the main project, we’d be constraining fellow developers to write code a certain way. We need a better way
Additional Handler?
How about adding extra logic to our custom middleware?
try:
...
except PlaygroundError:
...
except RuntimeError as err:
if isinstance(err.__cause__, PlaygroundError):
print("Error occured while making request to ACE")
return handle_exception(request, err.__cause__)
raise
This helps us handle the RuntimeErorr
Gracefully that occurs as a result of unhandled custom exception
---
There is a Middleware to handle errors on background tasks and an exception-handling hook for synchronous APIs. However, this feels hacky and took a lot of time to figure out. FastAPI developers deserve better both in terms of documentation and errors.
Other Things I Considered But Didn’t Do
Custom Background Task
In FastAPI all background task functions are wrapped around BackgroundTask
class. We can extend that to handle a custom error. But that’d be constraining developer behavior for future development
FastAPI Style Middleware
If you dig through FastAPI documentation enough you will find it recommending app.add_middleware
as a decorator or extending BaseHTTPMiddleware
Something like this
from fastapi import Request
from starlette.middleware.base import BaseHTTPMiddleware
class MyMiddleware(BaseHTTPMiddleware):
def __init__(
self,
app,
some_attribute: str,
):
super().__init__(app)
self.some_attribute = some_attribute
async def dispatch(self, request: Request, call_next):
# do something with the request object, for example
content_type = request.headers.get('Content-Type')
print(content_type)
# process the request and get the response
response = await call_next(request)
return response
This doesn’t work because the exceptions we are dealing with happen at the Starlette middleware stack level. Doesn’t matter how much I tried this particular case, the dispatch method was never reached. Maybe if I dig more I can find the why?