Stable Diffusion Inpainting example with Hugging Face inference Endpoints
Inpainting refers to the process of replacing/deteriorating or filling in missing data in an artwork to complete the image. This process was commonly used in image restoration. But with the latest AI developments and breakthroughs, like DALL-E by OpenAI or Stable Diffusion by CompVis, Stability AI, and LAION, inpainting can be used with generative models and achieve impressive results.
You can try it out yourself at the RunwayML Stable Diffusion Inpainting Space
Suppose you are now as impressed as I am. In that case, you are probably asking yourself: “ok, how can I integrate inpainting into my applications in a scalable, reliable, and secure way, how can I use it as an API?”.
That's where Hugging Face Inference Endpoints can help you! 🤗 Inference Endpoints offers a secure production solution to easily deploy Machine Learning models on dedicated and autoscaling infrastructure managed by Hugging Face.
Inference Endpoints already has support for text-to-image generation using Stable Diffusion, which enables you to generate Images from a text prompt. With this blog post, you will learn how to enable inpainting workflows with Inference Endpoints using the custom handler feature. Custom handlers allow users to modify, customize and extend the inference step of your model.
Before we can get started, make sure you meet all of the following requirements:
- An Organization/User with an active credit card. (Add billing here)
- You can access the UI at: https://ui.endpoints.huggingface.co
The Tutorial will cover how to:
- Create Inpainting Inference Handler
- Deploy Stable Diffusion 2 Inpainting as Inference Endpoint
- Integrate Stable Diffusion Inpainting as API and send HTTP requests using Python
TL;DR;
You can directly hit “deploy” on this repository to get started: https://huggingface.co/philschmid/stable-diffusion-2-inpainting-endpoint
1. Create Inpainting Inference Handler
This tutorial is not covering how you create the custom handler for inference. If you want to learn how to create a custom Handler for Inference Endpoints, you can either checkout the documentation or go through “Custom Inference with Hugging Face Inference Endpoints”
We are going to deploy philschmid/stable-diffusion-2-inpainting-endpoint, which implements the following handler.py
for stabilityai/stable-diffusion-2-inpainting.
2. Deploy Stable Diffusion 2 Inpainting as Inference Endpoint
UI: https://ui.endpoints.huggingface.co/
The first step is to deploy our model as an Inference Endpoint. We can deploy our custom Custom Handler the same way as a regular Inference Endpoint.
Select the repository, the cloud, and the region, adjust the instance and security settings, and deploy.
The Inference Endpoint Service will check during the creation of your Endpoint if there is a handler.py
available and valid and will use it for serving requests no matter which “Task” you select.
The UI will automatically select the preferred instance type for us after it recognizes the model. We can create you endpoint by clicking on “Create Endpoint”. This will now create our image artifact and then deploy our model, this should take a few minutes.
3. Integrate Stable Diffusion Inpainting as API and send HTTP requests using Python
Hugging Face Inference endpoints can directly work with binary data, this means that we can directly send our image from our document to the endpoint. We are going to use requests
to send our requests. (make your you have it installed pip install requests
). We need to replace the ENDPOINT_URL
and HF_TOKEN
with our values and then can send a request. Since we are using it as an API, we need to provide our base image, an image with the mask, and our text prompt.
If you want to test an example, you can find the dog.png
and mask_dog.png
in the repository at: https://huggingface.co/philschmid/stable-diffusion-2-inpainting-endpoint/tree/main
The result of the request should be a PIL
image we can display:
Conclusion
We successfully created and deployed a Stable Diffusion Inpainting inference handler to Hugging Face Inference Endpoints in less than 30 minutes.
Having scalable, secure API Endpoints will allow you to move from the experimenting (space) to integrated production workloads, e.g., Javascript Frontend/Desktop App and API Backend.
Now, it's your turn! Sign up and create your custom handler within a few minutes!
Thanks for reading! If you have any questions, feel free to contact me on Twitter or LinkedIn.