Page Not Found
Page not found. Your pixels are in another canvas.
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Page not found. Your pixels are in another canvas.
About
This is a page not in th emain menu
Published:
Serge is a chat interface that allows you to run Alpaca models without the need for any API keys. It is entirely self-hosted, fits on 4GB of RAM, and can run on the CPU. In this guide, we’ll walk you through the steps required to deploy Serge on Google Cloud Platform (GCP).
Before we get started, make sure you have a GCP account set up and are familiar with the basic concepts of GCP, including creating and managing virtual machines, and setting up firewall rules.
The first step in deploying Serge on GCP is to create a virtual machine. We recommend using a machine type with at least 4GB of RAM and a CPU. You can choose any operating system you like, but I recommend using a Linux-based distribution such as Ubuntu or Debian.
To create a virtual machine in GCP, follow these steps:
Create Instance
buttonus-central1
region and us-central1-c
zone. Also, we are gonna use a e2-standard-8
machine so that we can run models up to 30 billion parameters:SELECT
button:HTTP/HTTPS
traffic:CREATE
button to create your virtual machinedefault
Firewall
and then click on the Create a firewall rule
buttonTarget tags
field fill in the http-server
tag. In the Source IPv4 ranges enable all by filling in 0.0.0.0/0
. In the Protocols and ports section click on TCP
and open port 8008
(we will need it for later). Click on the SAVE
button:To get started with serge
, follow these steps:
SSH
buttonserge
repository:git clone https://github.com/nsarrazin/serge.git
cd serge
sudo docker-compose up -d
sudo docker-compose exec serge python3 /usr/src/app/api/utils/download.py tokenizer 7B
sudo docker-compose exec serge python3 /usr/src/app/api/utils/download.py tokenizer 13B
sudo docker-compose exec serge python3 /usr/src/app/api/utils/download.py tokenizer 30B
Please note that the models occupy the following storage space: 7B requires 4.21G, 13B requires 8.14G, and 30B requires 20.3G
Once you’ve installed the dependencies and started the Docker container, you can access the serge
API by following these steps:
Navigate to VM Instances and copy the External IP
of your machine
Open your web browser and navigate to http://external-ip:8008/ and Voilà:
You should see the serge
homepage, which means that the API is up and running!
To use the API, make requests to http://external-ip:8008/api/:
That’s it! You’re now ready to use serge
on GCP. Happy coding!
A personal blog website powered by Jekyll & AcademicPages, a fork of Minimal Mistakes
Published in Journal 1, 2009
This paper is about the number 1. The number 2 is left for future work.
Recommended citation: Your Name, You. (2009). "Paper Title Number 1." Journal 1. 1(1). http://academicpages.github.io/files/paper1.pdf
Published in Journal 1, 2010
This paper is about the number 2. The number 3 is left for future work.
Recommended citation: Your Name, You. (2010). "Paper Title Number 2." Journal 1. 1(2). http://academicpages.github.io/files/paper2.pdf
Published in Journal 1, 2015
This paper is about the number 3. The number 4 is left for future work.
Recommended citation: Your Name, You. (2015). "Paper Title Number 3." Journal 1. 1(3). http://academicpages.github.io/files/paper3.pdf
Published:
This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown!
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post.
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post.