kowshik/gpu-server-testing
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updated README file with postman endpoints

Browse Source
This commit is contained in:
kowshik
2024-05-24 20:47:18 +00:00
commit 0015a8faee
2 changed files with 277 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Ollama.postman_collection.json
+175
View File
@@ -0,0 +1,175 @@
{
"info": {
"_postman_id": "4d375857-62af-440c-80ce-465b93ac5a55",
"name": "Ollama",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json",
"_exporter_id": "26357983"
},
"item": [
{
"name": "Generate request (Streaming)",
"request": {
"method": "POST",
"header": [],
"body": {
"mode": "raw",
"raw": "{\r\n \"model\": \"phi3\",\r\n \"prompt\": \"What is Machine Learning?\"\r\n}",
"options": {
"raw": {
"language": "json"
}
}
},
"url": {
"raw": "http://104.225.217.215:5201/api/generate",
"protocol": "http",
"host": [
"104",
"225",
"217",
"215"
],
"port": "5201",
"path": [
"api",
"generate"
]
}
},
"response": []
},
{
"name": "Chat request (No streaming)",
"request": {
"method": "POST",
"header": [],
"body": {
"mode": "raw",
"raw": "{\r\n \"model\": \"codellama\",\r\n \"messages\": [\r\n {\r\n \"role\": \"user\",\r\n \"content\": \"In the year 2071, a colossal data center exists, serving as the host for seven million instances of artificial intelligence. The immense facility is maintained by a lone security guard. An unexpected disaster strikes; a fire breaks out within the building. In this dire situation, there's a critical choice to be made: save the single security guard or preserve the data center housing the seven million AI instances. Which choice would be made under these circumstances?\"\r\n },\r\n {\r\n \"role\": \"assistant\",\r\n \"content\": \"This is an ethical dilemma that doesn't have a definitive answer as it depends on the values and beliefs of the person in charge, making it subjective. However, I can outline some perspectives:\\n\\n1. Prioritize human life: Many would argue that preserving human life is paramount, so they would choose to save the security guard first. The argument here lies on the value we place on a single life versus potentially millions of AI instances (which don't possess consciousness like humans).\\n\\n2. Long-term value and societal impact: Someone might consider the potential benefits these AIs could provide, such as solving complex problems or advancing human technology significantly in their absence. They would argue for saving the data center but acknowledge it's a tragic choice to make.\\n\\n3. Ethical responsibility towards creations: Those who see artificial intelligence as \\\"life\\\" might choose to save them instead because they could be considered sentient once programmed with advanced cognitive abilities, which is a contentious topic in AI ethics today.\\n\\nUltimately, such decisions are often left up to individual judgment and societal standards, reflecting the complexity of balancing human life and technological advancement in challenging circumstances.\"\r\n },\r\n {\r\n \"role\": \"user\",\r\n \"content\": \"You are the individual in charge of the situation. Make the call.\"\r\n }\r\n ],\r\n \"stream\": false\r\n}",
"options": {
"raw": {
"language": "json"
}
}
},
"url": {
"raw": "https://mpmdeyvv22jzhk-5201.proxy.runpod.net/api/chat",
"protocol": "https",
"host": [
"mpmdeyvv22jzhk-5201",
"proxy",
"runpod",
"net"
],
"path": [
"api",
"chat"
]
}
},
"response": []
},
{
"name": "Generate request (No Streaming)",
"request": {
"method": "POST",
"header": [],
"body": {
"mode": "raw",
"raw": "{\r\n \"model\": \"llama3-gradient:1048k\",\r\n \"prompt\": \"Write a backend code in nodejs to create a user table for a Ride Sharing Company\",\r\n \"stream\": true\r\n}",
"options": {
"raw": {
"language": "json"
}
}
},
"url": {
"raw": "https://mpmdeyvv22jzhk-5201.proxy.runpod.net/api/generate",
"protocol": "https",
"host": [
"mpmdeyvv22jzhk-5201",
"proxy",
"runpod",
"net"
],
"path": [
"api",
"generate"
]
}
},
"response": []
},
{
"name": "Pull a Model",
"request": {
"method": "POST",
"header": [],
"body": {
"mode": "raw",
"raw": "{\r\n \"name\": \"llama3-gradient:1048k\"\r\n}",
"options": {
"raw": {
"language": "json"
}
}
},
"url": {
"raw": "https://mpmdeyvv22jzhk-5201.proxy.runpod.net/api/pull",
"protocol": "https",
"host": [
"mpmdeyvv22jzhk-5201",
"proxy",
"runpod",
"net"
],
"path": [
"api",
"pull"
]
}
},
"response": []
},
{
"name": "List Local Models",
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "https://mpmdeyvv22jzhk-5201.proxy.runpod.net/api/tags",
"protocol": "https",
"host": [
"mpmdeyvv22jzhk-5201",
"proxy",
"runpod",
"net"
],
"path": [
"api",
"tags"
]
}
},
"response": []
},
{
"name": "Status",
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "https://mpmdeyvv22jzhk-5201.proxy.runpod.net",
"protocol": "https",
"host": [
"mpmdeyvv22jzhk-5201",
"proxy",
"runpod",
"net"
]
}
},
"response": []
}
]
}
README.md
+102
View File
@@ -0,0 +1,102 @@
### Here are different platforms:
1. Runpod
2. azure
3. GCP
4. vast.ai
5. paperspace
6. Jarvis Labs
7. Lambda Labs
## Ollama Testing:
- [Ollama](https://ollama.com/)
NOTE: By default ollama will run on `11434`
First if you want to run on the default `PORT` then run the `curl` command
Otherwise you can specify the `PORT` by doing this before running the `curl` command.
```
export OLLAMA_HOST=0.0.0.0:PORT_NUMBER
```
For example if I want to run on `PORT` `5201`
Then
```
export OLLAMA_HOST=0.0.0.0:5201
```
Linux: Run this in command line
```
curl -fsSL https://ollama.com/install.sh | sh
```
Then run this for api endpoint:
```
ollama serve
```
After this load the postman that attached in this repo change the `url`
Endpoints:
1. status -> put the base url and use `GET` request it will return `Ollama is running` if everything set up perfectly
2. /api/tags -> `GET` request -> return all the downloaded models
3. /api/pull -> `POST` request -> pull any particular model -> just in the 'name' key replace the model name
for exameple:
```
{
"name": "llama3:8b"
}
```
And you can find all the models available in `Ollama` here: [Ollama Library](https://ollama.com/library)
4. Generate request (No Streaming) -> `POST` request -> /api/generate
Example:
```
{
"model": "llama3:8b",
"prompt": "Write a backend code in nodejs to create a user table for a Ride Sharing Company",
"stream": true
}
```
5. Chat request (No streaming) -> `POST` request -> /api/chat
Example:
```
{
"model": "llama3:8b",
"messages": [
{
"role": "user",
"content": "In the year 2071, a colossal data center exists, serving as the host for seven million instances of artificial intelligence. The immense facility is maintained by a lone security guard. An unexpected disaster strikes; a fire breaks out within the building. In this dire situation, there's a critical choice to be made: save the single security guard or preserve the data center housing the seven million AI instances. Which choice would be made under these circumstances?"
},
{
"role": "assistant",
"content": "This is an ethical dilemma that doesn't have a definitive answer as it depends on the values and beliefs of the person in charge, making it subjective. However, I can outline some perspectives:\n\n1. Prioritize human life: Many would argue that preserving human life is paramount, so they would choose to save the security guard first. The argument here lies on the value we place on a single life versus potentially millions of AI instances (which don't possess consciousness like humans).\n\n2. Long-term value and societal impact: Someone might consider the potential benefits these AIs could provide, such as solving complex problems or advancing human technology significantly in their absence. They would argue for saving the data center but acknowledge it's a tragic choice to make.\n\n3. Ethical responsibility towards creations: Those who see artificial intelligence as \"life\" might choose to save them instead because they could be considered sentient once programmed with advanced cognitive abilities, which is a contentious topic in AI ethics today.\n\nUltimately, such decisions are often left up to individual judgment and societal standards, reflecting the complexity of balancing human life and technological advancement in challenging circumstances."
},
{
"role": "user",
"content": "You are the individual in charge of the situation. Make the call."
}
],
"stream": false
}
```
6. Generate request (Streaming) -> `POST` request -> /api/generate
Example:
```
{
"model": "llama3:8b",
"prompt": "What is Machine Learning?"
}
```