gpu-server-testing/README.md

Here are different platforms:

1. Runpod
2. azure
3. GCP
4. vast.ai
5. paperspace
6. Jarvis Labs 7. Lambda Labs

Ollama Testing:

• Ollama

NOTE: By default ollama will run on 11434

First if you want to run on the default PORT then run the curl command Otherwise you can specify the PORT by doing this before running the curl command.

export OLLAMA_HOST=0.0.0.0:PORT_NUMBER

For example if I want to run on PORT 5201 Then

export OLLAMA_HOST=0.0.0.0:5201

Linux: Run this in command line

    curl -fsSL https://ollama.com/install.sh | sh

Then run this for api endpoint:

ollama serve

After this load the postman that attached in this repo change the url Endpoints:

1. status -> put the base url and use GET request it will return Ollama is running if everything set up perfectly

2. /api/tags -> GET request -> return all the downloaded models

3. /api/pull -> POST request -> pull any particular model -> just in the 'name' key replace the model name for exameple:

{
  "name": "llama3:8b"
}

And you can find all the models available in Ollama here: Ollama Library

4. Generate request (No Streaming) -> POST request -> /api/generate

Example:

{
  "model": "llama3:8b",
  "prompt": "Write a backend code in nodejs to create a user table for a Ride Sharing Company",
  "stream": true
}

5. Chat request (No streaming) -> POST request -> /api/chat

Example:

{
  "model": "llama3:8b",
  "messages": [
    {
      "role": "user",
      "content": "In the year 2071, a colossal data center exists, serving as the host for seven million instances of artificial intelligence. The immense facility is maintained by a lone security guard. An unexpected disaster strikes; a fire breaks out within the building. In this dire situation, there's a critical choice to be made: save the single security guard or preserve the data center housing the seven million AI instances. Which choice would be made under these circumstances?"
    },
    {
        "role": "assistant",
        "content": "This is an ethical dilemma that doesn't have a definitive answer as it depends on the values and beliefs of the person in charge, making it subjective. However, I can outline some perspectives:\n\n1. Prioritize human life: Many would argue that preserving human life is paramount, so they would choose to save the security guard first. The argument here lies on the value we place on a single life versus potentially millions of AI instances (which don't possess consciousness like humans).\n\n2. Long-term value and societal impact: Someone might consider the potential benefits these AIs could provide, such as solving complex problems or advancing human technology significantly in their absence. They would argue for saving the data center but acknowledge it's a tragic choice to make.\n\n3. Ethical responsibility towards creations: Those who see artificial intelligence as \"life\" might choose to save them instead because they could be considered sentient once programmed with advanced cognitive abilities, which is a contentious topic in AI ethics today.\n\nUltimately, such decisions are often left up to individual judgment and societal standards, reflecting the complexity of balancing human life and technological advancement in challenging circumstances."
    },
    {
      "role": "user",
      "content": "You are the individual in charge of the situation. Make the call."
    }
  ],
  "stream": false
}

6. Generate request (Streaming) -> POST request -> /api/generate

Example:

{
    "model": "llama3:8b",
    "prompt": "What is Machine Learning?"
}